Please use this identifier to cite or link to this item:
http://hdl.handle.net/1942/25209
Full metadata record
DC Field | Value | Language |
---|---|---|
dc.contributor.author | De Mulder, Wim | - |
dc.contributor.author | Rengs, Bernhard | - |
dc.contributor.author | MOLENBERGHS, Geert | - |
dc.contributor.author | Fent, Thomas | - |
dc.contributor.author | VERBEKE, Geert | - |
dc.date.accessioned | 2017-11-21T14:41:18Z | - |
dc.date.available | 2017-11-21T14:41:18Z | - |
dc.date.issued | 2016 | - |
dc.identifier.citation | International Journal on Advances in Systems and Measurements, 9(3-4), p. 188-198 | - |
dc.identifier.issn | 1942-261x | - |
dc.identifier.uri | http://hdl.handle.net/1942/25209 | - |
dc.description.abstract | A common way to evaluate surrogate models is by using validation measures. This amounts to applying a chosen validation measure to a test data set that was not used to train the surrogate model. The selection of a validation measure is typically motivated by diverse guidelines, such as simplicity of the measure, ease of implementation, popularity of the measure, etc., which are often not related to characteristics of the measure itself. However, it should be recognized that the validity of a model is not only dependent on the model, as desired, but also on the behavior of the chosen validation measure. Some, although very limited, research has been devoted to the evaluation of validation measures, by applying them to a given model that is trained on a data set with some known properties, and then evaluating whether the considered measures validate the model in an expected way. In this paper, we perform an evaluation of some statistical and non statistical validation measures from another point of view. We consider a test data set generated by an agentbased model and we successively remove those elements from it for which our previously developed Gaussian process emulator, a surrogate model, produces the worst approximation to the true output value, according to a selected validation measure. All considered validation measures are then applied to the sequence of increasingly smaller test data sets. It is desired that a validation measure shows improvement of a model when test data points on which the model poorly performs are removed, irrespective of the validation measure that is used to detect such data points. Our experiments show that only the considered statistical validation measures have this desired behavior. | - |
dc.description.sponsorship | The authors acknowledge funding from the KU Leuven funded Geconcerteerde Onderzoeksacties (GOA) project New approaches to the social dynamics of long-term fertility change [grant 20142018;GOA/14/001]. | - |
dc.language.iso | en | - |
dc.rights | Copyright © 2016 IARIA | - |
dc.subject.other | Gaussian process emulation; agent-based models; validation | - |
dc.title | Evaluation of some validation measures for Gaussian process emulation: a case study with an agent-based model | - |
dc.type | Journal Contribution | - |
dc.identifier.epage | 198 | - |
dc.identifier.issue | 3-4 | - |
dc.identifier.spage | 188 | - |
dc.identifier.volume | 9 | - |
local.bibliographicCitation.jcat | A1 | - |
local.type.refereed | Refereed | - |
local.type.specified | Article | - |
item.validation | vabb 2020 | - |
item.contributor | De Mulder, Wim | - |
item.contributor | Rengs, Bernhard | - |
item.contributor | MOLENBERGHS, Geert | - |
item.contributor | Fent, Thomas | - |
item.contributor | VERBEKE, Geert | - |
item.fulltext | With Fulltext | - |
item.accessRights | Open Access | - |
item.fullcitation | De Mulder, Wim; Rengs, Bernhard; MOLENBERGHS, Geert; Fent, Thomas & VERBEKE, Geert (2016) Evaluation of some validation measures for Gaussian process emulation: a case study with an agent-based model. In: International Journal on Advances in Systems and Measurements, 9(3-4), p. 188-198. | - |
crisitem.journal.issn | 1942-261x | - |
Appears in Collections: | Research publications |
Google ScholarTM
Check
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.