Expert judgement in a risk assessment model for Salmonella spp. in pork: The performance of different weighting schemes

Abstract

A structured expert judgement study was carried out in order to obtain input parameters for a quantitative microbial risk assessment (QMRA) model. This model aimed to estimate the risk of human Salmonella infections associated with the consumption of minced pork meat. judgements of 11 experts were used to derive subjective probability density functions (PDFs) to quantify the uncertainty on the model input parameters. The performance of experts as probability assessors was measured by the experts' ability to correctly and precisely provide estimates for a set of seed variables (=variables from the experts' area of expertise for which the true values were known to the analyst). Subsequently different weighting schemes or "decision makers" (DMs) were applied using Cooke's classical model in order to obtain combined PDFs as a weighted linear combination of the expert's individual PDFs. The aim of this study was to compare the performance of four DMs namely the equal weight DM (each expert's opinion received equal weight), the user weight DM (weights are determined by the expert's self-perceived level of expertise) and two performance-based DMs: the global weight DM and the item weight DIM. Weights in the performance-based DMs were calculated based on the expert's calibration and information performance as measured on the set of seed variables. The item weight DIM obtained the highest performance with a calibration score of 0.62 and an information score of 0.52, as compared to the other DMs. The weights of the performance-based DMs outperformed those of the best expert in the panel. The correlation between the scores for self-rating of expertise and the weights based on the experts' performance on the calibration variables was low and not significant (r = 0.37, p = 0.13). The applied classical model provided a rational basis to use the combined distributions obtained by the item weight DM as input in the QMRA model since this DM yielded generally more informative distributions for the variables of interest than those obtained by the equal weight and user weight DM. Attention should be paid to find adequate and relevant seed variables, since this is important for the validation of the results of the weighting scheme. (C) 2009 Elsevier B.V. All rights reserved.