Peabody picture vocabulary test - Revised data : a Bayesian approach to item response theory

Abstract

Background: Item Response Theory is the area of psychometry that deals with the problem of constructing and analyzing psychological and sociological tests. By applying a fully Bayesian approach to this methodology, we analyze a data set obtained administering the Italian translation of the well-known Peabody Picture Vocabulary Test - Revised (PPVT-R) to a sample of Italian children. In the original English version the items are believed to be in increasing difficulty order. One main aim of this thesis is to evaluate if and how much the translation leads to violations of the increasing difficulty ordering. This aspect is important since, in the original version, basal and ceiling level of the test are determined assuming items in increasing difficulty order. Methods: Classical item response models, as 1PL and 2PL have been applied to PPVT-R data. These models have been extended by including covariates. Parameters estimation has been performed using a complete Bayesian approach that in this caseresulted more flexible than the classical approach. In particular, the flexibility of the Bayesian approach has been underlined with respect to the analysis of an incomplete data matrix, due to the nature of the stopping rule, and to the item difficulty comparisons. Several decision rules for the comparison of item difficulties have been analyzed. We propose a further more general alternative method that allows to compare item characteristic curves taking into account also the ability distribution. Results and conclusions: 1PL, 2PL models and model with covariates have been estimated using MCMC methodology: the goodness of fit of the models has been analyzed using posterior predictive p-values and the performance of the three models has been compared using AIC, BIC and DIC indices. The model with covariates resulted to be the best in terms of information criteria. Therefore the comparisons of the item difficulties were performed using this model. The different decision rules for the item difficulty comparisons have been compared and an ordering of the items for each criterion has been drawn. The criteria agree on concluding violation of the increasing difficulty order: from the analysis of the results, we can conclude that the test can be improved by modifying the ordering and by translating the English terms in Italian words of more common use.