Applying Psychometric Validation Methodology to Longitudinal Clinical Trial Data

Abstract

When the biostatistician and the clinician are designing a new clinical study, they should have good information on the psychometric properties of the measurements that are planned to be done in clinical studies. Indeed, performing clinical studies is resource demanding and therefore it would irresponsible to built upon unreliable measurements. The strategy must be to use a scale, or measurements in general, which has been validated before and for which reliability (test-retest, inter-rater and internal consistency) and validity (content, construct and criterion) are established. The psychometric validation is usually done on a selected small sample from the population for which the scale is intended to be used. If the population of the trial is different, a new battery of reliability and validity testing might be warranted. In this thesis, most of the focus was on quantifying reliability. Reliability reflects on the amount or measurement error which is inherent in any measurement, and therefore, reliability also reflects the extent to which a measurement instrument can differentiate among individuals. As differentiation between subjects randomized to different treatments is core business in clinical trials, it is obvious that reliability is of utmost importance! Generalizability Theory, as natural extension of reliability and Classical Theory can therefore be an extremely valuable tool to assess which factors influence reliability. As noted by Shalvelson, Webb, and Rowley (1989), “GT is not widely applied in psychological research because of its formidable mathematical development”. As noted by Dunn (1989), there is also a need for larger sample sizes, otherwise, many of the estimates of variance components will be practically worthless. In this work, we proposed a framework to study trial or population specific reliability and generalizability based on longitudinal biomedical trial data. The goal is to use clinical trial data at hand and to evaluate psychometric properties of the measurement....