Investigating the criterion validity of psychiatric symptom scales using surrogate marker validation methodology

Abstract

This work investigates whether techniques that are generally used for the validation of surrogate markers in clinical trials can be applied in the validation of psychiatric health measurements (often scales) and more generally to investigate relationships between treatment effects on different measurements. However, the categorical nature of some scales makes these techniques inapplicable in the way they were originally defined. In this work, we show a possible extension of this methodology to the setting in which one of the scales is an ordinal categorical variable. When psychiatric health measurements are either developed or used in a new population, reliability and validity must be investigated. Reliability, more specifically internal consistency, test-retest reliability, and inter-rater reliability, is focused on the reproducibility of the measurement. Validity is defined as the degree to which the scale measures what it purports to measure. This can be performed through the analysis of content, construct, and criterion validity. We argue that recent methodology, in particular developed to study surrogate endpoints, can be used to examine criterion validity, concurrent validity, and predictive validity. In concurrent validity, we correlate the measurement with a criterion measure, both of which are given at the same time. In predictive validity, the criterion will not be available to some point in time in the future. The surrogate methods were applied on pooled data from five trials in schizophrenia.