Evaluation of Rolling Surveillance Methods in Context of Prior Aberrations: A Simulation Study With Routine Data From Low- and Middle-Income Countries

Abstract

Syndromic surveillance integrated into routine health management information systems could improve timely detection of disease outbreaks, particularly in low- and middle-income countries that have limited diagnostic data. This study evaluates the impact of prior anomalies referred to as "aberrations," such as historical outbreaks, that can distort "baseline data" on the accuracy of rolling surveillance methods that track ongoing disease trends. We assessed five widely used outbreak detection algorithms-EARS, Farrington, Holt-Winters, and two versions of the Weinberger-Fulcher model (negative binomial (WF NB) and quasipoisson (WF QP))-under simulation scenarios motivated by 5 years of acute respiratory infection data from Liberia. We evaluated seven data-generating mechanisms that cover a wide range of temporal and seasonal patterns. We assessed the accuracy of the outbreak detection algorithms under varied size and timing of outbreaks and aberrations. Accuracy was measured through sensitivity and specificity, with a joint assessment of both metrics using pseudo-ROC curves. Results showed that the introduction of aberrations reduced sensitivity in general, but the algorithms' relative performances were highly context-dependent. EARS and WF models demonstrated high sensitivity for detecting outbreaks when no recent aberrations were present. However, when aberrations occurred within the last year of baseline data, Holt-Winters-unless there was evidence of strong time trends-and WF QP maintained better overall balance between sensitivity and specificity. The Farrington algorithm exhibited strong sensitivity with recent aberrations but at the cost of lower specificity. These findings provide actionable insights and practical recommendations for implementing rolling surveillance in resource-constrained environments, emphasizing the need to consider historical data disturbances and rigorously evaluate sensitivity and specificity jointly.