Agreement and Accuracy of Non-Expert Ophthalmologists in Assessing Glaucomatous Changes in Serial Stereo Optic Disc Photographs

Abstract

Purpose: To compare the interobserver agreement in detecting glaucomatous optic disc changes using serial stereophotography between a large group of non-expert ophthalmologists and glaucoma specialists; to assess the accuracy of non-experts; to investigate whether the interobserver agreement and the accuracy of non-experts changed after a training session. Design: Masked interobserver agreement study. Participants: Serial optic disc stereophotos from 40 patients with glaucoma. Methods: Three independent experienced glaucoma specialists (readers of the European Glaucoma Prevention Study) evaluated a set of 2 serial optic disc color stereo-slides for glaucomatous change, obtained with a delay varying from 2 to 7 years of 40 patients, masked from the temporal sequence of the slides. Each patient was graded as changed or stable by agreement of 2 of 3 of the experts (the reference standard). Thirty-seven non-expert ophthalmologists independently evaluated the same set of serial optic disc stereo-slides twice, with the second evaluation on the same day, masked from the results of the previous evaluation, after a training session on a separate slide set. Main Outcome Measures: Interobserver agreement of non-experts and experts in detecting glaucomatous optic disc changes (expressed as kappa coefficient); agreement of non-experts with the reference standard (accuracy) before and after a training session. Results: The interobserver kappa coefficient (kappa) of the non-experts and experts was 0.20 (95% confidence interval [CI], 0.19-0.21) and 0.51 (95% CI, 0.33-0.69), respectively (P < 0.0001). The mean kappa of the non-experts with the reference standard was 0.33 (95% CI, 0.27-0.39). After a training session, the interobserver agreement of the non-experts increased from 0.20 to 0.27 (95% CI, 0.26-0.28) (P < 0.0001). The percentage agreement of the non-experts with the reference standard improved from 68.5% before to 71.4% after the training session (Wilcoxon signed-rank test, P = 0.034). Conclusions: The interobserver agreement of non-expert ophthalmologists in detecting glaucomatous optic disc changes using serial stereophotos was significantly lower than that of experts, which was moderate. After a training session, the interobserver agreement and the accuracy of the non-experts showed a small but statistically significant improvement. Financial Disclosure(s): The author(s) have no proprietary or commercial interest in any materials discussed in this article. Ophthalmology 2011;118:742-746 (C) 2011 by the American Academy of Ophthalmology.