Please use this identifier to cite or link to this item:
http://hdl.handle.net/1942/33812
Title: | Systematic Comparison of Heatmapping Techniques in Deep Learning in the Context of Diabetic Retinopathy Lesion Detection | Authors: | Van Craenendonck, Toon Elen, Bart Gerrits, Nele DE BOEVER, Patrick |
Issue Date: | 2020 | Publisher: | ASSOC RESEARCH VISION OPHTHALMOLOGY INC | Source: | Translational Vision Science & Technology, 9 (2) (Art N° 64) | Abstract: | Purpose: Heatmapping techniques can support explainability of deep learning (DL) predictions inmedical image analysis. However, individual techniques have been mainly applied in a descriptive way without an objective and systematic evaluation. We investigated comparative performances using diabetic retinopathy lesion detection as a benchmark task. Methods: The Indian Diabetic Retinopathy Image Dataset (IDRiD) publicly available database contains fundus images of diabetes patients with pixel level annotations of diabetic retinopathy (DR) lesions, the ground truth for this study. Three in advance trained DL models (ResNet50, VGG16 or InceptionV3) were used for DR detection in these images. Next, explainability was visualized with each of the 10 most used heatmapping techniques. The quantitative correspondence between the output of a heatmap and the ground truth was evaluated with the Explainability Consistency Score (ECS), a metric between 0 and 1, developed for this comparative task. Results: In case of the overall DR lesions detection, the ECS ranged from 0.21 to 0.51 for all model/heatmapping combinations. The highest score was for VGG16+Grad-CAM (ECS= 0.51; 95% confidence interval [CI]: [0.46; 0.55]). For individual lesions, VGG16+Grad-CAM performed best on hemorrhages and hard exudates. ResNet50+SmoothGrad performed best for soft exudates and ResNet50+Guided Backpropagation performed best for microaneurysms. Conclusions: Our empirical evaluation on the IDRiD database demonstrated that the combination DL model/heatmapping affects explainability when considering common DR lesions. Our approach found considerable disagreement between regions highlighted by heatmaps and expert annotations. | Notes: | De Boever, P (corresponding author), Univ Antwerp, Dept Biol, Univ Pl 1, B-2610 Antwerp, Belgium. patrick.deboever@uantwerpen.be |
Other: | De Boever, P (corresponding author), Univ Antwerp, Dept Biol, Univ Pl 1, B-2610 Antwerp, Belgium. patrick.deboever@uantwerpen.be | Keywords: | deep learning;heatmap;explainability;diabetic retinopathy | Document URI: | http://hdl.handle.net/1942/33812 | ISSN: | 2164-2591 | e-ISSN: | 2164-2591 | DOI: | 10.1167/tvst.9.2.64 | ISI #: | WOS:000618898700002 | Rights: | Copyright 2020 The Authors This work is licensed under a Creative Commons Attribution 4.0 International License | Category: | A1 | Type: | Journal Contribution | Validations: | ecoom 2022 |
Appears in Collections: | Research publications |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
i2164-2591-9-2-64_1609228844.82789.pdf | Published version | 1.44 MB | Adobe PDF | View/Open |
WEB OF SCIENCETM
Citations
11
checked on Apr 24, 2024
Page view(s)
36
checked on Sep 5, 2022
Download(s)
12
checked on Sep 5, 2022
Google ScholarTM
Check
Altmetric
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.