Comparison of uncertainty estimation methods for diabetic retinopathy classification using deep learning

Abstract

Deep learning models have shown potential in automated diabetic retinopathy classification, but they lack certainty in their predictions, which is crucial in clinical settings. Thus, uncertainty estimation is receiving increased attention in this field. This work compares uncertainty estimation methods for the classification of diabetic retinopathy in a 5-class scheme, including Evidential Deep Learning. To address the absence of ground truth for uncertainty, a novel evaluation framework is proposed. The framework utilizes a threshold-based system that assumes higher uncertainty for images from distributions other than the training distribution. It aims to distinguish between training distribution images and those from other distributions based on uncertainty estimates. Experiments evaluate the performance of the models in scenarios representing aleatoric and epistemic uncertainties. The results reveal the varying behavior of the methods based on the severity of the shift and the type of uncertainty. While ethnicity and disease shifts, as well as low-quality images, pose challenges as models confidently classify them, artificial noisy images and out-of-distribution samples are correctly identified as uncertain. Notably, Evidential Deep Learning demonstrates effective uncertainty modeling even in challenging scenarios. Overall, this work emphasizes the importance of uncertainty estimation for diabetic retinopathy classification, addresses limitations for its clinical applicability, and provides insights for future research in this domain.