Uncertainty-inspired open set learning for retinal anomaly identification
Abstract Failure to recognize samples from the classes unseen during training is a major limitation of artificial intelligence in the real-world implementation for recognition and classification of retinal anomalies. We establish an uncertainty-inspired open set (UIOS) model, which is trained with f...
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-10-01
|
| Series: | Nature Communications |
| Online Access: | https://doi.org/10.1038/s41467-023-42444-7 |
| _version_ | 1827709839258681344 |
|---|---|
| author | Meng Wang Tian Lin Lianyu Wang Aidi Lin Ke Zou Xinxing Xu Yi Zhou Yuanyuan Peng Qingquan Meng Yiming Qian Guoyao Deng Zhiqun Wu Junhong Chen Jianhong Lin Mingzhi Zhang Weifang Zhu Changqing Zhang Daoqiang Zhang Rick Siow Mong Goh Yong Liu Chi Pui Pang Xinjian Chen Haoyu Chen Huazhu Fu |
| author_facet | Meng Wang Tian Lin Lianyu Wang Aidi Lin Ke Zou Xinxing Xu Yi Zhou Yuanyuan Peng Qingquan Meng Yiming Qian Guoyao Deng Zhiqun Wu Junhong Chen Jianhong Lin Mingzhi Zhang Weifang Zhu Changqing Zhang Daoqiang Zhang Rick Siow Mong Goh Yong Liu Chi Pui Pang Xinjian Chen Haoyu Chen Huazhu Fu |
| author_sort | Meng Wang |
| collection | DOAJ |
| description | Abstract Failure to recognize samples from the classes unseen during training is a major limitation of artificial intelligence in the real-world implementation for recognition and classification of retinal anomalies. We establish an uncertainty-inspired open set (UIOS) model, which is trained with fundus images of 9 retinal conditions. Besides assessing the probability of each category, UIOS also calculates an uncertainty score to express its confidence. Our UIOS model with thresholding strategy achieves an F1 score of 99.55%, 97.01% and 91.91% for the internal testing set, external target categories (TC)-JSIEC dataset and TC-unseen testing set, respectively, compared to the F1 score of 92.20%, 80.69% and 64.74% by the standard AI model. Furthermore, UIOS correctly predicts high uncertainty scores, which would prompt the need for a manual check in the datasets of non-target categories retinal diseases, low-quality fundus images, and non-fundus images. UIOS provides a robust method for real-world screening of retinal anomalies. |
| first_indexed | 2024-03-10T17:28:34Z |
| format | Article |
| id | doaj.art-8ff54aca808d4c9b972ca81caf0f4bf8 |
| institution | Directory Open Access Journal |
| issn | 2041-1723 |
| language | English |
| last_indexed | 2024-03-10T17:28:34Z |
| publishDate | 2023-10-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Nature Communications |
| spelling | doaj.art-8ff54aca808d4c9b972ca81caf0f4bf82023-11-20T10:05:39ZengNature PortfolioNature Communications2041-17232023-10-0114111110.1038/s41467-023-42444-7Uncertainty-inspired open set learning for retinal anomaly identificationMeng Wang0Tian Lin1Lianyu Wang2Aidi Lin3Ke Zou4Xinxing Xu5Yi Zhou6Yuanyuan Peng7Qingquan Meng8Yiming Qian9Guoyao Deng10Zhiqun Wu11Junhong Chen12Jianhong Lin13Mingzhi Zhang14Weifang Zhu15Changqing Zhang16Daoqiang Zhang17Rick Siow Mong Goh18Yong Liu19Chi Pui Pang20Xinjian Chen21Haoyu Chen22Huazhu Fu23Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong KongCollege of Computer Science and Technology, Nanjing University of Aeronautics and AstronauticsJoint Shantou International Eye Center, Shantou University and the Chinese University of Hong KongNational Key Laboratory of Fundamental Science on Synthetic Vision and the College of Computer Science, Sichuan UniversityInstitute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)School of Electronics and Information Engineering, Soochow UniversitySchool of Biomedical Engineering, Anhui Medical UniversitySchool of Electronics and Information Engineering, Soochow UniversityInstitute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)National Key Laboratory of Fundamental Science on Synthetic Vision and the College of Computer Science, Sichuan UniversityLongchuan People’s HospitalPuning People’s HospitalHaifeng PengPai Memory HospitalJoint Shantou International Eye Center, Shantou University and the Chinese University of Hong KongSchool of Electronics and Information Engineering, Soochow UniversityCollege of Intelligence and Computing, Tianjin UniversityCollege of Computer Science and Technology, Nanjing University of Aeronautics and AstronauticsInstitute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)Joint Shantou International Eye Center, Shantou University and the Chinese University of Hong KongSchool of Electronics and Information Engineering, Soochow UniversityJoint Shantou International Eye Center, Shantou University and the Chinese University of Hong KongInstitute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)Abstract Failure to recognize samples from the classes unseen during training is a major limitation of artificial intelligence in the real-world implementation for recognition and classification of retinal anomalies. We establish an uncertainty-inspired open set (UIOS) model, which is trained with fundus images of 9 retinal conditions. Besides assessing the probability of each category, UIOS also calculates an uncertainty score to express its confidence. Our UIOS model with thresholding strategy achieves an F1 score of 99.55%, 97.01% and 91.91% for the internal testing set, external target categories (TC)-JSIEC dataset and TC-unseen testing set, respectively, compared to the F1 score of 92.20%, 80.69% and 64.74% by the standard AI model. Furthermore, UIOS correctly predicts high uncertainty scores, which would prompt the need for a manual check in the datasets of non-target categories retinal diseases, low-quality fundus images, and non-fundus images. UIOS provides a robust method for real-world screening of retinal anomalies.https://doi.org/10.1038/s41467-023-42444-7 |
| spellingShingle | Meng Wang Tian Lin Lianyu Wang Aidi Lin Ke Zou Xinxing Xu Yi Zhou Yuanyuan Peng Qingquan Meng Yiming Qian Guoyao Deng Zhiqun Wu Junhong Chen Jianhong Lin Mingzhi Zhang Weifang Zhu Changqing Zhang Daoqiang Zhang Rick Siow Mong Goh Yong Liu Chi Pui Pang Xinjian Chen Haoyu Chen Huazhu Fu Uncertainty-inspired open set learning for retinal anomaly identification Nature Communications |
| title | Uncertainty-inspired open set learning for retinal anomaly identification |
| title_full | Uncertainty-inspired open set learning for retinal anomaly identification |
| title_fullStr | Uncertainty-inspired open set learning for retinal anomaly identification |
| title_full_unstemmed | Uncertainty-inspired open set learning for retinal anomaly identification |
| title_short | Uncertainty-inspired open set learning for retinal anomaly identification |
| title_sort | uncertainty inspired open set learning for retinal anomaly identification |
| url | https://doi.org/10.1038/s41467-023-42444-7 |
| work_keys_str_mv | AT mengwang uncertaintyinspiredopensetlearningforretinalanomalyidentification AT tianlin uncertaintyinspiredopensetlearningforretinalanomalyidentification AT lianyuwang uncertaintyinspiredopensetlearningforretinalanomalyidentification AT aidilin uncertaintyinspiredopensetlearningforretinalanomalyidentification AT kezou uncertaintyinspiredopensetlearningforretinalanomalyidentification AT xinxingxu uncertaintyinspiredopensetlearningforretinalanomalyidentification AT yizhou uncertaintyinspiredopensetlearningforretinalanomalyidentification AT yuanyuanpeng uncertaintyinspiredopensetlearningforretinalanomalyidentification AT qingquanmeng uncertaintyinspiredopensetlearningforretinalanomalyidentification AT yimingqian uncertaintyinspiredopensetlearningforretinalanomalyidentification AT guoyaodeng uncertaintyinspiredopensetlearningforretinalanomalyidentification AT zhiqunwu uncertaintyinspiredopensetlearningforretinalanomalyidentification AT junhongchen uncertaintyinspiredopensetlearningforretinalanomalyidentification AT jianhonglin uncertaintyinspiredopensetlearningforretinalanomalyidentification AT mingzhizhang uncertaintyinspiredopensetlearningforretinalanomalyidentification AT weifangzhu uncertaintyinspiredopensetlearningforretinalanomalyidentification AT changqingzhang uncertaintyinspiredopensetlearningforretinalanomalyidentification AT daoqiangzhang uncertaintyinspiredopensetlearningforretinalanomalyidentification AT ricksiowmonggoh uncertaintyinspiredopensetlearningforretinalanomalyidentification AT yongliu uncertaintyinspiredopensetlearningforretinalanomalyidentification AT chipuipang uncertaintyinspiredopensetlearningforretinalanomalyidentification AT xinjianchen uncertaintyinspiredopensetlearningforretinalanomalyidentification AT haoyuchen uncertaintyinspiredopensetlearningforretinalanomalyidentification AT huazhufu uncertaintyinspiredopensetlearningforretinalanomalyidentification |