A Multi-Task Learning and Multi-Branch Network for DR and DME Joint Grading
Diabetic Retinopathy (DR) is one of the most common microvascular complications of diabetes. Diabetic Macular Edema (DME) is a concomitant symptom of DR. As the grade of lesion of DR and DME increase, the possibility of blindness can also increase significantly. In order to take the early interventi...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/14/1/138 |
| _version_ | 1797359176374026240 |
|---|---|
| author | Xiaoxue Xing Shenbo Mao Minghan Yan He Yu Dongfang Yuan Cancan Zhu Cong Zhang Jian Zhou Tingfa Xu |
| author_facet | Xiaoxue Xing Shenbo Mao Minghan Yan He Yu Dongfang Yuan Cancan Zhu Cong Zhang Jian Zhou Tingfa Xu |
| author_sort | Xiaoxue Xing |
| collection | DOAJ |
| description | Diabetic Retinopathy (DR) is one of the most common microvascular complications of diabetes. Diabetic Macular Edema (DME) is a concomitant symptom of DR. As the grade of lesion of DR and DME increase, the possibility of blindness can also increase significantly. In order to take the early interventions as soon as possible to reduce the likelihood of blindness, it is necessary to perform both DR and DME grading. We design a joint grading model based on multi-task learning and multi-branch networks (MaMNet) for DR and DME grading. The model mainly includes a multi-branch network (MbN), a feature fusion module, and a disease classification module. The MbN is formed by four branch structures, which can extract the low-level feature information of DME and DR in a targeted way; the feature fusion module is composed of a self-feature extraction module (SFEN), cross-feature extraction module (CFEN) and atrous spatial pyramid pooling module (ASPP). By combining various features collected from the aforementioned modules, the feature fusion module can provide more thorough discriminative features, which benefits the joint grading accuracy. The ISBI-2018-IDRiD challenge dataset is used to evaluate the performance of the proposed model. The experimental results show that based on the multi-task strategy the two grading tasks of DR and DME can provide each other with additional useful information. The joint accuracy of the model, the accuracy of DR and the accuracy of DME are 61.2%, 64.1% and 79.4% respectively. |
| first_indexed | 2024-03-08T15:12:52Z |
| format | Article |
| id | doaj.art-eeed7455e8b94585a4820a457f1ecd83 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-08T15:12:52Z |
| publishDate | 2023-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-eeed7455e8b94585a4820a457f1ecd832024-01-10T14:51:04ZengMDPI AGApplied Sciences2076-34172023-12-0114113810.3390/app14010138A Multi-Task Learning and Multi-Branch Network for DR and DME Joint GradingXiaoxue Xing0Shenbo Mao1Minghan Yan2He Yu3Dongfang Yuan4Cancan Zhu5Cong Zhang6Jian Zhou7Tingfa Xu8College of Electronic Information Engineering, Changchun University, Changchun 130012, ChinaCollege of Electronic Information Engineering, Changchun University, Changchun 130012, ChinaCollege of Electronic Information Engineering, Changchun University, Changchun 130012, ChinaCollege of Electronic Information Engineering, Changchun University, Changchun 130012, ChinaCollege of Electronic Information Engineering, Changchun University, Changchun 130012, ChinaCollege of Electronic Information Engineering, Changchun University, Changchun 130012, ChinaCollege of Electronic Information Engineering, Changchun University, Changchun 130012, ChinaCollege of Electronic Information Engineering, Changchun University, Changchun 130012, ChinaImage Engineering & Video Technology Lab, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, ChinaDiabetic Retinopathy (DR) is one of the most common microvascular complications of diabetes. Diabetic Macular Edema (DME) is a concomitant symptom of DR. As the grade of lesion of DR and DME increase, the possibility of blindness can also increase significantly. In order to take the early interventions as soon as possible to reduce the likelihood of blindness, it is necessary to perform both DR and DME grading. We design a joint grading model based on multi-task learning and multi-branch networks (MaMNet) for DR and DME grading. The model mainly includes a multi-branch network (MbN), a feature fusion module, and a disease classification module. The MbN is formed by four branch structures, which can extract the low-level feature information of DME and DR in a targeted way; the feature fusion module is composed of a self-feature extraction module (SFEN), cross-feature extraction module (CFEN) and atrous spatial pyramid pooling module (ASPP). By combining various features collected from the aforementioned modules, the feature fusion module can provide more thorough discriminative features, which benefits the joint grading accuracy. The ISBI-2018-IDRiD challenge dataset is used to evaluate the performance of the proposed model. The experimental results show that based on the multi-task strategy the two grading tasks of DR and DME can provide each other with additional useful information. The joint accuracy of the model, the accuracy of DR and the accuracy of DME are 61.2%, 64.1% and 79.4% respectively.https://www.mdpi.com/2076-3417/14/1/138DRDMEjoint gradingmulti-branch networkmulti-task learning |
| spellingShingle | Xiaoxue Xing Shenbo Mao Minghan Yan He Yu Dongfang Yuan Cancan Zhu Cong Zhang Jian Zhou Tingfa Xu A Multi-Task Learning and Multi-Branch Network for DR and DME Joint Grading Applied Sciences DR DME joint grading multi-branch network multi-task learning |
| title | A Multi-Task Learning and Multi-Branch Network for DR and DME Joint Grading |
| title_full | A Multi-Task Learning and Multi-Branch Network for DR and DME Joint Grading |
| title_fullStr | A Multi-Task Learning and Multi-Branch Network for DR and DME Joint Grading |
| title_full_unstemmed | A Multi-Task Learning and Multi-Branch Network for DR and DME Joint Grading |
| title_short | A Multi-Task Learning and Multi-Branch Network for DR and DME Joint Grading |
| title_sort | multi task learning and multi branch network for dr and dme joint grading |
| topic | DR DME joint grading multi-branch network multi-task learning |
| url | https://www.mdpi.com/2076-3417/14/1/138 |
| work_keys_str_mv | AT xiaoxuexing amultitasklearningandmultibranchnetworkfordranddmejointgrading AT shenbomao amultitasklearningandmultibranchnetworkfordranddmejointgrading AT minghanyan amultitasklearningandmultibranchnetworkfordranddmejointgrading AT heyu amultitasklearningandmultibranchnetworkfordranddmejointgrading AT dongfangyuan amultitasklearningandmultibranchnetworkfordranddmejointgrading AT cancanzhu amultitasklearningandmultibranchnetworkfordranddmejointgrading AT congzhang amultitasklearningandmultibranchnetworkfordranddmejointgrading AT jianzhou amultitasklearningandmultibranchnetworkfordranddmejointgrading AT tingfaxu amultitasklearningandmultibranchnetworkfordranddmejointgrading AT xiaoxuexing multitasklearningandmultibranchnetworkfordranddmejointgrading AT shenbomao multitasklearningandmultibranchnetworkfordranddmejointgrading AT minghanyan multitasklearningandmultibranchnetworkfordranddmejointgrading AT heyu multitasklearningandmultibranchnetworkfordranddmejointgrading AT dongfangyuan multitasklearningandmultibranchnetworkfordranddmejointgrading AT cancanzhu multitasklearningandmultibranchnetworkfordranddmejointgrading AT congzhang multitasklearningandmultibranchnetworkfordranddmejointgrading AT jianzhou multitasklearningandmultibranchnetworkfordranddmejointgrading AT tingfaxu multitasklearningandmultibranchnetworkfordranddmejointgrading |