Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network
Objective To identify ulcerative colitis (UC) and Crohn's disease (CD) with aid of deep learning technology for endoscopists. Methods From January 2018 to November 2020, the endoscopic images of 1 576 subjects (including 34 300 CD, UC and normal images) were collected from the Department of Gas...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Office of Journal of Army Medical University
2023-02-01
|
| Series: | 陆军军医大学学报 |
| Subjects: | |
| Online Access: | http://aammt.tmmu.edu.cn/Upload/rhtml/202209187.htm |
| _version_ | 1811166210175270912 |
|---|---|
| author | QI Jing RUAN Guangcong YANG Yi WU Yi CAO Qian |
| author_facet | QI Jing RUAN Guangcong YANG Yi WU Yi CAO Qian |
| author_sort | QI Jing |
| collection | DOAJ |
| description | Objective To identify ulcerative colitis (UC) and Crohn's disease (CD) with aid of deep learning technology for endoscopists. Methods From January 2018 to November 2020, the endoscopic images of 1 576 subjects (including 34 300 CD, UC and normal images) were collected from the Department of Gastroenterology of Army Medical Center of PLA and Sir Run Run Shaw Hospital.The training set and test set were randomly divided according to the ratio of 9:1 to train and test the neural network.A novel spatial and bilinear deep network (SABA-ResNet) was constructed on the basis of ResNet50.The spatial attention mechanism was introduced, and the receptive field was expanded by dilated convolution to leverage contextual information, which was combined with the local induction of standard convolution to adaptively focus the lesion region.Bilinear attention was applied to improve the feature representation ability of the network, and the second-order information was used to weight the channel information of the feature map, so as to improve the classification performance of the model. Results The overall accuracy of SABA-ResNet for the recognition of CD, UC and normal tissues on the test set was 92.67%(95%CI: 91.91~93.37), the AUC value was 0.978(95%CI: 0.972~0.983), 0.977(95%CI: 0.971~0.982) and 0.999(95%CI: 0.998~1.000), the sensitivity was 88.40%, 89.07% and 98.89%, the specificity was 95.49%, 94.88% and 98.93%, and the F1 value was 88.80%, 89.01% and 98.60%, respectively.The ablation experiment and the class activation map suggested that spatial attention and bilinear attention could help the model capture more features of the lesion region. Conclusion Our constructed network combines spatial attention and bilinear attention, achieves excellent performance in the recognition of CD, UC and normal tissue, and effectively assist endoscopists in the diagnosis of UC and CD.
|
| first_indexed | 2024-04-10T15:48:52Z |
| format | Article |
| id | doaj.art-b4bdfdfc08db49b6b94b562dd6352642 |
| institution | Directory Open Access Journal |
| issn | 2097-0927 |
| language | zho |
| last_indexed | 2024-04-10T15:48:52Z |
| publishDate | 2023-02-01 |
| publisher | Editorial Office of Journal of Army Medical University |
| record_format | Article |
| series | 陆军军医大学学报 |
| spelling | doaj.art-b4bdfdfc08db49b6b94b562dd63526422023-02-12T02:19:04ZzhoEditorial Office of Journal of Army Medical University陆军军医大学学报2097-09272023-02-0145322723410.16016/j.2097-0927.202209187Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention networkQI Jing0RUAN Guangcong1YANG Yi2WU Yi3CAO Qian4Department of Digital Medicine, School of Biomedical Engineering and Imaging Medicine, Army Medical University(Third Military Medical University), Chongqing, 400038Department of Gastroenterology, Army Medical Center of PLA, Chongqing, 400042 Department of Digital Medicine, School of Biomedical Engineering and Imaging Medicine, Army Medical University(Third Military Medical University), Chongqing, 400038 Department of Digital Medicine, School of Biomedical Engineering and Imaging Medicine, Army Medical University(Third Military Medical University), Chongqing, 400038Department of Gastroenterology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou, Zhejiang Province, 310016, ChinaObjective To identify ulcerative colitis (UC) and Crohn's disease (CD) with aid of deep learning technology for endoscopists. Methods From January 2018 to November 2020, the endoscopic images of 1 576 subjects (including 34 300 CD, UC and normal images) were collected from the Department of Gastroenterology of Army Medical Center of PLA and Sir Run Run Shaw Hospital.The training set and test set were randomly divided according to the ratio of 9:1 to train and test the neural network.A novel spatial and bilinear deep network (SABA-ResNet) was constructed on the basis of ResNet50.The spatial attention mechanism was introduced, and the receptive field was expanded by dilated convolution to leverage contextual information, which was combined with the local induction of standard convolution to adaptively focus the lesion region.Bilinear attention was applied to improve the feature representation ability of the network, and the second-order information was used to weight the channel information of the feature map, so as to improve the classification performance of the model. Results The overall accuracy of SABA-ResNet for the recognition of CD, UC and normal tissues on the test set was 92.67%(95%CI: 91.91~93.37), the AUC value was 0.978(95%CI: 0.972~0.983), 0.977(95%CI: 0.971~0.982) and 0.999(95%CI: 0.998~1.000), the sensitivity was 88.40%, 89.07% and 98.89%, the specificity was 95.49%, 94.88% and 98.93%, and the F1 value was 88.80%, 89.01% and 98.60%, respectively.The ablation experiment and the class activation map suggested that spatial attention and bilinear attention could help the model capture more features of the lesion region. Conclusion Our constructed network combines spatial attention and bilinear attention, achieves excellent performance in the recognition of CD, UC and normal tissue, and effectively assist endoscopists in the diagnosis of UC and CD. http://aammt.tmmu.edu.cn/Upload/rhtml/202209187.htminflammatory bowel diseasedeep learningulcerative colitiscrohn's disease |
| spellingShingle | QI Jing RUAN Guangcong YANG Yi WU Yi CAO Qian Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network 陆军军医大学学报 inflammatory bowel disease deep learning ulcerative colitis crohn's disease |
| title | Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network |
| title_full | Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network |
| title_fullStr | Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network |
| title_full_unstemmed | Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network |
| title_short | Identification of ulcerative colitis and Crohn's disease based on spatial and bilinear attention network |
| title_sort | identification of ulcerative colitis and crohn s disease based on spatial and bilinear attention network |
| topic | inflammatory bowel disease deep learning ulcerative colitis crohn's disease |
| url | http://aammt.tmmu.edu.cn/Upload/rhtml/202209187.htm |
| work_keys_str_mv | AT qijing identificationofulcerativecolitisandcrohnsdiseasebasedonspatialandbilinearattentionnetwork AT ruanguangcong identificationofulcerativecolitisandcrohnsdiseasebasedonspatialandbilinearattentionnetwork AT yangyi identificationofulcerativecolitisandcrohnsdiseasebasedonspatialandbilinearattentionnetwork AT wuyi identificationofulcerativecolitisandcrohnsdiseasebasedonspatialandbilinearattentionnetwork AT caoqian identificationofulcerativecolitisandcrohnsdiseasebasedonspatialandbilinearattentionnetwork |