Vison transformer adapter-based hyperbolic embeddings for multi-lesion segmentation in diabetic retinopathy
Abstract Diabetic Retinopathy (DR) is a major cause of blindness worldwide. Early detection and treatment are crucial to prevent vision loss, making accurate and timely diagnosis critical. Deep learning technology has shown promise in the automated diagnosis of DR, and in particular, multi-lesion se...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-07-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-38320-5 |
| _version_ | 1827899893617786880 |
|---|---|
| author | Zijian Wang Haimei Lu Haixin Yan Hongxing Kan Li Jin |
| author_facet | Zijian Wang Haimei Lu Haixin Yan Hongxing Kan Li Jin |
| author_sort | Zijian Wang |
| collection | DOAJ |
| description | Abstract Diabetic Retinopathy (DR) is a major cause of blindness worldwide. Early detection and treatment are crucial to prevent vision loss, making accurate and timely diagnosis critical. Deep learning technology has shown promise in the automated diagnosis of DR, and in particular, multi-lesion segmentation tasks. In this paper, we propose a novel Transformer-based model for DR segmentation that incorporates hyperbolic embeddings and a spatial prior module. The proposed model is primarily built on a traditional Vision Transformer encoder and further enhanced by incorporating a spatial prior module for image convolution and feature continuity, followed by feature interaction processing using the spatial feature injector and extractor. Hyperbolic embeddings are used to classify feature matrices from the model at the pixel level. We evaluated the proposed model’s performance on the publicly available datasets and compared it with other widely used DR segmentation models. The results show that our model outperforms these widely used DR segmentation models. The incorporation of hyperbolic embeddings and a spatial prior module into the Vision Transformer-based model significantly improves the accuracy of DR segmentation. The hyperbolic embeddings enable us to better capture the underlying geometric structure of the feature matrices, which is important for accurate segmentation. The spatial prior module improves the continuity of the features and helps to better distinguish between lesions and normal tissues. Overall, our proposed model has potential for clinical use in automated DR diagnosis, improving accuracy and speed of diagnosis. Our study shows that the integration of hyperbolic embeddings and a spatial prior module with a Vision Transformer-based model improves the performance of DR segmentation models. Future research can explore the application of our model to other medical imaging tasks, as well as further optimization and validation in real-world clinical settings. |
| first_indexed | 2024-03-12T23:23:50Z |
| format | Article |
| id | doaj.art-00502a0136504dd4becb81b8f5198973 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-03-12T23:23:50Z |
| publishDate | 2023-07-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-00502a0136504dd4becb81b8f51989732023-07-16T11:16:20ZengNature PortfolioScientific Reports2045-23222023-07-0113111310.1038/s41598-023-38320-5Vison transformer adapter-based hyperbolic embeddings for multi-lesion segmentation in diabetic retinopathyZijian Wang0Haimei Lu1Haixin Yan2Hongxing Kan3Li Jin4School of Medicine and Information Engineering, Anhui University of Chinese MedicineSchool of Basic Medical Sciences, Anhui Medical UniversityHefei University of TechnologySchool of Medicine and Information Engineering, Anhui University of Chinese MedicineSchool of Medicine and Information Engineering, Anhui University of Chinese MedicineAbstract Diabetic Retinopathy (DR) is a major cause of blindness worldwide. Early detection and treatment are crucial to prevent vision loss, making accurate and timely diagnosis critical. Deep learning technology has shown promise in the automated diagnosis of DR, and in particular, multi-lesion segmentation tasks. In this paper, we propose a novel Transformer-based model for DR segmentation that incorporates hyperbolic embeddings and a spatial prior module. The proposed model is primarily built on a traditional Vision Transformer encoder and further enhanced by incorporating a spatial prior module for image convolution and feature continuity, followed by feature interaction processing using the spatial feature injector and extractor. Hyperbolic embeddings are used to classify feature matrices from the model at the pixel level. We evaluated the proposed model’s performance on the publicly available datasets and compared it with other widely used DR segmentation models. The results show that our model outperforms these widely used DR segmentation models. The incorporation of hyperbolic embeddings and a spatial prior module into the Vision Transformer-based model significantly improves the accuracy of DR segmentation. The hyperbolic embeddings enable us to better capture the underlying geometric structure of the feature matrices, which is important for accurate segmentation. The spatial prior module improves the continuity of the features and helps to better distinguish between lesions and normal tissues. Overall, our proposed model has potential for clinical use in automated DR diagnosis, improving accuracy and speed of diagnosis. Our study shows that the integration of hyperbolic embeddings and a spatial prior module with a Vision Transformer-based model improves the performance of DR segmentation models. Future research can explore the application of our model to other medical imaging tasks, as well as further optimization and validation in real-world clinical settings.https://doi.org/10.1038/s41598-023-38320-5 |
| spellingShingle | Zijian Wang Haimei Lu Haixin Yan Hongxing Kan Li Jin Vison transformer adapter-based hyperbolic embeddings for multi-lesion segmentation in diabetic retinopathy Scientific Reports |
| title | Vison transformer adapter-based hyperbolic embeddings for multi-lesion segmentation in diabetic retinopathy |
| title_full | Vison transformer adapter-based hyperbolic embeddings for multi-lesion segmentation in diabetic retinopathy |
| title_fullStr | Vison transformer adapter-based hyperbolic embeddings for multi-lesion segmentation in diabetic retinopathy |
| title_full_unstemmed | Vison transformer adapter-based hyperbolic embeddings for multi-lesion segmentation in diabetic retinopathy |
| title_short | Vison transformer adapter-based hyperbolic embeddings for multi-lesion segmentation in diabetic retinopathy |
| title_sort | vison transformer adapter based hyperbolic embeddings for multi lesion segmentation in diabetic retinopathy |
| url | https://doi.org/10.1038/s41598-023-38320-5 |
| work_keys_str_mv | AT zijianwang visontransformeradapterbasedhyperbolicembeddingsformultilesionsegmentationindiabeticretinopathy AT haimeilu visontransformeradapterbasedhyperbolicembeddingsformultilesionsegmentationindiabeticretinopathy AT haixinyan visontransformeradapterbasedhyperbolicembeddingsformultilesionsegmentationindiabeticretinopathy AT hongxingkan visontransformeradapterbasedhyperbolicembeddingsformultilesionsegmentationindiabeticretinopathy AT lijin visontransformeradapterbasedhyperbolicembeddingsformultilesionsegmentationindiabeticretinopathy |