Progressively Multi-Scale Feature Fusion for Image Inpainting
The rapid advancement of Wise Information Technology of med (WITMED) has made the integration of traditional Chinese medicine tongue diagnosis and computer technology an increasingly significant area of research. The doctor obtains patient’s tongue images to make a further diagnosis. However, the to...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-12-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/24/4908 |
| _version_ | 1797380217960923136 |
|---|---|
| author | Wu Wen Tianhao Li Amr Tolba Ziyi Liu Kai Shao |
| author_facet | Wu Wen Tianhao Li Amr Tolba Ziyi Liu Kai Shao |
| author_sort | Wu Wen |
| collection | DOAJ |
| description | The rapid advancement of Wise Information Technology of med (WITMED) has made the integration of traditional Chinese medicine tongue diagnosis and computer technology an increasingly significant area of research. The doctor obtains patient’s tongue images to make a further diagnosis. However, the tongue image may be broken during the process of collecting the tongue image. Due to the extremely complex texture of the tongue and significant individual differences, existing methods fail to fully obtain sufficient feature information, which result in inaccurate inpainted tongue images. To address this problem, we propose a recurrent tongue image inpainting algorithm based on multi-scale feature fusion called Multi-Scale Fusion Module and Recurrent Attention Mechanism Network (MSFM-RAM-Net). We first propose Multi-Scale Fusion Module (MSFM), which preserves the feature information of tongue images at different scales and enhances the consistency between structures. To simultaneously accelerate the inpainting process and enhance the quality of the inpainted results, Recurrent Attention Mechanism (RAM) is proposed. RAM focuses the network’s attention on important areas and uses known information to gradually inpaint image, which can avoid redundant feature information and the problem of texture confusion caused by large missing areas. Finally, we establish a tongue image dataset and use this dataset to qualitatively and quantitatively evaluate the MSFM-RAM-Net. The results shows that the MSFM-RAM-Net has a better effect on tongue image inpainting, with PSNR and SSIM increasing by 2.1% and 3.3%, respectively. |
| first_indexed | 2024-03-08T20:34:10Z |
| format | Article |
| id | doaj.art-bb74be4f0ad547af9e280c60e8dff7b6 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-08T20:34:10Z |
| publishDate | 2023-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-bb74be4f0ad547af9e280c60e8dff7b62023-12-22T14:23:16ZengMDPI AGMathematics2227-73902023-12-011124490810.3390/math11244908Progressively Multi-Scale Feature Fusion for Image InpaintingWu Wen0Tianhao Li1Amr Tolba2Ziyi Liu3Kai Shao4School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, ChinaSchool of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, ChinaComputer Science Department, Community College, King Saud University, Riyadh 11437, Saudi ArabiaSchool of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, ChinaSchool of Software, Dalian University of Technology, Dalian 116024, ChinaThe rapid advancement of Wise Information Technology of med (WITMED) has made the integration of traditional Chinese medicine tongue diagnosis and computer technology an increasingly significant area of research. The doctor obtains patient’s tongue images to make a further diagnosis. However, the tongue image may be broken during the process of collecting the tongue image. Due to the extremely complex texture of the tongue and significant individual differences, existing methods fail to fully obtain sufficient feature information, which result in inaccurate inpainted tongue images. To address this problem, we propose a recurrent tongue image inpainting algorithm based on multi-scale feature fusion called Multi-Scale Fusion Module and Recurrent Attention Mechanism Network (MSFM-RAM-Net). We first propose Multi-Scale Fusion Module (MSFM), which preserves the feature information of tongue images at different scales and enhances the consistency between structures. To simultaneously accelerate the inpainting process and enhance the quality of the inpainted results, Recurrent Attention Mechanism (RAM) is proposed. RAM focuses the network’s attention on important areas and uses known information to gradually inpaint image, which can avoid redundant feature information and the problem of texture confusion caused by large missing areas. Finally, we establish a tongue image dataset and use this dataset to qualitatively and quantitatively evaluate the MSFM-RAM-Net. The results shows that the MSFM-RAM-Net has a better effect on tongue image inpainting, with PSNR and SSIM increasing by 2.1% and 3.3%, respectively.https://www.mdpi.com/2227-7390/11/24/4908tongue image inpaintingMSFMRAMtongue image dataset |
| spellingShingle | Wu Wen Tianhao Li Amr Tolba Ziyi Liu Kai Shao Progressively Multi-Scale Feature Fusion for Image Inpainting Mathematics tongue image inpainting MSFM RAM tongue image dataset |
| title | Progressively Multi-Scale Feature Fusion for Image Inpainting |
| title_full | Progressively Multi-Scale Feature Fusion for Image Inpainting |
| title_fullStr | Progressively Multi-Scale Feature Fusion for Image Inpainting |
| title_full_unstemmed | Progressively Multi-Scale Feature Fusion for Image Inpainting |
| title_short | Progressively Multi-Scale Feature Fusion for Image Inpainting |
| title_sort | progressively multi scale feature fusion for image inpainting |
| topic | tongue image inpainting MSFM RAM tongue image dataset |
| url | https://www.mdpi.com/2227-7390/11/24/4908 |
| work_keys_str_mv | AT wuwen progressivelymultiscalefeaturefusionforimageinpainting AT tianhaoli progressivelymultiscalefeaturefusionforimageinpainting AT amrtolba progressivelymultiscalefeaturefusionforimageinpainting AT ziyiliu progressivelymultiscalefeaturefusionforimageinpainting AT kaishao progressivelymultiscalefeaturefusionforimageinpainting |