A virtual restoration network of ancient murals via global–local feature extraction and structural information guidance
Abstract Ancient murals are precious cultural heritages. They suffer from various damages due to man-made destruction and long-time exposure to the environment. It is urgent to protect and restore the damaged ancient murals. Virtual restoration of ancient murals aims to fill damaged mural regions by...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2023-12-01
|
| Series: | Heritage Science |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40494-023-01109-w |
| _version_ | 1797388168284078080 |
|---|---|
| author | Hao Ge Ying Yu Le Zhang |
| author_facet | Hao Ge Ying Yu Le Zhang |
| author_sort | Hao Ge |
| collection | DOAJ |
| description | Abstract Ancient murals are precious cultural heritages. They suffer from various damages due to man-made destruction and long-time exposure to the environment. It is urgent to protect and restore the damaged ancient murals. Virtual restoration of ancient murals aims to fill damaged mural regions by using modern computer techniques. Most existing restoration approaches fail to fill the loss mural regions with rich details and complex structures. In this paper, we propose a virtual restoration network of ancient murals based on global–local feature extraction and structural information guidance (GLSI). The proposed network consists of two major sub-networks: the structural information generator (SIG) and the image content generator (ICG). In the first sub-network, SIG can predict the structural information and the coarse contents for the missing mural regions. In the second sub-network, ICG utilizes the predicted structural information and the coarse contents to generate the refined image contents for the missing mural regions. Moreover, we design an innovative BranchBlock module that can effectively extract and integrate the local and global features. We introduce a Fast Fourier Convolution (FFC) to improve the color restoration for the missing mural regions. We conduct experiments over simulated and real damaged murals. Experimental results show that our proposed method outperforms other three comparative state-of-the-art approaches in terms of structural continuity, color harmony and visual rationality of the restored mural images. In addition, the mural restoration results of our method can achieve comparatively high quantitative evaluation metrics. |
| first_indexed | 2024-03-08T22:36:57Z |
| format | Article |
| id | doaj.art-f89e68d02d254d1da68b759238e369fe |
| institution | Directory Open Access Journal |
| issn | 2050-7445 |
| language | English |
| last_indexed | 2024-03-08T22:36:57Z |
| publishDate | 2023-12-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Heritage Science |
| spelling | doaj.art-f89e68d02d254d1da68b759238e369fe2023-12-17T12:26:41ZengSpringerOpenHeritage Science2050-74452023-12-0111111710.1186/s40494-023-01109-wA virtual restoration network of ancient murals via global–local feature extraction and structural information guidanceHao Ge0Ying Yu1Le Zhang2School of Information Science and Engineering, Yunnan UniversitySchool of Information Science and Engineering, Yunnan UniversitySchool of Information Science and Engineering, Yunnan UniversityAbstract Ancient murals are precious cultural heritages. They suffer from various damages due to man-made destruction and long-time exposure to the environment. It is urgent to protect and restore the damaged ancient murals. Virtual restoration of ancient murals aims to fill damaged mural regions by using modern computer techniques. Most existing restoration approaches fail to fill the loss mural regions with rich details and complex structures. In this paper, we propose a virtual restoration network of ancient murals based on global–local feature extraction and structural information guidance (GLSI). The proposed network consists of two major sub-networks: the structural information generator (SIG) and the image content generator (ICG). In the first sub-network, SIG can predict the structural information and the coarse contents for the missing mural regions. In the second sub-network, ICG utilizes the predicted structural information and the coarse contents to generate the refined image contents for the missing mural regions. Moreover, we design an innovative BranchBlock module that can effectively extract and integrate the local and global features. We introduce a Fast Fourier Convolution (FFC) to improve the color restoration for the missing mural regions. We conduct experiments over simulated and real damaged murals. Experimental results show that our proposed method outperforms other three comparative state-of-the-art approaches in terms of structural continuity, color harmony and visual rationality of the restored mural images. In addition, the mural restoration results of our method can achieve comparatively high quantitative evaluation metrics.https://doi.org/10.1186/s40494-023-01109-wMural restorationStructure informationBranchBlockFFC |
| spellingShingle | Hao Ge Ying Yu Le Zhang A virtual restoration network of ancient murals via global–local feature extraction and structural information guidance Heritage Science Mural restoration Structure information BranchBlock FFC |
| title | A virtual restoration network of ancient murals via global–local feature extraction and structural information guidance |
| title_full | A virtual restoration network of ancient murals via global–local feature extraction and structural information guidance |
| title_fullStr | A virtual restoration network of ancient murals via global–local feature extraction and structural information guidance |
| title_full_unstemmed | A virtual restoration network of ancient murals via global–local feature extraction and structural information guidance |
| title_short | A virtual restoration network of ancient murals via global–local feature extraction and structural information guidance |
| title_sort | virtual restoration network of ancient murals via global local feature extraction and structural information guidance |
| topic | Mural restoration Structure information BranchBlock FFC |
| url | https://doi.org/10.1186/s40494-023-01109-w |
| work_keys_str_mv | AT haoge avirtualrestorationnetworkofancientmuralsviagloballocalfeatureextractionandstructuralinformationguidance AT yingyu avirtualrestorationnetworkofancientmuralsviagloballocalfeatureextractionandstructuralinformationguidance AT lezhang avirtualrestorationnetworkofancientmuralsviagloballocalfeatureextractionandstructuralinformationguidance AT haoge virtualrestorationnetworkofancientmuralsviagloballocalfeatureextractionandstructuralinformationguidance AT yingyu virtualrestorationnetworkofancientmuralsviagloballocalfeatureextractionandstructuralinformationguidance AT lezhang virtualrestorationnetworkofancientmuralsviagloballocalfeatureextractionandstructuralinformationguidance |