EXTRACTION OF PAINT LOSS IN ANCIENT MURALS BASED ON 3D RESIDUAL NEURAL NETWORK
The ancient murals of Qutan Temple in Qinghai Province have a very serious loss of paint. Moreover, the main components of the base color paint layer in the paint loss area and the white patterns in the murals are both calcified, which are similar in color and spectral features. Thus, it is difficul...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2023-12-01
|
| Series: | ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://isprs-annals.copernicus.org/articles/X-1-W1-2023/199/2023/isprs-annals-X-1-W1-2023-199-2023.pdf |
| _version_ | 1797403419208581120 |
|---|---|
| author | S. Y. Li S. Y. Li M. L. Hou M. L. Hou P. H. Cao S. Q. Lyu S. Q. Lyu |
| author_facet | S. Y. Li S. Y. Li M. L. Hou M. L. Hou P. H. Cao S. Q. Lyu S. Q. Lyu |
| author_sort | S. Y. Li |
| collection | DOAJ |
| description | The ancient murals of Qutan Temple in Qinghai Province have a very serious loss of paint. Moreover, the main components of the base color paint layer in the paint loss area and the white patterns in the murals are both calcified, which are similar in color and spectral features. Thus, it is difficult to distinguish them by only using spectral features. A method of paint loss area extraction based on 3D residual network with multi-scale feature fusion is proposed. Firstly, the hyperspectral images with paint loss regions were collected by hyperspectral images. They are pre-processed to establish the training data set. Secondly, 3D residual network models are constructed using 3×3×3, 3×3×5 and 5×5×3 convolution kernels to realize the extraction and fusion of spatial and spectral features at different scales of hyperspectral images. The produced mural hyperspectral dataset is used for network training to obtain the prediction model. Finally, the hyperspectral images are input into the trained model to achieve the extraction of paint loss. After comparing different methods, the experimental result shows that the proposed method can improve the extraction accuracy of mural paint loss and serve as a reference for other deteriorations extraction. |
| first_indexed | 2024-03-09T02:38:16Z |
| format | Article |
| id | doaj.art-455651985b254529b34f0327c1c72a73 |
| institution | Directory Open Access Journal |
| issn | 2194-9042 2194-9050 |
| language | English |
| last_indexed | 2024-03-09T02:38:16Z |
| publishDate | 2023-12-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj.art-455651985b254529b34f0327c1c72a732023-12-06T07:05:14ZengCopernicus PublicationsISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences2194-90422194-90502023-12-01X-1-W1-202319920610.5194/isprs-annals-X-1-W1-2023-199-2023EXTRACTION OF PAINT LOSS IN ANCIENT MURALS BASED ON 3D RESIDUAL NEURAL NETWORKS. Y. Li0S. Y. Li1M. L. Hou2M. L. Hou3P. H. Cao4S. Q. Lyu5S. Q. Lyu6School of Geomatics and Urban Spatial Informatics, Beijing University of Civil Engineering and Architecture, No.15 Yongyuan Road, Daxing District, Beijing, ChinaBeijing Key Laboratory for Architectural Heritage Fine Reconstruction & Health Monitoring, No.15 Yongyuan Road, Daxing District, Beijing, ChinaSchool of Geomatics and Urban Spatial Informatics, Beijing University of Civil Engineering and Architecture, No.15 Yongyuan Road, Daxing District, Beijing, ChinaBeijing Key Laboratory for Architectural Heritage Fine Reconstruction & Health Monitoring, No.15 Yongyuan Road, Daxing District, Beijing, ChinaShenzhen Feima Robotics Technology Co., LTD., No.8 Heiquan Road, Haidian Distruct, Beijing, ChinaSchool of Geomatics and Urban Spatial Informatics, Beijing University of Civil Engineering and Architecture, No.15 Yongyuan Road, Daxing District, Beijing, ChinaBeijing Key Laboratory for Architectural Heritage Fine Reconstruction & Health Monitoring, No.15 Yongyuan Road, Daxing District, Beijing, ChinaThe ancient murals of Qutan Temple in Qinghai Province have a very serious loss of paint. Moreover, the main components of the base color paint layer in the paint loss area and the white patterns in the murals are both calcified, which are similar in color and spectral features. Thus, it is difficult to distinguish them by only using spectral features. A method of paint loss area extraction based on 3D residual network with multi-scale feature fusion is proposed. Firstly, the hyperspectral images with paint loss regions were collected by hyperspectral images. They are pre-processed to establish the training data set. Secondly, 3D residual network models are constructed using 3×3×3, 3×3×5 and 5×5×3 convolution kernels to realize the extraction and fusion of spatial and spectral features at different scales of hyperspectral images. The produced mural hyperspectral dataset is used for network training to obtain the prediction model. Finally, the hyperspectral images are input into the trained model to achieve the extraction of paint loss. After comparing different methods, the experimental result shows that the proposed method can improve the extraction accuracy of mural paint loss and serve as a reference for other deteriorations extraction.https://isprs-annals.copernicus.org/articles/X-1-W1-2023/199/2023/isprs-annals-X-1-W1-2023-199-2023.pdf |
| spellingShingle | S. Y. Li S. Y. Li M. L. Hou M. L. Hou P. H. Cao S. Q. Lyu S. Q. Lyu EXTRACTION OF PAINT LOSS IN ANCIENT MURALS BASED ON 3D RESIDUAL NEURAL NETWORK ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | EXTRACTION OF PAINT LOSS IN ANCIENT MURALS BASED ON 3D RESIDUAL NEURAL NETWORK |
| title_full | EXTRACTION OF PAINT LOSS IN ANCIENT MURALS BASED ON 3D RESIDUAL NEURAL NETWORK |
| title_fullStr | EXTRACTION OF PAINT LOSS IN ANCIENT MURALS BASED ON 3D RESIDUAL NEURAL NETWORK |
| title_full_unstemmed | EXTRACTION OF PAINT LOSS IN ANCIENT MURALS BASED ON 3D RESIDUAL NEURAL NETWORK |
| title_short | EXTRACTION OF PAINT LOSS IN ANCIENT MURALS BASED ON 3D RESIDUAL NEURAL NETWORK |
| title_sort | extraction of paint loss in ancient murals based on 3d residual neural network |
| url | https://isprs-annals.copernicus.org/articles/X-1-W1-2023/199/2023/isprs-annals-X-1-W1-2023-199-2023.pdf |
| work_keys_str_mv | AT syli extractionofpaintlossinancientmuralsbasedon3dresidualneuralnetwork AT syli extractionofpaintlossinancientmuralsbasedon3dresidualneuralnetwork AT mlhou extractionofpaintlossinancientmuralsbasedon3dresidualneuralnetwork AT mlhou extractionofpaintlossinancientmuralsbasedon3dresidualneuralnetwork AT phcao extractionofpaintlossinancientmuralsbasedon3dresidualneuralnetwork AT sqlyu extractionofpaintlossinancientmuralsbasedon3dresidualneuralnetwork AT sqlyu extractionofpaintlossinancientmuralsbasedon3dresidualneuralnetwork |