Deep learning for terahertz image denoising in nondestructive historical document analysis
Abstract Historical documents contain essential information about the past, including places, people, or events. Many of these valuable cultural artifacts cannot be further examined due to aging or external influences, as they are too fragile to be opened or turned over, so their rich contents remai...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-12-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-26957-7 |
| _version_ | 1797973644702384128 |
|---|---|
| author | Balaka Dutta Konstantin Root Ingrid Ullmann Fabian Wagner Martin Mayr Mathias Seuret Mareike Thies Daniel Stromer Vincent Christlein Jan Schür Andreas Maier Yixing Huang |
| author_facet | Balaka Dutta Konstantin Root Ingrid Ullmann Fabian Wagner Martin Mayr Mathias Seuret Mareike Thies Daniel Stromer Vincent Christlein Jan Schür Andreas Maier Yixing Huang |
| author_sort | Balaka Dutta |
| collection | DOAJ |
| description | Abstract Historical documents contain essential information about the past, including places, people, or events. Many of these valuable cultural artifacts cannot be further examined due to aging or external influences, as they are too fragile to be opened or turned over, so their rich contents remain hidden. Terahertz (THz) imaging is a nondestructive 3D imaging technique that can be used to reveal the hidden contents without damaging the documents. As noise or imaging artifacts are predominantly present in reconstructed images processed by standard THz reconstruction algorithms, this work intends to improve THz image quality with deep learning. To overcome the data scarcity problem in training a supervised deep learning model, an unsupervised deep learning network (CycleGAN) is first applied to generate paired noisy THz images from clean images (clean images are generated by a handwriting generator). With such synthetic noisy-to-clean paired images, a supervised deep learning model using Pix2pixGAN is trained, which is effective to enhance real noisy THz images. After Pix2pixGAN denoising, 99% characters written on one-side of the Xuan paper can be clearly recognized, while 61% characters written on one-side of the standard paper are sufficiently recognized. The average perceptual indices of Pix2pixGAN processed images are 16.83, which is very close to the average perceptual index 16.19 of clean handwriting images. Our work has important value for THz-imaging-based nondestructive historical document analysis. |
| first_indexed | 2024-04-11T04:07:33Z |
| format | Article |
| id | doaj.art-96de7ce983b74b0f9ac1be755ef06162 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-11T04:07:33Z |
| publishDate | 2022-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-96de7ce983b74b0f9ac1be755ef061622023-01-01T12:18:42ZengNature PortfolioScientific Reports2045-23222022-12-0112111110.1038/s41598-022-26957-7Deep learning for terahertz image denoising in nondestructive historical document analysisBalaka Dutta0Konstantin Root1Ingrid Ullmann2Fabian Wagner3Martin Mayr4Mathias Seuret5Mareike Thies6Daniel Stromer7Vincent Christlein8Jan Schür9Andreas Maier10Yixing Huang11Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-NürnbergInstitute of Microwaves and Photonics, Friedrich-Alexander-Universität Erlangen-NürnbergInstitute of Microwaves and Photonics, Friedrich-Alexander-Universität Erlangen-NürnbergPattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-NürnbergPattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-NürnbergPattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-NürnbergPattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-NürnbergPattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-NürnbergPattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-NürnbergInstitute of Microwaves and Photonics, Friedrich-Alexander-Universität Erlangen-NürnbergPattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-NürnbergDepartment of Radiation Oncology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-NürnbergAbstract Historical documents contain essential information about the past, including places, people, or events. Many of these valuable cultural artifacts cannot be further examined due to aging or external influences, as they are too fragile to be opened or turned over, so their rich contents remain hidden. Terahertz (THz) imaging is a nondestructive 3D imaging technique that can be used to reveal the hidden contents without damaging the documents. As noise or imaging artifacts are predominantly present in reconstructed images processed by standard THz reconstruction algorithms, this work intends to improve THz image quality with deep learning. To overcome the data scarcity problem in training a supervised deep learning model, an unsupervised deep learning network (CycleGAN) is first applied to generate paired noisy THz images from clean images (clean images are generated by a handwriting generator). With such synthetic noisy-to-clean paired images, a supervised deep learning model using Pix2pixGAN is trained, which is effective to enhance real noisy THz images. After Pix2pixGAN denoising, 99% characters written on one-side of the Xuan paper can be clearly recognized, while 61% characters written on one-side of the standard paper are sufficiently recognized. The average perceptual indices of Pix2pixGAN processed images are 16.83, which is very close to the average perceptual index 16.19 of clean handwriting images. Our work has important value for THz-imaging-based nondestructive historical document analysis.https://doi.org/10.1038/s41598-022-26957-7 |
| spellingShingle | Balaka Dutta Konstantin Root Ingrid Ullmann Fabian Wagner Martin Mayr Mathias Seuret Mareike Thies Daniel Stromer Vincent Christlein Jan Schür Andreas Maier Yixing Huang Deep learning for terahertz image denoising in nondestructive historical document analysis Scientific Reports |
| title | Deep learning for terahertz image denoising in nondestructive historical document analysis |
| title_full | Deep learning for terahertz image denoising in nondestructive historical document analysis |
| title_fullStr | Deep learning for terahertz image denoising in nondestructive historical document analysis |
| title_full_unstemmed | Deep learning for terahertz image denoising in nondestructive historical document analysis |
| title_short | Deep learning for terahertz image denoising in nondestructive historical document analysis |
| title_sort | deep learning for terahertz image denoising in nondestructive historical document analysis |
| url | https://doi.org/10.1038/s41598-022-26957-7 |
| work_keys_str_mv | AT balakadutta deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis AT konstantinroot deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis AT ingridullmann deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis AT fabianwagner deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis AT martinmayr deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis AT mathiasseuret deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis AT mareikethies deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis AT danielstromer deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis AT vincentchristlein deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis AT janschur deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis AT andreasmaier deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis AT yixinghuang deeplearningforterahertzimagedenoisinginnondestructivehistoricaldocumentanalysis |