Progressive With Purpose: Guiding Progressive Inpainting DNNs Through Context and Structure
The advent of deep learning in the past decade has significantly helped advance image inpainting. Although achieving promising performance, deep learning-based inpainting algorithms still struggle from the distortion caused by the fusion of structural and contextual features, which are commonly obta...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10005125/ |
| _version_ | 1797956358145835008 |
|---|---|
| author | Kangdi Shi Muhammad Alrabeiah Jun Chen |
| author_facet | Kangdi Shi Muhammad Alrabeiah Jun Chen |
| author_sort | Kangdi Shi |
| collection | DOAJ |
| description | The advent of deep learning in the past decade has significantly helped advance image inpainting. Although achieving promising performance, deep learning-based inpainting algorithms still struggle from the distortion caused by the fusion of structural and contextual features, which are commonly obtained from, respectively, deep and shallow layers of a convolutional encoder. Motivated by this observation, we propose a novel progressive inpainting network that maintains the structural and contextual integrity of a processed image. More specifically, inspired by the Gaussian and Laplacian pyramids, the core of the proposed network is a feature extraction module named GLE. Stacking GLE modules enables the network to extract image features from different image frequency components. This ability is important to maintain structural and contextual integrity, for high frequency components correspond to structural information while low frequency components correspond to contextual information. The proposed network utilizes the GLE features to progressively fill in missing regions in a corrupted image in an iterative manner. Our benchmarking experiments demonstrate that the proposed method achieves clear improvement in performance over many state-of-the-art inpainting algorithms. |
| first_indexed | 2024-04-10T23:47:43Z |
| format | Article |
| id | doaj.art-1b74c820b44346ee9b44c16b7a7ce69b |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-10T23:47:43Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-1b74c820b44346ee9b44c16b7a7ce69b2023-01-11T00:00:29ZengIEEEIEEE Access2169-35362023-01-01112023203410.1109/ACCESS.2022.323387410005125Progressive With Purpose: Guiding Progressive Inpainting DNNs Through Context and StructureKangdi Shi0https://orcid.org/0000-0002-9949-4252Muhammad Alrabeiah1https://orcid.org/0000-0001-7586-2631Jun Chen2https://orcid.org/0000-0002-8084-9332Department of Electrical and Computer Engineering, McMaster University, Hamilton, CanadaElectrical Engineering Department, King Saud University, Riyadh, Saudi ArabiaDepartment of Electrical and Computer Engineering, McMaster University, Hamilton, CanadaThe advent of deep learning in the past decade has significantly helped advance image inpainting. Although achieving promising performance, deep learning-based inpainting algorithms still struggle from the distortion caused by the fusion of structural and contextual features, which are commonly obtained from, respectively, deep and shallow layers of a convolutional encoder. Motivated by this observation, we propose a novel progressive inpainting network that maintains the structural and contextual integrity of a processed image. More specifically, inspired by the Gaussian and Laplacian pyramids, the core of the proposed network is a feature extraction module named GLE. Stacking GLE modules enables the network to extract image features from different image frequency components. This ability is important to maintain structural and contextual integrity, for high frequency components correspond to structural information while low frequency components correspond to contextual information. The proposed network utilizes the GLE features to progressively fill in missing regions in a corrupted image in an iterative manner. Our benchmarking experiments demonstrate that the proposed method achieves clear improvement in performance over many state-of-the-art inpainting algorithms.https://ieeexplore.ieee.org/document/10005125/Deep image inpaintingimage pyramid |
| spellingShingle | Kangdi Shi Muhammad Alrabeiah Jun Chen Progressive With Purpose: Guiding Progressive Inpainting DNNs Through Context and Structure IEEE Access Deep image inpainting image pyramid |
| title | Progressive With Purpose: Guiding Progressive Inpainting DNNs Through Context and Structure |
| title_full | Progressive With Purpose: Guiding Progressive Inpainting DNNs Through Context and Structure |
| title_fullStr | Progressive With Purpose: Guiding Progressive Inpainting DNNs Through Context and Structure |
| title_full_unstemmed | Progressive With Purpose: Guiding Progressive Inpainting DNNs Through Context and Structure |
| title_short | Progressive With Purpose: Guiding Progressive Inpainting DNNs Through Context and Structure |
| title_sort | progressive with purpose guiding progressive inpainting dnns through context and structure |
| topic | Deep image inpainting image pyramid |
| url | https://ieeexplore.ieee.org/document/10005125/ |
| work_keys_str_mv | AT kangdishi progressivewithpurposeguidingprogressiveinpaintingdnnsthroughcontextandstructure AT muhammadalrabeiah progressivewithpurposeguidingprogressiveinpaintingdnnsthroughcontextandstructure AT junchen progressivewithpurposeguidingprogressiveinpaintingdnnsthroughcontextandstructure |