Leveraging Feature Extraction and Context Information for Image Relighting
Example-based image relighting aims to relight an input image to follow the lighting settings of another target example image. Deep learning-based methods for such tasks have become highly popular. However, they are often limited by the geometric priors or suffer from shadow reconstruction and lack...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-10-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/12/20/4301 |
| _version_ | 1797574052238327808 |
|---|---|
| author | Chenrong Fang Ju Wang Kan Chen Ran Su Chi-Fu Lai Qian Sun |
| author_facet | Chenrong Fang Ju Wang Kan Chen Ran Su Chi-Fu Lai Qian Sun |
| author_sort | Chenrong Fang |
| collection | DOAJ |
| description | Example-based image relighting aims to relight an input image to follow the lighting settings of another target example image. Deep learning-based methods for such tasks have become highly popular. However, they are often limited by the geometric priors or suffer from shadow reconstruction and lack of texture details. In this paper, we propose an image-to-image translation network called <i>DGATRN</i> to tackle this problem by enhancing feature extraction and unveiling context information to achieve visually plausible example-based image relighting. Specifically, the proposed <i>DGATRN</i> consists of a scene extraction, a shadow calibration, and a rendering network, and our key contribution lies in the first two networks. We propose an up- and downsampling approach to improve the feature extraction capability to capture scene and texture details better. We also introduce a feature attention downsampling block and a knowledge transfer to utilize the attention impact and underlying knowledge connection between scene and shadow. Experiments were conducted to evaluate the usefulness and effectiveness of the proposed method. |
| first_indexed | 2024-03-10T21:17:46Z |
| format | Article |
| id | doaj.art-9c2b582ba47e4795842e02a09d9e45d1 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-10T21:17:46Z |
| publishDate | 2023-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-9c2b582ba47e4795842e02a09d9e45d12023-11-19T16:19:42ZengMDPI AGElectronics2079-92922023-10-011220430110.3390/electronics12204301Leveraging Feature Extraction and Context Information for Image RelightingChenrong Fang0Ju Wang1Kan Chen2Ran Su3Chi-Fu Lai4Qian Sun5College of Intelligence and Computing, Tianjin University, Tianjin 300072, ChinaCollege of Intelligence and Computing, Tianjin University, Tianjin 300072, ChinaInfocomm Technology Cluster, Singapore Institute of Technology, Singapore 138683, SingaporeCollege of Intelligence and Computing, Tianjin University, Tianjin 300072, ChinaSchool of Arts and Social Sciences, Hong Kong Metropolitan University, Hong Kong, ChinaSchool of Electronic and Information Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, ChinaExample-based image relighting aims to relight an input image to follow the lighting settings of another target example image. Deep learning-based methods for such tasks have become highly popular. However, they are often limited by the geometric priors or suffer from shadow reconstruction and lack of texture details. In this paper, we propose an image-to-image translation network called <i>DGATRN</i> to tackle this problem by enhancing feature extraction and unveiling context information to achieve visually plausible example-based image relighting. Specifically, the proposed <i>DGATRN</i> consists of a scene extraction, a shadow calibration, and a rendering network, and our key contribution lies in the first two networks. We propose an up- and downsampling approach to improve the feature extraction capability to capture scene and texture details better. We also introduce a feature attention downsampling block and a knowledge transfer to utilize the attention impact and underlying knowledge connection between scene and shadow. Experiments were conducted to evaluate the usefulness and effectiveness of the proposed method.https://www.mdpi.com/2079-9292/12/20/4301image relightingupsampling and downsamplingattentionknowledge transferneural network |
| spellingShingle | Chenrong Fang Ju Wang Kan Chen Ran Su Chi-Fu Lai Qian Sun Leveraging Feature Extraction and Context Information for Image Relighting Electronics image relighting upsampling and downsampling attention knowledge transfer neural network |
| title | Leveraging Feature Extraction and Context Information for Image Relighting |
| title_full | Leveraging Feature Extraction and Context Information for Image Relighting |
| title_fullStr | Leveraging Feature Extraction and Context Information for Image Relighting |
| title_full_unstemmed | Leveraging Feature Extraction and Context Information for Image Relighting |
| title_short | Leveraging Feature Extraction and Context Information for Image Relighting |
| title_sort | leveraging feature extraction and context information for image relighting |
| topic | image relighting upsampling and downsampling attention knowledge transfer neural network |
| url | https://www.mdpi.com/2079-9292/12/20/4301 |
| work_keys_str_mv | AT chenrongfang leveragingfeatureextractionandcontextinformationforimagerelighting AT juwang leveragingfeatureextractionandcontextinformationforimagerelighting AT kanchen leveragingfeatureextractionandcontextinformationforimagerelighting AT ransu leveragingfeatureextractionandcontextinformationforimagerelighting AT chifulai leveragingfeatureextractionandcontextinformationforimagerelighting AT qiansun leveragingfeatureextractionandcontextinformationforimagerelighting |