Robust Visual Recognition in Poor Visibility Conditions: A Prior Knowledge-Guided Adversarial Learning Approach
Deep learning has achieved remarkable success in numerous computer vision tasks. However, recent research reveals that deep neural networks are vulnerable to natural perturbations from poor visibility conditions, limiting their practical applications. While several studies have focused on enhancing...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/12/17/3711 |
| _version_ | 1797582622929453056 |
|---|---|
| author | Jiangang Yang Jianfei Yang Luqing Luo Yun Wang Shizheng Wang Jian Liu |
| author_facet | Jiangang Yang Jianfei Yang Luqing Luo Yun Wang Shizheng Wang Jian Liu |
| author_sort | Jiangang Yang |
| collection | DOAJ |
| description | Deep learning has achieved remarkable success in numerous computer vision tasks. However, recent research reveals that deep neural networks are vulnerable to natural perturbations from poor visibility conditions, limiting their practical applications. While several studies have focused on enhancing model robustness in poor visibility conditions through techniques such as image restoration, data augmentation, and unsupervised domain adaptation, these efforts are predominantly confined to specific scenarios and fail to address multiple poor visibility scenarios encountered in real-world settings. Furthermore, the valuable prior knowledge inherent in poor visibility images is seldom utilized to aid in resolving high-level computer vision tasks. In light of these challenges, we propose a novel deep learning paradigm designed to bolster the robustness of object recognition across diverse poor visibility scenes. By observing the prior information in diverse poor visibility scenes, we integrate a feature matching module based on this prior knowledge into our proposed learning paradigm, aiming to facilitate deep models in learning more robust generic features at shallow levels. Moreover, to further enhance the robustness of deep features, we employ an adversarial learning strategy based on mutual information. This strategy combines the feature matching module to extract task-specific representations from low visibility scenes in a more robust manner, thereby enhancing the robustness of object recognition. We evaluate our approach on self-constructed datasets containing diverse poor visibility scenes, including visual blur, fog, rain, snow, and low illuminance. Extensive experiments demonstrate that our proposed method yields significant improvements over existing solutions across various poor visibility conditions. |
| first_indexed | 2024-03-10T23:25:00Z |
| format | Article |
| id | doaj.art-bc128a7f40d8423cb56a01b74d3f1c20 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-10T23:25:00Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-bc128a7f40d8423cb56a01b74d3f1c202023-11-19T08:02:57ZengMDPI AGElectronics2079-92922023-09-011217371110.3390/electronics12173711Robust Visual Recognition in Poor Visibility Conditions: A Prior Knowledge-Guided Adversarial Learning ApproachJiangang Yang0Jianfei Yang1Luqing Luo2Yun Wang3Shizheng Wang4Jian Liu5Institute of Microelectronics, Chinese Academy of Sciences, Beijing 100029, ChinaSchool of Electrical and Electronics Engineering, Nanyang Technological University, Singapore 639798, SingaporeInstitute of Microelectronics, Chinese Academy of Sciences, Beijing 100029, ChinaGuangdong Greater Bay Area Institute of Integrated Circuit and System, Guangzhou 510535, ChinaR&D Center for Internet of Things, Chinese Academy of Sciences, Wuxi 214200, ChinaInstitute of Microelectronics, Chinese Academy of Sciences, Beijing 100029, ChinaDeep learning has achieved remarkable success in numerous computer vision tasks. However, recent research reveals that deep neural networks are vulnerable to natural perturbations from poor visibility conditions, limiting their practical applications. While several studies have focused on enhancing model robustness in poor visibility conditions through techniques such as image restoration, data augmentation, and unsupervised domain adaptation, these efforts are predominantly confined to specific scenarios and fail to address multiple poor visibility scenarios encountered in real-world settings. Furthermore, the valuable prior knowledge inherent in poor visibility images is seldom utilized to aid in resolving high-level computer vision tasks. In light of these challenges, we propose a novel deep learning paradigm designed to bolster the robustness of object recognition across diverse poor visibility scenes. By observing the prior information in diverse poor visibility scenes, we integrate a feature matching module based on this prior knowledge into our proposed learning paradigm, aiming to facilitate deep models in learning more robust generic features at shallow levels. Moreover, to further enhance the robustness of deep features, we employ an adversarial learning strategy based on mutual information. This strategy combines the feature matching module to extract task-specific representations from low visibility scenes in a more robust manner, thereby enhancing the robustness of object recognition. We evaluate our approach on self-constructed datasets containing diverse poor visibility scenes, including visual blur, fog, rain, snow, and low illuminance. Extensive experiments demonstrate that our proposed method yields significant improvements over existing solutions across various poor visibility conditions.https://www.mdpi.com/2079-9292/12/17/3711robust visual recognitionpoor visibility conditionsunsupervised domain adaptationimage restoration |
| spellingShingle | Jiangang Yang Jianfei Yang Luqing Luo Yun Wang Shizheng Wang Jian Liu Robust Visual Recognition in Poor Visibility Conditions: A Prior Knowledge-Guided Adversarial Learning Approach Electronics robust visual recognition poor visibility conditions unsupervised domain adaptation image restoration |
| title | Robust Visual Recognition in Poor Visibility Conditions: A Prior Knowledge-Guided Adversarial Learning Approach |
| title_full | Robust Visual Recognition in Poor Visibility Conditions: A Prior Knowledge-Guided Adversarial Learning Approach |
| title_fullStr | Robust Visual Recognition in Poor Visibility Conditions: A Prior Knowledge-Guided Adversarial Learning Approach |
| title_full_unstemmed | Robust Visual Recognition in Poor Visibility Conditions: A Prior Knowledge-Guided Adversarial Learning Approach |
| title_short | Robust Visual Recognition in Poor Visibility Conditions: A Prior Knowledge-Guided Adversarial Learning Approach |
| title_sort | robust visual recognition in poor visibility conditions a prior knowledge guided adversarial learning approach |
| topic | robust visual recognition poor visibility conditions unsupervised domain adaptation image restoration |
| url | https://www.mdpi.com/2079-9292/12/17/3711 |
| work_keys_str_mv | AT jiangangyang robustvisualrecognitioninpoorvisibilityconditionsapriorknowledgeguidedadversariallearningapproach AT jianfeiyang robustvisualrecognitioninpoorvisibilityconditionsapriorknowledgeguidedadversariallearningapproach AT luqingluo robustvisualrecognitioninpoorvisibilityconditionsapriorknowledgeguidedadversariallearningapproach AT yunwang robustvisualrecognitioninpoorvisibilityconditionsapriorknowledgeguidedadversariallearningapproach AT shizhengwang robustvisualrecognitioninpoorvisibilityconditionsapriorknowledgeguidedadversariallearningapproach AT jianliu robustvisualrecognitioninpoorvisibilityconditionsapriorknowledgeguidedadversariallearningapproach |