Margin-Based Modal Adaptive Learning for Visible-Infrared Person Re-Identification
Visible-infrared person re-identification (VIPR) has great potential for intelligent transportation systems for constructing smart cities, but it is challenging to utilize due to the huge modal discrepancy between visible and infrared images. Although visible and infrared data can appear to be two d...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/3/1426 |
| _version_ | 1797623199004884992 |
|---|---|
| author | Qianqian Zhao Hanxiao Wu Jianqing Zhu |
| author_facet | Qianqian Zhao Hanxiao Wu Jianqing Zhu |
| author_sort | Qianqian Zhao |
| collection | DOAJ |
| description | Visible-infrared person re-identification (VIPR) has great potential for intelligent transportation systems for constructing smart cities, but it is challenging to utilize due to the huge modal discrepancy between visible and infrared images. Although visible and infrared data can appear to be two domains, VIPR is not identical to domain adaptation as it can massively eliminate modal discrepancies. Because VIPR has complete identity information on both visible and infrared modalities, once the domain adaption is overemphasized, the discriminative appearance information on the visible and infrared domains would drain. For that, we propose a novel margin-based modal adaptive learning (MMAL) method for VIPR in this paper. On each domain, we apply triplet and label smoothing cross-entropy functions to learn appearance-discriminative features. Between the two domains, we design a simple yet effective marginal maximum mean discrepancy (M<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msup><mrow></mrow><mn>3</mn></msup></semantics></math></inline-formula>D) loss function to avoid an excessive suppression of modal discrepancies to protect the features’ discriminative ability on each domain. As a result, our MMAL method could learn modal-invariant yet appearance-discriminative features for improving VIPR. The experimental results show that our MMAL method acquires state-of-the-art VIPR performance, e.g., on the RegDB dataset in the visible-to-infrared retrieval mode, the rank-1 accuracy is 93.24% and the mean average precision is 83.77%. |
| first_indexed | 2024-03-11T09:25:21Z |
| format | Article |
| id | doaj.art-806e0f935bb347f8a72afc7b70b3d9f2 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T09:25:21Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-806e0f935bb347f8a72afc7b70b3d9f22023-11-16T18:00:55ZengMDPI AGSensors1424-82202023-01-01233142610.3390/s23031426Margin-Based Modal Adaptive Learning for Visible-Infrared Person Re-IdentificationQianqian Zhao0Hanxiao Wu1Jianqing Zhu2College of Information Science and Engineering, Huaqiao University, Xiamen 361021, ChinaCollege of Information Science and Engineering, Huaqiao University, Xiamen 361021, ChinaCollege of Engineering, Huaqiao University, Quanzhou 362021, ChinaVisible-infrared person re-identification (VIPR) has great potential for intelligent transportation systems for constructing smart cities, but it is challenging to utilize due to the huge modal discrepancy between visible and infrared images. Although visible and infrared data can appear to be two domains, VIPR is not identical to domain adaptation as it can massively eliminate modal discrepancies. Because VIPR has complete identity information on both visible and infrared modalities, once the domain adaption is overemphasized, the discriminative appearance information on the visible and infrared domains would drain. For that, we propose a novel margin-based modal adaptive learning (MMAL) method for VIPR in this paper. On each domain, we apply triplet and label smoothing cross-entropy functions to learn appearance-discriminative features. Between the two domains, we design a simple yet effective marginal maximum mean discrepancy (M<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msup><mrow></mrow><mn>3</mn></msup></semantics></math></inline-formula>D) loss function to avoid an excessive suppression of modal discrepancies to protect the features’ discriminative ability on each domain. As a result, our MMAL method could learn modal-invariant yet appearance-discriminative features for improving VIPR. The experimental results show that our MMAL method acquires state-of-the-art VIPR performance, e.g., on the RegDB dataset in the visible-to-infrared retrieval mode, the rank-1 accuracy is 93.24% and the mean average precision is 83.77%.https://www.mdpi.com/1424-8220/23/3/1426deep learningmaximum mean discrepancyvisible-infrared person re-identification |
| spellingShingle | Qianqian Zhao Hanxiao Wu Jianqing Zhu Margin-Based Modal Adaptive Learning for Visible-Infrared Person Re-Identification Sensors deep learning maximum mean discrepancy visible-infrared person re-identification |
| title | Margin-Based Modal Adaptive Learning for Visible-Infrared Person Re-Identification |
| title_full | Margin-Based Modal Adaptive Learning for Visible-Infrared Person Re-Identification |
| title_fullStr | Margin-Based Modal Adaptive Learning for Visible-Infrared Person Re-Identification |
| title_full_unstemmed | Margin-Based Modal Adaptive Learning for Visible-Infrared Person Re-Identification |
| title_short | Margin-Based Modal Adaptive Learning for Visible-Infrared Person Re-Identification |
| title_sort | margin based modal adaptive learning for visible infrared person re identification |
| topic | deep learning maximum mean discrepancy visible-infrared person re-identification |
| url | https://www.mdpi.com/1424-8220/23/3/1426 |
| work_keys_str_mv | AT qianqianzhao marginbasedmodaladaptivelearningforvisibleinfraredpersonreidentification AT hanxiaowu marginbasedmodaladaptivelearningforvisibleinfraredpersonreidentification AT jianqingzhu marginbasedmodaladaptivelearningforvisibleinfraredpersonreidentification |