Hyperspectral Video Tracker Based on Spectral Deviation Reduction and a Double Siamese Network
The advent of hyperspectral cameras has popularized the study of hyperspectral video trackers. Although hyperspectral images can better distinguish the targets compared to their RGB counterparts, the occlusion and rotation of the target affect the effectiveness of the target. For instance, occlusion...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-03-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/6/1579 |
| _version_ | 1827747828885094400 |
|---|---|
| author | Zhe Zhang Bin Hu Mengyuan Wang Pattathal V. Arun Dong Zhao Xuguang Zhu Jianling Hu Huan Li Huixin Zhou Kun Qian |
| author_facet | Zhe Zhang Bin Hu Mengyuan Wang Pattathal V. Arun Dong Zhao Xuguang Zhu Jianling Hu Huan Li Huixin Zhou Kun Qian |
| author_sort | Zhe Zhang |
| collection | DOAJ |
| description | The advent of hyperspectral cameras has popularized the study of hyperspectral video trackers. Although hyperspectral images can better distinguish the targets compared to their RGB counterparts, the occlusion and rotation of the target affect the effectiveness of the target. For instance, occlusion obscures the target, reducing the tracking accuracy and even causing tracking failure. In this regard, this paper proposes a novel hyperspectral video tracker where the double Siamese network (D-Siam) forms the basis of the framework. Moreover, AlexNet serves as the backbone of D-Siam. The current study also adopts a novel spectral–deviation-based dimensionality reduction approach on the learned features to match the input requirements of the AlexNet. It should be noted that the proposed dimensionality reduction method increases the distinction between the target and background. The two response maps, namely the initial response map and the adjacent response map, obtained using the D-Siam network, were fused using an adaptive weight estimation strategy. Finally, a confidence judgment module is proposed to regulate the update for the whole framework. A comparative analysis of the proposed approach with state-of-the-art trackers and an extensive ablation study were conducted on a publicly available benchmark hyperspectral dataset. The results show that the proposed tracker outperforms the existing state-of-the-art approaches against most of the challenges. |
| first_indexed | 2024-03-11T05:57:45Z |
| format | Article |
| id | doaj.art-ea0385abd9424b8885733c03c32d92bb |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-11T05:57:45Z |
| publishDate | 2023-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-ea0385abd9424b8885733c03c32d92bb2023-11-17T13:39:03ZengMDPI AGRemote Sensing2072-42922023-03-01156157910.3390/rs15061579Hyperspectral Video Tracker Based on Spectral Deviation Reduction and a Double Siamese NetworkZhe Zhang0Bin Hu1Mengyuan Wang2Pattathal V. Arun3Dong Zhao4Xuguang Zhu5Jianling Hu6Huan Li7Huixin Zhou8Kun Qian9School of Physics, Xidian University, Xi’an 710071, ChinaSchool of Electronics and Information Engineering, Wuxi University, Wuxi 214105, ChinaSchool of Electronics and Information Engineering, Wuxi University, Wuxi 214105, ChinaComputer Science and Engineering Group, Indian Institute of Information Technology, Sri City 441108, IndiaSchool of Physics, Xidian University, Xi’an 710071, ChinaSchool of Electronics and Information Engineering, Wuxi University, Wuxi 214105, ChinaSchool of Electronics and Information Engineering, Wuxi University, Wuxi 214105, ChinaSchool of Physics, Xidian University, Xi’an 710071, ChinaSchool of Physics, Xidian University, Xi’an 710071, ChinaSchool of Artificial Intelligence and Computer, Jiangnan University, Wuxi 214122, ChinaThe advent of hyperspectral cameras has popularized the study of hyperspectral video trackers. Although hyperspectral images can better distinguish the targets compared to their RGB counterparts, the occlusion and rotation of the target affect the effectiveness of the target. For instance, occlusion obscures the target, reducing the tracking accuracy and even causing tracking failure. In this regard, this paper proposes a novel hyperspectral video tracker where the double Siamese network (D-Siam) forms the basis of the framework. Moreover, AlexNet serves as the backbone of D-Siam. The current study also adopts a novel spectral–deviation-based dimensionality reduction approach on the learned features to match the input requirements of the AlexNet. It should be noted that the proposed dimensionality reduction method increases the distinction between the target and background. The two response maps, namely the initial response map and the adjacent response map, obtained using the D-Siam network, were fused using an adaptive weight estimation strategy. Finally, a confidence judgment module is proposed to regulate the update for the whole framework. A comparative analysis of the proposed approach with state-of-the-art trackers and an extensive ablation study were conducted on a publicly available benchmark hyperspectral dataset. The results show that the proposed tracker outperforms the existing state-of-the-art approaches against most of the challenges.https://www.mdpi.com/2072-4292/15/6/1579hyperspectral video trackerdouble Siamese networkspectral deviation reductionadaptive weightsconfidence judgment module |
| spellingShingle | Zhe Zhang Bin Hu Mengyuan Wang Pattathal V. Arun Dong Zhao Xuguang Zhu Jianling Hu Huan Li Huixin Zhou Kun Qian Hyperspectral Video Tracker Based on Spectral Deviation Reduction and a Double Siamese Network Remote Sensing hyperspectral video tracker double Siamese network spectral deviation reduction adaptive weights confidence judgment module |
| title | Hyperspectral Video Tracker Based on Spectral Deviation Reduction and a Double Siamese Network |
| title_full | Hyperspectral Video Tracker Based on Spectral Deviation Reduction and a Double Siamese Network |
| title_fullStr | Hyperspectral Video Tracker Based on Spectral Deviation Reduction and a Double Siamese Network |
| title_full_unstemmed | Hyperspectral Video Tracker Based on Spectral Deviation Reduction and a Double Siamese Network |
| title_short | Hyperspectral Video Tracker Based on Spectral Deviation Reduction and a Double Siamese Network |
| title_sort | hyperspectral video tracker based on spectral deviation reduction and a double siamese network |
| topic | hyperspectral video tracker double Siamese network spectral deviation reduction adaptive weights confidence judgment module |
| url | https://www.mdpi.com/2072-4292/15/6/1579 |
| work_keys_str_mv | AT zhezhang hyperspectralvideotrackerbasedonspectraldeviationreductionandadoublesiamesenetwork AT binhu hyperspectralvideotrackerbasedonspectraldeviationreductionandadoublesiamesenetwork AT mengyuanwang hyperspectralvideotrackerbasedonspectraldeviationreductionandadoublesiamesenetwork AT pattathalvarun hyperspectralvideotrackerbasedonspectraldeviationreductionandadoublesiamesenetwork AT dongzhao hyperspectralvideotrackerbasedonspectraldeviationreductionandadoublesiamesenetwork AT xuguangzhu hyperspectralvideotrackerbasedonspectraldeviationreductionandadoublesiamesenetwork AT jianlinghu hyperspectralvideotrackerbasedonspectraldeviationreductionandadoublesiamesenetwork AT huanli hyperspectralvideotrackerbasedonspectraldeviationreductionandadoublesiamesenetwork AT huixinzhou hyperspectralvideotrackerbasedonspectraldeviationreductionandadoublesiamesenetwork AT kunqian hyperspectralvideotrackerbasedonspectraldeviationreductionandadoublesiamesenetwork |