Semantic‐guided fusion for multiple object tracking and RGB‐T tracking
Abstract The attention mechanism has produced impressive results in object tracking, but for a good trade‐off between performance and efficiency, CNN‐based approaches still dominate, owing to quadratic complexity of attention. Here, the SGF module is proposed, an efficient feature fusion block for e...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-09-01
|
| Series: | IET Image Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ipr2.12861 |
| _version_ | 1797693102766424064 |
|---|---|
| author | Xiaohu Liu Yichuang Luo Yan Zhang Zhiyong Lei |
| author_facet | Xiaohu Liu Yichuang Luo Yan Zhang Zhiyong Lei |
| author_sort | Xiaohu Liu |
| collection | DOAJ |
| description | Abstract The attention mechanism has produced impressive results in object tracking, but for a good trade‐off between performance and efficiency, CNN‐based approaches still dominate, owing to quadratic complexity of attention. Here, the SGF module is proposed, an efficient feature fusion block for effective object tracking with reduced linear complexity of attention. The proposed method fuses feature with attention in a coarse‐to‐fine manner. In the low‐resolution semantic branch, the top K regions with highest attention scores are selected; in the high‐resolution detail branch, attention is only calculated within regions corresponding to the top K regions. Thus, the features from the high‐resolution branch can be efficiently fused under the guidance of low‐resolution branch. Experiments on RGB and RGB‐T datasets with reformed FairMOT and MDNet+RGBT trackers demonstrated the effectiveness of the proposed method. |
| first_indexed | 2024-03-12T02:38:42Z |
| format | Article |
| id | doaj.art-d1fe766d52de41068a4b6370cec5375d |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-03-12T02:38:42Z |
| publishDate | 2023-09-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-d1fe766d52de41068a4b6370cec5375d2023-09-04T10:54:49ZengWileyIET Image Processing1751-96591751-96672023-09-0117113281329110.1049/ipr2.12861Semantic‐guided fusion for multiple object tracking and RGB‐T trackingXiaohu Liu0Yichuang Luo1Yan Zhang2Zhiyong Lei3School of Mechatronic Engineering Xi'an Technological University Xi'anChinaMeritData Technology Co., Ltd. Xi'anChinaSchool of Mechatronic Engineering Xi'an Technological University Xi'anChinaSchool of Electronic and Information Engineering Xi'an Technological University Xi'anChinaAbstract The attention mechanism has produced impressive results in object tracking, but for a good trade‐off between performance and efficiency, CNN‐based approaches still dominate, owing to quadratic complexity of attention. Here, the SGF module is proposed, an efficient feature fusion block for effective object tracking with reduced linear complexity of attention. The proposed method fuses feature with attention in a coarse‐to‐fine manner. In the low‐resolution semantic branch, the top K regions with highest attention scores are selected; in the high‐resolution detail branch, attention is only calculated within regions corresponding to the top K regions. Thus, the features from the high‐resolution branch can be efficiently fused under the guidance of low‐resolution branch. Experiments on RGB and RGB‐T datasets with reformed FairMOT and MDNet+RGBT trackers demonstrated the effectiveness of the proposed method.https://doi.org/10.1049/ipr2.12861computer visionimage fusionimage processingobject tracking |
| spellingShingle | Xiaohu Liu Yichuang Luo Yan Zhang Zhiyong Lei Semantic‐guided fusion for multiple object tracking and RGB‐T tracking IET Image Processing computer vision image fusion image processing object tracking |
| title | Semantic‐guided fusion for multiple object tracking and RGB‐T tracking |
| title_full | Semantic‐guided fusion for multiple object tracking and RGB‐T tracking |
| title_fullStr | Semantic‐guided fusion for multiple object tracking and RGB‐T tracking |
| title_full_unstemmed | Semantic‐guided fusion for multiple object tracking and RGB‐T tracking |
| title_short | Semantic‐guided fusion for multiple object tracking and RGB‐T tracking |
| title_sort | semantic guided fusion for multiple object tracking and rgb t tracking |
| topic | computer vision image fusion image processing object tracking |
| url | https://doi.org/10.1049/ipr2.12861 |
| work_keys_str_mv | AT xiaohuliu semanticguidedfusionformultipleobjecttrackingandrgbttracking AT yichuangluo semanticguidedfusionformultipleobjecttrackingandrgbttracking AT yanzhang semanticguidedfusionformultipleobjecttrackingandrgbttracking AT zhiyonglei semanticguidedfusionformultipleobjecttrackingandrgbttracking |