AMST2: aggregated multi-level spatial and temporal context-based transformer for robust aerial tracking
Abstract Recently, many existing visual trackers have made significant progress by incorporating either spatial information from multi-level convolution layers or temporal information for tracking. However, the complementary advantages of both spatial and temporal information cannot be leveraged whe...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-06-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-36131-2 |
| _version_ | 1797811572168458240 |
|---|---|
| author | Hasil Park Injae Lee Dasol Jeong Joonki Paik |
| author_facet | Hasil Park Injae Lee Dasol Jeong Joonki Paik |
| author_sort | Hasil Park |
| collection | DOAJ |
| description | Abstract Recently, many existing visual trackers have made significant progress by incorporating either spatial information from multi-level convolution layers or temporal information for tracking. However, the complementary advantages of both spatial and temporal information cannot be leveraged when these two types of information are used separately. In this paper, we present a new approach for robust visual tracking using a transformer-based model that incorporates both spatial and temporal context information at multiple levels. To integrate the refined similarity maps through multi-level spatial and temporal encoders, we propose an aggregation encoder. Consequently, the output of the proposed aggregation encoder contains useful features that integrate the global contexts of multi-level spatial and the temporal contexts. The feature we propose offers a contrasting yet complementary representation of multi-level spatial and temporal contexts. This characteristic is particularly beneficial in complex aerial scenarios, where tracking failures can occur due to occlusion, motion blur, small objects, and scale variations. Also, our tracker utilizes a light-weight network backbone, ensuring fast and effective object tracking in aerial datasets. Additionally, the proposed architecture can achieve more robust object tracking against significant variations by updating the features of the latest object while retaining the initial template information. Extensive experiments on seven challenging short-term and long-term aerial tracking benchmarks have demonstrated that the proposed tracker outperforms state-of-the-art tracking methods in terms of both real-time processing speed and performance. |
| first_indexed | 2024-03-13T07:24:41Z |
| format | Article |
| id | doaj.art-c8d6bd767834446391e1e8a278d4f013 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-03-13T07:24:41Z |
| publishDate | 2023-06-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-c8d6bd767834446391e1e8a278d4f0132023-06-04T11:27:09ZengNature PortfolioScientific Reports2045-23222023-06-0113111810.1038/s41598-023-36131-2AMST2: aggregated multi-level spatial and temporal context-based transformer for robust aerial trackingHasil Park0Injae Lee1Dasol Jeong2Joonki Paik3Department of Image, Chung-Ang UniversityDepartment of Artificial Intelligence, Chung-Ang UniversityDepartment of Image, Chung-Ang UniversityDepartment of Image, Chung-Ang UniversityAbstract Recently, many existing visual trackers have made significant progress by incorporating either spatial information from multi-level convolution layers or temporal information for tracking. However, the complementary advantages of both spatial and temporal information cannot be leveraged when these two types of information are used separately. In this paper, we present a new approach for robust visual tracking using a transformer-based model that incorporates both spatial and temporal context information at multiple levels. To integrate the refined similarity maps through multi-level spatial and temporal encoders, we propose an aggregation encoder. Consequently, the output of the proposed aggregation encoder contains useful features that integrate the global contexts of multi-level spatial and the temporal contexts. The feature we propose offers a contrasting yet complementary representation of multi-level spatial and temporal contexts. This characteristic is particularly beneficial in complex aerial scenarios, where tracking failures can occur due to occlusion, motion blur, small objects, and scale variations. Also, our tracker utilizes a light-weight network backbone, ensuring fast and effective object tracking in aerial datasets. Additionally, the proposed architecture can achieve more robust object tracking against significant variations by updating the features of the latest object while retaining the initial template information. Extensive experiments on seven challenging short-term and long-term aerial tracking benchmarks have demonstrated that the proposed tracker outperforms state-of-the-art tracking methods in terms of both real-time processing speed and performance.https://doi.org/10.1038/s41598-023-36131-2 |
| spellingShingle | Hasil Park Injae Lee Dasol Jeong Joonki Paik AMST2: aggregated multi-level spatial and temporal context-based transformer for robust aerial tracking Scientific Reports |
| title | AMST2: aggregated multi-level spatial and temporal context-based transformer for robust aerial tracking |
| title_full | AMST2: aggregated multi-level spatial and temporal context-based transformer for robust aerial tracking |
| title_fullStr | AMST2: aggregated multi-level spatial and temporal context-based transformer for robust aerial tracking |
| title_full_unstemmed | AMST2: aggregated multi-level spatial and temporal context-based transformer for robust aerial tracking |
| title_short | AMST2: aggregated multi-level spatial and temporal context-based transformer for robust aerial tracking |
| title_sort | amst2 aggregated multi level spatial and temporal context based transformer for robust aerial tracking |
| url | https://doi.org/10.1038/s41598-023-36131-2 |
| work_keys_str_mv | AT hasilpark amst2aggregatedmultilevelspatialandtemporalcontextbasedtransformerforrobustaerialtracking AT injaelee amst2aggregatedmultilevelspatialandtemporalcontextbasedtransformerforrobustaerialtracking AT dasoljeong amst2aggregatedmultilevelspatialandtemporalcontextbasedtransformerforrobustaerialtracking AT joonkipaik amst2aggregatedmultilevelspatialandtemporalcontextbasedtransformerforrobustaerialtracking |