Siamese Tracking Network with Spatial-Semantic-Aware Attention and Flexible Spatiotemporal Constraint
Siamese trackers based on classification and regression have drawn extensive attention due to their appropriate balance between accuracy and efficiency. However, most of them are prone to failure in the face of abrupt motion or appearance changes. This paper proposes a Siamese-based tracker that inc...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-01-01
|
| Series: | Symmetry |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-8994/16/1/61 |
| _version_ | 1797342554797113344 |
|---|---|
| author | Huanlong Zhang Panyun Wang Jie Zhang Fengxian Wang Xiaohui Song Hebin Zhou |
| author_facet | Huanlong Zhang Panyun Wang Jie Zhang Fengxian Wang Xiaohui Song Hebin Zhou |
| author_sort | Huanlong Zhang |
| collection | DOAJ |
| description | Siamese trackers based on classification and regression have drawn extensive attention due to their appropriate balance between accuracy and efficiency. However, most of them are prone to failure in the face of abrupt motion or appearance changes. This paper proposes a Siamese-based tracker that incorporates spatial-semantic-aware attention and flexible spatiotemporal constraint. First, we develop a spatial-semantic-aware attention model, which identifies the importance of each feature region and channel to target representation through the single convolution attention network with a loss function and increases the corresponding weights in the spatial and channel dimensions to reinforce the target region and semantic information on the target feature map. Secondly, considering that the traditional method unreasonably weights the target response in abrupt motion, we design a flexible spatiotemporal constraint. This constraint adaptively adjusts the constraint weights on the response map by evaluating the tracking result. Finally, we propose a new template updating the strategy. This strategy adaptively adjusts the contribution weights of the tracking result to the new template using depth correlation assessment criteria, thereby enhancing the reliability of the template. The Siamese network used in this paper is a symmetric neural network with dual input branches sharing weights. The experimental results on five challenging datasets show that our method outperformed other advanced algorithms. |
| first_indexed | 2024-03-08T10:34:59Z |
| format | Article |
| id | doaj.art-f664edb6f4674a30805da5a90a3bd398 |
| institution | Directory Open Access Journal |
| issn | 2073-8994 |
| language | English |
| last_indexed | 2024-03-08T10:34:59Z |
| publishDate | 2024-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Symmetry |
| spelling | doaj.art-f664edb6f4674a30805da5a90a3bd3982024-01-26T18:38:47ZengMDPI AGSymmetry2073-89942024-01-011616110.3390/sym16010061Siamese Tracking Network with Spatial-Semantic-Aware Attention and Flexible Spatiotemporal ConstraintHuanlong Zhang0Panyun Wang1Jie Zhang2Fengxian Wang3Xiaohui Song4Hebin Zhou5College of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450003, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450003, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450003, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450003, ChinaHenan Academy of Science, Zhengzhou 450008, ChinaCollege of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450003, ChinaSiamese trackers based on classification and regression have drawn extensive attention due to their appropriate balance between accuracy and efficiency. However, most of them are prone to failure in the face of abrupt motion or appearance changes. This paper proposes a Siamese-based tracker that incorporates spatial-semantic-aware attention and flexible spatiotemporal constraint. First, we develop a spatial-semantic-aware attention model, which identifies the importance of each feature region and channel to target representation through the single convolution attention network with a loss function and increases the corresponding weights in the spatial and channel dimensions to reinforce the target region and semantic information on the target feature map. Secondly, considering that the traditional method unreasonably weights the target response in abrupt motion, we design a flexible spatiotemporal constraint. This constraint adaptively adjusts the constraint weights on the response map by evaluating the tracking result. Finally, we propose a new template updating the strategy. This strategy adaptively adjusts the contribution weights of the tracking result to the new template using depth correlation assessment criteria, thereby enhancing the reliability of the template. The Siamese network used in this paper is a symmetric neural network with dual input branches sharing weights. The experimental results on five challenging datasets show that our method outperformed other advanced algorithms.https://www.mdpi.com/2073-8994/16/1/61object trackingaware attention modelspatiotemporal constrainttemplate updating |
| spellingShingle | Huanlong Zhang Panyun Wang Jie Zhang Fengxian Wang Xiaohui Song Hebin Zhou Siamese Tracking Network with Spatial-Semantic-Aware Attention and Flexible Spatiotemporal Constraint Symmetry object tracking aware attention model spatiotemporal constraint template updating |
| title | Siamese Tracking Network with Spatial-Semantic-Aware Attention and Flexible Spatiotemporal Constraint |
| title_full | Siamese Tracking Network with Spatial-Semantic-Aware Attention and Flexible Spatiotemporal Constraint |
| title_fullStr | Siamese Tracking Network with Spatial-Semantic-Aware Attention and Flexible Spatiotemporal Constraint |
| title_full_unstemmed | Siamese Tracking Network with Spatial-Semantic-Aware Attention and Flexible Spatiotemporal Constraint |
| title_short | Siamese Tracking Network with Spatial-Semantic-Aware Attention and Flexible Spatiotemporal Constraint |
| title_sort | siamese tracking network with spatial semantic aware attention and flexible spatiotemporal constraint |
| topic | object tracking aware attention model spatiotemporal constraint template updating |
| url | https://www.mdpi.com/2073-8994/16/1/61 |
| work_keys_str_mv | AT huanlongzhang siamesetrackingnetworkwithspatialsemanticawareattentionandflexiblespatiotemporalconstraint AT panyunwang siamesetrackingnetworkwithspatialsemanticawareattentionandflexiblespatiotemporalconstraint AT jiezhang siamesetrackingnetworkwithspatialsemanticawareattentionandflexiblespatiotemporalconstraint AT fengxianwang siamesetrackingnetworkwithspatialsemanticawareattentionandflexiblespatiotemporalconstraint AT xiaohuisong siamesetrackingnetworkwithspatialsemanticawareattentionandflexiblespatiotemporalconstraint AT hebinzhou siamesetrackingnetworkwithspatialsemanticawareattentionandflexiblespatiotemporalconstraint |