REPS: Rotation equivariant Siamese network enhanced by probability segmentation for satellite video tracking
Satellite video is an emerging surface observation data that has drawn increasing interest due to its potential in spatiotemporal dynamic analysis. Single object tracking of satellite videos allows the continuous acquisition of the positions and ranges of objects and establishes the correspondences...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-04-01
|
| Series: | International Journal of Applied Earth Observations and Geoinformation |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1569843224000955 |
| _version_ | 1797224324890886144 |
|---|---|
| author | Yuzeng Chen Yuqi Tang Qiangqiang Yuan Liangpei Zhang |
| author_facet | Yuzeng Chen Yuqi Tang Qiangqiang Yuan Liangpei Zhang |
| author_sort | Yuzeng Chen |
| collection | DOAJ |
| description | Satellite video is an emerging surface observation data that has drawn increasing interest due to its potential in spatiotemporal dynamic analysis. Single object tracking of satellite videos allows the continuous acquisition of the positions and ranges of objects and establishes the correspondences in the video sequence. However, small-sized objects are vulnerable to rotation and non-rigid deformation. Moreover, the horizontal bounding box of most trackers has difficulty in providing accurate semantic representations such as object position, orientation, and spatial distribution. In this article, we propose a unified framework, named rotation equivariant Siamese network enhanced by probability segmentation (REPS), to enhance the tracking accuracy and semantic representations simultaneously. First, to deal with the inconsistency of representations, we design a rotation equivariant (RE) Siamese network architecture to detect the rotation variations of objects right from the start frame, achieving the RE tracking. Second, a pixel-level (PL) refinement is proposed to refine the spatial distribution of objects. In addition, we proposed an adaptive Gaussian fusion that synergizes tracking and segmentation results to obtain compact outputs for satellite object representations. Extensive experiments on satellite videos demonstrate the superiority of the proposed approach. The code will be available at https://github.com/YZCU/REPS |
| first_indexed | 2024-04-24T13:51:19Z |
| format | Article |
| id | doaj.art-4503dc66397f47da9e6b37a0235d7358 |
| institution | Directory Open Access Journal |
| issn | 1569-8432 |
| language | English |
| last_indexed | 2024-04-24T13:51:19Z |
| publishDate | 2024-04-01 |
| publisher | Elsevier |
| record_format | Article |
| series | International Journal of Applied Earth Observations and Geoinformation |
| spelling | doaj.art-4503dc66397f47da9e6b37a0235d73582024-04-04T05:03:42ZengElsevierInternational Journal of Applied Earth Observations and Geoinformation1569-84322024-04-01128103741REPS: Rotation equivariant Siamese network enhanced by probability segmentation for satellite video trackingYuzeng Chen0Yuqi Tang1Qiangqiang Yuan2Liangpei Zhang3School of Geodesy and Geomatics, Wuhan University, Wuhan, ChinaSchool of Geosciences and Info-Physics, Central South University, Changsha, China; Corresponding author.School of Geodesy and Geomatics, Wuhan University, Wuhan, ChinaState Key Laboratory of Information Engineering, Survey Mapping and Remote Sensing, Wuhan University, Wuhan, ChinaSatellite video is an emerging surface observation data that has drawn increasing interest due to its potential in spatiotemporal dynamic analysis. Single object tracking of satellite videos allows the continuous acquisition of the positions and ranges of objects and establishes the correspondences in the video sequence. However, small-sized objects are vulnerable to rotation and non-rigid deformation. Moreover, the horizontal bounding box of most trackers has difficulty in providing accurate semantic representations such as object position, orientation, and spatial distribution. In this article, we propose a unified framework, named rotation equivariant Siamese network enhanced by probability segmentation (REPS), to enhance the tracking accuracy and semantic representations simultaneously. First, to deal with the inconsistency of representations, we design a rotation equivariant (RE) Siamese network architecture to detect the rotation variations of objects right from the start frame, achieving the RE tracking. Second, a pixel-level (PL) refinement is proposed to refine the spatial distribution of objects. In addition, we proposed an adaptive Gaussian fusion that synergizes tracking and segmentation results to obtain compact outputs for satellite object representations. Extensive experiments on satellite videos demonstrate the superiority of the proposed approach. The code will be available at https://github.com/YZCU/REPShttp://www.sciencedirect.com/science/article/pii/S1569843224000955Video satelliteObject trackingSiamese networkSemantic representation |
| spellingShingle | Yuzeng Chen Yuqi Tang Qiangqiang Yuan Liangpei Zhang REPS: Rotation equivariant Siamese network enhanced by probability segmentation for satellite video tracking International Journal of Applied Earth Observations and Geoinformation Video satellite Object tracking Siamese network Semantic representation |
| title | REPS: Rotation equivariant Siamese network enhanced by probability segmentation for satellite video tracking |
| title_full | REPS: Rotation equivariant Siamese network enhanced by probability segmentation for satellite video tracking |
| title_fullStr | REPS: Rotation equivariant Siamese network enhanced by probability segmentation for satellite video tracking |
| title_full_unstemmed | REPS: Rotation equivariant Siamese network enhanced by probability segmentation for satellite video tracking |
| title_short | REPS: Rotation equivariant Siamese network enhanced by probability segmentation for satellite video tracking |
| title_sort | reps rotation equivariant siamese network enhanced by probability segmentation for satellite video tracking |
| topic | Video satellite Object tracking Siamese network Semantic representation |
| url | http://www.sciencedirect.com/science/article/pii/S1569843224000955 |
| work_keys_str_mv | AT yuzengchen repsrotationequivariantsiamesenetworkenhancedbyprobabilitysegmentationforsatellitevideotracking AT yuqitang repsrotationequivariantsiamesenetworkenhancedbyprobabilitysegmentationforsatellitevideotracking AT qiangqiangyuan repsrotationequivariantsiamesenetworkenhancedbyprobabilitysegmentationforsatellitevideotracking AT liangpeizhang repsrotationequivariantsiamesenetworkenhancedbyprobabilitysegmentationforsatellitevideotracking |