Attention‐based video object segmentation algorithm
Abstract To improve the segmentation performance on videos with large object motion or deformation, a novel scheme is proposed which has two branches. In one branch, the attention mechanism is first utilized to highlight objects‐related features. Then, to well consider the temporal coherence of vide...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2021-06-01
|
| Series: | IET Image Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ipr2.12135 |
| _version_ | 1811251961543000064 |
|---|---|
| author | Ying Cao Lijuan Sun Chong Han Jian Guo |
| author_facet | Ying Cao Lijuan Sun Chong Han Jian Guo |
| author_sort | Ying Cao |
| collection | DOAJ |
| description | Abstract To improve the segmentation performance on videos with large object motion or deformation, a novel scheme is proposed which has two branches. In one branch, the attention mechanism is first utilized to highlight objects‐related features. Then, to well consider the temporal coherence of videos, Conv3D is integrated to capture short‐term temporal features, and the designed attention residual convolutional long–short‐term memory is adopted to capture the long–short‐term temporal information of objects under the interference of redundant video frames. Meanwhile, considering the negative effect of background motion, in another branch, the optical flow‐based prediction model is introduced to predict objects regions in subsequent video frames with the annotated initial frame. At last, based on the fused results of two branches, the global thresholds and noising area clean method are employed to obtain segmented objects. The experiments on DAVIS2016 and CDnet2014 exhibit the competitive performance of the proposed scheme. |
| first_indexed | 2024-04-12T16:27:51Z |
| format | Article |
| id | doaj.art-dc7bef564c4a45398c0fce84359b4ab4 |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-04-12T16:27:51Z |
| publishDate | 2021-06-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-dc7bef564c4a45398c0fce84359b4ab42022-12-22T03:25:18ZengWileyIET Image Processing1751-96591751-96672021-06-011581668167810.1049/ipr2.12135Attention‐based video object segmentation algorithmYing Cao0Lijuan Sun1Chong Han2Jian Guo3Henan University Kaifeng Henan ChinaNanjing University of Posts and Telecommunications Nanjing ChinaNanjing University of Posts and Telecommunications Nanjing ChinaNanjing University of Posts and Telecommunications Nanjing ChinaAbstract To improve the segmentation performance on videos with large object motion or deformation, a novel scheme is proposed which has two branches. In one branch, the attention mechanism is first utilized to highlight objects‐related features. Then, to well consider the temporal coherence of videos, Conv3D is integrated to capture short‐term temporal features, and the designed attention residual convolutional long–short‐term memory is adopted to capture the long–short‐term temporal information of objects under the interference of redundant video frames. Meanwhile, considering the negative effect of background motion, in another branch, the optical flow‐based prediction model is introduced to predict objects regions in subsequent video frames with the annotated initial frame. At last, based on the fused results of two branches, the global thresholds and noising area clean method are employed to obtain segmented objects. The experiments on DAVIS2016 and CDnet2014 exhibit the competitive performance of the proposed scheme.https://doi.org/10.1049/ipr2.12135Optical, image and video signal processingComputer vision and image processing techniquesVideo signal processingNeural nets |
| spellingShingle | Ying Cao Lijuan Sun Chong Han Jian Guo Attention‐based video object segmentation algorithm IET Image Processing Optical, image and video signal processing Computer vision and image processing techniques Video signal processing Neural nets |
| title | Attention‐based video object segmentation algorithm |
| title_full | Attention‐based video object segmentation algorithm |
| title_fullStr | Attention‐based video object segmentation algorithm |
| title_full_unstemmed | Attention‐based video object segmentation algorithm |
| title_short | Attention‐based video object segmentation algorithm |
| title_sort | attention based video object segmentation algorithm |
| topic | Optical, image and video signal processing Computer vision and image processing techniques Video signal processing Neural nets |
| url | https://doi.org/10.1049/ipr2.12135 |
| work_keys_str_mv | AT yingcao attentionbasedvideoobjectsegmentationalgorithm AT lijuansun attentionbasedvideoobjectsegmentationalgorithm AT chonghan attentionbasedvideoobjectsegmentationalgorithm AT jianguo attentionbasedvideoobjectsegmentationalgorithm |