UMOTMA: Underwater multiple object tracking with memory aggregation
Underwater multi-object tracking (UMOT) is an important technology in marine animal ethology. It is affected by complex factors such as scattering, background interference, and occlusion, which makes it a challenging computer vision task. As a result, the stable continuation of trajectories among di...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-11-01
|
| Series: | Frontiers in Marine Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fmars.2022.1071618/full |
| _version_ | 1811211392532873216 |
|---|---|
| author | Zhicheng Hao Jun Qiu Haimiao Zhang Guangbo Ren Chang Liu |
| author_facet | Zhicheng Hao Jun Qiu Haimiao Zhang Guangbo Ren Chang Liu |
| author_sort | Zhicheng Hao |
| collection | DOAJ |
| description | Underwater multi-object tracking (UMOT) is an important technology in marine animal ethology. It is affected by complex factors such as scattering, background interference, and occlusion, which makes it a challenging computer vision task. As a result, the stable continuation of trajectories among different targets has been the key to the tracking performance of UMOT tasks. To solve such challenges, we propose an underwater multi-object tracking algorithm based on memory aggregation (UMOTMA) to effectively associate multiple frames with targets. First, we propose a long short-term memory (LSTM)-based memory aggregation module (LSMAM) to enhance memory utilization between multiple frames. Next, LSMAM embeds LSTM into the transformer structure to save and aggregate features between multiple frames. Then, an underwater image enhancement module ME is introduced to process the original underwater images, which improves the quality and visibility of the underwater images so that the model can extract better features from the images. Finally, LSMAM and ME are integrated with a backbone network to implement the entire algorithm framework, which can fully utilize the historical information of the tracked targets. Experiments on the UMOT datasets and the underwater fish school datasets show that UMOTMA generally outperforms existing models and can maintain the stability of the target trajectory while ensuring high-quality detection. The code is available via Github. |
| first_indexed | 2024-04-12T05:13:35Z |
| format | Article |
| id | doaj.art-2cb5f9160c1746adad9b41f93be79f89 |
| institution | Directory Open Access Journal |
| issn | 2296-7745 |
| language | English |
| last_indexed | 2024-04-12T05:13:35Z |
| publishDate | 2022-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Marine Science |
| spelling | doaj.art-2cb5f9160c1746adad9b41f93be79f892022-12-22T03:46:42ZengFrontiers Media S.A.Frontiers in Marine Science2296-77452022-11-01910.3389/fmars.2022.10716181071618UMOTMA: Underwater multiple object tracking with memory aggregationZhicheng Hao0Jun Qiu1Haimiao Zhang2Guangbo Ren3Chang Liu4Institute of Applied Mathematics, Beijing Information Science and Technology University, Beijing, ChinaInstitute of Applied Mathematics, Beijing Information Science and Technology University, Beijing, ChinaInstitute of Applied Mathematics, Beijing Information Science and Technology University, Beijing, ChinaFirst Institute of Oceanography, Ministry of Natural Resources, Qingdao, Shandong, ChinaInstitute of Applied Mathematics, Beijing Information Science and Technology University, Beijing, ChinaUnderwater multi-object tracking (UMOT) is an important technology in marine animal ethology. It is affected by complex factors such as scattering, background interference, and occlusion, which makes it a challenging computer vision task. As a result, the stable continuation of trajectories among different targets has been the key to the tracking performance of UMOT tasks. To solve such challenges, we propose an underwater multi-object tracking algorithm based on memory aggregation (UMOTMA) to effectively associate multiple frames with targets. First, we propose a long short-term memory (LSTM)-based memory aggregation module (LSMAM) to enhance memory utilization between multiple frames. Next, LSMAM embeds LSTM into the transformer structure to save and aggregate features between multiple frames. Then, an underwater image enhancement module ME is introduced to process the original underwater images, which improves the quality and visibility of the underwater images so that the model can extract better features from the images. Finally, LSMAM and ME are integrated with a backbone network to implement the entire algorithm framework, which can fully utilize the historical information of the tracked targets. Experiments on the UMOT datasets and the underwater fish school datasets show that UMOTMA generally outperforms existing models and can maintain the stability of the target trajectory while ensuring high-quality detection. The code is available via Github.https://www.frontiersin.org/articles/10.3389/fmars.2022.1071618/fullartificial intelligenceunderwater multiple object trackingmarine environmentlong-short term memoryvision transformer |
| spellingShingle | Zhicheng Hao Jun Qiu Haimiao Zhang Guangbo Ren Chang Liu UMOTMA: Underwater multiple object tracking with memory aggregation Frontiers in Marine Science artificial intelligence underwater multiple object tracking marine environment long-short term memory vision transformer |
| title | UMOTMA: Underwater multiple object tracking with memory aggregation |
| title_full | UMOTMA: Underwater multiple object tracking with memory aggregation |
| title_fullStr | UMOTMA: Underwater multiple object tracking with memory aggregation |
| title_full_unstemmed | UMOTMA: Underwater multiple object tracking with memory aggregation |
| title_short | UMOTMA: Underwater multiple object tracking with memory aggregation |
| title_sort | umotma underwater multiple object tracking with memory aggregation |
| topic | artificial intelligence underwater multiple object tracking marine environment long-short term memory vision transformer |
| url | https://www.frontiersin.org/articles/10.3389/fmars.2022.1071618/full |
| work_keys_str_mv | AT zhichenghao umotmaunderwatermultipleobjecttrackingwithmemoryaggregation AT junqiu umotmaunderwatermultipleobjecttrackingwithmemoryaggregation AT haimiaozhang umotmaunderwatermultipleobjecttrackingwithmemoryaggregation AT guangboren umotmaunderwatermultipleobjecttrackingwithmemoryaggregation AT changliu umotmaunderwatermultipleobjecttrackingwithmemoryaggregation |