Anomalous behavior recognition of underwater creatures using lite 3D full-convolution network
Abstract Global warming and pollution could lead to the destruction of marine habitats and loss of species. The anomalous behavior of underwater creatures can be used as a biometer for assessing the health status of our ocean. Advances in behavior recognition have been driven by the active applicati...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-11-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-47128-2 |
| _version_ | 1797576704097517568 |
|---|---|
| author | Jung-Hua Wang Te-Hua Hsu Yi-Chung Lai Yan-Tsung Peng Zhen-Yao Chen Ying-Ren Lin Chang-Wen Huang Chung-Ping Chiang |
| author_facet | Jung-Hua Wang Te-Hua Hsu Yi-Chung Lai Yan-Tsung Peng Zhen-Yao Chen Ying-Ren Lin Chang-Wen Huang Chung-Ping Chiang |
| author_sort | Jung-Hua Wang |
| collection | DOAJ |
| description | Abstract Global warming and pollution could lead to the destruction of marine habitats and loss of species. The anomalous behavior of underwater creatures can be used as a biometer for assessing the health status of our ocean. Advances in behavior recognition have been driven by the active application of deep learning methods, yet many of them render superior accuracy at the cost of high computational complexity and slow inference. This paper presents a real-time anomalous behavior recognition approach that incorporates a lightweight deep learning model (Lite3D), object detection, and multitarget tracking. Lite3D is characterized in threefold: (1) image frames contain only regions of interest (ROI) generated by an object detector; (2) no fully connected layers are needed, the prediction head itself is a flatten layer of 1 × $${\mathcal{l}}$$ l @ 1× 1, $${\mathcal{l}}$$ l = number of categories; (3) all the convolution kernels are 3D, except the first layer degenerated to 2D. Through the tracking, a sequence of ROI-only frames is subjected to 3D convolutions for stacked feature extraction. Compared to other 3D models, Lite3D is 50 times smaller in size and 57 times lighter in terms of trainable parameters and can achieve 99% of F1-score. Lite3D is ideal for mounting on ROV or AUV to perform real-time edge computing. |
| first_indexed | 2024-03-10T21:57:30Z |
| format | Article |
| id | doaj.art-7192bfb320be441e86352f778dc5be87 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-03-10T21:57:30Z |
| publishDate | 2023-11-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-7192bfb320be441e86352f778dc5be872023-11-19T13:05:09ZengNature PortfolioScientific Reports2045-23222023-11-0113111110.1038/s41598-023-47128-2Anomalous behavior recognition of underwater creatures using lite 3D full-convolution networkJung-Hua Wang0Te-Hua Hsu1Yi-Chung Lai2Yan-Tsung Peng3Zhen-Yao Chen4Ying-Ren Lin5Chang-Wen Huang6Chung-Ping Chiang7Deptartment of Electrical Engineering, National Taiwan Ocean UniversityDepartment of Aquaculture, National Taiwan Ocean UniversityDeptartment of Electrical Engineering, National Taiwan Ocean UniversityDeptartment of Computer Science, National Chengchi UniversityDeptartment of Electrical Engineering, National Taiwan Ocean UniversityDeptartment of Electrical Engineering, National Taiwan Ocean UniversityDepartment of Aquaculture, National Taiwan Ocean UniversityDepartment of Aquaculture, National Taiwan Ocean UniversityAbstract Global warming and pollution could lead to the destruction of marine habitats and loss of species. The anomalous behavior of underwater creatures can be used as a biometer for assessing the health status of our ocean. Advances in behavior recognition have been driven by the active application of deep learning methods, yet many of them render superior accuracy at the cost of high computational complexity and slow inference. This paper presents a real-time anomalous behavior recognition approach that incorporates a lightweight deep learning model (Lite3D), object detection, and multitarget tracking. Lite3D is characterized in threefold: (1) image frames contain only regions of interest (ROI) generated by an object detector; (2) no fully connected layers are needed, the prediction head itself is a flatten layer of 1 × $${\mathcal{l}}$$ l @ 1× 1, $${\mathcal{l}}$$ l = number of categories; (3) all the convolution kernels are 3D, except the first layer degenerated to 2D. Through the tracking, a sequence of ROI-only frames is subjected to 3D convolutions for stacked feature extraction. Compared to other 3D models, Lite3D is 50 times smaller in size and 57 times lighter in terms of trainable parameters and can achieve 99% of F1-score. Lite3D is ideal for mounting on ROV or AUV to perform real-time edge computing.https://doi.org/10.1038/s41598-023-47128-2 |
| spellingShingle | Jung-Hua Wang Te-Hua Hsu Yi-Chung Lai Yan-Tsung Peng Zhen-Yao Chen Ying-Ren Lin Chang-Wen Huang Chung-Ping Chiang Anomalous behavior recognition of underwater creatures using lite 3D full-convolution network Scientific Reports |
| title | Anomalous behavior recognition of underwater creatures using lite 3D full-convolution network |
| title_full | Anomalous behavior recognition of underwater creatures using lite 3D full-convolution network |
| title_fullStr | Anomalous behavior recognition of underwater creatures using lite 3D full-convolution network |
| title_full_unstemmed | Anomalous behavior recognition of underwater creatures using lite 3D full-convolution network |
| title_short | Anomalous behavior recognition of underwater creatures using lite 3D full-convolution network |
| title_sort | anomalous behavior recognition of underwater creatures using lite 3d full convolution network |
| url | https://doi.org/10.1038/s41598-023-47128-2 |
| work_keys_str_mv | AT junghuawang anomalousbehaviorrecognitionofunderwatercreaturesusinglite3dfullconvolutionnetwork AT tehuahsu anomalousbehaviorrecognitionofunderwatercreaturesusinglite3dfullconvolutionnetwork AT yichunglai anomalousbehaviorrecognitionofunderwatercreaturesusinglite3dfullconvolutionnetwork AT yantsungpeng anomalousbehaviorrecognitionofunderwatercreaturesusinglite3dfullconvolutionnetwork AT zhenyaochen anomalousbehaviorrecognitionofunderwatercreaturesusinglite3dfullconvolutionnetwork AT yingrenlin anomalousbehaviorrecognitionofunderwatercreaturesusinglite3dfullconvolutionnetwork AT changwenhuang anomalousbehaviorrecognitionofunderwatercreaturesusinglite3dfullconvolutionnetwork AT chungpingchiang anomalousbehaviorrecognitionofunderwatercreaturesusinglite3dfullconvolutionnetwork |