Machine learning applied to big data from marine cabled observatories: A case study of sablefish monitoring in the NE Pacific
Ocean observatories collect large volumes of video data, with some data archives now spanning well over a few decades, and bringing the challenges of analytical capacity beyond conventional processing tools. The analysis of such vast and complex datasets can only be achieved with appropriate machine...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-08-01
|
| Series: | Frontiers in Marine Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fmars.2022.842946/full |
| _version_ | 1828432480466632704 |
|---|---|
| author | Federico Bonofiglio Fabio C. De Leo Fabio C. De Leo Connor Yee Damianos Chatzievangelou Jacopo Aguzzi Jacopo Aguzzi Simone Marini Simone Marini |
| author_facet | Federico Bonofiglio Fabio C. De Leo Fabio C. De Leo Connor Yee Damianos Chatzievangelou Jacopo Aguzzi Jacopo Aguzzi Simone Marini Simone Marini |
| author_sort | Federico Bonofiglio |
| collection | DOAJ |
| description | Ocean observatories collect large volumes of video data, with some data archives now spanning well over a few decades, and bringing the challenges of analytical capacity beyond conventional processing tools. The analysis of such vast and complex datasets can only be achieved with appropriate machine learning and Artificial Intelligence (AI) tools. The implementation of AI monitoring programs for animal tracking and classification becomes necessary in the particular case of deep-sea cabled observatories, as those operated by Ocean Networks Canada (ONC), where Petabytes of data are now collected each and every year since their installation. Here, we present a machine-learning and computer vision automated pipeline to detect and count sablefish (Anoplopoma fimbria), a key commercially exploited species in the N-NE Pacific. We used 651 hours of video footage obtained from three long-term monitoring sites in the NEPTUNE cabled observatory, in Barkley Canyon, on the nearby slope, and at depths ranging from 420 to 985 m. Our proposed AI sablefish detection and classification pipeline was tested and validated for an initial 4.5 month period (Sep 18 2019-Jan 2 2020), and was a first step towards validation for future processing of the now decade-long video archives from Barkley Canyon. For the validation period, we trained a YOLO neural network on 2917 manually annotated frames containing sablefish images to obtain an automatic detector with a 92% Average Precision (AP) on 730 test images, and a 5-fold cross-validation AP of 93% (± 3.7%). We then ran the detector on all video material (i.e., 651 hours from a 4.5 month period), to automatically detect and annotate sablefish. We finally applied a tracking algorithm on detection results, to approximate counts of individual fishes moving on scene and obtain a time series of proxy sablefish abundance. Those proxy abundance estimates are among the first to be made using such a large volume of video data from deep-sea settings. We discuss our AI results for application on a decade-long video monitoring program, and particularly with potential for complementing fisheries management practices of a commercially important species. |
| first_indexed | 2024-12-10T18:20:04Z |
| format | Article |
| id | doaj.art-de163e8b7a2a4b9f8159dbfe85b1f7ec |
| institution | Directory Open Access Journal |
| issn | 2296-7745 |
| language | English |
| last_indexed | 2024-12-10T18:20:04Z |
| publishDate | 2022-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Marine Science |
| spelling | doaj.art-de163e8b7a2a4b9f8159dbfe85b1f7ec2022-12-22T01:38:14ZengFrontiers Media S.A.Frontiers in Marine Science2296-77452022-08-01910.3389/fmars.2022.842946842946Machine learning applied to big data from marine cabled observatories: A case study of sablefish monitoring in the NE PacificFederico Bonofiglio0Fabio C. De Leo1Fabio C. De Leo2Connor Yee3Damianos Chatzievangelou4Jacopo Aguzzi5Jacopo Aguzzi6Simone Marini7Simone Marini8Institute of Marine Sciences (ISMAR), National Research Council of Italy, Lerici, ItalyOcean Networks Canada, University of Victoria, Victoria, BC, CanadaDepartment of Biology, University of Victoria, Victoria, BC, CanadaDepartment of Biology, University of Victoria, Victoria, BC, CanadaFunctioning and Vulnerability of Marine Ecosystems Group, Department of Renewable Marine Resources, Instituto de Ciencias del Mar - Consejo Superior de Investigaciones Científicas (ICM-CSIC), Barcelona, SpainFunctioning and Vulnerability of Marine Ecosystems Group, Department of Renewable Marine Resources, Instituto de Ciencias del Mar - Consejo Superior de Investigaciones Científicas (ICM-CSIC), Barcelona, SpainStazione Zoologica Anton Dohrn (SZN), Naples, ItalyInstitute of Marine Sciences (ISMAR), National Research Council of Italy, Lerici, ItalyStazione Zoologica Anton Dohrn (SZN), Naples, ItalyOcean observatories collect large volumes of video data, with some data archives now spanning well over a few decades, and bringing the challenges of analytical capacity beyond conventional processing tools. The analysis of such vast and complex datasets can only be achieved with appropriate machine learning and Artificial Intelligence (AI) tools. The implementation of AI monitoring programs for animal tracking and classification becomes necessary in the particular case of deep-sea cabled observatories, as those operated by Ocean Networks Canada (ONC), where Petabytes of data are now collected each and every year since their installation. Here, we present a machine-learning and computer vision automated pipeline to detect and count sablefish (Anoplopoma fimbria), a key commercially exploited species in the N-NE Pacific. We used 651 hours of video footage obtained from three long-term monitoring sites in the NEPTUNE cabled observatory, in Barkley Canyon, on the nearby slope, and at depths ranging from 420 to 985 m. Our proposed AI sablefish detection and classification pipeline was tested and validated for an initial 4.5 month period (Sep 18 2019-Jan 2 2020), and was a first step towards validation for future processing of the now decade-long video archives from Barkley Canyon. For the validation period, we trained a YOLO neural network on 2917 manually annotated frames containing sablefish images to obtain an automatic detector with a 92% Average Precision (AP) on 730 test images, and a 5-fold cross-validation AP of 93% (± 3.7%). We then ran the detector on all video material (i.e., 651 hours from a 4.5 month period), to automatically detect and annotate sablefish. We finally applied a tracking algorithm on detection results, to approximate counts of individual fishes moving on scene and obtain a time series of proxy sablefish abundance. Those proxy abundance estimates are among the first to be made using such a large volume of video data from deep-sea settings. We discuss our AI results for application on a decade-long video monitoring program, and particularly with potential for complementing fisheries management practices of a commercially important species.https://www.frontiersin.org/articles/10.3389/fmars.2022.842946/fullbig datamachine learningmarine observatoriesautomated video analysisfishery independent monitoringocean network Canada |
| spellingShingle | Federico Bonofiglio Fabio C. De Leo Fabio C. De Leo Connor Yee Damianos Chatzievangelou Jacopo Aguzzi Jacopo Aguzzi Simone Marini Simone Marini Machine learning applied to big data from marine cabled observatories: A case study of sablefish monitoring in the NE Pacific Frontiers in Marine Science big data machine learning marine observatories automated video analysis fishery independent monitoring ocean network Canada |
| title | Machine learning applied to big data from marine cabled observatories: A case study of sablefish monitoring in the NE Pacific |
| title_full | Machine learning applied to big data from marine cabled observatories: A case study of sablefish monitoring in the NE Pacific |
| title_fullStr | Machine learning applied to big data from marine cabled observatories: A case study of sablefish monitoring in the NE Pacific |
| title_full_unstemmed | Machine learning applied to big data from marine cabled observatories: A case study of sablefish monitoring in the NE Pacific |
| title_short | Machine learning applied to big data from marine cabled observatories: A case study of sablefish monitoring in the NE Pacific |
| title_sort | machine learning applied to big data from marine cabled observatories a case study of sablefish monitoring in the ne pacific |
| topic | big data machine learning marine observatories automated video analysis fishery independent monitoring ocean network Canada |
| url | https://www.frontiersin.org/articles/10.3389/fmars.2022.842946/full |
| work_keys_str_mv | AT federicobonofiglio machinelearningappliedtobigdatafrommarinecabledobservatoriesacasestudyofsablefishmonitoringinthenepacific AT fabiocdeleo machinelearningappliedtobigdatafrommarinecabledobservatoriesacasestudyofsablefishmonitoringinthenepacific AT fabiocdeleo machinelearningappliedtobigdatafrommarinecabledobservatoriesacasestudyofsablefishmonitoringinthenepacific AT connoryee machinelearningappliedtobigdatafrommarinecabledobservatoriesacasestudyofsablefishmonitoringinthenepacific AT damianoschatzievangelou machinelearningappliedtobigdatafrommarinecabledobservatoriesacasestudyofsablefishmonitoringinthenepacific AT jacopoaguzzi machinelearningappliedtobigdatafrommarinecabledobservatoriesacasestudyofsablefishmonitoringinthenepacific AT jacopoaguzzi machinelearningappliedtobigdatafrommarinecabledobservatoriesacasestudyofsablefishmonitoringinthenepacific AT simonemarini machinelearningappliedtobigdatafrommarinecabledobservatoriesacasestudyofsablefishmonitoringinthenepacific AT simonemarini machinelearningappliedtobigdatafrommarinecabledobservatoriesacasestudyofsablefishmonitoringinthenepacific |