Wild salmon enumeration and monitoring using deep learning empowered detection and tracking
Pacific salmon have experienced declining abundance and unpredictable returns, yet remain vital to livelihoods, food security, and cultures of coastal communities around the Pacific Rim, creating a need for reliable and timely monitoring to inform sustainable fishery management. Currently, spawning...
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-09-01
|
| Series: | Frontiers in Marine Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fmars.2023.1200408/full |
| _version_ | 1797679161953746944 |
|---|---|
| author | William I. Atlas Sami Ma Yi Ching Chou Katrina Connors Daniel Scurfield Brandon Nam Xiaoqiang Ma Mark Cleveland Janvier Doire Jonathan W. Moore Ryan Shea Jiangchuan Liu |
| author_facet | William I. Atlas Sami Ma Yi Ching Chou Katrina Connors Daniel Scurfield Brandon Nam Xiaoqiang Ma Mark Cleveland Janvier Doire Jonathan W. Moore Ryan Shea Jiangchuan Liu |
| author_sort | William I. Atlas |
| collection | DOAJ |
| description | Pacific salmon have experienced declining abundance and unpredictable returns, yet remain vital to livelihoods, food security, and cultures of coastal communities around the Pacific Rim, creating a need for reliable and timely monitoring to inform sustainable fishery management. Currently, spawning salmon abundance is often monitored with in-river video or sonar cameras. However, reviewing video for estimates of salmon abundance from these programs requires thousands of hours of staff time, and data are typically not available until after the fishing season is completed. Computer vision deep learning can enable rapid and reliable processing of data, with potentially transformative applications in salmon population assessment and fishery management. Working with two First Nations fishery programs in British Columbia, Canada, we developed, trained, and tested deep learning models to perform object detection and multi-object tracking for automated video enumeration of salmon passing two First Nation-run weirs. We gathered and annotated more than 500,000 frames of video data encompassing 12 species, including seven species of anadromous salmonids, and trained models for multi-object tracking and species detection. Our top performing model achieved a mean average precision (mAP) of 67.6%, and species-specific mAP scores > 90% for coho and > 80% for sockeye salmon when trained with a combined dataset of Kitwanga and Bear Rivers’ salmon annotations. We also tested and deployed a prototype for a real-time monitoring system that can perform computer vision deep learning analyses on site. Computer vision models and off-grid monitoring systems show promise for automated counting and species identification. A key future priority will be working with stewardship practitioners and fishery managers to apply salmon computer vision, testing and applying edge-capable computing solutions for in-situ analysis at remote sites, and developing tools for independent user-led computer vision analysis by non-computer scientists. These efforts can advance in-season monitoring and decision making to support adaptive management of sustainable wild salmon fisheries. |
| first_indexed | 2024-03-11T23:10:30Z |
| format | Article |
| id | doaj.art-7b9c5ab60640430a8c870d8f81462ef7 |
| institution | Directory Open Access Journal |
| issn | 2296-7745 |
| language | English |
| last_indexed | 2024-03-11T23:10:30Z |
| publishDate | 2023-09-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Marine Science |
| spelling | doaj.art-7b9c5ab60640430a8c870d8f81462ef72023-09-21T09:06:01ZengFrontiers Media S.A.Frontiers in Marine Science2296-77452023-09-011010.3389/fmars.2023.12004081200408Wild salmon enumeration and monitoring using deep learning empowered detection and trackingWilliam I. Atlas0Sami Ma1Yi Ching Chou2Katrina Connors3Daniel Scurfield4Brandon Nam5Xiaoqiang Ma6Mark Cleveland7Janvier Doire8Jonathan W. Moore9Ryan Shea10Jiangchuan Liu11Wild Salmon Center, Portland, OR, United StatesSchool of Computing Science, Simon Fraser University, Burnaby, BC, CanadaSchool of Computing Science, Simon Fraser University, Burnaby, BC, CanadaPacific Salmon Foundation, Salmon Watersheds Program, Vancouver, BC, CanadaSalmon Watersheds Lab, Simon Fraser University, Biological Sciences, Burnaby, BC, CanadaSalmon Watersheds Lab, Simon Fraser University, Biological Sciences, Burnaby, BC, CanadaComputing Science Department, Douglas College, New Westminster, BC, CanadaGitanyow Fisheries Authority, Gitanyow, BC, CanadaSkeena Fisheries Commission, Kispiox, BC, CanadaSalmon Watersheds Lab, Simon Fraser University, Biological Sciences, Burnaby, BC, CanadaSchool of Computing Science, Simon Fraser University, Burnaby, BC, CanadaSchool of Computing Science, Simon Fraser University, Burnaby, BC, CanadaPacific salmon have experienced declining abundance and unpredictable returns, yet remain vital to livelihoods, food security, and cultures of coastal communities around the Pacific Rim, creating a need for reliable and timely monitoring to inform sustainable fishery management. Currently, spawning salmon abundance is often monitored with in-river video or sonar cameras. However, reviewing video for estimates of salmon abundance from these programs requires thousands of hours of staff time, and data are typically not available until after the fishing season is completed. Computer vision deep learning can enable rapid and reliable processing of data, with potentially transformative applications in salmon population assessment and fishery management. Working with two First Nations fishery programs in British Columbia, Canada, we developed, trained, and tested deep learning models to perform object detection and multi-object tracking for automated video enumeration of salmon passing two First Nation-run weirs. We gathered and annotated more than 500,000 frames of video data encompassing 12 species, including seven species of anadromous salmonids, and trained models for multi-object tracking and species detection. Our top performing model achieved a mean average precision (mAP) of 67.6%, and species-specific mAP scores > 90% for coho and > 80% for sockeye salmon when trained with a combined dataset of Kitwanga and Bear Rivers’ salmon annotations. We also tested and deployed a prototype for a real-time monitoring system that can perform computer vision deep learning analyses on site. Computer vision models and off-grid monitoring systems show promise for automated counting and species identification. A key future priority will be working with stewardship practitioners and fishery managers to apply salmon computer vision, testing and applying edge-capable computing solutions for in-situ analysis at remote sites, and developing tools for independent user-led computer vision analysis by non-computer scientists. These efforts can advance in-season monitoring and decision making to support adaptive management of sustainable wild salmon fisheries.https://www.frontiersin.org/articles/10.3389/fmars.2023.1200408/fullcomputer visiondeep learningfisheries managementin-season fishery managementindigenous sciencewild salmon |
| spellingShingle | William I. Atlas Sami Ma Yi Ching Chou Katrina Connors Daniel Scurfield Brandon Nam Xiaoqiang Ma Mark Cleveland Janvier Doire Jonathan W. Moore Ryan Shea Jiangchuan Liu Wild salmon enumeration and monitoring using deep learning empowered detection and tracking Frontiers in Marine Science computer vision deep learning fisheries management in-season fishery management indigenous science wild salmon |
| title | Wild salmon enumeration and monitoring using deep learning empowered detection and tracking |
| title_full | Wild salmon enumeration and monitoring using deep learning empowered detection and tracking |
| title_fullStr | Wild salmon enumeration and monitoring using deep learning empowered detection and tracking |
| title_full_unstemmed | Wild salmon enumeration and monitoring using deep learning empowered detection and tracking |
| title_short | Wild salmon enumeration and monitoring using deep learning empowered detection and tracking |
| title_sort | wild salmon enumeration and monitoring using deep learning empowered detection and tracking |
| topic | computer vision deep learning fisheries management in-season fishery management indigenous science wild salmon |
| url | https://www.frontiersin.org/articles/10.3389/fmars.2023.1200408/full |
| work_keys_str_mv | AT williamiatlas wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking AT samima wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking AT yichingchou wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking AT katrinaconnors wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking AT danielscurfield wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking AT brandonnam wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking AT xiaoqiangma wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking AT markcleveland wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking AT janvierdoire wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking AT jonathanwmoore wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking AT ryanshea wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking AT jiangchuanliu wildsalmonenumerationandmonitoringusingdeeplearningempowereddetectionandtracking |