DeepLOKI- a deep learning based approach to identify zooplankton taxa on high-resolution images from the optical plankton recorder LOKI
Zooplankton play a crucial role in the ocean’s ecology, as they form a foundational component in the food chain by consuming phytoplankton or other zooplankton, supporting various marine species and influencing nutrient cycling. The vertical distribution of zooplankton in the ocean is patchy, and it...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-11-01
|
| Series: | Frontiers in Marine Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fmars.2023.1280510/full |
| _version_ | 1827630134797008896 |
|---|---|
| author | Ellen Oldenburg Ellen Oldenburg Raphael M. Kronberg Barbara Niehoff Oliver Ebenhöh Oliver Ebenhöh Ovidiu Popa |
| author_facet | Ellen Oldenburg Ellen Oldenburg Raphael M. Kronberg Barbara Niehoff Oliver Ebenhöh Oliver Ebenhöh Ovidiu Popa |
| author_sort | Ellen Oldenburg |
| collection | DOAJ |
| description | Zooplankton play a crucial role in the ocean’s ecology, as they form a foundational component in the food chain by consuming phytoplankton or other zooplankton, supporting various marine species and influencing nutrient cycling. The vertical distribution of zooplankton in the ocean is patchy, and its relation to hydrographical conditions cannot be fully deciphered using traditional net casts due to the large depth intervals sampled. The Lightframe On-sight Keyspecies Investigation (LOKI) concentrates zooplankton with a net that leads to a flow-through chamber with a camera taking images. These high-resolution images allow for the determination of zooplankton taxa, often even to genus or species level, and, in the case of copepods, developmental stages. Each cruise produces a substantial volume of images, ideally requiring onboard analysis, which presently consumes a significant amount of time and necessitates internet connectivity to access the EcoTaxa Web service. To enhance the analyses, we developed an AI-based software framework named DeepLOKI, utilizing Deep Transfer Learning with a Convolution Neural Network Backbone. Our DeepLOKI can be applied directly on board. We trained and validated the model on pre-labeled images from four cruises, while images from a fifth cruise were used for testing. The best-performing model, utilizing the self-supervised pre-trained ResNet18 Backbone, achieved a notable average classification accuracy of 83.9%, surpassing the regularly and frequently used method EcoTaxa (default) in this field by a factor of two. In summary, we developed a tool for pre-sorting high-resolution black and white zooplankton images with high accuracy, which will simplify and quicken the final annotation process. In addition, we provide a user-friendly graphical interface for the DeepLOKI framework for efficient and concise processes leading up to the classification stage. Moreover, performing latent space analysis on the self-supervised pre-trained ResNet18 Backbone could prove advantageous in identifying anomalies such as deviations in image parameter settings. This, in turn, enhances the quality control of the data. Our methodology remains agnostic to the specific imaging end system used, such as Loki, UVP, or ZooScan, as long as there is a sufficient amount of appropriately labeled data available to enable effective task performance by our algorithms. |
| first_indexed | 2024-03-09T14:01:54Z |
| format | Article |
| id | doaj.art-5fcdb3ed2cf64e1492548278cb388d4b |
| institution | Directory Open Access Journal |
| issn | 2296-7745 |
| language | English |
| last_indexed | 2024-03-09T14:01:54Z |
| publishDate | 2023-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Marine Science |
| spelling | doaj.art-5fcdb3ed2cf64e1492548278cb388d4b2023-11-30T07:29:28ZengFrontiers Media S.A.Frontiers in Marine Science2296-77452023-11-011010.3389/fmars.2023.12805101280510DeepLOKI- a deep learning based approach to identify zooplankton taxa on high-resolution images from the optical plankton recorder LOKIEllen Oldenburg0Ellen Oldenburg1Raphael M. Kronberg2Barbara Niehoff3Oliver Ebenhöh4Oliver Ebenhöh5Ovidiu Popa6Institute of Theoretical and Quantitative Biology, Heinrich Heine University, Düsseldorf, North Rhine-Westphalia, GermanyCluster of Excellence on Plant Sciences, Heinrich Heine University, Düsseldorf, North Rhine-Westphalia, GermanyMathematical Modelling of Biological Systems, Heinrich Heine University, Düsseldorf, North Rhine-Westphalia, GermanyPolar Biological Oceanography, Alfred- Wegener- Institut Helmholtz Center for Polar and Marine Research, Bremerhaven, GermanyInstitute of Theoretical and Quantitative Biology, Heinrich Heine University, Düsseldorf, North Rhine-Westphalia, GermanyCluster of Excellence on Plant Sciences, Heinrich Heine University, Düsseldorf, North Rhine-Westphalia, GermanyInstitute of Theoretical and Quantitative Biology, Heinrich Heine University, Düsseldorf, North Rhine-Westphalia, GermanyZooplankton play a crucial role in the ocean’s ecology, as they form a foundational component in the food chain by consuming phytoplankton or other zooplankton, supporting various marine species and influencing nutrient cycling. The vertical distribution of zooplankton in the ocean is patchy, and its relation to hydrographical conditions cannot be fully deciphered using traditional net casts due to the large depth intervals sampled. The Lightframe On-sight Keyspecies Investigation (LOKI) concentrates zooplankton with a net that leads to a flow-through chamber with a camera taking images. These high-resolution images allow for the determination of zooplankton taxa, often even to genus or species level, and, in the case of copepods, developmental stages. Each cruise produces a substantial volume of images, ideally requiring onboard analysis, which presently consumes a significant amount of time and necessitates internet connectivity to access the EcoTaxa Web service. To enhance the analyses, we developed an AI-based software framework named DeepLOKI, utilizing Deep Transfer Learning with a Convolution Neural Network Backbone. Our DeepLOKI can be applied directly on board. We trained and validated the model on pre-labeled images from four cruises, while images from a fifth cruise were used for testing. The best-performing model, utilizing the self-supervised pre-trained ResNet18 Backbone, achieved a notable average classification accuracy of 83.9%, surpassing the regularly and frequently used method EcoTaxa (default) in this field by a factor of two. In summary, we developed a tool for pre-sorting high-resolution black and white zooplankton images with high accuracy, which will simplify and quicken the final annotation process. In addition, we provide a user-friendly graphical interface for the DeepLOKI framework for efficient and concise processes leading up to the classification stage. Moreover, performing latent space analysis on the self-supervised pre-trained ResNet18 Backbone could prove advantageous in identifying anomalies such as deviations in image parameter settings. This, in turn, enhances the quality control of the data. Our methodology remains agnostic to the specific imaging end system used, such as Loki, UVP, or ZooScan, as long as there is a sufficient amount of appropriately labeled data available to enable effective task performance by our algorithms.https://www.frontiersin.org/articles/10.3389/fmars.2023.1280510/fullcomputer visiondeep learningDeepLOKIkeystone speciesmarine and zooplankton |
| spellingShingle | Ellen Oldenburg Ellen Oldenburg Raphael M. Kronberg Barbara Niehoff Oliver Ebenhöh Oliver Ebenhöh Ovidiu Popa DeepLOKI- a deep learning based approach to identify zooplankton taxa on high-resolution images from the optical plankton recorder LOKI Frontiers in Marine Science computer vision deep learning DeepLOKI keystone species marine and zooplankton |
| title | DeepLOKI- a deep learning based approach to identify zooplankton taxa on high-resolution images from the optical plankton recorder LOKI |
| title_full | DeepLOKI- a deep learning based approach to identify zooplankton taxa on high-resolution images from the optical plankton recorder LOKI |
| title_fullStr | DeepLOKI- a deep learning based approach to identify zooplankton taxa on high-resolution images from the optical plankton recorder LOKI |
| title_full_unstemmed | DeepLOKI- a deep learning based approach to identify zooplankton taxa on high-resolution images from the optical plankton recorder LOKI |
| title_short | DeepLOKI- a deep learning based approach to identify zooplankton taxa on high-resolution images from the optical plankton recorder LOKI |
| title_sort | deeploki a deep learning based approach to identify zooplankton taxa on high resolution images from the optical plankton recorder loki |
| topic | computer vision deep learning DeepLOKI keystone species marine and zooplankton |
| url | https://www.frontiersin.org/articles/10.3389/fmars.2023.1280510/full |
| work_keys_str_mv | AT ellenoldenburg deeplokiadeeplearningbasedapproachtoidentifyzooplanktontaxaonhighresolutionimagesfromtheopticalplanktonrecorderloki AT ellenoldenburg deeplokiadeeplearningbasedapproachtoidentifyzooplanktontaxaonhighresolutionimagesfromtheopticalplanktonrecorderloki AT raphaelmkronberg deeplokiadeeplearningbasedapproachtoidentifyzooplanktontaxaonhighresolutionimagesfromtheopticalplanktonrecorderloki AT barbaraniehoff deeplokiadeeplearningbasedapproachtoidentifyzooplanktontaxaonhighresolutionimagesfromtheopticalplanktonrecorderloki AT oliverebenhoh deeplokiadeeplearningbasedapproachtoidentifyzooplanktontaxaonhighresolutionimagesfromtheopticalplanktonrecorderloki AT oliverebenhoh deeplokiadeeplearningbasedapproachtoidentifyzooplanktontaxaonhighresolutionimagesfromtheopticalplanktonrecorderloki AT ovidiupopa deeplokiadeeplearningbasedapproachtoidentifyzooplanktontaxaonhighresolutionimagesfromtheopticalplanktonrecorderloki |