On Improving the Training of Models for the Semantic Segmentation of Benthic Communities from Orthographic Imagery
The semantic segmentation of underwater imagery is an important step in the ecological analysis of coral habitats. To date, scientists produce fine-scale area annotations <i>manually</i>, an exceptionally time-consuming task that could be efficiently automatized by modern CNNs. This pape...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-09-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/12/18/3106 |
| _version_ | 1797552884928217088 |
|---|---|
| author | Gaia Pavoni Massimiliano Corsini Marco Callieri Giuseppe Fiameni Clinton Edwards Paolo Cignoni |
| author_facet | Gaia Pavoni Massimiliano Corsini Marco Callieri Giuseppe Fiameni Clinton Edwards Paolo Cignoni |
| author_sort | Gaia Pavoni |
| collection | DOAJ |
| description | The semantic segmentation of underwater imagery is an important step in the ecological analysis of coral habitats. To date, scientists produce fine-scale area annotations <i>manually</i>, an exceptionally time-consuming task that could be efficiently automatized by modern CNNs. This paper extends our previous work presented at the 3DUW’19 conference, outlining the workflow for the automated annotation of imagery from the first step of dataset preparation, to the last step of prediction reassembly. In particular, we propose an ecologically inspired strategy for an efficient dataset partition, an over-sampling methodology targeted on ortho-imagery, and a score fusion strategy. We also investigate the use of different loss functions in the optimization of a Deeplab V3+ model, to mitigate the class-imbalance problem and improve prediction accuracy on coral instance boundaries. The experimental results demonstrate the effectiveness of the ecologically inspired split in improving model performance, and quantify the advantages and limitations of the proposed over-sampling strategy. The extensive comparison of the loss functions gives numerous insights on the segmentation task; the Focal Tversky, typically used in the context of medical imaging (but not in remote sensing), results in the most convenient choice. By improving the accuracy of automated ortho image processing, the results presented here promise to meet the fundamental challenge of increasing the spatial and temporal scale of coral reef research, allowing researchers greater predictive ability to better manage coral reef resilience in the context of a changing environment. |
| first_indexed | 2024-03-10T16:07:23Z |
| format | Article |
| id | doaj.art-2dba6afc7c284064a6b6f64283dedba2 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T16:07:23Z |
| publishDate | 2020-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-2dba6afc7c284064a6b6f64283dedba22023-11-20T14:43:26ZengMDPI AGRemote Sensing2072-42922020-09-011218310610.3390/rs12183106On Improving the Training of Models for the Semantic Segmentation of Benthic Communities from Orthographic ImageryGaia Pavoni0Massimiliano Corsini1Marco Callieri2Giuseppe Fiameni3Clinton Edwards4Paolo Cignoni5Visual Computing Lab (ISTI-CNR), 56124 Pisa, ItalyVisual Computing Lab (ISTI-CNR), 56124 Pisa, ItalyVisual Computing Lab (ISTI-CNR), 56124 Pisa, ItalyNVIDIA AI Technology Centre (NVAITC), 40134 Bologna, ItalyScripps Institution of Oceanography, UC San Diego, La Jolla, CA 92037, USAVisual Computing Lab (ISTI-CNR), 56124 Pisa, ItalyThe semantic segmentation of underwater imagery is an important step in the ecological analysis of coral habitats. To date, scientists produce fine-scale area annotations <i>manually</i>, an exceptionally time-consuming task that could be efficiently automatized by modern CNNs. This paper extends our previous work presented at the 3DUW’19 conference, outlining the workflow for the automated annotation of imagery from the first step of dataset preparation, to the last step of prediction reassembly. In particular, we propose an ecologically inspired strategy for an efficient dataset partition, an over-sampling methodology targeted on ortho-imagery, and a score fusion strategy. We also investigate the use of different loss functions in the optimization of a Deeplab V3+ model, to mitigate the class-imbalance problem and improve prediction accuracy on coral instance boundaries. The experimental results demonstrate the effectiveness of the ecologically inspired split in improving model performance, and quantify the advantages and limitations of the proposed over-sampling strategy. The extensive comparison of the loss functions gives numerous insights on the segmentation task; the Focal Tversky, typically used in the context of medical imaging (but not in remote sensing), results in the most convenient choice. By improving the accuracy of automated ortho image processing, the results presented here promise to meet the fundamental challenge of increasing the spatial and temporal scale of coral reef research, allowing researchers greater predictive ability to better manage coral reef resilience in the context of a changing environment.https://www.mdpi.com/2072-4292/12/18/3106coral reef monitoringorthomosaicsorthoprojectionssemantic segmentationdeep learning |
| spellingShingle | Gaia Pavoni Massimiliano Corsini Marco Callieri Giuseppe Fiameni Clinton Edwards Paolo Cignoni On Improving the Training of Models for the Semantic Segmentation of Benthic Communities from Orthographic Imagery Remote Sensing coral reef monitoring orthomosaics orthoprojections semantic segmentation deep learning |
| title | On Improving the Training of Models for the Semantic Segmentation of Benthic Communities from Orthographic Imagery |
| title_full | On Improving the Training of Models for the Semantic Segmentation of Benthic Communities from Orthographic Imagery |
| title_fullStr | On Improving the Training of Models for the Semantic Segmentation of Benthic Communities from Orthographic Imagery |
| title_full_unstemmed | On Improving the Training of Models for the Semantic Segmentation of Benthic Communities from Orthographic Imagery |
| title_short | On Improving the Training of Models for the Semantic Segmentation of Benthic Communities from Orthographic Imagery |
| title_sort | on improving the training of models for the semantic segmentation of benthic communities from orthographic imagery |
| topic | coral reef monitoring orthomosaics orthoprojections semantic segmentation deep learning |
| url | https://www.mdpi.com/2072-4292/12/18/3106 |
| work_keys_str_mv | AT gaiapavoni onimprovingthetrainingofmodelsforthesemanticsegmentationofbenthiccommunitiesfromorthographicimagery AT massimilianocorsini onimprovingthetrainingofmodelsforthesemanticsegmentationofbenthiccommunitiesfromorthographicimagery AT marcocallieri onimprovingthetrainingofmodelsforthesemanticsegmentationofbenthiccommunitiesfromorthographicimagery AT giuseppefiameni onimprovingthetrainingofmodelsforthesemanticsegmentationofbenthiccommunitiesfromorthographicimagery AT clintonedwards onimprovingthetrainingofmodelsforthesemanticsegmentationofbenthiccommunitiesfromorthographicimagery AT paolocignoni onimprovingthetrainingofmodelsforthesemanticsegmentationofbenthiccommunitiesfromorthographicimagery |