RandCrowns: A Quantitative Metric for Imprecisely Labeled Tree Crown Delineation
Supervised methods for object delineation in remote sensing require labeled ground-truth data. Gathering sufficient high quality ground-truth data is difficult, especially when targets are of irregular shape or difficult to distinguish from background or neighboring objects. Tree crown delineation p...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9585420/ |
| _version_ | 1818901076643938304 |
|---|---|
| author | Dylan Stewart Alina Zare Sergio Marconi Ben G. Weinstein Ethan P. White Sarah J. Graves Stephanie A. Bohlman Aditya Singh |
| author_facet | Dylan Stewart Alina Zare Sergio Marconi Ben G. Weinstein Ethan P. White Sarah J. Graves Stephanie A. Bohlman Aditya Singh |
| author_sort | Dylan Stewart |
| collection | DOAJ |
| description | Supervised methods for object delineation in remote sensing require labeled ground-truth data. Gathering sufficient high quality ground-truth data is difficult, especially when targets are of irregular shape or difficult to distinguish from background or neighboring objects. Tree crown delineation provides key information from remote sensing images for forestry, ecology, and management. However, tree crowns in remote sensing imagery are often difficult to label and annotate due to irregular shape, overlapping canopies, shadowing, and indistinct edges. There are also multiple approaches to annotation in this field (e.g., rectangular boxes vs. convex polygons) that further contribute to annotation imprecision. However, current evaluation methods do not account for this uncertainty in annotations, and quantitative metrics for evaluation can vary across multiple annotators. In this article, we address these limitations by developing an adaptation of the Rand index (RI) for weakly labeled crown delineation that we call RandCrowns (RC). Our new RC evaluation metric provides a method to appropriately evaluate delineated tree crowns while taking into account imprecision in the ground-truth delineations. The RC metric reformulates the RI by adjusting the areas over which each term of the index is computed to account for uncertain and imprecise object delineation labels. Quantitative comparisons to the commonly used intersection over union method show a decrease in the variance generated by differences among multiple annotators. Combined with qualitative examples, our results suggest that the RC metric is more robust for scoring target delineations in the presence of uncertainty and imprecision in annotations that are inherent to tree crown delineation. |
| first_indexed | 2024-12-19T20:14:00Z |
| format | Article |
| id | doaj.art-b3004e22a27c4369a7331fd04e2a81c8 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-12-19T20:14:00Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-b3004e22a27c4369a7331fd04e2a81c82022-12-21T20:07:14ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352021-01-0114112291123910.1109/JSTARS.2021.31223459585420RandCrowns: A Quantitative Metric for Imprecisely Labeled Tree Crown DelineationDylan Stewart0https://orcid.org/0000-0003-4505-657XAlina Zare1https://orcid.org/0000-0002-4847-7604Sergio Marconi2https://orcid.org/0000-0002-8096-754XBen G. Weinstein3https://orcid.org/0000-0002-2176-7935Ethan P. White4https://orcid.org/0000-0001-6728-7745Sarah J. Graves5https://orcid.org/0000-0003-3805-4242Stephanie A. Bohlman6https://orcid.org/0000-0002-4935-7321Aditya Singh7https://orcid.org/0000-0001-5559-9151Department of Electrical, and Computer Engineering, University of Florida, Gainesville, FL, USADepartment of Electrical, and Computer Engineering, University of Florida, Gainesville, FL, USASchool of Forest, Fisheries, and Geomatics Sciences, University of Florida, Gainesville, FL, USADepartment of Wildlife Ecology, and Conservation, University of Florida, Gainesville, FL, USADepartment of Wildlife Ecology, and Conservation, University of Florida, Gainesville, FL, USANelson Institute for Environmental Studies, University of Wisconsin-Madison, Madison, WI, USASchool of Forest, Fisheries, and Geomatics Sciences, University of Florida, Gainesville, FL, USADepartment of Agricultural, and Biological Engineering, University of Florida, Gainesville, FL, USASupervised methods for object delineation in remote sensing require labeled ground-truth data. Gathering sufficient high quality ground-truth data is difficult, especially when targets are of irregular shape or difficult to distinguish from background or neighboring objects. Tree crown delineation provides key information from remote sensing images for forestry, ecology, and management. However, tree crowns in remote sensing imagery are often difficult to label and annotate due to irregular shape, overlapping canopies, shadowing, and indistinct edges. There are also multiple approaches to annotation in this field (e.g., rectangular boxes vs. convex polygons) that further contribute to annotation imprecision. However, current evaluation methods do not account for this uncertainty in annotations, and quantitative metrics for evaluation can vary across multiple annotators. In this article, we address these limitations by developing an adaptation of the Rand index (RI) for weakly labeled crown delineation that we call RandCrowns (RC). Our new RC evaluation metric provides a method to appropriately evaluate delineated tree crowns while taking into account imprecision in the ground-truth delineations. The RC metric reformulates the RI by adjusting the areas over which each term of the index is computed to account for uncertain and imprecise object delineation labels. Quantitative comparisons to the commonly used intersection over union method show a decrease in the variance generated by differences among multiple annotators. Combined with qualitative examples, our results suggest that the RC metric is more robust for scoring target delineations in the presence of uncertainty and imprecision in annotations that are inherent to tree crown delineation.https://ieeexplore.ieee.org/document/9585420/Imprecise labelsquantitative evaluationremote sensingtree crown delineation |
| spellingShingle | Dylan Stewart Alina Zare Sergio Marconi Ben G. Weinstein Ethan P. White Sarah J. Graves Stephanie A. Bohlman Aditya Singh RandCrowns: A Quantitative Metric for Imprecisely Labeled Tree Crown Delineation IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Imprecise labels quantitative evaluation remote sensing tree crown delineation |
| title | RandCrowns: A Quantitative Metric for Imprecisely Labeled Tree Crown Delineation |
| title_full | RandCrowns: A Quantitative Metric for Imprecisely Labeled Tree Crown Delineation |
| title_fullStr | RandCrowns: A Quantitative Metric for Imprecisely Labeled Tree Crown Delineation |
| title_full_unstemmed | RandCrowns: A Quantitative Metric for Imprecisely Labeled Tree Crown Delineation |
| title_short | RandCrowns: A Quantitative Metric for Imprecisely Labeled Tree Crown Delineation |
| title_sort | randcrowns a quantitative metric for imprecisely labeled tree crown delineation |
| topic | Imprecise labels quantitative evaluation remote sensing tree crown delineation |
| url | https://ieeexplore.ieee.org/document/9585420/ |
| work_keys_str_mv | AT dylanstewart randcrownsaquantitativemetricforimpreciselylabeledtreecrowndelineation AT alinazare randcrownsaquantitativemetricforimpreciselylabeledtreecrowndelineation AT sergiomarconi randcrownsaquantitativemetricforimpreciselylabeledtreecrowndelineation AT bengweinstein randcrownsaquantitativemetricforimpreciselylabeledtreecrowndelineation AT ethanpwhite randcrownsaquantitativemetricforimpreciselylabeledtreecrowndelineation AT sarahjgraves randcrownsaquantitativemetricforimpreciselylabeledtreecrowndelineation AT stephanieabohlman randcrownsaquantitativemetricforimpreciselylabeledtreecrowndelineation AT adityasingh randcrownsaquantitativemetricforimpreciselylabeledtreecrowndelineation |