Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation
With the increase in the digitization efforts of herbarium collections worldwide, dataset repositories such as iDigBio and GBIF now have hundreds of thousands of herbarium sheet images ready for exploration. Although this serves as a new source of plant leaves data, herbarium datasets have an inhere...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/13/4549 |
| _version_ | 1797527876594040832 |
|---|---|
| author | Burhan Rashid Hussein Owais Ahmed Malik Wee-Hong Ong Johan Willem Frederik Slik |
| author_facet | Burhan Rashid Hussein Owais Ahmed Malik Wee-Hong Ong Johan Willem Frederik Slik |
| author_sort | Burhan Rashid Hussein |
| collection | DOAJ |
| description | With the increase in the digitization efforts of herbarium collections worldwide, dataset repositories such as iDigBio and GBIF now have hundreds of thousands of herbarium sheet images ready for exploration. Although this serves as a new source of plant leaves data, herbarium datasets have an inherent challenge to deal with the sheets containing other non-plant objects such as color charts, barcodes, and labels. Even for the plant part itself, a combination of different overlapping, damaged, and intact individual leaves exist together with other plant organs such as stems and fruits, which increases the complexity of leaf trait extraction and analysis. Focusing on segmentation and trait extraction on individual intact herbarium leaves, this study proposes a pipeline consisting of deep learning semantic segmentation model (DeepLabv3+), connected component analysis, and a single-leaf classifier trained on binary images to automate the extraction of an intact individual leaf with phenotypic traits. The proposed method achieved a higher F1-score for both the in-house dataset (96%) and on a publicly available herbarium dataset (93%) compared to object detection-based approaches including Faster R-CNN and YOLOv5. Furthermore, using the proposed approach, the phenotypic measurements extracted from the segmented individual leaves were closer to the ground truth measurements, which suggests the importance of the segmentation process in handling background noise. Compared to the object detection-based approaches, the proposed method showed a promising direction toward an autonomous tool for the extraction of individual leaves together with their trait data directly from herbarium specimen images. |
| first_indexed | 2024-03-10T09:50:03Z |
| format | Article |
| id | doaj.art-9a6569c86ca54ad093f0c3647d33c63c |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T09:50:03Z |
| publishDate | 2021-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-9a6569c86ca54ad093f0c3647d33c63c2023-11-22T02:50:58ZengMDPI AGSensors1424-82202021-07-012113454910.3390/s21134549Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic SegmentationBurhan Rashid Hussein0Owais Ahmed Malik1Wee-Hong Ong2Johan Willem Frederik Slik3Digital Science, Faculty of Science, Universiti Brunei Darussalam, Tungku Link, Gadong BE1410, BruneiDigital Science, Faculty of Science, Universiti Brunei Darussalam, Tungku Link, Gadong BE1410, BruneiDigital Science, Faculty of Science, Universiti Brunei Darussalam, Tungku Link, Gadong BE1410, BruneiDepartment of Environmental Life Sciences, Faculty of Science, Universiti Brunei Darussalam, Tungku Link, Gadong BE1410, BruneiWith the increase in the digitization efforts of herbarium collections worldwide, dataset repositories such as iDigBio and GBIF now have hundreds of thousands of herbarium sheet images ready for exploration. Although this serves as a new source of plant leaves data, herbarium datasets have an inherent challenge to deal with the sheets containing other non-plant objects such as color charts, barcodes, and labels. Even for the plant part itself, a combination of different overlapping, damaged, and intact individual leaves exist together with other plant organs such as stems and fruits, which increases the complexity of leaf trait extraction and analysis. Focusing on segmentation and trait extraction on individual intact herbarium leaves, this study proposes a pipeline consisting of deep learning semantic segmentation model (DeepLabv3+), connected component analysis, and a single-leaf classifier trained on binary images to automate the extraction of an intact individual leaf with phenotypic traits. The proposed method achieved a higher F1-score for both the in-house dataset (96%) and on a publicly available herbarium dataset (93%) compared to object detection-based approaches including Faster R-CNN and YOLOv5. Furthermore, using the proposed approach, the phenotypic measurements extracted from the segmented individual leaves were closer to the ground truth measurements, which suggests the importance of the segmentation process in handling background noise. Compared to the object detection-based approaches, the proposed method showed a promising direction toward an autonomous tool for the extraction of individual leaves together with their trait data directly from herbarium specimen images.https://www.mdpi.com/1424-8220/21/13/4549semantic segmentationobject detectionleaf extractionconnected component analysisplant species identificationherbarium leaf dataset |
| spellingShingle | Burhan Rashid Hussein Owais Ahmed Malik Wee-Hong Ong Johan Willem Frederik Slik Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation Sensors semantic segmentation object detection leaf extraction connected component analysis plant species identification herbarium leaf dataset |
| title | Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation |
| title_full | Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation |
| title_fullStr | Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation |
| title_full_unstemmed | Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation |
| title_short | Automated Extraction of Phenotypic Leaf Traits of Individual Intact Herbarium Leaves from Herbarium Specimen Images Using Deep Learning Based Semantic Segmentation |
| title_sort | automated extraction of phenotypic leaf traits of individual intact herbarium leaves from herbarium specimen images using deep learning based semantic segmentation |
| topic | semantic segmentation object detection leaf extraction connected component analysis plant species identification herbarium leaf dataset |
| url | https://www.mdpi.com/1424-8220/21/13/4549 |
| work_keys_str_mv | AT burhanrashidhussein automatedextractionofphenotypicleaftraitsofindividualintactherbariumleavesfromherbariumspecimenimagesusingdeeplearningbasedsemanticsegmentation AT owaisahmedmalik automatedextractionofphenotypicleaftraitsofindividualintactherbariumleavesfromherbariumspecimenimagesusingdeeplearningbasedsemanticsegmentation AT weehongong automatedextractionofphenotypicleaftraitsofindividualintactherbariumleavesfromherbariumspecimenimagesusingdeeplearningbasedsemanticsegmentation AT johanwillemfrederikslik automatedextractionofphenotypicleaftraitsofindividualintactherbariumleavesfromherbariumspecimenimagesusingdeeplearningbasedsemanticsegmentation |