Fossil image identification using deep learning ensembles of data augmented multiviews
Abstract Identification of fossil species is crucial to evolutionary studies. Recent advances from deep learning have shown promising prospects in fossil image identification. However, the quantity and quality of labelled fossil images are often limited due to fossil preservation, conditioned sampli...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-12-01
|
| Series: | Methods in Ecology and Evolution |
| Subjects: | |
| Online Access: | https://doi.org/10.1111/2041-210X.14229 |
| _version_ | 1827594453326495744 |
|---|---|
| author | Chengbin Hou Xinyu Lin Hanhui Huang Sheng Xu Junxuan Fan Yukun Shi Hairong Lv |
| author_facet | Chengbin Hou Xinyu Lin Hanhui Huang Sheng Xu Junxuan Fan Yukun Shi Hairong Lv |
| author_sort | Chengbin Hou |
| collection | DOAJ |
| description | Abstract Identification of fossil species is crucial to evolutionary studies. Recent advances from deep learning have shown promising prospects in fossil image identification. However, the quantity and quality of labelled fossil images are often limited due to fossil preservation, conditioned sampling and expensive and inconsistent label annotation by domain experts, which pose great challenges to training deep learning‐based image classification models. To address these challenges, we follow the idea of the wisdom of crowds and propose a multiview ensemble framework, which collects Original (O), Grey (G) and Skeleton (S) views of each fossil image reflecting its different characteristics to train multiple base models, and then makes the final decision via soft voting. Experiments on the largest fusulinid dataset with 2400 images show that the proposed OGS consistently outperforms baselines (using a single model for each view), and obtains superior or comparable performance compared to OOO (using three base models for three the same Original views). Besides, as the training data decreases, the proposed framework achieves more gains. While considering the identification consistency estimation with respect to human experts, OGS receives the highest agreement with the original labels of dataset and with the re‐identifications of two human experts. The validation performance provides a quantitative estimation of consistency across different experts and genera. We conclude that the proposed framework can present state‐of‐the‐art performance in the fusulinid fossil identification case study. This framework is designed for general fossil identification and it is expected to see applications to other fossil datasets in future work. Notably, the result, which shows more performance gains as train set size decreases or over a smaller imbalance fossil dataset, suggests the potential application to identify rare fossil images. The proposed framework also demonstrates its potential for assessing and resolving inconsistencies in fossil identification. |
| first_indexed | 2024-03-09T02:35:49Z |
| format | Article |
| id | doaj.art-c5889770fc5b4b4d9f0b336ab98efded |
| institution | Directory Open Access Journal |
| issn | 2041-210X |
| language | English |
| last_indexed | 2024-03-09T02:35:49Z |
| publishDate | 2023-12-01 |
| publisher | Wiley |
| record_format | Article |
| series | Methods in Ecology and Evolution |
| spelling | doaj.art-c5889770fc5b4b4d9f0b336ab98efded2023-12-06T10:20:16ZengWileyMethods in Ecology and Evolution2041-210X2023-12-0114123020303410.1111/2041-210X.14229Fossil image identification using deep learning ensembles of data augmented multiviewsChengbin Hou0Xinyu Lin1Hanhui Huang2Sheng Xu3Junxuan Fan4Yukun Shi5Hairong Lv6Ministry of Education Key Laboratory of Bioinformatics Bioinformatics Division Beijing National Research Center for Information Science and Technology Department of Automation Tsinghua University Beijing ChinaFuzhou Institute of Data Technology Fuzhou ChinaSchool of Earth Sciences and Engineering and Frontiers Science Center for Critical Earth Material Cycling Nanjing University Nanjing ChinaCollege of Physics and Information Engineering Fuzhou University Fuzhou ChinaSchool of Earth Sciences and Engineering and Frontiers Science Center for Critical Earth Material Cycling Nanjing University Nanjing ChinaSchool of Earth Sciences and Engineering and Frontiers Science Center for Critical Earth Material Cycling Nanjing University Nanjing ChinaMinistry of Education Key Laboratory of Bioinformatics Bioinformatics Division Beijing National Research Center for Information Science and Technology Department of Automation Tsinghua University Beijing ChinaAbstract Identification of fossil species is crucial to evolutionary studies. Recent advances from deep learning have shown promising prospects in fossil image identification. However, the quantity and quality of labelled fossil images are often limited due to fossil preservation, conditioned sampling and expensive and inconsistent label annotation by domain experts, which pose great challenges to training deep learning‐based image classification models. To address these challenges, we follow the idea of the wisdom of crowds and propose a multiview ensemble framework, which collects Original (O), Grey (G) and Skeleton (S) views of each fossil image reflecting its different characteristics to train multiple base models, and then makes the final decision via soft voting. Experiments on the largest fusulinid dataset with 2400 images show that the proposed OGS consistently outperforms baselines (using a single model for each view), and obtains superior or comparable performance compared to OOO (using three base models for three the same Original views). Besides, as the training data decreases, the proposed framework achieves more gains. While considering the identification consistency estimation with respect to human experts, OGS receives the highest agreement with the original labels of dataset and with the re‐identifications of two human experts. The validation performance provides a quantitative estimation of consistency across different experts and genera. We conclude that the proposed framework can present state‐of‐the‐art performance in the fusulinid fossil identification case study. This framework is designed for general fossil identification and it is expected to see applications to other fossil datasets in future work. Notably, the result, which shows more performance gains as train set size decreases or over a smaller imbalance fossil dataset, suggests the potential application to identify rare fossil images. The proposed framework also demonstrates its potential for assessing and resolving inconsistencies in fossil identification.https://doi.org/10.1111/2041-210X.14229deep learningensemblefossil identificationfusulinididentification inconsistencyimage classification |
| spellingShingle | Chengbin Hou Xinyu Lin Hanhui Huang Sheng Xu Junxuan Fan Yukun Shi Hairong Lv Fossil image identification using deep learning ensembles of data augmented multiviews Methods in Ecology and Evolution deep learning ensemble fossil identification fusulinid identification inconsistency image classification |
| title | Fossil image identification using deep learning ensembles of data augmented multiviews |
| title_full | Fossil image identification using deep learning ensembles of data augmented multiviews |
| title_fullStr | Fossil image identification using deep learning ensembles of data augmented multiviews |
| title_full_unstemmed | Fossil image identification using deep learning ensembles of data augmented multiviews |
| title_short | Fossil image identification using deep learning ensembles of data augmented multiviews |
| title_sort | fossil image identification using deep learning ensembles of data augmented multiviews |
| topic | deep learning ensemble fossil identification fusulinid identification inconsistency image classification |
| url | https://doi.org/10.1111/2041-210X.14229 |
| work_keys_str_mv | AT chengbinhou fossilimageidentificationusingdeeplearningensemblesofdataaugmentedmultiviews AT xinyulin fossilimageidentificationusingdeeplearningensemblesofdataaugmentedmultiviews AT hanhuihuang fossilimageidentificationusingdeeplearningensemblesofdataaugmentedmultiviews AT shengxu fossilimageidentificationusingdeeplearningensemblesofdataaugmentedmultiviews AT junxuanfan fossilimageidentificationusingdeeplearningensemblesofdataaugmentedmultiviews AT yukunshi fossilimageidentificationusingdeeplearningensemblesofdataaugmentedmultiviews AT haironglv fossilimageidentificationusingdeeplearningensemblesofdataaugmentedmultiviews |