Plant recognition by AI: Deep neural nets, transformers, and kNN in deep embeddings
The article reviews and benchmarks machine learning methods for automatic image-based plant species recognition and proposes a novel retrieval-based method for recognition by nearest neighbor classification in a deep embedding space. The image retrieval method relies on a model trained via the Recal...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-09-01
|
| Series: | Frontiers in Plant Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpls.2022.787527/full |
| _version_ | 1811204036862410752 |
|---|---|
| author | Lukáš Picek Milan Šulc Yash Patel Jiří Matas |
| author_facet | Lukáš Picek Milan Šulc Yash Patel Jiří Matas |
| author_sort | Lukáš Picek |
| collection | DOAJ |
| description | The article reviews and benchmarks machine learning methods for automatic image-based plant species recognition and proposes a novel retrieval-based method for recognition by nearest neighbor classification in a deep embedding space. The image retrieval method relies on a model trained via the Recall@k surrogate loss. State-of-the-art approaches to image classification, based on Convolutional Neural Networks (CNN) and Vision Transformers (ViT), are benchmarked and compared with the proposed image retrieval-based method. The impact of performance-enhancing techniques, e.g., class prior adaptation, image augmentations, learning rate scheduling, and loss functions, is studied. The evaluation is carried out on the PlantCLEF 2017, the ExpertLifeCLEF 2018, and the iNaturalist 2018 Datasets—the largest publicly available datasets for plant recognition. The evaluation of CNN and ViT classifiers shows a gradual improvement in classification accuracy. The current state-of-the-art Vision Transformer model, ViT-Large/16, achieves 91.15% and 83.54% accuracy on the PlantCLEF 2017 and ExpertLifeCLEF 2018 test sets, respectively; the best CNN model (ResNeSt-269e) error rate dropped by 22.91% and 28.34%. Apart from that, additional tricks increased the performance for the ViT-Base/32 by 3.72% on ExpertLifeCLEF 2018 and by 4.67% on PlantCLEF 2017. The retrieval approach achieved superior performance in all measured scenarios with accuracy margins of 0.28%, 4.13%, and 10.25% on ExpertLifeCLEF 2018, PlantCLEF 2017, and iNat2018–Plantae, respectively. |
| first_indexed | 2024-04-12T03:06:11Z |
| format | Article |
| id | doaj.art-4f411174fd1a49b8884e73033293aa62 |
| institution | Directory Open Access Journal |
| issn | 1664-462X |
| language | English |
| last_indexed | 2024-04-12T03:06:11Z |
| publishDate | 2022-09-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Plant Science |
| spelling | doaj.art-4f411174fd1a49b8884e73033293aa622022-12-22T03:50:31ZengFrontiers Media S.A.Frontiers in Plant Science1664-462X2022-09-011310.3389/fpls.2022.787527787527Plant recognition by AI: Deep neural nets, transformers, and kNN in deep embeddingsLukáš Picek0Milan Šulc1Yash Patel2Jiří Matas3Department of Cybernetics, Faculty of Applied Sciences, University of West Bohemia, Pilsen, CzechiaVisual Recognition Group, Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, CzechiaVisual Recognition Group, Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, CzechiaVisual Recognition Group, Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, CzechiaThe article reviews and benchmarks machine learning methods for automatic image-based plant species recognition and proposes a novel retrieval-based method for recognition by nearest neighbor classification in a deep embedding space. The image retrieval method relies on a model trained via the Recall@k surrogate loss. State-of-the-art approaches to image classification, based on Convolutional Neural Networks (CNN) and Vision Transformers (ViT), are benchmarked and compared with the proposed image retrieval-based method. The impact of performance-enhancing techniques, e.g., class prior adaptation, image augmentations, learning rate scheduling, and loss functions, is studied. The evaluation is carried out on the PlantCLEF 2017, the ExpertLifeCLEF 2018, and the iNaturalist 2018 Datasets—the largest publicly available datasets for plant recognition. The evaluation of CNN and ViT classifiers shows a gradual improvement in classification accuracy. The current state-of-the-art Vision Transformer model, ViT-Large/16, achieves 91.15% and 83.54% accuracy on the PlantCLEF 2017 and ExpertLifeCLEF 2018 test sets, respectively; the best CNN model (ResNeSt-269e) error rate dropped by 22.91% and 28.34%. Apart from that, additional tricks increased the performance for the ViT-Base/32 by 3.72% on ExpertLifeCLEF 2018 and by 4.67% on PlantCLEF 2017. The retrieval approach achieved superior performance in all measured scenarios with accuracy margins of 0.28%, 4.13%, and 10.25% on ExpertLifeCLEF 2018, PlantCLEF 2017, and iNat2018–Plantae, respectively.https://www.frontiersin.org/articles/10.3389/fpls.2022.787527/fullplantspeciesclassificationrecognitionmachine learningcomputer vision |
| spellingShingle | Lukáš Picek Milan Šulc Yash Patel Jiří Matas Plant recognition by AI: Deep neural nets, transformers, and kNN in deep embeddings Frontiers in Plant Science plant species classification recognition machine learning computer vision |
| title | Plant recognition by AI: Deep neural nets, transformers, and kNN in deep embeddings |
| title_full | Plant recognition by AI: Deep neural nets, transformers, and kNN in deep embeddings |
| title_fullStr | Plant recognition by AI: Deep neural nets, transformers, and kNN in deep embeddings |
| title_full_unstemmed | Plant recognition by AI: Deep neural nets, transformers, and kNN in deep embeddings |
| title_short | Plant recognition by AI: Deep neural nets, transformers, and kNN in deep embeddings |
| title_sort | plant recognition by ai deep neural nets transformers and knn in deep embeddings |
| topic | plant species classification recognition machine learning computer vision |
| url | https://www.frontiersin.org/articles/10.3389/fpls.2022.787527/full |
| work_keys_str_mv | AT lukaspicek plantrecognitionbyaideepneuralnetstransformersandknnindeepembeddings AT milansulc plantrecognitionbyaideepneuralnetstransformersandknnindeepembeddings AT yashpatel plantrecognitionbyaideepneuralnetstransformersandknnindeepembeddings AT jirimatas plantrecognitionbyaideepneuralnetstransformersandknnindeepembeddings |