Crop Identification Using Deep Learning on LUCAS Crop Cover Photos
Massive and high-quality in situ data are essential for Earth-observation-based agricultural monitoring. However, field surveying requires considerable organizational effort and money. Using computer vision to recognize crop types on geo-tagged photos could be a game changer allowing for the provisi...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/14/6298 |
| _version_ | 1827731828986871808 |
|---|---|
| author | Momchil Yordanov Raphaël d’Andrimont Laura Martinez-Sanchez Guido Lemoine Dominique Fasbender Marijn van der Velde |
| author_facet | Momchil Yordanov Raphaël d’Andrimont Laura Martinez-Sanchez Guido Lemoine Dominique Fasbender Marijn van der Velde |
| author_sort | Momchil Yordanov |
| collection | DOAJ |
| description | Massive and high-quality in situ data are essential for Earth-observation-based agricultural monitoring. However, field surveying requires considerable organizational effort and money. Using computer vision to recognize crop types on geo-tagged photos could be a game changer allowing for the provision of timely and accurate crop-specific information. This study presents the first use of the largest multi-year set of labelled close-up in situ photos systematically collected across the European Union from the Land Use Cover Area frame Survey (LUCAS). Benefiting from this unique in situ dataset, this study aims to benchmark and test computer vision models to recognize major crops on close-up photos statistically distributed spatially and through time between 2006 and 2018 in a practical agricultural policy relevant context. The methodology makes use of crop calendars from various sources to ascertain the mature stage of the crop, of an extensive paradigm for the hyper-parameterization of MobileNet from random parameter initialization, and of various techniques from information theory in order to carry out more accurate post-processing filtering on results. The work has produced a dataset of 169,460 images of mature crops for the 12 classes, out of which 15,876 were manually selected as representing a clean sample without any foreign objects or unfavorable conditions. The best-performing model achieved a macro F1 (M-F1) of 0.75 on an imbalanced test dataset of 8642 photos. Using metrics from information theory, namely the equivalence reference probability, resulted in an increase of 6%. The most unfavorable conditions for taking such images, across all crop classes, were found to be too early or late in the season. The proposed methodology shows the possibility of using minimal auxiliary data outside the images themselves in order to achieve an M-F1 of 0.82 for labelling between 12 major European crops. |
| first_indexed | 2024-03-11T00:41:03Z |
| format | Article |
| id | doaj.art-6b2792b0cb884ad6b01e6754be11651a |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T00:41:03Z |
| publishDate | 2023-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-6b2792b0cb884ad6b01e6754be11651a2023-11-18T21:15:46ZengMDPI AGSensors1424-82202023-07-012314629810.3390/s23146298Crop Identification Using Deep Learning on LUCAS Crop Cover PhotosMomchil Yordanov0Raphaël d’Andrimont1Laura Martinez-Sanchez2Guido Lemoine3Dominique Fasbender4Marijn van der Velde5SEIDOR Consulting S.L., 08500 Barcelona, SpainEuropean Commission, Joint Research Centre (JRC), 21027 Ispra, ItalyEuropean Commission, Joint Research Centre (JRC), 21027 Ispra, ItalyEuropean Commission, Joint Research Centre (JRC), 21027 Ispra, ItalyEuropean Commission, Joint Research Centre (JRC), 21027 Ispra, ItalyEuropean Commission, Joint Research Centre (JRC), 21027 Ispra, ItalyMassive and high-quality in situ data are essential for Earth-observation-based agricultural monitoring. However, field surveying requires considerable organizational effort and money. Using computer vision to recognize crop types on geo-tagged photos could be a game changer allowing for the provision of timely and accurate crop-specific information. This study presents the first use of the largest multi-year set of labelled close-up in situ photos systematically collected across the European Union from the Land Use Cover Area frame Survey (LUCAS). Benefiting from this unique in situ dataset, this study aims to benchmark and test computer vision models to recognize major crops on close-up photos statistically distributed spatially and through time between 2006 and 2018 in a practical agricultural policy relevant context. The methodology makes use of crop calendars from various sources to ascertain the mature stage of the crop, of an extensive paradigm for the hyper-parameterization of MobileNet from random parameter initialization, and of various techniques from information theory in order to carry out more accurate post-processing filtering on results. The work has produced a dataset of 169,460 images of mature crops for the 12 classes, out of which 15,876 were manually selected as representing a clean sample without any foreign objects or unfavorable conditions. The best-performing model achieved a macro F1 (M-F1) of 0.75 on an imbalanced test dataset of 8642 photos. Using metrics from information theory, namely the equivalence reference probability, resulted in an increase of 6%. The most unfavorable conditions for taking such images, across all crop classes, were found to be too early or late in the season. The proposed methodology shows the possibility of using minimal auxiliary data outside the images themselves in order to achieve an M-F1 of 0.82 for labelling between 12 major European crops.https://www.mdpi.com/1424-8220/23/14/6298plant recognitionagriculturecomputer visiondeep learningdata valorizationmapping from imagery |
| spellingShingle | Momchil Yordanov Raphaël d’Andrimont Laura Martinez-Sanchez Guido Lemoine Dominique Fasbender Marijn van der Velde Crop Identification Using Deep Learning on LUCAS Crop Cover Photos Sensors plant recognition agriculture computer vision deep learning data valorization mapping from imagery |
| title | Crop Identification Using Deep Learning on LUCAS Crop Cover Photos |
| title_full | Crop Identification Using Deep Learning on LUCAS Crop Cover Photos |
| title_fullStr | Crop Identification Using Deep Learning on LUCAS Crop Cover Photos |
| title_full_unstemmed | Crop Identification Using Deep Learning on LUCAS Crop Cover Photos |
| title_short | Crop Identification Using Deep Learning on LUCAS Crop Cover Photos |
| title_sort | crop identification using deep learning on lucas crop cover photos |
| topic | plant recognition agriculture computer vision deep learning data valorization mapping from imagery |
| url | https://www.mdpi.com/1424-8220/23/14/6298 |
| work_keys_str_mv | AT momchilyordanov cropidentificationusingdeeplearningonlucascropcoverphotos AT raphaeldandrimont cropidentificationusingdeeplearningonlucascropcoverphotos AT lauramartinezsanchez cropidentificationusingdeeplearningonlucascropcoverphotos AT guidolemoine cropidentificationusingdeeplearningonlucascropcoverphotos AT dominiquefasbender cropidentificationusingdeeplearningonlucascropcoverphotos AT marijnvandervelde cropidentificationusingdeeplearningonlucascropcoverphotos |