Transformer in UAV Image-Based Weed Mapping
Weeds affect crop yield and quality due to competition for resources. In order to reduce the risk of yield losses due to weeds, herbicides or non-chemical measures are applied. Weeds, especially creeping perennial species, are generally distributed in patches within arable fields. Hence, instead of...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-10-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/21/5165 |
| _version_ | 1797631344967155712 |
|---|---|
| author | Jiangsan Zhao Therese With Berge Jakob Geipel |
| author_facet | Jiangsan Zhao Therese With Berge Jakob Geipel |
| author_sort | Jiangsan Zhao |
| collection | DOAJ |
| description | Weeds affect crop yield and quality due to competition for resources. In order to reduce the risk of yield losses due to weeds, herbicides or non-chemical measures are applied. Weeds, especially creeping perennial species, are generally distributed in patches within arable fields. Hence, instead of applying control measures uniformly, precision weeding or site-specific weed management (SSWM) is highly recommended. Unmanned aerial vehicle (UAV) imaging is known for wide area coverage and flexible operation frequency, making it a potential solution to generate weed maps at a reasonable cost. Efficient weed mapping algorithms need to be developed together with UAV imagery to facilitate SSWM. Different machine learning (ML) approaches have been developed for image-based weed mapping, either classical ML models or the more up-to-date deep learning (DL) models taking full advantage of parallel computation on a GPU (graphics processing unit). Attention-based transformer DL models, which have seen a recent boom, are expected to overtake classical convolutional neural network (CNN) DL models. This inspired us to develop a transformer DL model for segmenting weeds, cereal crops, and ‘other’ in low-resolution RGB UAV imagery (about 33 mm ground sampling distance, g.s.d.) captured after the cereal crop had turned yellow. Images were acquired during three years in 15 fields with three cereal species (<i>Triticum aestivum</i>, <i>Hordeum vulgare</i>, and <i>Avena sativa</i>) and various weed flora dominated by creeping perennials (mainly <i>Cirsium arvense</i> and <i>Elymus repens</i>). The performance of our transformer model, 1Dtransformer, was evaluated through comparison with a classical DL model, 1DCNN, and two classical ML methods, i.e., random forest (RF) and k-nearest neighbor (KNN). The transformer model showed the best performance with an overall accuracy of 98.694% on pixels set aside for validation. It also agreed best and relatively well with ground reference data on total weed coverage, R2 = 0.598. In this study, we showed the outstanding performance and robustness of a 1Dtransformer model for weed mapping based on UAV imagery for the first time. The model can be used to obtain weed maps in cereals fields known to be infested by perennial weeds. These maps can be used as basis for the generation of prescription maps for SSWM, either pre-harvest, post-harvest, or in the next crop, by applying herbicides or non-chemical measures. |
| first_indexed | 2024-03-11T11:22:25Z |
| format | Article |
| id | doaj.art-90b28e2b4a7a4e37bec8ac09f235e866 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-11T11:22:25Z |
| publishDate | 2023-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-90b28e2b4a7a4e37bec8ac09f235e8662023-11-10T15:11:15ZengMDPI AGRemote Sensing2072-42922023-10-011521516510.3390/rs15215165Transformer in UAV Image-Based Weed MappingJiangsan Zhao0Therese With Berge1Jakob Geipel2Department of Agricultural Technology, Norwegian Institute of Bioeconomy Research (NIBIO), P.O. Box 115, NO-1431 Ås, NorwayDepartment of Invertebrate Pests and Weeds in Forestry, Agriculture and Horticulture, Norwegian Institute of Bioeconomy Research (NIBIO), P.O. Box 115, NO-1431 Ås, NorwayDepartment of Agricultural Technology, Norwegian Institute of Bioeconomy Research (NIBIO), P.O. Box 115, NO-1431 Ås, NorwayWeeds affect crop yield and quality due to competition for resources. In order to reduce the risk of yield losses due to weeds, herbicides or non-chemical measures are applied. Weeds, especially creeping perennial species, are generally distributed in patches within arable fields. Hence, instead of applying control measures uniformly, precision weeding or site-specific weed management (SSWM) is highly recommended. Unmanned aerial vehicle (UAV) imaging is known for wide area coverage and flexible operation frequency, making it a potential solution to generate weed maps at a reasonable cost. Efficient weed mapping algorithms need to be developed together with UAV imagery to facilitate SSWM. Different machine learning (ML) approaches have been developed for image-based weed mapping, either classical ML models or the more up-to-date deep learning (DL) models taking full advantage of parallel computation on a GPU (graphics processing unit). Attention-based transformer DL models, which have seen a recent boom, are expected to overtake classical convolutional neural network (CNN) DL models. This inspired us to develop a transformer DL model for segmenting weeds, cereal crops, and ‘other’ in low-resolution RGB UAV imagery (about 33 mm ground sampling distance, g.s.d.) captured after the cereal crop had turned yellow. Images were acquired during three years in 15 fields with three cereal species (<i>Triticum aestivum</i>, <i>Hordeum vulgare</i>, and <i>Avena sativa</i>) and various weed flora dominated by creeping perennials (mainly <i>Cirsium arvense</i> and <i>Elymus repens</i>). The performance of our transformer model, 1Dtransformer, was evaluated through comparison with a classical DL model, 1DCNN, and two classical ML methods, i.e., random forest (RF) and k-nearest neighbor (KNN). The transformer model showed the best performance with an overall accuracy of 98.694% on pixels set aside for validation. It also agreed best and relatively well with ground reference data on total weed coverage, R2 = 0.598. In this study, we showed the outstanding performance and robustness of a 1Dtransformer model for weed mapping based on UAV imagery for the first time. The model can be used to obtain weed maps in cereals fields known to be infested by perennial weeds. These maps can be used as basis for the generation of prescription maps for SSWM, either pre-harvest, post-harvest, or in the next crop, by applying herbicides or non-chemical measures.https://www.mdpi.com/2072-4292/15/21/51651Dtransformerperennial weed mappinglow-resolution UAV image |
| spellingShingle | Jiangsan Zhao Therese With Berge Jakob Geipel Transformer in UAV Image-Based Weed Mapping Remote Sensing 1Dtransformer perennial weed mapping low-resolution UAV image |
| title | Transformer in UAV Image-Based Weed Mapping |
| title_full | Transformer in UAV Image-Based Weed Mapping |
| title_fullStr | Transformer in UAV Image-Based Weed Mapping |
| title_full_unstemmed | Transformer in UAV Image-Based Weed Mapping |
| title_short | Transformer in UAV Image-Based Weed Mapping |
| title_sort | transformer in uav image based weed mapping |
| topic | 1Dtransformer perennial weed mapping low-resolution UAV image |
| url | https://www.mdpi.com/2072-4292/15/21/5165 |
| work_keys_str_mv | AT jiangsanzhao transformerinuavimagebasedweedmapping AT theresewithberge transformerinuavimagebasedweedmapping AT jakobgeipel transformerinuavimagebasedweedmapping |