A Detection Method for Individual Infected Pine Trees with Pine Wilt Disease Based on Deep Learning
Pine wilt disease (PWD) can cause destructive death in many species of pine trees within a short period. The recognition of infected pine trees in unmanned aerial vehicle (UAV) forest images is a key technology for automatic monitoring and early warning of pests. This paper collected UAV visible and...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Forests |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-4907/13/11/1880 |
| _version_ | 1797468206801092608 |
|---|---|
| author | Yan Zhou Wenping Liu Haojie Bi Riqiang Chen Shixiang Zong Youqing Luo |
| author_facet | Yan Zhou Wenping Liu Haojie Bi Riqiang Chen Shixiang Zong Youqing Luo |
| author_sort | Yan Zhou |
| collection | DOAJ |
| description | Pine wilt disease (PWD) can cause destructive death in many species of pine trees within a short period. The recognition of infected pine trees in unmanned aerial vehicle (UAV) forest images is a key technology for automatic monitoring and early warning of pests. This paper collected UAV visible and multispectral images of Korean pines (<i>Pinus koraiensis</i>) and Chinese pines (<i>P. tabulaeformis</i>) infected by PWD and divided the PWD infection into early, middle, and late stages. With the open-source annotation tool, LabelImg, we labeled the category of infected pine trees at each stage. After coordinate-correction preprocessing of the ground truth, the Korean pine and Chinese pine datasets were established. As a means of detecting infected pine trees of PWD and determining different infection stages, a multi-band image-fusion infected pine tree detector (MFTD) based on deep learning was proposed. Firstly, the Halfway Fusion mode was adopted to fuse the network based on four YOLOv5 variants. Simultaneously, the Backbone network was initially designed as a dual branching network that includes visible and multispectral subnets. Moreover, the features of visible and multispectral images were extracted. To fully utilize the features of visible and multispectral images, a multi-band feature fusion transformer (MFFT) with a multi-head attention mechanism and a feed-forward network was constructed to enhance the information correlation between visible and multispectral feature maps. Finally, following the MFFT module, the two feature maps were fused and input into Neck and Head to predict the categories and positions of infected pine trees. The best-performing MFTD model achieved the highest detection accuracy with mean average precision values (mAP@50) of 88.5% and 86.8% on Korean pine and Chinese pine datasets, respectively, which improved by 8.6% and 10.8% compared to the original YOLOv5 models trained only with visible images. In addition, the average precision values (AP@50) are 87.2%, 93.5%, and 84.8% for early, middle, and late stages on the KP dataset and 81.2%, 92.9%, and 86.2% on the CP dataset. Furthermore, the largest improvement is observed in the early stage with 14.3% and 11.6%, respectively. The results show that MFTD can accurately detect the infected pine trees, especially those at the early stage, and improve the early warning ability of PWD. |
| first_indexed | 2024-03-09T19:04:12Z |
| format | Article |
| id | doaj.art-1cd96e49f70343d09599fa27e54a6d4b |
| institution | Directory Open Access Journal |
| issn | 1999-4907 |
| language | English |
| last_indexed | 2024-03-09T19:04:12Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Forests |
| spelling | doaj.art-1cd96e49f70343d09599fa27e54a6d4b2023-11-24T04:44:44ZengMDPI AGForests1999-49072022-11-011311188010.3390/f13111880A Detection Method for Individual Infected Pine Trees with Pine Wilt Disease Based on Deep LearningYan Zhou0Wenping Liu1Haojie Bi2Riqiang Chen3Shixiang Zong4Youqing Luo5College of Information, Beijing Forestry University, Beijing 100083, ChinaCollege of Information, Beijing Forestry University, Beijing 100083, ChinaCollege of Forestry, Beijing Forestry University, Beijing 100083, ChinaCollege of Information, Beijing Forestry University, Beijing 100083, ChinaCollege of Forestry, Beijing Forestry University, Beijing 100083, ChinaCollege of Forestry, Beijing Forestry University, Beijing 100083, ChinaPine wilt disease (PWD) can cause destructive death in many species of pine trees within a short period. The recognition of infected pine trees in unmanned aerial vehicle (UAV) forest images is a key technology for automatic monitoring and early warning of pests. This paper collected UAV visible and multispectral images of Korean pines (<i>Pinus koraiensis</i>) and Chinese pines (<i>P. tabulaeformis</i>) infected by PWD and divided the PWD infection into early, middle, and late stages. With the open-source annotation tool, LabelImg, we labeled the category of infected pine trees at each stage. After coordinate-correction preprocessing of the ground truth, the Korean pine and Chinese pine datasets were established. As a means of detecting infected pine trees of PWD and determining different infection stages, a multi-band image-fusion infected pine tree detector (MFTD) based on deep learning was proposed. Firstly, the Halfway Fusion mode was adopted to fuse the network based on four YOLOv5 variants. Simultaneously, the Backbone network was initially designed as a dual branching network that includes visible and multispectral subnets. Moreover, the features of visible and multispectral images were extracted. To fully utilize the features of visible and multispectral images, a multi-band feature fusion transformer (MFFT) with a multi-head attention mechanism and a feed-forward network was constructed to enhance the information correlation between visible and multispectral feature maps. Finally, following the MFFT module, the two feature maps were fused and input into Neck and Head to predict the categories and positions of infected pine trees. The best-performing MFTD model achieved the highest detection accuracy with mean average precision values (mAP@50) of 88.5% and 86.8% on Korean pine and Chinese pine datasets, respectively, which improved by 8.6% and 10.8% compared to the original YOLOv5 models trained only with visible images. In addition, the average precision values (AP@50) are 87.2%, 93.5%, and 84.8% for early, middle, and late stages on the KP dataset and 81.2%, 92.9%, and 86.2% on the CP dataset. Furthermore, the largest improvement is observed in the early stage with 14.3% and 11.6%, respectively. The results show that MFTD can accurately detect the infected pine trees, especially those at the early stage, and improve the early warning ability of PWD.https://www.mdpi.com/1999-4907/13/11/1880unmanned aerial vehiclepine wood nematodeconvolutional neural networkobject detectionYOLOv5 |
| spellingShingle | Yan Zhou Wenping Liu Haojie Bi Riqiang Chen Shixiang Zong Youqing Luo A Detection Method for Individual Infected Pine Trees with Pine Wilt Disease Based on Deep Learning Forests unmanned aerial vehicle pine wood nematode convolutional neural network object detection YOLOv5 |
| title | A Detection Method for Individual Infected Pine Trees with Pine Wilt Disease Based on Deep Learning |
| title_full | A Detection Method for Individual Infected Pine Trees with Pine Wilt Disease Based on Deep Learning |
| title_fullStr | A Detection Method for Individual Infected Pine Trees with Pine Wilt Disease Based on Deep Learning |
| title_full_unstemmed | A Detection Method for Individual Infected Pine Trees with Pine Wilt Disease Based on Deep Learning |
| title_short | A Detection Method for Individual Infected Pine Trees with Pine Wilt Disease Based on Deep Learning |
| title_sort | detection method for individual infected pine trees with pine wilt disease based on deep learning |
| topic | unmanned aerial vehicle pine wood nematode convolutional neural network object detection YOLOv5 |
| url | https://www.mdpi.com/1999-4907/13/11/1880 |
| work_keys_str_mv | AT yanzhou adetectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning AT wenpingliu adetectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning AT haojiebi adetectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning AT riqiangchen adetectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning AT shixiangzong adetectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning AT youqingluo adetectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning AT yanzhou detectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning AT wenpingliu detectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning AT haojiebi detectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning AT riqiangchen detectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning AT shixiangzong detectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning AT youqingluo detectionmethodforindividualinfectedpinetreeswithpinewiltdiseasebasedondeeplearning |