Tomato Disease Classification and Identification Method Based on Multimodal Fusion Deep Learning
Considering that the occurrence and spread of diseases are closely related to the planting environment, a tomato disease diagnosis method based on Multi-ResNet34 multi-modal fusion learning based on residual learning is proposed for the problem of limited recognition rate of a single RGB image of a...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Agriculture |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-0472/12/12/2014 |
| _version_ | 1797461943036936192 |
|---|---|
| author | Ning Zhang Huarui Wu Huaji Zhu Ying Deng Xiao Han |
| author_facet | Ning Zhang Huarui Wu Huaji Zhu Ying Deng Xiao Han |
| author_sort | Ning Zhang |
| collection | DOAJ |
| description | Considering that the occurrence and spread of diseases are closely related to the planting environment, a tomato disease diagnosis method based on Multi-ResNet34 multi-modal fusion learning based on residual learning is proposed for the problem of limited recognition rate of a single RGB image of a tomato disease. Based on the ResNet34 backbone network, this paper introduces transfer learning to speed up training, reduce data dependencies, and prevent overfitting due to a small amount of sample data; it also integrates multi-source data (tomato disease image data and environmental parameters). The feature-level multi-modal data fusion method is used to retain the key information of the data to identify the feature, so that the different modal data can complement, support and correct each other, and obtain a more accurate identification effect. Firstly, Mask R-CNN was used to extract partial images of leaves from complex background tomato disease images to reduce the influence of background regions on disease identification. Then, the formed image environment data set was input into the multi-modal fusion model to obtain the identification results of disease types. The proposed multi-modal fusion model Multi-ResNet34 has a classification accuracy of 98.9% for six tomato diseases: bacterial spot, late blight, leaf mold, yellow aspergillosis, gray mold, and early blight, which is higher than that of the single-modal model. With the increase by 1.1%, the effect is obvious. The method in this paper can provide an important basis for the analysis and diagnosis of tomato intelligent greenhouse diseases in the context of agricultural informatization. |
| first_indexed | 2024-03-09T17:27:12Z |
| format | Article |
| id | doaj.art-aff2bae6d318474bba4fdcc0c7ceef9a |
| institution | Directory Open Access Journal |
| issn | 2077-0472 |
| language | English |
| last_indexed | 2024-03-09T17:27:12Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Agriculture |
| spelling | doaj.art-aff2bae6d318474bba4fdcc0c7ceef9a2023-11-24T12:39:52ZengMDPI AGAgriculture2077-04722022-11-011212201410.3390/agriculture12122014Tomato Disease Classification and Identification Method Based on Multimodal Fusion Deep LearningNing Zhang0Huarui Wu1Huaji Zhu2Ying Deng3Xiao Han4National Engineering Research Center for Information Technology in Agriculture, Beijing 100097, ChinaNational Engineering Research Center for Information Technology in Agriculture, Beijing 100097, ChinaNational Engineering Research Center for Information Technology in Agriculture, Beijing 100097, ChinaNational Engineering Research Center for Information Technology in Agriculture, Beijing 100097, ChinaNational Engineering Research Center for Information Technology in Agriculture, Beijing 100097, ChinaConsidering that the occurrence and spread of diseases are closely related to the planting environment, a tomato disease diagnosis method based on Multi-ResNet34 multi-modal fusion learning based on residual learning is proposed for the problem of limited recognition rate of a single RGB image of a tomato disease. Based on the ResNet34 backbone network, this paper introduces transfer learning to speed up training, reduce data dependencies, and prevent overfitting due to a small amount of sample data; it also integrates multi-source data (tomato disease image data and environmental parameters). The feature-level multi-modal data fusion method is used to retain the key information of the data to identify the feature, so that the different modal data can complement, support and correct each other, and obtain a more accurate identification effect. Firstly, Mask R-CNN was used to extract partial images of leaves from complex background tomato disease images to reduce the influence of background regions on disease identification. Then, the formed image environment data set was input into the multi-modal fusion model to obtain the identification results of disease types. The proposed multi-modal fusion model Multi-ResNet34 has a classification accuracy of 98.9% for six tomato diseases: bacterial spot, late blight, leaf mold, yellow aspergillosis, gray mold, and early blight, which is higher than that of the single-modal model. With the increase by 1.1%, the effect is obvious. The method in this paper can provide an important basis for the analysis and diagnosis of tomato intelligent greenhouse diseases in the context of agricultural informatization.https://www.mdpi.com/2077-0472/12/12/2014multimodal fusiontransfer learningResNet34residual networkdisease diagnosis |
| spellingShingle | Ning Zhang Huarui Wu Huaji Zhu Ying Deng Xiao Han Tomato Disease Classification and Identification Method Based on Multimodal Fusion Deep Learning Agriculture multimodal fusion transfer learning ResNet34 residual network disease diagnosis |
| title | Tomato Disease Classification and Identification Method Based on Multimodal Fusion Deep Learning |
| title_full | Tomato Disease Classification and Identification Method Based on Multimodal Fusion Deep Learning |
| title_fullStr | Tomato Disease Classification and Identification Method Based on Multimodal Fusion Deep Learning |
| title_full_unstemmed | Tomato Disease Classification and Identification Method Based on Multimodal Fusion Deep Learning |
| title_short | Tomato Disease Classification and Identification Method Based on Multimodal Fusion Deep Learning |
| title_sort | tomato disease classification and identification method based on multimodal fusion deep learning |
| topic | multimodal fusion transfer learning ResNet34 residual network disease diagnosis |
| url | https://www.mdpi.com/2077-0472/12/12/2014 |
| work_keys_str_mv | AT ningzhang tomatodiseaseclassificationandidentificationmethodbasedonmultimodalfusiondeeplearning AT huaruiwu tomatodiseaseclassificationandidentificationmethodbasedonmultimodalfusiondeeplearning AT huajizhu tomatodiseaseclassificationandidentificationmethodbasedonmultimodalfusiondeeplearning AT yingdeng tomatodiseaseclassificationandidentificationmethodbasedonmultimodalfusiondeeplearning AT xiaohan tomatodiseaseclassificationandidentificationmethodbasedonmultimodalfusiondeeplearning |