Utilisation of Deep Learning with Multimodal Data Fusion for Determination of Pineapple Quality Using Thermal Imaging
Fruit quality is an important aspect in determining the consumer preference in the supply chain. Thermal imaging was used to determine different pineapple varieties according to the physicochemical changes of the fruit by means of the deep learning method. Deep learning has gained attention in fruit...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Agronomy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-4395/13/2/401 |
| _version_ | 1797622901180989440 |
|---|---|
| author | Maimunah Mohd Ali Norhashila Hashim Samsuzana Abd Aziz Ola Lasekan |
| author_facet | Maimunah Mohd Ali Norhashila Hashim Samsuzana Abd Aziz Ola Lasekan |
| author_sort | Maimunah Mohd Ali |
| collection | DOAJ |
| description | Fruit quality is an important aspect in determining the consumer preference in the supply chain. Thermal imaging was used to determine different pineapple varieties according to the physicochemical changes of the fruit by means of the deep learning method. Deep learning has gained attention in fruit classification and recognition in unimodal processing. This paper proposes a multimodal data fusion framework for the determination of pineapple quality using deep learning methods based on the feature extraction acquired from thermal imaging. Feature extraction was selected from the thermal images that provided a correlation with the quality attributes of the fruit in developing the deep learning models. Three different types of deep learning architectures, including ResNet, VGG16, and InceptionV3, were built to develop the multimodal data fusion framework for the classification of pineapple varieties based on the concatenation of multiple features extracted by the robust networks. The multimodal data fusion coupled with powerful convolutional neural network architectures can remarkably distinguish different pineapple varieties. The proposed multimodal data fusion framework provides a reliable determination of fruit quality that can improve the recognition accuracy and the model performance up to 0.9687. The effectiveness of multimodal deep learning data fusion and thermal imaging has huge potential in monitoring the real-time determination of physicochemical changes of fruit. |
| first_indexed | 2024-03-11T09:16:52Z |
| format | Article |
| id | doaj.art-4dfe2782026345d881f6eb05379e9a3b |
| institution | Directory Open Access Journal |
| issn | 2073-4395 |
| language | English |
| last_indexed | 2024-03-11T09:16:52Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Agronomy |
| spelling | doaj.art-4dfe2782026345d881f6eb05379e9a3b2023-11-16T18:34:04ZengMDPI AGAgronomy2073-43952023-01-0113240110.3390/agronomy13020401Utilisation of Deep Learning with Multimodal Data Fusion for Determination of Pineapple Quality Using Thermal ImagingMaimunah Mohd Ali0Norhashila Hashim1Samsuzana Abd Aziz2Ola Lasekan3Department of Biological and Agricultural Engineering, Faculty of Engineering, Universiti Putra Malaysia, Serdang 43400, Selangor, MalaysiaDepartment of Biological and Agricultural Engineering, Faculty of Engineering, Universiti Putra Malaysia, Serdang 43400, Selangor, MalaysiaDepartment of Biological and Agricultural Engineering, Faculty of Engineering, Universiti Putra Malaysia, Serdang 43400, Selangor, MalaysiaDepartment of Food Technology, Faculty of Food Science and Technology, Universiti Putra Malaysia, Serdang 43400, Selangor, MalaysiaFruit quality is an important aspect in determining the consumer preference in the supply chain. Thermal imaging was used to determine different pineapple varieties according to the physicochemical changes of the fruit by means of the deep learning method. Deep learning has gained attention in fruit classification and recognition in unimodal processing. This paper proposes a multimodal data fusion framework for the determination of pineapple quality using deep learning methods based on the feature extraction acquired from thermal imaging. Feature extraction was selected from the thermal images that provided a correlation with the quality attributes of the fruit in developing the deep learning models. Three different types of deep learning architectures, including ResNet, VGG16, and InceptionV3, were built to develop the multimodal data fusion framework for the classification of pineapple varieties based on the concatenation of multiple features extracted by the robust networks. The multimodal data fusion coupled with powerful convolutional neural network architectures can remarkably distinguish different pineapple varieties. The proposed multimodal data fusion framework provides a reliable determination of fruit quality that can improve the recognition accuracy and the model performance up to 0.9687. The effectiveness of multimodal deep learning data fusion and thermal imaging has huge potential in monitoring the real-time determination of physicochemical changes of fruit.https://www.mdpi.com/2073-4395/13/2/401deep learningthermal imagingfruit qualityconvolutional neural networkmultimodal data fusion |
| spellingShingle | Maimunah Mohd Ali Norhashila Hashim Samsuzana Abd Aziz Ola Lasekan Utilisation of Deep Learning with Multimodal Data Fusion for Determination of Pineapple Quality Using Thermal Imaging Agronomy deep learning thermal imaging fruit quality convolutional neural network multimodal data fusion |
| title | Utilisation of Deep Learning with Multimodal Data Fusion for Determination of Pineapple Quality Using Thermal Imaging |
| title_full | Utilisation of Deep Learning with Multimodal Data Fusion for Determination of Pineapple Quality Using Thermal Imaging |
| title_fullStr | Utilisation of Deep Learning with Multimodal Data Fusion for Determination of Pineapple Quality Using Thermal Imaging |
| title_full_unstemmed | Utilisation of Deep Learning with Multimodal Data Fusion for Determination of Pineapple Quality Using Thermal Imaging |
| title_short | Utilisation of Deep Learning with Multimodal Data Fusion for Determination of Pineapple Quality Using Thermal Imaging |
| title_sort | utilisation of deep learning with multimodal data fusion for determination of pineapple quality using thermal imaging |
| topic | deep learning thermal imaging fruit quality convolutional neural network multimodal data fusion |
| url | https://www.mdpi.com/2073-4395/13/2/401 |
| work_keys_str_mv | AT maimunahmohdali utilisationofdeeplearningwithmultimodaldatafusionfordeterminationofpineapplequalityusingthermalimaging AT norhashilahashim utilisationofdeeplearningwithmultimodaldatafusionfordeterminationofpineapplequalityusingthermalimaging AT samsuzanaabdaziz utilisationofdeeplearningwithmultimodaldatafusionfordeterminationofpineapplequalityusingthermalimaging AT olalasekan utilisationofdeeplearningwithmultimodaldatafusionfordeterminationofpineapplequalityusingthermalimaging |