ResNet incorporating the fusion data of RGB & hyperspectral images improves classification accuracy of vegetable soybean freshness
Abstract The freshness of vegetable soybean (VS) is an important indicator for quality evaluation. Currently, deep learning-based image recognition technology provides a fast, efficient, and low-cost method for analyzing the freshness of food. The RGB (red, green, and blue) image recognition technol...
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-01-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-024-51668-6 |
| _version_ | 1797274642673565696 |
|---|---|
| author | Yuanpeng Bu Jinxuan Hu Cheng Chen Songhang Bai Zuohui Chen Tianyu Hu Guwen Zhang Na Liu Chang Cai Yuhao Li Qi Xuan Ye Wang Zhongjing Su Yun Xiang Yaming Gong |
| author_facet | Yuanpeng Bu Jinxuan Hu Cheng Chen Songhang Bai Zuohui Chen Tianyu Hu Guwen Zhang Na Liu Chang Cai Yuhao Li Qi Xuan Ye Wang Zhongjing Su Yun Xiang Yaming Gong |
| author_sort | Yuanpeng Bu |
| collection | DOAJ |
| description | Abstract The freshness of vegetable soybean (VS) is an important indicator for quality evaluation. Currently, deep learning-based image recognition technology provides a fast, efficient, and low-cost method for analyzing the freshness of food. The RGB (red, green, and blue) image recognition technology is widely used in the study of food appearance evaluation. In addition, the hyperspectral image has outstanding performance in predicting the nutrient content of samples. However, there are few reports on the research of classification models based on the fusion data of these two sources of images. We collected RGB and hyperspectral images at four different storage times of VS. The ENVI software was adopted to extract the hyperspectral information, and the RGB images were reconstructed based on the downsampling technology. Then, the one-dimensional hyperspectral data was transformed into a two-dimensional space, which allows it to be overlaid and concatenated with the RGB image data in the channel direction, thereby generating fused data. Compared with four commonly used machine learning models, the deep learning model ResNet18 has higher classification accuracy and computational efficiency. Based on the above results, a novel classification model named ResNet-R &H, which is based on the residual networks (ResNet) structure and incorporates the fusion data of RGB and hyperspectral images, was proposed. The ResNet-R &H can achieve a testing accuracy of 97.6%, which demonstrates a significant enhancement of 4.0% and 7.2% compared to the distinct utilization of hyperspectral data and RGB data, respectively. Overall, this research is significant in providing a unique, efficient, and more accurate classification approach in evaluating the freshness of vegetable soybean. The method proposed in this study can provide a theoretical reference for classifying the freshness of fruits and vegetables to improve classification accuracy and reduce human error and variability. |
| first_indexed | 2024-03-07T15:02:13Z |
| format | Article |
| id | doaj.art-470181892ac04253b8f1238d0601b025 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-03-07T15:02:13Z |
| publishDate | 2024-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-470181892ac04253b8f1238d0601b0252024-03-05T19:05:16ZengNature PortfolioScientific Reports2045-23222024-01-0114111310.1038/s41598-024-51668-6ResNet incorporating the fusion data of RGB & hyperspectral images improves classification accuracy of vegetable soybean freshnessYuanpeng Bu0Jinxuan Hu1Cheng Chen2Songhang Bai3Zuohui Chen4Tianyu Hu5Guwen Zhang6Na Liu7Chang Cai8Yuhao Li9Qi Xuan10Ye Wang11Zhongjing Su12Yun Xiang13Yaming Gong14Institute of Vegetables, Key Laboratory of Vegetable Legumes Germplasm Enhancement and Southern China of the Ministry of Agriculture and Rural Affairs, Zhejiang Academy of Agricultural SciencesInstitute of Cyberspace Security, Zhejiang University of TechnologyZhejiang Yuncheng Information technology Co Ltd.Institute of Cyberspace Security, Zhejiang University of TechnologyInstitute of Cyberspace Security, Zhejiang University of TechnologyInstitute of Cyberspace Security, Zhejiang University of TechnologyInstitute of Vegetables, Key Laboratory of Vegetable Legumes Germplasm Enhancement and Southern China of the Ministry of Agriculture and Rural Affairs, Zhejiang Academy of Agricultural SciencesInstitute of Vegetables, Key Laboratory of Vegetable Legumes Germplasm Enhancement and Southern China of the Ministry of Agriculture and Rural Affairs, Zhejiang Academy of Agricultural SciencesInstitute of Cyberspace Security, Zhejiang University of TechnologyInstitute of Cyberspace Security, Zhejiang University of TechnologyInstitute of Cyberspace Security, Zhejiang University of TechnologyFaculty of Engineering, Lishui UniversityInstitute of Cyberspace Security, Zhejiang University of TechnologyInstitute of Cyberspace Security, Zhejiang University of TechnologyInstitute of Vegetables, Key Laboratory of Vegetable Legumes Germplasm Enhancement and Southern China of the Ministry of Agriculture and Rural Affairs, Zhejiang Academy of Agricultural SciencesAbstract The freshness of vegetable soybean (VS) is an important indicator for quality evaluation. Currently, deep learning-based image recognition technology provides a fast, efficient, and low-cost method for analyzing the freshness of food. The RGB (red, green, and blue) image recognition technology is widely used in the study of food appearance evaluation. In addition, the hyperspectral image has outstanding performance in predicting the nutrient content of samples. However, there are few reports on the research of classification models based on the fusion data of these two sources of images. We collected RGB and hyperspectral images at four different storage times of VS. The ENVI software was adopted to extract the hyperspectral information, and the RGB images were reconstructed based on the downsampling technology. Then, the one-dimensional hyperspectral data was transformed into a two-dimensional space, which allows it to be overlaid and concatenated with the RGB image data in the channel direction, thereby generating fused data. Compared with four commonly used machine learning models, the deep learning model ResNet18 has higher classification accuracy and computational efficiency. Based on the above results, a novel classification model named ResNet-R &H, which is based on the residual networks (ResNet) structure and incorporates the fusion data of RGB and hyperspectral images, was proposed. The ResNet-R &H can achieve a testing accuracy of 97.6%, which demonstrates a significant enhancement of 4.0% and 7.2% compared to the distinct utilization of hyperspectral data and RGB data, respectively. Overall, this research is significant in providing a unique, efficient, and more accurate classification approach in evaluating the freshness of vegetable soybean. The method proposed in this study can provide a theoretical reference for classifying the freshness of fruits and vegetables to improve classification accuracy and reduce human error and variability.https://doi.org/10.1038/s41598-024-51668-6 |
| spellingShingle | Yuanpeng Bu Jinxuan Hu Cheng Chen Songhang Bai Zuohui Chen Tianyu Hu Guwen Zhang Na Liu Chang Cai Yuhao Li Qi Xuan Ye Wang Zhongjing Su Yun Xiang Yaming Gong ResNet incorporating the fusion data of RGB & hyperspectral images improves classification accuracy of vegetable soybean freshness Scientific Reports |
| title | ResNet incorporating the fusion data of RGB & hyperspectral images improves classification accuracy of vegetable soybean freshness |
| title_full | ResNet incorporating the fusion data of RGB & hyperspectral images improves classification accuracy of vegetable soybean freshness |
| title_fullStr | ResNet incorporating the fusion data of RGB & hyperspectral images improves classification accuracy of vegetable soybean freshness |
| title_full_unstemmed | ResNet incorporating the fusion data of RGB & hyperspectral images improves classification accuracy of vegetable soybean freshness |
| title_short | ResNet incorporating the fusion data of RGB & hyperspectral images improves classification accuracy of vegetable soybean freshness |
| title_sort | resnet incorporating the fusion data of rgb hyperspectral images improves classification accuracy of vegetable soybean freshness |
| url | https://doi.org/10.1038/s41598-024-51668-6 |
| work_keys_str_mv | AT yuanpengbu resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT jinxuanhu resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT chengchen resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT songhangbai resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT zuohuichen resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT tianyuhu resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT guwenzhang resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT naliu resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT changcai resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT yuhaoli resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT qixuan resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT yewang resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT zhongjingsu resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT yunxiang resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness AT yaminggong resnetincorporatingthefusiondataofrgbhyperspectralimagesimprovesclassificationaccuracyofvegetablesoybeanfreshness |