CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniques
Abstract Modern people who value healthy eating habits have shown increasing interest in plum (Prunus mume) fruits, primarily owing to their nutritiousness and proven efficacy. As consumption increases, it becomes important to monitor work to prevent Prunus mume fruits from falling out. Moreover, de...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-12-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-25260-9 |
| _version_ | 1811190473822306304 |
|---|---|
| author | EungChan Kim Suk-Ju Hong Sang-Yeon Kim Chang-Hyup Lee Sungjay Kim Hyuck-Joo Kim Ghiseok Kim |
| author_facet | EungChan Kim Suk-Ju Hong Sang-Yeon Kim Chang-Hyup Lee Sungjay Kim Hyuck-Joo Kim Ghiseok Kim |
| author_sort | EungChan Kim |
| collection | DOAJ |
| description | Abstract Modern people who value healthy eating habits have shown increasing interest in plum (Prunus mume) fruits, primarily owing to their nutritiousness and proven efficacy. As consumption increases, it becomes important to monitor work to prevent Prunus mume fruits from falling out. Moreover, determining the growth status of Prunus mume is also crucial and is attracting increasing attention. In this study, convolutional neural network (CNN)-based deep learning object detection was developed using RGBD images collected from Prunus mume farms. These RGBD images consider various environments, including the depth information of objects in the outdoor field. A faster region-based convolutional neural network (R-CNN), EfficientDet, Retinanet, and Single Shot Multibox Detector (SSD) were applied for detection, and the performance of all models was estimated by comparing their respective computing speeds and average precisions (APs). The test results show that the EfficientDet model is the most accurate, and SSD MobileNet is the fastest among the four models. In addition, the algorithm was developed to acquire the growth status of P. mume fruits by applying the coordinates and score values of bounding boxes to the depth map. Compared to the diameters of the artificial Prunus mume fruits used as the experimental group, the calculated diameters were very similar to those of the artificial objects. Collectively, the results demonstrate that the CNN-based deep learning Prunus mume detection and growth estimation method can be applied to real farmlands. |
| first_indexed | 2024-04-11T14:51:40Z |
| format | Article |
| id | doaj.art-6183dfd08b554663824f847de3c9eb30 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-11T14:51:40Z |
| publishDate | 2022-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-6183dfd08b554663824f847de3c9eb302022-12-22T04:17:26ZengNature PortfolioScientific Reports2045-23222022-12-0112111610.1038/s41598-022-25260-9CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniquesEungChan Kim0Suk-Ju Hong1Sang-Yeon Kim2Chang-Hyup Lee3Sungjay Kim4Hyuck-Joo Kim5Ghiseok Kim6Department of Biosystems Engineering, Seoul National UniversityDepartment of Biosystems Engineering, Seoul National UniversityDepartment of Biosystems Engineering, Seoul National UniversityDepartment of Biosystems Engineering, Seoul National UniversityDepartment of Biosystems Engineering, Seoul National UniversityDepartment of Convergent Biosystems Engineering, Sunchon National UniversityDepartment of Biosystems Engineering, Seoul National UniversityAbstract Modern people who value healthy eating habits have shown increasing interest in plum (Prunus mume) fruits, primarily owing to their nutritiousness and proven efficacy. As consumption increases, it becomes important to monitor work to prevent Prunus mume fruits from falling out. Moreover, determining the growth status of Prunus mume is also crucial and is attracting increasing attention. In this study, convolutional neural network (CNN)-based deep learning object detection was developed using RGBD images collected from Prunus mume farms. These RGBD images consider various environments, including the depth information of objects in the outdoor field. A faster region-based convolutional neural network (R-CNN), EfficientDet, Retinanet, and Single Shot Multibox Detector (SSD) were applied for detection, and the performance of all models was estimated by comparing their respective computing speeds and average precisions (APs). The test results show that the EfficientDet model is the most accurate, and SSD MobileNet is the fastest among the four models. In addition, the algorithm was developed to acquire the growth status of P. mume fruits by applying the coordinates and score values of bounding boxes to the depth map. Compared to the diameters of the artificial Prunus mume fruits used as the experimental group, the calculated diameters were very similar to those of the artificial objects. Collectively, the results demonstrate that the CNN-based deep learning Prunus mume detection and growth estimation method can be applied to real farmlands.https://doi.org/10.1038/s41598-022-25260-9 |
| spellingShingle | EungChan Kim Suk-Ju Hong Sang-Yeon Kim Chang-Hyup Lee Sungjay Kim Hyuck-Joo Kim Ghiseok Kim CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniques Scientific Reports |
| title | CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniques |
| title_full | CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniques |
| title_fullStr | CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniques |
| title_full_unstemmed | CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniques |
| title_short | CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniques |
| title_sort | cnn based object detection and growth estimation of plum fruit prunus mume using rgb and depth imaging techniques |
| url | https://doi.org/10.1038/s41598-022-25260-9 |
| work_keys_str_mv | AT eungchankim cnnbasedobjectdetectionandgrowthestimationofplumfruitprunusmumeusingrgbanddepthimagingtechniques AT sukjuhong cnnbasedobjectdetectionandgrowthestimationofplumfruitprunusmumeusingrgbanddepthimagingtechniques AT sangyeonkim cnnbasedobjectdetectionandgrowthestimationofplumfruitprunusmumeusingrgbanddepthimagingtechniques AT changhyuplee cnnbasedobjectdetectionandgrowthestimationofplumfruitprunusmumeusingrgbanddepthimagingtechniques AT sungjaykim cnnbasedobjectdetectionandgrowthestimationofplumfruitprunusmumeusingrgbanddepthimagingtechniques AT hyuckjookim cnnbasedobjectdetectionandgrowthestimationofplumfruitprunusmumeusingrgbanddepthimagingtechniques AT ghiseokkim cnnbasedobjectdetectionandgrowthestimationofplumfruitprunusmumeusingrgbanddepthimagingtechniques |