Vid2Cuts: A Framework for Enabling AI-Guided Grapevine Pruning
Recent advances in machine learning and computer vision promoted a surge in the development of AI-based approaches aimed at improving various agricultural tasks. In this work, we focus on grapevine pruning, which is one of the labor-intensive tasks in viticulture that requires experienced workers an...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10384649/ |
| _version_ | 1797357077401698304 |
|---|---|
| author | Simon Haring Sophie Folawiyo Mariia Podguzova Stephan kraub Didier Stricker |
| author_facet | Simon Haring Sophie Folawiyo Mariia Podguzova Stephan kraub Didier Stricker |
| author_sort | Simon Haring |
| collection | DOAJ |
| description | Recent advances in machine learning and computer vision promoted a surge in the development of AI-based approaches aimed at improving various agricultural tasks. In this work, we focus on grapevine pruning, which is one of the labor-intensive tasks in viticulture that requires experienced workers and has a huge impact on grapevine health, future yields and grape quality. Our objective is to develop an AI-based application that provides pruning suggestions according to the “gentle pruning” strategy enabling non-experts in the field to easily engage in the process. To achieve that, we have to deal with multiple challenges such as a large number of grapevine varieties, complicated outdoor conditions characterized by varied light, weather and complex grapevine structures with multiple occlusions. In this work, we present a framework, which allows the generation of pruning suggestions using a video recorded by a smartphone and visualize them in a mobile AR application. Thus, our contributions are the following: 1) we present the collection of a large image segmentation dataset of dormant grapevines; 2) we propose a novel distributed approach to generate pruning suggestions via a semantic 3D grapevine model generated from a smartphone video; 3) we propose a mobile AR application to visualize the pruning suggestions. Results show the robustness of our approach to outdoor conditions as well as reasonable pruning suggestions according to evaluation by domain experts in 71% of cases. We demonstrate the main challenges that must be addressed for such an application and propose a distributed solution to handle them. |
| first_indexed | 2024-03-08T14:38:05Z |
| format | Article |
| id | doaj.art-4546922277b04f97b18a35b44de62768 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T14:38:05Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-4546922277b04f97b18a35b44de627682024-01-12T00:02:25ZengIEEEIEEE Access2169-35362024-01-01125814583610.1109/ACCESS.2024.335043210384649Vid2Cuts: A Framework for Enabling AI-Guided Grapevine PruningSimon Haring0https://orcid.org/0000-0003-2827-0435Sophie Folawiyo1https://orcid.org/0009-0008-2913-8544Mariia Podguzova2https://orcid.org/0009-0004-4839-0236Stephan kraub3https://orcid.org/0009-0007-6007-3032Didier Stricker4Department of Computer Science, University of Kaiserslautern-Landau (RPTU), Kaiserslautern, GermanyDepartment of Computer Science, University of Kaiserslautern-Landau (RPTU), Kaiserslautern, GermanyDepartment of Computer Science, University of Kaiserslautern-Landau (RPTU), Kaiserslautern, GermanyDepartment of Augmented Vision, German Research Center for Artificial Intelligence (DFKI), Kaiserslautern, GermanyDepartment of Computer Science, University of Kaiserslautern-Landau (RPTU), Kaiserslautern, GermanyRecent advances in machine learning and computer vision promoted a surge in the development of AI-based approaches aimed at improving various agricultural tasks. In this work, we focus on grapevine pruning, which is one of the labor-intensive tasks in viticulture that requires experienced workers and has a huge impact on grapevine health, future yields and grape quality. Our objective is to develop an AI-based application that provides pruning suggestions according to the “gentle pruning” strategy enabling non-experts in the field to easily engage in the process. To achieve that, we have to deal with multiple challenges such as a large number of grapevine varieties, complicated outdoor conditions characterized by varied light, weather and complex grapevine structures with multiple occlusions. In this work, we present a framework, which allows the generation of pruning suggestions using a video recorded by a smartphone and visualize them in a mobile AR application. Thus, our contributions are the following: 1) we present the collection of a large image segmentation dataset of dormant grapevines; 2) we propose a novel distributed approach to generate pruning suggestions via a semantic 3D grapevine model generated from a smartphone video; 3) we propose a mobile AR application to visualize the pruning suggestions. Results show the robustness of our approach to outdoor conditions as well as reasonable pruning suggestions according to evaluation by domain experts in 71% of cases. We demonstrate the main challenges that must be addressed for such an application and propose a distributed solution to handle them.https://ieeexplore.ieee.org/document/10384649/3D reconstructionaugmented realitycomputer visiondeep learninggrapevine pruningsemantic segmentation |
| spellingShingle | Simon Haring Sophie Folawiyo Mariia Podguzova Stephan kraub Didier Stricker Vid2Cuts: A Framework for Enabling AI-Guided Grapevine Pruning IEEE Access 3D reconstruction augmented reality computer vision deep learning grapevine pruning semantic segmentation |
| title | Vid2Cuts: A Framework for Enabling AI-Guided Grapevine Pruning |
| title_full | Vid2Cuts: A Framework for Enabling AI-Guided Grapevine Pruning |
| title_fullStr | Vid2Cuts: A Framework for Enabling AI-Guided Grapevine Pruning |
| title_full_unstemmed | Vid2Cuts: A Framework for Enabling AI-Guided Grapevine Pruning |
| title_short | Vid2Cuts: A Framework for Enabling AI-Guided Grapevine Pruning |
| title_sort | vid2cuts a framework for enabling ai guided grapevine pruning |
| topic | 3D reconstruction augmented reality computer vision deep learning grapevine pruning semantic segmentation |
| url | https://ieeexplore.ieee.org/document/10384649/ |
| work_keys_str_mv | AT simonharing vid2cutsaframeworkforenablingaiguidedgrapevinepruning AT sophiefolawiyo vid2cutsaframeworkforenablingaiguidedgrapevinepruning AT mariiapodguzova vid2cutsaframeworkforenablingaiguidedgrapevinepruning AT stephankraub vid2cutsaframeworkforenablingaiguidedgrapevinepruning AT didierstricker vid2cutsaframeworkforenablingaiguidedgrapevinepruning |