Autonomous Visual Navigation for a Flower Pollination Drone
In this paper, we present the development of a visual navigation capability for a small drone enabling it to autonomously approach flowers. This is a very important step towards the development of a fully autonomous flower pollinating nanodrone. The drone we developed is totally autonomous and relie...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-05-01
|
| Series: | Machines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-1702/10/5/364 |
| _version_ | 1797498398411063296 |
|---|---|
| author | Dries Hulens Wiebe Van Ranst Ying Cao Toon Goedemé |
| author_facet | Dries Hulens Wiebe Van Ranst Ying Cao Toon Goedemé |
| author_sort | Dries Hulens |
| collection | DOAJ |
| description | In this paper, we present the development of a visual navigation capability for a small drone enabling it to autonomously approach flowers. This is a very important step towards the development of a fully autonomous flower pollinating nanodrone. The drone we developed is totally autonomous and relies for its navigation on a small on-board color camera, complemented with one simple ToF distance sensor, to detect and approach the flower. The proposed solution uses a DJI Tello drone carrying a Maix Bit processing board capable of running all deep-learning-based image processing and navigation algorithms on-board. We developed a two-stage visual servoing algorithm that first uses a highly optimized object detection CNN to localize the flowers and fly towards it. The second phase, approaching the flower, is implemented by a direct visual steering CNN. This enables the drone to detect any flower in the neighborhood, steer the drone towards the flower and make the drone’s pollinating rod touch the flower. We trained all deep learning models based on an artificial dataset with a mix of images of real flowers, artificial (synthetic) flowers and virtually rendered flowers. Our experiments demonstrate that the approach is technically feasible. The drone is able to detect, approach and touch the flowers totally autonomously. Our 10 cm sized prototype is trained on sunflowers, but the methodology presented in this paper can be retrained for any flower type. |
| first_indexed | 2024-03-10T03:32:53Z |
| format | Article |
| id | doaj.art-75de071ed6e24612a1a999c37bca1842 |
| institution | Directory Open Access Journal |
| issn | 2075-1702 |
| language | English |
| last_indexed | 2024-03-10T03:32:53Z |
| publishDate | 2022-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machines |
| spelling | doaj.art-75de071ed6e24612a1a999c37bca18422023-11-23T11:52:54ZengMDPI AGMachines2075-17022022-05-0110536410.3390/machines10050364Autonomous Visual Navigation for a Flower Pollination DroneDries Hulens0Wiebe Van Ranst1Ying Cao2Toon Goedemé3EAVISE (Embedded and Artificially intelligent VISion Engineering), KU Leuven, 2860 Sint-Katelijne-Waver, BelgiumEAVISE (Embedded and Artificially intelligent VISion Engineering), KU Leuven, 2860 Sint-Katelijne-Waver, BelgiumMagics Technologies NV, 2440 Geel, BelgiumEAVISE (Embedded and Artificially intelligent VISion Engineering), KU Leuven, 2860 Sint-Katelijne-Waver, BelgiumIn this paper, we present the development of a visual navigation capability for a small drone enabling it to autonomously approach flowers. This is a very important step towards the development of a fully autonomous flower pollinating nanodrone. The drone we developed is totally autonomous and relies for its navigation on a small on-board color camera, complemented with one simple ToF distance sensor, to detect and approach the flower. The proposed solution uses a DJI Tello drone carrying a Maix Bit processing board capable of running all deep-learning-based image processing and navigation algorithms on-board. We developed a two-stage visual servoing algorithm that first uses a highly optimized object detection CNN to localize the flowers and fly towards it. The second phase, approaching the flower, is implemented by a direct visual steering CNN. This enables the drone to detect any flower in the neighborhood, steer the drone towards the flower and make the drone’s pollinating rod touch the flower. We trained all deep learning models based on an artificial dataset with a mix of images of real flowers, artificial (synthetic) flowers and virtually rendered flowers. Our experiments demonstrate that the approach is technically feasible. The drone is able to detect, approach and touch the flowers totally autonomously. Our 10 cm sized prototype is trained on sunflowers, but the methodology presented in this paper can be retrained for any flower type.https://www.mdpi.com/2075-1702/10/5/364pollination dronevisual servoingtwo-stage approachneural network |
| spellingShingle | Dries Hulens Wiebe Van Ranst Ying Cao Toon Goedemé Autonomous Visual Navigation for a Flower Pollination Drone Machines pollination drone visual servoing two-stage approach neural network |
| title | Autonomous Visual Navigation for a Flower Pollination Drone |
| title_full | Autonomous Visual Navigation for a Flower Pollination Drone |
| title_fullStr | Autonomous Visual Navigation for a Flower Pollination Drone |
| title_full_unstemmed | Autonomous Visual Navigation for a Flower Pollination Drone |
| title_short | Autonomous Visual Navigation for a Flower Pollination Drone |
| title_sort | autonomous visual navigation for a flower pollination drone |
| topic | pollination drone visual servoing two-stage approach neural network |
| url | https://www.mdpi.com/2075-1702/10/5/364 |
| work_keys_str_mv | AT drieshulens autonomousvisualnavigationforaflowerpollinationdrone AT wiebevanranst autonomousvisualnavigationforaflowerpollinationdrone AT yingcao autonomousvisualnavigationforaflowerpollinationdrone AT toongoedeme autonomousvisualnavigationforaflowerpollinationdrone |