Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim
In this work, we propose a method for estimating depth for an image of a monocular camera in order to avoid a collision for the autonomous flight of a drone. The highest flight speed of a drone is generally approximate 22.2 m/s, and long-distant depth information is crucial for autonomous flights si...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/6/2097 |
| _version_ | 1827625740134252544 |
|---|---|
| author | Tomoyasu Shimada Hiroki Nishikawa Xiangbo Kong Hiroyuki Tomiyama |
| author_facet | Tomoyasu Shimada Hiroki Nishikawa Xiangbo Kong Hiroyuki Tomiyama |
| author_sort | Tomoyasu Shimada |
| collection | DOAJ |
| description | In this work, we propose a method for estimating depth for an image of a monocular camera in order to avoid a collision for the autonomous flight of a drone. The highest flight speed of a drone is generally approximate 22.2 m/s, and long-distant depth information is crucial for autonomous flights since if the long-distance information is not available, the drone flying at high speeds is prone to collisions. However, long-range, measurable depth cameras are too heavy to be equipped on a drone. This work applies Pix2Pix, which is a kind of Conditional Generative Adversarial Nets (CGAN). Pix2Pix generates depth images from a monocular camera. Additionally, this work applies optical flow to enhance the accuracy of depth estimation. In this work, we propose a highly accurate depth estimation method that effectively embeds an optical flow map into a monocular image. The models are trained with taking advantage of AirSim, which is one of the flight simulators. AirSim can take both monocular and depth images over a hundred meter in the virtual environment, and our model generates a depth image that provides the long-distance information than images captured by a common depth camera. We evaluate accuracy and error of our proposed method using test images in AirSim. In addition, the proposed method is utilized for flight simulation to evaluate the effectiveness to collision avoidance. As a result, our proposed method is higher accuracy and lower error than a state of work. Moreover, our proposed method is lower collision than a state of work. |
| first_indexed | 2024-03-09T12:42:50Z |
| format | Article |
| id | doaj.art-6ce2a6591ec5488d91e054b8de9e3833 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T12:42:50Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-6ce2a6591ec5488d91e054b8de9e38332023-11-30T22:16:11ZengMDPI AGSensors1424-82202022-03-01226209710.3390/s22062097Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSimTomoyasu Shimada0Hiroki Nishikawa1Xiangbo Kong2Hiroyuki Tomiyama3Graduate School of Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, JapanGraduate School of Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, JapanGraduate School of Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, JapanGraduate School of Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, JapanIn this work, we propose a method for estimating depth for an image of a monocular camera in order to avoid a collision for the autonomous flight of a drone. The highest flight speed of a drone is generally approximate 22.2 m/s, and long-distant depth information is crucial for autonomous flights since if the long-distance information is not available, the drone flying at high speeds is prone to collisions. However, long-range, measurable depth cameras are too heavy to be equipped on a drone. This work applies Pix2Pix, which is a kind of Conditional Generative Adversarial Nets (CGAN). Pix2Pix generates depth images from a monocular camera. Additionally, this work applies optical flow to enhance the accuracy of depth estimation. In this work, we propose a highly accurate depth estimation method that effectively embeds an optical flow map into a monocular image. The models are trained with taking advantage of AirSim, which is one of the flight simulators. AirSim can take both monocular and depth images over a hundred meter in the virtual environment, and our model generates a depth image that provides the long-distance information than images captured by a common depth camera. We evaluate accuracy and error of our proposed method using test images in AirSim. In addition, the proposed method is utilized for flight simulation to evaluate the effectiveness to collision avoidance. As a result, our proposed method is higher accuracy and lower error than a state of work. Moreover, our proposed method is lower collision than a state of work.https://www.mdpi.com/1424-8220/22/6/2097depth estimationoptical flowAirSim |
| spellingShingle | Tomoyasu Shimada Hiroki Nishikawa Xiangbo Kong Hiroyuki Tomiyama Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim Sensors depth estimation optical flow AirSim |
| title | Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim |
| title_full | Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim |
| title_fullStr | Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim |
| title_full_unstemmed | Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim |
| title_short | Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim |
| title_sort | pix2pix based monocular depth estimation for drones with optical flow on airsim |
| topic | depth estimation optical flow AirSim |
| url | https://www.mdpi.com/1424-8220/22/6/2097 |
| work_keys_str_mv | AT tomoyasushimada pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim AT hirokinishikawa pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim AT xiangbokong pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim AT hiroyukitomiyama pix2pixbasedmonoculardepthestimationfordroneswithopticalflowonairsim |