FastDepth: Fast monocular depth estimation on embedded systems
Depth sensing is a critical function for robotic tasks such as localization, mapping and obstacle detection. There has been a significant and growing interest in depth estimation from a single RGB image, due to the relatively low cost and size of monocular cameras. However, state-of-the-art single-v...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020
|
| Online Access: | https://hdl.handle.net/1721.1/126546 |
| _version_ | 1811087480341921792 |
|---|---|
| author | Wofk, Diana Ma, Fangchang Yang, Tien-Ju Karaman, Sertac Sze, Vivienne |
| author2 | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics |
| author_facet | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Wofk, Diana Ma, Fangchang Yang, Tien-Ju Karaman, Sertac Sze, Vivienne |
| author_sort | Wofk, Diana |
| collection | MIT |
| description | Depth sensing is a critical function for robotic tasks such as localization, mapping and obstacle detection. There has been a significant and growing interest in depth estimation from a single RGB image, due to the relatively low cost and size of monocular cameras. However, state-of-the-art single-view depth estimation algorithms are based on fairly complex deep neural networks that are too slow for real-time inference on an embedded platform, for instance, mounted on a micro aerial vehicle. In this paper, we address the problem of fast depth estimation on embedded systems. We propose an efficient and lightweight encoder-decoder network architecture and apply network pruning to further reduce computational complexity and latency. In particular, we focus on the design of a low-latency decoder. Our methodology demonstrates that it is possible to achieve similar accuracy as prior work on depth estimation, but at inference speeds that are an order of magnitude faster. Our proposed network, FastDepth, runs at 178 fps on an NVIDIA Jetson TX2 GPU and at 27 fps when using only the TX2 CPU, with active power consumption under 10 W. FastDepth achieves close to state-of-the-art accuracy on the NYU Depth v2 dataset. To the best of the authors' knowledge, this paper demonstrates real-time monocular depth estimation using a deep neural network with the lowest latency and highest throughput on an embedded platform that can be carried by a micro aerial vehicle. |
| first_indexed | 2024-09-23T13:46:46Z |
| format | Article |
| id | mit-1721.1/126546 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T13:46:46Z |
| publishDate | 2020 |
| publisher | IEEE |
| record_format | dspace |
| spelling | mit-1721.1/1265462022-10-01T17:06:34Z FastDepth: Fast monocular depth estimation on embedded systems Wofk, Diana Ma, Fangchang Yang, Tien-Ju Karaman, Sertac Sze, Vivienne Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Microsystems Technology Laboratories Depth sensing is a critical function for robotic tasks such as localization, mapping and obstacle detection. There has been a significant and growing interest in depth estimation from a single RGB image, due to the relatively low cost and size of monocular cameras. However, state-of-the-art single-view depth estimation algorithms are based on fairly complex deep neural networks that are too slow for real-time inference on an embedded platform, for instance, mounted on a micro aerial vehicle. In this paper, we address the problem of fast depth estimation on embedded systems. We propose an efficient and lightweight encoder-decoder network architecture and apply network pruning to further reduce computational complexity and latency. In particular, we focus on the design of a low-latency decoder. Our methodology demonstrates that it is possible to achieve similar accuracy as prior work on depth estimation, but at inference speeds that are an order of magnitude faster. Our proposed network, FastDepth, runs at 178 fps on an NVIDIA Jetson TX2 GPU and at 27 fps when using only the TX2 CPU, with active power consumption under 10 W. FastDepth achieves close to state-of-the-art accuracy on the NYU Depth v2 dataset. To the best of the authors' knowledge, this paper demonstrates real-time monocular depth estimation using a deep neural network with the lowest latency and highest throughput on an embedded platform that can be carried by a micro aerial vehicle. 2020-08-12T17:45:42Z 2020-08-12T17:45:42Z 2019-05 2019-10-29T15:58:10Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/126546 Wofk, Diana et al. “FastDepth: Fast monocular depth estimation on embedded systems.” Paper presented at the 2019 International Conference on Robotics and Automation (ICRA), Montreal, Canada, May 20-24, 2019, © 2019 The Author(s) en 10.1109/ICRA.2019.8794182 2019 International Conference on Robotics and Automation (ICRA) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf IEEE arXiv |
| spellingShingle | Wofk, Diana Ma, Fangchang Yang, Tien-Ju Karaman, Sertac Sze, Vivienne FastDepth: Fast monocular depth estimation on embedded systems |
| title | FastDepth: Fast monocular depth estimation on embedded systems |
| title_full | FastDepth: Fast monocular depth estimation on embedded systems |
| title_fullStr | FastDepth: Fast monocular depth estimation on embedded systems |
| title_full_unstemmed | FastDepth: Fast monocular depth estimation on embedded systems |
| title_short | FastDepth: Fast monocular depth estimation on embedded systems |
| title_sort | fastdepth fast monocular depth estimation on embedded systems |
| url | https://hdl.handle.net/1721.1/126546 |
| work_keys_str_mv | AT wofkdiana fastdepthfastmonoculardepthestimationonembeddedsystems AT mafangchang fastdepthfastmonoculardepthestimationonembeddedsystems AT yangtienju fastdepthfastmonoculardepthestimationonembeddedsystems AT karamansertac fastdepthfastmonoculardepthestimationonembeddedsystems AT szevivienne fastdepthfastmonoculardepthestimationonembeddedsystems |