Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones
This paper presents Navion, an energy-efficient accelerator for visual-inertial odometry (VIO) that enables autonomous navigation of miniaturized robots (e.g., nano drones), and virtual/augmented reality on portable devices. The chip uses inertial measurements and mono/stereo images to estimate the...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2018
|
| Online Access: | http://hdl.handle.net/1721.1/116810 https://orcid.org/0000-0002-0376-4220 https://orcid.org/0000-0002-0619-8199 https://orcid.org/0000-0003-1884-5397 https://orcid.org/0000-0002-2225-7275 https://orcid.org/0000-0003-4841-3990 |
| _version_ | 1826189295602368512 |
|---|---|
| author | Suleiman, Amr AbdulZahir Zhang, Zhengdong Carlone, Luca Karaman, Sertac Sze, Vivienne |
| author2 | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics |
| author_facet | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Suleiman, Amr AbdulZahir Zhang, Zhengdong Carlone, Luca Karaman, Sertac Sze, Vivienne |
| author_sort | Suleiman, Amr AbdulZahir |
| collection | MIT |
| description | This paper presents Navion, an energy-efficient accelerator for visual-inertial odometry (VIO) that enables autonomous navigation of miniaturized robots (e.g., nano drones), and virtual/augmented reality on portable devices. The chip uses inertial measurements and mono/stereo images to estimate the
drone’s trajectory and a 3D map of the environment. This estimate is obtained by running a state-of-the-art algorithm based on non-linear factor graph optimization, which requires large irregularly structured memories and heterogeneous computation flow. To reduce the energy consumption and footprint, the entire VIO system is fully integrated on chip to eliminate costly off-chip processing and storage. This work uses compression and exploits
both structured and unstructured sparsity to reduce on-chip memory size by 4.1x. Parallelism is used under tight area constraints to increase throughput by 43%. The chip is fabricated in 65nm CMOS, and can process 752x480 stereo images at up to 171 fps and inertial measurements at up to 52 kHz, while consuming an average of 24mW. The chip is configurable to maximize accuracy, throughput and energy-efficiency across different environments. To the best of our knowledge, this is the first fully integrated VIO system in an ASIC. Keywords: VIO, localization, mapping, nano drones, navigation |
| first_indexed | 2024-09-23T08:12:51Z |
| format | Article |
| id | mit-1721.1/116810 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T08:12:51Z |
| publishDate | 2018 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1168102022-09-30T08:20:21Z Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones Suleiman, Amr AbdulZahir Zhang, Zhengdong Carlone, Luca Karaman, Sertac Sze, Vivienne Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Sze, Vivienne Suleiman, Amr AbdulZahir Zhang, Zhengdong Carlone, Luca Karaman, Sertac Sze, Vivienne This paper presents Navion, an energy-efficient accelerator for visual-inertial odometry (VIO) that enables autonomous navigation of miniaturized robots (e.g., nano drones), and virtual/augmented reality on portable devices. The chip uses inertial measurements and mono/stereo images to estimate the drone’s trajectory and a 3D map of the environment. This estimate is obtained by running a state-of-the-art algorithm based on non-linear factor graph optimization, which requires large irregularly structured memories and heterogeneous computation flow. To reduce the energy consumption and footprint, the entire VIO system is fully integrated on chip to eliminate costly off-chip processing and storage. This work uses compression and exploits both structured and unstructured sparsity to reduce on-chip memory size by 4.1x. Parallelism is used under tight area constraints to increase throughput by 43%. The chip is fabricated in 65nm CMOS, and can process 752x480 stereo images at up to 171 fps and inertial measurements at up to 52 kHz, while consuming an average of 24mW. The chip is configurable to maximize accuracy, throughput and energy-efficiency across different environments. To the best of our knowledge, this is the first fully integrated VIO system in an ASIC. Keywords: VIO, localization, mapping, nano drones, navigation National Science Foundation (U.S.) (CAREER Grant 1350685) United States. Air Force. Office of Scientific Research. Young Investigator Program (FA9550-16-1-0228) 2018-07-06T13:15:47Z 2018-07-06T13:15:47Z 2018-06 Article http://purl.org/eprint/type/ConferencePaper http://hdl.handle.net/1721.1/116810 Suleiman, Amr et al. "Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones." IEEE Symposium on VLSI Circuits, 18-22 June, 2018, Honolulu, Hawaii, IEEE, 2018. https://orcid.org/0000-0002-0376-4220 https://orcid.org/0000-0002-0619-8199 https://orcid.org/0000-0003-1884-5397 https://orcid.org/0000-0002-2225-7275 https://orcid.org/0000-0003-4841-3990 en_US http://vlsisymposium.org/wp-content/uploads/2018/06/18-glance-web-combined.pdf IEEE Symposium on VLSI Circuits Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) Prof. Sze |
| spellingShingle | Suleiman, Amr AbdulZahir Zhang, Zhengdong Carlone, Luca Karaman, Sertac Sze, Vivienne Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title | Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title_full | Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title_fullStr | Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title_full_unstemmed | Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title_short | Navion: A Fully Integrated Energy-Efficient Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title_sort | navion a fully integrated energy efficient visual inertial odometry accelerator for autonomous navigation of nano drones |
| url | http://hdl.handle.net/1721.1/116810 https://orcid.org/0000-0002-0376-4220 https://orcid.org/0000-0002-0619-8199 https://orcid.org/0000-0003-1884-5397 https://orcid.org/0000-0002-2225-7275 https://orcid.org/0000-0003-4841-3990 |
| work_keys_str_mv | AT suleimanamrabdulzahir navionafullyintegratedenergyefficientvisualinertialodometryacceleratorforautonomousnavigationofnanodrones AT zhangzhengdong navionafullyintegratedenergyefficientvisualinertialodometryacceleratorforautonomousnavigationofnanodrones AT carloneluca navionafullyintegratedenergyefficientvisualinertialodometryacceleratorforautonomousnavigationofnanodrones AT karamansertac navionafullyintegratedenergyefficientvisualinertialodometryacceleratorforautonomousnavigationofnanodrones AT szevivienne navionafullyintegratedenergyefficientvisualinertialodometryacceleratorforautonomousnavigationofnanodrones |