Navion: A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones
© 1966-2012 IEEE. This paper presents Navion, an energy-efficient accelerator for visual-inertial odometry (VIO) that enables autonomous navigation of miniaturized robots (e.g., nano drones), and virtual reality (VR)/augmented reality (AR) on portable devices. The chip uses inertial measurements and...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/137120 |
| _version_ | 1826207669218705408 |
|---|---|
| author | Suleiman, Amr Zhang, Zhengdong Carlone, Luca Karaman, Sertac Sze, Vivienne |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Suleiman, Amr Zhang, Zhengdong Carlone, Luca Karaman, Sertac Sze, Vivienne |
| author_sort | Suleiman, Amr |
| collection | MIT |
| description | © 1966-2012 IEEE. This paper presents Navion, an energy-efficient accelerator for visual-inertial odometry (VIO) that enables autonomous navigation of miniaturized robots (e.g., nano drones), and virtual reality (VR)/augmented reality (AR) on portable devices. The chip uses inertial measurements and mono/stereo images to estimate the drone's trajectory and a 3-D map of the environment. This estimate is obtained by running a state-of-the-art VIO algorithm based on non-linear factor graph optimization, which requires large irregularly structured memories and heterogeneous computation flow. To reduce the energy consumption and footprint, the entire VIO system is fully integrated on-chip to eliminate costly off-chip processing and storage. This paper uses compression and exploits both structured and unstructured sparsity to reduce on-chip memory size by 4.1 ×. Parallelism is used under tight area constraints to increase throughput by 43%. The chip is fabricated in 65-nm CMOS and can process 752 × 480 stereo images from EuRoC data set in real time at 20 frames per second (fps) consuming only an average power of 2 mW. At its peak performance, Navion can process stereo images at up to 171 fps and inertial measurements at up to 52 kHz, while consuming an average of 24 mW. The chip is configurable to maximize accuracy, throughput, and energy-efficiency tradeoffs and to adapt to different environments. To the best of our knowledge, this is the first fully integrated VIO system in an application-specified integrated circuit (ASIC). |
| first_indexed | 2024-09-23T13:53:06Z |
| format | Article |
| id | mit-1721.1/137120 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T13:53:06Z |
| publishDate | 2021 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1371202023-06-21T18:47:36Z Navion: A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones Suleiman, Amr Zhang, Zhengdong Carlone, Luca Karaman, Sertac Sze, Vivienne Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Department of Aeronautics and Astronautics © 1966-2012 IEEE. This paper presents Navion, an energy-efficient accelerator for visual-inertial odometry (VIO) that enables autonomous navigation of miniaturized robots (e.g., nano drones), and virtual reality (VR)/augmented reality (AR) on portable devices. The chip uses inertial measurements and mono/stereo images to estimate the drone's trajectory and a 3-D map of the environment. This estimate is obtained by running a state-of-the-art VIO algorithm based on non-linear factor graph optimization, which requires large irregularly structured memories and heterogeneous computation flow. To reduce the energy consumption and footprint, the entire VIO system is fully integrated on-chip to eliminate costly off-chip processing and storage. This paper uses compression and exploits both structured and unstructured sparsity to reduce on-chip memory size by 4.1 ×. Parallelism is used under tight area constraints to increase throughput by 43%. The chip is fabricated in 65-nm CMOS and can process 752 × 480 stereo images from EuRoC data set in real time at 20 frames per second (fps) consuming only an average power of 2 mW. At its peak performance, Navion can process stereo images at up to 171 fps and inertial measurements at up to 52 kHz, while consuming an average of 24 mW. The chip is configurable to maximize accuracy, throughput, and energy-efficiency tradeoffs and to adapt to different environments. To the best of our knowledge, this is the first fully integrated VIO system in an application-specified integrated circuit (ASIC). 2021-11-02T17:14:36Z 2021-11-02T17:14:36Z 2019-04 2019-07-03T16:37:44Z Article http://purl.org/eprint/type/JournalArticle 0018-9200 1558-173X https://hdl.handle.net/1721.1/137120 Suleiman, Amr, Zhang, Zhengdong, Carlone, Luca, Karaman, Sertac and Sze, Vivienne. 2019. "Navion: A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones." 54 (4). en 10.1109/jssc.2018.2886342 Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) arXiv |
| spellingShingle | Suleiman, Amr Zhang, Zhengdong Carlone, Luca Karaman, Sertac Sze, Vivienne Navion: A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title | Navion: A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title_full | Navion: A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title_fullStr | Navion: A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title_full_unstemmed | Navion: A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title_short | Navion: A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones |
| title_sort | navion a 2 mw fully integrated real time visual inertial odometry accelerator for autonomous navigation of nano drones |
| url | https://hdl.handle.net/1721.1/137120 |
| work_keys_str_mv | AT suleimanamr naviona2mwfullyintegratedrealtimevisualinertialodometryacceleratorforautonomousnavigationofnanodrones AT zhangzhengdong naviona2mwfullyintegratedrealtimevisualinertialodometryacceleratorforautonomousnavigationofnanodrones AT carloneluca naviona2mwfullyintegratedrealtimevisualinertialodometryacceleratorforautonomousnavigationofnanodrones AT karamansertac naviona2mwfullyintegratedrealtimevisualinertialodometryacceleratorforautonomousnavigationofnanodrones AT szevivienne naviona2mwfullyintegratedrealtimevisualinertialodometryacceleratorforautonomousnavigationofnanodrones |