Attention and anticipation in fast visual-inertial navigation
Visual attention is the cognitive process that allows humans to parse a large amount of sensory data by selecting relevant information and filtering out irrelevant stimuli. This papers develops a computational approach for visual attention in robots. We consider a Visual-Inertial Navigation (VIN) pr...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2018
|
| Online Access: | http://hdl.handle.net/1721.1/114740 https://orcid.org/0000-0003-1884-5397 https://orcid.org/0000-0002-2225-7275 |
| _version_ | 1826215578339115008 |
|---|---|
| author | Carlone, Luca Karaman, Sertac |
| author2 | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics |
| author_facet | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Carlone, Luca Karaman, Sertac |
| author_sort | Carlone, Luca |
| collection | MIT |
| description | Visual attention is the cognitive process that allows humans to parse a large amount of sensory data by selecting relevant information and filtering out irrelevant stimuli. This papers develops a computational approach for visual attention in robots. We consider a Visual-Inertial Navigation (VIN) problem in which a robot needs to estimate its state using an on-board camera and an inertial sensor. The robot can allocate limited resources to VIN, due to time and energy constraints. Therefore, we answer the following question: under limited resources, what are the most relevant visual cues to maximize the performance of visual-inertial navigation? Our approach has four key features. First, it is task-driven, in that the selection of the visual cues is guided by a metric quantifying the task performance. Second, it exploits the notion of anticipation, since it uses a simplified model for forward-simulation of robot dynamics, predicting the utility of a set of visual cues over a time horizon. Third, it is efficient and easy to implement, since it leads to a greedy algorithm for the selection of the most relevant visual cues. Fourth, it provides formal performance guarantees: we leverage submodularity to prove that the greedy selection cannot be far from the optimal (combinatorial) selection. Simulations and real experiments on agile micro aerial vehicles show that our approach leads to dramatic improvements in the VIN performance. In the easy scenarios, our approach outperforms the state of the art in terms of localization errors. In the most challenging scenarios, it enables accurate visual-inertial navigation while the state of the art fails to track robot's motion during aggressive maneuvers. |
| first_indexed | 2024-09-23T16:35:54Z |
| format | Article |
| id | mit-1721.1/114740 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T16:35:54Z |
| publishDate | 2018 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1147402022-10-02T08:22:24Z Attention and anticipation in fast visual-inertial navigation Carlone, Luca Karaman, Sertac Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Massachusetts Institute of Technology. Laboratory for Information and Decision Systems Carlone, Luca Karaman, Sertac Visual attention is the cognitive process that allows humans to parse a large amount of sensory data by selecting relevant information and filtering out irrelevant stimuli. This papers develops a computational approach for visual attention in robots. We consider a Visual-Inertial Navigation (VIN) problem in which a robot needs to estimate its state using an on-board camera and an inertial sensor. The robot can allocate limited resources to VIN, due to time and energy constraints. Therefore, we answer the following question: under limited resources, what are the most relevant visual cues to maximize the performance of visual-inertial navigation? Our approach has four key features. First, it is task-driven, in that the selection of the visual cues is guided by a metric quantifying the task performance. Second, it exploits the notion of anticipation, since it uses a simplified model for forward-simulation of robot dynamics, predicting the utility of a set of visual cues over a time horizon. Third, it is efficient and easy to implement, since it leads to a greedy algorithm for the selection of the most relevant visual cues. Fourth, it provides formal performance guarantees: we leverage submodularity to prove that the greedy selection cannot be far from the optimal (combinatorial) selection. Simulations and real experiments on agile micro aerial vehicles show that our approach leads to dramatic improvements in the VIN performance. In the easy scenarios, our approach outperforms the state of the art in terms of localization errors. In the most challenging scenarios, it enables accurate visual-inertial navigation while the state of the art fails to track robot's motion during aggressive maneuvers. 2018-04-13T22:39:06Z 2018-04-13T22:39:06Z 2017-07 2017-05 2018-03-22T16:58:27Z Article http://purl.org/eprint/type/ConferencePaper 978-1-5090-4633-1 978-1-5090-4634-8 http://hdl.handle.net/1721.1/114740 Carlone, Luca, and Sertac Karaman. “Attention and Anticipation in Fast Visual-Inertial Navigation.” 2017 IEEE International Conference on Robotics and Automation (ICRA), May 2017, Singapore, Singapore, 2017. https://orcid.org/0000-0003-1884-5397 https://orcid.org/0000-0002-2225-7275 http://dx.doi.org/10.1109/ICRA.2017.7989448 2017 IEEE International Conference on Robotics and Automation (ICRA) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) arXiv |
| spellingShingle | Carlone, Luca Karaman, Sertac Attention and anticipation in fast visual-inertial navigation |
| title | Attention and anticipation in fast visual-inertial navigation |
| title_full | Attention and anticipation in fast visual-inertial navigation |
| title_fullStr | Attention and anticipation in fast visual-inertial navigation |
| title_full_unstemmed | Attention and anticipation in fast visual-inertial navigation |
| title_short | Attention and anticipation in fast visual-inertial navigation |
| title_sort | attention and anticipation in fast visual inertial navigation |
| url | http://hdl.handle.net/1721.1/114740 https://orcid.org/0000-0003-1884-5397 https://orcid.org/0000-0002-2225-7275 |
| work_keys_str_mv | AT carloneluca attentionandanticipationinfastvisualinertialnavigation AT karamansertac attentionandanticipationinfastvisualinertialnavigation |