Towards visual ego-motion learning in robots
Many model-based Visual Odometry (VO) algorithms have been proposed in the past decade, often restricted to the type of camera optics, or the underlying motion manifold observed. We envision robots to be able to learn and perform these tasks, in a minimally supervised setting, as they gain more expe...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2019
|
| Online Access: | http://hdl.handle.net/1721.1/119893 https://orcid.org/0000-0001-7198-1772 https://orcid.org/0000-0002-8863-6550 |
| _version_ | 1811096595088801792 |
|---|---|
| author | Pillai, Sudeep Leonard, John J |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Pillai, Sudeep Leonard, John J |
| author_sort | Pillai, Sudeep |
| collection | MIT |
| description | Many model-based Visual Odometry (VO) algorithms have been proposed in the past decade, often restricted to the type of camera optics, or the underlying motion manifold observed. We envision robots to be able to learn and perform these tasks, in a minimally supervised setting, as they gain more experience. To this end, we propose a fully trainable solution to visual ego-motion estimation for varied camera optics. We propose a visual ego-motion learning architecture that maps observed optical flow vectors to an ego-motion density estimate via a Mixture Density Network (MDN). By modeling the architecture as a Conditional Variational Autoencoder (C-VAE), our model is able to provide introspective reasoning and prediction for ego-motion induced scene-flow. Additionally, our proposed model is especially amenable to bootstrapped ego-motion learning in robots where the supervision in ego-motion estimation for a particular camera sensor can be obtained from standard navigation-based sensor fusion strategies (GPS/INS and wheel-odometry fusion). Through experiments, we show the utility of our proposed approach in enabling the concept of self-supervised learning for visual ego-motion estimation in autonomous robots. |
| first_indexed | 2024-09-23T16:46:10Z |
| format | Article |
| id | mit-1721.1/119893 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T16:46:10Z |
| publishDate | 2019 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1198932022-09-29T21:22:49Z Towards visual ego-motion learning in robots Pillai, Sudeep Leonard, John J Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology. Department of Mechanical Engineering Pillai, Sudeep Leonard, John J Many model-based Visual Odometry (VO) algorithms have been proposed in the past decade, often restricted to the type of camera optics, or the underlying motion manifold observed. We envision robots to be able to learn and perform these tasks, in a minimally supervised setting, as they gain more experience. To this end, we propose a fully trainable solution to visual ego-motion estimation for varied camera optics. We propose a visual ego-motion learning architecture that maps observed optical flow vectors to an ego-motion density estimate via a Mixture Density Network (MDN). By modeling the architecture as a Conditional Variational Autoencoder (C-VAE), our model is able to provide introspective reasoning and prediction for ego-motion induced scene-flow. Additionally, our proposed model is especially amenable to bootstrapped ego-motion learning in robots where the supervision in ego-motion estimation for a particular camera sensor can be obtained from standard navigation-based sensor fusion strategies (GPS/INS and wheel-odometry fusion). Through experiments, we show the utility of our proposed approach in enabling the concept of self-supervised learning for visual ego-motion estimation in autonomous robots. United States. Office of Naval Research (Grant N00014-11-1-0688) United States. Office of Naval Research (Grant N00014- 13-1-0588) National Science Foundation (U.S.) (Grant IIS-1318392) 2019-01-09T19:09:47Z 2019-01-09T19:09:47Z 2017-12 2017-09 2018-12-12T15:08:20Z Article http://purl.org/eprint/type/ConferencePaper 978-1-5386-2682-5 http://hdl.handle.net/1721.1/119893 Pillai, Sudeep, and John J. Leonard. “Towards Visual Ego-Motion Learning in Robots.” 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 24-28 September, 2017, Vancouver, BC, Canada, IEEE, 2017. © 2017 IEEE https://orcid.org/0000-0001-7198-1772 https://orcid.org/0000-0002-8863-6550 http://dx.doi.org/10.1109/IROS.2017.8206441 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) arXiv |
| spellingShingle | Pillai, Sudeep Leonard, John J Towards visual ego-motion learning in robots |
| title | Towards visual ego-motion learning in robots |
| title_full | Towards visual ego-motion learning in robots |
| title_fullStr | Towards visual ego-motion learning in robots |
| title_full_unstemmed | Towards visual ego-motion learning in robots |
| title_short | Towards visual ego-motion learning in robots |
| title_sort | towards visual ego motion learning in robots |
| url | http://hdl.handle.net/1721.1/119893 https://orcid.org/0000-0001-7198-1772 https://orcid.org/0000-0002-8863-6550 |
| work_keys_str_mv | AT pillaisudeep towardsvisualegomotionlearninginrobots AT leonardjohnj towardsvisualegomotionlearninginrobots |