Augmented Reality Driving Using Semantic Geo-Registration
We propose a new approach that utilizes semantic information to register 2D monocular video frames to the world using 3D geo-referenced data, for augmented reality driving applications. The geo-registration process uses our predicted vehicle pose to generate a rendered depth map for each frame, allo...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2018
|
| Online Access: | http://hdl.handle.net/1721.1/119669 https://orcid.org/0000-0001-9741-1102 |
| _version_ | 1811083577166659584 |
|---|---|
| author | Villamil, Ryan Samarasekera, Supun Chiu, Han-Pang Murali, Varun Munoz Kessler, Rodrigo Arturo Kumar, Rakesh |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Villamil, Ryan Samarasekera, Supun Chiu, Han-Pang Murali, Varun Munoz Kessler, Rodrigo Arturo Kumar, Rakesh |
| author_sort | Villamil, Ryan |
| collection | MIT |
| description | We propose a new approach that utilizes semantic information to register 2D monocular video frames to the world using 3D geo-referenced data, for augmented reality driving applications. The geo-registration process uses our predicted vehicle pose to generate a rendered depth map for each frame, allowing 3D graphics to be convincingly blended with the real world view. We also estimate absolute depth values for dynamic objects, up to 120 meters, based on the rendered depth map and update the rendered depth map to reflect scene changes over time. This process also creates opportunistic global heading measurements, which are fused with other sensors, to improve estimates of the 6 degrees-of-freedom global pose of the vehicle over state-of-the-art outdoor augmented reality systems [5, 18, 19]. We evaluate the navigation accuracy and depth map quality of our system on a driving vehicle within various large-scale environments for producing realistic augmentations. |
| first_indexed | 2024-09-23T12:35:17Z |
| format | Article |
| id | mit-1721.1/119669 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T12:35:17Z |
| publishDate | 2018 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1196692022-10-01T09:56:07Z Augmented Reality Driving Using Semantic Geo-Registration Villamil, Ryan Samarasekera, Supun Chiu, Han-Pang Murali, Varun Munoz Kessler, Rodrigo Arturo Kumar, Rakesh Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Varun Murali Chiu, Han-Pang Murali, Varun Munoz Kessler, Rodrigo Arturo Kumar, Rakesh We propose a new approach that utilizes semantic information to register 2D monocular video frames to the world using 3D geo-referenced data, for augmented reality driving applications. The geo-registration process uses our predicted vehicle pose to generate a rendered depth map for each frame, allowing 3D graphics to be convincingly blended with the real world view. We also estimate absolute depth values for dynamic objects, up to 120 meters, based on the rendered depth map and update the rendered depth map to reflect scene changes over time. This process also creates opportunistic global heading measurements, which are fused with other sensors, to improve estimates of the 6 degrees-of-freedom global pose of the vehicle over state-of-the-art outdoor augmented reality systems [5, 18, 19]. We evaluate the navigation accuracy and depth map quality of our system on a driving vehicle within various large-scale environments for producing realistic augmentations. 2018-12-18T13:44:32Z 2018-12-18T13:44:32Z 2018-03 2018-12-04T17:44:51Z Article http://purl.org/eprint/type/ConferencePaper 978-1-5386-3365-6 http://hdl.handle.net/1721.1/119669 Chiu, Han-Pang, Varun Murali, Ryan Villamil, G. Drew Kessler, Supun Samarasekera, and Rakesh Kumar. “Augmented Reality Driving Using Semantic Geo-Registration.” 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (March 2018). https://orcid.org/0000-0001-9741-1102 http://dx.doi.org/10.1109/VR.2018.8447560 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) Varun Murali |
| spellingShingle | Villamil, Ryan Samarasekera, Supun Chiu, Han-Pang Murali, Varun Munoz Kessler, Rodrigo Arturo Kumar, Rakesh Augmented Reality Driving Using Semantic Geo-Registration |
| title | Augmented Reality Driving Using Semantic Geo-Registration |
| title_full | Augmented Reality Driving Using Semantic Geo-Registration |
| title_fullStr | Augmented Reality Driving Using Semantic Geo-Registration |
| title_full_unstemmed | Augmented Reality Driving Using Semantic Geo-Registration |
| title_short | Augmented Reality Driving Using Semantic Geo-Registration |
| title_sort | augmented reality driving using semantic geo registration |
| url | http://hdl.handle.net/1721.1/119669 https://orcid.org/0000-0001-9741-1102 |
| work_keys_str_mv | AT villamilryan augmentedrealitydrivingusingsemanticgeoregistration AT samarasekerasupun augmentedrealitydrivingusingsemanticgeoregistration AT chiuhanpang augmentedrealitydrivingusingsemanticgeoregistration AT muralivarun augmentedrealitydrivingusingsemanticgeoregistration AT munozkesslerrodrigoarturo augmentedrealitydrivingusingsemanticgeoregistration AT kumarrakesh augmentedrealitydrivingusingsemanticgeoregistration |