Photorealistic sensor simulation for perception-driven robotics using virtual reality
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | eng |
| Published: |
Massachusetts Institute of Technology
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/1721.1/124244 |
| _version_ | 1826200284915826688 |
|---|---|
| author | Guerra, Winter Joseph. |
| author2 | Sertac Karaman. |
| author_facet | Sertac Karaman. Guerra, Winter Joseph. |
| author_sort | Guerra, Winter Joseph. |
| collection | MIT |
| description | This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. |
| first_indexed | 2024-09-23T11:34:08Z |
| format | Thesis |
| id | mit-1721.1/124244 |
| institution | Massachusetts Institute of Technology |
| language | eng |
| last_indexed | 2024-09-23T11:34:08Z |
| publishDate | 2020 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1242442020-03-25T03:26:11Z Photorealistic sensor simulation for perception-driven robotics using virtual reality Guerra, Winter Joseph. Sertac Karaman. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Electrical Engineering and Computer Science. This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019 Cataloged from student-submitted PDF version of thesis. Includes bibliographical references (pages 103-112). In recent years, intensive research has centered around using small, perception-driven robotic systems (e.g. quadrotor vehicles) for complex tasks at operational speeds. Although much progress has been made towards that end in the fields of online-planning, fast-perception, and agile-control, most robotic systems are still confined to controlled laboratory settings due to cost, safety, and repeatability. In this thesis, we introduce a few novel contributions that we believe could assist the greater robotics community to bring their robotics systems out of the lab and into the real world. First, we introduce FlightGoggles, a photorealistic sensor simulator for perception-driven robotic vehicles. FlightGoggles provides photorealistic exteroceptive sensor simulation using graphics assets generated with photogrammetry and provides the ability to combine synthetic exteroceptive measurements generated in silico in real time and vehicle dynamics and proprioceptive measurements generated in motio by vehicle(s) in flight in a motion-capture facility. Second, we present The Blackbird Dataset, a large-scale dataset for UAV perception in aggressive flight containing over 10 hours of flight data across 171 flights at velocities up to 13.8ms⁻¹ in 5 environments with some dynamic elements. Third, we introduce a virtual reality framework for FlightGoggles that enables multi-agent or robot-human interaction in a safe manner by superimposing position data from multiple motion capture spaces into a unified virtual reality environment. Fourth, we propose an extension of FlightGoggles using augmented reality for aircraft-in-the-loop experiments that aims to aid sim-to-real transfer from simulated to real-world cameras. Lastly, we study applications of FlightGoggles in the greater robotics community through the AlphaPilot autonomous drone racing challenge and survey approaches and results from the top AlphaPilot teams, which may be of independent interest. by Winter Joseph Guerra. M. Eng. M.Eng. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science 2020-03-24T15:36:04Z 2020-03-24T15:36:04Z 2019 2019 Thesis https://hdl.handle.net/1721.1/124244 1145019349 eng MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission. http://dspace.mit.edu/handle/1721.1/7582 112 pages application/pdf Massachusetts Institute of Technology |
| spellingShingle | Electrical Engineering and Computer Science. Guerra, Winter Joseph. Photorealistic sensor simulation for perception-driven robotics using virtual reality |
| title | Photorealistic sensor simulation for perception-driven robotics using virtual reality |
| title_full | Photorealistic sensor simulation for perception-driven robotics using virtual reality |
| title_fullStr | Photorealistic sensor simulation for perception-driven robotics using virtual reality |
| title_full_unstemmed | Photorealistic sensor simulation for perception-driven robotics using virtual reality |
| title_short | Photorealistic sensor simulation for perception-driven robotics using virtual reality |
| title_sort | photorealistic sensor simulation for perception driven robotics using virtual reality |
| topic | Electrical Engineering and Computer Science. |
| url | https://hdl.handle.net/1721.1/124244 |
| work_keys_str_mv | AT guerrawinterjoseph photorealisticsensorsimulationforperceptiondrivenroboticsusingvirtualreality |