Where to go: Interpreting natural directions using global inference
An important component of human-robot interaction is that people need to be able to instruct robots to move to other locations using naturally given directions. When giving directions, people often make mistakes such as labelling errors (e.g., left vs. right) and errors of omission (skipping importa...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Institute of Electrical and Electronics Engineers
2011
|
| Online Access: | http://hdl.handle.net/1721.1/66168 https://orcid.org/0000-0002-8293-0492 |
| _version_ | 1826188855869440000 |
|---|---|
| author | Wei, Yuan Brunskill, Emma Kollar, Thomas Fleming Roy, Nicholas |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Wei, Yuan Brunskill, Emma Kollar, Thomas Fleming Roy, Nicholas |
| author_sort | Wei, Yuan |
| collection | MIT |
| description | An important component of human-robot interaction is that people need to be able to instruct robots to move to other locations using naturally given directions. When giving directions, people often make mistakes such as labelling errors (e.g., left vs. right) and errors of omission (skipping important decision points in a sequence). Furthermore, people often use multiple levels of granularity in specifying directions, referring to locations using single object landmarks, multiple landmarks in a given location, or identifying large regions as a single location. The challenge is to identify the correct path to a destination from a sequence of noisy, possibly erroneous directions. In our work we cast this problem as probabilistic inference: given a set of directions, an agent should automatically find the path with the geometry and physical appearance to maximize the likelihood of those directions. We use a specific variant of a Markov Random Field (MRF) to represent our model, and gather multi-granularity representation information using existing large tagged datasets. On a dataset of route directions collected in a large third floor university building, we found that our algorithm correctly inferred the true final destination in 47 out of the 55 cases successfully followed by humans volunteers. These results suggest that our algorithm is performing well relative to human users. In the future this work will be included in a broader system for autonomously constructing environmental representations that support natural human-robot interaction for direction giving. |
| first_indexed | 2024-09-23T08:05:45Z |
| format | Article |
| id | mit-1721.1/66168 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T08:05:45Z |
| publishDate | 2011 |
| publisher | Institute of Electrical and Electronics Engineers |
| record_format | dspace |
| spelling | mit-1721.1/661682022-09-30T07:29:04Z Where to go: Interpreting natural directions using global inference Wei, Yuan Brunskill, Emma Kollar, Thomas Fleming Roy, Nicholas Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Roy, Nicholas Wei, Yuan Brunskill, Emma Kollar, Thomas Fleming Roy, Nicholas An important component of human-robot interaction is that people need to be able to instruct robots to move to other locations using naturally given directions. When giving directions, people often make mistakes such as labelling errors (e.g., left vs. right) and errors of omission (skipping important decision points in a sequence). Furthermore, people often use multiple levels of granularity in specifying directions, referring to locations using single object landmarks, multiple landmarks in a given location, or identifying large regions as a single location. The challenge is to identify the correct path to a destination from a sequence of noisy, possibly erroneous directions. In our work we cast this problem as probabilistic inference: given a set of directions, an agent should automatically find the path with the geometry and physical appearance to maximize the likelihood of those directions. We use a specific variant of a Markov Random Field (MRF) to represent our model, and gather multi-granularity representation information using existing large tagged datasets. On a dataset of route directions collected in a large third floor university building, we found that our algorithm correctly inferred the true final destination in 47 out of the 55 cases successfully followed by humans volunteers. These results suggest that our algorithm is performing well relative to human users. In the future this work will be included in a broader system for autonomously constructing environmental representations that support natural human-robot interaction for direction giving. United States. Air Force Office of Scientific Research (Agile Robotics project, contract number 7000038334) National Science Foundation (U.S.) (NSF Division of Information and Intelligent Systems under grant # 0546467) Massachusetts Institute of Technology (Hugh Hampton Young Memorial Fund Fellowship) United States. Office of Naval Research (MURI N00014-07-1-0749) 2011-10-03T20:28:33Z 2011-10-03T20:28:33Z 2009-05 Article http://purl.org/eprint/type/ConferencePaper 978-1-4244-2788-8 1050-4729 http://hdl.handle.net/1721.1/66168 Yuan Wei et al. “Where to go: Interpreting natural directions using global inference.” Robotics and Automation, 2009. ICRA’09. IEEE International Conference on. 2009. 3761-3767. © 2009 IEEE. https://orcid.org/0000-0002-8293-0492 en_US http://dx.doi.org/10.1109/ROBOT.2009.5152775 IEEE International Conference on Robotics and Automation, 2009. ICRA '09 Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf Institute of Electrical and Electronics Engineers IEEE |
| spellingShingle | Wei, Yuan Brunskill, Emma Kollar, Thomas Fleming Roy, Nicholas Where to go: Interpreting natural directions using global inference |
| title | Where to go: Interpreting natural directions using global inference |
| title_full | Where to go: Interpreting natural directions using global inference |
| title_fullStr | Where to go: Interpreting natural directions using global inference |
| title_full_unstemmed | Where to go: Interpreting natural directions using global inference |
| title_short | Where to go: Interpreting natural directions using global inference |
| title_sort | where to go interpreting natural directions using global inference |
| url | http://hdl.handle.net/1721.1/66168 https://orcid.org/0000-0002-8293-0492 |
| work_keys_str_mv | AT weiyuan wheretogointerpretingnaturaldirectionsusingglobalinference AT brunskillemma wheretogointerpretingnaturaldirectionsusingglobalinference AT kollarthomasfleming wheretogointerpretingnaturaldirectionsusingglobalinference AT roynicholas wheretogointerpretingnaturaldirectionsusingglobalinference |