Efficient Grounding of Abstract Spatial Concepts for Natural Language Interaction with Robot Manipulators
Our goal is to develop models that allow a robot to understand natural language instructions in the context of its world representation. Contemporary models learn possible correspondences between parsed instructions and candidate groundings that include objects, regions and motion constraints. Howev...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Robotics: Science and Systems Foundation
2018
|
| Online Access: | http://hdl.handle.net/1721.1/116438 https://orcid.org/0000-0002-9693-2237 https://orcid.org/0000-0002-8293-0492 |
| _version_ | 1826192681465806848 |
|---|---|
| author | Paul, Rohan Arkin, Jacob Roy, Nicholas M. Howard, Thomas |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Paul, Rohan Arkin, Jacob Roy, Nicholas M. Howard, Thomas |
| author_sort | Paul, Rohan |
| collection | MIT |
| description | Our goal is to develop models that allow a robot to understand natural language instructions in the context of its world representation. Contemporary models learn possible correspondences between parsed instructions and candidate groundings that include objects, regions and motion constraints. However, these models cannot reason about abstract concepts expressed in an instruction like, “pick up the middle block in the row of five blocks”. In this work, we introduce a probabilistic model that incorporates an expressive space of abstract spatial concepts as well as notions of cardinality and ordinality. The graph is structured according to the parse structure of language
and introduces a factorisation over abstract concepts correlated with concrete constituents. Inference in the model is posed as an approximate search procedure that leverages partitioning of the joint in terms of concrete and abstract factors. The algorithm
first estimates a set of probable concrete constituents that constrains the search procedure to a reduced space of abstract concepts, pruning away improbable portions of the exponentiallylarge search space. Empirical evaluation demonstrates accurate grounding of abstract concepts embedded in complex natural language instructions commanding a robot manipulator. The proposed inference method leads to significant efficiency gains compared to the baseline, with minimal trade-off in accuracy. |
| first_indexed | 2024-09-23T09:27:38Z |
| format | Article |
| id | mit-1721.1/116438 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T09:27:38Z |
| publishDate | 2018 |
| publisher | Robotics: Science and Systems Foundation |
| record_format | dspace |
| spelling | mit-1721.1/1164382022-09-26T11:33:39Z Efficient Grounding of Abstract Spatial Concepts for Natural Language Interaction with Robot Manipulators Paul, Rohan Arkin, Jacob Roy, Nicholas M. Howard, Thomas Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Paul, Rohan Roy, Nicholas Our goal is to develop models that allow a robot to understand natural language instructions in the context of its world representation. Contemporary models learn possible correspondences between parsed instructions and candidate groundings that include objects, regions and motion constraints. However, these models cannot reason about abstract concepts expressed in an instruction like, “pick up the middle block in the row of five blocks”. In this work, we introduce a probabilistic model that incorporates an expressive space of abstract spatial concepts as well as notions of cardinality and ordinality. The graph is structured according to the parse structure of language and introduces a factorisation over abstract concepts correlated with concrete constituents. Inference in the model is posed as an approximate search procedure that leverages partitioning of the joint in terms of concrete and abstract factors. The algorithm first estimates a set of probable concrete constituents that constrains the search procedure to a reduced space of abstract concepts, pruning away improbable portions of the exponentiallylarge search space. Empirical evaluation demonstrates accurate grounding of abstract concepts embedded in complex natural language instructions commanding a robot manipulator. The proposed inference method leads to significant efficiency gains compared to the baseline, with minimal trade-off in accuracy. United States. Army Research Laboratory. Robotics Consortium (Collaborative Technology Alliance Program) National Science Foundation (U.S.) (Grant No.1427547) 2018-06-19T19:45:35Z 2018-06-19T19:45:35Z 2016 2018-04-09T18:37:51Z Article http://purl.org/eprint/type/ConferencePaper 9780992374723 http://hdl.handle.net/1721.1/116438 Paul, Rohan, Jacob Arkin, Nicholas Roy, and Thomas M. Howard. “Efficient Grounding of Abstract Spatial Concepts for Natural Language Interaction with Robot Manipulators.” Robotics: Science and Systems XII (n.d.), Ann Arbor, Michigan, Robotics: Science and Systems Foundation, 2016. https://orcid.org/0000-0002-9693-2237 https://orcid.org/0000-0002-8293-0492 http://dx.doi.org/10.15607/RSS.2016.XII.037 Robotics: Science and Systems XII Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Robotics: Science and Systems Foundation Other repository |
| spellingShingle | Paul, Rohan Arkin, Jacob Roy, Nicholas M. Howard, Thomas Efficient Grounding of Abstract Spatial Concepts for Natural Language Interaction with Robot Manipulators |
| title | Efficient Grounding of Abstract Spatial Concepts for Natural Language Interaction with Robot Manipulators |
| title_full | Efficient Grounding of Abstract Spatial Concepts for Natural Language Interaction with Robot Manipulators |
| title_fullStr | Efficient Grounding of Abstract Spatial Concepts for Natural Language Interaction with Robot Manipulators |
| title_full_unstemmed | Efficient Grounding of Abstract Spatial Concepts for Natural Language Interaction with Robot Manipulators |
| title_short | Efficient Grounding of Abstract Spatial Concepts for Natural Language Interaction with Robot Manipulators |
| title_sort | efficient grounding of abstract spatial concepts for natural language interaction with robot manipulators |
| url | http://hdl.handle.net/1721.1/116438 https://orcid.org/0000-0002-9693-2237 https://orcid.org/0000-0002-8293-0492 |
| work_keys_str_mv | AT paulrohan efficientgroundingofabstractspatialconceptsfornaturallanguageinteractionwithrobotmanipulators AT arkinjacob efficientgroundingofabstractspatialconceptsfornaturallanguageinteractionwithrobotmanipulators AT roynicholas efficientgroundingofabstractspatialconceptsfornaturallanguageinteractionwithrobotmanipulators AT mhowardthomas efficientgroundingofabstractspatialconceptsfornaturallanguageinteractionwithrobotmanipulators |