Approaching the Symbol Grounding Problem with Probabilistic Graphical Models
In order for robots to engage in dialog with human teammates, they must have the ability to map between words in the language and aspects of the external world. A solution to this symbol grounding problem (Harnad, 1990) would enable a robot to interpret commands such as “Drive over to receiving and...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for the Advancement of Artificial Intelligence
2012
|
| Online Access: | http://hdl.handle.net/1721.1/73542 https://orcid.org/0000-0002-8293-0492 |
| _version_ | 1826207247089270784 |
|---|---|
| author | Tellex, Stefanie A. Kollar, Thomas Fleming Dickerson, Steven R. Walter, Matthew R. Banerjee, Ashis Teller, Seth Roy, Nicholas |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Tellex, Stefanie A. Kollar, Thomas Fleming Dickerson, Steven R. Walter, Matthew R. Banerjee, Ashis Teller, Seth Roy, Nicholas |
| author_sort | Tellex, Stefanie A. |
| collection | MIT |
| description | In order for robots to engage in dialog with human teammates, they must have the ability to map between words in the language and aspects of the external world. A solution to this symbol grounding problem (Harnad, 1990) would enable a robot to interpret commands such as “Drive over to receiving and pick up the tire pallet.” In this article we describe several of our results that use probabilistic inference to address the symbol grounding problem. Our specific approach is to develop models that factor according to the linguistic structure of a command. We first describe an early result, a generative model that factors according to the sequential structure of language, and then discuss our new framework, generalized grounding graphs (G3). The G3 framework dynamically instantiates a probabilistic graphical model for a natural language input, enabling a mapping between words in language and concrete objects, places, paths and events in the external world. We report on corpus-based experiments where the robot is able to learn and use word meanings in three real-world tasks: indoor navigation, spatial language video retrieval, and mobile manipulation. |
| first_indexed | 2024-09-23T13:46:20Z |
| format | Article |
| id | mit-1721.1/73542 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T13:46:20Z |
| publishDate | 2012 |
| publisher | Association for the Advancement of Artificial Intelligence |
| record_format | dspace |
| spelling | mit-1721.1/735422022-09-28T16:04:55Z Approaching the Symbol Grounding Problem with Probabilistic Graphical Models Tellex, Stefanie A. Kollar, Thomas Fleming Dickerson, Steven R. Walter, Matthew R. Banerjee, Ashis Teller, Seth Roy, Nicholas Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Tellex, Stefanie A. Kollar, Thomas Fleming Dickerson, Steven R. Walter, Matthew R. Banerjee, Ashis Teller, Seth Roy, Nicholas In order for robots to engage in dialog with human teammates, they must have the ability to map between words in the language and aspects of the external world. A solution to this symbol grounding problem (Harnad, 1990) would enable a robot to interpret commands such as “Drive over to receiving and pick up the tire pallet.” In this article we describe several of our results that use probabilistic inference to address the symbol grounding problem. Our specific approach is to develop models that factor according to the linguistic structure of a command. We first describe an early result, a generative model that factors according to the sequential structure of language, and then discuss our new framework, generalized grounding graphs (G3). The G3 framework dynamically instantiates a probabilistic graphical model for a natural language input, enabling a mapping between words in language and concrete objects, places, paths and events in the external world. We report on corpus-based experiments where the robot is able to learn and use word meanings in three real-world tasks: indoor navigation, spatial language video retrieval, and mobile manipulation. U.S. Army Research Laboratory. Collaborative Technology Alliance Program (Cooperative Agreement W911NF-10-2-0016) United States. Office of Naval Research (MURI N00014-07-1-0749) 2012-10-02T14:51:12Z 2012-10-02T14:51:12Z 2011 Article http://purl.org/eprint/type/JournalArticle 0738-4602 http://hdl.handle.net/1721.1/73542 Tellex, S. et al. "Approaching the Symbol Grounding Problem with Probabilistic Graphical Models" AI Magazine 32.4, Winter 2011. https://orcid.org/0000-0002-8293-0492 en_US http://www.aaai.org/ojs/index.php/aimagazine/article/view/2384 AI Magazine Creative Commons Attribution-Noncommercial-Share Alike 3.0 http://creativecommons.org/licenses/by-nc-sa/3.0/ application/pdf Association for the Advancement of Artificial Intelligence MIT web domain |
| spellingShingle | Tellex, Stefanie A. Kollar, Thomas Fleming Dickerson, Steven R. Walter, Matthew R. Banerjee, Ashis Teller, Seth Roy, Nicholas Approaching the Symbol Grounding Problem with Probabilistic Graphical Models |
| title | Approaching the Symbol Grounding Problem with Probabilistic Graphical Models |
| title_full | Approaching the Symbol Grounding Problem with Probabilistic Graphical Models |
| title_fullStr | Approaching the Symbol Grounding Problem with Probabilistic Graphical Models |
| title_full_unstemmed | Approaching the Symbol Grounding Problem with Probabilistic Graphical Models |
| title_short | Approaching the Symbol Grounding Problem with Probabilistic Graphical Models |
| title_sort | approaching the symbol grounding problem with probabilistic graphical models |
| url | http://hdl.handle.net/1721.1/73542 https://orcid.org/0000-0002-8293-0492 |
| work_keys_str_mv | AT tellexstefaniea approachingthesymbolgroundingproblemwithprobabilisticgraphicalmodels AT kollarthomasfleming approachingthesymbolgroundingproblemwithprobabilisticgraphicalmodels AT dickersonstevenr approachingthesymbolgroundingproblemwithprobabilisticgraphicalmodels AT waltermatthewr approachingthesymbolgroundingproblemwithprobabilisticgraphicalmodels AT banerjeeashis approachingthesymbolgroundingproblemwithprobabilisticgraphicalmodels AT tellerseth approachingthesymbolgroundingproblemwithprobabilisticgraphicalmodels AT roynicholas approachingthesymbolgroundingproblemwithprobabilisticgraphicalmodels |