Dialogue-driven Multi-Agent Activity Planning
A fundamental challenge in robotics is to build a general-purpose system with multiple agents that can perform a wide range of tasks based on specifications provided in natural language. This work presents a novel dialogue-driven activity planning framework for multiagent scenarios. We present a met...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2024
|
| Online Access: | https://hdl.handle.net/1721.1/156325 https://orcid.org/0000-0003-0478-0254 |
| _version_ | 1826193317036032000 |
|---|---|
| author | Sonar, Anoopkumar S. |
| author2 | Williams, Brian C. |
| author_facet | Williams, Brian C. Sonar, Anoopkumar S. |
| author_sort | Sonar, Anoopkumar S. |
| collection | MIT |
| description | A fundamental challenge in robotics is to build a general-purpose system with multiple agents that can perform a wide range of tasks based on specifications provided in natural language. This work presents a novel dialogue-driven activity planning framework for multiagent scenarios. We present a method that accepts commands from a user in natural language and translates it to an intermediate form called a state plan by leveraging large language models. We further experiment with chain-of-thought prompting to improve the translation from natural language to state plans. In conjunction with an action model, this state plan is utilized by a constraint-based generative planner called ctBurton which outputs a full grounded plan in the form of a state and control trajectory. We demonstrate the utility of our method across three different scenarios– a presentation system, search-and-rescue, and multi-agent assembly– along with experiments on its scalability. |
| first_indexed | 2024-09-23T09:37:04Z |
| format | Thesis |
| id | mit-1721.1/156325 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T09:37:04Z |
| publishDate | 2024 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1563252024-08-22T03:53:38Z Dialogue-driven Multi-Agent Activity Planning Sonar, Anoopkumar S. Williams, Brian C. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science A fundamental challenge in robotics is to build a general-purpose system with multiple agents that can perform a wide range of tasks based on specifications provided in natural language. This work presents a novel dialogue-driven activity planning framework for multiagent scenarios. We present a method that accepts commands from a user in natural language and translates it to an intermediate form called a state plan by leveraging large language models. We further experiment with chain-of-thought prompting to improve the translation from natural language to state plans. In conjunction with an action model, this state plan is utilized by a constraint-based generative planner called ctBurton which outputs a full grounded plan in the form of a state and control trajectory. We demonstrate the utility of our method across three different scenarios– a presentation system, search-and-rescue, and multi-agent assembly– along with experiments on its scalability. S.M. 2024-08-21T18:56:54Z 2024-08-21T18:56:54Z 2024-05 2024-07-10T12:59:59.141Z Thesis https://hdl.handle.net/1721.1/156325 https://orcid.org/0000-0003-0478-0254 Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) Copyright retained by author(s) https://creativecommons.org/licenses/by-nc-nd/4.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Sonar, Anoopkumar S. Dialogue-driven Multi-Agent Activity Planning |
| title | Dialogue-driven Multi-Agent Activity Planning |
| title_full | Dialogue-driven Multi-Agent Activity Planning |
| title_fullStr | Dialogue-driven Multi-Agent Activity Planning |
| title_full_unstemmed | Dialogue-driven Multi-Agent Activity Planning |
| title_short | Dialogue-driven Multi-Agent Activity Planning |
| title_sort | dialogue driven multi agent activity planning |
| url | https://hdl.handle.net/1721.1/156325 https://orcid.org/0000-0003-0478-0254 |
| work_keys_str_mv | AT sonaranoopkumars dialoguedrivenmultiagentactivityplanning |