<scp>InSCIt</scp>: Information-Seeking Conversations with Mixed-Initiative Interactions
AbstractIn an information-seeking conversation, a user may ask questions that are under-specified or unanswerable. An ideal agent would interact by initiating different response types according to the available knowledge sources. However, most current studies either fail to or artifi...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
The MIT Press
2023-05-01
|
| Series: | Transactions of the Association for Computational Linguistics |
| Online Access: | https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00559/116048/InSCIt-Information-Seeking-Conversations-with |
| _version_ | 1797796630173319168 |
|---|---|
| author | Zeqiu Wu Ryu Parish Hao Cheng Sewon Min Prithviraj Ammanabrolu Mari Ostendorf Hannaneh Hajishirzi |
| author_facet | Zeqiu Wu Ryu Parish Hao Cheng Sewon Min Prithviraj Ammanabrolu Mari Ostendorf Hannaneh Hajishirzi |
| author_sort | Zeqiu Wu |
| collection | DOAJ |
| description |
AbstractIn an information-seeking conversation, a user may ask questions that are under-specified or unanswerable. An ideal agent would interact by initiating different response types according to the available knowledge sources. However, most current studies either fail to or artificially incorporate such agent-side initiative. This work presents InSCIt, a dataset for Information-Seeking Conversations with mixed-initiative Interactions. It contains 4.7K user-agent turns from 805 human-human conversations where the agent searches over Wikipedia and either directly answers, asks for clarification, or provides relevant information to address user queries. The data supports two subtasks, evidence passage identification and response generation, as well as a human evaluation protocol to assess model performance. We report results of two systems based on state-of-the-art models of conversational knowledge identification and open-domain question answering. Both systems significantly underperform humans, suggesting ample room for improvement in future studies.1 |
| first_indexed | 2024-03-13T03:36:56Z |
| format | Article |
| id | doaj.art-ece55ee3d6294fe3b867f5436a755131 |
| institution | Directory Open Access Journal |
| issn | 2307-387X |
| language | English |
| last_indexed | 2024-03-13T03:36:56Z |
| publishDate | 2023-05-01 |
| publisher | The MIT Press |
| record_format | Article |
| series | Transactions of the Association for Computational Linguistics |
| spelling | doaj.art-ece55ee3d6294fe3b867f5436a7551312023-06-23T18:57:59ZengThe MIT PressTransactions of the Association for Computational Linguistics2307-387X2023-05-011145346810.1162/tacl_a_00559 <scp>InSCIt</scp>: Information-Seeking Conversations with Mixed-Initiative InteractionsZeqiu Wu0Ryu Parish1Hao Cheng2Sewon Min3Prithviraj Ammanabrolu4Mari Ostendorf5Hannaneh Hajishirzi6University of Washington, USA. zeqiuwu1@uw.eduUniversity of Washington, USA. rparish@uw.eduMicrosoft Research, USA. chehao@microsoft.comUniversity of Washington, USA. sewon@uw.eduAllen Institute for AI, USA. raja@allenai.orgUniversity of Washington, USA. ostendor@uw.eduUniversity of Washington, USA. hannaneh@uw.edu AbstractIn an information-seeking conversation, a user may ask questions that are under-specified or unanswerable. An ideal agent would interact by initiating different response types according to the available knowledge sources. However, most current studies either fail to or artificially incorporate such agent-side initiative. This work presents InSCIt, a dataset for Information-Seeking Conversations with mixed-initiative Interactions. It contains 4.7K user-agent turns from 805 human-human conversations where the agent searches over Wikipedia and either directly answers, asks for clarification, or provides relevant information to address user queries. The data supports two subtasks, evidence passage identification and response generation, as well as a human evaluation protocol to assess model performance. We report results of two systems based on state-of-the-art models of conversational knowledge identification and open-domain question answering. Both systems significantly underperform humans, suggesting ample room for improvement in future studies.1https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00559/116048/InSCIt-Information-Seeking-Conversations-with |
| spellingShingle | Zeqiu Wu Ryu Parish Hao Cheng Sewon Min Prithviraj Ammanabrolu Mari Ostendorf Hannaneh Hajishirzi <scp>InSCIt</scp>: Information-Seeking Conversations with Mixed-Initiative Interactions Transactions of the Association for Computational Linguistics |
| title |
<scp>InSCIt</scp>: Information-Seeking Conversations with Mixed-Initiative Interactions |
| title_full |
<scp>InSCIt</scp>: Information-Seeking Conversations with Mixed-Initiative Interactions |
| title_fullStr |
<scp>InSCIt</scp>: Information-Seeking Conversations with Mixed-Initiative Interactions |
| title_full_unstemmed |
<scp>InSCIt</scp>: Information-Seeking Conversations with Mixed-Initiative Interactions |
| title_short |
<scp>InSCIt</scp>: Information-Seeking Conversations with Mixed-Initiative Interactions |
| title_sort | scp inscit scp information seeking conversations with mixed initiative interactions |
| url | https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00559/116048/InSCIt-Information-Seeking-Conversations-with |
| work_keys_str_mv | AT zeqiuwu scpinscitscpinformationseekingconversationswithmixedinitiativeinteractions AT ryuparish scpinscitscpinformationseekingconversationswithmixedinitiativeinteractions AT haocheng scpinscitscpinformationseekingconversationswithmixedinitiativeinteractions AT sewonmin scpinscitscpinformationseekingconversationswithmixedinitiativeinteractions AT prithvirajammanabrolu scpinscitscpinformationseekingconversationswithmixedinitiativeinteractions AT mariostendorf scpinscitscpinformationseekingconversationswithmixedinitiativeinteractions AT hannanehhajishirzi scpinscitscpinformationseekingconversationswithmixedinitiativeinteractions |