CEG: A joint model for causal commonsense events enhanced story ending generation.
With the success of pre-trained language models, the performance of story ending generation has been dramatically improved while remaining challenging due to the lack of commonsense reasoning ability. Most previous works mainly focus on using commonsense knowledge to enhance the implicit correlation...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2023-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0286049 |
| _version_ | 1827923497694789632 |
|---|---|
| author | Yushi Zhang Yan Yang Ming Gu Feng Gao Chengcai Chen Liang He |
| author_facet | Yushi Zhang Yan Yang Ming Gu Feng Gao Chengcai Chen Liang He |
| author_sort | Yushi Zhang |
| collection | DOAJ |
| description | With the success of pre-trained language models, the performance of story ending generation has been dramatically improved while remaining challenging due to the lack of commonsense reasoning ability. Most previous works mainly focus on using commonsense knowledge to enhance the implicit correlations between words but ignore the hidden causality of sentences or events. In this paper, we propose Causal commonsense Enhanced joint model for story ending Generation (CEG), which incorporates causal commonsense events knowledge to generate a reasonable story ending. Specifically, we first develop a commonsense events inference model trained on GLUCOSE, which converts static knowledge into a dynamic generation model to discover unseen knowledge. It uses prompts to produce various commonsense events behind the stories as pseudo-labels of the dataset. Then, we propose a joint model for the causal events inference task and the story ending generation task to inject inference knowledge into the generation, which consists of a shared encoder, an inference decoder, and a generation decoder. In the causal events inference task, we use the shared encoder and the inference decoder to reason the causal events behind each sentence of the story context to help the model better understand the story and provide long-distance dependencies for the story ending generation. In story ending generation, we combine the hidden states of the causal events with the story context to generate the story ending by the shared encoder and the generation decoder. We jointly train the model on two tasks so that the generation decoder produces the story endings that better match the clues. Experimental results on the ROCStories dataset show that our model outperforms the previous works, demonstrating the effectiveness of the joint model and the generated causal events. |
| first_indexed | 2024-03-13T04:58:45Z |
| format | Article |
| id | doaj.art-6f727b20e7cb4514a6706bed91e35a1d |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-03-13T04:58:45Z |
| publishDate | 2023-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-6f727b20e7cb4514a6706bed91e35a1d2023-06-17T05:32:01ZengPublic Library of Science (PLoS)PLoS ONE1932-62032023-01-01185e028604910.1371/journal.pone.0286049CEG: A joint model for causal commonsense events enhanced story ending generation.Yushi ZhangYan YangMing GuFeng GaoChengcai ChenLiang HeWith the success of pre-trained language models, the performance of story ending generation has been dramatically improved while remaining challenging due to the lack of commonsense reasoning ability. Most previous works mainly focus on using commonsense knowledge to enhance the implicit correlations between words but ignore the hidden causality of sentences or events. In this paper, we propose Causal commonsense Enhanced joint model for story ending Generation (CEG), which incorporates causal commonsense events knowledge to generate a reasonable story ending. Specifically, we first develop a commonsense events inference model trained on GLUCOSE, which converts static knowledge into a dynamic generation model to discover unseen knowledge. It uses prompts to produce various commonsense events behind the stories as pseudo-labels of the dataset. Then, we propose a joint model for the causal events inference task and the story ending generation task to inject inference knowledge into the generation, which consists of a shared encoder, an inference decoder, and a generation decoder. In the causal events inference task, we use the shared encoder and the inference decoder to reason the causal events behind each sentence of the story context to help the model better understand the story and provide long-distance dependencies for the story ending generation. In story ending generation, we combine the hidden states of the causal events with the story context to generate the story ending by the shared encoder and the generation decoder. We jointly train the model on two tasks so that the generation decoder produces the story endings that better match the clues. Experimental results on the ROCStories dataset show that our model outperforms the previous works, demonstrating the effectiveness of the joint model and the generated causal events.https://doi.org/10.1371/journal.pone.0286049 |
| spellingShingle | Yushi Zhang Yan Yang Ming Gu Feng Gao Chengcai Chen Liang He CEG: A joint model for causal commonsense events enhanced story ending generation. PLoS ONE |
| title | CEG: A joint model for causal commonsense events enhanced story ending generation. |
| title_full | CEG: A joint model for causal commonsense events enhanced story ending generation. |
| title_fullStr | CEG: A joint model for causal commonsense events enhanced story ending generation. |
| title_full_unstemmed | CEG: A joint model for causal commonsense events enhanced story ending generation. |
| title_short | CEG: A joint model for causal commonsense events enhanced story ending generation. |
| title_sort | ceg a joint model for causal commonsense events enhanced story ending generation |
| url | https://doi.org/10.1371/journal.pone.0286049 |
| work_keys_str_mv | AT yushizhang cegajointmodelforcausalcommonsenseeventsenhancedstoryendinggeneration AT yanyang cegajointmodelforcausalcommonsenseeventsenhancedstoryendinggeneration AT minggu cegajointmodelforcausalcommonsenseeventsenhancedstoryendinggeneration AT fenggao cegajointmodelforcausalcommonsenseeventsenhancedstoryendinggeneration AT chengcaichen cegajointmodelforcausalcommonsenseeventsenhancedstoryendinggeneration AT lianghe cegajointmodelforcausalcommonsenseeventsenhancedstoryendinggeneration |