Keyphrase Enhanced Diverse Beam Search: A Content-Introducing Approach to Neural Text Generation
Neural text generation has been a challenging task, among which the text representation and the beam search are crucial techniques. By improving these techniques, we propose a novel model to generate texts of higher quality in this paper. First, we leverage the global and local contextual features b...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8726288/ |
| _version_ | 1828890451655000064 |
|---|---|
| author | Xuewen Chen Jinlong Li Haihan Wang |
| author_facet | Xuewen Chen Jinlong Li Haihan Wang |
| author_sort | Xuewen Chen |
| collection | DOAJ |
| description | Neural text generation has been a challenging task, among which the text representation and the beam search are crucial techniques. By improving these techniques, we propose a novel model to generate texts of higher quality in this paper. First, we leverage the global and local contextual features by combining the structure of both the recurrent neural network (RNN) and convolutional neural network (CNN) to learn a joint representation for the source text. Next, we introduce a modified diverse beam search to foster the diversity in the generated sentences during decoding, and then we rank these sentences according to its saliency score which measures the co-occurrence of keyphrases with the source text. Such a ranking mechanism promotes the semantical relevance between the source text and the generated sentence. To evaluate our model, we conduct extensive experiments on two neural generation tasks, including document summarization and headline generation. The results on both tasks show that our proposed model contributes to promising improvement in performance compared with the state-of-the-art baselines. |
| first_indexed | 2024-12-13T12:59:15Z |
| format | Article |
| id | doaj.art-65c3fd18b05f40e9a7f0dab1d5dc8f3f |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-13T12:59:15Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-65c3fd18b05f40e9a7f0dab1d5dc8f3f2022-12-21T23:45:04ZengIEEEIEEE Access2169-35362019-01-017727167272510.1109/ACCESS.2019.29199748726288Keyphrase Enhanced Diverse Beam Search: A Content-Introducing Approach to Neural Text GenerationXuewen Chen0https://orcid.org/0000-0002-9058-8332Jinlong Li1Haihan Wang2USTC-Birmingham Joint Research Institute, University of Science and Technology of China, Hefei, ChinaUSTC-Birmingham Joint Research Institute, University of Science and Technology of China, Hefei, ChinaUSTC-Birmingham Joint Research Institute, University of Science and Technology of China, Hefei, ChinaNeural text generation has been a challenging task, among which the text representation and the beam search are crucial techniques. By improving these techniques, we propose a novel model to generate texts of higher quality in this paper. First, we leverage the global and local contextual features by combining the structure of both the recurrent neural network (RNN) and convolutional neural network (CNN) to learn a joint representation for the source text. Next, we introduce a modified diverse beam search to foster the diversity in the generated sentences during decoding, and then we rank these sentences according to its saliency score which measures the co-occurrence of keyphrases with the source text. Such a ranking mechanism promotes the semantical relevance between the source text and the generated sentence. To evaluate our model, we conduct extensive experiments on two neural generation tasks, including document summarization and headline generation. The results on both tasks show that our proposed model contributes to promising improvement in performance compared with the state-of-the-art baselines.https://ieeexplore.ieee.org/document/8726288/Text generationneural networkssequence to sequencebeam searchkeyphrase |
| spellingShingle | Xuewen Chen Jinlong Li Haihan Wang Keyphrase Enhanced Diverse Beam Search: A Content-Introducing Approach to Neural Text Generation IEEE Access Text generation neural networks sequence to sequence beam search keyphrase |
| title | Keyphrase Enhanced Diverse Beam Search: A Content-Introducing Approach to Neural Text Generation |
| title_full | Keyphrase Enhanced Diverse Beam Search: A Content-Introducing Approach to Neural Text Generation |
| title_fullStr | Keyphrase Enhanced Diverse Beam Search: A Content-Introducing Approach to Neural Text Generation |
| title_full_unstemmed | Keyphrase Enhanced Diverse Beam Search: A Content-Introducing Approach to Neural Text Generation |
| title_short | Keyphrase Enhanced Diverse Beam Search: A Content-Introducing Approach to Neural Text Generation |
| title_sort | keyphrase enhanced diverse beam search a content introducing approach to neural text generation |
| topic | Text generation neural networks sequence to sequence beam search keyphrase |
| url | https://ieeexplore.ieee.org/document/8726288/ |
| work_keys_str_mv | AT xuewenchen keyphraseenhanceddiversebeamsearchacontentintroducingapproachtoneuraltextgeneration AT jinlongli keyphraseenhanceddiversebeamsearchacontentintroducingapproachtoneuraltextgeneration AT haihanwang keyphraseenhanceddiversebeamsearchacontentintroducingapproachtoneuraltextgeneration |