Warm-Starting for Improving the Novelty of Abstractive Summarization
Abstractive summarization is distinguished by using novel phrases that are not found in the source text. However, most previous research ignores this feature in favour of enhancing syntactical similarity with the reference. To improve novelty aspects, we have used multiple warm-started models with v...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10272590/ |
| _version_ | 1827791823471378432 |
|---|---|
| author | Ayham Alomari Ahmad Sami Al-Shamayleh Norisma Idris Aznul Qalid Md Sabri Izzat Alsmadi Danah Omary |
| author_facet | Ayham Alomari Ahmad Sami Al-Shamayleh Norisma Idris Aznul Qalid Md Sabri Izzat Alsmadi Danah Omary |
| author_sort | Ayham Alomari |
| collection | DOAJ |
| description | Abstractive summarization is distinguished by using novel phrases that are not found in the source text. However, most previous research ignores this feature in favour of enhancing syntactical similarity with the reference. To improve novelty aspects, we have used multiple warm-started models with varying encoder and decoder checkpoints and vocabulary. These models are then adapted to the paraphrasing task and the sampling decoding strategy to further boost the levels of novelty and quality. In addition, to avoid relying only on the syntactical similarity assessment, two additional abstractive summarization metrics are introduced: 1) NovScore: a new novelty metric that delivers a summary novelty score; and 2) NSSF: a new comprehensive metric that ensembles Novelty, Syntactic, Semantic, and Faithfulness features into a single score to simulate human assessment in providing a reliable evaluation. Finally, we compare our models to the state-of-the-art sequence-to-sequence models using the current and the proposed metrics. As a result, warm-starting, sampling, and paraphrasing improve novelty degrees by 2%, 5%, and 14%, respectively, while maintaining comparable scores on other metrics. |
| first_indexed | 2024-03-11T17:53:34Z |
| format | Article |
| id | doaj.art-d4304272523b463e8f6ffeb5bfdde4f8 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-11T17:53:34Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-d4304272523b463e8f6ffeb5bfdde4f82023-10-17T23:00:32ZengIEEEIEEE Access2169-35362023-01-011111248311250110.1109/ACCESS.2023.332222610272590Warm-Starting for Improving the Novelty of Abstractive SummarizationAyham Alomari0https://orcid.org/0000-0002-3507-5796Ahmad Sami Al-Shamayleh1https://orcid.org/0000-0002-7222-2433Norisma Idris2Aznul Qalid Md Sabri3https://orcid.org/0000-0002-4758-5400Izzat Alsmadi4https://orcid.org/0000-0001-7832-5081Danah Omary5Department of Computer Science, Faculty of Information Technology, Applied Science Private University, Amman, JordanDepartment of Data Science and Artificial Intelligence, Al-Ahliyya Amman University, Amman, JordanDepartment of Artificial Intelligence, Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, MalaysiaDepartment of Artificial Intelligence, Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, MalaysiaDepartment of Computing and Cybersecurity, Texas A&M University-San Antonio, San Antonio, TX, USADepartment of Electrical Engineering, University of North Texas, Denton, TX, USAAbstractive summarization is distinguished by using novel phrases that are not found in the source text. However, most previous research ignores this feature in favour of enhancing syntactical similarity with the reference. To improve novelty aspects, we have used multiple warm-started models with varying encoder and decoder checkpoints and vocabulary. These models are then adapted to the paraphrasing task and the sampling decoding strategy to further boost the levels of novelty and quality. In addition, to avoid relying only on the syntactical similarity assessment, two additional abstractive summarization metrics are introduced: 1) NovScore: a new novelty metric that delivers a summary novelty score; and 2) NSSF: a new comprehensive metric that ensembles Novelty, Syntactic, Semantic, and Faithfulness features into a single score to simulate human assessment in providing a reliable evaluation. Finally, we compare our models to the state-of-the-art sequence-to-sequence models using the current and the proposed metrics. As a result, warm-starting, sampling, and paraphrasing improve novelty degrees by 2%, 5%, and 14%, respectively, while maintaining comparable scores on other metrics.https://ieeexplore.ieee.org/document/10272590/Abstractive summarizationnoveltywarm-started modelsdeep learningmetrics |
| spellingShingle | Ayham Alomari Ahmad Sami Al-Shamayleh Norisma Idris Aznul Qalid Md Sabri Izzat Alsmadi Danah Omary Warm-Starting for Improving the Novelty of Abstractive Summarization IEEE Access Abstractive summarization novelty warm-started models deep learning metrics |
| title | Warm-Starting for Improving the Novelty of Abstractive Summarization |
| title_full | Warm-Starting for Improving the Novelty of Abstractive Summarization |
| title_fullStr | Warm-Starting for Improving the Novelty of Abstractive Summarization |
| title_full_unstemmed | Warm-Starting for Improving the Novelty of Abstractive Summarization |
| title_short | Warm-Starting for Improving the Novelty of Abstractive Summarization |
| title_sort | warm starting for improving the novelty of abstractive summarization |
| topic | Abstractive summarization novelty warm-started models deep learning metrics |
| url | https://ieeexplore.ieee.org/document/10272590/ |
| work_keys_str_mv | AT ayhamalomari warmstartingforimprovingthenoveltyofabstractivesummarization AT ahmadsamialshamayleh warmstartingforimprovingthenoveltyofabstractivesummarization AT norismaidris warmstartingforimprovingthenoveltyofabstractivesummarization AT aznulqalidmdsabri warmstartingforimprovingthenoveltyofabstractivesummarization AT izzatalsmadi warmstartingforimprovingthenoveltyofabstractivesummarization AT danahomary warmstartingforimprovingthenoveltyofabstractivesummarization |