Token-Level Fact Correction in Abstractive Summarization
This paper addresses fact correction for abstractive summarization of which aim is to edit a system-generated summary into a new source-consistent summary. The summaries generated by abstractive summarization models often contain various kinds of factual errors. Thus, fact correction becomes essenti...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10005108/ |
| _version_ | 1797956338674827264 |
|---|---|
| author | Jeongwan Shin Seong-Bae Park Hyun-Je Song |
| author_facet | Jeongwan Shin Seong-Bae Park Hyun-Je Song |
| author_sort | Jeongwan Shin |
| collection | DOAJ |
| description | This paper addresses fact correction for abstractive summarization of which aim is to edit a system-generated summary into a new source-consistent summary. The summaries generated by abstractive summarization models often contain various kinds of factual errors. Thus, fact correction becomes essential to apply abstractive summarization to real-world applications. However, most existing methods for fact correction focus only on entity-level errors, which occasions the error correction methods to miss non-entity errors such as inconsistent tokens or mentions. Therefore, this paper presents a token-level fact correction that resolves inconsistencies of a system-generated summary at the token level. Since a token is the smallest meaning-bearing unit, all kinds of errors can be corrected if the errors are rectified at this level. The proposed fact corrector examines the consistency of a summary at the summary level like existing methods, but corrects the found inconsistencies at the token level. Thus, the proposed corrector consists of three modules of a summary fact checker, a token fact checker, and a fact emender. The summary fact checker inspects if a system-generated summary is factually consistent with a source text, the token fact checker finds out the tokens which cause inconsistency, and the fact emender actually replaces the inconsistency-causing tokens with correct tokens in the source text. Since these modules are closely related and affect one another, they are jointly trained to improve the performance of each module. The effectiveness of the proposed fact corrector is empirically proven from two viewpoints of consistency and summarization performance. For correcting inconsistencies in a summary, it is shown that the summaries by the proposed corrector are more factually consistent than those by its competitors. In addition, the proposed corrector outperforms the current state-of-the-art corrector even in automatic summarization performance. |
| first_indexed | 2024-04-10T23:48:31Z |
| format | Article |
| id | doaj.art-d11dad918aee4de79fbd4649012c0547 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-10T23:48:31Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-d11dad918aee4de79fbd4649012c05472023-01-11T00:00:40ZengIEEEIEEE Access2169-35362023-01-01111934194310.1109/ACCESS.2022.323385410005108Token-Level Fact Correction in Abstractive SummarizationJeongwan Shin0https://orcid.org/0000-0002-6990-2892Seong-Bae Park1https://orcid.org/0000-0002-6453-0348Hyun-Je Song2https://orcid.org/0000-0002-9109-2611School of Computer Science and Engineering, Kyungpook National University, Daegu, South KoreaSchool of Computer Science and Engineering, Kyung Hee University, Yongin-si, South KoreaDepartment of Information Technology and Engineering, Jeonbuk National University, Jeonju-si, South KoreaThis paper addresses fact correction for abstractive summarization of which aim is to edit a system-generated summary into a new source-consistent summary. The summaries generated by abstractive summarization models often contain various kinds of factual errors. Thus, fact correction becomes essential to apply abstractive summarization to real-world applications. However, most existing methods for fact correction focus only on entity-level errors, which occasions the error correction methods to miss non-entity errors such as inconsistent tokens or mentions. Therefore, this paper presents a token-level fact correction that resolves inconsistencies of a system-generated summary at the token level. Since a token is the smallest meaning-bearing unit, all kinds of errors can be corrected if the errors are rectified at this level. The proposed fact corrector examines the consistency of a summary at the summary level like existing methods, but corrects the found inconsistencies at the token level. Thus, the proposed corrector consists of three modules of a summary fact checker, a token fact checker, and a fact emender. The summary fact checker inspects if a system-generated summary is factually consistent with a source text, the token fact checker finds out the tokens which cause inconsistency, and the fact emender actually replaces the inconsistency-causing tokens with correct tokens in the source text. Since these modules are closely related and affect one another, they are jointly trained to improve the performance of each module. The effectiveness of the proposed fact corrector is empirically proven from two viewpoints of consistency and summarization performance. For correcting inconsistencies in a summary, it is shown that the summaries by the proposed corrector are more factually consistent than those by its competitors. In addition, the proposed corrector outperforms the current state-of-the-art corrector even in automatic summarization performance.https://ieeexplore.ieee.org/document/10005108/Token-level fact correctionfactual consistency in abstractive summarizationsummary fact checkertoken fact checkerfact emender |
| spellingShingle | Jeongwan Shin Seong-Bae Park Hyun-Je Song Token-Level Fact Correction in Abstractive Summarization IEEE Access Token-level fact correction factual consistency in abstractive summarization summary fact checker token fact checker fact emender |
| title | Token-Level Fact Correction in Abstractive Summarization |
| title_full | Token-Level Fact Correction in Abstractive Summarization |
| title_fullStr | Token-Level Fact Correction in Abstractive Summarization |
| title_full_unstemmed | Token-Level Fact Correction in Abstractive Summarization |
| title_short | Token-Level Fact Correction in Abstractive Summarization |
| title_sort | token level fact correction in abstractive summarization |
| topic | Token-level fact correction factual consistency in abstractive summarization summary fact checker token fact checker fact emender |
| url | https://ieeexplore.ieee.org/document/10005108/ |
| work_keys_str_mv | AT jeongwanshin tokenlevelfactcorrectioninabstractivesummarization AT seongbaepark tokenlevelfactcorrectioninabstractivesummarization AT hyunjesong tokenlevelfactcorrectioninabstractivesummarization |