Check-worthy claim detection across topics for automated fact-checking
An important component of an automated fact-checking system is the claim check-worthiness detection system, which ranks sentences by prioritising them based on their need to be checked. Despite a body of research tackling the task, previous research has overlooked the challenging nature of identifyi...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2023-05-01
|
| Series: | PeerJ Computer Science |
| Subjects: | |
| Online Access: | https://peerj.com/articles/cs-1365.pdf |
| _version_ | 1827944586114236416 |
|---|---|
| author | Amani S. Abumansour Arkaitz Zubiaga |
| author_facet | Amani S. Abumansour Arkaitz Zubiaga |
| author_sort | Amani S. Abumansour |
| collection | DOAJ |
| description | An important component of an automated fact-checking system is the claim check-worthiness detection system, which ranks sentences by prioritising them based on their need to be checked. Despite a body of research tackling the task, previous research has overlooked the challenging nature of identifying check-worthy claims across different topics. In this article, we assess and quantify the challenge of detecting check-worthy claims for new, unseen topics. After highlighting the problem, we propose the AraCWA model to mitigate the performance deterioration when detecting check-worthy claims across topics. The AraCWA model enables boosting the performance for new topics by incorporating two components for few-shot learning and data augmentation. Using a publicly available dataset of Arabic tweets consisting of 14 different topics, we demonstrate that our proposed data augmentation strategy achieves substantial improvements across topics overall, where the extent of the improvement varies across topics. Further, we analyse the semantic similarities between topics, suggesting that the similarity metric could be used as a proxy to determine the difficulty level of an unseen topic prior to undertaking the task of labelling the underlying sentences. |
| first_indexed | 2024-03-13T10:32:34Z |
| format | Article |
| id | doaj.art-f14b6477bb2a4c4c9048c8bc9b577ccd |
| institution | Directory Open Access Journal |
| issn | 2376-5992 |
| language | English |
| last_indexed | 2024-03-13T10:32:34Z |
| publishDate | 2023-05-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ Computer Science |
| spelling | doaj.art-f14b6477bb2a4c4c9048c8bc9b577ccd2023-05-18T15:05:10ZengPeerJ Inc.PeerJ Computer Science2376-59922023-05-019e136510.7717/peerj-cs.1365Check-worthy claim detection across topics for automated fact-checkingAmani S. Abumansour0Arkaitz Zubiaga1Queen Mary University of London, London, United KingdomQueen Mary University of London, London, United KingdomAn important component of an automated fact-checking system is the claim check-worthiness detection system, which ranks sentences by prioritising them based on their need to be checked. Despite a body of research tackling the task, previous research has overlooked the challenging nature of identifying check-worthy claims across different topics. In this article, we assess and quantify the challenge of detecting check-worthy claims for new, unseen topics. After highlighting the problem, we propose the AraCWA model to mitigate the performance deterioration when detecting check-worthy claims across topics. The AraCWA model enables boosting the performance for new topics by incorporating two components for few-shot learning and data augmentation. Using a publicly available dataset of Arabic tweets consisting of 14 different topics, we demonstrate that our proposed data augmentation strategy achieves substantial improvements across topics overall, where the extent of the improvement varies across topics. Further, we analyse the semantic similarities between topics, suggesting that the similarity metric could be used as a proxy to determine the difficulty level of an unseen topic prior to undertaking the task of labelling the underlying sentences.https://peerj.com/articles/cs-1365.pdfCheck-worthinessCheck-worthyClaim detection cross-topicAutomated fact-checking system |
| spellingShingle | Amani S. Abumansour Arkaitz Zubiaga Check-worthy claim detection across topics for automated fact-checking PeerJ Computer Science Check-worthiness Check-worthy Claim detection cross-topic Automated fact-checking system |
| title | Check-worthy claim detection across topics for automated fact-checking |
| title_full | Check-worthy claim detection across topics for automated fact-checking |
| title_fullStr | Check-worthy claim detection across topics for automated fact-checking |
| title_full_unstemmed | Check-worthy claim detection across topics for automated fact-checking |
| title_short | Check-worthy claim detection across topics for automated fact-checking |
| title_sort | check worthy claim detection across topics for automated fact checking |
| topic | Check-worthiness Check-worthy Claim detection cross-topic Automated fact-checking system |
| url | https://peerj.com/articles/cs-1365.pdf |
| work_keys_str_mv | AT amanisabumansour checkworthyclaimdetectionacrosstopicsforautomatedfactchecking AT arkaitzzubiaga checkworthyclaimdetectionacrosstopicsforautomatedfactchecking |