A System to Support Readers in Automatically Acquiring Complete Summarized Information on an Event from Different Sources
Today, most newspapers utilize social media to disseminate news. On the one hand, this results in an overload of related articles for social media users. On the other hand, since social media tends to form echo chambers around their users, different opinions and information may be hidden. Enabling u...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Algorithms |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-4893/16/11/513 |
| _version_ | 1827640888492294144 |
|---|---|
| author | Pietro Dell’Oglio Alessandro Bondielli Francesco Marcelloni |
| author_facet | Pietro Dell’Oglio Alessandro Bondielli Francesco Marcelloni |
| author_sort | Pietro Dell’Oglio |
| collection | DOAJ |
| description | Today, most newspapers utilize social media to disseminate news. On the one hand, this results in an overload of related articles for social media users. On the other hand, since social media tends to form echo chambers around their users, different opinions and information may be hidden. Enabling users to access different information (possibly outside of their echo chambers, without the burden of reading entire articles, often containing redundant information) may be a step forward in allowing them to form their own opinions. To address this challenge, we propose a system that integrates Transformer neural models and text summarization models along with decision rules. Given a reference article already read by the user, our system first collects articles related to the same topic from a configurable number of different sources. Then, it identifies and summarizes the information that differs from the reference article and outputs the summary to the user. The core of the system is the sentence classification algorithm, which classifies sentences in the collected articles into three classes based on similarity with the reference article: sentences classified as dissimilar are summarized by using a pre-trained abstractive summarization model. We evaluated the proposed system in two steps. First, we assessed its effectiveness in identifying content differences between the reference article and the related articles by using human judgments obtained through crowdsourcing as ground truth. We obtained an average F1 score of 0.772 against average F1 scores of 0.797 and 0.676 achieved by two state-of-the-art approaches based, respectively, on model tuning and prompt tuning, which require an appropriate tuning phase and, therefore, greater computational effort. Second, we asked a sample of people to evaluate how well the summary generated by the system represents the information that is not present in the article read by the user. The results are extremely encouraging. Finally, we present a use case. |
| first_indexed | 2024-03-09T17:05:52Z |
| format | Article |
| id | doaj.art-d52ad5fa2fad4780ab2c0e8ae53afd2e |
| institution | Directory Open Access Journal |
| issn | 1999-4893 |
| language | English |
| last_indexed | 2024-03-09T17:05:52Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Algorithms |
| spelling | doaj.art-d52ad5fa2fad4780ab2c0e8ae53afd2e2023-11-24T14:24:24ZengMDPI AGAlgorithms1999-48932023-11-01161151310.3390/a16110513A System to Support Readers in Automatically Acquiring Complete Summarized Information on an Event from Different SourcesPietro Dell’Oglio0Alessandro Bondielli1Francesco Marcelloni2Department of Information Engineering, University of Florence, Via di S. Marta, 3, 50039 Florence, ItalyDepartment of Computer Science, University of Pisa, Largo Bruno Pontecorvo, 3, 56127 Pisa, ItalyDepartment of Information Engineering, University of Pisa, Largo Lucio Lazzarino, 1, 56122 Pisa, ItalyToday, most newspapers utilize social media to disseminate news. On the one hand, this results in an overload of related articles for social media users. On the other hand, since social media tends to form echo chambers around their users, different opinions and information may be hidden. Enabling users to access different information (possibly outside of their echo chambers, without the burden of reading entire articles, often containing redundant information) may be a step forward in allowing them to form their own opinions. To address this challenge, we propose a system that integrates Transformer neural models and text summarization models along with decision rules. Given a reference article already read by the user, our system first collects articles related to the same topic from a configurable number of different sources. Then, it identifies and summarizes the information that differs from the reference article and outputs the summary to the user. The core of the system is the sentence classification algorithm, which classifies sentences in the collected articles into three classes based on similarity with the reference article: sentences classified as dissimilar are summarized by using a pre-trained abstractive summarization model. We evaluated the proposed system in two steps. First, we assessed its effectiveness in identifying content differences between the reference article and the related articles by using human judgments obtained through crowdsourcing as ground truth. We obtained an average F1 score of 0.772 against average F1 scores of 0.797 and 0.676 achieved by two state-of-the-art approaches based, respectively, on model tuning and prompt tuning, which require an appropriate tuning phase and, therefore, greater computational effort. Second, we asked a sample of people to evaluate how well the summary generated by the system represents the information that is not present in the article read by the user. The results are extremely encouraging. Finally, we present a use case.https://www.mdpi.com/1999-4893/16/11/513natural language processingtext similarityTransformersneural language modelsnewspaper articles |
| spellingShingle | Pietro Dell’Oglio Alessandro Bondielli Francesco Marcelloni A System to Support Readers in Automatically Acquiring Complete Summarized Information on an Event from Different Sources Algorithms natural language processing text similarity Transformers neural language models newspaper articles |
| title | A System to Support Readers in Automatically Acquiring Complete Summarized Information on an Event from Different Sources |
| title_full | A System to Support Readers in Automatically Acquiring Complete Summarized Information on an Event from Different Sources |
| title_fullStr | A System to Support Readers in Automatically Acquiring Complete Summarized Information on an Event from Different Sources |
| title_full_unstemmed | A System to Support Readers in Automatically Acquiring Complete Summarized Information on an Event from Different Sources |
| title_short | A System to Support Readers in Automatically Acquiring Complete Summarized Information on an Event from Different Sources |
| title_sort | system to support readers in automatically acquiring complete summarized information on an event from different sources |
| topic | natural language processing text similarity Transformers neural language models newspaper articles |
| url | https://www.mdpi.com/1999-4893/16/11/513 |
| work_keys_str_mv | AT pietrodelloglio asystemtosupportreadersinautomaticallyacquiringcompletesummarizedinformationonaneventfromdifferentsources AT alessandrobondielli asystemtosupportreadersinautomaticallyacquiringcompletesummarizedinformationonaneventfromdifferentsources AT francescomarcelloni asystemtosupportreadersinautomaticallyacquiringcompletesummarizedinformationonaneventfromdifferentsources AT pietrodelloglio systemtosupportreadersinautomaticallyacquiringcompletesummarizedinformationonaneventfromdifferentsources AT alessandrobondielli systemtosupportreadersinautomaticallyacquiringcompletesummarizedinformationonaneventfromdifferentsources AT francescomarcelloni systemtosupportreadersinautomaticallyacquiringcompletesummarizedinformationonaneventfromdifferentsources |