Emerging Research Topic Detection Using Filtered-LDA
Comparing two sets of documents to identify new topics is useful in many applications, like discovering trending topics from sets of scientific papers, emerging topic detection in microblogs, and interpreting sentiment variations in Twitter. In this paper, the main topic-modeling-based approaches to...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-10-01
|
| Series: | AI |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2673-2688/2/4/35 |
| _version_ | 1797506984909471744 |
|---|---|
| author | Fuad Alattar Khaled Shaalan |
| author_facet | Fuad Alattar Khaled Shaalan |
| author_sort | Fuad Alattar |
| collection | DOAJ |
| description | Comparing two sets of documents to identify new topics is useful in many applications, like discovering trending topics from sets of scientific papers, emerging topic detection in microblogs, and interpreting sentiment variations in Twitter. In this paper, the main topic-modeling-based approaches to address this task are examined to identify limitations and necessary enhancements. To overcome these limitations, we introduce two separate frameworks to discover emerging topics through a filtered latent Dirichlet allocation (filtered-LDA) model. The model acts as a filter that identifies old topics from a timestamped set of documents, removes all documents that focus on old topics, and keeps documents that discuss new topics. Filtered-LDA also genuinely reduces the chance of using keywords from old topics to represent emerging topics. The final stage of the filter uses multiple topic visualization formats to improve human interpretability of the filtered topics, and it presents the most-representative document for each topic. |
| first_indexed | 2024-03-10T04:40:15Z |
| format | Article |
| id | doaj.art-2557ead5884e4964848ff313765b9c22 |
| institution | Directory Open Access Journal |
| issn | 2673-2688 |
| language | English |
| last_indexed | 2024-03-10T04:40:15Z |
| publishDate | 2021-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | AI |
| spelling | doaj.art-2557ead5884e4964848ff313765b9c222023-11-23T03:24:30ZengMDPI AGAI2673-26882021-10-012457859910.3390/ai2040035Emerging Research Topic Detection Using Filtered-LDAFuad Alattar0Khaled Shaalan1Faculty of Engineering and IT, The British University in Dubai, Dubai 345015, United Arab EmiratesFaculty of Engineering and IT, The British University in Dubai, Dubai 345015, United Arab EmiratesComparing two sets of documents to identify new topics is useful in many applications, like discovering trending topics from sets of scientific papers, emerging topic detection in microblogs, and interpreting sentiment variations in Twitter. In this paper, the main topic-modeling-based approaches to address this task are examined to identify limitations and necessary enhancements. To overcome these limitations, we introduce two separate frameworks to discover emerging topics through a filtered latent Dirichlet allocation (filtered-LDA) model. The model acts as a filter that identifies old topics from a timestamped set of documents, removes all documents that focus on old topics, and keeps documents that discuss new topics. Filtered-LDA also genuinely reduces the chance of using keywords from old topics to represent emerging topics. The final stage of the filter uses multiple topic visualization formats to improve human interpretability of the filtered topics, and it presents the most-representative document for each topic.https://www.mdpi.com/2673-2688/2/4/35emerging topic detectionresearch trend detectiontopic discoverytopic modelinghot topicstrending topics |
| spellingShingle | Fuad Alattar Khaled Shaalan Emerging Research Topic Detection Using Filtered-LDA AI emerging topic detection research trend detection topic discovery topic modeling hot topics trending topics |
| title | Emerging Research Topic Detection Using Filtered-LDA |
| title_full | Emerging Research Topic Detection Using Filtered-LDA |
| title_fullStr | Emerging Research Topic Detection Using Filtered-LDA |
| title_full_unstemmed | Emerging Research Topic Detection Using Filtered-LDA |
| title_short | Emerging Research Topic Detection Using Filtered-LDA |
| title_sort | emerging research topic detection using filtered lda |
| topic | emerging topic detection research trend detection topic discovery topic modeling hot topics trending topics |
| url | https://www.mdpi.com/2673-2688/2/4/35 |
| work_keys_str_mv | AT fuadalattar emergingresearchtopicdetectionusingfilteredlda AT khaledshaalan emergingresearchtopicdetectionusingfilteredlda |