Automatic topic detection of news
The aim of this project is to explore the topic of Natural Language Processing and how to implement it into automatic topic detection, namely categorization and topic generation of news articles. The project will mainly focus on using unsupervised learning methods for implementation to reduce the...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/140587 |
| _version_ | 1811676710715785216 |
|---|---|
| author | Liu, Fengyuan |
| author2 | Mao Kezhi |
| author_facet | Mao Kezhi Liu, Fengyuan |
| author_sort | Liu, Fengyuan |
| collection | NTU |
| description | The aim of this project is to explore the topic of Natural Language Processing and how to
implement it into automatic topic detection, namely categorization and topic generation
of news articles. The project will mainly focus on using unsupervised learning methods
for implementation to reduce the amount of manual work and fulfill the “automatic”
component of the project [1].
Choosing the “right” information to read on the internet is a growing issue today. It is
especially true for the news segment due to the vast amount of news available online.
This brings our attention to one of the current solutions which is filtering or categorizing
news into different sections and topics. However, manually categorizing the news is slow
and prone to error since personal opinion is involved. Hence, the drive of the project
would be to explore news topic detection using machine learning.
The first half of the project explores topic modeling [2] and how to categorize news text
using machine learning. The methodology chosen is Latent Dirichlet Allocation [3]. This
model is trained on the “20 Newsgroup” dataset which contains 20,000 news documents
across 20 different fields [4]. The second half of the project used the categorized results
and further fine-grained the categories by generating new topic titles to choose from. The
methodology used is Word2vec pre-trained on “Text8” corpus and fine-tuned using the
“20 Newsgroup” dataset. This project also experiments on different approaches and
hyperparameters to further analyze the results for both techniques. |
| first_indexed | 2024-10-01T02:25:48Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/140587 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T02:25:48Z |
| publishDate | 2020 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1405872023-07-07T18:47:33Z Automatic topic detection of news Liu, Fengyuan Mao Kezhi School of Electrical and Electronic Engineering ekzmao@ntu.edu.sg Engineering::Electrical and electronic engineering The aim of this project is to explore the topic of Natural Language Processing and how to implement it into automatic topic detection, namely categorization and topic generation of news articles. The project will mainly focus on using unsupervised learning methods for implementation to reduce the amount of manual work and fulfill the “automatic” component of the project [1]. Choosing the “right” information to read on the internet is a growing issue today. It is especially true for the news segment due to the vast amount of news available online. This brings our attention to one of the current solutions which is filtering or categorizing news into different sections and topics. However, manually categorizing the news is slow and prone to error since personal opinion is involved. Hence, the drive of the project would be to explore news topic detection using machine learning. The first half of the project explores topic modeling [2] and how to categorize news text using machine learning. The methodology chosen is Latent Dirichlet Allocation [3]. This model is trained on the “20 Newsgroup” dataset which contains 20,000 news documents across 20 different fields [4]. The second half of the project used the categorized results and further fine-grained the categories by generating new topic titles to choose from. The methodology used is Word2vec pre-trained on “Text8” corpus and fine-tuned using the “20 Newsgroup” dataset. This project also experiments on different approaches and hyperparameters to further analyze the results for both techniques. Bachelor of Engineering (Information Engineering and Media) 2020-05-31T11:47:56Z 2020-05-31T11:47:56Z 2020 Final Year Project (FYP) https://hdl.handle.net/10356/140587 en A1119-191 application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Electrical and electronic engineering Liu, Fengyuan Automatic topic detection of news |
| title | Automatic topic detection of news |
| title_full | Automatic topic detection of news |
| title_fullStr | Automatic topic detection of news |
| title_full_unstemmed | Automatic topic detection of news |
| title_short | Automatic topic detection of news |
| title_sort | automatic topic detection of news |
| topic | Engineering::Electrical and electronic engineering |
| url | https://hdl.handle.net/10356/140587 |
| work_keys_str_mv | AT liufengyuan automatictopicdetectionofnews |