An efficient and accurate deep learning architecture for text sentiment prediction
Sentiment analysis is an important process in learning individual opinions on a certain topic, product or service. Sentiment analysis offers many insights on a product or service based on the polarity of its reviews, the public sentiment of opinions such as the consumer confidence of an economy as w...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/138157 |
| _version_ | 1811689492838350848 |
|---|---|
| author | Tan, Jun En |
| author2 | Jagath C Rajapakse |
| author_facet | Jagath C Rajapakse Tan, Jun En |
| author_sort | Tan, Jun En |
| collection | NTU |
| description | Sentiment analysis is an important process in learning individual opinions on a certain topic, product or service. Sentiment analysis offers many insights on a product or service based on the polarity of its reviews, the public sentiment of opinions such as the consumer confidence of an economy as well as the public opinion on a certain political candidate or issue. Accurate and efficient sentiment analysis is important as it allows companies to create new marketing strategies or modify existing marketing strategies based on public opinion. The ability to gauge the sentiment of a topic means being better able to strategize and react appropriately to changes in sentiment. However, the human language is complex and is often layered with figurative language, the use of sarcasm and polysemy, which is the capacity for a word or phrase to have multiple meanings in different contexts. Therefore, it is a difficult task to effectively predict the sentiment of a sentence or document
We investigate the use of different deep-learning approaches on the problem of sentiment classification, especially on polarity detection, where a review or opinion will be either classified as positive or negative. In this project, we study the different approaches used for sentiment polarity detection and create a novel deep learning architecture to attempt to maximize the accuracy of sentiment classification. We also explore new approaches that can reduce the memory footprint of the proposed model to ensure a faster and more efficient method of sentiment analysis.
Several deep learning approaches that have been studied and applied in this project are transfer learning approaches in Natural Language Processing (NLP), Transformer model blocks and its architectures such as Bidirectional Encoder representations from Transformers (BERT) language representation models. This project intends to study the use of Transformer blocks with previous state-of-the-art deep learning model architectures such as pre-trained word embeddings and Recurrent Neural Network blocks. An architecture using a combination of embeddings, transformer blocks and Long Short-Term Memory networks is proposed and compared with existing state-of-the-art deep learning approaches in sentiment prediction.
Our proposed model achieved over 90% accuracy for the sentiment classification tasks without fine-tuning of different hyperparameters. Although we do not achieve the state-
3
of-the-art results using the proposed model architecture, our model architecture achieves a high level of sentiment classification with a smaller memory footprint, making it viable for use in commercial applications compared to the more memory-intensive state-of-the-art deep learning models.
We decided to further investigate the potential of the model architecture by including pre-trained weights and having a larger model size. The result obtained was comparable to the current state-of-the-art accuracies for text sentiment prediction. |
| first_indexed | 2024-10-01T05:48:58Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/138157 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:48:58Z |
| publishDate | 2020 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1381572020-04-27T07:21:36Z An efficient and accurate deep learning architecture for text sentiment prediction Tan, Jun En Jagath C Rajapakse School of Computer Science and Engineering asjagath@ntu.edu.sg Engineering::Computer science and engineering Sentiment analysis is an important process in learning individual opinions on a certain topic, product or service. Sentiment analysis offers many insights on a product or service based on the polarity of its reviews, the public sentiment of opinions such as the consumer confidence of an economy as well as the public opinion on a certain political candidate or issue. Accurate and efficient sentiment analysis is important as it allows companies to create new marketing strategies or modify existing marketing strategies based on public opinion. The ability to gauge the sentiment of a topic means being better able to strategize and react appropriately to changes in sentiment. However, the human language is complex and is often layered with figurative language, the use of sarcasm and polysemy, which is the capacity for a word or phrase to have multiple meanings in different contexts. Therefore, it is a difficult task to effectively predict the sentiment of a sentence or document We investigate the use of different deep-learning approaches on the problem of sentiment classification, especially on polarity detection, where a review or opinion will be either classified as positive or negative. In this project, we study the different approaches used for sentiment polarity detection and create a novel deep learning architecture to attempt to maximize the accuracy of sentiment classification. We also explore new approaches that can reduce the memory footprint of the proposed model to ensure a faster and more efficient method of sentiment analysis. Several deep learning approaches that have been studied and applied in this project are transfer learning approaches in Natural Language Processing (NLP), Transformer model blocks and its architectures such as Bidirectional Encoder representations from Transformers (BERT) language representation models. This project intends to study the use of Transformer blocks with previous state-of-the-art deep learning model architectures such as pre-trained word embeddings and Recurrent Neural Network blocks. An architecture using a combination of embeddings, transformer blocks and Long Short-Term Memory networks is proposed and compared with existing state-of-the-art deep learning approaches in sentiment prediction. Our proposed model achieved over 90% accuracy for the sentiment classification tasks without fine-tuning of different hyperparameters. Although we do not achieve the state- 3 of-the-art results using the proposed model architecture, our model architecture achieves a high level of sentiment classification with a smaller memory footprint, making it viable for use in commercial applications compared to the more memory-intensive state-of-the-art deep learning models. We decided to further investigate the potential of the model architecture by including pre-trained weights and having a larger model size. The result obtained was comparable to the current state-of-the-art accuracies for text sentiment prediction. Bachelor of Engineering (Computer Science) 2020-04-27T07:21:36Z 2020-04-27T07:21:36Z 2020 Final Year Project (FYP) https://hdl.handle.net/10356/138157 en SCSE19-0279 application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Computer science and engineering Tan, Jun En An efficient and accurate deep learning architecture for text sentiment prediction |
| title | An efficient and accurate deep learning architecture for text sentiment prediction |
| title_full | An efficient and accurate deep learning architecture for text sentiment prediction |
| title_fullStr | An efficient and accurate deep learning architecture for text sentiment prediction |
| title_full_unstemmed | An efficient and accurate deep learning architecture for text sentiment prediction |
| title_short | An efficient and accurate deep learning architecture for text sentiment prediction |
| title_sort | efficient and accurate deep learning architecture for text sentiment prediction |
| topic | Engineering::Computer science and engineering |
| url | https://hdl.handle.net/10356/138157 |
| work_keys_str_mv | AT tanjunen anefficientandaccuratedeeplearningarchitecturefortextsentimentprediction AT tanjunen efficientandaccuratedeeplearningarchitecturefortextsentimentprediction |