Dual Features Local-Global Attention Model with BERT for Aspect Sentiment Analysis
Aspect-based sentiment analysis aims to predict the sentiment polarity of a specific aspect in a sentence or document. Most of recent research uses attention mechanism to model the context. But there is a problem in that the context information needs to be considered according to different contexts...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press
2024-01-01
|
| Series: | Jisuanji kexue yu tansuo |
| Subjects: | |
| Online Access: | http://fcst.ceaj.org/fileup/1673-9418/PDF/2210012.pdf |
| _version_ | 1827387966583996416 |
|---|---|
| author | LI Jin, XIA Hongbin, LIU Yuan |
| author_facet | LI Jin, XIA Hongbin, LIU Yuan |
| author_sort | LI Jin, XIA Hongbin, LIU Yuan |
| collection | DOAJ |
| description | Aspect-based sentiment analysis aims to predict the sentiment polarity of a specific aspect in a sentence or document. Most of recent research uses attention mechanism to model the context. But there is a problem in that the context information needs to be considered according to different contexts when the BERT model is used to calculate the dependencies between representations to extract features by sentiment classification models, which leads to the lack of contextual knowledge of the modelled features. And the importance of aspect words is not given more attention, affecting the overall classification performance of the model. To address the problems above, this paper proposes a dual features local-global attention model with BERT (DFLGA-BERT). Local and global feature extraction modules are designed respectively to fully capture the semantic association between aspect words and context. Moreover, an improved quasi-attention mechanism is used in DFLGA-BERT, which leads to the model using minus attention in the fusion of attention to weaken the effect of noise on classification in the text. The feature fusion structure of local and global features is designed to better integrate regional and global features based on conditional layer normalization (CLN). Experiments are conducted on the SentiHood and SemEval 2014 Task 4 datasets. Experimental results show that the performance of the proposed model is significantly improved compared with the baselines after incorporating contextual features. |
| first_indexed | 2024-03-08T16:10:57Z |
| format | Article |
| id | doaj.art-f6d7cef60d964ceabe8b7fb5517e98b7 |
| institution | Directory Open Access Journal |
| issn | 1673-9418 |
| language | zho |
| last_indexed | 2024-03-08T16:10:57Z |
| publishDate | 2024-01-01 |
| publisher | Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press |
| record_format | Article |
| series | Jisuanji kexue yu tansuo |
| spelling | doaj.art-f6d7cef60d964ceabe8b7fb5517e98b72024-01-08T01:41:14ZzhoJournal of Computer Engineering and Applications Beijing Co., Ltd., Science PressJisuanji kexue yu tansuo1673-94182024-01-0118120521610.3778/j.issn.1673-9418.2210012Dual Features Local-Global Attention Model with BERT for Aspect Sentiment AnalysisLI Jin, XIA Hongbin, LIU Yuan01. School of Artificial Intelligence and Computer, Jiangnan University, Wuxi, Jiangsu 214122, China 2. Jiangsu Key Laboratory of Media Design and Software Technology, Wuxi, Jiangsu 214122, ChinaAspect-based sentiment analysis aims to predict the sentiment polarity of a specific aspect in a sentence or document. Most of recent research uses attention mechanism to model the context. But there is a problem in that the context information needs to be considered according to different contexts when the BERT model is used to calculate the dependencies between representations to extract features by sentiment classification models, which leads to the lack of contextual knowledge of the modelled features. And the importance of aspect words is not given more attention, affecting the overall classification performance of the model. To address the problems above, this paper proposes a dual features local-global attention model with BERT (DFLGA-BERT). Local and global feature extraction modules are designed respectively to fully capture the semantic association between aspect words and context. Moreover, an improved quasi-attention mechanism is used in DFLGA-BERT, which leads to the model using minus attention in the fusion of attention to weaken the effect of noise on classification in the text. The feature fusion structure of local and global features is designed to better integrate regional and global features based on conditional layer normalization (CLN). Experiments are conducted on the SentiHood and SemEval 2014 Task 4 datasets. Experimental results show that the performance of the proposed model is significantly improved compared with the baselines after incorporating contextual features.http://fcst.ceaj.org/fileup/1673-9418/PDF/2210012.pdfsentiment analysis; natural language understanding; quasi-attention mechanism; contextual attention |
| spellingShingle | LI Jin, XIA Hongbin, LIU Yuan Dual Features Local-Global Attention Model with BERT for Aspect Sentiment Analysis Jisuanji kexue yu tansuo sentiment analysis; natural language understanding; quasi-attention mechanism; contextual attention |
| title | Dual Features Local-Global Attention Model with BERT for Aspect Sentiment Analysis |
| title_full | Dual Features Local-Global Attention Model with BERT for Aspect Sentiment Analysis |
| title_fullStr | Dual Features Local-Global Attention Model with BERT for Aspect Sentiment Analysis |
| title_full_unstemmed | Dual Features Local-Global Attention Model with BERT for Aspect Sentiment Analysis |
| title_short | Dual Features Local-Global Attention Model with BERT for Aspect Sentiment Analysis |
| title_sort | dual features local global attention model with bert for aspect sentiment analysis |
| topic | sentiment analysis; natural language understanding; quasi-attention mechanism; contextual attention |
| url | http://fcst.ceaj.org/fileup/1673-9418/PDF/2210012.pdf |
| work_keys_str_mv | AT lijinxiahongbinliuyuan dualfeatureslocalglobalattentionmodelwithbertforaspectsentimentanalysis |