Sentiment Analysis of Review Text Based on BiGRU-Attention and Hybrid CNN
Convolutional neural networks (CNN), recurrent neural networks (RNN), attention, and their variants are extensively applied in the sentiment analysis, and the effect of fusion model is expected to be better. However, fusion model is confronted with some problems such as complicated structure, excess...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9562511/ |
| _version_ | 1818402993197809664 |
|---|---|
| author | Qiannan Zhu Xiaofan Jiang Renzhen Ye |
| author_facet | Qiannan Zhu Xiaofan Jiang Renzhen Ye |
| author_sort | Qiannan Zhu |
| collection | DOAJ |
| description | Convolutional neural networks (CNN), recurrent neural networks (RNN), attention, and their variants are extensively applied in the sentiment analysis, and the effect of fusion model is expected to be better. However, fusion model is confronted with some problems such as complicated structure, excessive trainable parameters, and long training time. The classification effect of traditional model with cross entropy loss as loss function is undesirable since sample category imbalance as well as ease and difficulty of sample classification is not taken into account. In order to solve these problems, the model BiGRU-Att-HCNN is proposed on the basis of bidirectional gated recurrent unit (BiGRU), attention, and hybrid convolutional neural networks. In this model, BiGRU and self-attention are combined to acquire global information, and key information weight is supplemented. Two parallel convolutions (dilated convolution and standard convolution) are used to obtain multi-scale characteristic information with relatively less parameters, and the standard convolution is replaced with depthwise separable convolution with two-step calculations. Traditional max-pooling and average-pooling are discarded, and global average pooling is applied to substitute the pooling layer and the fully-connected layer simultaneously, making it possible to substantially decrease the number of model parameters and reduce over-fitting. In our model, focal loss is used as the loss function to tackle the problems of unbalanced sample categories and hard samples. Experimental results illustrate that in terms of multiple indicators, our model outperforms the 15 benchmark models, even with intermediate number of trainable parameters. |
| first_indexed | 2024-12-14T08:17:11Z |
| format | Article |
| id | doaj.art-7238031c83f244d6be63df3ef0b37e04 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-14T08:17:11Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-7238031c83f244d6be63df3ef0b37e042022-12-21T23:09:54ZengIEEEIEEE Access2169-35362021-01-01914907714908810.1109/ACCESS.2021.31185379562511Sentiment Analysis of Review Text Based on BiGRU-Attention and Hybrid CNNQiannan Zhu0Xiaofan Jiang1Renzhen Ye2https://orcid.org/0000-0002-2569-9060College of Science, Huazhong Agricultural University, Wuhan, ChinaChina Construction Bank, Beijing, ChinaCollege of Science, Huazhong Agricultural University, Wuhan, ChinaConvolutional neural networks (CNN), recurrent neural networks (RNN), attention, and their variants are extensively applied in the sentiment analysis, and the effect of fusion model is expected to be better. However, fusion model is confronted with some problems such as complicated structure, excessive trainable parameters, and long training time. The classification effect of traditional model with cross entropy loss as loss function is undesirable since sample category imbalance as well as ease and difficulty of sample classification is not taken into account. In order to solve these problems, the model BiGRU-Att-HCNN is proposed on the basis of bidirectional gated recurrent unit (BiGRU), attention, and hybrid convolutional neural networks. In this model, BiGRU and self-attention are combined to acquire global information, and key information weight is supplemented. Two parallel convolutions (dilated convolution and standard convolution) are used to obtain multi-scale characteristic information with relatively less parameters, and the standard convolution is replaced with depthwise separable convolution with two-step calculations. Traditional max-pooling and average-pooling are discarded, and global average pooling is applied to substitute the pooling layer and the fully-connected layer simultaneously, making it possible to substantially decrease the number of model parameters and reduce over-fitting. In our model, focal loss is used as the loss function to tackle the problems of unbalanced sample categories and hard samples. Experimental results illustrate that in terms of multiple indicators, our model outperforms the 15 benchmark models, even with intermediate number of trainable parameters.https://ieeexplore.ieee.org/document/9562511/Bidirectional gated recurrent unitdepthwise separable convolutiondilated convolutionfocal lossself-attention mechanism |
| spellingShingle | Qiannan Zhu Xiaofan Jiang Renzhen Ye Sentiment Analysis of Review Text Based on BiGRU-Attention and Hybrid CNN IEEE Access Bidirectional gated recurrent unit depthwise separable convolution dilated convolution focal loss self-attention mechanism |
| title | Sentiment Analysis of Review Text Based on BiGRU-Attention and Hybrid CNN |
| title_full | Sentiment Analysis of Review Text Based on BiGRU-Attention and Hybrid CNN |
| title_fullStr | Sentiment Analysis of Review Text Based on BiGRU-Attention and Hybrid CNN |
| title_full_unstemmed | Sentiment Analysis of Review Text Based on BiGRU-Attention and Hybrid CNN |
| title_short | Sentiment Analysis of Review Text Based on BiGRU-Attention and Hybrid CNN |
| title_sort | sentiment analysis of review text based on bigru attention and hybrid cnn |
| topic | Bidirectional gated recurrent unit depthwise separable convolution dilated convolution focal loss self-attention mechanism |
| url | https://ieeexplore.ieee.org/document/9562511/ |
| work_keys_str_mv | AT qiannanzhu sentimentanalysisofreviewtextbasedonbigruattentionandhybridcnn AT xiaofanjiang sentimentanalysisofreviewtextbasedonbigruattentionandhybridcnn AT renzhenye sentimentanalysisofreviewtextbasedonbigruattentionandhybridcnn |