Attention Pooling-Based Bidirectional GRU Model for Sentimental Classification
Recurrent neural network (RNN) is one of the most popular architectures for addressing variable sequence text, and it shows outstanding results in many natural language processing (NLP) tasks and remarkable performance in capturing long-term dependencies. Many models have achieved excellent results...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
|
| Series: | International Journal of Computational Intelligence Systems |
| Subjects: | |
| Online Access: | https://www.atlantis-press.com/article/125913350/view |
| _version_ | 1811234201790316544 |
|---|---|
| author | Dejun Zhang Mingbo Hong Lu Zou Fei Han Fazhi He Zhigang Tu Yafeng Ren |
| author_facet | Dejun Zhang Mingbo Hong Lu Zou Fei Han Fazhi He Zhigang Tu Yafeng Ren |
| author_sort | Dejun Zhang |
| collection | DOAJ |
| description | Recurrent neural network (RNN) is one of the most popular architectures for addressing variable sequence text, and it shows outstanding results in many natural language processing (NLP) tasks and remarkable performance in capturing long-term dependencies. Many models have achieved excellent results based on RNN. However, most of these models overlook the locations of the keywords in a sentence and the semantic connections in different directions. As a consequence, these methods do not make full use of the available information. Considering that different words in a sequence usually have different importance, in this paper, we propose bidirectional gated recurrent units (BGRUs) which integrates a novel attention pooling mechanism with max-pooling operation to force the model to pay attention to the keywords in a sentence and maintain the most meaningful information of the text automatically. The presented model allows to encode longer sequences. Thus, it not only prevents important information from being discarded but also can be used to filter noises. To avoid full exposure of content without any control, we add an output gate to the GRU, which is named as text unit. The proposed model was evaluated on multiple tasks, including sentimental classification, movie review data, and a subjective classification dataset. Experimental results show that our model can achieve excellent performance on these tasks. |
| first_indexed | 2024-04-12T11:32:42Z |
| format | Article |
| id | doaj.art-00cb4a802b534f0ca21aeeafa3e10ef8 |
| institution | Directory Open Access Journal |
| issn | 1875-6883 |
| language | English |
| last_indexed | 2024-04-12T11:32:42Z |
| publisher | Springer |
| record_format | Article |
| series | International Journal of Computational Intelligence Systems |
| spelling | doaj.art-00cb4a802b534f0ca21aeeafa3e10ef82022-12-22T03:34:57ZengSpringerInternational Journal of Computational Intelligence Systems1875-688310.2991/ijcis.d.190710.001Attention Pooling-Based Bidirectional GRU Model for Sentimental ClassificationDejun ZhangMingbo HongLu ZouFei HanFazhi HeZhigang TuYafeng RenRecurrent neural network (RNN) is one of the most popular architectures for addressing variable sequence text, and it shows outstanding results in many natural language processing (NLP) tasks and remarkable performance in capturing long-term dependencies. Many models have achieved excellent results based on RNN. However, most of these models overlook the locations of the keywords in a sentence and the semantic connections in different directions. As a consequence, these methods do not make full use of the available information. Considering that different words in a sequence usually have different importance, in this paper, we propose bidirectional gated recurrent units (BGRUs) which integrates a novel attention pooling mechanism with max-pooling operation to force the model to pay attention to the keywords in a sentence and maintain the most meaningful information of the text automatically. The presented model allows to encode longer sequences. Thus, it not only prevents important information from being discarded but also can be used to filter noises. To avoid full exposure of content without any control, we add an output gate to the GRU, which is named as text unit. The proposed model was evaluated on multiple tasks, including sentimental classification, movie review data, and a subjective classification dataset. Experimental results show that our model can achieve excellent performance on these tasks.https://www.atlantis-press.com/article/125913350/viewNatural language processingNeural networkGated recurrent unitsText classification |
| spellingShingle | Dejun Zhang Mingbo Hong Lu Zou Fei Han Fazhi He Zhigang Tu Yafeng Ren Attention Pooling-Based Bidirectional GRU Model for Sentimental Classification International Journal of Computational Intelligence Systems Natural language processing Neural network Gated recurrent units Text classification |
| title | Attention Pooling-Based Bidirectional GRU Model for Sentimental Classification |
| title_full | Attention Pooling-Based Bidirectional GRU Model for Sentimental Classification |
| title_fullStr | Attention Pooling-Based Bidirectional GRU Model for Sentimental Classification |
| title_full_unstemmed | Attention Pooling-Based Bidirectional GRU Model for Sentimental Classification |
| title_short | Attention Pooling-Based Bidirectional GRU Model for Sentimental Classification |
| title_sort | attention pooling based bidirectional gru model for sentimental classification |
| topic | Natural language processing Neural network Gated recurrent units Text classification |
| url | https://www.atlantis-press.com/article/125913350/view |
| work_keys_str_mv | AT dejunzhang attentionpoolingbasedbidirectionalgrumodelforsentimentalclassification AT mingbohong attentionpoolingbasedbidirectionalgrumodelforsentimentalclassification AT luzou attentionpoolingbasedbidirectionalgrumodelforsentimentalclassification AT feihan attentionpoolingbasedbidirectionalgrumodelforsentimentalclassification AT fazhihe attentionpoolingbasedbidirectionalgrumodelforsentimentalclassification AT zhigangtu attentionpoolingbasedbidirectionalgrumodelforsentimentalclassification AT yafengren attentionpoolingbasedbidirectionalgrumodelforsentimentalclassification |