Convolution-Based Neural Attention With Applications to Sentiment Classification
Neural attention mechanism has achieved many successes in various tasks in natural language processing. However, existing neural attention models based on a densely connected network are loosely related to the attention mechanism found in psychology and neuroscience. Motivated by the finding in neur...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8648327/ |
| _version_ | 1818645024833798144 |
|---|---|
| author | Jiachen Du Lin Gui Yulan He Ruifeng Xu Xuan Wang |
| author_facet | Jiachen Du Lin Gui Yulan He Ruifeng Xu Xuan Wang |
| author_sort | Jiachen Du |
| collection | DOAJ |
| description | Neural attention mechanism has achieved many successes in various tasks in natural language processing. However, existing neural attention models based on a densely connected network are loosely related to the attention mechanism found in psychology and neuroscience. Motivated by the finding in neuroscience that human possesses the template-searching attention mechanism, we propose to use convolution operation to simulate attentions and give a mathematical explanation of our neural attention model. We then introduce a new network architecture, which combines a recurrent neural network with our convolution-based attention model and further stacks an attention-based neural model to build a hierarchical sentiment classification model. The experimental results show that our proposed models can capture salient parts of the text to improve the performance of sentiment classification at both the sentence level and the document level. |
| first_indexed | 2024-12-17T00:24:10Z |
| format | Article |
| id | doaj.art-c4399cb0d826436397c534091e64fb58 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-17T00:24:10Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-c4399cb0d826436397c534091e64fb582022-12-21T22:10:30ZengIEEEIEEE Access2169-35362019-01-017279832799210.1109/ACCESS.2019.29003358648327Convolution-Based Neural Attention With Applications to Sentiment ClassificationJiachen Du0https://orcid.org/0000-0002-6284-7691Lin Gui1Yulan He2Ruifeng Xu3Xuan Wang4School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen, ChinaDepartment of Computer Science, University of Warwick, Coventry, U.K.Department of Computer Science, University of Warwick, Coventry, U.K.School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen, ChinaSchool of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen, ChinaNeural attention mechanism has achieved many successes in various tasks in natural language processing. However, existing neural attention models based on a densely connected network are loosely related to the attention mechanism found in psychology and neuroscience. Motivated by the finding in neuroscience that human possesses the template-searching attention mechanism, we propose to use convolution operation to simulate attentions and give a mathematical explanation of our neural attention model. We then introduce a new network architecture, which combines a recurrent neural network with our convolution-based attention model and further stacks an attention-based neural model to build a hierarchical sentiment classification model. The experimental results show that our proposed models can capture salient parts of the text to improve the performance of sentiment classification at both the sentence level and the document level.https://ieeexplore.ieee.org/document/8648327/Natural language processingsentiment classificationconvolutional neural networksneural attention model |
| spellingShingle | Jiachen Du Lin Gui Yulan He Ruifeng Xu Xuan Wang Convolution-Based Neural Attention With Applications to Sentiment Classification IEEE Access Natural language processing sentiment classification convolutional neural networks neural attention model |
| title | Convolution-Based Neural Attention With Applications to Sentiment Classification |
| title_full | Convolution-Based Neural Attention With Applications to Sentiment Classification |
| title_fullStr | Convolution-Based Neural Attention With Applications to Sentiment Classification |
| title_full_unstemmed | Convolution-Based Neural Attention With Applications to Sentiment Classification |
| title_short | Convolution-Based Neural Attention With Applications to Sentiment Classification |
| title_sort | convolution based neural attention with applications to sentiment classification |
| topic | Natural language processing sentiment classification convolutional neural networks neural attention model |
| url | https://ieeexplore.ieee.org/document/8648327/ |
| work_keys_str_mv | AT jiachendu convolutionbasedneuralattentionwithapplicationstosentimentclassification AT lingui convolutionbasedneuralattentionwithapplicationstosentimentclassification AT yulanhe convolutionbasedneuralattentionwithapplicationstosentimentclassification AT ruifengxu convolutionbasedneuralattentionwithapplicationstosentimentclassification AT xuanwang convolutionbasedneuralattentionwithapplicationstosentimentclassification |