A text classification method based on LSTM and graph attention network
Text classification is a popular research topic in the natural language processing. Recently solving text classification problems with graph neural network (GNN) has received increasing attention. However, current graph-based studies ignore the hidden information in text syntax and sequence structur...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2022-12-01
|
| Series: | Connection Science |
| Subjects: | |
| Online Access: | http://dx.doi.org/10.1080/09540091.2022.2128047 |
| _version_ | 1797684018028740608 |
|---|---|
| author | Haitao Wang Fangbing Li |
| author_facet | Haitao Wang Fangbing Li |
| author_sort | Haitao Wang |
| collection | DOAJ |
| description | Text classification is a popular research topic in the natural language processing. Recently solving text classification problems with graph neural network (GNN) has received increasing attention. However, current graph-based studies ignore the hidden information in text syntax and sequence structure, and it is difficult to use the model directly for processing new documents because the text graph is built based on the whole corpus including the test set. To address the above problems, we propose a text classification model based on long short-term memory network (LSTM) and graph attention network (GAT). The model builds a separate graph based on the syntactic structure of each document, generates word embeddings with contextual information using LSTM, then learns the inductive representation of words by GAT, and finally fuses all the nodes in the graph together into the document embedding. Experimental results on four datasets show that our model outperforms existing text classification methods with faster convergence and less memory consumption than other graph-based methods. In addition, our model shows a more notable improvement when using less training data. Our model proves the importance of text syntax and sequence information for classification results. |
| first_indexed | 2024-03-12T00:23:16Z |
| format | Article |
| id | doaj.art-a0f9e89a597e48e486610c26f91a1963 |
| institution | Directory Open Access Journal |
| issn | 0954-0091 1360-0494 |
| language | English |
| last_indexed | 2024-03-12T00:23:16Z |
| publishDate | 2022-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | Connection Science |
| spelling | doaj.art-a0f9e89a597e48e486610c26f91a19632023-09-15T10:48:01ZengTaylor & Francis GroupConnection Science0954-00911360-04942022-12-013412466248010.1080/09540091.2022.21280472128047A text classification method based on LSTM and graph attention networkHaitao Wang0Fangbing Li1Henan Polytechnic UniversityHenan Polytechnic UniversityText classification is a popular research topic in the natural language processing. Recently solving text classification problems with graph neural network (GNN) has received increasing attention. However, current graph-based studies ignore the hidden information in text syntax and sequence structure, and it is difficult to use the model directly for processing new documents because the text graph is built based on the whole corpus including the test set. To address the above problems, we propose a text classification model based on long short-term memory network (LSTM) and graph attention network (GAT). The model builds a separate graph based on the syntactic structure of each document, generates word embeddings with contextual information using LSTM, then learns the inductive representation of words by GAT, and finally fuses all the nodes in the graph together into the document embedding. Experimental results on four datasets show that our model outperforms existing text classification methods with faster convergence and less memory consumption than other graph-based methods. In addition, our model shows a more notable improvement when using less training data. Our model proves the importance of text syntax and sequence information for classification results.http://dx.doi.org/10.1080/09540091.2022.2128047text classificationlstmgraph attention networkdependency syntaxdeep learning |
| spellingShingle | Haitao Wang Fangbing Li A text classification method based on LSTM and graph attention network Connection Science text classification lstm graph attention network dependency syntax deep learning |
| title | A text classification method based on LSTM and graph attention network |
| title_full | A text classification method based on LSTM and graph attention network |
| title_fullStr | A text classification method based on LSTM and graph attention network |
| title_full_unstemmed | A text classification method based on LSTM and graph attention network |
| title_short | A text classification method based on LSTM and graph attention network |
| title_sort | text classification method based on lstm and graph attention network |
| topic | text classification lstm graph attention network dependency syntax deep learning |
| url | http://dx.doi.org/10.1080/09540091.2022.2128047 |
| work_keys_str_mv | AT haitaowang atextclassificationmethodbasedonlstmandgraphattentionnetwork AT fangbingli atextclassificationmethodbasedonlstmandgraphattentionnetwork AT haitaowang textclassificationmethodbasedonlstmandgraphattentionnetwork AT fangbingli textclassificationmethodbasedonlstmandgraphattentionnetwork |