Hierarchical graph-based text classification framework with contextual node embedding and BERT-based dynamic fusion
We propose a novel hierarchical graph-based text classification framework that leverages the power of contextual node embedding and BERT-based dynamic fusion to capture the complex relationships between the nodes in the hierarchical graph and generate a more accurate classification of text. The fram...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-07-01
|
| Series: | Journal of King Saud University: Computer and Information Sciences |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1319157823001647 |
| _version_ | 1797749164788940800 |
|---|---|
| author | Aytuğ Onan |
| author_facet | Aytuğ Onan |
| author_sort | Aytuğ Onan |
| collection | DOAJ |
| description | We propose a novel hierarchical graph-based text classification framework that leverages the power of contextual node embedding and BERT-based dynamic fusion to capture the complex relationships between the nodes in the hierarchical graph and generate a more accurate classification of text. The framework consists of seven stages: Linguistic Feature Extraction, Hierarchical Node Construction with Domain-Specific Knowledge, Contextual Node Embedding, Multi-Level Graph Learning, Dynamic Text Sequential Feature Interaction, Attention-Based Graph Learning, and Dynamic Fusion with BERT. The first stage, Linguistic Feature Extraction, extracts the linguistic features of the text, including part-of-speech tags, dependency parsing, and named entities. The second stage constructs a hierarchical graph based on the domain-specific knowledge, which is used to capture the relationships between nodes in the graph. The third stage, Contextual Node Embedding, generates a vector representation for each node in the hierarchical graph, which captures its local context information, linguistic features, and domain-specific knowledge. The fourth stage, Multi-Level Graph Learning, uses a graph convolutional neural network to learn the hierarchical structure of the graph and extract the features of the nodes in the graph. The fifth stage, Dynamic Text Sequential Feature Interaction, captures the sequential information of the text and generates dynamic features for each node. The sixth stage, Attention-Based Graph earning, uses an attention mechanism to capture the important features of the nodes in the graph. Finally, the seventh stage, Dynamic Fusion with BERT, combines the output from the previous stages with the output from a pre-trained BERT model to obtain the final integrated vector representation of the text. This approach leverages the strengths of both the proposed framework and BERT, allowing for better performance on the classification task. The proposed framework was evaluated on several benchmark datasets and compared to state-of-the-art methods, demonstrating significant improvements in classification accuracy. |
| first_indexed | 2024-03-12T16:15:24Z |
| format | Article |
| id | doaj.art-48f45bae68674993bf7ff6216b29c65c |
| institution | Directory Open Access Journal |
| issn | 1319-1578 |
| language | English |
| last_indexed | 2024-03-12T16:15:24Z |
| publishDate | 2023-07-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Journal of King Saud University: Computer and Information Sciences |
| spelling | doaj.art-48f45bae68674993bf7ff6216b29c65c2023-08-09T04:32:04ZengElsevierJournal of King Saud University: Computer and Information Sciences1319-15782023-07-01357101610Hierarchical graph-based text classification framework with contextual node embedding and BERT-based dynamic fusionAytuğ Onan0İzmir Katip Çelebi University, Faculty of Engineering and Architecture, Department of Computer Engineering, 35620 İzmir, TurkeyWe propose a novel hierarchical graph-based text classification framework that leverages the power of contextual node embedding and BERT-based dynamic fusion to capture the complex relationships between the nodes in the hierarchical graph and generate a more accurate classification of text. The framework consists of seven stages: Linguistic Feature Extraction, Hierarchical Node Construction with Domain-Specific Knowledge, Contextual Node Embedding, Multi-Level Graph Learning, Dynamic Text Sequential Feature Interaction, Attention-Based Graph Learning, and Dynamic Fusion with BERT. The first stage, Linguistic Feature Extraction, extracts the linguistic features of the text, including part-of-speech tags, dependency parsing, and named entities. The second stage constructs a hierarchical graph based on the domain-specific knowledge, which is used to capture the relationships between nodes in the graph. The third stage, Contextual Node Embedding, generates a vector representation for each node in the hierarchical graph, which captures its local context information, linguistic features, and domain-specific knowledge. The fourth stage, Multi-Level Graph Learning, uses a graph convolutional neural network to learn the hierarchical structure of the graph and extract the features of the nodes in the graph. The fifth stage, Dynamic Text Sequential Feature Interaction, captures the sequential information of the text and generates dynamic features for each node. The sixth stage, Attention-Based Graph earning, uses an attention mechanism to capture the important features of the nodes in the graph. Finally, the seventh stage, Dynamic Fusion with BERT, combines the output from the previous stages with the output from a pre-trained BERT model to obtain the final integrated vector representation of the text. This approach leverages the strengths of both the proposed framework and BERT, allowing for better performance on the classification task. The proposed framework was evaluated on several benchmark datasets and compared to state-of-the-art methods, demonstrating significant improvements in classification accuracy.http://www.sciencedirect.com/science/article/pii/S1319157823001647Text classificationHierarchical graphPre-trained language modelsContextual embeddingAttention mechanism |
| spellingShingle | Aytuğ Onan Hierarchical graph-based text classification framework with contextual node embedding and BERT-based dynamic fusion Journal of King Saud University: Computer and Information Sciences Text classification Hierarchical graph Pre-trained language models Contextual embedding Attention mechanism |
| title | Hierarchical graph-based text classification framework with contextual node embedding and BERT-based dynamic fusion |
| title_full | Hierarchical graph-based text classification framework with contextual node embedding and BERT-based dynamic fusion |
| title_fullStr | Hierarchical graph-based text classification framework with contextual node embedding and BERT-based dynamic fusion |
| title_full_unstemmed | Hierarchical graph-based text classification framework with contextual node embedding and BERT-based dynamic fusion |
| title_short | Hierarchical graph-based text classification framework with contextual node embedding and BERT-based dynamic fusion |
| title_sort | hierarchical graph based text classification framework with contextual node embedding and bert based dynamic fusion |
| topic | Text classification Hierarchical graph Pre-trained language models Contextual embedding Attention mechanism |
| url | http://www.sciencedirect.com/science/article/pii/S1319157823001647 |
| work_keys_str_mv | AT aytugonan hierarchicalgraphbasedtextclassificationframeworkwithcontextualnodeembeddingandbertbaseddynamicfusion |