LA-HCN: label-based attention for hierarchical multi-label text classification neural network
Hierarchical multi-label text classification (HMTC) has been gaining popularity in recent years thanks to its applicability to a plethora of real-world applications. The existing HMTC algorithms largely focus on the design of classifiers, such as the local, global, or a combination of them. However,...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/160673 |
| _version_ | 1811689821422223360 |
|---|---|
| author | Zhang, Xinyi Xu, Jiahao Soh, Charlie Chen, Lihui |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Zhang, Xinyi Xu, Jiahao Soh, Charlie Chen, Lihui |
| author_sort | Zhang, Xinyi |
| collection | NTU |
| description | Hierarchical multi-label text classification (HMTC) has been gaining popularity in recent years thanks to its applicability to a plethora of real-world applications. The existing HMTC algorithms largely focus on the design of classifiers, such as the local, global, or a combination of them. However, very few studies have focused on hierarchical feature extraction and explore the association between the hierarchical labels and the text. In this paper, we propose a Label-based Attention for Hierarchical Multi-label Text Classification Neural Network (LA-HCN), where the novel label-based attention module is designed to hierarchically extract important information from the text based on the labels from different hierarchy levels. Besides, hierarchical information is shared across levels while preserving the hierarchical label-based information. Separate local and global document embeddings are obtained and used to facilitate the respective local and global classifications. In our experiments, LA-HCN outperforms other state-of-the-art neural network-based HMTC algorithms on four public HMTC datasets. The ablation study also demonstrates the effectiveness of the proposed label-based attention module as well as the novel local and global embeddings and classifications. By visualizing the learned attention (words), we find that LA-HCN is able to extract meaningful information corresponding to the different labels which provides explainability that may be helpful for the human analyst. |
| first_indexed | 2024-10-01T05:54:11Z |
| format | Journal Article |
| id | ntu-10356/160673 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:54:11Z |
| publishDate | 2022 |
| record_format | dspace |
| spelling | ntu-10356/1606732022-07-29T08:29:13Z LA-HCN: label-based attention for hierarchical multi-label text classification neural network Zhang, Xinyi Xu, Jiahao Soh, Charlie Chen, Lihui School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Deep Neural Network Attention Hierarchical multi-label text classification (HMTC) has been gaining popularity in recent years thanks to its applicability to a plethora of real-world applications. The existing HMTC algorithms largely focus on the design of classifiers, such as the local, global, or a combination of them. However, very few studies have focused on hierarchical feature extraction and explore the association between the hierarchical labels and the text. In this paper, we propose a Label-based Attention for Hierarchical Multi-label Text Classification Neural Network (LA-HCN), where the novel label-based attention module is designed to hierarchically extract important information from the text based on the labels from different hierarchy levels. Besides, hierarchical information is shared across levels while preserving the hierarchical label-based information. Separate local and global document embeddings are obtained and used to facilitate the respective local and global classifications. In our experiments, LA-HCN outperforms other state-of-the-art neural network-based HMTC algorithms on four public HMTC datasets. The ablation study also demonstrates the effectiveness of the proposed label-based attention module as well as the novel local and global embeddings and classifications. By visualizing the learned attention (words), we find that LA-HCN is able to extract meaningful information corresponding to the different labels which provides explainability that may be helpful for the human analyst. National Research Foundation (NRF) This research/project is supported by the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award No: AISG-100E-2019-031). 2022-07-29T08:29:13Z 2022-07-29T08:29:13Z 2022 Journal Article Zhang, X., Xu, J., Soh, C. & Chen, L. (2022). LA-HCN: label-based attention for hierarchical multi-label text classification neural network. Expert Systems With Applications, 187, 115922-. https://dx.doi.org/10.1016/j.eswa.2021.115922 0957-4174 https://hdl.handle.net/10356/160673 10.1016/j.eswa.2021.115922 2-s2.0-85116380503 187 115922 en AISG-100E-2019-031 Expert Systems with Applications © 2021 Elsevier Ltd. All rights reserved. |
| spellingShingle | Engineering::Electrical and electronic engineering Deep Neural Network Attention Zhang, Xinyi Xu, Jiahao Soh, Charlie Chen, Lihui LA-HCN: label-based attention for hierarchical multi-label text classification neural network |
| title | LA-HCN: label-based attention for hierarchical multi-label text classification neural network |
| title_full | LA-HCN: label-based attention for hierarchical multi-label text classification neural network |
| title_fullStr | LA-HCN: label-based attention for hierarchical multi-label text classification neural network |
| title_full_unstemmed | LA-HCN: label-based attention for hierarchical multi-label text classification neural network |
| title_short | LA-HCN: label-based attention for hierarchical multi-label text classification neural network |
| title_sort | la hcn label based attention for hierarchical multi label text classification neural network |
| topic | Engineering::Electrical and electronic engineering Deep Neural Network Attention |
| url | https://hdl.handle.net/10356/160673 |
| work_keys_str_mv | AT zhangxinyi lahcnlabelbasedattentionforhierarchicalmultilabeltextclassificationneuralnetwork AT xujiahao lahcnlabelbasedattentionforhierarchicalmultilabeltextclassificationneuralnetwork AT sohcharlie lahcnlabelbasedattentionforhierarchicalmultilabeltextclassificationneuralnetwork AT chenlihui lahcnlabelbasedattentionforhierarchicalmultilabeltextclassificationneuralnetwork |