Incorporating Label Co-Occurrence Into Neural Network-Based Models for Multi-Label Text Classification
Multi-label text classification (MLTC) addresses a fundamental problem in natural language processing, which assigns multiple relevant labels to each document. In recent years, Neural Network-based models (NN models) for MLTC have attracted much attention. In addition, NN models achieve favorable pe...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8936450/ |
| _version_ | 1819171018230464512 |
|---|---|
| author | Jiaqi Yao Keren Wang Jikun Yan |
| author_facet | Jiaqi Yao Keren Wang Jikun Yan |
| author_sort | Jiaqi Yao |
| collection | DOAJ |
| description | Multi-label text classification (MLTC) addresses a fundamental problem in natural language processing, which assigns multiple relevant labels to each document. In recent years, Neural Network-based models (NN models) for MLTC have attracted much attention. In addition, NN models achieve favorable performances because they can exploit label correlations in the penultimate layer. To further capture and explore label correlations, we propose a novel initialization to incorporate label co-occurrence into NN models. First, we represent each class as a column vector of the weight matrix in the penultimate layer, which we name the class embedding matrix. Second, we deduce an equation for correlating the class embedding matrix with the label co-occurrence matrix, ensuring that relevant classes are denoted by vectors with large correlations. Finally, we provide a theoretical analysis of the equation, and propose an algorithm to calculate the initial values of the class embedding matrix from the label co-occurrence matrix. We evaluate our approach with various text extractors, such as Recurrent Neural Network (RNN), Convolutional Neural Network (CNN) and Transformer on four public datasets. The experimental results demonstrate that our approach markedly improves the performance of existing NN models. |
| first_indexed | 2024-12-22T19:44:37Z |
| format | Article |
| id | doaj.art-0930318045f34b509130b090075806bd |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-22T19:44:37Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-0930318045f34b509130b090075806bd2022-12-21T18:14:43ZengIEEEIEEE Access2169-35362019-01-01718358018358810.1109/ACCESS.2019.29606268936450Incorporating Label Co-Occurrence Into Neural Network-Based Models for Multi-Label Text ClassificationJiaqi Yao0https://orcid.org/0000-0002-7390-5156Keren Wang1https://orcid.org/0000-0002-5846-7119Jikun Yan2https://orcid.org/0000-0003-2600-968XNational Key Laboratory of Science and Technology on Blind Signal Processing, Chengdu, ChinaNational Key Laboratory of Science and Technology on Blind Signal Processing, Chengdu, ChinaNational Key Laboratory of Science and Technology on Blind Signal Processing, Chengdu, ChinaMulti-label text classification (MLTC) addresses a fundamental problem in natural language processing, which assigns multiple relevant labels to each document. In recent years, Neural Network-based models (NN models) for MLTC have attracted much attention. In addition, NN models achieve favorable performances because they can exploit label correlations in the penultimate layer. To further capture and explore label correlations, we propose a novel initialization to incorporate label co-occurrence into NN models. First, we represent each class as a column vector of the weight matrix in the penultimate layer, which we name the class embedding matrix. Second, we deduce an equation for correlating the class embedding matrix with the label co-occurrence matrix, ensuring that relevant classes are denoted by vectors with large correlations. Finally, we provide a theoretical analysis of the equation, and propose an algorithm to calculate the initial values of the class embedding matrix from the label co-occurrence matrix. We evaluate our approach with various text extractors, such as Recurrent Neural Network (RNN), Convolutional Neural Network (CNN) and Transformer on four public datasets. The experimental results demonstrate that our approach markedly improves the performance of existing NN models.https://ieeexplore.ieee.org/document/8936450/Multi-label text classificationlabel co-occurrenceinitializationneural networkclass embedding |
| spellingShingle | Jiaqi Yao Keren Wang Jikun Yan Incorporating Label Co-Occurrence Into Neural Network-Based Models for Multi-Label Text Classification IEEE Access Multi-label text classification label co-occurrence initialization neural network class embedding |
| title | Incorporating Label Co-Occurrence Into Neural Network-Based Models for Multi-Label Text Classification |
| title_full | Incorporating Label Co-Occurrence Into Neural Network-Based Models for Multi-Label Text Classification |
| title_fullStr | Incorporating Label Co-Occurrence Into Neural Network-Based Models for Multi-Label Text Classification |
| title_full_unstemmed | Incorporating Label Co-Occurrence Into Neural Network-Based Models for Multi-Label Text Classification |
| title_short | Incorporating Label Co-Occurrence Into Neural Network-Based Models for Multi-Label Text Classification |
| title_sort | incorporating label co occurrence into neural network based models for multi label text classification |
| topic | Multi-label text classification label co-occurrence initialization neural network class embedding |
| url | https://ieeexplore.ieee.org/document/8936450/ |
| work_keys_str_mv | AT jiaqiyao incorporatinglabelcooccurrenceintoneuralnetworkbasedmodelsformultilabeltextclassification AT kerenwang incorporatinglabelcooccurrenceintoneuralnetworkbasedmodelsformultilabeltextclassification AT jikunyan incorporatinglabelcooccurrenceintoneuralnetworkbasedmodelsformultilabeltextclassification |