MLRNet: A Meta-Loss Reweighting Network for Biased Data on Text Classification
Artificially generated datasets often exhibit biases, leading conventional deep neural networks to overfit. Typically, a weighted function adjusts sample impact during model updates using weighted loss. Meta-neural networks, trained with meta-learning principles, generalize well across tasks, acquir...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/14/1/164 |
| _version_ | 1797359172836130816 |
|---|---|
| author | Hao Yu Xinfu Li |
| author_facet | Hao Yu Xinfu Li |
| author_sort | Hao Yu |
| collection | DOAJ |
| description | Artificially generated datasets often exhibit biases, leading conventional deep neural networks to overfit. Typically, a weighted function adjusts sample impact during model updates using weighted loss. Meta-neural networks, trained with meta-learning principles, generalize well across tasks, acquiring generalized weights. This enables the self-generation of tailored weighted functions for data biases. However, datasets may simultaneously exhibit imbalanced classes and corrupted labels, posing a challenge for current meta-models. To address this, this paper presents Meta-Loss Reweighting Network (MLRNet) with fusion attention features. MLRNet continually evolves sample loss values, integrating them with sample features from self-attention layers in a semantic space. This enhances discriminative power for biased samples. By employing minimal unbiased meta-data for guidance, mutual optimization between the classifier and the meta-model is conducted, endowing biased samples with more reasonable weights. Experiments on English and Chinese benchmark datasets including artificial and real-world biased data show MLRNet’s superior performance under biased data conditions. |
| first_indexed | 2024-03-08T15:12:49Z |
| format | Article |
| id | doaj.art-e21f3f5e0aca42d785a46880e2dea3c4 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-08T15:12:49Z |
| publishDate | 2023-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-e21f3f5e0aca42d785a46880e2dea3c42024-01-10T14:51:11ZengMDPI AGApplied Sciences2076-34172023-12-0114116410.3390/app14010164MLRNet: A Meta-Loss Reweighting Network for Biased Data on Text ClassificationHao Yu0Xinfu Li1School of Cyber Security and Computer, Hebei University, Baoding 071002, ChinaSchool of Cyber Security and Computer, Hebei University, Baoding 071002, ChinaArtificially generated datasets often exhibit biases, leading conventional deep neural networks to overfit. Typically, a weighted function adjusts sample impact during model updates using weighted loss. Meta-neural networks, trained with meta-learning principles, generalize well across tasks, acquiring generalized weights. This enables the self-generation of tailored weighted functions for data biases. However, datasets may simultaneously exhibit imbalanced classes and corrupted labels, posing a challenge for current meta-models. To address this, this paper presents Meta-Loss Reweighting Network (MLRNet) with fusion attention features. MLRNet continually evolves sample loss values, integrating them with sample features from self-attention layers in a semantic space. This enhances discriminative power for biased samples. By employing minimal unbiased meta-data for guidance, mutual optimization between the classifier and the meta-model is conducted, endowing biased samples with more reasonable weights. Experiments on English and Chinese benchmark datasets including artificial and real-world biased data show MLRNet’s superior performance under biased data conditions.https://www.mdpi.com/2076-3417/14/1/164biased dataclass imbalancecorrupted labelmeta-learningself-attentiontext classification |
| spellingShingle | Hao Yu Xinfu Li MLRNet: A Meta-Loss Reweighting Network for Biased Data on Text Classification Applied Sciences biased data class imbalance corrupted label meta-learning self-attention text classification |
| title | MLRNet: A Meta-Loss Reweighting Network for Biased Data on Text Classification |
| title_full | MLRNet: A Meta-Loss Reweighting Network for Biased Data on Text Classification |
| title_fullStr | MLRNet: A Meta-Loss Reweighting Network for Biased Data on Text Classification |
| title_full_unstemmed | MLRNet: A Meta-Loss Reweighting Network for Biased Data on Text Classification |
| title_short | MLRNet: A Meta-Loss Reweighting Network for Biased Data on Text Classification |
| title_sort | mlrnet a meta loss reweighting network for biased data on text classification |
| topic | biased data class imbalance corrupted label meta-learning self-attention text classification |
| url | https://www.mdpi.com/2076-3417/14/1/164 |
| work_keys_str_mv | AT haoyu mlrnetametalossreweightingnetworkforbiaseddataontextclassification AT xinfuli mlrnetametalossreweightingnetworkforbiaseddataontextclassification |