Compressed models for co-reference resolution: enhancing efficiency with debiased word embeddings
Abstract This work presents a comprehensive approach to reduce bias in word embedding vectors and evaluate the impact on various Natural Language Processing (NLP) tasks. Two GloVe variations (840B and 50) are debiased by identifying the gender direction in the word embedding space and then removing...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-10-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-45677-0 |
| _version_ | 1797647343498035200 |
|---|---|
| author | Georgios Ioannides Aishwarya Jadhav Aditi Sharma Samarth Navali Alan W. Black |
| author_facet | Georgios Ioannides Aishwarya Jadhav Aditi Sharma Samarth Navali Alan W. Black |
| author_sort | Georgios Ioannides |
| collection | DOAJ |
| description | Abstract This work presents a comprehensive approach to reduce bias in word embedding vectors and evaluate the impact on various Natural Language Processing (NLP) tasks. Two GloVe variations (840B and 50) are debiased by identifying the gender direction in the word embedding space and then removing or reducing the gender component from the embeddings of target words, while preserving useful semantic information. Their gender bias is assessed through the Word Embedding Association Test. The performance of co-reference resolution and text classification models trained on both original and debiased embeddings is evaluated in terms of accuracy. A compressed co-reference resolution model is examined to gauge the effectiveness of debiasing techniques on resource-efficient models. To the best of the authors’ knowledge, this is the first attempt to apply compression techniques to debiased models. By analyzing the context preservation of debiased embeddings using a Twitter misinformation dataset, this study contributes valuable insights into the practical implications of debiasing methods for real-world applications such as person profiling. |
| first_indexed | 2024-03-11T15:14:54Z |
| format | Article |
| id | doaj.art-87033d2a03fd4e73b5043d2b0f03e7db |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-03-11T15:14:54Z |
| publishDate | 2023-10-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-87033d2a03fd4e73b5043d2b0f03e7db2023-10-29T12:22:58ZengNature PortfolioScientific Reports2045-23222023-10-011311910.1038/s41598-023-45677-0Compressed models for co-reference resolution: enhancing efficiency with debiased word embeddingsGeorgios Ioannides0Aishwarya Jadhav1Aditi Sharma2Samarth Navali3Alan W. Black4Language Technologies Institute, Carnegie Mellon UniversityLanguage Technologies Institute, Carnegie Mellon UniversityLanguage Technologies Institute, Carnegie Mellon UniversityLanguage Technologies Institute, Carnegie Mellon UniversityLanguage Technologies Institute, Carnegie Mellon UniversityAbstract This work presents a comprehensive approach to reduce bias in word embedding vectors and evaluate the impact on various Natural Language Processing (NLP) tasks. Two GloVe variations (840B and 50) are debiased by identifying the gender direction in the word embedding space and then removing or reducing the gender component from the embeddings of target words, while preserving useful semantic information. Their gender bias is assessed through the Word Embedding Association Test. The performance of co-reference resolution and text classification models trained on both original and debiased embeddings is evaluated in terms of accuracy. A compressed co-reference resolution model is examined to gauge the effectiveness of debiasing techniques on resource-efficient models. To the best of the authors’ knowledge, this is the first attempt to apply compression techniques to debiased models. By analyzing the context preservation of debiased embeddings using a Twitter misinformation dataset, this study contributes valuable insights into the practical implications of debiasing methods for real-world applications such as person profiling.https://doi.org/10.1038/s41598-023-45677-0 |
| spellingShingle | Georgios Ioannides Aishwarya Jadhav Aditi Sharma Samarth Navali Alan W. Black Compressed models for co-reference resolution: enhancing efficiency with debiased word embeddings Scientific Reports |
| title | Compressed models for co-reference resolution: enhancing efficiency with debiased word embeddings |
| title_full | Compressed models for co-reference resolution: enhancing efficiency with debiased word embeddings |
| title_fullStr | Compressed models for co-reference resolution: enhancing efficiency with debiased word embeddings |
| title_full_unstemmed | Compressed models for co-reference resolution: enhancing efficiency with debiased word embeddings |
| title_short | Compressed models for co-reference resolution: enhancing efficiency with debiased word embeddings |
| title_sort | compressed models for co reference resolution enhancing efficiency with debiased word embeddings |
| url | https://doi.org/10.1038/s41598-023-45677-0 |
| work_keys_str_mv | AT georgiosioannides compressedmodelsforcoreferenceresolutionenhancingefficiencywithdebiasedwordembeddings AT aishwaryajadhav compressedmodelsforcoreferenceresolutionenhancingefficiencywithdebiasedwordembeddings AT aditisharma compressedmodelsforcoreferenceresolutionenhancingefficiencywithdebiasedwordembeddings AT samarthnavali compressedmodelsforcoreferenceresolutionenhancingefficiencywithdebiasedwordembeddings AT alanwblack compressedmodelsforcoreferenceresolutionenhancingefficiencywithdebiasedwordembeddings |