Unsupervised detection of contextualized embedding bias with application to ideology
We propose a fully unsupervised method to detect bias in contextualized embeddings. The method leverages the assortative information latently encoded by social networks and combines orthogonality regularization, structured sparsity learning, and graph neural networks to find the embedding subspace c...
| Glavni autori: | , , |
|---|---|
| Format: | Conference item |
| Jezik: | English |
| Izdano: |
Journal of Machine Learning Research
2022
|
| _version_ | 1826309038032289792 |
|---|---|
| author | Hofmann, V Pierrehumbert, J Schütze, H |
| author_facet | Hofmann, V Pierrehumbert, J Schütze, H |
| author_sort | Hofmann, V |
| collection | OXFORD |
| description | We propose a fully unsupervised method to detect bias in contextualized embeddings. The method leverages the assortative information latently encoded by social networks and combines orthogonality regularization, structured sparsity learning, and graph neural networks to find the embedding subspace capturing this information. As a concrete example, we focus on the phenomenon of ideological bias: we introduce the concept of an ideological subspace, show how it can be found by applying our method to online discussion forums, and present techniques to probe it. Our experiments suggest that the ideological subspace encodes abstract evaluative semantics and reflects changes in the political left-right spectrum during the presidency of Donald Trump. |
| first_indexed | 2024-03-07T07:29:45Z |
| format | Conference item |
| id | oxford-uuid:ff2aa4fa-fbe2-4b2d-9d76-1a7fbec6f20e |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T07:29:45Z |
| publishDate | 2022 |
| publisher | Journal of Machine Learning Research |
| record_format | dspace |
| spelling | oxford-uuid:ff2aa4fa-fbe2-4b2d-9d76-1a7fbec6f20e2022-12-12T11:44:59ZUnsupervised detection of contextualized embedding bias with application to ideologyConference itemhttp://purl.org/coar/resource_type/c_5794uuid:ff2aa4fa-fbe2-4b2d-9d76-1a7fbec6f20eEnglishSymplectic ElementsJournal of Machine Learning Research2022Hofmann, VPierrehumbert, JSchütze, HWe propose a fully unsupervised method to detect bias in contextualized embeddings. The method leverages the assortative information latently encoded by social networks and combines orthogonality regularization, structured sparsity learning, and graph neural networks to find the embedding subspace capturing this information. As a concrete example, we focus on the phenomenon of ideological bias: we introduce the concept of an ideological subspace, show how it can be found by applying our method to online discussion forums, and present techniques to probe it. Our experiments suggest that the ideological subspace encodes abstract evaluative semantics and reflects changes in the political left-right spectrum during the presidency of Donald Trump. |
| spellingShingle | Hofmann, V Pierrehumbert, J Schütze, H Unsupervised detection of contextualized embedding bias with application to ideology |
| title | Unsupervised detection of contextualized embedding bias with application to ideology |
| title_full | Unsupervised detection of contextualized embedding bias with application to ideology |
| title_fullStr | Unsupervised detection of contextualized embedding bias with application to ideology |
| title_full_unstemmed | Unsupervised detection of contextualized embedding bias with application to ideology |
| title_short | Unsupervised detection of contextualized embedding bias with application to ideology |
| title_sort | unsupervised detection of contextualized embedding bias with application to ideology |
| work_keys_str_mv | AT hofmannv unsuperviseddetectionofcontextualizedembeddingbiaswithapplicationtoideology AT pierrehumbertj unsuperviseddetectionofcontextualizedembeddingbiaswithapplicationtoideology AT schutzeh unsuperviseddetectionofcontextualizedembeddingbiaswithapplicationtoideology |