Neural Embedding Allocation: Distributed Representations of Topic Models
We propose a method that uses neural embeddings to improve the performance of any given LDA-style topic model. Our method, called neural embedding allocation (NEA), deconstructs topic models (LDA or otherwise) into interpretable vector-space embeddings of words, topics, documents, authors, and so on...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
The MIT Press
2022-08-01
|
| Series: | Computational Linguistics |
| Online Access: | http://dx.doi.org/10.1162/coli_a_00457 |
| _version_ | 1797795369511288832 |
|---|---|
| author | Kamrun Naher Keya Yannis Papanikolaou James R. Foulds |
| author_facet | Kamrun Naher Keya Yannis Papanikolaou James R. Foulds |
| author_sort | Kamrun Naher Keya |
| collection | DOAJ |
| description | We propose a method that uses neural embeddings to improve the performance of any given LDA-style topic model. Our method, called neural embedding allocation (NEA), deconstructs topic models (LDA or otherwise) into interpretable vector-space embeddings of words, topics, documents, authors, and so on, by learning neural embeddings to mimic the topic model. We demonstrate that NEA improves coherence scores of the original topic model by smoothing out the noisy topics when the number of topics is large. Furthermore, we show NEA’s effectiveness and generality in deconstructing and smoothing LDA, author-topic models, and the recent mixed membership skip-gram topic model and achieve better performance with the embeddings compared to several state-of-the-art models. |
| first_indexed | 2024-03-13T03:18:03Z |
| format | Article |
| id | doaj.art-4f518a22b5844cf6895a7258e5b704a9 |
| institution | Directory Open Access Journal |
| issn | 1530-9312 |
| language | English |
| last_indexed | 2024-03-13T03:18:03Z |
| publishDate | 2022-08-01 |
| publisher | The MIT Press |
| record_format | Article |
| series | Computational Linguistics |
| spelling | doaj.art-4f518a22b5844cf6895a7258e5b704a92023-06-25T14:50:05ZengThe MIT PressComputational Linguistics1530-93122022-08-0148410.1162/coli_a_00457Neural Embedding Allocation: Distributed Representations of Topic ModelsKamrun Naher KeyaYannis PapanikolaouJames R. FouldsWe propose a method that uses neural embeddings to improve the performance of any given LDA-style topic model. Our method, called neural embedding allocation (NEA), deconstructs topic models (LDA or otherwise) into interpretable vector-space embeddings of words, topics, documents, authors, and so on, by learning neural embeddings to mimic the topic model. We demonstrate that NEA improves coherence scores of the original topic model by smoothing out the noisy topics when the number of topics is large. Furthermore, we show NEA’s effectiveness and generality in deconstructing and smoothing LDA, author-topic models, and the recent mixed membership skip-gram topic model and achieve better performance with the embeddings compared to several state-of-the-art models.http://dx.doi.org/10.1162/coli_a_00457 |
| spellingShingle | Kamrun Naher Keya Yannis Papanikolaou James R. Foulds Neural Embedding Allocation: Distributed Representations of Topic Models Computational Linguistics |
| title | Neural Embedding Allocation: Distributed Representations of Topic Models |
| title_full | Neural Embedding Allocation: Distributed Representations of Topic Models |
| title_fullStr | Neural Embedding Allocation: Distributed Representations of Topic Models |
| title_full_unstemmed | Neural Embedding Allocation: Distributed Representations of Topic Models |
| title_short | Neural Embedding Allocation: Distributed Representations of Topic Models |
| title_sort | neural embedding allocation distributed representations of topic models |
| url | http://dx.doi.org/10.1162/coli_a_00457 |
| work_keys_str_mv | AT kamrunnaherkeya neuralembeddingallocationdistributedrepresentationsoftopicmodels AT yannispapanikolaou neuralembeddingallocationdistributedrepresentationsoftopicmodels AT jamesrfoulds neuralembeddingallocationdistributedrepresentationsoftopicmodels |