Collapsed variational inference for HDP
A wide variety of Dirichlet-multinomial 'topic' models have found interesting applications in recent years. While Gibbs sampling remains an important method of inference in such models, variational techniques have certain advantages such as easy assessment of convergence, easy optimization...
| Main Authors: | , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
2009
|
| _version_ | 1826290593746124800 |
|---|---|
| author | Teh, Y Kurihara, K Welling, M |
| author_facet | Teh, Y Kurihara, K Welling, M |
| author_sort | Teh, Y |
| collection | OXFORD |
| description | A wide variety of Dirichlet-multinomial 'topic' models have found interesting applications in recent years. While Gibbs sampling remains an important method of inference in such models, variational techniques have certain advantages such as easy assessment of convergence, easy optimization without the need to maintain detailed balance, a bound on the marginal likelihood, and side-stepping of issues with topic-identifiability. The most accurate variational technique thus far, namely collapsed variational latent Dirichlet allocation, did not deal with model selection nor did it include inference for hyperparameters. We address both issues by generalizing the technique, obtaining the first variational algorithm to deal with the hierarchical Dirichlet process and to deal with hyperparameters of Dirichlet variables. Experiments show a significant improvement in accuracy. |
| first_indexed | 2024-03-07T02:46:34Z |
| format | Journal article |
| id | oxford-uuid:ac3f313c-1899-43a3-9d70-777c1f0cd680 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T02:46:34Z |
| publishDate | 2009 |
| record_format | dspace |
| spelling | oxford-uuid:ac3f313c-1899-43a3-9d70-777c1f0cd6802022-03-27T03:27:36ZCollapsed variational inference for HDPJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:ac3f313c-1899-43a3-9d70-777c1f0cd680EnglishSymplectic Elements at Oxford2009Teh, YKurihara, KWelling, MA wide variety of Dirichlet-multinomial 'topic' models have found interesting applications in recent years. While Gibbs sampling remains an important method of inference in such models, variational techniques have certain advantages such as easy assessment of convergence, easy optimization without the need to maintain detailed balance, a bound on the marginal likelihood, and side-stepping of issues with topic-identifiability. The most accurate variational technique thus far, namely collapsed variational latent Dirichlet allocation, did not deal with model selection nor did it include inference for hyperparameters. We address both issues by generalizing the technique, obtaining the first variational algorithm to deal with the hierarchical Dirichlet process and to deal with hyperparameters of Dirichlet variables. Experiments show a significant improvement in accuracy. |
| spellingShingle | Teh, Y Kurihara, K Welling, M Collapsed variational inference for HDP |
| title | Collapsed variational inference for HDP |
| title_full | Collapsed variational inference for HDP |
| title_fullStr | Collapsed variational inference for HDP |
| title_full_unstemmed | Collapsed variational inference for HDP |
| title_short | Collapsed variational inference for HDP |
| title_sort | collapsed variational inference for hdp |
| work_keys_str_mv | AT tehy collapsedvariationalinferenceforhdp AT kuriharak collapsedvariationalinferenceforhdp AT wellingm collapsedvariationalinferenceforhdp |