Stochastic collapsed variational inference for hidden Markov models
Stochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm for hidden Markov models, in a sequential data setting. Given a collapsed hidden Markov Model,...
| Váldodahkkit: | , |
|---|---|
| Materiálatiipa: | Conference item |
| Almmustuhtton: |
Neural Information Processing Systems
2015
|
| _version_ | 1826286425209831424 |
|---|---|
| author | Wang, P Blunsom, P |
| author_facet | Wang, P Blunsom, P |
| author_sort | Wang, P |
| collection | OXFORD |
| description | Stochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm for hidden Markov models, in a sequential data setting. Given a collapsed hidden Markov Model, we break its long Markov chain into a set of short subchains. We propose a novel sum-product algorithm to update the posteriors of the subchains, taking into account their boundary transitions due to the sequential dependencies. Our experiments on two discrete datasets show that our collapsed algorithm is scalable to very large datasets, memory efficient and significantly more accurate than the existing uncollapsed algorithm. |
| first_indexed | 2024-03-07T01:43:34Z |
| format | Conference item |
| id | oxford-uuid:97a6319e-04cd-467d-a3c1-96e3c81c0d0c |
| institution | University of Oxford |
| last_indexed | 2024-03-07T01:43:34Z |
| publishDate | 2015 |
| publisher | Neural Information Processing Systems |
| record_format | dspace |
| spelling | oxford-uuid:97a6319e-04cd-467d-a3c1-96e3c81c0d0c2022-03-27T00:01:25ZStochastic collapsed variational inference for hidden Markov modelsConference itemhttp://purl.org/coar/resource_type/c_5794uuid:97a6319e-04cd-467d-a3c1-96e3c81c0d0cSymplectic Elements at OxfordNeural Information Processing Systems2015Wang, PBlunsom, PStochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm for hidden Markov models, in a sequential data setting. Given a collapsed hidden Markov Model, we break its long Markov chain into a set of short subchains. We propose a novel sum-product algorithm to update the posteriors of the subchains, taking into account their boundary transitions due to the sequential dependencies. Our experiments on two discrete datasets show that our collapsed algorithm is scalable to very large datasets, memory efficient and significantly more accurate than the existing uncollapsed algorithm. |
| spellingShingle | Wang, P Blunsom, P Stochastic collapsed variational inference for hidden Markov models |
| title | Stochastic collapsed variational inference for hidden Markov models |
| title_full | Stochastic collapsed variational inference for hidden Markov models |
| title_fullStr | Stochastic collapsed variational inference for hidden Markov models |
| title_full_unstemmed | Stochastic collapsed variational inference for hidden Markov models |
| title_short | Stochastic collapsed variational inference for hidden Markov models |
| title_sort | stochastic collapsed variational inference for hidden markov models |
| work_keys_str_mv | AT wangp stochasticcollapsedvariationalinferenceforhiddenmarkovmodels AT blunsomp stochasticcollapsedvariationalinferenceforhiddenmarkovmodels |