Memorized Variational Continual Learning for Dirichlet Process Mixtures
Bayesian nonparametric models are theoretically suitable for streaming data due to their ability to adapt model complexity with the observed data. However, very limited work has addressed posterior inference in a streaming fashion, and most of the existing variational inference algorithms require tr...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8871157/ |
| _version_ | 1819174683735490560 |
|---|---|
| author | Yang Yang Bo Chen Hongwei Liu |
| author_facet | Yang Yang Bo Chen Hongwei Liu |
| author_sort | Yang Yang |
| collection | DOAJ |
| description | Bayesian nonparametric models are theoretically suitable for streaming data due to their ability to adapt model complexity with the observed data. However, very limited work has addressed posterior inference in a streaming fashion, and most of the existing variational inference algorithms require truncation on variational distributions which cannot vary with the data. In this paper, we focus Dirichlet process mixture models and develop the corresponding variational continual learning approach by maintaining memorized sufficient statistics for previous tasks, called memorized variational continual learning (MVCL), which is able to handle both the posterior update and data in a continual learning setting. Furthermore, we extend MVCL for two cases of mixture models which can handle different data types. The experiments demonstrate the comparable inference capability of our MVCL for both discrete and real-valued datasets with automatically inferring the number of mixture components. |
| first_indexed | 2024-12-22T20:42:52Z |
| format | Article |
| id | doaj.art-2044faff9baf4dc4a306559189d139c4 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-22T20:42:52Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-2044faff9baf4dc4a306559189d139c42022-12-21T18:13:18ZengIEEEIEEE Access2169-35362019-01-01715085115086210.1109/ACCESS.2019.29477228871157Memorized Variational Continual Learning for Dirichlet Process MixturesYang Yang0https://orcid.org/0000-0002-8851-1790Bo Chen1Hongwei Liu2National Laboratory of Radar Signal Processing, Xidian University, Xi’an, ChinaNational Laboratory of Radar Signal Processing, Xidian University, Xi’an, ChinaNational Laboratory of Radar Signal Processing, Xidian University, Xi’an, ChinaBayesian nonparametric models are theoretically suitable for streaming data due to their ability to adapt model complexity with the observed data. However, very limited work has addressed posterior inference in a streaming fashion, and most of the existing variational inference algorithms require truncation on variational distributions which cannot vary with the data. In this paper, we focus Dirichlet process mixture models and develop the corresponding variational continual learning approach by maintaining memorized sufficient statistics for previous tasks, called memorized variational continual learning (MVCL), which is able to handle both the posterior update and data in a continual learning setting. Furthermore, we extend MVCL for two cases of mixture models which can handle different data types. The experiments demonstrate the comparable inference capability of our MVCL for both discrete and real-valued datasets with automatically inferring the number of mixture components.https://ieeexplore.ieee.org/document/8871157/Bayesian nonparametricstreaming datavariational continual learningDirichlet process mixturememorized sufficient statisticsdiscrete and real-valued datasets |
| spellingShingle | Yang Yang Bo Chen Hongwei Liu Memorized Variational Continual Learning for Dirichlet Process Mixtures IEEE Access Bayesian nonparametric streaming data variational continual learning Dirichlet process mixture memorized sufficient statistics discrete and real-valued datasets |
| title | Memorized Variational Continual Learning for Dirichlet Process Mixtures |
| title_full | Memorized Variational Continual Learning for Dirichlet Process Mixtures |
| title_fullStr | Memorized Variational Continual Learning for Dirichlet Process Mixtures |
| title_full_unstemmed | Memorized Variational Continual Learning for Dirichlet Process Mixtures |
| title_short | Memorized Variational Continual Learning for Dirichlet Process Mixtures |
| title_sort | memorized variational continual learning for dirichlet process mixtures |
| topic | Bayesian nonparametric streaming data variational continual learning Dirichlet process mixture memorized sufficient statistics discrete and real-valued datasets |
| url | https://ieeexplore.ieee.org/document/8871157/ |
| work_keys_str_mv | AT yangyang memorizedvariationalcontinuallearningfordirichletprocessmixtures AT bochen memorizedvariationalcontinuallearningfordirichletprocessmixtures AT hongweiliu memorizedvariationalcontinuallearningfordirichletprocessmixtures |