Bayesian coreset construction via greedy iterative geodesic ascent
Coherent uncertainty quantification is a key strength of Bayesian methods. But modern algorithms for approximate Bayesian posterior inference often sacrifice accurate posterior uncertainty estimation in the pursuit of scalability. This work shows that previous Bayesian coreset construction algorithm...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
MIT Press
2020
|
| Online Access: | https://hdl.handle.net/1721.1/128781 |
| _version_ | 1826214205985914880 |
|---|---|
| author | Campbell, Trevor David Broderick, Tamara A |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Campbell, Trevor David Broderick, Tamara A |
| author_sort | Campbell, Trevor David |
| collection | MIT |
| description | Coherent uncertainty quantification is a key strength of Bayesian methods. But modern algorithms for approximate Bayesian posterior inference often sacrifice accurate posterior uncertainty estimation in the pursuit of scalability. This work shows that previous Bayesian coreset construction algorithms-which build a small, weighted subset of the data that approximates the full dataset-are no exception. We demonstrate that these algorithms scale the coreset log-likelihood suboptimally, resulting in underestimated posterior uncertainty. To address this shortcoming, we develop greedy iterative geodesic ascent (GIGA), a novel algorithm for Bayesian coreset construction that scales the coreset log-likelihood optimally. GIGA provides geometric decay in posterior approximation error as a function of coreset size, and maintains the fast running time of its predecessors. The paper concludes with validation of GIGA on both synthetic and real datasets, demonstrating that it reduces posterior approximation error by orders of magnitude compared with previous coreset constructions. |
| first_indexed | 2024-09-23T16:01:47Z |
| format | Article |
| id | mit-1721.1/128781 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T16:01:47Z |
| publishDate | 2020 |
| publisher | MIT Press |
| record_format | dspace |
| spelling | mit-1721.1/1287812022-09-29T17:44:40Z Bayesian coreset construction via greedy iterative geodesic ascent Campbell, Trevor David Broderick, Tamara A Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Coherent uncertainty quantification is a key strength of Bayesian methods. But modern algorithms for approximate Bayesian posterior inference often sacrifice accurate posterior uncertainty estimation in the pursuit of scalability. This work shows that previous Bayesian coreset construction algorithms-which build a small, weighted subset of the data that approximates the full dataset-are no exception. We demonstrate that these algorithms scale the coreset log-likelihood suboptimally, resulting in underestimated posterior uncertainty. To address this shortcoming, we develop greedy iterative geodesic ascent (GIGA), a novel algorithm for Bayesian coreset construction that scales the coreset log-likelihood optimally. GIGA provides geometric decay in posterior approximation error as a function of coreset size, and maintains the fast running time of its predecessors. The paper concludes with validation of GIGA on both synthetic and real datasets, demonstrating that it reduces posterior approximation error by orders of magnitude compared with previous coreset constructions. United States. Office of Naval Research (Grant N00014-17-1-2072) 2020-12-10T21:32:04Z 2020-12-10T21:32:04Z 2018-07 2020-12-03T17:59:09Z Article http://purl.org/eprint/type/ConferencePaper 1533-7928 1532-4435 https://hdl.handle.net/1721.1/128781 Campbell, Trevor and Tamara Broderick. “Bayesian coreset construction via greedy iterative geodesic ascent.” Proceedings of the 35th International Conference on Machine Learning, PMLR, 80 (July 2018): 698-706 © 2018 The Author(s) en Proceedings of the 35th International Conference on Machine Learning, PMLR Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf MIT Press arXiv |
| spellingShingle | Campbell, Trevor David Broderick, Tamara A Bayesian coreset construction via greedy iterative geodesic ascent |
| title | Bayesian coreset construction via greedy iterative geodesic ascent |
| title_full | Bayesian coreset construction via greedy iterative geodesic ascent |
| title_fullStr | Bayesian coreset construction via greedy iterative geodesic ascent |
| title_full_unstemmed | Bayesian coreset construction via greedy iterative geodesic ascent |
| title_short | Bayesian coreset construction via greedy iterative geodesic ascent |
| title_sort | bayesian coreset construction via greedy iterative geodesic ascent |
| url | https://hdl.handle.net/1721.1/128781 |
| work_keys_str_mv | AT campbelltrevordavid bayesiancoresetconstructionviagreedyiterativegeodesicascent AT brodericktamaraa bayesiancoresetconstructionviagreedyiterativegeodesicascent |