MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning
Efficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally efficient approximations. We showcase the usefulness of the pro...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-05-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/21/6/551 |
| _version_ | 1798039695756623872 |
|---|---|
| author | Diego Granziol Binxin Ru Stefan Zohren Xiaowen Dong Michael Osborne Stephen Roberts |
| author_facet | Diego Granziol Binxin Ru Stefan Zohren Xiaowen Dong Michael Osborne Stephen Roberts |
| author_sort | Diego Granziol |
| collection | DOAJ |
| description | Efficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally efficient approximations. We showcase the usefulness of the proposed method, its equivalence to constrained Bayesian variational inference and demonstrate its superiority over existing approaches in two applications, namely, fast log determinant estimation and information-theoretic Bayesian optimisation. |
| first_indexed | 2024-04-11T21:57:18Z |
| format | Article |
| id | doaj.art-14429d1f4da8433aac39e50f42fa7433 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-04-11T21:57:18Z |
| publishDate | 2019-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-14429d1f4da8433aac39e50f42fa74332022-12-22T04:01:03ZengMDPI AGEntropy1099-43002019-05-0121655110.3390/e21060551e21060551MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine LearningDiego Granziol0Binxin Ru1Stefan Zohren2Xiaowen Dong3Michael Osborne4Stephen Roberts5Machine Learning Research Group, University of Oxford, Walton Well Rd, Oxford OX2 6ED, UKMachine Learning Research Group, University of Oxford, Walton Well Rd, Oxford OX2 6ED, UKMachine Learning Research Group, University of Oxford, Walton Well Rd, Oxford OX2 6ED, UKMachine Learning Research Group, University of Oxford, Walton Well Rd, Oxford OX2 6ED, UKMachine Learning Research Group, University of Oxford, Walton Well Rd, Oxford OX2 6ED, UKMachine Learning Research Group, University of Oxford, Walton Well Rd, Oxford OX2 6ED, UKEfficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally efficient approximations. We showcase the usefulness of the proposed method, its equivalence to constrained Bayesian variational inference and demonstrate its superiority over existing approaches in two applications, namely, fast log determinant estimation and information-theoretic Bayesian optimisation.https://www.mdpi.com/1099-4300/21/6/551maximum entropylog determinant estimationBayesian optimisation |
| spellingShingle | Diego Granziol Binxin Ru Stefan Zohren Xiaowen Dong Michael Osborne Stephen Roberts MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning Entropy maximum entropy log determinant estimation Bayesian optimisation |
| title | MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning |
| title_full | MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning |
| title_fullStr | MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning |
| title_full_unstemmed | MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning |
| title_short | MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning |
| title_sort | meme an accurate maximum entropy method for efficient approximations in large scale machine learning |
| topic | maximum entropy log determinant estimation Bayesian optimisation |
| url | https://www.mdpi.com/1099-4300/21/6/551 |
| work_keys_str_mv | AT diegogranziol memeanaccuratemaximumentropymethodforefficientapproximationsinlargescalemachinelearning AT binxinru memeanaccuratemaximumentropymethodforefficientapproximationsinlargescalemachinelearning AT stefanzohren memeanaccuratemaximumentropymethodforefficientapproximationsinlargescalemachinelearning AT xiaowendong memeanaccuratemaximumentropymethodforefficientapproximationsinlargescalemachinelearning AT michaelosborne memeanaccuratemaximumentropymethodforefficientapproximationsinlargescalemachinelearning AT stephenroberts memeanaccuratemaximumentropymethodforefficientapproximationsinlargescalemachinelearning |