Multimodal probabilistic reasoning for prediction and coordination problems in machine learning
<p>In this thesis we consider the role of multimodality in decision making and coordination problems in machine learning. We propose the use of a series of multimodal probabilistic methods, using extensions of (finite) mixture models to tackle challenges in time series forecasting, efficient u...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
2021
|
| Subjects: |
| _version_ | 1817931812368809984 |
|---|---|
| author | Zand, J |
| author2 | Roberts, S |
| author_facet | Roberts, S Zand, J |
| author_sort | Zand, J |
| collection | OXFORD |
| description | <p>In this thesis we consider the role of multimodality in decision making and coordination problems in machine learning. We propose the use of a series of multimodal probabilistic methods, using extensions of (finite) mixture models to tackle challenges in time series forecasting, efficient uncertainty quantification in neural networks, adversarial models and multi-agent coordination.</p>
<p>In the first part of the thesis, we focus on the usage of multimodal uncertainty estimation for time series forecasting, showing that such approaches offer tractable, beneficial alternatives to point estimation methods, which remains a prevalent method of choice for prediction. We discuss the significance of multimodal uncertainty and show the necessity for more adept approaches to estimate posterior target distributions. We present a series of computationally efficient, yet capable, methods for estimating rich multimodal posterior distributions. We compare our models to techniques that estimate uncertainty with point measures or a unimodal distribution and conclude this part with an extension of the approaches developed, inspired by generative adversarial networks. We show that this method provides state-of-the-art robustness to additive noise, making it particularly suitable for data sets in which unknown stochastics abound.</p>
<p>In the second part of this work, we investigate the importance of multi-modal models for Cooperative Multi-Agent Systems (CMASs), extending our work to take a probabilistic approach. To date, the majority of research in this area has been confined to considering a self-play paradigm, even if the approaches tackle a variety of challenging problems. Whilst these advances are significant, the use of arbitrary conventions in self-play leads to coordination problems when agents play outside this setting. We consider ad-hoc CMAS settings, moving away from the self-play framework. This is a particularly challenging area in machine learning and one that has attracted significant attention in recent years, offering the promise of AI agents able to interact effectively with humans (and other agents) in real-world settings. We tackle the problem of ad-hoc coordination by formulating posterior beliefs over other agents' strategies. This is achieved using an extension of Gibbs sampling to obtain close-to-optimal ad-hoc performance. We test our algorithm on the challenging game of Hanabi, one of the most prominent testbeds for cooperative multi-agent reinforcement learning and one which has gained momentum as a benchmark in recent years. We show that our method can achieve strong cross-play even with unseen partners, achieving successful ad-hoc coordination without up-front knowledge of the partners' strategies.</p> |
| first_indexed | 2024-03-07T07:10:11Z |
| format | Thesis |
| id | oxford-uuid:08a7e25c-af77-43b7-9ccd-e1346a2af5c9 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-12-09T03:27:58Z |
| publishDate | 2021 |
| record_format | dspace |
| spelling | oxford-uuid:08a7e25c-af77-43b7-9ccd-e1346a2af5c92024-12-01T10:53:37ZMultimodal probabilistic reasoning for prediction and coordination problems in machine learningThesishttp://purl.org/coar/resource_type/c_db06uuid:08a7e25c-af77-43b7-9ccd-e1346a2af5c9Machine learningEnglishHyrax Deposit2021Zand, JRoberts, S<p>In this thesis we consider the role of multimodality in decision making and coordination problems in machine learning. We propose the use of a series of multimodal probabilistic methods, using extensions of (finite) mixture models to tackle challenges in time series forecasting, efficient uncertainty quantification in neural networks, adversarial models and multi-agent coordination.</p> <p>In the first part of the thesis, we focus on the usage of multimodal uncertainty estimation for time series forecasting, showing that such approaches offer tractable, beneficial alternatives to point estimation methods, which remains a prevalent method of choice for prediction. We discuss the significance of multimodal uncertainty and show the necessity for more adept approaches to estimate posterior target distributions. We present a series of computationally efficient, yet capable, methods for estimating rich multimodal posterior distributions. We compare our models to techniques that estimate uncertainty with point measures or a unimodal distribution and conclude this part with an extension of the approaches developed, inspired by generative adversarial networks. We show that this method provides state-of-the-art robustness to additive noise, making it particularly suitable for data sets in which unknown stochastics abound.</p> <p>In the second part of this work, we investigate the importance of multi-modal models for Cooperative Multi-Agent Systems (CMASs), extending our work to take a probabilistic approach. To date, the majority of research in this area has been confined to considering a self-play paradigm, even if the approaches tackle a variety of challenging problems. Whilst these advances are significant, the use of arbitrary conventions in self-play leads to coordination problems when agents play outside this setting. We consider ad-hoc CMAS settings, moving away from the self-play framework. This is a particularly challenging area in machine learning and one that has attracted significant attention in recent years, offering the promise of AI agents able to interact effectively with humans (and other agents) in real-world settings. We tackle the problem of ad-hoc coordination by formulating posterior beliefs over other agents' strategies. This is achieved using an extension of Gibbs sampling to obtain close-to-optimal ad-hoc performance. We test our algorithm on the challenging game of Hanabi, one of the most prominent testbeds for cooperative multi-agent reinforcement learning and one which has gained momentum as a benchmark in recent years. We show that our method can achieve strong cross-play even with unseen partners, achieving successful ad-hoc coordination without up-front knowledge of the partners' strategies.</p> |
| spellingShingle | Machine learning Zand, J Multimodal probabilistic reasoning for prediction and coordination problems in machine learning |
| title | Multimodal probabilistic reasoning for prediction and coordination problems in machine learning |
| title_full | Multimodal probabilistic reasoning for prediction and coordination problems in machine learning |
| title_fullStr | Multimodal probabilistic reasoning for prediction and coordination problems in machine learning |
| title_full_unstemmed | Multimodal probabilistic reasoning for prediction and coordination problems in machine learning |
| title_short | Multimodal probabilistic reasoning for prediction and coordination problems in machine learning |
| title_sort | multimodal probabilistic reasoning for prediction and coordination problems in machine learning |
| topic | Machine learning |
| work_keys_str_mv | AT zandj multimodalprobabilisticreasoningforpredictionandcoordinationproblemsinmachinelearning |