| Summary: | <p>Probabilistic models enable us to infer the underlying relationships within data and make decisions based on this information. Certain models are more commonly used not because they are more appropriate to imitate a particular system, but because they are simple enough to analyze given the current computational resources and available inference algorithms. However, there are a broad range of non-standard models that we could use to describe real-world tasks better; they may have complex dependency structures, variables following non-common distributions, and/or a stochastic number of variables. But we typically avoid using them as these features substantially complicate the inference procedure such that many conventional inference algorithms would fail when dealing with these models.</p>
<p>To alleviate the computational concerns of data scientists or domain experts and free them to develop non-standard, complex models as needed, more sophisticated inference methods need to be more easily accessible. Additionally, because it is usually difficult to extract helpful information of non-standard models ahead of inference, like where the modes might be, such inference techniques should be expected to work reasonably well without too much such information. In an ideal world, domain experts would be able to specify any model of interest, and no longer need to worry about the technical details of inference algorithms or how to implement them. Moreover, these algorithms are provided as generic engines and can be used to reason the models in an automated manner. Achieving so is the ambitious, long-term goal of the emerging field called <b>probabilistic programming</b>.</p>
<p>The aim of this thesis is to make inroads to this ultimate goal: we develop and automate advanced inference methods that are applicable for a broad range of non-standard probabilistic models. In particular, we introduce a novel class of adaptive inference algorithms which can be implemented in a black-box manner. They are especially useful for the models with multiple, separated modes where many off-the-shelf options struggle. Furthermore, we investigate both a restricted class of probabilistic programming systems (PPSs) that impose strong constraints on the model class to ensure inference efficiency, and universal PPSs which are designed around the long-term goal and aim to support any possible model but substantially complicate the inference procedures. For the former, we propose a principled way to extend them to incorporate a broader range of models and inference engines that are not supported before. For the latter, we develop a general framework to handle one class of the most challenging models where the support of a model can be stochastic.</p>
|
|---|