| Summary: | <p>Optimization and machine learning are both extremely active research topics. In
this thesis, we explore problems at the intersection of the two fields. In particular,
we will develop two main ideas.</p>
<p>First, optimization can be used to improve machine learning. We illustrate this
idea by considering computer vision tasks that are modelled with dense conditional
random fields. Existing solvers for these models are either slow or inaccurate. We
show that, by introducing a specialized solver based on proximal minimization and
fast filtering, these models can be solved both quickly and accurately. Similarly, we
introduce a specialized linear programming solver for block bounded problems, a
common class of problems encountered in machine learning. This solver is efficient,
easy to tune and simple to integrate inside larger machine learning algorithms.</p>
<p>Second, machine learning can be used to improve optimization, in particular for
NP-hard problems. For problems solved by using hand-tuned heuristics, machine
learning can be used to discover and improve these heuristics. We show that,
for the problem of super-optimization, a better heuristic to explore the space of
programs can be learnt using reinforcement learning. For problems where no such
heuristics exist, machine learning can be used to get an approximate solution of the
original problem. We use this idea to tackle the problem of program synthesis by
reformulating it as the problem of learning a program that performs the required
task. We introduce a new differentiable formulation of the execution and show that
the fastest programs can be recovered for simple tasks.
</p>
|
|---|