Summary: | The metareasoning framework aims to enable autonomous
agents to factor in planning costs when making decisions. In
this work, we develop the first non-myopic metareasoning
algorithm for planning with Markov decision processes. Our
method learns the behaviour of anytime probabilistic planning
algorithms from performance data. Specifically, we propose
a novel model for metareasoning, based on contextual performance profiles that predict the value of the planner’s current
solution given the time spent planning, the state of the planning algorithm’s internal parameters, and the difficulty of the
planning problem being solved. This model removes the need
to assume that the current solution quality is always known,
broadening the class of metareasoning problems that can be
addressed. We then employ deep reinforcement learning to
learn a policy that decides, at each timestep, whether to continue planning or start executing the current plan, and how to
set hyperparameters of the planner to enhance its performance.
We demonstrate our algorithm’s ability to perform effective
metareasoning in two domains.
|