Quantum-Enhanced Reinforcement Learning for Finite-Episode Games with Discrete State Spaces
Quantum annealing algorithms belong to the class of metaheuristic tools, applicable for solving binary optimization problems. Hardware implementations of quantum annealing, such as the quantum annealing machines produced by D-Wave Systems [1], have been subject to multiple analyses in research, with...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2018-02-01
|
| Series: | Frontiers in Physics |
| Subjects: | |
| Online Access: | http://journal.frontiersin.org/article/10.3389/fphy.2017.00071/full |
| _version_ | 1819176324714987520 |
|---|---|
| author | Florian Neukart David Von Dollen Christian Seidel Gabriele Compostella |
| author_facet | Florian Neukart David Von Dollen Christian Seidel Gabriele Compostella |
| author_sort | Florian Neukart |
| collection | DOAJ |
| description | Quantum annealing algorithms belong to the class of metaheuristic tools, applicable for solving binary optimization problems. Hardware implementations of quantum annealing, such as the quantum annealing machines produced by D-Wave Systems [1], have been subject to multiple analyses in research, with the aim of characterizing the technology's usefulness for optimization and sampling tasks [2–16]. Here, we present a way to partially embed both Monte Carlo policy iteration for finding an optimal policy on random observations, as well as how to embed n sub-optimal state-value functions for approximating an improved state-value function given a policy for finite horizon games with discrete state spaces on a D-Wave 2000Q quantum processing unit (QPU). We explain how both problems can be expressed as a quadratic unconstrained binary optimization (QUBO) problem, and show that quantum-enhanced Monte Carlo policy evaluation allows for finding equivalent or better state-value functions for a given policy with the same number episodes compared to a purely classical Monte Carlo algorithm. Additionally, we describe a quantum-classical policy learning algorithm. Our first and foremost aim is to explain how to represent and solve parts of these problems with the help of the QPU, and not to prove supremacy over every existing classical policy evaluation algorithm. |
| first_indexed | 2024-12-22T21:08:57Z |
| format | Article |
| id | doaj.art-d30aff00d1c84d068c253e39f400d5b2 |
| institution | Directory Open Access Journal |
| issn | 2296-424X |
| language | English |
| last_indexed | 2024-12-22T21:08:57Z |
| publishDate | 2018-02-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Physics |
| spelling | doaj.art-d30aff00d1c84d068c253e39f400d5b22022-12-21T18:12:35ZengFrontiers Media S.A.Frontiers in Physics2296-424X2018-02-01510.3389/fphy.2017.00071308956Quantum-Enhanced Reinforcement Learning for Finite-Episode Games with Discrete State SpacesFlorian Neukart0David Von Dollen1Christian Seidel2Gabriele Compostella3Volkswagen Group of America, Herndon, VA, United StatesVolkswagen Group of America, Herndon, VA, United StatesVolkswagen Data:Lab, Wolfsburg, GermanyVolkswagen Data:Lab, Wolfsburg, GermanyQuantum annealing algorithms belong to the class of metaheuristic tools, applicable for solving binary optimization problems. Hardware implementations of quantum annealing, such as the quantum annealing machines produced by D-Wave Systems [1], have been subject to multiple analyses in research, with the aim of characterizing the technology's usefulness for optimization and sampling tasks [2–16]. Here, we present a way to partially embed both Monte Carlo policy iteration for finding an optimal policy on random observations, as well as how to embed n sub-optimal state-value functions for approximating an improved state-value function given a policy for finite horizon games with discrete state spaces on a D-Wave 2000Q quantum processing unit (QPU). We explain how both problems can be expressed as a quadratic unconstrained binary optimization (QUBO) problem, and show that quantum-enhanced Monte Carlo policy evaluation allows for finding equivalent or better state-value functions for a given policy with the same number episodes compared to a purely classical Monte Carlo algorithm. Additionally, we describe a quantum-classical policy learning algorithm. Our first and foremost aim is to explain how to represent and solve parts of these problems with the help of the QPU, and not to prove supremacy over every existing classical policy evaluation algorithm.http://journal.frontiersin.org/article/10.3389/fphy.2017.00071/fullquantum annealingquantum computingreinforcement learningquantum-enhanced algorithmsquantum-classical |
| spellingShingle | Florian Neukart David Von Dollen Christian Seidel Gabriele Compostella Quantum-Enhanced Reinforcement Learning for Finite-Episode Games with Discrete State Spaces Frontiers in Physics quantum annealing quantum computing reinforcement learning quantum-enhanced algorithms quantum-classical |
| title | Quantum-Enhanced Reinforcement Learning for Finite-Episode Games with Discrete State Spaces |
| title_full | Quantum-Enhanced Reinforcement Learning for Finite-Episode Games with Discrete State Spaces |
| title_fullStr | Quantum-Enhanced Reinforcement Learning for Finite-Episode Games with Discrete State Spaces |
| title_full_unstemmed | Quantum-Enhanced Reinforcement Learning for Finite-Episode Games with Discrete State Spaces |
| title_short | Quantum-Enhanced Reinforcement Learning for Finite-Episode Games with Discrete State Spaces |
| title_sort | quantum enhanced reinforcement learning for finite episode games with discrete state spaces |
| topic | quantum annealing quantum computing reinforcement learning quantum-enhanced algorithms quantum-classical |
| url | http://journal.frontiersin.org/article/10.3389/fphy.2017.00071/full |
| work_keys_str_mv | AT florianneukart quantumenhancedreinforcementlearningforfiniteepisodegameswithdiscretestatespaces AT davidvondollen quantumenhancedreinforcementlearningforfiniteepisodegameswithdiscretestatespaces AT christianseidel quantumenhancedreinforcementlearningforfiniteepisodegameswithdiscretestatespaces AT gabrielecompostella quantumenhancedreinforcementlearningforfiniteepisodegameswithdiscretestatespaces |