Efficient Elitist Cooperative Evolutionary Algorithm for Multi-Objective Reinforcement Learning
Sequential decision-making problems with multiple objectives are known as multi-objective reinforcement learning. In these scenarios, decision-makers require a complete Pareto front that consists of Pareto optimal solutions. Such a front enables decision-makers to understand the relationship between...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10113623/ |
| _version_ | 1797829122699821056 |
|---|---|
| author | Dan Zhou Jiqing Du Sachiyo Arai |
| author_facet | Dan Zhou Jiqing Du Sachiyo Arai |
| author_sort | Dan Zhou |
| collection | DOAJ |
| description | Sequential decision-making problems with multiple objectives are known as multi-objective reinforcement learning. In these scenarios, decision-makers require a complete Pareto front that consists of Pareto optimal solutions. Such a front enables decision-makers to understand the relationship between objectives and make informed decisions from a broad range of solutions. However, existing methods may be unable to search for solutions in concave regions of the Pareto front or lack global optimization ability, leading to incomplete Pareto fronts. To address this issue, we propose an efficient elitist cooperative evolutionary algorithm that maintains both an evolving population and an elite archive. The elite archive uses cooperative operations with various genetic operators to guide the evolving population, resulting in efficient searches for Pareto optimal solutions. The experimental results on submarine treasure hunting benchmarks demonstrate the effectiveness of the proposed method in solving various multi-objective reinforcement learning problems and providing decision-makers with a set of trade-off solutions between travel time and treasure amount, enabling them to make flexible and informed decisions based on their preferences. Therefore, the proposed method has the potential to be a useful tool for implementing real-world applications. |
| first_indexed | 2024-04-09T13:15:24Z |
| format | Article |
| id | doaj.art-3c2dcab280274e4f994b6f59af3e4f83 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-09T13:15:24Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-3c2dcab280274e4f994b6f59af3e4f832023-05-11T23:00:47ZengIEEEIEEE Access2169-35362023-01-0111431284313910.1109/ACCESS.2023.327211510113623Efficient Elitist Cooperative Evolutionary Algorithm for Multi-Objective Reinforcement LearningDan Zhou0https://orcid.org/0000-0002-4875-1781Jiqing Du1Sachiyo Arai2Department of Urban Environment Systems, Division of Earth and Environmental Sciences, Graduate School of Science and Engineering, Chiba University, Chiba, JapanDepartment of Urban Environment Systems, Division of Earth and Environmental Sciences, Graduate School of Science and Engineering, Chiba University, Chiba, JapanDepartment of Urban Environment Systems, Division of Earth and Environmental Sciences, Graduate School of Science and Engineering, Chiba University, Chiba, JapanSequential decision-making problems with multiple objectives are known as multi-objective reinforcement learning. In these scenarios, decision-makers require a complete Pareto front that consists of Pareto optimal solutions. Such a front enables decision-makers to understand the relationship between objectives and make informed decisions from a broad range of solutions. However, existing methods may be unable to search for solutions in concave regions of the Pareto front or lack global optimization ability, leading to incomplete Pareto fronts. To address this issue, we propose an efficient elitist cooperative evolutionary algorithm that maintains both an evolving population and an elite archive. The elite archive uses cooperative operations with various genetic operators to guide the evolving population, resulting in efficient searches for Pareto optimal solutions. The experimental results on submarine treasure hunting benchmarks demonstrate the effectiveness of the proposed method in solving various multi-objective reinforcement learning problems and providing decision-makers with a set of trade-off solutions between travel time and treasure amount, enabling them to make flexible and informed decisions based on their preferences. Therefore, the proposed method has the potential to be a useful tool for implementing real-world applications.https://ieeexplore.ieee.org/document/10113623/Multi-objective reinforcement learningefficientcooperativePareto frontelite archive |
| spellingShingle | Dan Zhou Jiqing Du Sachiyo Arai Efficient Elitist Cooperative Evolutionary Algorithm for Multi-Objective Reinforcement Learning IEEE Access Multi-objective reinforcement learning efficient cooperative Pareto front elite archive |
| title | Efficient Elitist Cooperative Evolutionary Algorithm for Multi-Objective Reinforcement Learning |
| title_full | Efficient Elitist Cooperative Evolutionary Algorithm for Multi-Objective Reinforcement Learning |
| title_fullStr | Efficient Elitist Cooperative Evolutionary Algorithm for Multi-Objective Reinforcement Learning |
| title_full_unstemmed | Efficient Elitist Cooperative Evolutionary Algorithm for Multi-Objective Reinforcement Learning |
| title_short | Efficient Elitist Cooperative Evolutionary Algorithm for Multi-Objective Reinforcement Learning |
| title_sort | efficient elitist cooperative evolutionary algorithm for multi objective reinforcement learning |
| topic | Multi-objective reinforcement learning efficient cooperative Pareto front elite archive |
| url | https://ieeexplore.ieee.org/document/10113623/ |
| work_keys_str_mv | AT danzhou efficientelitistcooperativeevolutionaryalgorithmformultiobjectivereinforcementlearning AT jiqingdu efficientelitistcooperativeevolutionaryalgorithmformultiobjectivereinforcementlearning AT sachiyoarai efficientelitistcooperativeevolutionaryalgorithmformultiobjectivereinforcementlearning |