A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression
Truss layout design aims to find the optimal layout, considering node locations, connection topology between nodes, and cross-sectional areas of connecting bars. The design process of trusses can be represented as a reinforcement learning problem by formulating the optimization task into a Markov De...
Main Authors: | , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2022-08-01
|
Series: | Applied Sciences |
Subjects: | |
Online Access: | https://www.mdpi.com/2076-3417/12/16/8227 |
_version_ | 1797439611364966400 |
---|---|
author | Ruifeng Luo Yifan Wang Zhiyuan Liu Weifang Xiao Xianzhong Zhao |
author_facet | Ruifeng Luo Yifan Wang Zhiyuan Liu Weifang Xiao Xianzhong Zhao |
author_sort | Ruifeng Luo |
collection | DOAJ |
description | Truss layout design aims to find the optimal layout, considering node locations, connection topology between nodes, and cross-sectional areas of connecting bars. The design process of trusses can be represented as a reinforcement learning problem by formulating the optimization task into a Markov Decision Process (MDP). The optimization variables such as node positions need to be transformed into discrete actions in this MDP; however, the common method is to uniformly discretize the design domain by generating a set of candidate actions, which brings dimension explosion problems in spatial truss design. In this paper, a reinforcement learning algorithm is proposed to deal with continuous action spaces in truss layout design problems by using kernel regression. It is a nonparametric regression way to sample the continuous action space and generalize the information about action value between sampled actions and unexplored parts of the action space. As the number of searches increases, the algorithm can gradually increase the candidate action set by appending actions of high confidence value from the continuous action space. The value correlation between actions is mapped by the Gaussian function and Euclidean distance. In this sampling strategy, a modified Confidence Upper Bound formula is proposed to evaluate the heuristics of sampled actions, including both 2D and 3D cases. The proposed algorithm was tested in various layout design problems of planar and spatial trusses. The results indicate that the proposed algorithm has a good performance in finding the truss layout with minimum weight. This implies the validity and efficiency of the established algorithm. |
first_indexed | 2024-03-09T11:56:35Z |
format | Article |
id | doaj.art-6ea5a5bfc0544d9db2f299e4094fc348 |
institution | Directory Open Access Journal |
issn | 2076-3417 |
language | English |
last_indexed | 2024-03-09T11:56:35Z |
publishDate | 2022-08-01 |
publisher | MDPI AG |
record_format | Article |
series | Applied Sciences |
spelling | doaj.art-6ea5a5bfc0544d9db2f299e4094fc3482023-11-30T23:08:41ZengMDPI AGApplied Sciences2076-34172022-08-011216822710.3390/app12168227A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel RegressionRuifeng Luo0Yifan Wang1Zhiyuan Liu2Weifang Xiao3Xianzhong Zhao4College of Civil Engineering, Tongji University, Shanghai 200092, ChinaShanghai Qi Zhi Institute, Shanghai 200232, ChinaCollege of Civil Engineering, Tongji University, Shanghai 200092, ChinaCollege of Civil Engineering, Tongji University, Shanghai 200092, ChinaCollege of Civil Engineering, Tongji University, Shanghai 200092, ChinaTruss layout design aims to find the optimal layout, considering node locations, connection topology between nodes, and cross-sectional areas of connecting bars. The design process of trusses can be represented as a reinforcement learning problem by formulating the optimization task into a Markov Decision Process (MDP). The optimization variables such as node positions need to be transformed into discrete actions in this MDP; however, the common method is to uniformly discretize the design domain by generating a set of candidate actions, which brings dimension explosion problems in spatial truss design. In this paper, a reinforcement learning algorithm is proposed to deal with continuous action spaces in truss layout design problems by using kernel regression. It is a nonparametric regression way to sample the continuous action space and generalize the information about action value between sampled actions and unexplored parts of the action space. As the number of searches increases, the algorithm can gradually increase the candidate action set by appending actions of high confidence value from the continuous action space. The value correlation between actions is mapped by the Gaussian function and Euclidean distance. In this sampling strategy, a modified Confidence Upper Bound formula is proposed to evaluate the heuristics of sampled actions, including both 2D and 3D cases. The proposed algorithm was tested in various layout design problems of planar and spatial trusses. The results indicate that the proposed algorithm has a good performance in finding the truss layout with minimum weight. This implies the validity and efficiency of the established algorithm.https://www.mdpi.com/2076-3417/12/16/8227generative designoptimal truss layoutreinforcement learningMonte Carlo Tree Searchkernel regressiondesign automation |
spellingShingle | Ruifeng Luo Yifan Wang Zhiyuan Liu Weifang Xiao Xianzhong Zhao A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression Applied Sciences generative design optimal truss layout reinforcement learning Monte Carlo Tree Search kernel regression design automation |
title | A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression |
title_full | A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression |
title_fullStr | A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression |
title_full_unstemmed | A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression |
title_short | A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression |
title_sort | reinforcement learning method for layout design of planar and spatial trusses using kernel regression |
topic | generative design optimal truss layout reinforcement learning Monte Carlo Tree Search kernel regression design automation |
url | https://www.mdpi.com/2076-3417/12/16/8227 |
work_keys_str_mv | AT ruifengluo areinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression AT yifanwang areinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression AT zhiyuanliu areinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression AT weifangxiao areinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression AT xianzhongzhao areinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression AT ruifengluo reinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression AT yifanwang reinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression AT zhiyuanliu reinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression AT weifangxiao reinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression AT xianzhongzhao reinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression |