A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression

Truss layout design aims to find the optimal layout, considering node locations, connection topology between nodes, and cross-sectional areas of connecting bars. The design process of trusses can be represented as a reinforcement learning problem by formulating the optimization task into a Markov De...

Full description

Bibliographic Details
Main Authors: Ruifeng Luo, Yifan Wang, Zhiyuan Liu, Weifang Xiao, Xianzhong Zhao
Format: Article
Language:English
Published: MDPI AG 2022-08-01
Series:Applied Sciences
Subjects:
Online Access:https://www.mdpi.com/2076-3417/12/16/8227
_version_ 1797439611364966400
author Ruifeng Luo
Yifan Wang
Zhiyuan Liu
Weifang Xiao
Xianzhong Zhao
author_facet Ruifeng Luo
Yifan Wang
Zhiyuan Liu
Weifang Xiao
Xianzhong Zhao
author_sort Ruifeng Luo
collection DOAJ
description Truss layout design aims to find the optimal layout, considering node locations, connection topology between nodes, and cross-sectional areas of connecting bars. The design process of trusses can be represented as a reinforcement learning problem by formulating the optimization task into a Markov Decision Process (MDP). The optimization variables such as node positions need to be transformed into discrete actions in this MDP; however, the common method is to uniformly discretize the design domain by generating a set of candidate actions, which brings dimension explosion problems in spatial truss design. In this paper, a reinforcement learning algorithm is proposed to deal with continuous action spaces in truss layout design problems by using kernel regression. It is a nonparametric regression way to sample the continuous action space and generalize the information about action value between sampled actions and unexplored parts of the action space. As the number of searches increases, the algorithm can gradually increase the candidate action set by appending actions of high confidence value from the continuous action space. The value correlation between actions is mapped by the Gaussian function and Euclidean distance. In this sampling strategy, a modified Confidence Upper Bound formula is proposed to evaluate the heuristics of sampled actions, including both 2D and 3D cases. The proposed algorithm was tested in various layout design problems of planar and spatial trusses. The results indicate that the proposed algorithm has a good performance in finding the truss layout with minimum weight. This implies the validity and efficiency of the established algorithm.
first_indexed 2024-03-09T11:56:35Z
format Article
id doaj.art-6ea5a5bfc0544d9db2f299e4094fc348
institution Directory Open Access Journal
issn 2076-3417
language English
last_indexed 2024-03-09T11:56:35Z
publishDate 2022-08-01
publisher MDPI AG
record_format Article
series Applied Sciences
spelling doaj.art-6ea5a5bfc0544d9db2f299e4094fc3482023-11-30T23:08:41ZengMDPI AGApplied Sciences2076-34172022-08-011216822710.3390/app12168227A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel RegressionRuifeng Luo0Yifan Wang1Zhiyuan Liu2Weifang Xiao3Xianzhong Zhao4College of Civil Engineering, Tongji University, Shanghai 200092, ChinaShanghai Qi Zhi Institute, Shanghai 200232, ChinaCollege of Civil Engineering, Tongji University, Shanghai 200092, ChinaCollege of Civil Engineering, Tongji University, Shanghai 200092, ChinaCollege of Civil Engineering, Tongji University, Shanghai 200092, ChinaTruss layout design aims to find the optimal layout, considering node locations, connection topology between nodes, and cross-sectional areas of connecting bars. The design process of trusses can be represented as a reinforcement learning problem by formulating the optimization task into a Markov Decision Process (MDP). The optimization variables such as node positions need to be transformed into discrete actions in this MDP; however, the common method is to uniformly discretize the design domain by generating a set of candidate actions, which brings dimension explosion problems in spatial truss design. In this paper, a reinforcement learning algorithm is proposed to deal with continuous action spaces in truss layout design problems by using kernel regression. It is a nonparametric regression way to sample the continuous action space and generalize the information about action value between sampled actions and unexplored parts of the action space. As the number of searches increases, the algorithm can gradually increase the candidate action set by appending actions of high confidence value from the continuous action space. The value correlation between actions is mapped by the Gaussian function and Euclidean distance. In this sampling strategy, a modified Confidence Upper Bound formula is proposed to evaluate the heuristics of sampled actions, including both 2D and 3D cases. The proposed algorithm was tested in various layout design problems of planar and spatial trusses. The results indicate that the proposed algorithm has a good performance in finding the truss layout with minimum weight. This implies the validity and efficiency of the established algorithm.https://www.mdpi.com/2076-3417/12/16/8227generative designoptimal truss layoutreinforcement learningMonte Carlo Tree Searchkernel regressiondesign automation
spellingShingle Ruifeng Luo
Yifan Wang
Zhiyuan Liu
Weifang Xiao
Xianzhong Zhao
A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression
Applied Sciences
generative design
optimal truss layout
reinforcement learning
Monte Carlo Tree Search
kernel regression
design automation
title A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression
title_full A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression
title_fullStr A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression
title_full_unstemmed A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression
title_short A Reinforcement Learning Method for Layout Design of Planar and Spatial Trusses using Kernel Regression
title_sort reinforcement learning method for layout design of planar and spatial trusses using kernel regression
topic generative design
optimal truss layout
reinforcement learning
Monte Carlo Tree Search
kernel regression
design automation
url https://www.mdpi.com/2076-3417/12/16/8227
work_keys_str_mv AT ruifengluo areinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression
AT yifanwang areinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression
AT zhiyuanliu areinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression
AT weifangxiao areinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression
AT xianzhongzhao areinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression
AT ruifengluo reinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression
AT yifanwang reinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression
AT zhiyuanliu reinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression
AT weifangxiao reinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression
AT xianzhongzhao reinforcementlearningmethodforlayoutdesignofplanarandspatialtrussesusingkernelregression