Adaptive Laser Welding Control: A Reinforcement Learning Approach
Despite extensive research efforts in the field of laser welding, the imperfect repeatability of the weld quality still represents an open topic. Indeed, the inherent complexity of the underlying physical phenomena prevents the implementation of an effective controller using conventional regulators....
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9102251/ |
| _version_ | 1818874548421918720 |
|---|---|
| author | Giulio Masinelli Tri Le-Quang Silvio Zanoli Kilian Wasmer Sergey A. Shevchik |
| author_facet | Giulio Masinelli Tri Le-Quang Silvio Zanoli Kilian Wasmer Sergey A. Shevchik |
| author_sort | Giulio Masinelli |
| collection | DOAJ |
| description | Despite extensive research efforts in the field of laser welding, the imperfect repeatability of the weld quality still represents an open topic. Indeed, the inherent complexity of the underlying physical phenomena prevents the implementation of an effective controller using conventional regulators. To close this gap, we propose the application of Reinforcement Learning for closed-loop adaptive control of welding processes. The presented system is able to autonomously learn a control law that achieves a predefined weld quality independently from the starting conditions and without prior knowledge of the process dynamics. Specifically, our control unit influences the welding process by modulating the laser power and uses optical and acoustic emission signals as sensory input. The algorithm consists of three elements: a smart agent interacting with the process, a feedback network for quality monitoring, and an encoder that retains only the quality critic events from the sensory input. Based on the data representation provided by the encoder, the smart agent decides the output laser power accordingly. The corresponding input signals are then analyzed by the feedback network to determine the resulting process quality. Depending on the distance to the targeted quality, a reward is given to the agent. The latter is designed to learn from its experience by taking the actions that maximize not just its immediate reward, but the sum of all the rewards that it will receive from that moment on. Two learning schemes were tested for the agent, namely ${Q}$ -Learning and Policy Gradient. The required training time to reach the targeted quality was 20 min for the former technique and 33 min for the latter. |
| first_indexed | 2024-12-19T13:12:21Z |
| format | Article |
| id | doaj.art-465ac00c9ee84454a28523a214b670f9 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-19T13:12:21Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-465ac00c9ee84454a28523a214b670f92022-12-21T20:19:54ZengIEEEIEEE Access2169-35362020-01-01810380310381410.1109/ACCESS.2020.29980529102251Adaptive Laser Welding Control: A Reinforcement Learning ApproachGiulio Masinelli0https://orcid.org/0000-0003-1924-7735Tri Le-Quang1https://orcid.org/0000-0001-7129-5393Silvio Zanoli2https://orcid.org/0000-0002-0316-1657Kilian Wasmer3https://orcid.org/0000-0002-3294-3244Sergey A. Shevchik4https://orcid.org/0000-0003-0073-3450Laboratory for Advanced Materials Processing, Swiss Federal Laboratories for Materials Science and Technology (EMPA), Thun, SwitzerlandLaboratory for Advanced Materials Processing, Swiss Federal Laboratories for Materials Science and Technology (EMPA), Thun, SwitzerlandEmbedded Systems Laboratory, Swiss Federal Institute of Technology in Lausanne (EPFL), Lausanne, SwitzerlandLaboratory for Advanced Materials Processing, Swiss Federal Laboratories for Materials Science and Technology (EMPA), Thun, SwitzerlandLaboratory for Advanced Materials Processing, Swiss Federal Laboratories for Materials Science and Technology (EMPA), Thun, SwitzerlandDespite extensive research efforts in the field of laser welding, the imperfect repeatability of the weld quality still represents an open topic. Indeed, the inherent complexity of the underlying physical phenomena prevents the implementation of an effective controller using conventional regulators. To close this gap, we propose the application of Reinforcement Learning for closed-loop adaptive control of welding processes. The presented system is able to autonomously learn a control law that achieves a predefined weld quality independently from the starting conditions and without prior knowledge of the process dynamics. Specifically, our control unit influences the welding process by modulating the laser power and uses optical and acoustic emission signals as sensory input. The algorithm consists of three elements: a smart agent interacting with the process, a feedback network for quality monitoring, and an encoder that retains only the quality critic events from the sensory input. Based on the data representation provided by the encoder, the smart agent decides the output laser power accordingly. The corresponding input signals are then analyzed by the feedback network to determine the resulting process quality. Depending on the distance to the targeted quality, a reward is given to the agent. The latter is designed to learn from its experience by taking the actions that maximize not just its immediate reward, but the sum of all the rewards that it will receive from that moment on. Two learning schemes were tested for the agent, namely ${Q}$ -Learning and Policy Gradient. The required training time to reach the targeted quality was 20 min for the former technique and 33 min for the latter.https://ieeexplore.ieee.org/document/9102251/Laser weldinglaser material processingreinforcement learningpolicy gradient<italic xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">Q</italic>-learningclosed-loop control |
| spellingShingle | Giulio Masinelli Tri Le-Quang Silvio Zanoli Kilian Wasmer Sergey A. Shevchik Adaptive Laser Welding Control: A Reinforcement Learning Approach IEEE Access Laser welding laser material processing reinforcement learning policy gradient <italic xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">Q</italic>-learning closed-loop control |
| title | Adaptive Laser Welding Control: A Reinforcement Learning Approach |
| title_full | Adaptive Laser Welding Control: A Reinforcement Learning Approach |
| title_fullStr | Adaptive Laser Welding Control: A Reinforcement Learning Approach |
| title_full_unstemmed | Adaptive Laser Welding Control: A Reinforcement Learning Approach |
| title_short | Adaptive Laser Welding Control: A Reinforcement Learning Approach |
| title_sort | adaptive laser welding control a reinforcement learning approach |
| topic | Laser welding laser material processing reinforcement learning policy gradient <italic xmlns:ali="http://www.niso.org/schemas/ali/1.0/" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">Q</italic>-learning closed-loop control |
| url | https://ieeexplore.ieee.org/document/9102251/ |
| work_keys_str_mv | AT giuliomasinelli adaptivelaserweldingcontrolareinforcementlearningapproach AT trilequang adaptivelaserweldingcontrolareinforcementlearningapproach AT silviozanoli adaptivelaserweldingcontrolareinforcementlearningapproach AT kilianwasmer adaptivelaserweldingcontrolareinforcementlearningapproach AT sergeyashevchik adaptivelaserweldingcontrolareinforcementlearningapproach |