An analytic end-to-end collaborative learning algorithm
In most control applications, theoretical analysis of the systems is crucial in ensuring stability or convergence, so as to ensure safe and reliable operations and also to gain a better understanding of the systems for further developments. However, most current deep learning methods are black-box a...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Conference Paper |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/173152 https://ieeexplore.ieee.org/xpl/conhome/10383192/proceeding |
| _version_ | 1824456557806485504 |
|---|---|
| author | Li, Sitan Cheah, Chien Chern |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Li, Sitan Cheah, Chien Chern |
| author_sort | Li, Sitan |
| collection | NTU |
| description | In most control applications, theoretical analysis of the systems is crucial in ensuring stability or convergence, so as to ensure safe and reliable operations and also to gain a better understanding of the systems for further developments. However, most current deep learning methods are black-box approaches that are more focused on empirical studies. Recently, some results have been obtained for convergence analysis of end-to end deep learning based on non-smooth ReLU activation functions, which may result in chattering for control tasks. This paper presents a convergence analysis for end-to-end deep learning of fully connected neural networks (FNN) with smooth activation functions. The proposed method therefore avoids any potential chattering problem, and it also does not easily lead to gradient vanishing problems. The proposed End-to-End algorithm trains multiple two-layer fully connected networks concurrently and collaborative learning can be used to further combine their strengths to improve accuracy. A classification case study based on fully connected networks and MNIST dataset was done to demonstrate the performance of the proposed approach. Then an online kinematics control task of a UR5e robot arm was performed to illustrate the regression approximation and online updating ability of our algorithm. |
| first_indexed | 2025-02-19T03:56:00Z |
| format | Conference Paper |
| id | ntu-10356/173152 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2025-02-19T03:56:00Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1731522024-03-22T15:40:03Z An analytic end-to-end collaborative learning algorithm Li, Sitan Cheah, Chien Chern School of Electrical and Electronic Engineering 2023 62nd IEEE Conference on Decision and Control (CDC) Engineering End-to-End Convergence analysis In most control applications, theoretical analysis of the systems is crucial in ensuring stability or convergence, so as to ensure safe and reliable operations and also to gain a better understanding of the systems for further developments. However, most current deep learning methods are black-box approaches that are more focused on empirical studies. Recently, some results have been obtained for convergence analysis of end-to end deep learning based on non-smooth ReLU activation functions, which may result in chattering for control tasks. This paper presents a convergence analysis for end-to-end deep learning of fully connected neural networks (FNN) with smooth activation functions. The proposed method therefore avoids any potential chattering problem, and it also does not easily lead to gradient vanishing problems. The proposed End-to-End algorithm trains multiple two-layer fully connected networks concurrently and collaborative learning can be used to further combine their strengths to improve accuracy. A classification case study based on fully connected networks and MNIST dataset was done to demonstrate the performance of the proposed approach. Then an online kinematics control task of a UR5e robot arm was performed to illustrate the regression approximation and online updating ability of our algorithm. Ministry of Education (MOE) Submitted/Accepted version This work was supported by the Ministry of Education (MOE) Singapore, Academic Research Fund (AcRF) Tier 1, under Grant RG65/22. 2024-03-22T01:13:48Z 2024-03-22T01:13:48Z 2024 Conference Paper Li, S. & Cheah, C. C. (2024). An analytic end-to-end collaborative learning algorithm. 2023 62nd IEEE Conference on Decision and Control (CDC). 979-8-3503-0124-3 https://hdl.handle.net/10356/173152 https://ieeexplore.ieee.org/xpl/conhome/10383192/proceeding en RG65/22 © 2024 IEEE. All rights reserved. This article may be downloaded for personal use only. Any other use requires prior permission of the copyright holder. The Version of Record is available online under IEEE Control Systems Letters at http://doi.org/10.1109/LCSYS.2023.3292034. application/pdf |
| spellingShingle | Engineering End-to-End Convergence analysis Li, Sitan Cheah, Chien Chern An analytic end-to-end collaborative learning algorithm |
| title | An analytic end-to-end collaborative learning algorithm |
| title_full | An analytic end-to-end collaborative learning algorithm |
| title_fullStr | An analytic end-to-end collaborative learning algorithm |
| title_full_unstemmed | An analytic end-to-end collaborative learning algorithm |
| title_short | An analytic end-to-end collaborative learning algorithm |
| title_sort | analytic end to end collaborative learning algorithm |
| topic | Engineering End-to-End Convergence analysis |
| url | https://hdl.handle.net/10356/173152 https://ieeexplore.ieee.org/xpl/conhome/10383192/proceeding |
| work_keys_str_mv | AT lisitan ananalyticendtoendcollaborativelearningalgorithm AT cheahchienchern ananalyticendtoendcollaborativelearningalgorithm AT lisitan analyticendtoendcollaborativelearningalgorithm AT cheahchienchern analyticendtoendcollaborativelearningalgorithm |