ODE-RU: a dynamical system view on recurrent neural networks
The core of the demonstration of this paper is to interpret the forward propagation process of machine learning as a parameter estimation problem of nonlinear dynamical systems. This process is to establish a connection between the Recurrent Neural Network and the discrete differential equation, so...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AIMS Press
2022-01-01
|
| Series: | Electronic Research Archive |
| Subjects: | |
| Online Access: | https://www.aimspress.com/article/doi/10.3934/era.2022014?viewType=HTML |
| _version_ | 1811182439089831936 |
|---|---|
| author | Pinchao Meng Xinyu Wang Weishi Yin |
| author_facet | Pinchao Meng Xinyu Wang Weishi Yin |
| author_sort | Pinchao Meng |
| collection | DOAJ |
| description | The core of the demonstration of this paper is to interpret the forward propagation process of machine learning as a parameter estimation problem of nonlinear dynamical systems. This process is to establish a connection between the Recurrent Neural Network and the discrete differential equation, so as to construct a new network structure: ODE-RU. At the same time, under the inspiration of the theory of ordinary differential equations, we propose a new forward propagation mode. In a large number of simulations and experiments, the forward propagation not only shows the trainability of the new architecture, but also achieves a low training error on the basis of main-taining the stability of the network. For the problem requiring long-term memory, we specifically study the obstacle shape reconstruction problem using the backscattering far-field features data set, and demonstrate the effectiveness of the proposed architecture using the data set. The results show that the network can effectively reduce the sensitivity to small changes in the input feature. And the error generated by the ordinary differential equation cyclic unit network in inverting the shape and position of obstacles is less than $ 10^{-2} $. |
| first_indexed | 2024-04-11T09:32:26Z |
| format | Article |
| id | doaj.art-94755c7da465488ba9532a4f08bfbd08 |
| institution | Directory Open Access Journal |
| issn | 2688-1594 |
| language | English |
| last_indexed | 2024-04-11T09:32:26Z |
| publishDate | 2022-01-01 |
| publisher | AIMS Press |
| record_format | Article |
| series | Electronic Research Archive |
| spelling | doaj.art-94755c7da465488ba9532a4f08bfbd082022-12-22T04:31:50ZengAIMS PressElectronic Research Archive2688-15942022-01-0130125727110.3934/era.2022014ODE-RU: a dynamical system view on recurrent neural networksPinchao Meng0Xinyu Wang 1Weishi Yin2Changchun University of Science and Technology, changchun, ChinaChangchun University of Science and Technology, changchun, ChinaChangchun University of Science and Technology, changchun, ChinaThe core of the demonstration of this paper is to interpret the forward propagation process of machine learning as a parameter estimation problem of nonlinear dynamical systems. This process is to establish a connection between the Recurrent Neural Network and the discrete differential equation, so as to construct a new network structure: ODE-RU. At the same time, under the inspiration of the theory of ordinary differential equations, we propose a new forward propagation mode. In a large number of simulations and experiments, the forward propagation not only shows the trainability of the new architecture, but also achieves a low training error on the basis of main-taining the stability of the network. For the problem requiring long-term memory, we specifically study the obstacle shape reconstruction problem using the backscattering far-field features data set, and demonstrate the effectiveness of the proposed architecture using the data set. The results show that the network can effectively reduce the sensitivity to small changes in the input feature. And the error generated by the ordinary differential equation cyclic unit network in inverting the shape and position of obstacles is less than $ 10^{-2} $.https://www.aimspress.com/article/doi/10.3934/era.2022014?viewType=HTMLordinary differential equationstabilityobstacle inversionrecurrent neural networkgated cell structure |
| spellingShingle | Pinchao Meng Xinyu Wang Weishi Yin ODE-RU: a dynamical system view on recurrent neural networks Electronic Research Archive ordinary differential equation stability obstacle inversion recurrent neural network gated cell structure |
| title | ODE-RU: a dynamical system view on recurrent neural networks |
| title_full | ODE-RU: a dynamical system view on recurrent neural networks |
| title_fullStr | ODE-RU: a dynamical system view on recurrent neural networks |
| title_full_unstemmed | ODE-RU: a dynamical system view on recurrent neural networks |
| title_short | ODE-RU: a dynamical system view on recurrent neural networks |
| title_sort | ode ru a dynamical system view on recurrent neural networks |
| topic | ordinary differential equation stability obstacle inversion recurrent neural network gated cell structure |
| url | https://www.aimspress.com/article/doi/10.3934/era.2022014?viewType=HTML |
| work_keys_str_mv | AT pinchaomeng oderuadynamicalsystemviewonrecurrentneuralnetworks AT xinyuwang oderuadynamicalsystemviewonrecurrentneuralnetworks AT weishiyin oderuadynamicalsystemviewonrecurrentneuralnetworks |