Influence Area of Overlap Singularity in Multilayer Perceptrons
The existing overlap singularities in the parameter space significantly affect the learning dynamics of the multilayer perceptrons. From the obtained theoretical learning trajectories near overlap singularity, when the learning process has been affected by the overlap singularity, the influence area...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2018-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8481340/ |
| _version_ | 1818618206959435776 |
|---|---|
| author | Weili Guo Yuan Yang Yingjiang Zhou Yushun Tan Haikun Wei Aiguo Song Guochen Pang |
| author_facet | Weili Guo Yuan Yang Yingjiang Zhou Yushun Tan Haikun Wei Aiguo Song Guochen Pang |
| author_sort | Weili Guo |
| collection | DOAJ |
| description | The existing overlap singularities in the parameter space significantly affect the learning dynamics of the multilayer perceptrons. From the obtained theoretical learning trajectories near overlap singularity, when the learning process has been affected by the overlap singularity, the influence area of the overlap singularity is just the line space where the two hidden units equal to each other. However, in the practical applications, different case has been observed and the influence area of such singularity may be larger. By analyzing the generalization error of multilayer perceptrons, we find that the error surface is much flatter near overlap singularity and the singularity would have much larger influence area. Finally, the validity of the obtained results are verified by taking an artificial experiment and two real-data experiments. |
| first_indexed | 2024-12-16T17:17:55Z |
| format | Article |
| id | doaj.art-15bf6df800af4e54aeb12fd34dd8e3a4 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-16T17:17:55Z |
| publishDate | 2018-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-15bf6df800af4e54aeb12fd34dd8e3a42022-12-21T22:23:14ZengIEEEIEEE Access2169-35362018-01-016602146022310.1109/ACCESS.2018.28738118481340Influence Area of Overlap Singularity in Multilayer PerceptronsWeili Guo0https://orcid.org/0000-0003-4459-9978Yuan Yang1Yingjiang Zhou2https://orcid.org/0000-0002-4481-3746Yushun Tan3Haikun Wei4Aiguo Song5https://orcid.org/0000-0002-1982-6780Guochen Pang6Key Laboratory of Remote Measurement and Control of Jiangsu Province, School of Instrument Science and Engineering, Southeast University, Nanjing, ChinaKey Laboratory of Remote Measurement and Control of Jiangsu Province, School of Instrument Science and Engineering, Southeast University, Nanjing, ChinaSchool of Automation, Nanjing University of Posts and Telecommunications, Nanjing, ChinaKey Laboratory of Measurement and Control of CSE, School of Automation, Ministry of Education, Southeast University, Nanjing, ChinaKey Laboratory of Measurement and Control of CSE, School of Automation, Ministry of Education, Southeast University, Nanjing, ChinaKey Laboratory of Remote Measurement and Control of Jiangsu Province, School of Instrument Science and Engineering, Southeast University, Nanjing, ChinaSchool of Automation and Electrical Engineering, Linyi University, Linyi, ChinaThe existing overlap singularities in the parameter space significantly affect the learning dynamics of the multilayer perceptrons. From the obtained theoretical learning trajectories near overlap singularity, when the learning process has been affected by the overlap singularity, the influence area of the overlap singularity is just the line space where the two hidden units equal to each other. However, in the practical applications, different case has been observed and the influence area of such singularity may be larger. By analyzing the generalization error of multilayer perceptrons, we find that the error surface is much flatter near overlap singularity and the singularity would have much larger influence area. Finally, the validity of the obtained results are verified by taking an artificial experiment and two real-data experiments.https://ieeexplore.ieee.org/document/8481340/Multilayer perceptronsdynamicsinformation geometryoverlap singularityinfluence area |
| spellingShingle | Weili Guo Yuan Yang Yingjiang Zhou Yushun Tan Haikun Wei Aiguo Song Guochen Pang Influence Area of Overlap Singularity in Multilayer Perceptrons IEEE Access Multilayer perceptrons dynamics information geometry overlap singularity influence area |
| title | Influence Area of Overlap Singularity in Multilayer Perceptrons |
| title_full | Influence Area of Overlap Singularity in Multilayer Perceptrons |
| title_fullStr | Influence Area of Overlap Singularity in Multilayer Perceptrons |
| title_full_unstemmed | Influence Area of Overlap Singularity in Multilayer Perceptrons |
| title_short | Influence Area of Overlap Singularity in Multilayer Perceptrons |
| title_sort | influence area of overlap singularity in multilayer perceptrons |
| topic | Multilayer perceptrons dynamics information geometry overlap singularity influence area |
| url | https://ieeexplore.ieee.org/document/8481340/ |
| work_keys_str_mv | AT weiliguo influenceareaofoverlapsingularityinmultilayerperceptrons AT yuanyang influenceareaofoverlapsingularityinmultilayerperceptrons AT yingjiangzhou influenceareaofoverlapsingularityinmultilayerperceptrons AT yushuntan influenceareaofoverlapsingularityinmultilayerperceptrons AT haikunwei influenceareaofoverlapsingularityinmultilayerperceptrons AT aiguosong influenceareaofoverlapsingularityinmultilayerperceptrons AT guochenpang influenceareaofoverlapsingularityinmultilayerperceptrons |