Fisher Information Matrix and its Application of Bipolar Activation Function Based Multilayer Perceptrons With General Gaussian Input
For the widely used multilayer perceptrons (MLPs), there exist singularities in the parameter space where Fisher information matrix (FIM) degenerates on these subspaces. The singularities seriously influence the learning dynamics of MLPs which have attracted many researchers’ attentions....
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9973293/ |
| _version_ | 1811200377595363328 |
|---|---|
| author | Weili Guo Guangyu Li Jianfeng Lu |
| author_facet | Weili Guo Guangyu Li Jianfeng Lu |
| author_sort | Weili Guo |
| collection | DOAJ |
| description | For the widely used multilayer perceptrons (MLPs), there exist singularities in the parameter space where Fisher information matrix (FIM) degenerates on these subspaces. The singularities seriously influence the learning dynamics of MLPs which have attracted many researchers’ attentions. As FIM plays key role in investigating the singular learning dynamics of MLPs, it is very important to obtain the analytical form of FIM. In this paper, for the bipolar activation function based MLPs with general Gaussian input, by choosing bipolar error function as the activation function, the analytical form of FIM are obtained. Then the validity of obtained results are verified by taking two experiments. |
| first_indexed | 2024-04-12T02:03:54Z |
| format | Article |
| id | doaj.art-991ae8b854554a8dbe7f5f3d5ee0f7fe |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T02:03:54Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-991ae8b854554a8dbe7f5f3d5ee0f7fe2022-12-22T03:52:36ZengIEEEIEEE Access2169-35362022-01-011012825012826210.1109/ACCESS.2022.32274279973293Fisher Information Matrix and its Application of Bipolar Activation Function Based Multilayer Perceptrons With General Gaussian InputWeili Guo0https://orcid.org/0000-0003-4459-9978Guangyu Li1Jianfeng Lu2https://orcid.org/0000-0002-9190-507XSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSchool of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaFor the widely used multilayer perceptrons (MLPs), there exist singularities in the parameter space where Fisher information matrix (FIM) degenerates on these subspaces. The singularities seriously influence the learning dynamics of MLPs which have attracted many researchers’ attentions. As FIM plays key role in investigating the singular learning dynamics of MLPs, it is very important to obtain the analytical form of FIM. In this paper, for the bipolar activation function based MLPs with general Gaussian input, by choosing bipolar error function as the activation function, the analytical form of FIM are obtained. Then the validity of obtained results are verified by taking two experiments.https://ieeexplore.ieee.org/document/9973293/Fisher information matrixmultilayer perceptronssingularitybipolar error functiongeneral Gaussian input |
| spellingShingle | Weili Guo Guangyu Li Jianfeng Lu Fisher Information Matrix and its Application of Bipolar Activation Function Based Multilayer Perceptrons With General Gaussian Input IEEE Access Fisher information matrix multilayer perceptrons singularity bipolar error function general Gaussian input |
| title | Fisher Information Matrix and its Application of Bipolar Activation Function Based Multilayer Perceptrons With General Gaussian Input |
| title_full | Fisher Information Matrix and its Application of Bipolar Activation Function Based Multilayer Perceptrons With General Gaussian Input |
| title_fullStr | Fisher Information Matrix and its Application of Bipolar Activation Function Based Multilayer Perceptrons With General Gaussian Input |
| title_full_unstemmed | Fisher Information Matrix and its Application of Bipolar Activation Function Based Multilayer Perceptrons With General Gaussian Input |
| title_short | Fisher Information Matrix and its Application of Bipolar Activation Function Based Multilayer Perceptrons With General Gaussian Input |
| title_sort | fisher information matrix and its application of bipolar activation function based multilayer perceptrons with general gaussian input |
| topic | Fisher information matrix multilayer perceptrons singularity bipolar error function general Gaussian input |
| url | https://ieeexplore.ieee.org/document/9973293/ |
| work_keys_str_mv | AT weiliguo fisherinformationmatrixanditsapplicationofbipolaractivationfunctionbasedmultilayerperceptronswithgeneralgaussianinput AT guangyuli fisherinformationmatrixanditsapplicationofbipolaractivationfunctionbasedmultilayerperceptronswithgeneralgaussianinput AT jianfenglu fisherinformationmatrixanditsapplicationofbipolaractivationfunctionbasedmultilayerperceptronswithgeneralgaussianinput |