Neural Networks Regularization With Graph-Based Local Resampling
This paper presents the concept of Graph-based Local Resampling of perceptron-like neural networks with random projections (RN-ELM) which aims at regularization of the yielded model. The addition of synthetic noise to the learning set finds some similarity with data augmentation approaches that are...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9383228/ |
| _version_ | 1798002046171873280 |
|---|---|
| author | Alex D. Assis Luiz C. B. Torres Lourenco R. G. Araujo Vitor M. Hanriot Antonio P. Braga |
| author_facet | Alex D. Assis Luiz C. B. Torres Lourenco R. G. Araujo Vitor M. Hanriot Antonio P. Braga |
| author_sort | Alex D. Assis |
| collection | DOAJ |
| description | This paper presents the concept of Graph-based Local Resampling of perceptron-like neural networks with random projections (RN-ELM) which aims at regularization of the yielded model. The addition of synthetic noise to the learning set finds some similarity with data augmentation approaches that are currently adopted in many deep learning strategies. With the graph-based approach, however, it is possible to direct resample in the margin region instead of exhaustively cover the whole input space. The goal is to train neural networks with added noise in the margin region, located by structural information extracted from a planar graph. The so-called structural vectors, which are the training set vertices near the class boundary, are obtained from the structural information using Gabriel Graph. Synthetic samples are added to the learning set around the geometric vectors, improving generalization performance. A mathematical formulation that shows that the addition of synthetic samples has the same effect as the Tikhonov regularization is presented. Friedman and pos-hoc Nemenyi tests indicate that outcomes from the proposed method are statistically equivalent to the ones obtained by objective-function regularization, implying that both methods yield smoother solutions, reducing the effects of overfitting. |
| first_indexed | 2024-04-11T11:45:57Z |
| format | Article |
| id | doaj.art-d86c9144cb4243b7bab9662bf8fa2f00 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T11:45:57Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-d86c9144cb4243b7bab9662bf8fa2f002022-12-22T04:25:37ZengIEEEIEEE Access2169-35362021-01-019507275073710.1109/ACCESS.2021.30681279383228Neural Networks Regularization With Graph-Based Local ResamplingAlex D. Assis0https://orcid.org/0000-0002-1293-5642Luiz C. B. Torres1https://orcid.org/0000-0002-4991-8395Lourenco R. G. Araujo2https://orcid.org/0000-0001-9075-8787Vitor M. Hanriot3Antonio P. Braga4https://orcid.org/0000-0002-9007-0920Department of Economics, Universidade Federal de Juiz de Fora (UFJF), Governador Valadares, BrazilDepartment of Computing and Systems, Universidade Federal de Ouro Preto (UFOP), João Monlevade, BrazilGraduate Program in Electrical Engineering, Universidade Federal de Minas Gerais (UFMG), Belo Horizonte, BrazilDepartment of Electronics Engineering, Universidade Federal de Minas Gerais (UFMG), Belo Horizonte, BrazilDepartment of Electronics Engineering, Universidade Federal de Minas Gerais (UFMG), Belo Horizonte, BrazilThis paper presents the concept of Graph-based Local Resampling of perceptron-like neural networks with random projections (RN-ELM) which aims at regularization of the yielded model. The addition of synthetic noise to the learning set finds some similarity with data augmentation approaches that are currently adopted in many deep learning strategies. With the graph-based approach, however, it is possible to direct resample in the margin region instead of exhaustively cover the whole input space. The goal is to train neural networks with added noise in the margin region, located by structural information extracted from a planar graph. The so-called structural vectors, which are the training set vertices near the class boundary, are obtained from the structural information using Gabriel Graph. Synthetic samples are added to the learning set around the geometric vectors, improving generalization performance. A mathematical formulation that shows that the addition of synthetic samples has the same effect as the Tikhonov regularization is presented. Friedman and pos-hoc Nemenyi tests indicate that outcomes from the proposed method are statistically equivalent to the ones obtained by objective-function regularization, implying that both methods yield smoother solutions, reducing the effects of overfitting.https://ieeexplore.ieee.org/document/9383228/Classifierneural networkregularizationtraining with noise |
| spellingShingle | Alex D. Assis Luiz C. B. Torres Lourenco R. G. Araujo Vitor M. Hanriot Antonio P. Braga Neural Networks Regularization With Graph-Based Local Resampling IEEE Access Classifier neural network regularization training with noise |
| title | Neural Networks Regularization With Graph-Based Local Resampling |
| title_full | Neural Networks Regularization With Graph-Based Local Resampling |
| title_fullStr | Neural Networks Regularization With Graph-Based Local Resampling |
| title_full_unstemmed | Neural Networks Regularization With Graph-Based Local Resampling |
| title_short | Neural Networks Regularization With Graph-Based Local Resampling |
| title_sort | neural networks regularization with graph based local resampling |
| topic | Classifier neural network regularization training with noise |
| url | https://ieeexplore.ieee.org/document/9383228/ |
| work_keys_str_mv | AT alexdassis neuralnetworksregularizationwithgraphbasedlocalresampling AT luizcbtorres neuralnetworksregularizationwithgraphbasedlocalresampling AT lourencorgaraujo neuralnetworksregularizationwithgraphbasedlocalresampling AT vitormhanriot neuralnetworksregularizationwithgraphbasedlocalresampling AT antoniopbraga neuralnetworksregularizationwithgraphbasedlocalresampling |