Concentration Inequalities and Optimal Number of Layers for Stochastic Deep Neural Networks
We state concentration inequalities for the output of the hidden layers of a stochastic deep neural network (SDNN), as well as for the output of the whole SDNN. These results allow us to introduce an expected classifier (EC), and to give probabilistic upper bound for the classification error of the...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10103873/ |
| _version_ | 1797840110102773760 |
|---|---|
| author | Michele Caprio Sayan Mukherjee |
| author_facet | Michele Caprio Sayan Mukherjee |
| author_sort | Michele Caprio |
| collection | DOAJ |
| description | We state concentration inequalities for the output of the hidden layers of a stochastic deep neural network (SDNN), as well as for the output of the whole SDNN. These results allow us to introduce an expected classifier (EC), and to give probabilistic upper bound for the classification error of the EC. We also state the optimal number of layers for the SDNN via an optimal stopping procedure. We apply our analysis to a stochastic version of a feedforward neural network with ReLU activation function. |
| first_indexed | 2024-04-09T16:09:15Z |
| format | Article |
| id | doaj.art-e56696bfa99b4f1fb594c98513defbdb |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-09T16:09:15Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e56696bfa99b4f1fb594c98513defbdb2023-04-24T23:00:39ZengIEEEIEEE Access2169-35362023-01-0111384583847010.1109/ACCESS.2023.326803410103873Concentration Inequalities and Optimal Number of Layers for Stochastic Deep Neural NetworksMichele Caprio0https://orcid.org/0000-0002-7569-097XSayan Mukherjee1Department of Computer and Information Science, PRECISE Center, University of Pennsylvania, Philadelphia, PA, USACenter for Scalable Data Analytics and Artificial Intelligence, Universität Leipzig, Leipzig, GermanyWe state concentration inequalities for the output of the hidden layers of a stochastic deep neural network (SDNN), as well as for the output of the whole SDNN. These results allow us to introduce an expected classifier (EC), and to give probabilistic upper bound for the classification error of the EC. We also state the optimal number of layers for the SDNN via an optimal stopping procedure. We apply our analysis to a stochastic version of a feedforward neural network with ReLU activation function.https://ieeexplore.ieee.org/document/10103873/Stochastic deep neural networkfeedforward neural networkReLU activationconcentration inequalitymartingalesoptimal stopping |
| spellingShingle | Michele Caprio Sayan Mukherjee Concentration Inequalities and Optimal Number of Layers for Stochastic Deep Neural Networks IEEE Access Stochastic deep neural network feedforward neural network ReLU activation concentration inequality martingales optimal stopping |
| title | Concentration Inequalities and Optimal Number of Layers for Stochastic Deep Neural Networks |
| title_full | Concentration Inequalities and Optimal Number of Layers for Stochastic Deep Neural Networks |
| title_fullStr | Concentration Inequalities and Optimal Number of Layers for Stochastic Deep Neural Networks |
| title_full_unstemmed | Concentration Inequalities and Optimal Number of Layers for Stochastic Deep Neural Networks |
| title_short | Concentration Inequalities and Optimal Number of Layers for Stochastic Deep Neural Networks |
| title_sort | concentration inequalities and optimal number of layers for stochastic deep neural networks |
| topic | Stochastic deep neural network feedforward neural network ReLU activation concentration inequality martingales optimal stopping |
| url | https://ieeexplore.ieee.org/document/10103873/ |
| work_keys_str_mv | AT michelecaprio concentrationinequalitiesandoptimalnumberoflayersforstochasticdeepneuralnetworks AT sayanmukherjee concentrationinequalitiesandoptimalnumberoflayersforstochasticdeepneuralnetworks |