On the impact of the activation function on deep neural networks training
The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure. An inappropriate selection can lead to the loss of information of the input during forward propagation and the exponential vanishing/exploding of gradient...
| Main Authors: | , , |
|---|---|
| Format: | Conference item |
| Published: |
Journal of Machine Learning Research
2019
|