On the Universally Optimal Activation Function for a Class of Residual Neural Networks

On the Universally Optimal Activation Function for a Class of Residual Neural Networks

While non-linear activation functions play vital roles in artificial neural networks, it is generally unclear how the non-linearity can improve the quality of function approximations. In this paper, we present a theoretical framework to rigorously analyze the performance gain of using non-linear act...

Full description

Bibliographic Details
Main Authors: Feng Zhao, Shao-Lun Huang
Format: Article
Language:English
Published: MDPI AG 2022-10-01
Series:AppliedMath
Subjects:
activation function
function approximation
Hermite polynomials
perturbation analysis
Online Access:https://www.mdpi.com/2673-9909/2/4/33

Internet

https://www.mdpi.com/2673-9909/2/4/33

Similar Items