Universal characteristics of deep neural network loss surfaces from random matrix theory
This paper considers several aspects of random matrix universality in deep neural networks (DNNs). Motivated by recent experimental work, we use universal properties of random matrices related to local statistics to derive practical implications for DNNs based on a realistic model of their Hessians....
Main Authors: | Baskerville, N, Keating, JP, Mezzadri, F, Najnudel, J, Granziol, D |
---|---|
Format: | Journal article |
Language: | English |
Published: |
IOP Publishing
2022
|
Similar Items
-
Appearance of random matrix theory in deep learning
by: Baskerville, NP, et al.
Published: (2021) -
The loss surfaces of neural networks with general activation functions
by: Baskerville, N, et al.
Published: (2021) -
A spin glass model for the loss surfaces of generative adversarial networks
by: Baskerville, N, et al.
Published: (2022) -
Learning rates as a function of batch size: a random matrix theory approach to neural network training
by: Granziol, D, et al.
Published: (2022) -
Freezing transitions and extreme values: random matrix theory, and disordered landscapes
by: Fyodorov, YV, et al.
Published: (2014)