Low-Rank Gradient Descent

Low-Rank Gradient Descent

Several recent empirical studies demonstrate that important machine learning tasks such as training deep neural networks, exhibit a low-rank structure, where most of the variation in the loss function occurs only in a few directions of the input space. In this article, we leverage such low-rank stru...

Cur síos iomlán

Sonraí bibleagrafaíochta
Príomhchruthaitheoirí: Romain Cosson, Ali Jadbabaie, Anuran Makur, Amirhossein Reisizadeh, Devavrat Shah
Formáid: Alt
Teanga:English
Foilsithe / Cruthaithe: IEEE 2023-01-01
Sraith:IEEE Open Journal of Control Systems
Ábhair:
Active subspace
first order optimization
low-rank functions
oracle complexity
Rochtain ar líne:https://ieeexplore.ieee.org/document/10250907/

Ar líne

https://ieeexplore.ieee.org/document/10250907/

Míreanna comhchosúla