Research on three-step accelerated gradient algorithm in deep learning

Research on three-step accelerated gradient algorithm in deep learning

Gradient descent (GD) algorithm is the widely used optimisation method in training machine learning and deep learning models. In this paper, based on GD, Polyak's momentum (PM), and Nesterov accelerated gradient (NAG), we give the convergence of the algorithms from an initial value to the optim...

Full description

Bibliographic Details
Main Authors: Yongqiang Lian, Yincai Tang, Shirong Zhou
Format: Article
Language:English
Published: Taylor & Francis Group 2022-01-01
Series:Statistical Theory and Related Fields
Subjects:
accelerated algorithm
backpropagation
deep learning
learning rate
momentum
stochastic gradient descent
Online Access:http://dx.doi.org/10.1080/24754269.2020.1846414

Internet

http://dx.doi.org/10.1080/24754269.2020.1846414

Similar Items