Low-Rank Gradient Descent by Romain Cosson, Ali Jadbabaie, Anuran Makur, Amirhossein Reisizadeh, Devavrat Shah

“…In this article, we leverage such low-rank structure to reduce the high computational cost of canonical gradient-based methods such as gradient descent (<monospace>GD</monospace>). Our proposed <italic>Low-Rank Gradient Descent</italic> (<monospace>LRGD</monospace>) algorithm finds an <inline-formula><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula>-approximate stationary point of a <inline-formula><tex-math notation="LaTeX">$p$</tex-math></inline-formula>-dimensional function by first identifying <inline-formula><tex-math notation="LaTeX">$r \leq p$</tex-math></inline-formula> significant directions, and then estimating the true <inline-formula><tex-math notation="LaTeX">$p$</tex-math></inline-formula>-dimensional gradient at every iteration by computing directional derivatives only along those <inline-formula><tex-math notation="LaTeX">$r$</tex-math></inline-formula> directions. …”
Get full text

On-manifold projected gradient descent by Aaron Mahler, Tyrus Berry, Tom Stephens, Harbir Antil, Michael Merritt, Jeanie Schreiber, Ioannis Kevrekidis

“…The tools are applied to the setting of neural network image classifiers, where we generate novel, on-manifold data samples and implement a projected gradient descent algorithm for on-manifold adversarial training. …”
Get full text

Gradient-Descent-like Ghost Imaging by Wen-Kai Yu, Chen-Xi Zhu, Ya-Xin Li, Shuo-Fei Wang, Chong Cao

Subjects: Get full text

Carathéodory sampling for stochastic gradient descent by Cosentino, F, Oberhauser, H, Abate, A

“…Many problems require to optimize empirical risk functions over large data sets. Gradient descent methods that calculate the full gradient in every descent step do not scale to such datasets. …”

Carathéodory sampling for stochastic gradient descent by Cosentino, F, Oberhauser, H, Abate, A

“…Many problems require to optimize empirical risk functions over large data sets. Gradient descent methods that calculate the full gradient in every descent step do not scale to such datasets. …”

Dual space preconditioning for gradient descent by Maddison, CJ, Paulin, D, Teh, YW, Doucet, A

“…Thus, in principle our method is capable of improving the conditioning of gradient descent on problems with a non-Lipschitz gradient or nonstrongly convex structure. …”

Pipelined Stochastic Gradient Descent with Taylor Expansion by Bongwon Jang, Inchul Yoo, Dongsuk Yook

Subjects: Get full text

Accelerated Gradient Descent Driven by Lévy Perturbations by Yuquan Chen, Zhenlong Wu, Yixiang Lu, Yangquan Chen, Yong Wang

Subjects: “…accelerated gradient descent…”
Get full text

Gradient Descent Batch Clustering for Image Classification by Jae-Sam Park

Subjects: Get full text

Stochastic gradient descent for optimization for nuclear systems by Austin Williams, Noah Walton, Austin Maryanski, Sandra Bogetic, Wes Hines, Vladimir Sobes

“…ADAM is a gradient descent method that accounts for gradients with a stochastic nature. …”
Get full text

Stochastic gradient descent for wind farm optimization by J. Quick, P.-E. Rethore, M. Mølgaard Pedersen, R. V. Rodrigues, M. Friis-Møller

“…This study presents stochastic gradient descent (SGD) for wind farm optimization, which is an approach that estimates the gradient of the AEP using Monte Carlo simulation, allowing for the consideration of an arbitrarily large number of atmospheric conditions. …”
Get full text

The complexity of gradient descent: CLS = PPAD ∩ PLS by Fearnley, J, Goldberg, P, Hollender, A, Savani, R

The complexity of gradient descent: CLS = PPAD∩PLS by Fearnley, J, Goldberg, P, Hollender, A, Savani, R

“…We study search problems that can be solved by performing Gradient Descent on a bounded convex polytopal domain and show that this class is equal to the intersection of two well-known classes: PPAD and PLS. …”

The Improved Stochastic Fractional Order Gradient Descent Algorithm by Yang Yang, Lipo Mo, Yusen Hu, Fei Long

Subjects: Get full text

Recent Advances in Stochastic Gradient Descent in Deep Learning by Yingjie Tian, Yuqi Zhang, Haibin Zhang

Subjects: “…stochastic gradient descent…”
Get full text

Granular Elastic Network Regression with Stochastic Gradient Descent by Linjie He, Yumin Chen, Caiming Zhong, Keshou Wu

“…After that, we conduct the derivative of the granular loss function and design the granular elastic network gradient descent optimization algorithm. Finally, we performed experiments on the UCI datasets to verify the validity of the granular elasticity network. …”
Get full text

Distributed Stochastic Gradient Descent With Compressed and Skipped Communication by Tran Thi Phuong, Le Trieu Phong, Kazuhide Fukushima

Subjects: “…distributed stochastic gradient descent…”
Get full text

Optimization of Gradient Descent Parameters in Attitude Estimation Algorithms by Karla Sever, Leonardo Max Golušin, Josip Lončar

Subjects: Get full text

Gradient-descent-like scheme for the Allen–Cahn equation by Dongsun Lee

“…From a numerical point of view, a linear, unconditionally energy stable splitting scheme is transformed into a gradient-descent-like scheme. Various numerical simulations are illustrated to demonstrate the validity of the proposed scheme. …”
Get full text

Gradient descent with adaptive momentum for active contour models by Guoqi Liu, Zhiheng Zhou, Huiqiang Zhong, Shengli Xie

Subjects: Get full text