Why random reshuffling beats stochastic gradient descent

Why random reshuffling beats stochastic gradient descent

Abstract We analyze the convergence rate of the random reshuffling (RR) method, which is a randomized first-order incremental algorithm for minimizing a finite sum of convex component functions. RR proceeds in cycles, picking a uniformly random order (permutation) and processing the c...

Full description

Bibliographic Details
Main Authors:	Gürbüzbalaban, M., Ozdaglar, A., Parrilo, P. A
Other Authors:	Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Format:	Article
Language:	English
Published:	Springer Berlin Heidelberg 2021
Online Access:	https://hdl.handle.net/1721.1/132030

Similar Items

When cyclic coordinate descent outperforms randomized coordinate descent
by: Gurbuzbalaban, Mert, et al.
Published: (2019)

Randomness and permutations in coordinate descent methods
by: Gürbüzbalaban, Mert, et al.
Published: (2021)

Jokowi's cabinet reshuffle : will it beat COVID-19?
by: Borsuk, Richard
Published: (2021)

A universally optimal multistage accelerated stochastic gradient method
by: Aybat, NS, et al.
Published: (2021)

Carathéodory sampling for stochastic gradient descent
by: Cosentino, F, et al.
Published: (2020)

Carathéodory sampling for stochastic gradient descent
by: Cosentino, F, et al.
Published: (2020)

Semi-Stochastic Gradient Descent Methods
by: Jakub Konečný, et al.
Published: (2017-05-01)

Stochastic gradient descent for wind farm optimization
by: J. Quick, et al.
Published: (2023-08-01)

Stochastic gradient descent with random label noises: doubly stochastic models and inference stabilizer
by: Haoyi Xiong, et al.
Published: (2024-01-01)

Pipelined Stochastic Gradient Descent with Taylor Expansion
by: Bongwon Jang, et al.
Published: (2023-10-01)

Stochastic gradient descent for optimization for nuclear systems
by: Austin Williams, et al.
Published: (2023-05-01)

Generalization bounds for label noise stochastic gradient descent
by: Huh, JE, et al.
Published: (2023)

Generalization bounds for label noise stochastic gradient descent
by: Huh, JE, et al.
Published: (2024)

A globally convergent incremental Newton method
by: Gurbuzbalaban, Mert, et al.
Published: (2016)

On the Convergence Rate of Incremental Aggregated Gradient Algorithms
by: Gurbuzbalaban, Mert, et al.
Published: (2018)

Granular Elastic Network Regression with Stochastic Gradient Descent
by: Linjie He, et al.
Published: (2022-07-01)

The Improved Stochastic Fractional Order Gradient Descent Algorithm
by: Yang Yang, et al.
Published: (2023-08-01)

Recent Advances in Stochastic Gradient Descent in Deep Learning
by: Yingjie Tian, et al.
Published: (2023-01-01)

Distributed Stochastic Gradient Descent With Compressed and Skipped Communication
by: Tran Thi Phuong, et al.
Published: (2023-01-01)

Stochastic gradient descent for hybrid quantum-classical optimization
by: Ryan Sweke, et al.
Published: (2020-08-01)

AG-SGD: Angle-Based Stochastic Gradient Descent
by: Chongya Song, et al.
Published: (2021-01-01)

Featured Hybrid Recommendation System Using Stochastic Gradient Descent
by: Si Thin Nguyen, et al.
Published: (2021-01-01)

Stochastic gradient descent based fuzzy clustering for large data
by: Chen, Lihui, et al.
Published: (2015)

Design of Momentum Fractional Stochastic Gradient Descent for Recommender Systems
by: Zeshan Aslam Khan, et al.
Published: (2019-01-01)

Global Convergence Rate of Proximal Incremental Aggregated Gradient Methods
by: Vanli, Nuri Denizcan, et al.
Published: (2019)

Hyper-parameter optimization for support vector machines using stochastic gradient descent and dual coordinate descent
by: W.e.i. Jiang, et al.
Published: (2020-03-01)

Cabinet reshuffle imminent?
by: Daim, Nuradzimah, et al.
Published: (2016)

Normalized stochastic gradient descent learning of general complex‐valued models
by: T. Paireder, et al.
Published: (2021-06-01)

Adaptive Gradient Estimation Stochastic Parallel Gradient Descent Algorithm for Laser Beam Cleanup
by: Shiqing Ma, et al.
Published: (2021-05-01)

Robust Accelerated Gradient Methods for Smooth Strongly Convex Functions
by: Aybat, Necdet Serhat, et al.
Published: (2021)

Adaptive Stochastic Gradient Descent Method for Convex and Non-Convex Optimization
by: Ruijuan Chen, et al.
Published: (2022-11-01)

Implicit Stochastic Gradient Descent Method for Cross-Domain Recommendation System
by: Nam D. Vo, et al.
Published: (2020-04-01)

Damped Newton Stochastic Gradient Descent Method for Neural Networks Training
by: Jingcheng Zhou, et al.
Published: (2021-06-01)

Using the Stochastic Gradient Descent Optimization Algorithm on Estimating of Reactivity Ratios
by: Iosif Sorin Fazakas-Anca, et al.
Published: (2021-08-01)

A Geometric Interpretation of Stochastic Gradient Descent Using Diffusion Metrics
by: Rita Fioresi, et al.
Published: (2020-01-01)

A simplified convergence theory for Byzantine resilient stochastic gradient descent
by: Lindon Roberts, et al.
Published: (2022-01-01)

Beyond Stochastic Gradient Descent for Matrix Completion Based Indoor Localization
by: Wafa Njima, et al.
Published: (2019-06-01)

On the stochastic gradient descent matrix factorization in application to the supervised classification of microarrays
by: Vladimir Nikolaevich Nikulin
Published: (2013-04-01)

Convergence rates for the stochastic gradient descent method for non-convex objective functions
by: Fehrman, B, et al.
Published: (2020)

Decentralised learning with distributed gradient descent and random features
by: Richards, D, et al.
Published: (2020)