Acceleration in First Order Quasi-strongly Convex Optimization by ODE Discretization

Acceleration in First Order Quasi-strongly Convex Optimization by ODE Discretization

We study gradient-based optimization methods obtained by direct Runge-Kutta discretization of the ordinary differential equation (ODE) describing the movement of a heavy-ball under constant friction coefficient. When the function is high-order smooth and strongly convex, we show that directly simula...

书目详细资料
Main Authors:	Zhang, Jingzhao, Sra, Suvrit, Jadbabaie, Ali
其他作者:	Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
格式:	文件
语言:	English
出版:	Institute of Electrical and Electronics Engineers (IEEE) 2021
在线阅读:	https://hdl.handle.net/1721.1/130426

相似书籍

Direct Runge-Kutta discretization achieves acceleration
由: Zhang, Jingzhao, et al.
出版: (2021)

Direct Runge-Kutta discretization achieves acceleration
由: Zhang, Jingzhao, et al.
出版: (2022)

Achieving acceleration in distributed optimization via direct discretization of the heavy-ball ODE
由: Zhang, J, et al.
出版: (2023)

New concavity and convexity results for symmetric polynomials and their ratios
由: Sra, Suvrit
出版: (2021)

New concavity and convexity results for symmetric polynomials and their ratios
由: Sra, Suvrit
出版: (2021)

Efficiently testing local optimality and escaping saddles for RELu networks
由: Jadbabaie, Ali, et al.
出版: (2021)

Dynamical systems view of acceleration in first order optimization
由: Zhang, Jingzhao,S.M.Massachusetts Institute of Technology.
出版: (2019)

Functions with Positive Differences on Convex Cones
由: Niculescu, Constantin P., et al.
出版: (2023)

Small ReLU networks are powerful memorizers: A tight analysis of memorization capacity
由: Yun, Chulhee, et al.
出版: (2021)

Small nonlinearities in activation functions create bad local minima in neural networks
由: Yun, Chulee, et al.
出版: (2021)

On geodesically convex formulations for the brascamp-lieb constant
由: Sra, Suvrit, et al.
出版: (2022)

Logarithmic inequalities under a symmetric polynomial dominance order
由: Sra, Suvrit
出版: (2021)

Logarithmic inequalities under a symmetric polynomial dominance order
由: Sra, Suvrit
出版: (2021)

Acceleration in first-order optimization methods: promenading beyond convexity or smoothness, and applications
由: Martinez Rubio, D
出版: (2021)

Are deep ResNets provably better than linear predictors?
由: Yun, Chulhee, et al.
出版: (2022)

Exponentiated strongly Rayleigh distributions
由: Mariet, Zelda, et al.
出版: (2022)

Modular proximal optimization for multidimensional total-variation regularization
由: Sra, Suvrit
出版: (2021)

Provably Efficient Algorithms for Multi-Objective Competitive RL
由: Yu, Tiancheng, et al.
出版: (2022)

Flexible modeling of diversity with strongly log-concave distributions
由: Robinson, Joshua, et al.
出版: (2022)

High-order tuners for convex optimization : stability and accelerated learning
由: Moreu Gamazo, José M.(José María)
出版: (2021)

Fast mixing Markov chains for strongly rayleigh measures, DPPs, and constrained sampling
由: Li, Chengtao, et al.
出版: (2021)

Robust Accelerated Gradient Methods for Smooth Strongly Convex Functions
由: Aybat, Necdet Serhat, et al.
出版: (2021)

On Generalized Strongly p-Convex Functions of Higher Order
由: Muhammad Shoaib Saleem, et al.
出版: (2020-01-01)

On the strongly starlikeness of multivalently convex functions of order α
由: Mamoru Nunokawa, et al.
出版: (2001-01-01)

Riemannian Optimization via Frank-Wolfe Methods
由: Weber, Melanie, et al.
出版: (2022)

Projection-free nonconvex stochastic optimization on Riemannian manifolds
由: Weber, Melanie, et al.
出版: (2022)

Symmetry Classification of First Integrals for Scalar Linearizable Second-Order ODEs
由: K. S. Mahomed, et al.
出版: (2012-01-01)

Relatively Smooth Convex Optimization by First-Order Methods, and Applications
由: Nesterov, Yurii, et al.
出版: (2019)

An alternative to EM for Gaussian mixture models: batch and stochastic Riemannian optimization
由: Hosseini, Reshad, et al.
出版: (2021)

Greed, hedging, and acceleration in convex optimization
由: Altschuler, Jason (Jason M.)
出版: (2019)

Starlikeness, Convexity, Close-to-Convexity, and Quasi-Convexity for Functions with Fixed Initial Coefficients
由: Mohanad Kadhim Ahmed Alkarafi, et al.
出版: (2024-10-01)

Conditional gradient methods via stochastic path-integrated differential estimator
由: Sra, Suvrit
出版: (2021)

3-Point implicit block multistep method for solution of first order ODEs.
由: Mehrkanoon, S., et al.
出版: (2012)

New technique of VSVO coupled block method for solving first order ODEs.
由: Abdul Majid, Zanariah, et al.
出版: (2011)

Explicit Runge-Kutta method with trigonometrically-fitted for solving first order ODEs
由: Alshareeda, Firas Adel Fawzi, et al.
出版: (2016)

Implementation of 2-point 2-step methods for the solution of first order ODEs.
由: Mehrkanoon, S., et al.
出版: (2010)

Variable steps implicit block multistep method for solving first order ODEs
由: S., Mehrkanoon, et al.
出版: (2010)

An Implementable First-Order Primal-Dual Algorithm for Structured Convex Optimization
由: Feng Ma, et al.
出版: (2014-01-01)

On geodesic strongly E-convex sets and geodesic strongly E-convex functions
由: Kilicman, Adem, et al.
出版: (2015)

Algebraic Properties of First Integrals for Scalar Linear Third-Order ODEs of Maximal Symmetry
由: K. S. Mahomed, et al.
出版: (2013-01-01)