Convergence of Stochastic Proximal Gradient Algorithm
Abstract We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case, as well as almost sure convergence and convergence rates for the iterate...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Springer US
2021
|
| Online Access: | https://hdl.handle.net/1721.1/131842 |
| _version_ | 1826205435099611136 |
|---|---|
| author | Rosasco, Lorenzo Villa, Silvia Vũ, Bằng C |
| author2 | Center for Brains, Minds, and Machines |
| author_facet | Center for Brains, Minds, and Machines Rosasco, Lorenzo Villa, Silvia Vũ, Bằng C |
| author_sort | Rosasco, Lorenzo |
| collection | MIT |
| description | Abstract
We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case, as well as almost sure convergence and convergence rates for the iterates under further convexity assumptions. Our analysis avoid averaging the iterates and error summability assumptions which might not be satisfied in applications, e.g. in machine learning. Our proofing technique extends classical ideas from the analysis of deterministic proximal gradient algorithms. |
| first_indexed | 2024-09-23T13:13:06Z |
| format | Article |
| id | mit-1721.1/131842 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T13:13:06Z |
| publishDate | 2021 |
| publisher | Springer US |
| record_format | dspace |
| spelling | mit-1721.1/1318422023-11-03T21:00:48Z Convergence of Stochastic Proximal Gradient Algorithm Rosasco, Lorenzo Villa, Silvia Vũ, Bằng C Center for Brains, Minds, and Machines Abstract We study the extension of the proximal gradient algorithm where only a stochastic gradient estimate is available and a relaxation step is allowed. We establish convergence rates for function values in the convex case, as well as almost sure convergence and convergence rates for the iterates under further convexity assumptions. Our analysis avoid averaging the iterates and error summability assumptions which might not be satisfied in applications, e.g. in machine learning. Our proofing technique extends classical ideas from the analysis of deterministic proximal gradient algorithms. 2021-09-20T17:30:33Z 2021-09-20T17:30:33Z 2019-10-15 2020-10-28T04:27:59Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/131842 en https://doi.org/10.1007/s00245-019-09617-7 Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. Springer Science+Business Media, LLC, part of Springer Nature application/pdf Springer US Springer US |
| spellingShingle | Rosasco, Lorenzo Villa, Silvia Vũ, Bằng C Convergence of Stochastic Proximal Gradient Algorithm |
| title | Convergence of Stochastic Proximal Gradient Algorithm |
| title_full | Convergence of Stochastic Proximal Gradient Algorithm |
| title_fullStr | Convergence of Stochastic Proximal Gradient Algorithm |
| title_full_unstemmed | Convergence of Stochastic Proximal Gradient Algorithm |
| title_short | Convergence of Stochastic Proximal Gradient Algorithm |
| title_sort | convergence of stochastic proximal gradient algorithm |
| url | https://hdl.handle.net/1721.1/131842 |
| work_keys_str_mv | AT rosascolorenzo convergenceofstochasticproximalgradientalgorithm AT villasilvia convergenceofstochasticproximalgradientalgorithm AT vubangc convergenceofstochasticproximalgradientalgorithm |