Accumulated decoupled learning with gradient staleness mitigation for convolutional neural networks

Accumulated decoupled learning with gradient staleness mitigation for convolutional neural networks

Gradient staleness is a major side effect in decoupled learning when training convolutional neural networks asynchronously. Existing methods that ignore this effect might result in reduced generalization and even divergence. In this paper, we propose an accumulated decoupled learning (ADL), wh...

Full description

Bibliographic Details
Main Authors: Zhuang, Huiping, Weng, Zhenyu, Luo, Fulin, Toh, Kar-Ann, Li, Haizhou, Lin, Zhiping
Other Authors: School of Electrical and Electronic Engineering
Format: Conference Paper
Language:English
Published: 2024
Subjects:
Computer and Information Science
Convolutional neural networks
Delayed gradients-based methods
Online Access:https://hdl.handle.net/10356/174480
https://icml.cc/virtual/2021/index.html

Similar Items

Similar Items