Gradient Decomposition Methods for Training Neural Networks With Non-ideal Synaptic Devices

Gradient Decomposition Methods for Training Neural Networks With Non-ideal Synaptic Devices

While promising for high-capacity machine learning accelerators, memristor devices have non-idealities that prevent software-equivalent accuracies when used for online training. This work uses a combination of Mini-Batch Gradient Descent (MBGD) to average gradients, stochastic rounding to avoid vani...

Full description

Bibliographic Details
Main Authors: Junyun Zhao, Siyuan Huang, Osama Yousuf, Yutong Gao, Brian D. Hoskins, Gina C. Adam
Format: Article
Language:English
Published: Frontiers Media S.A. 2021-11-01
Series:Frontiers in Neuroscience
Subjects:
non-negative matrix factorization
gradient data decomposition
principal component analysis
memristor
non-idealities
ReRAM
Online Access:https://www.frontiersin.org/articles/10.3389/fnins.2021.749811/full

Similar Items

Similar Items