Streaming Normalization: Towards Simpler and More Biologically-plausible Normalizations for Online and Recurrent Learning

We systematically explored a spectrum of normalization algorithms related to Batch Normalization (BN) and propose a generalized formulation that simultaneously solves two major limitations of BN: (1) online learning and (2) recurrent learning. Our proposal is simpler and more biologically-plausible....

Full description

Bibliographic Details
Main Authors: Liao, Qianli, Kawaguchi, Kenji, Poggio, Tomaso
Format: Technical Report
Language:en_US
Published: Center for Brains, Minds and Machines (CBMM), arXiv 2016
Subjects:
Online Access:http://hdl.handle.net/1721.1/104906