Regularity Normalization: Neuroscience-Inspired Unsupervised Attention across Neural Network Layers
Inspired by the adaptation phenomenon of neuronal firing, we propose the regularity normalization (RN) as an unsupervised attention mechanism (UAM) which computes the statistical regularity in the implicit space of neural networks under the Minimum Description Length (MDL) principle. Treating the ne...
Main Author: | Baihan Lin |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2021-12-01
|
Series: | Entropy |
Subjects: | |
Online Access: | https://www.mdpi.com/1099-4300/24/1/59 |
Similar Items
-
Model Selection for Non-Negative Tensor Factorization with Minimum Description Length
by: Yunhui Fu, et al.
Published: (2019-06-01) -
Biologically-inspired neuronal adaptation improves learning in neural networks
by: Yoshimasa Kubo, et al.
Published: (2023-12-01) -
Minimum Description Length Codes Are Critical
by: Ryan John Cubero, et al.
Published: (2018-10-01) -
Word2vec Skip-Gram Dimensionality Selection via Sequential Normalized Maximum Likelihood
by: Pham Thuc Hung, et al.
Published: (2021-07-01) -
Data Discovery and Anomaly Detection Using Atypicality for Real-Valued Data
by: Elyas Sabeti, et al.
Published: (2019-02-01)