Information-bottleneck under mean field initialization
This work explores the sensitivity of mutual information (MI) flow in hidden layers of very deep neural networks (DNNs) as a function of the initialization variance. Specifically, we demonstrate that information-bottleneck (IB) interpretations of DNNs are significantly affected by their choice of no...
| Main Authors: | , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
2020
|