Formation of Representations in Neural Networks
Understanding neural representations will help open the black box of neural networks and advance our scientific understanding of modern AI systems. However, how complex, structured, and transferable representations emerge in modern neural networks has remained a mystery. Building on previous results...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Published: |
Center for Brains, Minds and Machines (CBMM)
2024
|
| Online Access: | https://hdl.handle.net/1721.1/157132 |
| _version_ | 1824458448720363520 |
|---|---|
| author | Ziyin, Liu Chuang, Isaac Galanti, Tomer Poggio, Tomaso |
| author_facet | Ziyin, Liu Chuang, Isaac Galanti, Tomer Poggio, Tomaso |
| author_sort | Ziyin, Liu |
| collection | MIT |
| description | Understanding neural representations will help open the black box of neural networks and advance our scientific understanding of modern AI systems. However, how complex, structured, and transferable representations emerge in modern neural networks has remained a mystery. Building on previous results, we propose the Canonical Representation Hypothesis (CRH), which posits a set of six alignment relations to universally govern the formation of representations in most hidden layers of a neural network. Under the CRH, the latent representations (R), weights (W), and neuron gradients (G) become mutually aligned during training. This alignment implies that neural networks naturally learn compact representations, where neurons and weights are invariant to task-irrelevant transformations. We then show that the breaking of CRH leads to the emergence of reciprocal power-law relations between R, W, and G, which we refer to as the Polynomial Alignment Hypothesis (PAH). We present a minimal-assumption theory demonstrating that the balance between gradient noise and regularization is crucial for the emergence the canonical representation. The CRH and PAH lead to an exciting possibility of unifying major key deep learning phenomena, including neural collapse and the neural feature ansatz, in a single framework. |
| first_indexed | 2025-02-19T04:26:03Z |
| format | Article |
| id | mit-1721.1/157132 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2025-02-19T04:26:03Z |
| publishDate | 2024 |
| publisher | Center for Brains, Minds and Machines (CBMM) |
| record_format | dspace |
| spelling | mit-1721.1/1571322024-10-09T03:01:45Z Formation of Representations in Neural Networks Ziyin, Liu Chuang, Isaac Galanti, Tomer Poggio, Tomaso Understanding neural representations will help open the black box of neural networks and advance our scientific understanding of modern AI systems. However, how complex, structured, and transferable representations emerge in modern neural networks has remained a mystery. Building on previous results, we propose the Canonical Representation Hypothesis (CRH), which posits a set of six alignment relations to universally govern the formation of representations in most hidden layers of a neural network. Under the CRH, the latent representations (R), weights (W), and neuron gradients (G) become mutually aligned during training. This alignment implies that neural networks naturally learn compact representations, where neurons and weights are invariant to task-irrelevant transformations. We then show that the breaking of CRH leads to the emergence of reciprocal power-law relations between R, W, and G, which we refer to as the Polynomial Alignment Hypothesis (PAH). We present a minimal-assumption theory demonstrating that the balance between gradient noise and regularization is crucial for the emergence the canonical representation. The CRH and PAH lead to an exciting possibility of unifying major key deep learning phenomena, including neural collapse and the neural feature ansatz, in a single framework. This material is based upon work supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF-1231216. 2024-10-08T14:32:03Z 2024-10-08T14:32:03Z 2024-10-07 Article Technical Report Working Paper https://hdl.handle.net/1721.1/157132 CBMM Memo;150 application/pdf Center for Brains, Minds and Machines (CBMM) |
| spellingShingle | Ziyin, Liu Chuang, Isaac Galanti, Tomer Poggio, Tomaso Formation of Representations in Neural Networks |
| title | Formation of Representations in Neural Networks |
| title_full | Formation of Representations in Neural Networks |
| title_fullStr | Formation of Representations in Neural Networks |
| title_full_unstemmed | Formation of Representations in Neural Networks |
| title_short | Formation of Representations in Neural Networks |
| title_sort | formation of representations in neural networks |
| url | https://hdl.handle.net/1721.1/157132 |
| work_keys_str_mv | AT ziyinliu formationofrepresentationsinneuralnetworks AT chuangisaac formationofrepresentationsinneuralnetworks AT galantitomer formationofrepresentationsinneuralnetworks AT poggiotomaso formationofrepresentationsinneuralnetworks |