Feature learning in deep classifiers through Intermediate Neural Collapse

Feature learning in deep classifiers through Intermediate Neural Collapse

In this paper, we conduct an empirical study of the feature learning process in deep classifiers. Recent research has identified a training phenomenon called Neural Collapse (NC), in which the top-layer feature embeddings of samples from the same class tend to concentrate around their means, and the...

書目詳細資料
Main Authors:	Rangamani, Akshay, Lindegaard, Marius, Galanti, Tomer, Poggio, Tomaso
格式:	Article
出版:	Center for Brains, Minds and Machines (CBMM) 2023
在線閱讀:	https://hdl.handle.net/1721.1/148239

相似書籍

SGD Noise and Implicit Low-Rank Bias in Deep Neural Networks
由: Galanti, Tomer, et al.
出版: (2022)

The Janus effects of SGD vs GD: high noise and low rank
由: Xu, Mengjia, et al.
出版: (2023)

On Generalization Bounds for Neural Networks with Low Rank Layers
由: Pinto, Andrea, et al.
出版: (2024)

SGD and Weight Decay Provably Induce a Low-Rank Bias in Deep Neural Networks
由: Galanti, Tomer, et al.
出版: (2023)

Norm-Based Generalization Bounds for Compositionally Sparse Neural Network
由: Galanti, Tomer, et al.
出版: (2023)

Formation of Representations in Neural Networks
由: Ziyin, Liu, et al.
出版: (2024)

For interpolating kernel machines, the minimum norm ERM solution is the most stable
由: Rangamani, Akshay, et al.
出版: (2020)

Lossy intermediate deep learning feature compression and evaluation
由: Chen, Zhuo, et al.
出版: (2020)

On the Power of Decision Trees in Auto-Regressive Language Modeling
由: Gan, Yulu, et al.
出版: (2024)

Do deep neural networks suffer from crowding?
由: Poggio, Tomaso, et al.
出版: (2021)

Object-Oriented Deep Learning
由: Liao, Qianli, et al.
出版: (2017)

Do Deep Neural Networks Suffer from Crowding?
由: Volokitin, Anna, et al.
出版: (2017)

Do deep neural networks suffer from crowding?
由: Volokitin, Anna, et al.
出版: (2022)

Theory II: Landscape of the Empirical Risk in Deep Learning
由: Poggio, Tomaso, et al.
出版: (2017)

Toward intelligent sensing : intermediate deep feature compression
由: Chen, Zhuo, et al.
出版: (2020)

From Associative Memories to Deep Networks
由: Poggio, Tomaso
出版: (2021)

Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex
由: Liao, Qianli, et al.
出版: (2016)

Learning with group invariant features: A Kernel perspective
由: Mroueh, Youssef, et al.
出版: (2017)

Learning Real and Boolean Functions: When Is Deep Better Than Shallow
由: Mhaskar, Hrushikesh, et al.
出版: (2016)

Learn to navigate through deep neural networks
由: Wu, Keyu
出版: (2020)

One class based feature learning approach for defect detection using deep autoencoders
由: Abdul Mujeeb, et al.
出版: (2020)

Diagnosis of heart valve disorders through trapezoidal features and hybrid classifier.
由: Safara, Fatemeh, et al.
出版: (2013)

An adaptive deep learning neural network model to enhance machine-learning-based classifiers for intrusion detection in smart grids
由: Li, Xue Jun, et al.
出版: (2023)

Compression of Deep Neural Networks for Image Instance Retrieval
由: Chandrasekhar, Vijay, et al.
出版: (2017)

Deep Learning for Seismic Inverse Problems: Toward the Acceleration of Geophysical Analysis Workflows
由: Adler, Amir, et al.
出版: (2022)

Implicit dynamic regularization in deep networks
由: Poggio, Tomaso, et al.
出版: (2020)

Understanding the Role of Recurrent Connections in Assembly Calculus
由: Rangamani, Akshay, et al.
出版: (2022)

Testing deep image classifiers using generative machine learning
由: Dunn, I
出版: (2022)

An Overview of Some Issues in the Theory of Deep Networks
由: Poggio, Tomaso, et al.
出版: (2022)

An Overview of Some Issues in the Theory of Deep Networks
由: Poggio, Tomaso, et al.
出版: (2021)

Feature and classifier development for human detection
由: Amit Satpathy
出版: (2014)

Feature Selection and Ensemble Meta Classifier for Multiclass Imbalance Data Learning
由: Sainin, Mohd Shamrie, et al.
出版: (2018)

Eccentricity dependent deep neural networks: Modeling invariance in human vision
由: Chen, Francis X., et al.
出版: (2017)

Musings on Deep Learning: Properties of SGD
由: Zhang, Chiyuan, et al.
出版: (2017)

An analysis of training and generalization errors in shallow and deep networks
由: Mhaskar, Hrushikesh, et al.
出版: (2018)

Deep vs. shallow networks : An approximation theory perspective
由: Mhaskar, Hrushikesh, et al.
出版: (2016)

Reducing model bias in a deep learning classifier using domain adversarial neural networks in the MINERvA experiment
由: Perdue, GN, et al.
出版: (2018)

Stable Foundations for Learning: a foundational framework for learning theory in both the classical and modern regime.
由: Poggio, Tomaso
出版: (2020)

DeepETC: a deep convolutional neural network architecture for investigating and classifying electron transport chain's complexes
由: Le, Nguyen Quoc Khanh, et al.
出版: (2022)

Theoretical Issues in Deep Networks
由: Poggio, Tomaso, et al.
出版: (2019)