For interpolating kernel machines, the minimum norm ERM solution is the most stable

For interpolating kernel machines, the minimum norm ERM solution is the most stable

We study the average CVloo stability of kernel ridge-less regression and derive corresponding risk bounds. We show that the interpolating solution with minimum norm has the best CVloo stability, which in turn is controlled by the condition number of the empirical kernel matrix. The latter can be cha...

Bibliográfalaš dieđut
Váldodahkkit:	Rangamani, Akshay, Rosasco, Lorenzo, Poggio, Tomaso
Materiálatiipa:	Technical Report
Almmustuhtton:	Center for Brains, Minds and Machines (CBMM) 2020
Liŋkkat:	https://hdl.handle.net/1721.1/125927

Geahča maid

Deep Convolutional Networks are Hierarchical Kernel Machines
Dahkki: Anselmi, Fabio, et al.
Almmustuhtton: (2015)

The Janus effects of SGD vs GD: high noise and low rank
Dahkki: Xu, Mengjia, et al.
Almmustuhtton: (2023)

Learning and Invariance in a Family of Hierarchical Kernels
Dahkki: Wibisono, Andre, et al.
Almmustuhtton: (2010)

On Generalization Bounds for Neural Networks with Low Rank Layers
Dahkki: Pinto, Andrea, et al.
Almmustuhtton: (2024)

Feature learning in deep classifiers through Intermediate Neural Collapse
Dahkki: Rangamani, Akshay, et al.
Almmustuhtton: (2023)

Regularization Predicts While Discovering Taxonomy
Dahkki: Mroueh, Youssef, et al.
Almmustuhtton: (2011)

Sufficient Conditions for Uniform Stability of Regularization Algorithms
Dahkki: Poggio, Tomaso, et al.
Almmustuhtton: (2009)

On Invariance and Selectivity in Representation Learning
Dahkki: Anselmi, Fabio, et al.
Almmustuhtton: (2015)

I-theory on depth vs width: hierarchical function composition
Dahkki: Poggio, Tomaso, et al.
Almmustuhtton: (2015)

Holographic Embeddings of Knowledge Graphs
Dahkki: Nickel, Maximilian, et al.
Almmustuhtton: (2015)

On invariance and selectivity in representation learning
Dahkki: Anselmi, Fabio, et al.
Almmustuhtton: (2017)

Holographic embeddings of knowledge graphs
Dahkki: Nickel, Maximilian, et al.
Almmustuhtton: (2017)

Symmetry Regularization
Dahkki: Anselmi, Fabio, et al.
Almmustuhtton: (2017)

Adaptive Kernel Methods Using the Balancing Principle
Dahkki: Rosasco, Lorenzo, et al.
Almmustuhtton: (2008)

Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization
Dahkki: Recht, Benjamin, et al.
Almmustuhtton: (2011)

Hidden minimum-norm problems in quadratic programming
Almmustuhtton: (2003)

Kernels for Vector-Valued Functions: a Review
Dahkki: Alvarez, Mauricio A., et al.
Almmustuhtton: (2011)

Convex learning of multiple tasks and their structure
Dahkki: Ciliberto, Carlo, et al.
Almmustuhtton: (2017)

On invariance in hierarchical models
Dahkki: Bouvrie, Jacob Vincent, et al.
Almmustuhtton: (2014)

Learning manifolds with k-means and k-flats
Dahkki: Canas, Guillermo D., et al.
Almmustuhtton: (2014)

Just interpolate: Kernel “Ridgeless” regression can generalize
Dahkki: Liang, Tengyuan, et al.
Almmustuhtton: (2021)

Sparse Correlation Kernel Analysis and Reconstruction
Dahkki: Papgeorgiou, Constantine P., et al.
Almmustuhtton: (2004)

Norm-Based Generalization Bounds for Compositionally Sparse Neural Network
Dahkki: Galanti, Tomer, et al.
Almmustuhtton: (2023)

Stable Foundations for Learning: a foundational framework for learning theory in both the classical and modern regime.
Dahkki: Poggio, Tomaso
Almmustuhtton: (2020)

Generalization and Properties of the Neural Response
Dahkki: Bouvrie, Jake, et al.
Almmustuhtton: (2010)

Learning An Invariant Speech Representation
Dahkki: Evangelopoulos, Georgios, et al.
Almmustuhtton: (2015)

Mathematics of the Neural Response
Dahkki: Caponnetto, Andrea, et al.
Almmustuhtton: (2008)

Multi-Class Learning: Simplex Coding And Relaxation Error
Dahkki: Mroueh, Youssef, et al.
Almmustuhtton: (2011)

The computational magic of the ventral stream: sketch of a theory (and why some deep architectures work).
Dahkki: Poggio, Tomaso, et al.
Almmustuhtton: (2013)

A Deep Representation for Invariance And Music Classification
Dahkki: Zhang, Chiyuan, et al.
Almmustuhtton: (2015)

Notes on Hierarchical Splines, DCLNs and i-theory
Dahkki: Poggio, Tomaso, et al.
Almmustuhtton: (2015)

Theory I: Why and When Can Deep Networks Avoid the Curse of Dimensionality?
Dahkki: Poggio, Tomaso, et al.
Almmustuhtton: (2016)

Vision by Man and Machine
Dahkki: Poggio, Tomaso
Almmustuhtton: (2004)

Review of enterprise risk management (ERM) literature
Dahkki: Haji Togok, S., et al.
Almmustuhtton: (2014)

Maximum and minimum norms for τ-NAF expansion on Koblitz Curve
Dahkki: Ali, Nur Adawiah, et al.
Almmustuhtton: (2016)

Learning Generic Invariances in Object Recognition: Translation and Scale
Dahkki: Leibo, Joel Z, et al.
Almmustuhtton: (2011)

Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review
Dahkki: Mhaskar, Hrushikesh, et al.
Almmustuhtton: (2017)

Neurons That Confuse Mirror-Symmetric Object Views
Dahkki: Mutch, Jim, et al.
Almmustuhtton: (2011)

Learning with group invariant features: A Kernel perspective
Dahkki: Mroueh, Youssef, et al.
Almmustuhtton: (2017)

Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers
Dahkki: Bao, Yujia, et al.
Almmustuhtton: (2022)