Feature-space selection with banded ridge regression
Encoding models provide a powerful framework to identify the information represented in brain recordings. In this framework, a stimulus representation is expressed within a feature space and is used in a regularized linear regression to predict brain activity. To account for a potential complementar...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2022-12-01
|
| Series: | NeuroImage |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1053811922008497 |
| _version_ | 1828120133889949696 |
|---|---|
| author | Tom Dupré la Tour Michael Eickenberg Anwar O. Nunez-Elizalde Jack L. Gallant |
| author_facet | Tom Dupré la Tour Michael Eickenberg Anwar O. Nunez-Elizalde Jack L. Gallant |
| author_sort | Tom Dupré la Tour |
| collection | DOAJ |
| description | Encoding models provide a powerful framework to identify the information represented in brain recordings. In this framework, a stimulus representation is expressed within a feature space and is used in a regularized linear regression to predict brain activity. To account for a potential complementarity of different feature spaces, a joint model is fit on multiple feature spaces simultaneously. To adapt regularization strength to each feature space, ridge regression is extended to banded ridge regression, which optimizes a different regularization hyperparameter per feature space. The present paper proposes a method to decompose over feature spaces the variance explained by a banded ridge regression model. It also describes how banded ridge regression performs a feature-space selection, effectively ignoring non-predictive and redundant feature spaces. This feature-space selection leads to better prediction accuracy and to better interpretability. Banded ridge regression is then mathematically linked to a number of other regression methods with similar feature-space selection mechanisms. Finally, several methods are proposed to address the computational challenge of fitting banded ridge regressions on large numbers of voxels and feature spaces. All implementations are released in an open-source Python package called Himalaya. |
| first_indexed | 2024-04-11T14:00:35Z |
| format | Article |
| id | doaj.art-796858c858a7430eb0548431722c389b |
| institution | Directory Open Access Journal |
| issn | 1095-9572 |
| language | English |
| last_indexed | 2024-04-11T14:00:35Z |
| publishDate | 2022-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | NeuroImage |
| spelling | doaj.art-796858c858a7430eb0548431722c389b2022-12-22T04:20:08ZengElsevierNeuroImage1095-95722022-12-01264119728Feature-space selection with banded ridge regressionTom Dupré la Tour0Michael Eickenberg1Anwar O. Nunez-Elizalde2Jack L. Gallant3Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USAHelen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USA; Center for Computational Mathematics, Flatiron Institute, 162 5th Ave, New York, NY 10100, USAHelen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USACorresponding author at: Department of Psychology, University of California, Berkeley, CA 94720, USA.; Helen Wills Neuroscience Institute, University of California, Berkeley, CA 94720, USA; Department of Psychology, University of California, Berkeley, CA 94720, USAEncoding models provide a powerful framework to identify the information represented in brain recordings. In this framework, a stimulus representation is expressed within a feature space and is used in a regularized linear regression to predict brain activity. To account for a potential complementarity of different feature spaces, a joint model is fit on multiple feature spaces simultaneously. To adapt regularization strength to each feature space, ridge regression is extended to banded ridge regression, which optimizes a different regularization hyperparameter per feature space. The present paper proposes a method to decompose over feature spaces the variance explained by a banded ridge regression model. It also describes how banded ridge regression performs a feature-space selection, effectively ignoring non-predictive and redundant feature spaces. This feature-space selection leads to better prediction accuracy and to better interpretability. Banded ridge regression is then mathematically linked to a number of other regression methods with similar feature-space selection mechanisms. Finally, several methods are proposed to address the computational challenge of fitting banded ridge regressions on large numbers of voxels and feature spaces. All implementations are released in an open-source Python package called Himalaya.http://www.sciencedirect.com/science/article/pii/S1053811922008497NeuroimagingEncoding modelsRegularized regressionVariance decompositionGroup sparsityHyperparameter optimization |
| spellingShingle | Tom Dupré la Tour Michael Eickenberg Anwar O. Nunez-Elizalde Jack L. Gallant Feature-space selection with banded ridge regression NeuroImage Neuroimaging Encoding models Regularized regression Variance decomposition Group sparsity Hyperparameter optimization |
| title | Feature-space selection with banded ridge regression |
| title_full | Feature-space selection with banded ridge regression |
| title_fullStr | Feature-space selection with banded ridge regression |
| title_full_unstemmed | Feature-space selection with banded ridge regression |
| title_short | Feature-space selection with banded ridge regression |
| title_sort | feature space selection with banded ridge regression |
| topic | Neuroimaging Encoding models Regularized regression Variance decomposition Group sparsity Hyperparameter optimization |
| url | http://www.sciencedirect.com/science/article/pii/S1053811922008497 |
| work_keys_str_mv | AT tomduprelatour featurespaceselectionwithbandedridgeregression AT michaeleickenberg featurespaceselectionwithbandedridgeregression AT anwaronunezelizalde featurespaceselectionwithbandedridgeregression AT jacklgallant featurespaceselectionwithbandedridgeregression |