Optimality and sub-optimality of PCA I: Spiked random matrix models
A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or “spike”) is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute of Mathematical Statistics
2020
|
| Online Access: | https://hdl.handle.net/1721.1/125398 |
| _version_ | 1826189611918950400 |
|---|---|
| author | Perry, Amelia E. Wein, Alexander Spence Bandeira, Afonso S. Moitra, Ankur |
| author2 | Massachusetts Institute of Technology. Department of Mathematics |
| author_facet | Massachusetts Institute of Technology. Department of Mathematics Perry, Amelia E. Wein, Alexander Spence Bandeira, Afonso S. Moitra, Ankur |
| author_sort | Perry, Amelia E. |
| collection | MIT |
| description | A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or “spike”) is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and Péché showed that the spiked Wishart ensemble exhibits a sharp phase transition asymptotically: when the spike strength is above a critical threshold, it is possible to detect the presence of a spike based on the top eigenvalue, and below the threshold the top eigenvalue provides no information. Such results form the basis of our understanding of when PCA can detect a low-rank signal in the presence of noise. However, under structural assumptions on the spike, not all information is necessarily contained in the spectrum. We study the statistical limits of tests for the presence of a spike, including nonspectral tests. Our results leverage Le Cam's notion of contiguity and include: (i) For the Gaussian Wigner ensemble, we show that PCA achieves the optimal detection threshold for certain natural priors for the spike. (ii) For any non-Gaussian Wigner ensemble, PCA is sub-optimal for detection. However, an efficient variant of PCA achieves the optimal threshold (for natural priors) by pre-transforming the matrix entries. (iii) For the Gaussian Wishart ensemble, the PCA threshold is optimal for positive spikes (for natural priors) but this is not always the case for negative spikes. Keywords: Random matrix; principal component analysis; hypothesis testing; deformed Wigner; spiked covariance; contiguity; power envelope; phase transition |
| first_indexed | 2024-09-23T08:18:23Z |
| format | Article |
| id | mit-1721.1/125398 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T08:18:23Z |
| publishDate | 2020 |
| publisher | Institute of Mathematical Statistics |
| record_format | dspace |
| spelling | mit-1721.1/1253982022-09-23T12:10:36Z Optimality and sub-optimality of PCA I: Spiked random matrix models Perry, Amelia E. Wein, Alexander Spence Bandeira, Afonso S. Moitra, Ankur Massachusetts Institute of Technology. Department of Mathematics A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or “spike”) is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and Péché showed that the spiked Wishart ensemble exhibits a sharp phase transition asymptotically: when the spike strength is above a critical threshold, it is possible to detect the presence of a spike based on the top eigenvalue, and below the threshold the top eigenvalue provides no information. Such results form the basis of our understanding of when PCA can detect a low-rank signal in the presence of noise. However, under structural assumptions on the spike, not all information is necessarily contained in the spectrum. We study the statistical limits of tests for the presence of a spike, including nonspectral tests. Our results leverage Le Cam's notion of contiguity and include: (i) For the Gaussian Wigner ensemble, we show that PCA achieves the optimal detection threshold for certain natural priors for the spike. (ii) For any non-Gaussian Wigner ensemble, PCA is sub-optimal for detection. However, an efficient variant of PCA achieves the optimal threshold (for natural priors) by pre-transforming the matrix entries. (iii) For the Gaussian Wishart ensemble, the PCA threshold is optimal for positive spikes (for natural priors) but this is not always the case for negative spikes. Keywords: Random matrix; principal component analysis; hypothesis testing; deformed Wigner; spiked covariance; contiguity; power envelope; phase transition NSF CAREER Award (Grant CCF-1453261) NSF Large (Grant CCF-156523) 2020-05-21T20:31:23Z 2020-05-21T20:31:23Z 2018-08 2017-07 2019-11-15T17:41:52Z Article http://purl.org/eprint/type/JournalArticle 0090-5364 https://hdl.handle.net/1721.1/125398 Perry, Amelia et al. "Optimality and sub-optimality of PCA I: Spiked random matrix models." Annals of Statistics 46, 5 (October 2018), 2416-2451. © 2018 Institute of Mathematical Statistics. en http://dx.doi.org/10.1214/17-aos1625 Annals of Statistics Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Mathematical Statistics arXiv |
| spellingShingle | Perry, Amelia E. Wein, Alexander Spence Bandeira, Afonso S. Moitra, Ankur Optimality and sub-optimality of PCA I: Spiked random matrix models |
| title | Optimality and sub-optimality of PCA I: Spiked random matrix models |
| title_full | Optimality and sub-optimality of PCA I: Spiked random matrix models |
| title_fullStr | Optimality and sub-optimality of PCA I: Spiked random matrix models |
| title_full_unstemmed | Optimality and sub-optimality of PCA I: Spiked random matrix models |
| title_short | Optimality and sub-optimality of PCA I: Spiked random matrix models |
| title_sort | optimality and sub optimality of pca i spiked random matrix models |
| url | https://hdl.handle.net/1721.1/125398 |
| work_keys_str_mv | AT perryameliae optimalityandsuboptimalityofpcaispikedrandommatrixmodels AT weinalexanderspence optimalityandsuboptimalityofpcaispikedrandommatrixmodels AT bandeiraafonsos optimalityandsuboptimalityofpcaispikedrandommatrixmodels AT moitraankur optimalityandsuboptimalityofpcaispikedrandommatrixmodels |