Computation of the maximum likelihood estimator in low-rank factor analysis
Abstract Factor analysis is a classical multivariate dimensionality reduction technique popularly used in statistics, econometrics and data science. Estimation for factor analysis is often carried out via the maximum likelihood principle, which seeks to maximize the Gaussian likelihoo...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Springer Berlin Heidelberg
2021
|
| Online Access: | https://hdl.handle.net/1721.1/131369 |
| _version_ | 1811078057852665856 |
|---|---|
| author | Khamaru, Koulik Mazumder, Rahul |
| author2 | Sloan School of Management |
| author_facet | Sloan School of Management Khamaru, Koulik Mazumder, Rahul |
| author_sort | Khamaru, Koulik |
| collection | MIT |
| description | Abstract
Factor analysis is a classical multivariate dimensionality reduction technique popularly used in statistics, econometrics and data science. Estimation for factor analysis is often carried out via the maximum likelihood principle, which seeks to maximize the Gaussian likelihood under the assumption that the positive definite covariance matrix can be decomposed as the sum of a low-rank positive semidefinite matrix and a diagonal matrix with nonnegative entries. This leads to a challenging rank constrained nonconvex optimization problem, for which very few reliable computational algorithms are available. We reformulate the low-rank maximum likelihood factor analysis task as a nonlinear nonsmooth semidefinite optimization problem, study various structural properties of this reformulation; and propose fast and scalable algorithms based on difference of convex optimization. Our approach has computational guarantees, gracefully scales to large problems, is applicable to situations where the sample covariance matrix is rank deficient and adapts to variants of the maximum likelihood problem with additional constraints on the model parameters. Our numerical experiments validate the usefulness of our approach over existing state-of-the-art approaches for maximum likelihood factor analysis. |
| first_indexed | 2024-09-23T10:52:42Z |
| format | Article |
| id | mit-1721.1/131369 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T10:52:42Z |
| publishDate | 2021 |
| publisher | Springer Berlin Heidelberg |
| record_format | dspace |
| spelling | mit-1721.1/1313692023-09-27T18:00:17Z Computation of the maximum likelihood estimator in low-rank factor analysis Khamaru, Koulik Mazumder, Rahul Sloan School of Management Massachusetts Institute of Technology. Operations Research Center Abstract Factor analysis is a classical multivariate dimensionality reduction technique popularly used in statistics, econometrics and data science. Estimation for factor analysis is often carried out via the maximum likelihood principle, which seeks to maximize the Gaussian likelihood under the assumption that the positive definite covariance matrix can be decomposed as the sum of a low-rank positive semidefinite matrix and a diagonal matrix with nonnegative entries. This leads to a challenging rank constrained nonconvex optimization problem, for which very few reliable computational algorithms are available. We reformulate the low-rank maximum likelihood factor analysis task as a nonlinear nonsmooth semidefinite optimization problem, study various structural properties of this reformulation; and propose fast and scalable algorithms based on difference of convex optimization. Our approach has computational guarantees, gracefully scales to large problems, is applicable to situations where the sample covariance matrix is rank deficient and adapts to variants of the maximum likelihood problem with additional constraints on the model parameters. Our numerical experiments validate the usefulness of our approach over existing state-of-the-art approaches for maximum likelihood factor analysis. 2021-09-20T17:16:45Z 2021-09-20T17:16:45Z 2019-03-02 2020-09-24T21:02:06Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/131369 en https://doi.org/10.1007/s10107-019-01370-7 Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society application/pdf Springer Berlin Heidelberg Springer Berlin Heidelberg |
| spellingShingle | Khamaru, Koulik Mazumder, Rahul Computation of the maximum likelihood estimator in low-rank factor analysis |
| title | Computation of the maximum likelihood estimator in low-rank factor analysis |
| title_full | Computation of the maximum likelihood estimator in low-rank factor analysis |
| title_fullStr | Computation of the maximum likelihood estimator in low-rank factor analysis |
| title_full_unstemmed | Computation of the maximum likelihood estimator in low-rank factor analysis |
| title_short | Computation of the maximum likelihood estimator in low-rank factor analysis |
| title_sort | computation of the maximum likelihood estimator in low rank factor analysis |
| url | https://hdl.handle.net/1721.1/131369 |
| work_keys_str_mv | AT khamarukoulik computationofthemaximumlikelihoodestimatorinlowrankfactoranalysis AT mazumderrahul computationofthemaximumlikelihoodestimatorinlowrankfactoranalysis |