Noisy Independent Factor Analysis Model for Density Estimation and Classification
We consider the problem of multivariate density estimation when the unknown density is assumed to follow a particular form of dimensionality reduction, a noisy independent factor analysis (IFA) model. In this model the data are generated by a number of latent independent components having unknown di...
| Main Authors: | , , , |
|---|---|
| Format: | Working Paper |
| Language: | en_US |
| Published: |
Cambridge, MA; Alfred P. Sloan School of Management, Massachusetts Institute of Technology
2011
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/66262 |
| _version_ | 1811097863971667968 |
|---|---|
| author | Amato, U. Antoniadis, A. Samarov, A. Tsybakov, A.B. |
| author_facet | Amato, U. Antoniadis, A. Samarov, A. Tsybakov, A.B. |
| author_sort | Amato, U. |
| collection | MIT |
| description | We consider the problem of multivariate density estimation when the unknown density is assumed to follow a particular form of dimensionality reduction, a noisy independent factor analysis (IFA) model. In this model the data are generated by a number of latent independent components having unknown distributions and are observed in Gaussian noise. We do not assume that either the number of components or the matrix mixing the components are known. We show that the densities of this form can be estimated with a fast rate. Using the mirror averaging aggregation algorithm, we construct a density estimator which achieves a nearly parametric rate (log1/4 n)/√n, independent of the dimensionality of the data, as the sample size n tends to infinity. This estimator is adaptive to the number of components, their distributions and the mixing matrix. We then apply this density estimator to construct nonparametric plug-in classifiers and show that they achieve the best obtainable rate of the excess Bayes risk, to within a logarithmic factor independent of the dimension of the data. Applications of this classifier to simulated data sets and to real data from a remote sensing experiment show promising results. |
| first_indexed | 2024-09-23T17:06:10Z |
| format | Working Paper |
| id | mit-1721.1/66262 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T17:06:10Z |
| publishDate | 2011 |
| publisher | Cambridge, MA; Alfred P. Sloan School of Management, Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/662622019-04-10T10:37:31Z Noisy Independent Factor Analysis Model for Density Estimation and Classification Amato, U. Antoniadis, A. Samarov, A. Tsybakov, A.B. Aggregation, Remote sensing Plug-in classifier Independent Factor Analysis Nonparametric Density Estimation We consider the problem of multivariate density estimation when the unknown density is assumed to follow a particular form of dimensionality reduction, a noisy independent factor analysis (IFA) model. In this model the data are generated by a number of latent independent components having unknown distributions and are observed in Gaussian noise. We do not assume that either the number of components or the matrix mixing the components are known. We show that the densities of this form can be estimated with a fast rate. Using the mirror averaging aggregation algorithm, we construct a density estimator which achieves a nearly parametric rate (log1/4 n)/√n, independent of the dimensionality of the data, as the sample size n tends to infinity. This estimator is adaptive to the number of components, their distributions and the mixing matrix. We then apply this density estimator to construct nonparametric plug-in classifiers and show that they achieve the best obtainable rate of the excess Bayes risk, to within a logarithmic factor independent of the dimension of the data. Applications of this classifier to simulated data sets and to real data from a remote sensing experiment show promising results. Financial support from the IAP research network of the Belgian government (Belgian Federal Science Policy) is gratefully acknowledged. Research of A. Samarov was partially supported by NSF grant DMS- 0505561 and by a grant from Singapore-MIT Alliance (CSB). Research of A.B. Tsybakov was partially supported by the grant ANR-06-BLAN-0194 and by the PASCAL Network of Excellence. 2011-10-14T19:35:28Z 2011-10-14T19:35:28Z 2009-06-09 Working Paper http://hdl.handle.net/1721.1/66262 en_US MIT Sloan School of Management Working Paper;4746-09 application/pdf Cambridge, MA; Alfred P. Sloan School of Management, Massachusetts Institute of Technology |
| spellingShingle | Aggregation, Remote sensing Plug-in classifier Independent Factor Analysis Nonparametric Density Estimation Amato, U. Antoniadis, A. Samarov, A. Tsybakov, A.B. Noisy Independent Factor Analysis Model for Density Estimation and Classification |
| title | Noisy Independent Factor Analysis Model for Density Estimation and Classification |
| title_full | Noisy Independent Factor Analysis Model for Density Estimation and Classification |
| title_fullStr | Noisy Independent Factor Analysis Model for Density Estimation and Classification |
| title_full_unstemmed | Noisy Independent Factor Analysis Model for Density Estimation and Classification |
| title_short | Noisy Independent Factor Analysis Model for Density Estimation and Classification |
| title_sort | noisy independent factor analysis model for density estimation and classification |
| topic | Aggregation, Remote sensing Plug-in classifier Independent Factor Analysis Nonparametric Density Estimation |
| url | http://hdl.handle.net/1721.1/66262 |
| work_keys_str_mv | AT amatou noisyindependentfactoranalysismodelfordensityestimationandclassification AT antoniadisa noisyindependentfactoranalysismodelfordensityestimationandclassification AT samarova noisyindependentfactoranalysismodelfordensityestimationandclassification AT tsybakovab noisyindependentfactoranalysismodelfordensityestimationandclassification |