Consistency and Generalization Bounds for Maximum Entropy Density Estimation
We investigate the statistical properties of maximum entropy density estimation, both for the complete data case and the incomplete data case. We show that under certain assumptions, the generalization error can be bounded in terms of the complexity of the underlying feature functions. This allows u...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2013-12-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1099-4300/15/12/5439 |
| _version_ | 1828395639105388544 |
|---|---|
| author | Shaojun Wang Russell Greiner Shaomin Wang |
| author_facet | Shaojun Wang Russell Greiner Shaomin Wang |
| author_sort | Shaojun Wang |
| collection | DOAJ |
| description | We investigate the statistical properties of maximum entropy density estimation, both for the complete data case and the incomplete data case. We show that under certain assumptions, the generalization error can be bounded in terms of the complexity of the underlying feature functions. This allows us to establish the universal consistency of maximum entropy density estimation. |
| first_indexed | 2024-12-10T08:16:32Z |
| format | Article |
| id | doaj.art-f9ee22c315664032b2b595eb909067be |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-12-10T08:16:32Z |
| publishDate | 2013-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-f9ee22c315664032b2b595eb909067be2022-12-22T01:56:27ZengMDPI AGEntropy1099-43002013-12-0115125439546310.3390/e15125439e15125439Consistency and Generalization Bounds for Maximum Entropy Density EstimationShaojun Wang0Russell Greiner1Shaomin Wang2Kno.e.sis Center, Department of Computer Science and Engineering, Wright State University, Dayton, OH 45435, USADepartment of Computing Science, University of Alberta, Edmonton, Alberta T6G 2E8, CanadaVisa Inc., San Francisco, CA 94128, USAWe investigate the statistical properties of maximum entropy density estimation, both for the complete data case and the incomplete data case. We show that under certain assumptions, the generalization error can be bounded in terms of the complexity of the underlying feature functions. This allows us to establish the universal consistency of maximum entropy density estimation.http://www.mdpi.com/1099-4300/15/12/5439maximum entropy principledensity estimationgeneralization boundconsistency |
| spellingShingle | Shaojun Wang Russell Greiner Shaomin Wang Consistency and Generalization Bounds for Maximum Entropy Density Estimation Entropy maximum entropy principle density estimation generalization bound consistency |
| title | Consistency and Generalization Bounds for Maximum Entropy Density Estimation |
| title_full | Consistency and Generalization Bounds for Maximum Entropy Density Estimation |
| title_fullStr | Consistency and Generalization Bounds for Maximum Entropy Density Estimation |
| title_full_unstemmed | Consistency and Generalization Bounds for Maximum Entropy Density Estimation |
| title_short | Consistency and Generalization Bounds for Maximum Entropy Density Estimation |
| title_sort | consistency and generalization bounds for maximum entropy density estimation |
| topic | maximum entropy principle density estimation generalization bound consistency |
| url | http://www.mdpi.com/1099-4300/15/12/5439 |
| work_keys_str_mv | AT shaojunwang consistencyandgeneralizationboundsformaximumentropydensityestimation AT russellgreiner consistencyandgeneralizationboundsformaximumentropydensityestimation AT shaominwang consistencyandgeneralizationboundsformaximumentropydensityestimation |