Maximum Entropy Discrimination
We present a general framework for discriminative estimation based on the maximum entropy principle and its extensions. All calculations involve distributions over structures and/or parameters rather than specific settings and reduce to relative entropy projections. This holds even when the d...
| Main Authors: | , , |
|---|---|
| Language: | en_US |
| Published: |
2004
|
| Online Access: | http://hdl.handle.net/1721.1/7089 |
| _version_ | 1811079532467191808 |
|---|---|
| author | Jaakkola, Tommi Meila, Marina Jebara, Tony |
| author_facet | Jaakkola, Tommi Meila, Marina Jebara, Tony |
| author_sort | Jaakkola, Tommi |
| collection | MIT |
| description | We present a general framework for discriminative estimation based on the maximum entropy principle and its extensions. All calculations involve distributions over structures and/or parameters rather than specific settings and reduce to relative entropy projections. This holds even when the data is not separable within the chosen parametric class, in the context of anomaly detection rather than classification, or when the labels in the training set are uncertain or incomplete. Support vector machines are naturally subsumed under this class and we provide several extensions. We are also able to estimate exactly and efficiently discriminative distributions over tree structures of class-conditional models within this framework. Preliminary experimental results are indicative of the potential in these techniques. |
| first_indexed | 2024-09-23T11:16:33Z |
| id | mit-1721.1/7089 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T11:16:33Z |
| publishDate | 2004 |
| record_format | dspace |
| spelling | mit-1721.1/70892019-04-12T08:33:54Z Maximum Entropy Discrimination Jaakkola, Tommi Meila, Marina Jebara, Tony We present a general framework for discriminative estimation based on the maximum entropy principle and its extensions. All calculations involve distributions over structures and/or parameters rather than specific settings and reduce to relative entropy projections. This holds even when the data is not separable within the chosen parametric class, in the context of anomaly detection rather than classification, or when the labels in the training set are uncertain or incomplete. Support vector machines are naturally subsumed under this class and we provide several extensions. We are also able to estimate exactly and efficiently discriminative distributions over tree structures of class-conditional models within this framework. Preliminary experimental results are indicative of the potential in these techniques. 2004-10-20T20:29:28Z 2004-10-20T20:29:28Z 1999-12-01 AITR-1668 http://hdl.handle.net/1721.1/7089 en_US AITR-1668 6420262 bytes 1702298 bytes application/postscript application/pdf application/postscript application/pdf |
| spellingShingle | Jaakkola, Tommi Meila, Marina Jebara, Tony Maximum Entropy Discrimination |
| title | Maximum Entropy Discrimination |
| title_full | Maximum Entropy Discrimination |
| title_fullStr | Maximum Entropy Discrimination |
| title_full_unstemmed | Maximum Entropy Discrimination |
| title_short | Maximum Entropy Discrimination |
| title_sort | maximum entropy discrimination |
| url | http://hdl.handle.net/1721.1/7089 |
| work_keys_str_mv | AT jaakkolatommi maximumentropydiscrimination AT meilamarina maximumentropydiscrimination AT jebaratony maximumentropydiscrimination |