Stable Mixing of Complete and Incomplete Information
An increasing number of parameter estimation tasks involve the use of at least two information sources, one complete but limited, the other abundant but incomplete. Standard algorithms such as EM (or em) used in this context are unfortunately not stable in the sense that they can lead to a dramatic...
| Main Authors: | , |
|---|---|
| Language: | en_US |
| Published: |
2004
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/6679 |
| _version_ | 1826212663821074432 |
|---|---|
| author | Corduneanu, Adrian Jaakkola, Tommi |
| author_facet | Corduneanu, Adrian Jaakkola, Tommi |
| author_sort | Corduneanu, Adrian |
| collection | MIT |
| description | An increasing number of parameter estimation tasks involve the use of at least two information sources, one complete but limited, the other abundant but incomplete. Standard algorithms such as EM (or em) used in this context are unfortunately not stable in the sense that they can lead to a dramatic loss of accuracy with the inclusion of incomplete observations. We provide a more controlled solution to this problem through differential equations that govern the evolution of locally optimal solutions (fixed points) as a function of the source weighting. This approach permits us to explicitly identify any critical (bifurcation) points leading to choices unsupported by the available complete data. The approach readily applies to any graphical model in O(n^3) time where n is the number of parameters. We use the naive Bayes model to illustrate these ideas and demonstrate the effectiveness of our approach in the context of text classification problems. |
| first_indexed | 2024-09-23T15:31:57Z |
| id | mit-1721.1/6679 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T15:31:57Z |
| publishDate | 2004 |
| record_format | dspace |
| spelling | mit-1721.1/66792019-04-11T02:52:49Z Stable Mixing of Complete and Incomplete Information Corduneanu, Adrian Jaakkola, Tommi AI semi-supervised learning incomplete data EM stable estimation An increasing number of parameter estimation tasks involve the use of at least two information sources, one complete but limited, the other abundant but incomplete. Standard algorithms such as EM (or em) used in this context are unfortunately not stable in the sense that they can lead to a dramatic loss of accuracy with the inclusion of incomplete observations. We provide a more controlled solution to this problem through differential equations that govern the evolution of locally optimal solutions (fixed points) as a function of the source weighting. This approach permits us to explicitly identify any critical (bifurcation) points leading to choices unsupported by the available complete data. The approach readily applies to any graphical model in O(n^3) time where n is the number of parameters. We use the naive Bayes model to illustrate these ideas and demonstrate the effectiveness of our approach in the context of text classification problems. 2004-10-08T20:37:18Z 2004-10-08T20:37:18Z 2001-11-08 AIM-2001-030 http://hdl.handle.net/1721.1/6679 en_US AIM-2001-030 9 p. 1207127 bytes 733599 bytes application/postscript application/pdf application/postscript application/pdf |
| spellingShingle | AI semi-supervised learning incomplete data EM stable estimation Corduneanu, Adrian Jaakkola, Tommi Stable Mixing of Complete and Incomplete Information |
| title | Stable Mixing of Complete and Incomplete Information |
| title_full | Stable Mixing of Complete and Incomplete Information |
| title_fullStr | Stable Mixing of Complete and Incomplete Information |
| title_full_unstemmed | Stable Mixing of Complete and Incomplete Information |
| title_short | Stable Mixing of Complete and Incomplete Information |
| title_sort | stable mixing of complete and incomplete information |
| topic | AI semi-supervised learning incomplete data EM stable estimation |
| url | http://hdl.handle.net/1721.1/6679 |
| work_keys_str_mv | AT corduneanuadrian stablemixingofcompleteandincompleteinformation AT jaakkolatommi stablemixingofcompleteandincompleteinformation |