Learning to classify from impure samples with high-dimensional data
A persistent challenge in practical classification tasks is that labeled training sets are not always available. In particle physics, this challenge is surmounted by the use of simulations. These simulations accurately reproduce most features of data, but cannot be trusted to capture all of the comp...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
American Physical Society
2018
|
| Online Access: | http://hdl.handle.net/1721.1/117099 |
| _version_ | 1826200054281535488 |
|---|---|
| author | Nachman, Benjamin Schwartz, Matthew D. Komiske, Patrick T. Metodiev, Eric Mario |
| author2 | Massachusetts Institute of Technology. Center for Theoretical Physics |
| author_facet | Massachusetts Institute of Technology. Center for Theoretical Physics Nachman, Benjamin Schwartz, Matthew D. Komiske, Patrick T. Metodiev, Eric Mario |
| author_sort | Nachman, Benjamin |
| collection | MIT |
| description | A persistent challenge in practical classification tasks is that labeled training sets are not always available. In particle physics, this challenge is surmounted by the use of simulations. These simulations accurately reproduce most features of data, but cannot be trusted to capture all of the complex correlations exploitable by modern machine learning methods. Recent work in weakly supervised learning has shown that simple, low-dimensional classifiers can be trained using only the impure mixtures present in data. Here, we demonstrate that complex, high-dimensional classifiers can also be trained on impure mixtures using weak supervision techniques, with performance comparable to what could be achieved with pure samples. Using weak supervision will therefore allow us to avoid relying exclusively on simulations for high-dimensional classification. This work opens the door to a new regime whereby complex models are trained directly on data, providing direct access to probe the underlying physics. |
| first_indexed | 2024-09-23T11:30:16Z |
| format | Article |
| id | mit-1721.1/117099 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T11:30:16Z |
| publishDate | 2018 |
| publisher | American Physical Society |
| record_format | dspace |
| spelling | mit-1721.1/1170992022-10-01T04:05:29Z Learning to classify from impure samples with high-dimensional data Nachman, Benjamin Schwartz, Matthew D. Komiske, Patrick T. Metodiev, Eric Mario Massachusetts Institute of Technology. Center for Theoretical Physics Massachusetts Institute of Technology. Department of Physics Komiske, Patrick T. Metodiev, Eric Mario A persistent challenge in practical classification tasks is that labeled training sets are not always available. In particle physics, this challenge is surmounted by the use of simulations. These simulations accurately reproduce most features of data, but cannot be trusted to capture all of the complex correlations exploitable by modern machine learning methods. Recent work in weakly supervised learning has shown that simple, low-dimensional classifiers can be trained using only the impure mixtures present in data. Here, we demonstrate that complex, high-dimensional classifiers can also be trained on impure mixtures using weak supervision techniques, with performance comparable to what could be achieved with pure samples. Using weak supervision will therefore allow us to avoid relying exclusively on simulations for high-dimensional classification. This work opens the door to a new regime whereby complex models are trained directly on data, providing direct access to probe the underlying physics. United States. Department of Energy. Office of Science (Contract No. DE-SC0013607) United States. Department of Energy. Office of Science (Contract No. DE-AC02-05CH11231) United States. Department of Energy. Office of Nuclear Physics (Contract No. DE-SC0011090) United States. Department of Energy. Office of High Energy Physics (Contract No. DE-SC0012567) 2018-07-24T22:38:10Z 2018-07-24T22:38:10Z 2018-07 2018-04 2018-07-16T18:00:17Z Article http://purl.org/eprint/type/JournalArticle 2470-0010 2470-0029 http://hdl.handle.net/1721.1/117099 Komiske, Patrick T., Eric M. Metodiev, Benjamin Nachman, and Matthew D. Schwartz. “Learning to Classify from Impure Samples with High-Dimensional Data.” Physical Review D 98, no. 1 (July 16, 2018). en http://dx.doi.org/10.1103/PhysRevD.98.011502 Physical Review D Creative Commons Attribution http://creativecommons.org/licenses/by/3.0 application/pdf American Physical Society American Physical Society |
| spellingShingle | Nachman, Benjamin Schwartz, Matthew D. Komiske, Patrick T. Metodiev, Eric Mario Learning to classify from impure samples with high-dimensional data |
| title | Learning to classify from impure samples with high-dimensional data |
| title_full | Learning to classify from impure samples with high-dimensional data |
| title_fullStr | Learning to classify from impure samples with high-dimensional data |
| title_full_unstemmed | Learning to classify from impure samples with high-dimensional data |
| title_short | Learning to classify from impure samples with high-dimensional data |
| title_sort | learning to classify from impure samples with high dimensional data |
| url | http://hdl.handle.net/1721.1/117099 |
| work_keys_str_mv | AT nachmanbenjamin learningtoclassifyfromimpuresampleswithhighdimensionaldata AT schwartzmatthewd learningtoclassifyfromimpuresampleswithhighdimensionaldata AT komiskepatrickt learningtoclassifyfromimpuresampleswithhighdimensionaldata AT metodievericmario learningtoclassifyfromimpuresampleswithhighdimensionaldata |