Uncertainty Quantification Using Neural Networks for Molecular Property Prediction
Uncertainty quantification (UQ) is an important component of molecular property prediction, particularly for drug discovery applications where model predictions direct experimental design and where unanticipated imprecision wastes valuable time and resources. The need for UQ is especially acute for...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
American Chemical Society (ACS)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/135244 |
| _version_ | 1826204057127092224 |
|---|---|
| author | Hirschfeld, Lior Swanson, Kyle Yang, Kevin Barzilay, Regina Coley, Connor W |
| author2 | Massachusetts Institute of Technology. Department of Chemical Engineering |
| author_facet | Massachusetts Institute of Technology. Department of Chemical Engineering Hirschfeld, Lior Swanson, Kyle Yang, Kevin Barzilay, Regina Coley, Connor W |
| author_sort | Hirschfeld, Lior |
| collection | MIT |
| description | Uncertainty quantification (UQ) is an important component of molecular property prediction, particularly for drug discovery applications where model predictions direct experimental design and where unanticipated imprecision wastes valuable time and resources. The need for UQ is especially acute for neural models, which are becoming increasingly standard yet are challenging to interpret. While several approaches to UQ have been proposed in the literature, there is no clear consensus on the comparative performance of these models. In this paper, we study this question in the context of regression tasks. We systematically evaluate several methods on five regression data sets using multiple complementary performance metrics. Our experiments show that none of the methods we tested is unequivocally superior to all others, and none produces a particularly reliable ranking of errors across multiple data sets. While we believe that these results show that existing UQ methods are not sufficient for all common use cases and further research is needed, we conclude with a practical recommendation as to which existing techniques seem to perform well relative to others. |
| first_indexed | 2024-09-23T12:48:14Z |
| format | Article |
| id | mit-1721.1/135244 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T12:48:14Z |
| publishDate | 2021 |
| publisher | American Chemical Society (ACS) |
| record_format | dspace |
| spelling | mit-1721.1/1352442023-02-23T16:31:33Z Uncertainty Quantification Using Neural Networks for Molecular Property Prediction Hirschfeld, Lior Swanson, Kyle Yang, Kevin Barzilay, Regina Coley, Connor W Massachusetts Institute of Technology. Department of Chemical Engineering Uncertainty quantification (UQ) is an important component of molecular property prediction, particularly for drug discovery applications where model predictions direct experimental design and where unanticipated imprecision wastes valuable time and resources. The need for UQ is especially acute for neural models, which are becoming increasingly standard yet are challenging to interpret. While several approaches to UQ have been proposed in the literature, there is no clear consensus on the comparative performance of these models. In this paper, we study this question in the context of regression tasks. We systematically evaluate several methods on five regression data sets using multiple complementary performance metrics. Our experiments show that none of the methods we tested is unequivocally superior to all others, and none produces a particularly reliable ranking of errors across multiple data sets. While we believe that these results show that existing UQ methods are not sufficient for all common use cases and further research is needed, we conclude with a practical recommendation as to which existing techniques seem to perform well relative to others. 2021-10-27T20:22:37Z 2021-10-27T20:22:37Z 2020 2020-12-01T18:07:21Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/135244 en 10.1021/acs.jcim.0c00502 Journal of Chemical Information and Modeling Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf American Chemical Society (ACS) arXiv |
| spellingShingle | Hirschfeld, Lior Swanson, Kyle Yang, Kevin Barzilay, Regina Coley, Connor W Uncertainty Quantification Using Neural Networks for Molecular Property Prediction |
| title | Uncertainty Quantification Using Neural Networks for Molecular Property Prediction |
| title_full | Uncertainty Quantification Using Neural Networks for Molecular Property Prediction |
| title_fullStr | Uncertainty Quantification Using Neural Networks for Molecular Property Prediction |
| title_full_unstemmed | Uncertainty Quantification Using Neural Networks for Molecular Property Prediction |
| title_short | Uncertainty Quantification Using Neural Networks for Molecular Property Prediction |
| title_sort | uncertainty quantification using neural networks for molecular property prediction |
| url | https://hdl.handle.net/1721.1/135244 |
| work_keys_str_mv | AT hirschfeldlior uncertaintyquantificationusingneuralnetworksformolecularpropertyprediction AT swansonkyle uncertaintyquantificationusingneuralnetworksformolecularpropertyprediction AT yangkevin uncertaintyquantificationusingneuralnetworksformolecularpropertyprediction AT barzilayregina uncertaintyquantificationusingneuralnetworksformolecularpropertyprediction AT coleyconnorw uncertaintyquantificationusingneuralnetworksformolecularpropertyprediction |