Gen: A General-Purpose Probabilistic Programming System with Programmable Inference
Probabilistic modeling and inference are central to many fields. A key challenge for wider adoption of probabilistic programming languages is designing systems that are both flexible and performant. This paper introduces Gen, a new probabilistic programming system with novel language con- structs fo...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | en_US |
| Published: |
2018
|
| Online Access: | http://hdl.handle.net/1721.1/119255 |
| _version_ | 1826200119099260928 |
|---|---|
| author | Cusumano-Towner, Marco F. Saad, Feras A. Lew, Alexander Mansinghka, Vikash K. |
| author_facet | Cusumano-Towner, Marco F. Saad, Feras A. Lew, Alexander Mansinghka, Vikash K. |
| author_sort | Cusumano-Towner, Marco F. |
| collection | MIT |
| description | Probabilistic modeling and inference are central to many fields. A key challenge for wider adoption of probabilistic programming languages is designing systems that are both flexible and performant. This paper introduces Gen, a new probabilistic programming system with novel language con- structs for modeling and for end-user customization and optimization of inference. Gen makes it practical to write probabilistic programs that solve problems from multiple fields. Gen programs can combine generative models written in Julia, neural networks written in TensorFlow, and custom inference algorithms based on an extensible library of Monte Carlo and numerical optimization techniques. This paper also presents techniques that enable Gen’s combination of flexibility and performance: (i) the generative function inter- face, an abstraction for encapsulating probabilistic and/or differentiable computations; (ii) domain-specific languages with custom compilers that strike different flexibility/per- formance tradeoffs; (iii) combinators that encode common patterns of conditional independence and repeated compu- tation, enabling speedups from caching; and (iv) a standard inference library that supports custom proposal distributions also written as programs in Gen. This paper shows that Gen outperforms state-of-the-art probabilistic programming systems, sometimes by multiple orders of magnitude, on problems such as nonlinear state-space modeling, structure learning for real-world time series data, robust regression, and 3D body pose estimation from depth images. |
| first_indexed | 2024-09-23T11:31:29Z |
| format | Article |
| id | mit-1721.1/119255 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T11:31:29Z |
| publishDate | 2018 |
| record_format | dspace |
| spelling | mit-1721.1/1192552019-04-12T23:02:33Z Gen: A General-Purpose Probabilistic Programming System with Programmable Inference Cusumano-Towner, Marco F. Saad, Feras A. Lew, Alexander Mansinghka, Vikash K. Probabilistic modeling and inference are central to many fields. A key challenge for wider adoption of probabilistic programming languages is designing systems that are both flexible and performant. This paper introduces Gen, a new probabilistic programming system with novel language con- structs for modeling and for end-user customization and optimization of inference. Gen makes it practical to write probabilistic programs that solve problems from multiple fields. Gen programs can combine generative models written in Julia, neural networks written in TensorFlow, and custom inference algorithms based on an extensible library of Monte Carlo and numerical optimization techniques. This paper also presents techniques that enable Gen’s combination of flexibility and performance: (i) the generative function inter- face, an abstraction for encapsulating probabilistic and/or differentiable computations; (ii) domain-specific languages with custom compilers that strike different flexibility/per- formance tradeoffs; (iii) combinators that encode common patterns of conditional independence and repeated compu- tation, enabling speedups from caching; and (iv) a standard inference library that supports custom proposal distributions also written as programs in Gen. This paper shows that Gen outperforms state-of-the-art probabilistic programming systems, sometimes by multiple orders of magnitude, on problems such as nonlinear state-space modeling, structure learning for real-world time series data, robust regression, and 3D body pose estimation from depth images. This research was supported in part by the US Department of the Air Force contract FA8750-17-C-0239, grants from the MIT Media Lab/Harvard Berkman Center Ethics & Gover- nance of AI Fund and the MIT CSAIL Systems that Learn Consortium, a gift from the Aphorism Foundation, and the US Department of Defense through the the National De- fense Science & Engineering Graduate Fellowship (NDSEG) Program. 2018-11-26T17:33:26Z 2018-11-26T17:33:26Z 2018-11-26 Article http://hdl.handle.net/1721.1/119255 en_US ;MIT-CSAIL-TR-2018-020 application/pdf |
| spellingShingle | Cusumano-Towner, Marco F. Saad, Feras A. Lew, Alexander Mansinghka, Vikash K. Gen: A General-Purpose Probabilistic Programming System with Programmable Inference |
| title | Gen: A General-Purpose Probabilistic Programming System with Programmable Inference |
| title_full | Gen: A General-Purpose Probabilistic Programming System with Programmable Inference |
| title_fullStr | Gen: A General-Purpose Probabilistic Programming System with Programmable Inference |
| title_full_unstemmed | Gen: A General-Purpose Probabilistic Programming System with Programmable Inference |
| title_short | Gen: A General-Purpose Probabilistic Programming System with Programmable Inference |
| title_sort | gen a general purpose probabilistic programming system with programmable inference |
| url | http://hdl.handle.net/1721.1/119255 |
| work_keys_str_mv | AT cusumanotownermarcof genageneralpurposeprobabilisticprogrammingsystemwithprogrammableinference AT saadferasa genageneralpurposeprobabilisticprogrammingsystemwithprogrammableinference AT lewalexander genageneralpurposeprobabilisticprogrammingsystemwithprogrammableinference AT mansinghkavikashk genageneralpurposeprobabilisticprogrammingsystemwithprogrammableinference |