Kernel-nased just-in-time learning for passing expectation propagation messages
We propose an efficient nonparametric strategy for learning a message operator in expectation propagation (EP), which takes as input the set of incoming messages to a factor node, and produces an outgoing message as output. This learned operator replaces the multivariate integral required in classic...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Conference item |
| Published: |
Association for Uncertainty in Artificial Intelligence
2015
|
| _version_ | 1826275540264288256 |
|---|---|
| author | Jitkrittum, W Gretton, A Heess, N Eslami, S Lakshminarayanan, B Sejdinovic, D Szabó, Z |
| author_facet | Jitkrittum, W Gretton, A Heess, N Eslami, S Lakshminarayanan, B Sejdinovic, D Szabó, Z |
| author_sort | Jitkrittum, W |
| collection | OXFORD |
| description | We propose an efficient nonparametric strategy for learning a message operator in expectation propagation (EP), which takes as input the set of incoming messages to a factor node, and produces an outgoing message as output. This learned operator replaces the multivariate integral required in classical EP, which may not have an analytic expression. We use kernel-based regression, which is trained on a set of probability distributions representing the incoming messages, and the associated outgoing messages. The kernel approach has two main advantages: first, it is fast, as it is implemented using a novel two-layer random feature representation of the input message distributions; second, it has principled uncertainty estimates, and can be cheaply updated online, meaning it can request and incorporate new training data when it encounters inputs on which it is uncertain. In experiments, our approach is able to solve learning problems where a single message operator is required for multiple, substantially different data sets (logistic regression for a variety of classification problems), where it is essential to accurately assess uncertainty and to efficiently and robustly update the message operator. |
| first_indexed | 2024-03-06T23:00:16Z |
| format | Conference item |
| id | oxford-uuid:61de56f1-24b4-43c7-81cc-6a1ec756b3bf |
| institution | University of Oxford |
| last_indexed | 2024-03-06T23:00:16Z |
| publishDate | 2015 |
| publisher | Association for Uncertainty in Artificial Intelligence |
| record_format | dspace |
| spelling | oxford-uuid:61de56f1-24b4-43c7-81cc-6a1ec756b3bf2022-03-26T18:02:37ZKernel-nased just-in-time learning for passing expectation propagation messagesConference itemhttp://purl.org/coar/resource_type/c_5794uuid:61de56f1-24b4-43c7-81cc-6a1ec756b3bfSymplectic Elements at OxfordAssociation for Uncertainty in Artificial Intelligence2015Jitkrittum, WGretton, AHeess, NEslami, SLakshminarayanan, BSejdinovic, DSzabó, ZWe propose an efficient nonparametric strategy for learning a message operator in expectation propagation (EP), which takes as input the set of incoming messages to a factor node, and produces an outgoing message as output. This learned operator replaces the multivariate integral required in classical EP, which may not have an analytic expression. We use kernel-based regression, which is trained on a set of probability distributions representing the incoming messages, and the associated outgoing messages. The kernel approach has two main advantages: first, it is fast, as it is implemented using a novel two-layer random feature representation of the input message distributions; second, it has principled uncertainty estimates, and can be cheaply updated online, meaning it can request and incorporate new training data when it encounters inputs on which it is uncertain. In experiments, our approach is able to solve learning problems where a single message operator is required for multiple, substantially different data sets (logistic regression for a variety of classification problems), where it is essential to accurately assess uncertainty and to efficiently and robustly update the message operator. |
| spellingShingle | Jitkrittum, W Gretton, A Heess, N Eslami, S Lakshminarayanan, B Sejdinovic, D Szabó, Z Kernel-nased just-in-time learning for passing expectation propagation messages |
| title | Kernel-nased just-in-time learning for passing expectation propagation messages |
| title_full | Kernel-nased just-in-time learning for passing expectation propagation messages |
| title_fullStr | Kernel-nased just-in-time learning for passing expectation propagation messages |
| title_full_unstemmed | Kernel-nased just-in-time learning for passing expectation propagation messages |
| title_short | Kernel-nased just-in-time learning for passing expectation propagation messages |
| title_sort | kernel nased just in time learning for passing expectation propagation messages |
| work_keys_str_mv | AT jitkrittumw kernelnasedjustintimelearningforpassingexpectationpropagationmessages AT grettona kernelnasedjustintimelearningforpassingexpectationpropagationmessages AT heessn kernelnasedjustintimelearningforpassingexpectationpropagationmessages AT eslamis kernelnasedjustintimelearningforpassingexpectationpropagationmessages AT lakshminarayananb kernelnasedjustintimelearningforpassingexpectationpropagationmessages AT sejdinovicd kernelnasedjustintimelearningforpassingexpectationpropagationmessages AT szaboz kernelnasedjustintimelearningforpassingexpectationpropagationmessages |