Inference and Learning in a Latent Variable Model for Beta Distributed Interval Data
Latent Variable Models (LVMs) are well established tools to accomplish a range of different data processing tasks. Applications exploit the ability of LVMs to identify latent data structure in order to improve data (e.g., through denoising) or to estimate the relation between latent causes and measu...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-04-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/23/5/552 |
| _version_ | 1797535739411431424 |
|---|---|
| author | Hamid Mousavi Mareike Buhl Enrico Guiraud Jakob Drefs Jörg Lücke |
| author_facet | Hamid Mousavi Mareike Buhl Enrico Guiraud Jakob Drefs Jörg Lücke |
| author_sort | Hamid Mousavi |
| collection | DOAJ |
| description | Latent Variable Models (LVMs) are well established tools to accomplish a range of different data processing tasks. Applications exploit the ability of LVMs to identify latent data structure in order to improve data (e.g., through denoising) or to estimate the relation between latent causes and measurements in medical data. In the latter case, LVMs in the form of noisy-OR Bayes nets represent the standard approach to relate binary latents (which represent diseases) to binary observables (which represent symptoms). Bayes nets with binary representation for symptoms may be perceived as a coarse approximation, however. In practice, real disease symptoms can range from absent over mild and intermediate to very severe. Therefore, using diseases/symptoms relations as motivation, we here ask how standard noisy-OR Bayes nets can be generalized to incorporate continuous observables, e.g., variables that model symptom severity in an interval from healthy to pathological. This transition from binary to interval data poses a number of challenges including a transition from a Bernoulli to a Beta distribution to model symptom statistics. While noisy-OR-like approaches are constrained to model how causes determine the observables’ mean values, the use of Beta distributions additionally provides (and also requires) that the causes determine the observables’ variances. To meet the challenges emerging when generalizing from Bernoulli to Beta distributed observables, we investigate a novel LVM that uses a maximum non-linearity to model how the latents determine means and variances of the observables. Given the model and the goal of likelihood maximization, we then leverage recent theoretical results to derive an Expectation Maximization (EM) algorithm for the suggested LVM. We further show how variational EM can be used to efficiently scale the approach to large networks. Experimental results finally illustrate the efficacy of the proposed model using both synthetic and real data sets. Importantly, we show that the model produces reliable results in estimating causes using proofs of concepts and first tests based on real medical data and on images. |
| first_indexed | 2024-03-10T11:49:41Z |
| format | Article |
| id | doaj.art-f0b0be813a154dbea9c0b08ba8321971 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T11:49:41Z |
| publishDate | 2021-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-f0b0be813a154dbea9c0b08ba83219712023-11-21T17:50:22ZengMDPI AGEntropy1099-43002021-04-0123555210.3390/e23050552Inference and Learning in a Latent Variable Model for Beta Distributed Interval DataHamid Mousavi0Mareike Buhl1Enrico Guiraud2Jakob Drefs3Jörg Lücke4Machine Learning Lab, Department of Medical Physics and Acoustics and Cluster of Excellence Hearing4all, University of Oldenburg, 26129 Oldenburg, GermanyMedical Physics Group, Department of Medical Physics and Acoustics and Cluster of Excellence Hearing4all, University of Oldenburg, 26129 Oldenburg, GermanyMachine Learning Lab, Department of Medical Physics and Acoustics and Cluster of Excellence Hearing4all, University of Oldenburg, 26129 Oldenburg, GermanyMachine Learning Lab, Department of Medical Physics and Acoustics and Cluster of Excellence Hearing4all, University of Oldenburg, 26129 Oldenburg, GermanyMachine Learning Lab, Department of Medical Physics and Acoustics and Cluster of Excellence Hearing4all, University of Oldenburg, 26129 Oldenburg, GermanyLatent Variable Models (LVMs) are well established tools to accomplish a range of different data processing tasks. Applications exploit the ability of LVMs to identify latent data structure in order to improve data (e.g., through denoising) or to estimate the relation between latent causes and measurements in medical data. In the latter case, LVMs in the form of noisy-OR Bayes nets represent the standard approach to relate binary latents (which represent diseases) to binary observables (which represent symptoms). Bayes nets with binary representation for symptoms may be perceived as a coarse approximation, however. In practice, real disease symptoms can range from absent over mild and intermediate to very severe. Therefore, using diseases/symptoms relations as motivation, we here ask how standard noisy-OR Bayes nets can be generalized to incorporate continuous observables, e.g., variables that model symptom severity in an interval from healthy to pathological. This transition from binary to interval data poses a number of challenges including a transition from a Bernoulli to a Beta distribution to model symptom statistics. While noisy-OR-like approaches are constrained to model how causes determine the observables’ mean values, the use of Beta distributions additionally provides (and also requires) that the causes determine the observables’ variances. To meet the challenges emerging when generalizing from Bernoulli to Beta distributed observables, we investigate a novel LVM that uses a maximum non-linearity to model how the latents determine means and variances of the observables. Given the model and the goal of likelihood maximization, we then leverage recent theoretical results to derive an Expectation Maximization (EM) algorithm for the suggested LVM. We further show how variational EM can be used to efficiently scale the approach to large networks. Experimental results finally illustrate the efficacy of the proposed model using both synthetic and real data sets. Importantly, we show that the model produces reliable results in estimating causes using proofs of concepts and first tests based on real medical data and on images.https://www.mdpi.com/1099-4300/23/5/552latent variable modelsBayes netsBeta distributionnoisy-ORexpectation maximizationvariational inference |
| spellingShingle | Hamid Mousavi Mareike Buhl Enrico Guiraud Jakob Drefs Jörg Lücke Inference and Learning in a Latent Variable Model for Beta Distributed Interval Data Entropy latent variable models Bayes nets Beta distribution noisy-OR expectation maximization variational inference |
| title | Inference and Learning in a Latent Variable Model for Beta Distributed Interval Data |
| title_full | Inference and Learning in a Latent Variable Model for Beta Distributed Interval Data |
| title_fullStr | Inference and Learning in a Latent Variable Model for Beta Distributed Interval Data |
| title_full_unstemmed | Inference and Learning in a Latent Variable Model for Beta Distributed Interval Data |
| title_short | Inference and Learning in a Latent Variable Model for Beta Distributed Interval Data |
| title_sort | inference and learning in a latent variable model for beta distributed interval data |
| topic | latent variable models Bayes nets Beta distribution noisy-OR expectation maximization variational inference |
| url | https://www.mdpi.com/1099-4300/23/5/552 |
| work_keys_str_mv | AT hamidmousavi inferenceandlearninginalatentvariablemodelforbetadistributedintervaldata AT mareikebuhl inferenceandlearninginalatentvariablemodelforbetadistributedintervaldata AT enricoguiraud inferenceandlearninginalatentvariablemodelforbetadistributedintervaldata AT jakobdrefs inferenceandlearninginalatentvariablemodelforbetadistributedintervaldata AT jorglucke inferenceandlearninginalatentvariablemodelforbetadistributedintervaldata |