Advances in Hierarchical Probabilistic Multimodal Data Fusion
Multimodal data fusion is the process of integrating disparate data sources into a shared representation suitable for complex reasoning. As a result, one can make more precise inferences about the underlying phenomenon than is possible with each data source used in isolation. In the thesis we adopt...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2022
|
| Online Access: | https://hdl.handle.net/1721.1/144802 |
| _version_ | 1826216390613270528 |
|---|---|
| author | Dean, Christopher L. |
| author2 | Fisher III, John W. |
| author_facet | Fisher III, John W. Dean, Christopher L. |
| author_sort | Dean, Christopher L. |
| collection | MIT |
| description | Multimodal data fusion is the process of integrating disparate data sources into a shared representation suitable for complex reasoning. As a result, one can make more precise inferences about the underlying phenomenon than is possible with each data source used in isolation. In the thesis we adopt a Bayesian view of multimodal data fusion, which formulates reasoning as posterior inference over latent variables. Within the Bayesian setting we present a novel method for data integration that we call lightweight data fusion (LDF). LDF addresses the case where the forward model for a subset of the data sources is unknown or poorly characterized. LDF leverages the remaining data sources to learn an inverse model suitable for posterior inference that combines both types of data. Additionally, we develop a multimodal extension to hierarchical Dirichlet processes (mmHDPs) where, in contrast to the setting for LDF, we lack observation-level correspondences across modalities and the data arise from an implicit latent variable model. Finally, we develop a novel representation for Dirichlet process and HDP mixture models that enables parallelization during inference and extends to more complex models including mmHDPs. |
| first_indexed | 2024-09-23T16:46:53Z |
| format | Thesis |
| id | mit-1721.1/144802 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T16:46:53Z |
| publishDate | 2022 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1448022022-08-30T03:02:39Z Advances in Hierarchical Probabilistic Multimodal Data Fusion Dean, Christopher L. Fisher III, John W. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Multimodal data fusion is the process of integrating disparate data sources into a shared representation suitable for complex reasoning. As a result, one can make more precise inferences about the underlying phenomenon than is possible with each data source used in isolation. In the thesis we adopt a Bayesian view of multimodal data fusion, which formulates reasoning as posterior inference over latent variables. Within the Bayesian setting we present a novel method for data integration that we call lightweight data fusion (LDF). LDF addresses the case where the forward model for a subset of the data sources is unknown or poorly characterized. LDF leverages the remaining data sources to learn an inverse model suitable for posterior inference that combines both types of data. Additionally, we develop a multimodal extension to hierarchical Dirichlet processes (mmHDPs) where, in contrast to the setting for LDF, we lack observation-level correspondences across modalities and the data arise from an implicit latent variable model. Finally, we develop a novel representation for Dirichlet process and HDP mixture models that enables parallelization during inference and extends to more complex models including mmHDPs. Ph.D. 2022-08-29T16:12:41Z 2022-08-29T16:12:41Z 2022-05 2022-06-21T19:15:38.609Z Thesis https://hdl.handle.net/1721.1/144802 In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Dean, Christopher L. Advances in Hierarchical Probabilistic Multimodal Data Fusion |
| title | Advances in Hierarchical Probabilistic Multimodal Data Fusion |
| title_full | Advances in Hierarchical Probabilistic Multimodal Data Fusion |
| title_fullStr | Advances in Hierarchical Probabilistic Multimodal Data Fusion |
| title_full_unstemmed | Advances in Hierarchical Probabilistic Multimodal Data Fusion |
| title_short | Advances in Hierarchical Probabilistic Multimodal Data Fusion |
| title_sort | advances in hierarchical probabilistic multimodal data fusion |
| url | https://hdl.handle.net/1721.1/144802 |
| work_keys_str_mv | AT deanchristopherl advancesinhierarchicalprobabilisticmultimodaldatafusion |