An Efficient Approach to Informative Feature Extraction from Multimodal Data
Copyright © 2019, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. One primary focus in multimodal feature extraction is to find the representations of individual modalities that are maximally correlated. As a well-known measure of dependence, the Hirsc...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Association for the Advancement of Artificial Intelligence (AAAI)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/137795 |
| _version_ | 1811072742184714240 |
|---|---|
| author | Wang, Lichen Wu, Jiaxiang Huang, Shao-Lun Zheng, Lizhong Xu, Xiangxiang Zhang, Lin Huang, Junzhou |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Wang, Lichen Wu, Jiaxiang Huang, Shao-Lun Zheng, Lizhong Xu, Xiangxiang Zhang, Lin Huang, Junzhou |
| author_sort | Wang, Lichen |
| collection | MIT |
| description | Copyright © 2019, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. One primary focus in multimodal feature extraction is to find the representations of individual modalities that are maximally correlated. As a well-known measure of dependence, the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation becomes an appealing objective because of its operational meaning and desirable properties. However, the strict whitening constraints formalized in the HGR maximal correlation limit its application. To address this problem, this paper proposes Soft-HGR, a novel framework to extract informative features from multiple data modalities. Specifically, our framework prevents the “hard” whitening constraints, while simultaneously preserving the same feature geometry as in the HGR maximal correlation. The objective of Soft-HGR is straightforward, only involving two inner products, which guarantees the efficiency and stability in optimization. We further generalize the framework to handle more than two modalities and missing modalities. When labels are partially available, we enhance the discriminative power of the feature representations by making a semi-supervised adaptation. Empirical evaluation implies that our approach learns more informative feature mappings and is more efficient to optimize. |
| first_indexed | 2024-09-23T09:11:33Z |
| format | Article |
| id | mit-1721.1/137795 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T09:11:33Z |
| publishDate | 2021 |
| publisher | Association for the Advancement of Artificial Intelligence (AAAI) |
| record_format | dspace |
| spelling | mit-1721.1/1377952023-02-10T18:30:44Z An Efficient Approach to Informative Feature Extraction from Multimodal Data Wang, Lichen Wu, Jiaxiang Huang, Shao-Lun Zheng, Lizhong Xu, Xiangxiang Zhang, Lin Huang, Junzhou Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Copyright © 2019, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. One primary focus in multimodal feature extraction is to find the representations of individual modalities that are maximally correlated. As a well-known measure of dependence, the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation becomes an appealing objective because of its operational meaning and desirable properties. However, the strict whitening constraints formalized in the HGR maximal correlation limit its application. To address this problem, this paper proposes Soft-HGR, a novel framework to extract informative features from multiple data modalities. Specifically, our framework prevents the “hard” whitening constraints, while simultaneously preserving the same feature geometry as in the HGR maximal correlation. The objective of Soft-HGR is straightforward, only involving two inner products, which guarantees the efficiency and stability in optimization. We further generalize the framework to handle more than two modalities and missing modalities. When labels are partially available, we enhance the discriminative power of the feature representations by making a semi-supervised adaptation. Empirical evaluation implies that our approach learns more informative feature mappings and is more efficient to optimize. 2021-11-08T19:30:22Z 2021-11-08T19:30:22Z 2019 2021-01-25T19:11:05Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/137795 Wang, Lichen, Wu, Jiaxiang, Huang, Shao-Lun, Zheng, Lizhong, Xu, Xiangxiang et al. 2019. "An Efficient Approach to Informative Feature Extraction from Multimodal Data." Proceedings of the AAAI Conference on Artificial Intelligence, 33. en 10.1609/AAAI.V33I01.33015281 Proceedings of the AAAI Conference on Artificial Intelligence Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Association for the Advancement of Artificial Intelligence (AAAI) arXiv |
| spellingShingle | Wang, Lichen Wu, Jiaxiang Huang, Shao-Lun Zheng, Lizhong Xu, Xiangxiang Zhang, Lin Huang, Junzhou An Efficient Approach to Informative Feature Extraction from Multimodal Data |
| title | An Efficient Approach to Informative Feature Extraction from Multimodal Data |
| title_full | An Efficient Approach to Informative Feature Extraction from Multimodal Data |
| title_fullStr | An Efficient Approach to Informative Feature Extraction from Multimodal Data |
| title_full_unstemmed | An Efficient Approach to Informative Feature Extraction from Multimodal Data |
| title_short | An Efficient Approach to Informative Feature Extraction from Multimodal Data |
| title_sort | efficient approach to informative feature extraction from multimodal data |
| url | https://hdl.handle.net/1721.1/137795 |
| work_keys_str_mv | AT wanglichen anefficientapproachtoinformativefeatureextractionfrommultimodaldata AT wujiaxiang anefficientapproachtoinformativefeatureextractionfrommultimodaldata AT huangshaolun anefficientapproachtoinformativefeatureextractionfrommultimodaldata AT zhenglizhong anefficientapproachtoinformativefeatureextractionfrommultimodaldata AT xuxiangxiang anefficientapproachtoinformativefeatureextractionfrommultimodaldata AT zhanglin anefficientapproachtoinformativefeatureextractionfrommultimodaldata AT huangjunzhou anefficientapproachtoinformativefeatureextractionfrommultimodaldata AT wanglichen efficientapproachtoinformativefeatureextractionfrommultimodaldata AT wujiaxiang efficientapproachtoinformativefeatureextractionfrommultimodaldata AT huangshaolun efficientapproachtoinformativefeatureextractionfrommultimodaldata AT zhenglizhong efficientapproachtoinformativefeatureextractionfrommultimodaldata AT xuxiangxiang efficientapproachtoinformativefeatureextractionfrommultimodaldata AT zhanglin efficientapproachtoinformativefeatureextractionfrommultimodaldata AT huangjunzhou efficientapproachtoinformativefeatureextractionfrommultimodaldata |