Perturbation-invariant Speech Representation Learning by Online Clustering
Despite success across various tasks, self-supervised speech models face significant challenges in enhancing content-related performance with unlabeled data, requiring substantial computational resources. Meanwhile, learning from clustered discrete units has been shown to facilitate accurate phoneti...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2024
|
| Online Access: | https://hdl.handle.net/1721.1/153784 https://orcid.org/0000-0002-1690-2610 |
| _version_ | 1811068413063200768 |
|---|---|
| author | Chang, Heng-Jui |
| author2 | Glass, James R. |
| author_facet | Glass, James R. Chang, Heng-Jui |
| author_sort | Chang, Heng-Jui |
| collection | MIT |
| description | Despite success across various tasks, self-supervised speech models face significant challenges in enhancing content-related performance with unlabeled data, requiring substantial computational resources. Meanwhile, learning from clustered discrete units has been shown to facilitate accurate phonetic representations. Thus, this thesis investigates speaker and noise-invariant speech representations. First, Speaker-invariant Clustering (Spin) is proposed to extract content representations through online clustering and speaker-invariant cross-view prediction. Second, Robust Spin (R-Spin) is devised to extend Spin to handle more distorted speech signals by leveraging acoustic pieces. Furthermore, this thesis includes a diverse set of evaluation and visualization techniques to quantitatively and qualitatively analyze the perturbation invariability of the proposed methods. This thesis offers approaches to producing perturbation-invariant speech representations and deeply investigates the characteristics of the learned representations, providing insights into these models and cultivating future extension possibilities. |
| first_indexed | 2024-09-23T07:55:37Z |
| format | Thesis |
| id | mit-1721.1/153784 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T07:55:37Z |
| publishDate | 2024 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1537842024-03-16T04:08:33Z Perturbation-invariant Speech Representation Learning by Online Clustering Chang, Heng-Jui Glass, James R. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Despite success across various tasks, self-supervised speech models face significant challenges in enhancing content-related performance with unlabeled data, requiring substantial computational resources. Meanwhile, learning from clustered discrete units has been shown to facilitate accurate phonetic representations. Thus, this thesis investigates speaker and noise-invariant speech representations. First, Speaker-invariant Clustering (Spin) is proposed to extract content representations through online clustering and speaker-invariant cross-view prediction. Second, Robust Spin (R-Spin) is devised to extend Spin to handle more distorted speech signals by leveraging acoustic pieces. Furthermore, this thesis includes a diverse set of evaluation and visualization techniques to quantitatively and qualitatively analyze the perturbation invariability of the proposed methods. This thesis offers approaches to producing perturbation-invariant speech representations and deeply investigates the characteristics of the learned representations, providing insights into these models and cultivating future extension possibilities. S.M. 2024-03-15T19:23:50Z 2024-03-15T19:23:50Z 2024-02 2024-02-21T17:10:05.097Z Thesis https://hdl.handle.net/1721.1/153784 https://orcid.org/0000-0002-1690-2610 In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Chang, Heng-Jui Perturbation-invariant Speech Representation Learning by Online Clustering |
| title | Perturbation-invariant Speech Representation Learning by Online Clustering |
| title_full | Perturbation-invariant Speech Representation Learning by Online Clustering |
| title_fullStr | Perturbation-invariant Speech Representation Learning by Online Clustering |
| title_full_unstemmed | Perturbation-invariant Speech Representation Learning by Online Clustering |
| title_short | Perturbation-invariant Speech Representation Learning by Online Clustering |
| title_sort | perturbation invariant speech representation learning by online clustering |
| url | https://hdl.handle.net/1721.1/153784 https://orcid.org/0000-0002-1690-2610 |
| work_keys_str_mv | AT changhengjui perturbationinvariantspeechrepresentationlearningbyonlineclustering |