Localizing category-related information in speech with multi-scale analyses.
Measurements of the physical outputs of speech-vocal tract geometry and acoustic energy-are high-dimensional, but linguistic theories posit a low-dimensional set of categories such as phonemes and phrase types. How can it be determined when and where in high-dimensional articulatory and acoustic sig...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2021-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0258178 |
| _version_ | 1819195159308402688 |
|---|---|
| author | Sam Tilsen Seung-Eun Kim Claire Wang |
| author_facet | Sam Tilsen Seung-Eun Kim Claire Wang |
| author_sort | Sam Tilsen |
| collection | DOAJ |
| description | Measurements of the physical outputs of speech-vocal tract geometry and acoustic energy-are high-dimensional, but linguistic theories posit a low-dimensional set of categories such as phonemes and phrase types. How can it be determined when and where in high-dimensional articulatory and acoustic signals there is information related to theoretical categories? For a variety of reasons, it is problematic to directly quantify mutual information between hypothesized categories and signals. To address this issue, a multi-scale analysis method is proposed for localizing category-related information in an ensemble of speech signals using machine learning algorithms. By analyzing how classification accuracy on unseen data varies as the temporal extent of training input is systematically restricted, inferences can be drawn regarding the temporal distribution of category-related information. The method can also be used to investigate redundancy between subsets of signal dimensions. Two types of theoretical categories are examined in this paper: phonemic/gestural categories and syntactic relative clause categories. Moreover, two different machine learning algorithms were examined: linear discriminant analysis and neural networks with long short-term memory units. Both algorithms detected category-related information earlier and later in signals than would be expected given standard theoretical assumptions about when linguistic categories should influence speech. The neural network algorithm was able to identify category-related information to a greater extent than the discriminant analyses. |
| first_indexed | 2024-12-23T02:08:19Z |
| format | Article |
| id | doaj.art-f6886a041f6c4acc9adc938b3350f949 |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-12-23T02:08:19Z |
| publishDate | 2021-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-f6886a041f6c4acc9adc938b3350f9492022-12-21T18:03:50ZengPublic Library of Science (PLoS)PLoS ONE1932-62032021-01-011610e025817810.1371/journal.pone.0258178Localizing category-related information in speech with multi-scale analyses.Sam TilsenSeung-Eun KimClaire WangMeasurements of the physical outputs of speech-vocal tract geometry and acoustic energy-are high-dimensional, but linguistic theories posit a low-dimensional set of categories such as phonemes and phrase types. How can it be determined when and where in high-dimensional articulatory and acoustic signals there is information related to theoretical categories? For a variety of reasons, it is problematic to directly quantify mutual information between hypothesized categories and signals. To address this issue, a multi-scale analysis method is proposed for localizing category-related information in an ensemble of speech signals using machine learning algorithms. By analyzing how classification accuracy on unseen data varies as the temporal extent of training input is systematically restricted, inferences can be drawn regarding the temporal distribution of category-related information. The method can also be used to investigate redundancy between subsets of signal dimensions. Two types of theoretical categories are examined in this paper: phonemic/gestural categories and syntactic relative clause categories. Moreover, two different machine learning algorithms were examined: linear discriminant analysis and neural networks with long short-term memory units. Both algorithms detected category-related information earlier and later in signals than would be expected given standard theoretical assumptions about when linguistic categories should influence speech. The neural network algorithm was able to identify category-related information to a greater extent than the discriminant analyses.https://doi.org/10.1371/journal.pone.0258178 |
| spellingShingle | Sam Tilsen Seung-Eun Kim Claire Wang Localizing category-related information in speech with multi-scale analyses. PLoS ONE |
| title | Localizing category-related information in speech with multi-scale analyses. |
| title_full | Localizing category-related information in speech with multi-scale analyses. |
| title_fullStr | Localizing category-related information in speech with multi-scale analyses. |
| title_full_unstemmed | Localizing category-related information in speech with multi-scale analyses. |
| title_short | Localizing category-related information in speech with multi-scale analyses. |
| title_sort | localizing category related information in speech with multi scale analyses |
| url | https://doi.org/10.1371/journal.pone.0258178 |
| work_keys_str_mv | AT samtilsen localizingcategoryrelatedinformationinspeechwithmultiscaleanalyses AT seungeunkim localizingcategoryrelatedinformationinspeechwithmultiscaleanalyses AT clairewang localizingcategoryrelatedinformationinspeechwithmultiscaleanalyses |