Transforming assessment of speech in children with cleft palate via online crowdsourcing.
OBJECTIVE:Speech intelligibility is fundamental to social interactions and a critical surgical outcome in patients with cleft palate. Online crowdsourcing is a burgeoning technology, with potential to mitigate the burden of limited accessibility to speech-language-pathologists (SLPs). This pilot stu...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2020-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0227686 |
| _version_ | 1818587854493712384 |
|---|---|
| author | Anne M Sescleifer Caitlin A Francoisse Janna C Webber Jeffrey D Rector Alexander Y Lin |
| author_facet | Anne M Sescleifer Caitlin A Francoisse Janna C Webber Jeffrey D Rector Alexander Y Lin |
| author_sort | Anne M Sescleifer |
| collection | DOAJ |
| description | OBJECTIVE:Speech intelligibility is fundamental to social interactions and a critical surgical outcome in patients with cleft palate. Online crowdsourcing is a burgeoning technology, with potential to mitigate the burden of limited accessibility to speech-language-pathologists (SLPs). This pilot study investigates the concordance of online crowdsourced evaluations of hypernasality with SLP ratings of children with cleft palate. METHODS:Six audio-phrases each from children with cleft palate were assessed by online crowdsourcing using Amazon Mechanical Turk (MTurk), and compared to SLP's gold-standard hypernasality score on the Pittsburgh Weighted Speech Score (PWSS). Phrases were presented to MTurk crowdsourced lay-raters to assess hypernasality on a Likert scale analogous to the PWSS. The survey included clickable reference audio samples for different levels of hypernasality. RESULTS:1,088 unique online crowdsourced speech ratings were collected on 16 sentences of 3 children with cleft palate aged 4-8 years, with audio averaging 6.5 years follow-up after cleft palate surgery. Patient 1 crowd-mean was 2.62 (SLP rated 2-3); Patient 2 crowd-mean 2.66 (SLP rated 3); and Patient 3 crowd-mean 1.76 (SLP rated 2). Rounded for consistency with PWSS scale, all patients matched SLP ratings. Different sentences had different accuracies compared to the SLP gold standard scores. CONCLUSION:Online crowdsourced ratings of hypernasal speech in children with cleft palate were concordant with SLP ratings, predicting SLP scores in all 3 patients. This novel technology has potential for translation in clinical speech assessments, and may serve as a valuable screening tool for non-experts to identify children requiring further assessment and intervention by a qualified speech language pathology expert. |
| first_indexed | 2024-12-16T09:15:28Z |
| format | Article |
| id | doaj.art-5284dcf59620409991190d4c7ebb82a7 |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-12-16T09:15:28Z |
| publishDate | 2020-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-5284dcf59620409991190d4c7ebb82a72022-12-21T22:36:53ZengPublic Library of Science (PLoS)PLoS ONE1932-62032020-01-01151e022768610.1371/journal.pone.0227686Transforming assessment of speech in children with cleft palate via online crowdsourcing.Anne M SescleiferCaitlin A FrancoisseJanna C WebberJeffrey D RectorAlexander Y LinOBJECTIVE:Speech intelligibility is fundamental to social interactions and a critical surgical outcome in patients with cleft palate. Online crowdsourcing is a burgeoning technology, with potential to mitigate the burden of limited accessibility to speech-language-pathologists (SLPs). This pilot study investigates the concordance of online crowdsourced evaluations of hypernasality with SLP ratings of children with cleft palate. METHODS:Six audio-phrases each from children with cleft palate were assessed by online crowdsourcing using Amazon Mechanical Turk (MTurk), and compared to SLP's gold-standard hypernasality score on the Pittsburgh Weighted Speech Score (PWSS). Phrases were presented to MTurk crowdsourced lay-raters to assess hypernasality on a Likert scale analogous to the PWSS. The survey included clickable reference audio samples for different levels of hypernasality. RESULTS:1,088 unique online crowdsourced speech ratings were collected on 16 sentences of 3 children with cleft palate aged 4-8 years, with audio averaging 6.5 years follow-up after cleft palate surgery. Patient 1 crowd-mean was 2.62 (SLP rated 2-3); Patient 2 crowd-mean 2.66 (SLP rated 3); and Patient 3 crowd-mean 1.76 (SLP rated 2). Rounded for consistency with PWSS scale, all patients matched SLP ratings. Different sentences had different accuracies compared to the SLP gold standard scores. CONCLUSION:Online crowdsourced ratings of hypernasal speech in children with cleft palate were concordant with SLP ratings, predicting SLP scores in all 3 patients. This novel technology has potential for translation in clinical speech assessments, and may serve as a valuable screening tool for non-experts to identify children requiring further assessment and intervention by a qualified speech language pathology expert.https://doi.org/10.1371/journal.pone.0227686 |
| spellingShingle | Anne M Sescleifer Caitlin A Francoisse Janna C Webber Jeffrey D Rector Alexander Y Lin Transforming assessment of speech in children with cleft palate via online crowdsourcing. PLoS ONE |
| title | Transforming assessment of speech in children with cleft palate via online crowdsourcing. |
| title_full | Transforming assessment of speech in children with cleft palate via online crowdsourcing. |
| title_fullStr | Transforming assessment of speech in children with cleft palate via online crowdsourcing. |
| title_full_unstemmed | Transforming assessment of speech in children with cleft palate via online crowdsourcing. |
| title_short | Transforming assessment of speech in children with cleft palate via online crowdsourcing. |
| title_sort | transforming assessment of speech in children with cleft palate via online crowdsourcing |
| url | https://doi.org/10.1371/journal.pone.0227686 |
| work_keys_str_mv | AT annemsescleifer transformingassessmentofspeechinchildrenwithcleftpalateviaonlinecrowdsourcing AT caitlinafrancoisse transformingassessmentofspeechinchildrenwithcleftpalateviaonlinecrowdsourcing AT jannacwebber transformingassessmentofspeechinchildrenwithcleftpalateviaonlinecrowdsourcing AT jeffreydrector transformingassessmentofspeechinchildrenwithcleftpalateviaonlinecrowdsourcing AT alexanderylin transformingassessmentofspeechinchildrenwithcleftpalateviaonlinecrowdsourcing |