Limitations of the Cough Sound-Based COVID-19 Diagnosis Artificial Intelligence Model and its Future Direction: Longitudinal Observation Study
BackgroundThe outbreak of SARS-CoV-2 in 2019 has necessitated the rapid and accurate detection of COVID-19 to manage patients effectively and implement public health measures. Artificial intelligence (AI) models analyzing cough sounds have emerged as promising tools for large...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2024-02-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2024/1/e51640 |
| _version_ | 1797322707728072704 |
|---|---|
| author | Jina Kim Yong Sung Choi Young Joo Lee Seung Geun Yeo Kyung Won Kim Min Seo Kim Masoud Rahmati Dong Keon Yon Jinseok Lee |
| author_facet | Jina Kim Yong Sung Choi Young Joo Lee Seung Geun Yeo Kyung Won Kim Min Seo Kim Masoud Rahmati Dong Keon Yon Jinseok Lee |
| author_sort | Jina Kim |
| collection | DOAJ |
| description |
BackgroundThe outbreak of SARS-CoV-2 in 2019 has necessitated the rapid and accurate detection of COVID-19 to manage patients effectively and implement public health measures. Artificial intelligence (AI) models analyzing cough sounds have emerged as promising tools for large-scale screening and early identification of potential cases.
ObjectiveThis study aimed to investigate the efficacy of using cough sounds as a diagnostic tool for COVID-19, considering the unique acoustic features that differentiate positive and negative cases. We investigated whether an AI model trained on cough sound recordings from specific periods, especially the early stages of the COVID-19 pandemic, were applicable to the ongoing situation with persistent variants.
MethodsWe used cough sound recordings from 3 data sets (Cambridge, Coswara, and Virufy) representing different stages of the pandemic and variants. Our AI model was trained using the Cambridge data set with subsequent evaluation against all data sets. The performance was analyzed based on the area under the receiver operating curve (AUC) across different data measurement periods and COVID-19 variants.
ResultsThe AI model demonstrated a high AUC when tested with the Cambridge data set, indicative of its initial effectiveness. However, the performance varied significantly with other data sets, particularly in detecting later variants such as Delta and Omicron, with a marked decline in AUC observed for the latter. These results highlight the challenges in maintaining the efficacy of AI models against the backdrop of an evolving virus.
ConclusionsWhile AI models analyzing cough sounds offer a promising noninvasive and rapid screening method for COVID-19, their effectiveness is challenged by the emergence of new virus variants. Ongoing research and adaptations in AI methodologies are crucial to address these limitations. The adaptability of AI models to evolve with the virus underscores their potential as a foundational technology for not only the current pandemic but also future outbreaks, contributing to a more agile and resilient global health infrastructure. |
| first_indexed | 2024-03-08T05:17:13Z |
| format | Article |
| id | doaj.art-8c425bbb1a5d4a07a8350c3a39af95c6 |
| institution | Directory Open Access Journal |
| issn | 1438-8871 |
| language | English |
| last_indexed | 2024-03-08T05:17:13Z |
| publishDate | 2024-02-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj.art-8c425bbb1a5d4a07a8350c3a39af95c62024-02-06T16:00:35ZengJMIR PublicationsJournal of Medical Internet Research1438-88712024-02-0126e5164010.2196/51640Limitations of the Cough Sound-Based COVID-19 Diagnosis Artificial Intelligence Model and its Future Direction: Longitudinal Observation StudyJina Kimhttps://orcid.org/0009-0006-4661-7905Yong Sung Choihttps://orcid.org/0000-0001-9181-7849Young Joo Leehttps://orcid.org/0000-0001-5294-7368Seung Geun Yeohttps://orcid.org/0000-0001-8021-1024Kyung Won Kimhttps://orcid.org/0000-0002-1532-5970Min Seo Kimhttps://orcid.org/0000-0003-2115-7835Masoud Rahmatihttps://orcid.org/0000-0003-4792-027XDong Keon Yonhttps://orcid.org/0000-0003-1628-9948Jinseok Leehttps://orcid.org/0000-0002-8580-490X BackgroundThe outbreak of SARS-CoV-2 in 2019 has necessitated the rapid and accurate detection of COVID-19 to manage patients effectively and implement public health measures. Artificial intelligence (AI) models analyzing cough sounds have emerged as promising tools for large-scale screening and early identification of potential cases. ObjectiveThis study aimed to investigate the efficacy of using cough sounds as a diagnostic tool for COVID-19, considering the unique acoustic features that differentiate positive and negative cases. We investigated whether an AI model trained on cough sound recordings from specific periods, especially the early stages of the COVID-19 pandemic, were applicable to the ongoing situation with persistent variants. MethodsWe used cough sound recordings from 3 data sets (Cambridge, Coswara, and Virufy) representing different stages of the pandemic and variants. Our AI model was trained using the Cambridge data set with subsequent evaluation against all data sets. The performance was analyzed based on the area under the receiver operating curve (AUC) across different data measurement periods and COVID-19 variants. ResultsThe AI model demonstrated a high AUC when tested with the Cambridge data set, indicative of its initial effectiveness. However, the performance varied significantly with other data sets, particularly in detecting later variants such as Delta and Omicron, with a marked decline in AUC observed for the latter. These results highlight the challenges in maintaining the efficacy of AI models against the backdrop of an evolving virus. ConclusionsWhile AI models analyzing cough sounds offer a promising noninvasive and rapid screening method for COVID-19, their effectiveness is challenged by the emergence of new virus variants. Ongoing research and adaptations in AI methodologies are crucial to address these limitations. The adaptability of AI models to evolve with the virus underscores their potential as a foundational technology for not only the current pandemic but also future outbreaks, contributing to a more agile and resilient global health infrastructure.https://www.jmir.org/2024/1/e51640 |
| spellingShingle | Jina Kim Yong Sung Choi Young Joo Lee Seung Geun Yeo Kyung Won Kim Min Seo Kim Masoud Rahmati Dong Keon Yon Jinseok Lee Limitations of the Cough Sound-Based COVID-19 Diagnosis Artificial Intelligence Model and its Future Direction: Longitudinal Observation Study Journal of Medical Internet Research |
| title | Limitations of the Cough Sound-Based COVID-19 Diagnosis Artificial Intelligence Model and its Future Direction: Longitudinal Observation Study |
| title_full | Limitations of the Cough Sound-Based COVID-19 Diagnosis Artificial Intelligence Model and its Future Direction: Longitudinal Observation Study |
| title_fullStr | Limitations of the Cough Sound-Based COVID-19 Diagnosis Artificial Intelligence Model and its Future Direction: Longitudinal Observation Study |
| title_full_unstemmed | Limitations of the Cough Sound-Based COVID-19 Diagnosis Artificial Intelligence Model and its Future Direction: Longitudinal Observation Study |
| title_short | Limitations of the Cough Sound-Based COVID-19 Diagnosis Artificial Intelligence Model and its Future Direction: Longitudinal Observation Study |
| title_sort | limitations of the cough sound based covid 19 diagnosis artificial intelligence model and its future direction longitudinal observation study |
| url | https://www.jmir.org/2024/1/e51640 |
| work_keys_str_mv | AT jinakim limitationsofthecoughsoundbasedcovid19diagnosisartificialintelligencemodelanditsfuturedirectionlongitudinalobservationstudy AT yongsungchoi limitationsofthecoughsoundbasedcovid19diagnosisartificialintelligencemodelanditsfuturedirectionlongitudinalobservationstudy AT youngjoolee limitationsofthecoughsoundbasedcovid19diagnosisartificialintelligencemodelanditsfuturedirectionlongitudinalobservationstudy AT seunggeunyeo limitationsofthecoughsoundbasedcovid19diagnosisartificialintelligencemodelanditsfuturedirectionlongitudinalobservationstudy AT kyungwonkim limitationsofthecoughsoundbasedcovid19diagnosisartificialintelligencemodelanditsfuturedirectionlongitudinalobservationstudy AT minseokim limitationsofthecoughsoundbasedcovid19diagnosisartificialintelligencemodelanditsfuturedirectionlongitudinalobservationstudy AT masoudrahmati limitationsofthecoughsoundbasedcovid19diagnosisartificialintelligencemodelanditsfuturedirectionlongitudinalobservationstudy AT dongkeonyon limitationsofthecoughsoundbasedcovid19diagnosisartificialintelligencemodelanditsfuturedirectionlongitudinalobservationstudy AT jinseoklee limitationsofthecoughsoundbasedcovid19diagnosisartificialintelligencemodelanditsfuturedirectionlongitudinalobservationstudy |