Symptom-based scoring technique by machine learning to predict COVID-19: a validation study
Abstract Background Coronavirus disease 2019 (COVID-19) surges, such as that which occurred when omicron variants emerged, may overwhelm healthcare systems. To function properly, such systems should balance detection and workloads by improving referrals using simple yet precise and sensitive diagnos...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-12-01
|
| Series: | BMC Infectious Diseases |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12879-023-08846-0 |
| _version_ | 1797388492713492480 |
|---|---|
| author | Amelia Nur Vidyanti Sekar Satiti Atitya Fithri Khairani Aditya Rifqi Fauzi Muhammad Hardhantyo Herdiantri Sufriyana Emily Chia-Yu Su |
| author_facet | Amelia Nur Vidyanti Sekar Satiti Atitya Fithri Khairani Aditya Rifqi Fauzi Muhammad Hardhantyo Herdiantri Sufriyana Emily Chia-Yu Su |
| author_sort | Amelia Nur Vidyanti |
| collection | DOAJ |
| description | Abstract Background Coronavirus disease 2019 (COVID-19) surges, such as that which occurred when omicron variants emerged, may overwhelm healthcare systems. To function properly, such systems should balance detection and workloads by improving referrals using simple yet precise and sensitive diagnostic predictions. A symptom-based scoring system was developed using machine learning for the general population, but no validation has occurred in healthcare settings. We aimed to validate a COVID-19 scoring system using self-reported symptoms, including loss of smell and taste as major indicators. Methods A cross-sectional study was conducted to evaluate medical records of patients admitted to Dr. Sardjito Hospital, Yogyakarta, Indonesia, from March 2020 to December 2021. Outcomes were defined by a reverse-transcription polymerase chain reaction (RT-PCR). We compared the symptom-based scoring system, as the index test, with antigen tests, antibody tests, and clinical judgements by primary care physicians. To validate use of the index test to improve referral, we evaluated positive predictive value (PPV) and sensitivity. Results After clinical judgement with a PPV of 61% (n = 327/530, 95% confidence interval [CI]: 60% to 62%), confirmation with the index test resulted in the highest PPV of 85% (n = 30/35, 95% CI: 83% to 87%) but the lowest sensitivity (n = 30/180, 17%, 95% CI: 15% to 19%). If this confirmation was defined by either positive predictive scoring or antigen tests, the PPV was 92% (n = 55/60, 95% CI: 90% to 94%). Meanwhile, the sensitivity was 88% (n = 55/62, 95% CI: 87% to 89%), which was higher than that when using only antigen tests (n = 29/41, 71%, 95% CI: 69% to 73%). Conclusions The symptom-based COVID-19 predictive score was validated in healthcare settings for its precision and sensitivity. However, an impact study is needed to confirm if this can balance detection and workload for the next COVID-19 surge. |
| first_indexed | 2024-03-08T22:41:37Z |
| format | Article |
| id | doaj.art-a263b86f9e4e4c8fb4319508bb8c2267 |
| institution | Directory Open Access Journal |
| issn | 1471-2334 |
| language | English |
| last_indexed | 2024-03-08T22:41:37Z |
| publishDate | 2023-12-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Infectious Diseases |
| spelling | doaj.art-a263b86f9e4e4c8fb4319508bb8c22672023-12-17T12:08:03ZengBMCBMC Infectious Diseases1471-23342023-12-0123111110.1186/s12879-023-08846-0Symptom-based scoring technique by machine learning to predict COVID-19: a validation studyAmelia Nur Vidyanti0Sekar Satiti1Atitya Fithri Khairani2Aditya Rifqi Fauzi3Muhammad Hardhantyo4Herdiantri Sufriyana5Emily Chia-Yu Su6Department of Neurology, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah MadaDepartment of Neurology, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah MadaDepartment of Neurology, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah MadaDepartment of Neurology, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah MadaCenter for Health Policy and Management, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah MadaGraduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical UniversityGraduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical UniversityAbstract Background Coronavirus disease 2019 (COVID-19) surges, such as that which occurred when omicron variants emerged, may overwhelm healthcare systems. To function properly, such systems should balance detection and workloads by improving referrals using simple yet precise and sensitive diagnostic predictions. A symptom-based scoring system was developed using machine learning for the general population, but no validation has occurred in healthcare settings. We aimed to validate a COVID-19 scoring system using self-reported symptoms, including loss of smell and taste as major indicators. Methods A cross-sectional study was conducted to evaluate medical records of patients admitted to Dr. Sardjito Hospital, Yogyakarta, Indonesia, from March 2020 to December 2021. Outcomes were defined by a reverse-transcription polymerase chain reaction (RT-PCR). We compared the symptom-based scoring system, as the index test, with antigen tests, antibody tests, and clinical judgements by primary care physicians. To validate use of the index test to improve referral, we evaluated positive predictive value (PPV) and sensitivity. Results After clinical judgement with a PPV of 61% (n = 327/530, 95% confidence interval [CI]: 60% to 62%), confirmation with the index test resulted in the highest PPV of 85% (n = 30/35, 95% CI: 83% to 87%) but the lowest sensitivity (n = 30/180, 17%, 95% CI: 15% to 19%). If this confirmation was defined by either positive predictive scoring or antigen tests, the PPV was 92% (n = 55/60, 95% CI: 90% to 94%). Meanwhile, the sensitivity was 88% (n = 55/62, 95% CI: 87% to 89%), which was higher than that when using only antigen tests (n = 29/41, 71%, 95% CI: 69% to 73%). Conclusions The symptom-based COVID-19 predictive score was validated in healthcare settings for its precision and sensitivity. However, an impact study is needed to confirm if this can balance detection and workload for the next COVID-19 surge.https://doi.org/10.1186/s12879-023-08846-0COVID-19Clinical prediction rulesValidation studyMachine learningHospital referral |
| spellingShingle | Amelia Nur Vidyanti Sekar Satiti Atitya Fithri Khairani Aditya Rifqi Fauzi Muhammad Hardhantyo Herdiantri Sufriyana Emily Chia-Yu Su Symptom-based scoring technique by machine learning to predict COVID-19: a validation study BMC Infectious Diseases COVID-19 Clinical prediction rules Validation study Machine learning Hospital referral |
| title | Symptom-based scoring technique by machine learning to predict COVID-19: a validation study |
| title_full | Symptom-based scoring technique by machine learning to predict COVID-19: a validation study |
| title_fullStr | Symptom-based scoring technique by machine learning to predict COVID-19: a validation study |
| title_full_unstemmed | Symptom-based scoring technique by machine learning to predict COVID-19: a validation study |
| title_short | Symptom-based scoring technique by machine learning to predict COVID-19: a validation study |
| title_sort | symptom based scoring technique by machine learning to predict covid 19 a validation study |
| topic | COVID-19 Clinical prediction rules Validation study Machine learning Hospital referral |
| url | https://doi.org/10.1186/s12879-023-08846-0 |
| work_keys_str_mv | AT amelianurvidyanti symptombasedscoringtechniquebymachinelearningtopredictcovid19avalidationstudy AT sekarsatiti symptombasedscoringtechniquebymachinelearningtopredictcovid19avalidationstudy AT atityafithrikhairani symptombasedscoringtechniquebymachinelearningtopredictcovid19avalidationstudy AT adityarifqifauzi symptombasedscoringtechniquebymachinelearningtopredictcovid19avalidationstudy AT muhammadhardhantyo symptombasedscoringtechniquebymachinelearningtopredictcovid19avalidationstudy AT herdiantrisufriyana symptombasedscoringtechniquebymachinelearningtopredictcovid19avalidationstudy AT emilychiayusu symptombasedscoringtechniquebymachinelearningtopredictcovid19avalidationstudy |