Estimating Patient-Level Uncertainty in Seizure Detection Using Group-Specific Out-of-Distribution Detection Technique
Epilepsy is a chronic neurological disorder affecting around 1% of the global population, characterized by recurrent epileptic seizures. Accurate diagnosis and treatment are crucial for reducing mortality rates. Recent advancements in machine learning (ML) algorithms have shown potential in aiding c...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/20/8375 |
| _version_ | 1797572330812080128 |
|---|---|
| author | Sheng Wong Anj Simmons Jessica Rivera Villicana Scott Barnett |
| author_facet | Sheng Wong Anj Simmons Jessica Rivera Villicana Scott Barnett |
| author_sort | Sheng Wong |
| collection | DOAJ |
| description | Epilepsy is a chronic neurological disorder affecting around 1% of the global population, characterized by recurrent epileptic seizures. Accurate diagnosis and treatment are crucial for reducing mortality rates. Recent advancements in machine learning (ML) algorithms have shown potential in aiding clinicians with seizure detection in electroencephalography (EEG) data. However, these algorithms face significant challenges due to the patient-specific variability in seizure patterns and the limited availability of high-quality EEG data for training, causing erratic predictions. These erratic predictions are harmful, especially for high-stake domains in healthcare, negatively affecting patients. Therefore, ensuring safety in AI is of the utmost importance. In this study, we propose a novel ensemble method for uncertainty quantification to identify patients with low-confidence predictions in ML-based seizure detection algorithms. Our approach aims to mitigate high-risk predictions in previously unseen seizure patients, thereby enhancing the robustness of existing seizure detection algorithms. Additionally, our method can be implemented with most of the deep learning (DL) models. We evaluated the proposed method against established uncertainty detection techniques, demonstrating its effectiveness in identifying patients for whom the model’s predictions are less certain. Our proposed method managed to achieve 87%, 89% and 75% in accuracy, specificity and sensitivity, respectively. This study represents a novel attempt to improve the reliability and robustness of DL algorithms in the domain of seizure detection. This study underscores the value of integrating uncertainty quantification into ML algorithms for seizure detection, offering clinicians a practical tool to gauge the applicability of ML models for individual patients. |
| first_indexed | 2024-03-10T20:54:44Z |
| format | Article |
| id | doaj.art-733d8f15cc8d44a5834d45e555d2b3c7 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T20:54:44Z |
| publishDate | 2023-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-733d8f15cc8d44a5834d45e555d2b3c72023-11-19T18:01:56ZengMDPI AGSensors1424-82202023-10-012320837510.3390/s23208375Estimating Patient-Level Uncertainty in Seizure Detection Using Group-Specific Out-of-Distribution Detection TechniqueSheng Wong0Anj Simmons1Jessica Rivera Villicana2Scott Barnett3Applied Artificial Intelligence Institute, Deakin University, Burwood, VIC 3125, AustraliaApplied Artificial Intelligence Institute, Deakin University, Burwood, VIC 3125, AustraliaSchool of Computing Technologies, RMIT University, Melbourne, VIC 3000, AustraliaApplied Artificial Intelligence Institute, Deakin University, Burwood, VIC 3125, AustraliaEpilepsy is a chronic neurological disorder affecting around 1% of the global population, characterized by recurrent epileptic seizures. Accurate diagnosis and treatment are crucial for reducing mortality rates. Recent advancements in machine learning (ML) algorithms have shown potential in aiding clinicians with seizure detection in electroencephalography (EEG) data. However, these algorithms face significant challenges due to the patient-specific variability in seizure patterns and the limited availability of high-quality EEG data for training, causing erratic predictions. These erratic predictions are harmful, especially for high-stake domains in healthcare, negatively affecting patients. Therefore, ensuring safety in AI is of the utmost importance. In this study, we propose a novel ensemble method for uncertainty quantification to identify patients with low-confidence predictions in ML-based seizure detection algorithms. Our approach aims to mitigate high-risk predictions in previously unseen seizure patients, thereby enhancing the robustness of existing seizure detection algorithms. Additionally, our method can be implemented with most of the deep learning (DL) models. We evaluated the proposed method against established uncertainty detection techniques, demonstrating its effectiveness in identifying patients for whom the model’s predictions are less certain. Our proposed method managed to achieve 87%, 89% and 75% in accuracy, specificity and sensitivity, respectively. This study represents a novel attempt to improve the reliability and robustness of DL algorithms in the domain of seizure detection. This study underscores the value of integrating uncertainty quantification into ML algorithms for seizure detection, offering clinicians a practical tool to gauge the applicability of ML models for individual patients.https://www.mdpi.com/1424-8220/23/20/8375EEGrule-based reasoninguncertainty estimationout-of-distributionseizure detection |
| spellingShingle | Sheng Wong Anj Simmons Jessica Rivera Villicana Scott Barnett Estimating Patient-Level Uncertainty in Seizure Detection Using Group-Specific Out-of-Distribution Detection Technique Sensors EEG rule-based reasoning uncertainty estimation out-of-distribution seizure detection |
| title | Estimating Patient-Level Uncertainty in Seizure Detection Using Group-Specific Out-of-Distribution Detection Technique |
| title_full | Estimating Patient-Level Uncertainty in Seizure Detection Using Group-Specific Out-of-Distribution Detection Technique |
| title_fullStr | Estimating Patient-Level Uncertainty in Seizure Detection Using Group-Specific Out-of-Distribution Detection Technique |
| title_full_unstemmed | Estimating Patient-Level Uncertainty in Seizure Detection Using Group-Specific Out-of-Distribution Detection Technique |
| title_short | Estimating Patient-Level Uncertainty in Seizure Detection Using Group-Specific Out-of-Distribution Detection Technique |
| title_sort | estimating patient level uncertainty in seizure detection using group specific out of distribution detection technique |
| topic | EEG rule-based reasoning uncertainty estimation out-of-distribution seizure detection |
| url | https://www.mdpi.com/1424-8220/23/20/8375 |
| work_keys_str_mv | AT shengwong estimatingpatientleveluncertaintyinseizuredetectionusinggroupspecificoutofdistributiondetectiontechnique AT anjsimmons estimatingpatientleveluncertaintyinseizuredetectionusinggroupspecificoutofdistributiondetectiontechnique AT jessicariveravillicana estimatingpatientleveluncertaintyinseizuredetectionusinggroupspecificoutofdistributiondetectiontechnique AT scottbarnett estimatingpatientleveluncertaintyinseizuredetectionusinggroupspecificoutofdistributiondetectiontechnique |