Prediction of emergency department revisits among child and youth mental health outpatients using deep learning techniques
Abstract Background The proportion of Canadian youth seeking mental health support from an emergency department (ED) has risen in recent years. As EDs typically address urgent mental health crises, revisiting an ED may represent unmet mental health needs. Accurate ED revisit prediction could aid ear...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2024-02-01
|
| Series: | BMC Medical Informatics and Decision Making |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12911-024-02450-1 |
| _version_ | 1797274342625640448 |
|---|---|
| author | Simran Saggu Hirad Daneshvar Reza Samavi Paulo Pires Roberto B. Sassi Thomas E. Doyle Judy Zhao Ahmad Mauluddin Laura Duncan |
| author_facet | Simran Saggu Hirad Daneshvar Reza Samavi Paulo Pires Roberto B. Sassi Thomas E. Doyle Judy Zhao Ahmad Mauluddin Laura Duncan |
| author_sort | Simran Saggu |
| collection | DOAJ |
| description | Abstract Background The proportion of Canadian youth seeking mental health support from an emergency department (ED) has risen in recent years. As EDs typically address urgent mental health crises, revisiting an ED may represent unmet mental health needs. Accurate ED revisit prediction could aid early intervention and ensure efficient healthcare resource allocation. We examine the potential increased accuracy and performance of graph neural network (GNN) machine learning models compared to recurrent neural network (RNN), and baseline conventional machine learning and regression models for predicting ED revisit in electronic health record (EHR) data. Methods This study used EHR data for children and youth aged 4–17 seeking services at McMaster Children’s Hospital’s Child and Youth Mental Health Program outpatient service to develop and evaluate GNN and RNN models to predict whether a child/youth with an ED visit had an ED revisit within 30 days. GNN and RNN models were developed and compared against conventional baseline models. Model performance for GNN, RNN, XGBoost, decision tree and logistic regression models was evaluated using F1 scores. Results The GNN model outperformed the RNN model by an F1-score increase of 0.0511 and the best performing conventional machine learning model by an F1-score increase of 0.0470. Precision, recall, receiver operating characteristic (ROC) curves, and positive and negative predictive values showed that the GNN model performed the best, and the RNN model performed similarly to the XGBoost model. Performance increases were most noticeable for recall and negative predictive value than for precision and positive predictive value. Conclusions This study demonstrates the improved accuracy and potential utility of GNN models in predicting ED revisits among children and youth, although model performance may not be sufficient for clinical implementation. Given the improvements in recall and negative predictive value, GNN models should be further explored to develop algorithms that can inform clinical decision-making in ways that facilitate targeted interventions, optimize resource allocation, and improve outcomes for children and youth. |
| first_indexed | 2024-03-07T14:57:04Z |
| format | Article |
| id | doaj.art-004911075cd94f68a39611fc6b65a5b6 |
| institution | Directory Open Access Journal |
| issn | 1472-6947 |
| language | English |
| last_indexed | 2024-03-07T14:57:04Z |
| publishDate | 2024-02-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Informatics and Decision Making |
| spelling | doaj.art-004911075cd94f68a39611fc6b65a5b62024-03-05T19:19:58ZengBMCBMC Medical Informatics and Decision Making1472-69472024-02-012411910.1186/s12911-024-02450-1Prediction of emergency department revisits among child and youth mental health outpatients using deep learning techniquesSimran Saggu0Hirad Daneshvar1Reza Samavi2Paulo Pires3Roberto B. Sassi4Thomas E. Doyle5Judy Zhao6Ahmad Mauluddin7Laura Duncan8Department of Health Research Methodology, Evidence & Impact, McMaster UniversityDepartment of Electrical, Computer and Biomedical Engineering, Toronto Metropolitan UniversityDepartment of Electrical, Computer and Biomedical Engineering, Toronto Metropolitan UniversityDepartment of Psychiatry & Behavioural Neurosciences, McMaster UniversityDepartment of Psychiatry, University of British Columbia, UBC Vancouver CampusDepartment of Electrical & Computer Engineering, McMaster UniversityMcMaster Children’s Hospital, Hamilton Health SciencesMcMaster Children’s Hospital, Hamilton Health SciencesDepartment of Psychiatry & Behavioural Neurosciences, McMaster UniversityAbstract Background The proportion of Canadian youth seeking mental health support from an emergency department (ED) has risen in recent years. As EDs typically address urgent mental health crises, revisiting an ED may represent unmet mental health needs. Accurate ED revisit prediction could aid early intervention and ensure efficient healthcare resource allocation. We examine the potential increased accuracy and performance of graph neural network (GNN) machine learning models compared to recurrent neural network (RNN), and baseline conventional machine learning and regression models for predicting ED revisit in electronic health record (EHR) data. Methods This study used EHR data for children and youth aged 4–17 seeking services at McMaster Children’s Hospital’s Child and Youth Mental Health Program outpatient service to develop and evaluate GNN and RNN models to predict whether a child/youth with an ED visit had an ED revisit within 30 days. GNN and RNN models were developed and compared against conventional baseline models. Model performance for GNN, RNN, XGBoost, decision tree and logistic regression models was evaluated using F1 scores. Results The GNN model outperformed the RNN model by an F1-score increase of 0.0511 and the best performing conventional machine learning model by an F1-score increase of 0.0470. Precision, recall, receiver operating characteristic (ROC) curves, and positive and negative predictive values showed that the GNN model performed the best, and the RNN model performed similarly to the XGBoost model. Performance increases were most noticeable for recall and negative predictive value than for precision and positive predictive value. Conclusions This study demonstrates the improved accuracy and potential utility of GNN models in predicting ED revisits among children and youth, although model performance may not be sufficient for clinical implementation. Given the improvements in recall and negative predictive value, GNN models should be further explored to develop algorithms that can inform clinical decision-making in ways that facilitate targeted interventions, optimize resource allocation, and improve outcomes for children and youth.https://doi.org/10.1186/s12911-024-02450-1Mental healthMachine learningGraph neural networkDeep learningEmergency departmentRevisits |
| spellingShingle | Simran Saggu Hirad Daneshvar Reza Samavi Paulo Pires Roberto B. Sassi Thomas E. Doyle Judy Zhao Ahmad Mauluddin Laura Duncan Prediction of emergency department revisits among child and youth mental health outpatients using deep learning techniques BMC Medical Informatics and Decision Making Mental health Machine learning Graph neural network Deep learning Emergency department Revisits |
| title | Prediction of emergency department revisits among child and youth mental health outpatients using deep learning techniques |
| title_full | Prediction of emergency department revisits among child and youth mental health outpatients using deep learning techniques |
| title_fullStr | Prediction of emergency department revisits among child and youth mental health outpatients using deep learning techniques |
| title_full_unstemmed | Prediction of emergency department revisits among child and youth mental health outpatients using deep learning techniques |
| title_short | Prediction of emergency department revisits among child and youth mental health outpatients using deep learning techniques |
| title_sort | prediction of emergency department revisits among child and youth mental health outpatients using deep learning techniques |
| topic | Mental health Machine learning Graph neural network Deep learning Emergency department Revisits |
| url | https://doi.org/10.1186/s12911-024-02450-1 |
| work_keys_str_mv | AT simransaggu predictionofemergencydepartmentrevisitsamongchildandyouthmentalhealthoutpatientsusingdeeplearningtechniques AT hiraddaneshvar predictionofemergencydepartmentrevisitsamongchildandyouthmentalhealthoutpatientsusingdeeplearningtechniques AT rezasamavi predictionofemergencydepartmentrevisitsamongchildandyouthmentalhealthoutpatientsusingdeeplearningtechniques AT paulopires predictionofemergencydepartmentrevisitsamongchildandyouthmentalhealthoutpatientsusingdeeplearningtechniques AT robertobsassi predictionofemergencydepartmentrevisitsamongchildandyouthmentalhealthoutpatientsusingdeeplearningtechniques AT thomasedoyle predictionofemergencydepartmentrevisitsamongchildandyouthmentalhealthoutpatientsusingdeeplearningtechniques AT judyzhao predictionofemergencydepartmentrevisitsamongchildandyouthmentalhealthoutpatientsusingdeeplearningtechniques AT ahmadmauluddin predictionofemergencydepartmentrevisitsamongchildandyouthmentalhealthoutpatientsusingdeeplearningtechniques AT lauraduncan predictionofemergencydepartmentrevisitsamongchildandyouthmentalhealthoutpatientsusingdeeplearningtechniques |