Assessing Detection of Children With Suicide-Related Emergencies: Evaluation and Development of Computable Phenotyping Approaches

BackgroundAlthough suicide is a leading cause of death among children, the optimal approach for using health care data sets to detect suicide-related emergencies among children is not known. ObjectiveThis study aimed to assess the performance of suicide-related In...

Full description

Bibliographic Details
Main Authors: Juliet Beni Edgcomb, Chi-hong Tseng, Mengtong Pan, Alexandra Klomhaus, Bonnie T Zima
Format: Article
Language:English
Published: JMIR Publications 2023-07-01
Series:JMIR Mental Health
Online Access:https://mental.jmir.org/2023/1/e47084
_version_ 1827857201784422400
author Juliet Beni Edgcomb
Chi-hong Tseng
Mengtong Pan
Alexandra Klomhaus
Bonnie T Zima
author_facet Juliet Beni Edgcomb
Chi-hong Tseng
Mengtong Pan
Alexandra Klomhaus
Bonnie T Zima
author_sort Juliet Beni Edgcomb
collection DOAJ
description BackgroundAlthough suicide is a leading cause of death among children, the optimal approach for using health care data sets to detect suicide-related emergencies among children is not known. ObjectiveThis study aimed to assess the performance of suicide-related International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes and suicide-related chief complaint in detecting self-injurious thoughts and behaviors (SITB) among children compared with clinician chart review. The study also aimed to examine variations in performance by child sociodemographics and type of self-injury, as well as develop machine learning models trained on codified health record data (features) and clinician chart review (gold standard) and test model detection performance. MethodsA gold standard classification of suicide-related emergencies was determined through clinician manual review of clinical notes from 600 emergency department visits between 2015 and 2019 by children aged 10 to 17 years. Visits classified with nonfatal suicide attempt or intentional self-harm using the Centers for Disease Control and Prevention surveillance case definition list of ICD-10-CM codes and suicide-related chief complaint were compared with the gold standard classification. Machine learning classifiers (least absolute shrinkage and selection operator–penalized logistic regression and random forest) were then trained and tested using codified health record data (eg, child sociodemographics, medications, disposition, and laboratory testing) and the gold standard classification. The accuracy, sensitivity, and specificity of each detection approach and relative importance of features were examined. ResultsSITB accounted for 47.3% (284/600) of the visits. Suicide-related diagnostic codes missed nearly one-third (82/284, 28.9%) and suicide-related chief complaints missed more than half (153/284, 53.9%) of the children presenting to emergency departments with SITB. Sensitivity was significantly lower for male children than for female children (0.69, 95% CI 0.61-0.77 vs 0.84, 95% CI 0.78-0.90, respectively) and for preteens compared with adolescents (0.66, 95% CI 0.54-0.78 vs 0.86, 95% CI 0.80-0.92, respectively). Specificity was significantly lower for detecting preparatory acts (0.68, 95% CI 0.64-0.72) and attempts (0.67, 95% CI 0.63-0.71) than for detecting ideation (0.79, 95% CI 0.75-0.82). Machine learning–based models significantly improved the sensitivity of detection compared with suicide-related codes and chief complaint alone. Models considering all 84 features performed similarly to models considering only mental health–related ICD-10-CM codes and chief complaints (34 features) and models considering non–ICD-10-CM code indicators and mental health–related chief complaints (53 features). ConclusionsThe capacity to detect children with SITB may be strengthened by applying a machine learning–based approach to codified health record data. To improve integration between clinical research informatics and child mental health care, future research is needed to evaluate the potential benefits of implementing detection approaches at the point of care and identifying precise targets for suicide prevention interventions in children.
first_indexed 2024-03-12T12:37:01Z
format Article
id doaj.art-1abf1ad06d614979b33eae7cf12d353a
institution Directory Open Access Journal
issn 2368-7959
language English
last_indexed 2024-03-12T12:37:01Z
publishDate 2023-07-01
publisher JMIR Publications
record_format Article
series JMIR Mental Health
spelling doaj.art-1abf1ad06d614979b33eae7cf12d353a2023-08-29T00:06:03ZengJMIR PublicationsJMIR Mental Health2368-79592023-07-0110e4708410.2196/47084Assessing Detection of Children With Suicide-Related Emergencies: Evaluation and Development of Computable Phenotyping ApproachesJuliet Beni Edgcombhttps://orcid.org/0000-0001-5530-8262Chi-hong Tsenghttps://orcid.org/0000-0002-2549-6167Mengtong Panhttps://orcid.org/0009-0009-8700-3434Alexandra Klomhaushttps://orcid.org/0000-0002-7676-3595Bonnie T Zimahttps://orcid.org/0000-0002-5542-1879 BackgroundAlthough suicide is a leading cause of death among children, the optimal approach for using health care data sets to detect suicide-related emergencies among children is not known. ObjectiveThis study aimed to assess the performance of suicide-related International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes and suicide-related chief complaint in detecting self-injurious thoughts and behaviors (SITB) among children compared with clinician chart review. The study also aimed to examine variations in performance by child sociodemographics and type of self-injury, as well as develop machine learning models trained on codified health record data (features) and clinician chart review (gold standard) and test model detection performance. MethodsA gold standard classification of suicide-related emergencies was determined through clinician manual review of clinical notes from 600 emergency department visits between 2015 and 2019 by children aged 10 to 17 years. Visits classified with nonfatal suicide attempt or intentional self-harm using the Centers for Disease Control and Prevention surveillance case definition list of ICD-10-CM codes and suicide-related chief complaint were compared with the gold standard classification. Machine learning classifiers (least absolute shrinkage and selection operator–penalized logistic regression and random forest) were then trained and tested using codified health record data (eg, child sociodemographics, medications, disposition, and laboratory testing) and the gold standard classification. The accuracy, sensitivity, and specificity of each detection approach and relative importance of features were examined. ResultsSITB accounted for 47.3% (284/600) of the visits. Suicide-related diagnostic codes missed nearly one-third (82/284, 28.9%) and suicide-related chief complaints missed more than half (153/284, 53.9%) of the children presenting to emergency departments with SITB. Sensitivity was significantly lower for male children than for female children (0.69, 95% CI 0.61-0.77 vs 0.84, 95% CI 0.78-0.90, respectively) and for preteens compared with adolescents (0.66, 95% CI 0.54-0.78 vs 0.86, 95% CI 0.80-0.92, respectively). Specificity was significantly lower for detecting preparatory acts (0.68, 95% CI 0.64-0.72) and attempts (0.67, 95% CI 0.63-0.71) than for detecting ideation (0.79, 95% CI 0.75-0.82). Machine learning–based models significantly improved the sensitivity of detection compared with suicide-related codes and chief complaint alone. Models considering all 84 features performed similarly to models considering only mental health–related ICD-10-CM codes and chief complaints (34 features) and models considering non–ICD-10-CM code indicators and mental health–related chief complaints (53 features). ConclusionsThe capacity to detect children with SITB may be strengthened by applying a machine learning–based approach to codified health record data. To improve integration between clinical research informatics and child mental health care, future research is needed to evaluate the potential benefits of implementing detection approaches at the point of care and identifying precise targets for suicide prevention interventions in children.https://mental.jmir.org/2023/1/e47084
spellingShingle Juliet Beni Edgcomb
Chi-hong Tseng
Mengtong Pan
Alexandra Klomhaus
Bonnie T Zima
Assessing Detection of Children With Suicide-Related Emergencies: Evaluation and Development of Computable Phenotyping Approaches
JMIR Mental Health
title Assessing Detection of Children With Suicide-Related Emergencies: Evaluation and Development of Computable Phenotyping Approaches
title_full Assessing Detection of Children With Suicide-Related Emergencies: Evaluation and Development of Computable Phenotyping Approaches
title_fullStr Assessing Detection of Children With Suicide-Related Emergencies: Evaluation and Development of Computable Phenotyping Approaches
title_full_unstemmed Assessing Detection of Children With Suicide-Related Emergencies: Evaluation and Development of Computable Phenotyping Approaches
title_short Assessing Detection of Children With Suicide-Related Emergencies: Evaluation and Development of Computable Phenotyping Approaches
title_sort assessing detection of children with suicide related emergencies evaluation and development of computable phenotyping approaches
url https://mental.jmir.org/2023/1/e47084
work_keys_str_mv AT julietbeniedgcomb assessingdetectionofchildrenwithsuiciderelatedemergenciesevaluationanddevelopmentofcomputablephenotypingapproaches
AT chihongtseng assessingdetectionofchildrenwithsuiciderelatedemergenciesevaluationanddevelopmentofcomputablephenotypingapproaches
AT mengtongpan assessingdetectionofchildrenwithsuiciderelatedemergenciesevaluationanddevelopmentofcomputablephenotypingapproaches
AT alexandraklomhaus assessingdetectionofchildrenwithsuiciderelatedemergenciesevaluationanddevelopmentofcomputablephenotypingapproaches
AT bonnietzima assessingdetectionofchildrenwithsuiciderelatedemergenciesevaluationanddevelopmentofcomputablephenotypingapproaches