Quickly identifying people at risk of opioid use disorder in emergency departments: trade-offs between a machine learning approach and a simple EHR flag strategy
Objectives Emergency departments (EDs) are an important point of contact for people with opioid use disorder (OUD). Universal screening for OUD is costly and often infeasible. Evidence on effective, selective screening is needed. We assessed the feasibility of using a risk factor-based machine learn...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2022-09-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/12/9/e059414.full |
| _version_ | 1798021168129638400 |
|---|---|
| author | Izabela E Annis Robyn Jordan Kathleen C Thomas |
| author_facet | Izabela E Annis Robyn Jordan Kathleen C Thomas |
| author_sort | Izabela E Annis |
| collection | DOAJ |
| description | Objectives Emergency departments (EDs) are an important point of contact for people with opioid use disorder (OUD). Universal screening for OUD is costly and often infeasible. Evidence on effective, selective screening is needed. We assessed the feasibility of using a risk factor-based machine learning model to identify OUD quickly among patients presenting in EDs.Design/settings/participants In this cohort study, all ED visits between January 2016 and March 2018 for patients aged 12 years and older were identified from electronic health records (EHRs) data from a large university health system. First, logistic regression modelling was used to describe and elucidate the associations between patient demographic and clinical characteristics and diagnosis of OUD. Second, a Gradient Boosting Classifier was applied to develop a predictive model to identify patients at risk of OUD. The predictive performance of the Gradient Boosting algorithm was assessed using F1 scores and area under the curve (AUC).Outcome The primary outcome was the diagnosis of OUD.Results Among 345 728 patient ED visits (mean (SD) patient age, 49.4 (21.0) years; 210 045 (60.8%) female), 1.16% had a diagnosis of OUD. Bivariate analyses indicated that history of OUD was the strongest predictor of current OUD (OR=13.4, CI: 11.8 to 15.1). When history of OUD was excluded in multivariate models, baseline use of medications for OUD (OR=3.4, CI: 2.9 to 4.0) and white race (OR=2.9, CI: 2.6 to 3.3) were the strongest predictors. The best Gradient Boosting model achieved an AUC of 0.71, accuracy of 0.96 but only 0.45 sensitivity.Conclusions Patients who present at the ED with OUD are high-need patients who are typically smokers with psychiatric, chronic pain and substance use disorders. A machine learning model did not improve predictive ability. A quick review of a patient’s EHR for history of OUD is an efficient strategy to identify those who are currently at greatest risk of OUD. |
| first_indexed | 2024-04-11T17:09:25Z |
| format | Article |
| id | doaj.art-ac2fa22512a4443685a7645fce4cb1d0 |
| institution | Directory Open Access Journal |
| issn | 2044-6055 |
| language | English |
| last_indexed | 2024-04-11T17:09:25Z |
| publishDate | 2022-09-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj.art-ac2fa22512a4443685a7645fce4cb1d02022-12-22T04:12:56ZengBMJ Publishing GroupBMJ Open2044-60552022-09-0112910.1136/bmjopen-2021-059414Quickly identifying people at risk of opioid use disorder in emergency departments: trade-offs between a machine learning approach and a simple EHR flag strategyIzabela E Annis0Robyn Jordan1Kathleen C Thomas2Division of Pharmaceutical Outcomes and Policy, University of North Carolina at Chapel Hill Eshelman School of Pharmacy, Chapel Hill, North Carolina, USADepartment of Psychiatry, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, North Carolina, USADivision of Pharmaceutical Outcomes and Policy, University of North Carolina at Chapel Hill Eshelman School of Pharmacy, Chapel Hill, North Carolina, USAObjectives Emergency departments (EDs) are an important point of contact for people with opioid use disorder (OUD). Universal screening for OUD is costly and often infeasible. Evidence on effective, selective screening is needed. We assessed the feasibility of using a risk factor-based machine learning model to identify OUD quickly among patients presenting in EDs.Design/settings/participants In this cohort study, all ED visits between January 2016 and March 2018 for patients aged 12 years and older were identified from electronic health records (EHRs) data from a large university health system. First, logistic regression modelling was used to describe and elucidate the associations between patient demographic and clinical characteristics and diagnosis of OUD. Second, a Gradient Boosting Classifier was applied to develop a predictive model to identify patients at risk of OUD. The predictive performance of the Gradient Boosting algorithm was assessed using F1 scores and area under the curve (AUC).Outcome The primary outcome was the diagnosis of OUD.Results Among 345 728 patient ED visits (mean (SD) patient age, 49.4 (21.0) years; 210 045 (60.8%) female), 1.16% had a diagnosis of OUD. Bivariate analyses indicated that history of OUD was the strongest predictor of current OUD (OR=13.4, CI: 11.8 to 15.1). When history of OUD was excluded in multivariate models, baseline use of medications for OUD (OR=3.4, CI: 2.9 to 4.0) and white race (OR=2.9, CI: 2.6 to 3.3) were the strongest predictors. The best Gradient Boosting model achieved an AUC of 0.71, accuracy of 0.96 but only 0.45 sensitivity.Conclusions Patients who present at the ED with OUD are high-need patients who are typically smokers with psychiatric, chronic pain and substance use disorders. A machine learning model did not improve predictive ability. A quick review of a patient’s EHR for history of OUD is an efficient strategy to identify those who are currently at greatest risk of OUD.https://bmjopen.bmj.com/content/12/9/e059414.full |
| spellingShingle | Izabela E Annis Robyn Jordan Kathleen C Thomas Quickly identifying people at risk of opioid use disorder in emergency departments: trade-offs between a machine learning approach and a simple EHR flag strategy BMJ Open |
| title | Quickly identifying people at risk of opioid use disorder in emergency departments: trade-offs between a machine learning approach and a simple EHR flag strategy |
| title_full | Quickly identifying people at risk of opioid use disorder in emergency departments: trade-offs between a machine learning approach and a simple EHR flag strategy |
| title_fullStr | Quickly identifying people at risk of opioid use disorder in emergency departments: trade-offs between a machine learning approach and a simple EHR flag strategy |
| title_full_unstemmed | Quickly identifying people at risk of opioid use disorder in emergency departments: trade-offs between a machine learning approach and a simple EHR flag strategy |
| title_short | Quickly identifying people at risk of opioid use disorder in emergency departments: trade-offs between a machine learning approach and a simple EHR flag strategy |
| title_sort | quickly identifying people at risk of opioid use disorder in emergency departments trade offs between a machine learning approach and a simple ehr flag strategy |
| url | https://bmjopen.bmj.com/content/12/9/e059414.full |
| work_keys_str_mv | AT izabelaeannis quicklyidentifyingpeopleatriskofopioidusedisorderinemergencydepartmentstradeoffsbetweenamachinelearningapproachandasimpleehrflagstrategy AT robynjordan quicklyidentifyingpeopleatriskofopioidusedisorderinemergencydepartmentstradeoffsbetweenamachinelearningapproachandasimpleehrflagstrategy AT kathleencthomas quicklyidentifyingpeopleatriskofopioidusedisorderinemergencydepartmentstradeoffsbetweenamachinelearningapproachandasimpleehrflagstrategy |