Negativity and Positivity in the ICU: Exploratory Development of Automated Sentiment Capture in the Electronic Health Record
OBJECTIVES:. To develop proof-of-concept algorithms using alternative approaches to capture provider sentiment in ICU notes. DESIGN:. Retrospective observational cohort study. SETTING:. The Multiparameter Intelligent Monitoring of Intensive Care III (MIMIC-III) and the University of California, San...
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wolters Kluwer
2023-10-01
|
| Series: | Critical Care Explorations |
| Online Access: | http://journals.lww.com/10.1097/CCE.0000000000000960 |
| _version_ | 1797671831508877312 |
|---|---|
| author | Chris J. Kennedy, PhD Catherine Chiu, MD Allyson Cook Chapman, MD Oksana Gologorskaya, MS Hassan Farhan, MD Mary Han, MD MacGregor Hodgson, MD Daniel Lazzareschi, MD Deepshikha Ashana, MD, MBA, MS Sei Lee, MD Alexander K. Smith, MD, MPH Edie Espejo, MA John Boscardin, PhD Romain Pirracchio, MD, PhD Julien Cobert, MD |
| author_facet | Chris J. Kennedy, PhD Catherine Chiu, MD Allyson Cook Chapman, MD Oksana Gologorskaya, MS Hassan Farhan, MD Mary Han, MD MacGregor Hodgson, MD Daniel Lazzareschi, MD Deepshikha Ashana, MD, MBA, MS Sei Lee, MD Alexander K. Smith, MD, MPH Edie Espejo, MA John Boscardin, PhD Romain Pirracchio, MD, PhD Julien Cobert, MD |
| author_sort | Chris J. Kennedy, PhD |
| collection | DOAJ |
| description | OBJECTIVES:. To develop proof-of-concept algorithms using alternative approaches to capture provider sentiment in ICU notes.
DESIGN:. Retrospective observational cohort study.
SETTING:. The Multiparameter Intelligent Monitoring of Intensive Care III (MIMIC-III) and the University of California, San Francisco (UCSF) deidentified notes databases.
PATIENTS:. Adult (≥18 yr old) patients admitted to the ICU.
MEASUREMENTS AND MAIN RESULTS:. We developed two sentiment models: 1) a keywords-based approach using a consensus-based clinical sentiment lexicon comprised of 72 positive and 103 negative phrases, including negations and 2) a Decoding-enhanced Bidirectional Encoder Representations from Transformers with disentangled attention-v3-based deep learning model (keywords-independent) trained on clinical sentiment labels. We applied the models to 198,944 notes across 52,997 ICU admissions in the MIMIC-III database. Analyses were replicated on an external sample of patients admitted to a UCSF ICU from 2018 to 2019. We also labeled sentiment in 1,493 note fragments and compared the predictive accuracy of our tools to three popular sentiment classifiers. Clinical sentiment terms were found in 99% of patient visits across 88% of notes. Our two sentiment tools were substantially more predictive (Spearman correlations of 0.62–0.84, p values < 0.00001) of labeled sentiment compared with general language algorithms (0.28–0.46).
CONCLUSION:. Our exploratory healthcare-specific sentiment models can more accurately detect positivity and negativity in clinical notes compared with general sentiment tools not designed for clinical usage. |
| first_indexed | 2024-03-11T21:21:12Z |
| format | Article |
| id | doaj.art-b36dd990d46e492bb90ed6f4a37be0b1 |
| institution | Directory Open Access Journal |
| issn | 2639-8028 |
| language | English |
| last_indexed | 2024-03-11T21:21:12Z |
| publishDate | 2023-10-01 |
| publisher | Wolters Kluwer |
| record_format | Article |
| series | Critical Care Explorations |
| spelling | doaj.art-b36dd990d46e492bb90ed6f4a37be0b12023-09-28T07:07:28ZengWolters KluwerCritical Care Explorations2639-80282023-10-01510e096010.1097/CCE.0000000000000960202310000-00001Negativity and Positivity in the ICU: Exploratory Development of Automated Sentiment Capture in the Electronic Health RecordChris J. Kennedy, PhD0Catherine Chiu, MD1Allyson Cook Chapman, MD2Oksana Gologorskaya, MS3Hassan Farhan, MD4Mary Han, MD5MacGregor Hodgson, MD6Daniel Lazzareschi, MD7Deepshikha Ashana, MD, MBA, MS8Sei Lee, MD9Alexander K. Smith, MD, MPH10Edie Espejo, MA11John Boscardin, PhD12Romain Pirracchio, MD, PhD13Julien Cobert, MD141 Department of Psychiatry, Harvard Medical School, Boston, MA.3 Department of Anesthesia and Perioperative Care, University of California San Francisco, San Francisco, CA.4 Critical Care and Palliative Medicine, Department of Internal Medicine, University of California San Francisco, San Francisco, CA.6 Bakar Computational Health Sciences Institute, University of California San Francisco, San Francisco, CA.7 Department of Anesthesiology, Perioperative and Pain Management, Stanford University, Stanford, CA.2 Center for Precision Psychiatry, Massachusetts General Hospital, Boston, MA.2 Center for Precision Psychiatry, Massachusetts General Hospital, Boston, MA.2 Center for Precision Psychiatry, Massachusetts General Hospital, Boston, MA.8 Division of Pulmonary, Allergy, and Critical Care Medicine, Duke University, Durham, NC.9 Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, CA.9 Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, CA.9 Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, CA.9 Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, CA.3 Department of Anesthesia and Perioperative Care, University of California San Francisco, San Francisco, CA.3 Department of Anesthesia and Perioperative Care, University of California San Francisco, San Francisco, CA.OBJECTIVES:. To develop proof-of-concept algorithms using alternative approaches to capture provider sentiment in ICU notes. DESIGN:. Retrospective observational cohort study. SETTING:. The Multiparameter Intelligent Monitoring of Intensive Care III (MIMIC-III) and the University of California, San Francisco (UCSF) deidentified notes databases. PATIENTS:. Adult (≥18 yr old) patients admitted to the ICU. MEASUREMENTS AND MAIN RESULTS:. We developed two sentiment models: 1) a keywords-based approach using a consensus-based clinical sentiment lexicon comprised of 72 positive and 103 negative phrases, including negations and 2) a Decoding-enhanced Bidirectional Encoder Representations from Transformers with disentangled attention-v3-based deep learning model (keywords-independent) trained on clinical sentiment labels. We applied the models to 198,944 notes across 52,997 ICU admissions in the MIMIC-III database. Analyses were replicated on an external sample of patients admitted to a UCSF ICU from 2018 to 2019. We also labeled sentiment in 1,493 note fragments and compared the predictive accuracy of our tools to three popular sentiment classifiers. Clinical sentiment terms were found in 99% of patient visits across 88% of notes. Our two sentiment tools were substantially more predictive (Spearman correlations of 0.62–0.84, p values < 0.00001) of labeled sentiment compared with general language algorithms (0.28–0.46). CONCLUSION:. Our exploratory healthcare-specific sentiment models can more accurately detect positivity and negativity in clinical notes compared with general sentiment tools not designed for clinical usage.http://journals.lww.com/10.1097/CCE.0000000000000960 |
| spellingShingle | Chris J. Kennedy, PhD Catherine Chiu, MD Allyson Cook Chapman, MD Oksana Gologorskaya, MS Hassan Farhan, MD Mary Han, MD MacGregor Hodgson, MD Daniel Lazzareschi, MD Deepshikha Ashana, MD, MBA, MS Sei Lee, MD Alexander K. Smith, MD, MPH Edie Espejo, MA John Boscardin, PhD Romain Pirracchio, MD, PhD Julien Cobert, MD Negativity and Positivity in the ICU: Exploratory Development of Automated Sentiment Capture in the Electronic Health Record Critical Care Explorations |
| title | Negativity and Positivity in the ICU: Exploratory Development of Automated Sentiment Capture in the Electronic Health Record |
| title_full | Negativity and Positivity in the ICU: Exploratory Development of Automated Sentiment Capture in the Electronic Health Record |
| title_fullStr | Negativity and Positivity in the ICU: Exploratory Development of Automated Sentiment Capture in the Electronic Health Record |
| title_full_unstemmed | Negativity and Positivity in the ICU: Exploratory Development of Automated Sentiment Capture in the Electronic Health Record |
| title_short | Negativity and Positivity in the ICU: Exploratory Development of Automated Sentiment Capture in the Electronic Health Record |
| title_sort | negativity and positivity in the icu exploratory development of automated sentiment capture in the electronic health record |
| url | http://journals.lww.com/10.1097/CCE.0000000000000960 |
| work_keys_str_mv | AT chrisjkennedyphd negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT catherinechiumd negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT allysoncookchapmanmd negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT oksanagologorskayams negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT hassanfarhanmd negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT maryhanmd negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT macgregorhodgsonmd negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT daniellazzareschimd negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT deepshikhaashanamdmbams negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT seileemd negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT alexanderksmithmdmph negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT edieespejoma negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT johnboscardinphd negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT romainpirracchiomdphd negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord AT juliencobertmd negativityandpositivityintheicuexploratorydevelopmentofautomatedsentimentcaptureintheelectronichealthrecord |