The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery
Abstract Background In patients who underwent colorectal surgery, an existing semi-automated surveillance algorithm based on structured data achieves high sensitivity in detecting deep surgical site infections (SSI), however, generates a significant number of false positives. The inclusion of unstru...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-10-01
|
| Series: | Antimicrobial Resistance and Infection Control |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13756-023-01316-x |
| _version_ | 1827780990871797760 |
|---|---|
| author | Janneke D.M. Verberk Suzanne D. van der Werff Rebecka Weegar Aron Henriksson Milan C. Richir Christian Buchli Maaike S.M. van Mourik Pontus Nauclér |
| author_facet | Janneke D.M. Verberk Suzanne D. van der Werff Rebecka Weegar Aron Henriksson Milan C. Richir Christian Buchli Maaike S.M. van Mourik Pontus Nauclér |
| author_sort | Janneke D.M. Verberk |
| collection | DOAJ |
| description | Abstract Background In patients who underwent colorectal surgery, an existing semi-automated surveillance algorithm based on structured data achieves high sensitivity in detecting deep surgical site infections (SSI), however, generates a significant number of false positives. The inclusion of unstructured, clinical narratives to the algorithm may decrease the number of patients requiring manual chart review. The aim of this study was to investigate the performance of this semi-automated surveillance algorithm augmented with a natural language processing (NLP) component to improve positive predictive value (PPV) and thus workload reduction (WR). Methods Retrospective, observational cohort study in patients who underwent colorectal surgery from January 1, 2015, through September 30, 2020. NLP was used to detect keyword counts in clinical notes. Several NLP-algorithms were developed with different count input types and classifiers, and added as component to the original semi-automated algorithm. Traditional manual surveillance was compared with the NLP-augmented surveillance algorithms and sensitivity, specificity, PPV and WR were calculated. Results From the NLP-augmented models, the decision tree models with discretized counts or binary counts had the best performance (sensitivity 95.1% (95%CI 83.5–99.4%), WR 60.9%) and improved PPV and WR by only 2.6% and 3.6%, respectively, compared to the original algorithm. Conclusions The addition of an NLP component to the existing algorithm had modest effect on WR (decrease of 1.4–12.5%), at the cost of sensitivity. For future implementation it will be a trade-off between optimal case-finding techniques versus practical considerations such as acceptability and availability of resources. |
| first_indexed | 2024-03-11T15:12:36Z |
| format | Article |
| id | doaj.art-3f24520fa77848019be6bba88c353d7d |
| institution | Directory Open Access Journal |
| issn | 2047-2994 |
| language | English |
| last_indexed | 2024-03-11T15:12:36Z |
| publishDate | 2023-10-01 |
| publisher | BMC |
| record_format | Article |
| series | Antimicrobial Resistance and Infection Control |
| spelling | doaj.art-3f24520fa77848019be6bba88c353d7d2023-10-29T12:36:56ZengBMCAntimicrobial Resistance and Infection Control2047-29942023-10-0112111010.1186/s13756-023-01316-xThe augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgeryJanneke D.M. Verberk0Suzanne D. van der Werff1Rebecka Weegar2Aron Henriksson3Milan C. Richir4Christian Buchli5Maaike S.M. van Mourik6Pontus Nauclér7Department of Medical Microbiology and Infection Prevention, University Medical Centre UtrechtDepartment of Medicine Solna, Division of Infectious Diseases, Karolinska InstitutetDepartment of Computer and Systems Sciences, Stockholm UniversityDepartment of Computer and Systems Sciences, Stockholm UniversityDepartment of Surgery, Cancer Centre, University Medical Centre UtrechtDepartment of Molecular Medicine and Surgery, Karolinska InstitutetDepartment of Medical Microbiology and Infection Prevention, University Medical Centre UtrechtDepartment of Medicine Solna, Division of Infectious Diseases, Karolinska InstitutetAbstract Background In patients who underwent colorectal surgery, an existing semi-automated surveillance algorithm based on structured data achieves high sensitivity in detecting deep surgical site infections (SSI), however, generates a significant number of false positives. The inclusion of unstructured, clinical narratives to the algorithm may decrease the number of patients requiring manual chart review. The aim of this study was to investigate the performance of this semi-automated surveillance algorithm augmented with a natural language processing (NLP) component to improve positive predictive value (PPV) and thus workload reduction (WR). Methods Retrospective, observational cohort study in patients who underwent colorectal surgery from January 1, 2015, through September 30, 2020. NLP was used to detect keyword counts in clinical notes. Several NLP-algorithms were developed with different count input types and classifiers, and added as component to the original semi-automated algorithm. Traditional manual surveillance was compared with the NLP-augmented surveillance algorithms and sensitivity, specificity, PPV and WR were calculated. Results From the NLP-augmented models, the decision tree models with discretized counts or binary counts had the best performance (sensitivity 95.1% (95%CI 83.5–99.4%), WR 60.9%) and improved PPV and WR by only 2.6% and 3.6%, respectively, compared to the original algorithm. Conclusions The addition of an NLP component to the existing algorithm had modest effect on WR (decrease of 1.4–12.5%), at the cost of sensitivity. For future implementation it will be a trade-off between optimal case-finding techniques versus practical considerations such as acceptability and availability of resources.https://doi.org/10.1186/s13756-023-01316-xAutomated surveillanceAlgorithmColorectal surgeryHealthcare-associated infectionsNatural language processingSurgical site infections |
| spellingShingle | Janneke D.M. Verberk Suzanne D. van der Werff Rebecka Weegar Aron Henriksson Milan C. Richir Christian Buchli Maaike S.M. van Mourik Pontus Nauclér The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery Antimicrobial Resistance and Infection Control Automated surveillance Algorithm Colorectal surgery Healthcare-associated infections Natural language processing Surgical site infections |
| title | The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery |
| title_full | The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery |
| title_fullStr | The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery |
| title_full_unstemmed | The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery |
| title_short | The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery |
| title_sort | augmented value of using clinical notes in semi automated surveillance of deep surgical site infections after colorectal surgery |
| topic | Automated surveillance Algorithm Colorectal surgery Healthcare-associated infections Natural language processing Surgical site infections |
| url | https://doi.org/10.1186/s13756-023-01316-x |
| work_keys_str_mv | AT jannekedmverberk theaugmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT suzannedvanderwerff theaugmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT rebeckaweegar theaugmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT aronhenriksson theaugmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT milancrichir theaugmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT christianbuchli theaugmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT maaikesmvanmourik theaugmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT pontusnaucler theaugmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT jannekedmverberk augmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT suzannedvanderwerff augmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT rebeckaweegar augmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT aronhenriksson augmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT milancrichir augmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT christianbuchli augmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT maaikesmvanmourik augmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery AT pontusnaucler augmentedvalueofusingclinicalnotesinsemiautomatedsurveillanceofdeepsurgicalsiteinfectionsaftercolorectalsurgery |