Measuring case severity: a novel tool for benchmarking and clinical documentation improvement
Abstract Background Severity of illness (SOI) is an All Patients Refined Diagnosis Related Groups (APR DRG) modifier based on comorbidity capture. Tracking SOI helps hospitals improve performance and resource distribution. Furthermore, benchmarking SOI plays a key role in Quality Improvement (QI) ef...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-04-01
|
| Series: | BMC Health Services Research |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12913-022-07935-1 |
| _version_ | 1818274496883195904 |
|---|---|
| author | Jie Xiang Paul W. Durance Louisa C. Griffes Yalei Chen Rishi R. Bakshi |
| author_facet | Jie Xiang Paul W. Durance Louisa C. Griffes Yalei Chen Rishi R. Bakshi |
| author_sort | Jie Xiang |
| collection | DOAJ |
| description | Abstract Background Severity of illness (SOI) is an All Patients Refined Diagnosis Related Groups (APR DRG) modifier based on comorbidity capture. Tracking SOI helps hospitals improve performance and resource distribution. Furthermore, benchmarking SOI plays a key role in Quality Improvement (QI) efforts such as Clinical Documentation Improvement (CDI) programs. The current SOI system highly relies on the 3 M APR DRG grouper that is updated annually, making it difficult to track severity longitudinally and benchmark against hospitals with different patient populations. Here, we describe an alternative SOI scoring system that is grouper-independent and that can be tracked longitudinally. Methods Admission data for 2019–2020 U.S. News and World Report Honor Roll facilities were downloaded from the Vizient Clinical Database and split into training and testing datasets. Elixhauser comorbidities, body systems developed from the Healthcare Cost and Utilization Project (HCUP), and ICD-10-CM complication and comorbidity (CC/MCC) indicators were selected as the predictors for orthogonal polynomial regression models to predict patients’ admission and discharge SOI. Receiver operating characteristic (ROC) and Precision-Recall (PR) analysis, and prediction accuracy were used to evaluate model performance. Results In the training dataset, the full model including both Elixhauser comorbidities and body system CC/MCC indicators had the highest ROC AUC, PR AUC and predication accuracy for both admission (ROC AUC: 92.9%; PR AUC: 91.0%; prediction accuracy: 85.4%) and discharge SOI (ROC AUC: 93.6%; PR AUC: 92.8%; prediction accuracy: 86.2%). The model including only body system CC/MCC indicators had similar performance for admission (ROC AUC: 92.4%; PR AUC: 90.4%; prediction accuracy: 84.8%) and discharge SOI (ROC AUC: 93.1%; PR AUC: 92.2%; prediction accuracy: 85.6%) as the full model. The model including only Elixhauser comorbidities exhibited the lowest performance. Similarly, in the validation dataset, the prediction accuracy was 86.2% for the full model, 85.6% for the body system model, and 79.3% for the comorbidity model. With fewer variables and less model complexity, the body system model was more efficient and was determined to be the optimal model. The probabilities generated from this model, named J_Score and J_Score_POA, successfully measured SOI and had practical applications in assessment of CDI performance. Conclusions The J_Scores generated from the body system model have significant value in evaluating admission and discharge severity of illness. We believe that this new scoring system will provide a useful tool for healthcare institutions to benchmark patients’ illness severity and augment Quality Improvement (QI) efforts. |
| first_indexed | 2024-12-12T22:14:47Z |
| format | Article |
| id | doaj.art-b7fd6a69c7094babb920a40eeb485673 |
| institution | Directory Open Access Journal |
| issn | 1472-6963 |
| language | English |
| last_indexed | 2024-12-12T22:14:47Z |
| publishDate | 2022-04-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Health Services Research |
| spelling | doaj.art-b7fd6a69c7094babb920a40eeb4856732022-12-22T00:10:06ZengBMCBMC Health Services Research1472-69632022-04-012211910.1186/s12913-022-07935-1Measuring case severity: a novel tool for benchmarking and clinical documentation improvementJie Xiang0Paul W. Durance1Louisa C. Griffes2Yalei Chen3Rishi R. Bakshi4Revenue Cycle Department, Michigan Medicine, University of Michigan HealthFinancial Services Department, Michigan Medicine, University of Michigan HealthQuality Analytics, Quality Department, Michigan Medicine, University of Michigan HealthDepartment of Public Health Sciences, Henry Ford Health SystemDepartments of Physical Medicine & Rehabilitation and Revenue Cycle, Michigan Medicine, University of Michigan HealthAbstract Background Severity of illness (SOI) is an All Patients Refined Diagnosis Related Groups (APR DRG) modifier based on comorbidity capture. Tracking SOI helps hospitals improve performance and resource distribution. Furthermore, benchmarking SOI plays a key role in Quality Improvement (QI) efforts such as Clinical Documentation Improvement (CDI) programs. The current SOI system highly relies on the 3 M APR DRG grouper that is updated annually, making it difficult to track severity longitudinally and benchmark against hospitals with different patient populations. Here, we describe an alternative SOI scoring system that is grouper-independent and that can be tracked longitudinally. Methods Admission data for 2019–2020 U.S. News and World Report Honor Roll facilities were downloaded from the Vizient Clinical Database and split into training and testing datasets. Elixhauser comorbidities, body systems developed from the Healthcare Cost and Utilization Project (HCUP), and ICD-10-CM complication and comorbidity (CC/MCC) indicators were selected as the predictors for orthogonal polynomial regression models to predict patients’ admission and discharge SOI. Receiver operating characteristic (ROC) and Precision-Recall (PR) analysis, and prediction accuracy were used to evaluate model performance. Results In the training dataset, the full model including both Elixhauser comorbidities and body system CC/MCC indicators had the highest ROC AUC, PR AUC and predication accuracy for both admission (ROC AUC: 92.9%; PR AUC: 91.0%; prediction accuracy: 85.4%) and discharge SOI (ROC AUC: 93.6%; PR AUC: 92.8%; prediction accuracy: 86.2%). The model including only body system CC/MCC indicators had similar performance for admission (ROC AUC: 92.4%; PR AUC: 90.4%; prediction accuracy: 84.8%) and discharge SOI (ROC AUC: 93.1%; PR AUC: 92.2%; prediction accuracy: 85.6%) as the full model. The model including only Elixhauser comorbidities exhibited the lowest performance. Similarly, in the validation dataset, the prediction accuracy was 86.2% for the full model, 85.6% for the body system model, and 79.3% for the comorbidity model. With fewer variables and less model complexity, the body system model was more efficient and was determined to be the optimal model. The probabilities generated from this model, named J_Score and J_Score_POA, successfully measured SOI and had practical applications in assessment of CDI performance. Conclusions The J_Scores generated from the body system model have significant value in evaluating admission and discharge severity of illness. We believe that this new scoring system will provide a useful tool for healthcare institutions to benchmark patients’ illness severity and augment Quality Improvement (QI) efforts.https://doi.org/10.1186/s12913-022-07935-1Severity of IllnessAPR DRGBody SystemsElixhauser ComorbiditiesJ ScoresBenchmark |
| spellingShingle | Jie Xiang Paul W. Durance Louisa C. Griffes Yalei Chen Rishi R. Bakshi Measuring case severity: a novel tool for benchmarking and clinical documentation improvement BMC Health Services Research Severity of Illness APR DRG Body Systems Elixhauser Comorbidities J Scores Benchmark |
| title | Measuring case severity: a novel tool for benchmarking and clinical documentation improvement |
| title_full | Measuring case severity: a novel tool for benchmarking and clinical documentation improvement |
| title_fullStr | Measuring case severity: a novel tool for benchmarking and clinical documentation improvement |
| title_full_unstemmed | Measuring case severity: a novel tool for benchmarking and clinical documentation improvement |
| title_short | Measuring case severity: a novel tool for benchmarking and clinical documentation improvement |
| title_sort | measuring case severity a novel tool for benchmarking and clinical documentation improvement |
| topic | Severity of Illness APR DRG Body Systems Elixhauser Comorbidities J Scores Benchmark |
| url | https://doi.org/10.1186/s12913-022-07935-1 |
| work_keys_str_mv | AT jiexiang measuringcaseseverityanoveltoolforbenchmarkingandclinicaldocumentationimprovement AT paulwdurance measuringcaseseverityanoveltoolforbenchmarkingandclinicaldocumentationimprovement AT louisacgriffes measuringcaseseverityanoveltoolforbenchmarkingandclinicaldocumentationimprovement AT yaleichen measuringcaseseverityanoveltoolforbenchmarkingandclinicaldocumentationimprovement AT rishirbakshi measuringcaseseverityanoveltoolforbenchmarkingandclinicaldocumentationimprovement |