Interrater agreement of multi-professional case review as reference standard for specialist palliative care need: a mixed-methods study
Abstract Background A wide variety of screening tools for the need for specialist palliative care (SPC) have been proposed for the use in oncology. However, as there is no established reference standard for SPC need to compare their results with, their sensitivity and specificity have not yet been d...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-11-01
|
| Series: | BMC Palliative Care |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12904-023-01281-7 |
| _version_ | 1797555754845077504 |
|---|---|
| author | Evelyn Müller Michael Josef Müller Katharina Seibel Christopher Boehlke Henning Schäfer Carsten Klein Maria Heckel Steffen T. Simon Gerhild Becker |
| author_facet | Evelyn Müller Michael Josef Müller Katharina Seibel Christopher Boehlke Henning Schäfer Carsten Klein Maria Heckel Steffen T. Simon Gerhild Becker |
| author_sort | Evelyn Müller |
| collection | DOAJ |
| description | Abstract Background A wide variety of screening tools for the need for specialist palliative care (SPC) have been proposed for the use in oncology. However, as there is no established reference standard for SPC need to compare their results with, their sensitivity and specificity have not yet been determined. The aim of the study was to explore whether SPC need assessment by means of multi-professional case review has sufficient interrater agreement to be employed as a reference standard. Methods Comprehensive case descriptions were prepared for 20 inpatients with advanced oncologic disease at the University Hospital Freiburg (Germany). All cases were presented to the palliative care teams of three different hospitals in independent, multi-professional case review sessions. The teams assessed whether patients had support needs in nine categories and subsequently concluded SPC need (yes / no). Interrater agreement regarding SPC need was determined by calculating Fleiss’ Kappa. Results In 17 out of 20 cases the three teams agreed regarding their appraisal of SPC need (substantial interrater agreement: Fleiss’ Kappa κ = 0.80 (95% CI: 0.55–1.0; p < 0.001)). The number of support needs was significantly lower for patients who all teams agreed had no SPC need than for those with agreed SPC need. Conclusions The proposed expert case review process shows sufficient reliability to be used as a reference standard. Key elements of the case review process (e.g. clear definition of SPC need, standardized review of the patients’ support needs) and possible modifications to simplify the process are discussed. Trial registration German Clinical Trials Register, DRKS00021686, registered 17.12.2020. |
| first_indexed | 2024-03-10T16:52:06Z |
| format | Article |
| id | doaj.art-a717bb13864b4813801a7e89ae87810e |
| institution | Directory Open Access Journal |
| issn | 1472-684X |
| language | English |
| last_indexed | 2024-03-10T16:52:06Z |
| publishDate | 2023-11-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Palliative Care |
| spelling | doaj.art-a717bb13864b4813801a7e89ae87810e2023-11-20T11:17:27ZengBMCBMC Palliative Care1472-684X2023-11-012211910.1186/s12904-023-01281-7Interrater agreement of multi-professional case review as reference standard for specialist palliative care need: a mixed-methods studyEvelyn Müller0Michael Josef Müller1Katharina Seibel2Christopher Boehlke3Henning Schäfer4Carsten Klein5Maria Heckel6Steffen T. Simon7Gerhild Becker8Department of Palliative Medicine, Medical Center – University of Freiburg, Faculty of Medicine, University of FreiburgDepartment of Palliative Medicine, Medical Center – University of Freiburg, Faculty of Medicine, University of FreiburgDepartment of Palliative Medicine, Medical Center – University of Freiburg, Faculty of Medicine, University of FreiburgDepartment of Palliative Care, University Hospital BaselDepartment of Radiation Oncology, Medical Center, Faculty of Medicine, University of Freiburg, German Cancer Consortium (DKTK) Partner Site Freiburg, German Cancer Research Center (DKFZ), HeidelbergDepartment of Palliative Medicine, University Hospital Erlangen-EMN, Comprehensive Cancer Center CCC Erlangen, Friedrich-Alexander-University Erlangen-NürnbergDepartment of Palliative Medicine, University Hospital Erlangen-EMN, Comprehensive Cancer Center CCC Erlangen, Friedrich-Alexander-University Erlangen-NürnbergDepartment of Palliative Medicine and Center for Integrated Oncology Aachen Bonn Cologne Dusseldorf (CIO ABCD), University Hospital of Cologne, Faculty of Medicine and University HospitalDepartment of Palliative Medicine, Medical Center – University of Freiburg, Faculty of Medicine, University of FreiburgAbstract Background A wide variety of screening tools for the need for specialist palliative care (SPC) have been proposed for the use in oncology. However, as there is no established reference standard for SPC need to compare their results with, their sensitivity and specificity have not yet been determined. The aim of the study was to explore whether SPC need assessment by means of multi-professional case review has sufficient interrater agreement to be employed as a reference standard. Methods Comprehensive case descriptions were prepared for 20 inpatients with advanced oncologic disease at the University Hospital Freiburg (Germany). All cases were presented to the palliative care teams of three different hospitals in independent, multi-professional case review sessions. The teams assessed whether patients had support needs in nine categories and subsequently concluded SPC need (yes / no). Interrater agreement regarding SPC need was determined by calculating Fleiss’ Kappa. Results In 17 out of 20 cases the three teams agreed regarding their appraisal of SPC need (substantial interrater agreement: Fleiss’ Kappa κ = 0.80 (95% CI: 0.55–1.0; p < 0.001)). The number of support needs was significantly lower for patients who all teams agreed had no SPC need than for those with agreed SPC need. Conclusions The proposed expert case review process shows sufficient reliability to be used as a reference standard. Key elements of the case review process (e.g. clear definition of SPC need, standardized review of the patients’ support needs) and possible modifications to simplify the process are discussed. Trial registration German Clinical Trials Register, DRKS00021686, registered 17.12.2020.https://doi.org/10.1186/s12904-023-01281-7Referral and consultationPalliative careNeoplasmsPsychometrics |
| spellingShingle | Evelyn Müller Michael Josef Müller Katharina Seibel Christopher Boehlke Henning Schäfer Carsten Klein Maria Heckel Steffen T. Simon Gerhild Becker Interrater agreement of multi-professional case review as reference standard for specialist palliative care need: a mixed-methods study BMC Palliative Care Referral and consultation Palliative care Neoplasms Psychometrics |
| title | Interrater agreement of multi-professional case review as reference standard for specialist palliative care need: a mixed-methods study |
| title_full | Interrater agreement of multi-professional case review as reference standard for specialist palliative care need: a mixed-methods study |
| title_fullStr | Interrater agreement of multi-professional case review as reference standard for specialist palliative care need: a mixed-methods study |
| title_full_unstemmed | Interrater agreement of multi-professional case review as reference standard for specialist palliative care need: a mixed-methods study |
| title_short | Interrater agreement of multi-professional case review as reference standard for specialist palliative care need: a mixed-methods study |
| title_sort | interrater agreement of multi professional case review as reference standard for specialist palliative care need a mixed methods study |
| topic | Referral and consultation Palliative care Neoplasms Psychometrics |
| url | https://doi.org/10.1186/s12904-023-01281-7 |
| work_keys_str_mv | AT evelynmuller interrateragreementofmultiprofessionalcasereviewasreferencestandardforspecialistpalliativecareneedamixedmethodsstudy AT michaeljosefmuller interrateragreementofmultiprofessionalcasereviewasreferencestandardforspecialistpalliativecareneedamixedmethodsstudy AT katharinaseibel interrateragreementofmultiprofessionalcasereviewasreferencestandardforspecialistpalliativecareneedamixedmethodsstudy AT christopherboehlke interrateragreementofmultiprofessionalcasereviewasreferencestandardforspecialistpalliativecareneedamixedmethodsstudy AT henningschafer interrateragreementofmultiprofessionalcasereviewasreferencestandardforspecialistpalliativecareneedamixedmethodsstudy AT carstenklein interrateragreementofmultiprofessionalcasereviewasreferencestandardforspecialistpalliativecareneedamixedmethodsstudy AT mariaheckel interrateragreementofmultiprofessionalcasereviewasreferencestandardforspecialistpalliativecareneedamixedmethodsstudy AT steffentsimon interrateragreementofmultiprofessionalcasereviewasreferencestandardforspecialistpalliativecareneedamixedmethodsstudy AT gerhildbecker interrateragreementofmultiprofessionalcasereviewasreferencestandardforspecialistpalliativecareneedamixedmethodsstudy |