PRO-Angoff method for remote standard setting: establishing clinical thresholds for the upper digestive disease tool
Abstract Background The Upper Digestive Disease (UDD) Tool™ is used to monitor symptom frequency, intensity, and interference across nine symptom domains and includes two Patient-Reported Outcome Measurement Information System (PROMIS) domains assessing physical and mental health. This study aimed t...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2024-03-01
|
| Series: | Journal of Patient-Reported Outcomes |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s41687-024-00707-x |
| _version_ | 1827315972209377280 |
|---|---|
| author | Minji K. Lee Mohamad K Abou Chaar Shanda H Blackmon Kathleen J Yost |
| author_facet | Minji K. Lee Mohamad K Abou Chaar Shanda H Blackmon Kathleen J Yost |
| author_sort | Minji K. Lee |
| collection | DOAJ |
| description | Abstract Background The Upper Digestive Disease (UDD) Tool™ is used to monitor symptom frequency, intensity, and interference across nine symptom domains and includes two Patient-Reported Outcome Measurement Information System (PROMIS) domains assessing physical and mental health. This study aimed to establish cut scores for updated symptom domains through standard setting exercises and evaluate the effectiveness and acceptability of virtual standard setting. Methods The extended Angoff method was employed to determine cut scores. Subject matter experts refined performance descriptions for symptom control categories and achieved consensus. Domains were categorized into good, moderate, and poor symptom control. Two cut scores were established, differentiating good vs. moderate and moderate vs. poor. Panelists estimated average scores for 100 borderline patients per item. Cut scores were computed based on the sum of the average ratings for individual questions, converted to 0-100 scale. Results Performance descriptions were refined. Panelists discussed that interpretation of the scores should take into account the timing of symptoms after surgery and patient populations, and the importance of items asking symptom frequency, severity, and interference with daily life. The good/moderate cut scores ranged from 21.3 to 35.0 (mean 28.6, SD 3.6) across domains, and moderate/poor ranged from 47.5 to 71.3 (mean 54.5, SD 7.0). Conclusions Panelists were confident in the virtual standard setting process, expecting valid cut scores. Future studies can further validate the cut scores using patient perspectives and collect patient and physician preferences for displaying contextual items on patient- and physician-facing dashboard. |
| first_indexed | 2024-04-24T23:05:53Z |
| format | Article |
| id | doaj.art-6a5b5627b451459faec21837644ace48 |
| institution | Directory Open Access Journal |
| issn | 2509-8020 |
| language | English |
| last_indexed | 2024-04-24T23:05:53Z |
| publishDate | 2024-03-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Journal of Patient-Reported Outcomes |
| spelling | doaj.art-6a5b5627b451459faec21837644ace482024-03-17T12:27:52ZengSpringerOpenJournal of Patient-Reported Outcomes2509-80202024-03-018111010.1186/s41687-024-00707-xPRO-Angoff method for remote standard setting: establishing clinical thresholds for the upper digestive disease toolMinji K. Lee0Mohamad K Abou Chaar1Shanda H Blackmon2Kathleen J Yost3Department of Quantitative Health Sciences, Mayo ClinicDepartment of Surgery, Division of Thoracic Surgery, Mayo ClinicDepartment of Surgery, Division of Thoracic Surgery, Mayo ClinicDepartment of Quantitative Health Sciences, Mayo ClinicAbstract Background The Upper Digestive Disease (UDD) Tool™ is used to monitor symptom frequency, intensity, and interference across nine symptom domains and includes two Patient-Reported Outcome Measurement Information System (PROMIS) domains assessing physical and mental health. This study aimed to establish cut scores for updated symptom domains through standard setting exercises and evaluate the effectiveness and acceptability of virtual standard setting. Methods The extended Angoff method was employed to determine cut scores. Subject matter experts refined performance descriptions for symptom control categories and achieved consensus. Domains were categorized into good, moderate, and poor symptom control. Two cut scores were established, differentiating good vs. moderate and moderate vs. poor. Panelists estimated average scores for 100 borderline patients per item. Cut scores were computed based on the sum of the average ratings for individual questions, converted to 0-100 scale. Results Performance descriptions were refined. Panelists discussed that interpretation of the scores should take into account the timing of symptoms after surgery and patient populations, and the importance of items asking symptom frequency, severity, and interference with daily life. The good/moderate cut scores ranged from 21.3 to 35.0 (mean 28.6, SD 3.6) across domains, and moderate/poor ranged from 47.5 to 71.3 (mean 54.5, SD 7.0). Conclusions Panelists were confident in the virtual standard setting process, expecting valid cut scores. Future studies can further validate the cut scores using patient perspectives and collect patient and physician preferences for displaying contextual items on patient- and physician-facing dashboard.https://doi.org/10.1186/s41687-024-00707-xRemote monitoring of symptomsUpper digestive diseaseStandard settingCut scoreAngoff method |
| spellingShingle | Minji K. Lee Mohamad K Abou Chaar Shanda H Blackmon Kathleen J Yost PRO-Angoff method for remote standard setting: establishing clinical thresholds for the upper digestive disease tool Journal of Patient-Reported Outcomes Remote monitoring of symptoms Upper digestive disease Standard setting Cut score Angoff method |
| title | PRO-Angoff method for remote standard setting: establishing clinical thresholds for the upper digestive disease tool |
| title_full | PRO-Angoff method for remote standard setting: establishing clinical thresholds for the upper digestive disease tool |
| title_fullStr | PRO-Angoff method for remote standard setting: establishing clinical thresholds for the upper digestive disease tool |
| title_full_unstemmed | PRO-Angoff method for remote standard setting: establishing clinical thresholds for the upper digestive disease tool |
| title_short | PRO-Angoff method for remote standard setting: establishing clinical thresholds for the upper digestive disease tool |
| title_sort | pro angoff method for remote standard setting establishing clinical thresholds for the upper digestive disease tool |
| topic | Remote monitoring of symptoms Upper digestive disease Standard setting Cut score Angoff method |
| url | https://doi.org/10.1186/s41687-024-00707-x |
| work_keys_str_mv | AT minjiklee proangoffmethodforremotestandardsettingestablishingclinicalthresholdsfortheupperdigestivediseasetool AT mohamadkabouchaar proangoffmethodforremotestandardsettingestablishingclinicalthresholdsfortheupperdigestivediseasetool AT shandahblackmon proangoffmethodforremotestandardsettingestablishingclinicalthresholdsfortheupperdigestivediseasetool AT kathleenjyost proangoffmethodforremotestandardsettingestablishingclinicalthresholdsfortheupperdigestivediseasetool |