A study on the standard setting, validity, and reliability of a standardized patient performance rating scale – student version
AbstractIntroduction The quality of the performances of standardized patients plays a significant role in the effectiveness of clinical skills education. Therefore, providing standardized patients with constant feedback is essential. It is especially important to get students’ perspectives immediate...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2023-12-01
|
| Series: | Annals of Medicine |
| Subjects: | |
| Online Access: | https://www.tandfonline.com/doi/10.1080/07853890.2023.2168744 |
| _version_ | 1797353723844886528 |
|---|---|
| author | Ipek Gonullu Celal Deha Dogan Sengul Erden Derya Gokmen |
| author_facet | Ipek Gonullu Celal Deha Dogan Sengul Erden Derya Gokmen |
| author_sort | Ipek Gonullu |
| collection | DOAJ |
| description | AbstractIntroduction The quality of the performances of standardized patients plays a significant role in the effectiveness of clinical skills education. Therefore, providing standardized patients with constant feedback is essential. It is especially important to get students’ perspectives immediately following their encounters with standardized patients. In the literature, there is no scale for use by students to evaluate the performance of standardized patients. Thus, the three main goals of this study were to: (1) develop a scale for use by students to evaluate the performance of standardized patients, (2) examine the psychometric properties of the scale, and (3) determine a cut-off score for the scale in a standard-settingMaterials and methods Seven hundred and two medical students participated in the scale- development process, the pilot test, and the validation process, and seven educators took part in the standard-setting process. After the evaluation of content validity, construct validity was assessed via exploratory and confirmatory factor analyses. For the standard-setting study, the extended Angoff method was used.Results The exploratory factor analysis revealed that the scale had a single-factor structure, which was confirmed by confirmatory factor analysis. The Cronbach’s alpha internal consistency coefficient was 0.91. The scale consists of nine items. The cut-off score was determined to be 24.11/45, which represents the minimum acceptable standard for standardized patient performance.Conclusions Our study outlined the critical steps in developing a measurement tool and produced a valid and reliable scale that allows medical students to assess the performance of standardized patients immediately following their interaction with the standardized patient. This scale constitutes an important contribution to the literature as it provides a tool for standardized patient trainers to assess standardized patients’ weaknesses and help them improve their performance.KEY MESSAGESEvaluation of SP performance is essential to ensure the educational quality of clinical skills training programs.Students are the most relevant stakeholders to give feedback about SP performance immediately after encounters.The ‘Standardized Patient Performance Rating Scale – Student Version’ is a valid, reliable scale that can be used by students for the evaluation of standardized patients’ strengths and weaknesses at individual-performance levels quickly. |
| first_indexed | 2024-03-08T13:35:10Z |
| format | Article |
| id | doaj.art-36202f24d58a4edaaea41800eeca677b |
| institution | Directory Open Access Journal |
| issn | 0785-3890 1365-2060 |
| language | English |
| last_indexed | 2024-03-08T13:35:10Z |
| publishDate | 2023-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | Annals of Medicine |
| spelling | doaj.art-36202f24d58a4edaaea41800eeca677b2024-01-16T19:13:21ZengTaylor & Francis GroupAnnals of Medicine0785-38901365-20602023-12-0155149050110.1080/07853890.2023.2168744A study on the standard setting, validity, and reliability of a standardized patient performance rating scale – student versionIpek Gonullu0Celal Deha Dogan1Sengul Erden2Derya Gokmen3Department of Medical Education and Informatics, Ankara University Faculty of Medicine, Ankara, TurkeyDepartment of Measurement and Evaluation, Ankara University Faculty of Education, Ankara, TurkeyDepartment of Medical Education and Informatics, Ankara University Faculty of Medicine, Ankara, TurkeyDepartment of Biostatistics, Ankara University Faculty of Medicine, Ankara, TurkeyAbstractIntroduction The quality of the performances of standardized patients plays a significant role in the effectiveness of clinical skills education. Therefore, providing standardized patients with constant feedback is essential. It is especially important to get students’ perspectives immediately following their encounters with standardized patients. In the literature, there is no scale for use by students to evaluate the performance of standardized patients. Thus, the three main goals of this study were to: (1) develop a scale for use by students to evaluate the performance of standardized patients, (2) examine the psychometric properties of the scale, and (3) determine a cut-off score for the scale in a standard-settingMaterials and methods Seven hundred and two medical students participated in the scale- development process, the pilot test, and the validation process, and seven educators took part in the standard-setting process. After the evaluation of content validity, construct validity was assessed via exploratory and confirmatory factor analyses. For the standard-setting study, the extended Angoff method was used.Results The exploratory factor analysis revealed that the scale had a single-factor structure, which was confirmed by confirmatory factor analysis. The Cronbach’s alpha internal consistency coefficient was 0.91. The scale consists of nine items. The cut-off score was determined to be 24.11/45, which represents the minimum acceptable standard for standardized patient performance.Conclusions Our study outlined the critical steps in developing a measurement tool and produced a valid and reliable scale that allows medical students to assess the performance of standardized patients immediately following their interaction with the standardized patient. This scale constitutes an important contribution to the literature as it provides a tool for standardized patient trainers to assess standardized patients’ weaknesses and help them improve their performance.KEY MESSAGESEvaluation of SP performance is essential to ensure the educational quality of clinical skills training programs.Students are the most relevant stakeholders to give feedback about SP performance immediately after encounters.The ‘Standardized Patient Performance Rating Scale – Student Version’ is a valid, reliable scale that can be used by students for the evaluation of standardized patients’ strengths and weaknesses at individual-performance levels quickly.https://www.tandfonline.com/doi/10.1080/07853890.2023.2168744Performance rating scalereliabilitystandard settingstandardized patientstudent ratingsvalidity |
| spellingShingle | Ipek Gonullu Celal Deha Dogan Sengul Erden Derya Gokmen A study on the standard setting, validity, and reliability of a standardized patient performance rating scale – student version Annals of Medicine Performance rating scale reliability standard setting standardized patient student ratings validity |
| title | A study on the standard setting, validity, and reliability of a standardized patient performance rating scale – student version |
| title_full | A study on the standard setting, validity, and reliability of a standardized patient performance rating scale – student version |
| title_fullStr | A study on the standard setting, validity, and reliability of a standardized patient performance rating scale – student version |
| title_full_unstemmed | A study on the standard setting, validity, and reliability of a standardized patient performance rating scale – student version |
| title_short | A study on the standard setting, validity, and reliability of a standardized patient performance rating scale – student version |
| title_sort | study on the standard setting validity and reliability of a standardized patient performance rating scale student version |
| topic | Performance rating scale reliability standard setting standardized patient student ratings validity |
| url | https://www.tandfonline.com/doi/10.1080/07853890.2023.2168744 |
| work_keys_str_mv | AT ipekgonullu astudyonthestandardsettingvalidityandreliabilityofastandardizedpatientperformanceratingscalestudentversion AT celaldehadogan astudyonthestandardsettingvalidityandreliabilityofastandardizedpatientperformanceratingscalestudentversion AT sengulerden astudyonthestandardsettingvalidityandreliabilityofastandardizedpatientperformanceratingscalestudentversion AT deryagokmen astudyonthestandardsettingvalidityandreliabilityofastandardizedpatientperformanceratingscalestudentversion AT ipekgonullu studyonthestandardsettingvalidityandreliabilityofastandardizedpatientperformanceratingscalestudentversion AT celaldehadogan studyonthestandardsettingvalidityandreliabilityofastandardizedpatientperformanceratingscalestudentversion AT sengulerden studyonthestandardsettingvalidityandreliabilityofastandardizedpatientperformanceratingscalestudentversion AT deryagokmen studyonthestandardsettingvalidityandreliabilityofastandardizedpatientperformanceratingscalestudentversion |