Leniency and halo effects in marking undergraduate short research projects
<p>Abstract</p> <p>Background</p> <p>Supervisors are often involved in the assessment of projects they have supervised themselves. Previous research suggests that detailed marking sheets may alleviate leniency and halo effects. We set out to determine if, despite using...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2004-11-01
|
| Series: | BMC Medical Education |
| Online Access: | http://www.biomedcentral.com/1472-6920/4/28 |
| _version_ | 1819182811896086528 |
|---|---|
| author | Elton Robert A Cameron Helen S McKinstry Brian H Riley Simon C |
| author_facet | Elton Robert A Cameron Helen S McKinstry Brian H Riley Simon C |
| author_sort | Elton Robert A |
| collection | DOAJ |
| description | <p>Abstract</p> <p>Background</p> <p>Supervisors are often involved in the assessment of projects they have supervised themselves. Previous research suggests that detailed marking sheets may alleviate leniency and halo effects. We set out to determine if, despite using such a marking schedule, leniency and halo effects were evident in the supervisors' marking of undergraduate short research projects (special study modules (SSM)).</p> <p>Methods</p> <p>Review of grades awarded by supervisors, second markers and control markers to the written reports of 4<sup>th </sup>year medical students who had participated in an SSM during two full academic years (n = 399). Paired t-tests were used to compare mean marks, Pearson correlation to look at agreement between marks and multiple linear regression to test the prediction of one mark from several others adjusted for one another.</p> <p>Results</p> <p>There was a highly significant difference of approximately half a grade between supervisors and second markers with supervisors marking higher. (t = 3.12, p < 0.01, difference in grade score = 0.42, 95% CI for mean difference 0.18–0.80). There was a high correlation between the two marks awarded for performance of the project and the written report by the supervisor (r = 0.75), but a low-modest correlation between supervisor and second marker (r = 0.28). Linear regression analysis of the influence of the supervisors' mark for performance on their mark for the report gave a non-significant result. This suggests a leniency effect but no halo effect.</p> <p>Conclusions</p> <p>This study shows that with the use of structured marking sheet for assessment of undergraduate medical students, supervisors marks are not associated with a halo effect, but leniency does occur. As supervisor assessment is becoming more common in both under graduate and postgraduate teaching new ways to improve objectivity in marking and to address the leniency of supervisors should be sought.</p> |
| first_indexed | 2024-12-22T22:52:04Z |
| format | Article |
| id | doaj.art-c2f8e5e2e5a04e10905651c6980a9322 |
| institution | Directory Open Access Journal |
| issn | 1472-6920 |
| language | English |
| last_indexed | 2024-12-22T22:52:04Z |
| publishDate | 2004-11-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Education |
| spelling | doaj.art-c2f8e5e2e5a04e10905651c6980a93222022-12-21T18:09:55ZengBMCBMC Medical Education1472-69202004-11-01412810.1186/1472-6920-4-28Leniency and halo effects in marking undergraduate short research projectsElton Robert ACameron Helen SMcKinstry Brian HRiley Simon C<p>Abstract</p> <p>Background</p> <p>Supervisors are often involved in the assessment of projects they have supervised themselves. Previous research suggests that detailed marking sheets may alleviate leniency and halo effects. We set out to determine if, despite using such a marking schedule, leniency and halo effects were evident in the supervisors' marking of undergraduate short research projects (special study modules (SSM)).</p> <p>Methods</p> <p>Review of grades awarded by supervisors, second markers and control markers to the written reports of 4<sup>th </sup>year medical students who had participated in an SSM during two full academic years (n = 399). Paired t-tests were used to compare mean marks, Pearson correlation to look at agreement between marks and multiple linear regression to test the prediction of one mark from several others adjusted for one another.</p> <p>Results</p> <p>There was a highly significant difference of approximately half a grade between supervisors and second markers with supervisors marking higher. (t = 3.12, p < 0.01, difference in grade score = 0.42, 95% CI for mean difference 0.18–0.80). There was a high correlation between the two marks awarded for performance of the project and the written report by the supervisor (r = 0.75), but a low-modest correlation between supervisor and second marker (r = 0.28). Linear regression analysis of the influence of the supervisors' mark for performance on their mark for the report gave a non-significant result. This suggests a leniency effect but no halo effect.</p> <p>Conclusions</p> <p>This study shows that with the use of structured marking sheet for assessment of undergraduate medical students, supervisors marks are not associated with a halo effect, but leniency does occur. As supervisor assessment is becoming more common in both under graduate and postgraduate teaching new ways to improve objectivity in marking and to address the leniency of supervisors should be sought.</p>http://www.biomedcentral.com/1472-6920/4/28 |
| spellingShingle | Elton Robert A Cameron Helen S McKinstry Brian H Riley Simon C Leniency and halo effects in marking undergraduate short research projects BMC Medical Education |
| title | Leniency and halo effects in marking undergraduate short research projects |
| title_full | Leniency and halo effects in marking undergraduate short research projects |
| title_fullStr | Leniency and halo effects in marking undergraduate short research projects |
| title_full_unstemmed | Leniency and halo effects in marking undergraduate short research projects |
| title_short | Leniency and halo effects in marking undergraduate short research projects |
| title_sort | leniency and halo effects in marking undergraduate short research projects |
| url | http://www.biomedcentral.com/1472-6920/4/28 |
| work_keys_str_mv | AT eltonroberta leniencyandhaloeffectsinmarkingundergraduateshortresearchprojects AT cameronhelens leniencyandhaloeffectsinmarkingundergraduateshortresearchprojects AT mckinstrybrianh leniencyandhaloeffectsinmarkingundergraduateshortresearchprojects AT rileysimonc leniencyandhaloeffectsinmarkingundergraduateshortresearchprojects |