A Model-Based Examination of Scale Effects in Student Evaluations of Teaching
Student evaluations of teaching are widely used to assess instructors and courses. Using a model-based approach and Bayesian methods, we examine how the direction of the scale, labels on scales, and the number of options affect the ratings. We conduct a within-participants experiment in which respon...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2021-08-01
|
| Series: | AERA Open |
| Online Access: | https://doi.org/10.1177/23328584211040083 |
| _version_ | 1819083969882226688 |
|---|---|
| author | Karyssa A. Courey Michael D. Lee |
| author_facet | Karyssa A. Courey Michael D. Lee |
| author_sort | Karyssa A. Courey |
| collection | DOAJ |
| description | Student evaluations of teaching are widely used to assess instructors and courses. Using a model-based approach and Bayesian methods, we examine how the direction of the scale, labels on scales, and the number of options affect the ratings. We conduct a within-participants experiment in which respondents evaluate instructors and lectures using different scales. We find that people tend to give positive ratings, especially when using letter scales compared with number scales. Furthermore, people tend to use the end-points less often when a scale is presented in reverse. Our model-based analysis allows us to infer how the features of scales shift responses to higher or lower ratings and how they compress scale use to make end-point responses more or less likely. The model also makes predictions about equivalent ratings across scales, which we demonstrate using real-world evaluation data. Our study has implications for the design of scales and for their use in assessment. |
| first_indexed | 2024-12-21T20:41:01Z |
| format | Article |
| id | doaj.art-3a353285999e4f1b87f424251fcff8fe |
| institution | Directory Open Access Journal |
| issn | 2332-8584 |
| language | English |
| last_indexed | 2024-12-21T20:41:01Z |
| publishDate | 2021-08-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | AERA Open |
| spelling | doaj.art-3a353285999e4f1b87f424251fcff8fe2022-12-21T18:50:58ZengSAGE PublishingAERA Open2332-85842021-08-01710.1177/23328584211040083A Model-Based Examination of Scale Effects in Student Evaluations of TeachingKaryssa A. CoureyMichael D. LeeStudent evaluations of teaching are widely used to assess instructors and courses. Using a model-based approach and Bayesian methods, we examine how the direction of the scale, labels on scales, and the number of options affect the ratings. We conduct a within-participants experiment in which respondents evaluate instructors and lectures using different scales. We find that people tend to give positive ratings, especially when using letter scales compared with number scales. Furthermore, people tend to use the end-points less often when a scale is presented in reverse. Our model-based analysis allows us to infer how the features of scales shift responses to higher or lower ratings and how they compress scale use to make end-point responses more or less likely. The model also makes predictions about equivalent ratings across scales, which we demonstrate using real-world evaluation data. Our study has implications for the design of scales and for their use in assessment.https://doi.org/10.1177/23328584211040083 |
| spellingShingle | Karyssa A. Courey Michael D. Lee A Model-Based Examination of Scale Effects in Student Evaluations of Teaching AERA Open |
| title | A Model-Based Examination of Scale Effects in Student Evaluations of Teaching |
| title_full | A Model-Based Examination of Scale Effects in Student Evaluations of Teaching |
| title_fullStr | A Model-Based Examination of Scale Effects in Student Evaluations of Teaching |
| title_full_unstemmed | A Model-Based Examination of Scale Effects in Student Evaluations of Teaching |
| title_short | A Model-Based Examination of Scale Effects in Student Evaluations of Teaching |
| title_sort | model based examination of scale effects in student evaluations of teaching |
| url | https://doi.org/10.1177/23328584211040083 |
| work_keys_str_mv | AT karyssaacourey amodelbasedexaminationofscaleeffectsinstudentevaluationsofteaching AT michaeldlee amodelbasedexaminationofscaleeffectsinstudentevaluationsofteaching AT karyssaacourey modelbasedexaminationofscaleeffectsinstudentevaluationsofteaching AT michaeldlee modelbasedexaminationofscaleeffectsinstudentevaluationsofteaching |