Judging behaviour and rater errors: an application of the many-facet rasch model
Of the potential sources of construct irrelevant variance or unwanted variability in performance assessment, those associated with raters have been found to be extensive, difficult to control, and impossible to eliminate. And as rater-related errors are non-trivial and threaten the validity of test...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Universiti Kebangsaan Malaysia (UKM)
2011
|
| Subjects: | |
| Online Access: | http://irep.iium.edu.my/13789/7/pp179_197.pdf |
| _version_ | 1796876101489786880 |
|---|---|
| author | Abu Kassim , Noor Lide |
| author_facet | Abu Kassim , Noor Lide |
| author_sort | Abu Kassim , Noor Lide |
| collection | IIUM |
| description | Of the potential sources of construct irrelevant variance or unwanted variability in performance assessment, those associated with raters have been found to be extensive, difficult to control, and impossible to eliminate. And as rater-related errors are non-trivial and threaten the validity of test results, it is necessary that these errors are accounted for and controlled in some way. This paper explains the different types of rater errors and illustrates how they can be identified using the Many-facet Rasch Model, as implemented by FACETS. It also demonstrates what these errors mean in terms of actual judging or rating behaviour and elucidates how they may affect the accuracy of estimation of performance. Rater errors that are explicated in this paper are those related to rater severity, restriction of range, central tendency, and internal consistency. As assessment and its procedures are central to student learning, matters related to valid and fair testing need to be taken seriously. It is hoped that with greater awareness of how we judge and a better understanding of how rater-related errors are introduced into the assessment process, we can be better raters and better teachers.
|
| first_indexed | 2024-03-05T22:47:41Z |
| format | Article |
| id | oai:generic.eprints.org:13789 |
| institution | International Islamic University Malaysia |
| language | English |
| last_indexed | 2024-03-05T22:47:41Z |
| publishDate | 2011 |
| publisher | Universiti Kebangsaan Malaysia (UKM) |
| record_format | dspace |
| spelling | oai:generic.eprints.org:137892013-07-03T04:01:09Z http://irep.iium.edu.my/13789/ Judging behaviour and rater errors: an application of the many-facet rasch model Abu Kassim , Noor Lide L Education (General) LB2361 Curriculum Of the potential sources of construct irrelevant variance or unwanted variability in performance assessment, those associated with raters have been found to be extensive, difficult to control, and impossible to eliminate. And as rater-related errors are non-trivial and threaten the validity of test results, it is necessary that these errors are accounted for and controlled in some way. This paper explains the different types of rater errors and illustrates how they can be identified using the Many-facet Rasch Model, as implemented by FACETS. It also demonstrates what these errors mean in terms of actual judging or rating behaviour and elucidates how they may affect the accuracy of estimation of performance. Rater errors that are explicated in this paper are those related to rater severity, restriction of range, central tendency, and internal consistency. As assessment and its procedures are central to student learning, matters related to valid and fair testing need to be taken seriously. It is hoped that with greater awareness of how we judge and a better understanding of how rater-related errors are introduced into the assessment process, we can be better raters and better teachers. Universiti Kebangsaan Malaysia (UKM) 2011-09 Article PeerReviewed application/pdf en http://irep.iium.edu.my/13789/7/pp179_197.pdf Abu Kassim , Noor Lide (2011) Judging behaviour and rater errors: an application of the many-facet rasch model. GEMA Online Journal of Language Studies, 11 (3). pp. 179-197. ISSN 1675-8021 http://www.ukm.my/ppbl/Gema/gemahome.html |
| spellingShingle | L Education (General) LB2361 Curriculum Abu Kassim , Noor Lide Judging behaviour and rater errors: an application of the many-facet rasch model |
| title | Judging behaviour and rater errors: an application of the many-facet rasch model |
| title_full | Judging behaviour and rater errors: an application of the many-facet rasch model |
| title_fullStr | Judging behaviour and rater errors: an application of the many-facet rasch model |
| title_full_unstemmed | Judging behaviour and rater errors: an application of the many-facet rasch model |
| title_short | Judging behaviour and rater errors: an application of the many-facet rasch model |
| title_sort | judging behaviour and rater errors an application of the many facet rasch model |
| topic | L Education (General) LB2361 Curriculum |
| url | http://irep.iium.edu.my/13789/7/pp179_197.pdf |
| work_keys_str_mv | AT abukassimnoorlide judgingbehaviourandratererrorsanapplicationofthemanyfacetraschmodel |