Exploring rater judging behaviour using the many-facet rasch model
Performance assessment, unlike the traditional fixed-response assessment, has features peculiar to its assessment setting (the task choice, the task processing conditions, the raters,the rating scale, and the rating procedures) that make it much more vulnerable to construct irrelevant variance (McN...
| Main Author: | |
|---|---|
| Format: | Book Section |
| Language: | English |
| Published: |
Faculty of Communication and Modern Languages, Universiti Utara Malaysia
2007
|
| Subjects: | |
| Online Access: | https://repo.uum.edu.my/id/eprint/3212/1/Noor1.pdf |
| _version_ | 1803624621172326400 |
|---|---|
| author | Abu Kassim, Noor Lide |
| author_facet | Abu Kassim, Noor Lide |
| author_sort | Abu Kassim, Noor Lide |
| collection | UUM |
| description | Performance assessment, unlike the traditional fixed-response assessment, has features peculiar to its assessment setting (the task choice, the task processing conditions, the raters,the rating scale, and the rating procedures) that make it much more vulnerable to construct
irrelevant variance (McNamara, 1997; Upshur & Turner, 1999). Of these potential sources of variability, those associated with raters are considered to be extensive and pose serious threats to the validity of ratings (Linacre, 1989; McNamara, 1996).For performance assessment to yield valid and reliable results, it is essential that these sources of variability are eliminated or minimized. This paper illustrates how sources of rater-related variability or rater effects can be identified and controlled for using the Many-facet Rasch Model.It also illustrates how the idiosyncratic rating behaviour of individual raters can be explicated through the use of this measurement model. In this illustration, the ratings of thirty-five English language instructors on 12 paragraphs written by new intake students at the Centre for Foundation Studies, IIUM were analysed using FACETS (Linacre, 2003), a computer
application which implements the Many-facet Rasch Model. The results of the analysis indicate substantial differences in rater severity, and the presence of other rater effects. |
| first_indexed | 2024-07-04T05:21:17Z |
| format | Book Section |
| id | uum-3212 |
| institution | Universiti Utara Malaysia |
| language | English |
| last_indexed | 2024-07-04T05:21:17Z |
| publishDate | 2007 |
| publisher | Faculty of Communication and Modern Languages, Universiti Utara Malaysia |
| record_format | dspace |
| spelling | uum-32122011-05-29T08:39:33Z https://repo.uum.edu.my/id/eprint/3212/ Exploring rater judging behaviour using the many-facet rasch model Abu Kassim, Noor Lide LB2300 Higher Education QA Mathematics Performance assessment, unlike the traditional fixed-response assessment, has features peculiar to its assessment setting (the task choice, the task processing conditions, the raters,the rating scale, and the rating procedures) that make it much more vulnerable to construct irrelevant variance (McNamara, 1997; Upshur & Turner, 1999). Of these potential sources of variability, those associated with raters are considered to be extensive and pose serious threats to the validity of ratings (Linacre, 1989; McNamara, 1996).For performance assessment to yield valid and reliable results, it is essential that these sources of variability are eliminated or minimized. This paper illustrates how sources of rater-related variability or rater effects can be identified and controlled for using the Many-facet Rasch Model.It also illustrates how the idiosyncratic rating behaviour of individual raters can be explicated through the use of this measurement model. In this illustration, the ratings of thirty-five English language instructors on 12 paragraphs written by new intake students at the Centre for Foundation Studies, IIUM were analysed using FACETS (Linacre, 2003), a computer application which implements the Many-facet Rasch Model. The results of the analysis indicate substantial differences in rater severity, and the presence of other rater effects. Faculty of Communication and Modern Languages, Universiti Utara Malaysia 2007 Book Section PeerReviewed application/pdf en https://repo.uum.edu.my/id/eprint/3212/1/Noor1.pdf Abu Kassim, Noor Lide (2007) Exploring rater judging behaviour using the many-facet rasch model. In: The Second Biennial International Conference on Teaching and Learning of English in Asia : Exploring New Frontiers (TELiA2), 14-16 June 2007, Holiday Villa Beach & Spa Resort, Langkawi. Faculty of Communication and Modern Languages, Universiti Utara Malaysia, Sintok, pp. 1-13. ISBN 978-983-42061-2-3 http://staf.uum.edu.my/tleia2/proceeding.htm |
| spellingShingle | LB2300 Higher Education QA Mathematics Abu Kassim, Noor Lide Exploring rater judging behaviour using the many-facet rasch model |
| title | Exploring rater judging behaviour using the many-facet rasch model |
| title_full | Exploring rater judging behaviour using the many-facet rasch model |
| title_fullStr | Exploring rater judging behaviour using the many-facet rasch model |
| title_full_unstemmed | Exploring rater judging behaviour using the many-facet rasch model |
| title_short | Exploring rater judging behaviour using the many-facet rasch model |
| title_sort | exploring rater judging behaviour using the many facet rasch model |
| topic | LB2300 Higher Education QA Mathematics |
| url | https://repo.uum.edu.my/id/eprint/3212/1/Noor1.pdf |
| work_keys_str_mv | AT abukassimnoorlide exploringraterjudgingbehaviourusingthemanyfacetraschmodel |