A Study of Raters’ Behavior in Scoring L2 Speaking Performance: Using Rater Discussion as a Training Tool
The studies conducted so far on the effectiveness of resolution methods including the discussion method in resolving discrepancies in rating have yielded mixed results. What is left unnoticed in the literature is the potential of discussion to be used as a training tool rather than a resolution meth...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Allameh Tabataba'i University Press
2019-06-01
|
| Series: | Issues in Language Teaching |
| Subjects: | |
| Online Access: | https://ilt.atu.ac.ir/article_10846_be0f49385d3a01a5d7ce39fffb2291d1.pdf |
| _version_ | 1797378163200753664 |
|---|---|
| author | Alireza Ahmadi |
| author_facet | Alireza Ahmadi |
| author_sort | Alireza Ahmadi |
| collection | DOAJ |
| description | The studies conducted so far on the effectiveness of resolution methods including the discussion method in resolving discrepancies in rating have yielded mixed results. What is left unnoticed in the literature is the potential of discussion to be used as a training tool rather than a resolution method. The present study addresses this research gap by exploring the data coming from rating behaviors of 5 Iranian raters of English. Qualitative analysis of the data indicated that the discussion method can serve the function of training raters. It helped raters rate more easily, quickly and confidently. Furthermore, it helped them improve their understanding and application of the rating criteria and enabled them justify their scoring decisions. Many-faceted Rasch analysis also supported the beneficial effects of discussion in terms of improvement in raters’ severity, consistency in scoring, and the use of scale categories. The findings provide insight into the potential of discussion to be used as a training tool especially in EFL contexts in which lack of access to expert raters can be an obstacle to holding training programs. The author argues for future studies to focus on how discussion may function depending on the rating scale used. |
| first_indexed | 2024-03-08T20:03:46Z |
| format | Article |
| id | doaj.art-1f656c3d98974477a2da7e48504d1380 |
| institution | Directory Open Access Journal |
| issn | 2322-3715 2476-6194 |
| language | English |
| last_indexed | 2024-03-08T20:03:46Z |
| publishDate | 2019-06-01 |
| publisher | Allameh Tabataba'i University Press |
| record_format | Article |
| series | Issues in Language Teaching |
| spelling | doaj.art-1f656c3d98974477a2da7e48504d13802023-12-23T10:47:13ZengAllameh Tabataba'i University PressIssues in Language Teaching2322-37152476-61942019-06-018119522410.22054/ilt.2020.49511.46110846A Study of Raters’ Behavior in Scoring L2 Speaking Performance: Using Rater Discussion as a Training ToolAlireza Ahmadi0Department of Foreign Languages and Linguistics, Faculty of letters and Literature, Shiraz University, Shiraz, IranThe studies conducted so far on the effectiveness of resolution methods including the discussion method in resolving discrepancies in rating have yielded mixed results. What is left unnoticed in the literature is the potential of discussion to be used as a training tool rather than a resolution method. The present study addresses this research gap by exploring the data coming from rating behaviors of 5 Iranian raters of English. Qualitative analysis of the data indicated that the discussion method can serve the function of training raters. It helped raters rate more easily, quickly and confidently. Furthermore, it helped them improve their understanding and application of the rating criteria and enabled them justify their scoring decisions. Many-faceted Rasch analysis also supported the beneficial effects of discussion in terms of improvement in raters’ severity, consistency in scoring, and the use of scale categories. The findings provide insight into the potential of discussion to be used as a training tool especially in EFL contexts in which lack of access to expert raters can be an obstacle to holding training programs. The author argues for future studies to focus on how discussion may function depending on the rating scale used.https://ilt.atu.ac.ir/article_10846_be0f49385d3a01a5d7ce39fffb2291d1.pdfdiscussionrater trainingl2 speakingmany-faceted rasch analysisresolution method |
| spellingShingle | Alireza Ahmadi A Study of Raters’ Behavior in Scoring L2 Speaking Performance: Using Rater Discussion as a Training Tool Issues in Language Teaching discussion rater training l2 speaking many-faceted rasch analysis resolution method |
| title | A Study of Raters’ Behavior in Scoring L2 Speaking Performance: Using Rater Discussion as a Training Tool |
| title_full | A Study of Raters’ Behavior in Scoring L2 Speaking Performance: Using Rater Discussion as a Training Tool |
| title_fullStr | A Study of Raters’ Behavior in Scoring L2 Speaking Performance: Using Rater Discussion as a Training Tool |
| title_full_unstemmed | A Study of Raters’ Behavior in Scoring L2 Speaking Performance: Using Rater Discussion as a Training Tool |
| title_short | A Study of Raters’ Behavior in Scoring L2 Speaking Performance: Using Rater Discussion as a Training Tool |
| title_sort | study of raters behavior in scoring l2 speaking performance using rater discussion as a training tool |
| topic | discussion rater training l2 speaking many-faceted rasch analysis resolution method |
| url | https://ilt.atu.ac.ir/article_10846_be0f49385d3a01a5d7ce39fffb2291d1.pdf |
| work_keys_str_mv | AT alirezaahmadi astudyofratersbehaviorinscoringl2speakingperformanceusingraterdiscussionasatrainingtool AT alirezaahmadi studyofratersbehaviorinscoringl2speakingperformanceusingraterdiscussionasatrainingtool |