Standard setting and internal validation of a novel approach adopted for assessing speaking
In 2016, Euroexam International launched a complex and ambitious project to redesign the rating process and practice of its subjectively scored papers. As part of the project, a series of level-specific performance checklists were developed for speaking and writing. This paper reports on the standar...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Peoples’ Friendship University of Russia (RUDN University)
2020-12-01
|
| Series: | Training, Language and Culture |
| Subjects: | |
| Online Access: | https://rudn.tlcjournal.org/archive/4(4)/4(4)-01.pdf |
| _version_ | 1819057352290074624 |
|---|---|
| author | Zoltán Lukácsi |
| author_facet | Zoltán Lukácsi |
| author_sort | Zoltán Lukácsi |
| collection | DOAJ |
| description | In 2016, Euroexam International launched a complex and ambitious project to redesign the rating process and practice of its subjectively scored papers. As part of the project, a series of level-specific performance checklists were developed for speaking and writing. This paper reports on the standard setting and internal validation procedures for B2 speaking in English as a foreign language (EFL) as executed in Spring, 2020. The participants of the study were 8 EFL teachers and oral examiners from international backgrounds with varying degrees of expertise in language testing. The method adopted for standard setting was the Body of Work method. The participants were allocated into either of two groups working independently on a random sample of 32 audio-recorded live speaking paper performances in a counterbalanced design. The study found that (a) a level-specific 30-item checklist for EFL speaking at B2 can adequately replace the operational rating scales for pass/fail decisions; (b) the granular nature of the checklist renders it more capable of covering the targeted content areas; and (c) the explicit checklist statements support fairness, transparency and accountability. |
| first_indexed | 2024-12-21T13:37:56Z |
| format | Article |
| id | doaj.art-0c0bf617f01948a68fbdb475b53f2b78 |
| institution | Directory Open Access Journal |
| issn | 2520-2073 2521-442X |
| language | English |
| last_indexed | 2024-12-21T13:37:56Z |
| publishDate | 2020-12-01 |
| publisher | Peoples’ Friendship University of Russia (RUDN University) |
| record_format | Article |
| series | Training, Language and Culture |
| spelling | doaj.art-0c0bf617f01948a68fbdb475b53f2b782022-12-21T19:02:06ZengPeoples’ Friendship University of Russia (RUDN University)Training, Language and Culture2520-20732521-442X2020-12-014492210.22363/2521-442X-2020-4-4-9-22Standard setting and internal validation of a novel approach adopted for assessing speakingZoltán Lukácsi0Euroexam InternationalIn 2016, Euroexam International launched a complex and ambitious project to redesign the rating process and practice of its subjectively scored papers. As part of the project, a series of level-specific performance checklists were developed for speaking and writing. This paper reports on the standard setting and internal validation procedures for B2 speaking in English as a foreign language (EFL) as executed in Spring, 2020. The participants of the study were 8 EFL teachers and oral examiners from international backgrounds with varying degrees of expertise in language testing. The method adopted for standard setting was the Body of Work method. The participants were allocated into either of two groups working independently on a random sample of 32 audio-recorded live speaking paper performances in a counterbalanced design. The study found that (a) a level-specific 30-item checklist for EFL speaking at B2 can adequately replace the operational rating scales for pass/fail decisions; (b) the granular nature of the checklist renders it more capable of covering the targeted content areas; and (c) the explicit checklist statements support fairness, transparency and accountability.https://rudn.tlcjournal.org/archive/4(4)/4(4)-01.pdfassessing speakingspeaking assessmentcheckliststandard settingvalidationtransparencyeflenglish as a foreign languageeuroexam |
| spellingShingle | Zoltán Lukácsi Standard setting and internal validation of a novel approach adopted for assessing speaking Training, Language and Culture assessing speaking speaking assessment checklist standard setting validation transparency efl english as a foreign language euroexam |
| title | Standard setting and internal validation of a novel approach adopted for assessing speaking |
| title_full | Standard setting and internal validation of a novel approach adopted for assessing speaking |
| title_fullStr | Standard setting and internal validation of a novel approach adopted for assessing speaking |
| title_full_unstemmed | Standard setting and internal validation of a novel approach adopted for assessing speaking |
| title_short | Standard setting and internal validation of a novel approach adopted for assessing speaking |
| title_sort | standard setting and internal validation of a novel approach adopted for assessing speaking |
| topic | assessing speaking speaking assessment checklist standard setting validation transparency efl english as a foreign language euroexam |
| url | https://rudn.tlcjournal.org/archive/4(4)/4(4)-01.pdf |
| work_keys_str_mv | AT zoltanlukacsi standardsettingandinternalvalidationofanovelapproachadoptedforassessingspeaking |