| Summary: | The Covid-19 pandemic caused the cancellation of 2020 public general qualification examinations in the United Kingdom (UK), and elsewhere. The implementation and subsequent rejection of policies of grade standardisation that followed across the UK’s four nations has the potential to have long-lasting effects on the UK’s qualification systems. The aim of this study was to explore the short- and long-term effects of the policies relating to the summer 2020 general qualification examination series, with respect to the key assessment principles of validity, reliability, fairness, and comparability. Validity arguments for the different approaches taken were developed, using a phased approach based on Crooks, Kane and Cohen’s (1996) validity chain model.
Focused on Wales as a main ‘home-international’ case study, while drawing on other UK nations for elucidation and comparison, this study found that, as well as the validity argument for standardisation, there were also two distinct validity arguments for the awarding of centre assessment grades (CAGs) determined by teachers. Both arguments de-prioritised comparability between examination series and centres, previous students’ legitimate expectations that their grades would continue to represent an equivalent level of attainment to the grades awarded in future, and the relational sense of fairness.
The first validity argument for CAGs was based on the impossibility of meeting the twin goals of awarding fair grades to individual students and achieving causal, cohort-level comparability of outcomes, given that teachers were known from previous years to be optimistic in their predicted grades. Analysis of grade data also suggested that teachers were faced with selecting a grade when, in many cases, a multitude of grades were possible, but none very likely. Student-level standards were prioritised, thus rejecting the logic of the standardisation model. As such, the first argument
2
represented a temporary re-prioritisation of key assessment principles to give more weight to CAGs, as the main source of student-level attainment data.
The second validity argument for CAGs is a more fundamental departure from the usual balance of principles, stating that teachers are not so much more optimistic as more knowledgeable about a student’s level of attainment, which the normal general qualifications assessment process also fails to capture, either because some students will have a bad day, or because outcomes are unfairly capped as a result of predictions used to ensure statistical comparability (Casella et al., 2021). Thus, the attainment measure from previous years was presented as insufficiently valid, and the notion of maintaining the attainment standard was rejected.
The validity arguments are then considered in relation to paradigms of assessment (Baird, 2018a) and paradigm change (Isaacs & Gorgen, 2018) to examine how the pandemic may affect proposals for future assessments and the balance of key principles they represent.
|
|---|