Item difficulty analysis of a high-stakes mathematics examination using Rasch analysis: The case of sequences and series
The National Senior Certificate examination is the most important school examination in South Africa. Analysis of learners’ performance in Mathematics in this examination is normally carried out and presented in terms of the percentage of learners who succeeded in the different bands of achievement....
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AOSIS
2014-05-01
|
| Series: | Pythagoras |
| Subjects: | |
| Online Access: | https://pythagoras.org.za/index.php/pythagoras/article/view/220 |
| _version_ | 1811291761451991040 |
|---|---|
| author | Mark Jacobs Duncan Mhakure Richard L. Fray Lorna Holtman Cyril Julie |
| author_facet | Mark Jacobs Duncan Mhakure Richard L. Fray Lorna Holtman Cyril Julie |
| author_sort | Mark Jacobs |
| collection | DOAJ |
| description | The National Senior Certificate examination is the most important school examination in South Africa. Analysis of learners’ performance in Mathematics in this examination is normally carried out and presented in terms of the percentage of learners who succeeded in the different bands of achievement. In some cases item difficulties are presented – item refers to the subsection of each examination question. Very little attention is paid to other diagnostic statistics, such as the discrimination indices and item difficulties taking into consideration partial scores examinees achieve on items. In this article we report on a study that, in addition to the usual item difficulties, includes a discrimination index of item difficulties taking into account partial scores examinees achieved. The items, considered individually, are analysed in relation to the other items on the test. The focus is on the topic sequences and series and the data were obtained from a stratified sample of the marked scripts of the candidates who wrote the National Senior Certificate examination in Mathematics in November 2010. Rasch procedures were used for the analysis. The findings indicate that learners perform differently on subsections of topics, herein referred to as items, and that focusing on scores for full topics potentially mask these differences. Mathematical explanations are attempted to account for difficulties learners exhibit in these subsections, using a hierarchy of scale. The findings and our analysis indicate that a form of measurement-driven testing could have beneficial results for teaching. Also, for some items the difficulty obtained from the work of examinees runs counter to the commonly perceived wisdom that an examination ought to be structured in such a way that the less difficult items are at the start of a topic. An explanatory device anchored around the construct of ‘familiarity with problem types through repeated productive practice’ is used to account for the manifested hierarchy of difficulty of the items. |
| first_indexed | 2024-04-13T04:35:11Z |
| format | Article |
| id | doaj.art-7d342b1e2b2b41728cdc7d05e88e3693 |
| institution | Directory Open Access Journal |
| issn | 1012-2346 2223-7895 |
| language | English |
| last_indexed | 2024-04-13T04:35:11Z |
| publishDate | 2014-05-01 |
| publisher | AOSIS |
| record_format | Article |
| series | Pythagoras |
| spelling | doaj.art-7d342b1e2b2b41728cdc7d05e88e36932022-12-22T03:02:12ZengAOSISPythagoras1012-23462223-78952014-05-01351e1e710.4102/pythagoras.v35i1.220158Item difficulty analysis of a high-stakes mathematics examination using Rasch analysis: The case of sequences and seriesMark Jacobs0Duncan Mhakure1Richard L. Fray2Lorna Holtman3Cyril Julie4Electrical Engineering Department, Faculty of Engineering, Cape Peninsula University of TechnologyNumeracy Centre, University of Cape Town, South AfricaDepartment of Mathematics and Applied Mathematics, University of the Western CapePostgraduate Studies, University of the Western CapeSchool of Science and Mathematics Education, University of the Western CapeThe National Senior Certificate examination is the most important school examination in South Africa. Analysis of learners’ performance in Mathematics in this examination is normally carried out and presented in terms of the percentage of learners who succeeded in the different bands of achievement. In some cases item difficulties are presented – item refers to the subsection of each examination question. Very little attention is paid to other diagnostic statistics, such as the discrimination indices and item difficulties taking into consideration partial scores examinees achieve on items. In this article we report on a study that, in addition to the usual item difficulties, includes a discrimination index of item difficulties taking into account partial scores examinees achieved. The items, considered individually, are analysed in relation to the other items on the test. The focus is on the topic sequences and series and the data were obtained from a stratified sample of the marked scripts of the candidates who wrote the National Senior Certificate examination in Mathematics in November 2010. Rasch procedures were used for the analysis. The findings indicate that learners perform differently on subsections of topics, herein referred to as items, and that focusing on scores for full topics potentially mask these differences. Mathematical explanations are attempted to account for difficulties learners exhibit in these subsections, using a hierarchy of scale. The findings and our analysis indicate that a form of measurement-driven testing could have beneficial results for teaching. Also, for some items the difficulty obtained from the work of examinees runs counter to the commonly perceived wisdom that an examination ought to be structured in such a way that the less difficult items are at the start of a topic. An explanatory device anchored around the construct of ‘familiarity with problem types through repeated productive practice’ is used to account for the manifested hierarchy of difficulty of the items.https://pythagoras.org.za/index.php/pythagoras/article/view/220Sequences and seriesHigh-stakes examinationItem difficultyRaschDiscrimination index |
| spellingShingle | Mark Jacobs Duncan Mhakure Richard L. Fray Lorna Holtman Cyril Julie Item difficulty analysis of a high-stakes mathematics examination using Rasch analysis: The case of sequences and series Pythagoras Sequences and series High-stakes examination Item difficulty Rasch Discrimination index |
| title | Item difficulty analysis of a high-stakes mathematics examination using Rasch analysis: The case of sequences and series |
| title_full | Item difficulty analysis of a high-stakes mathematics examination using Rasch analysis: The case of sequences and series |
| title_fullStr | Item difficulty analysis of a high-stakes mathematics examination using Rasch analysis: The case of sequences and series |
| title_full_unstemmed | Item difficulty analysis of a high-stakes mathematics examination using Rasch analysis: The case of sequences and series |
| title_short | Item difficulty analysis of a high-stakes mathematics examination using Rasch analysis: The case of sequences and series |
| title_sort | item difficulty analysis of a high stakes mathematics examination using rasch analysis the case of sequences and series |
| topic | Sequences and series High-stakes examination Item difficulty Rasch Discrimination index |
| url | https://pythagoras.org.za/index.php/pythagoras/article/view/220 |
| work_keys_str_mv | AT markjacobs itemdifficultyanalysisofahighstakesmathematicsexaminationusingraschanalysisthecaseofsequencesandseries AT duncanmhakure itemdifficultyanalysisofahighstakesmathematicsexaminationusingraschanalysisthecaseofsequencesandseries AT richardlfray itemdifficultyanalysisofahighstakesmathematicsexaminationusingraschanalysisthecaseofsequencesandseries AT lornaholtman itemdifficultyanalysisofahighstakesmathematicsexaminationusingraschanalysisthecaseofsequencesandseries AT cyriljulie itemdifficultyanalysisofahighstakesmathematicsexaminationusingraschanalysisthecaseofsequencesandseries |