Investigating rater–student interaction, gender bias, and major bias in the assessment of research seminar presentation
The study aims to investigate the severity of raters toward examinees' performances and to analyze biases subject to genders and academic majors based on the rater–examinee interactions. Data were collected across a 13-week research seminar course in English. Six raters were selected to rate 33...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-06-01
|
| Series: | Heliyon |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2405844023037556 |
| _version_ | 1797818662893125632 |
|---|---|
| author | Fitria Arifiyanti Soeharto Soeharto Stephen Amukune Son Van Nguyen Khalil Aburezeq Achmad Hidayatullah Eri Sarimanah |
| author_facet | Fitria Arifiyanti Soeharto Soeharto Stephen Amukune Son Van Nguyen Khalil Aburezeq Achmad Hidayatullah Eri Sarimanah |
| author_sort | Fitria Arifiyanti |
| collection | DOAJ |
| description | The study aims to investigate the severity of raters toward examinees' performances and to analyze biases subject to genders and academic majors based on the rater–examinee interactions. Data were collected across a 13-week research seminar course in English. Six raters were selected to rate 33 examinees using 18-item criteria. The many-facet Rasch measurement was utilized to analyze rater–examinee interactions and bias factors. The results confirmed that the instrument is reliable and valid. This study depicted the interaction between raters and other facets using a variable map, where the raters exhibited different levels of severity/leniency in scoring students' performances in oral presentations at the research seminar course. The result based on the Rasch analysis also confirmed that gender and academic majors contaminated rater assessment. Bias interaction between raters and student gender was detected, and Rater 6 displayed bias based on gender due to the tendency to give higher scores to female than male participants with a target contrast of 2.05 logits. Bias interaction between rater academic majors and student academic majors was also identified among raters with linguistics and psychology majors. |
| first_indexed | 2024-03-13T09:11:15Z |
| format | Article |
| id | doaj.art-b97d1e0c7df64922a5ab2e670d9642ea |
| institution | Directory Open Access Journal |
| issn | 2405-8440 |
| language | English |
| last_indexed | 2024-03-13T09:11:15Z |
| publishDate | 2023-06-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Heliyon |
| spelling | doaj.art-b97d1e0c7df64922a5ab2e670d9642ea2023-05-27T04:26:14ZengElsevierHeliyon2405-84402023-06-0196e16548Investigating rater–student interaction, gender bias, and major bias in the assessment of research seminar presentationFitria Arifiyanti0Soeharto Soeharto1Stephen Amukune2Son Van Nguyen3Khalil Aburezeq4Achmad Hidayatullah5Eri Sarimanah6Doctoral School of Education, University of Szeged, 30-34, Petőfi S. sgt., H-6722, Szeged, Hungary; Corresponding author.Doctoral School of Education, University of Szeged, 30-34, Petőfi S. sgt., H-6722, Szeged, HungaryDoctoral School of Education, University of Szeged, 30-34, Petőfi S. sgt., H-6722, Szeged, HungaryDoctoral School of Education, University of Szeged, 30-34, Petőfi S. sgt., H-6722, Szeged, Hungary; Thuyloi University, No. 175, Tay Son Street, Dong Da District, Hanoi, Viet NamDoctoral School of Education, University of Szeged, 30-34, Petőfi S. sgt., H-6722, Szeged, HungaryDoctoral School of Education, University of Szeged, 30-34, Petőfi S. sgt., H-6722, Szeged, HungaryFaculty of Language and Literature Education, Pakuan University, IndonesiaThe study aims to investigate the severity of raters toward examinees' performances and to analyze biases subject to genders and academic majors based on the rater–examinee interactions. Data were collected across a 13-week research seminar course in English. Six raters were selected to rate 33 examinees using 18-item criteria. The many-facet Rasch measurement was utilized to analyze rater–examinee interactions and bias factors. The results confirmed that the instrument is reliable and valid. This study depicted the interaction between raters and other facets using a variable map, where the raters exhibited different levels of severity/leniency in scoring students' performances in oral presentations at the research seminar course. The result based on the Rasch analysis also confirmed that gender and academic majors contaminated rater assessment. Bias interaction between raters and student gender was detected, and Rater 6 displayed bias based on gender due to the tendency to give higher scores to female than male participants with a target contrast of 2.05 logits. Bias interaction between rater academic majors and student academic majors was also identified among raters with linguistics and psychology majors.http://www.sciencedirect.com/science/article/pii/S2405844023037556Gender biasMajor biasRater–Student interactionMany-facet Rasch measurementAssessment |
| spellingShingle | Fitria Arifiyanti Soeharto Soeharto Stephen Amukune Son Van Nguyen Khalil Aburezeq Achmad Hidayatullah Eri Sarimanah Investigating rater–student interaction, gender bias, and major bias in the assessment of research seminar presentation Heliyon Gender bias Major bias Rater–Student interaction Many-facet Rasch measurement Assessment |
| title | Investigating rater–student interaction, gender bias, and major bias in the assessment of research seminar presentation |
| title_full | Investigating rater–student interaction, gender bias, and major bias in the assessment of research seminar presentation |
| title_fullStr | Investigating rater–student interaction, gender bias, and major bias in the assessment of research seminar presentation |
| title_full_unstemmed | Investigating rater–student interaction, gender bias, and major bias in the assessment of research seminar presentation |
| title_short | Investigating rater–student interaction, gender bias, and major bias in the assessment of research seminar presentation |
| title_sort | investigating rater student interaction gender bias and major bias in the assessment of research seminar presentation |
| topic | Gender bias Major bias Rater–Student interaction Many-facet Rasch measurement Assessment |
| url | http://www.sciencedirect.com/science/article/pii/S2405844023037556 |
| work_keys_str_mv | AT fitriaarifiyanti investigatingraterstudentinteractiongenderbiasandmajorbiasintheassessmentofresearchseminarpresentation AT soehartosoeharto investigatingraterstudentinteractiongenderbiasandmajorbiasintheassessmentofresearchseminarpresentation AT stephenamukune investigatingraterstudentinteractiongenderbiasandmajorbiasintheassessmentofresearchseminarpresentation AT sonvannguyen investigatingraterstudentinteractiongenderbiasandmajorbiasintheassessmentofresearchseminarpresentation AT khalilaburezeq investigatingraterstudentinteractiongenderbiasandmajorbiasintheassessmentofresearchseminarpresentation AT achmadhidayatullah investigatingraterstudentinteractiongenderbiasandmajorbiasintheassessmentofresearchseminarpresentation AT erisarimanah investigatingraterstudentinteractiongenderbiasandmajorbiasintheassessmentofresearchseminarpresentation |