Exploring sociodemographic subgroup differences in multiple mini-interview (MMI) performance based on MMI station type and the implications for the predictive fairness of the Hamburg MMI
Abstract Background Sociodemographic subgroup differences in multiple mini-interview (MMI) performance have been extensively studied within the MMI research literature, but heterogeneous findings demand a closer look at how specific aspects of MMI design (such as station type) affect these differenc...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2019-07-01
|
| Series: | BMC Medical Education |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s12909-019-1674-z |
| _version_ | 1818525471397117952 |
|---|---|
| author | Mirjana Knorr Hubertus Meyer Susanne Sehner Wolfgang Hampe Stefan Zimmermann |
| author_facet | Mirjana Knorr Hubertus Meyer Susanne Sehner Wolfgang Hampe Stefan Zimmermann |
| author_sort | Mirjana Knorr |
| collection | DOAJ |
| description | Abstract Background Sociodemographic subgroup differences in multiple mini-interview (MMI) performance have been extensively studied within the MMI research literature, but heterogeneous findings demand a closer look at how specific aspects of MMI design (such as station type) affect these differences. So far, it has not been investigated whether sociodemographic subgroup differences imply that an MMI is biased, particularly in terms of its predictive validity. Methods Between 2010 and 2017, the University Medical Centre Hamburg-Eppendorf (UKE) tested 1438 candidates in an MMI who also provided sociodemographic data and agreed to participate in this study. Out of these, 400 candidates were admitted and underwent a first objective structured clinical examination (OSCE) after one and a half years, including one station assessing communication skills. First, we analyzed the relationship between gender, age, native language and medical family background and MMI station performance including interaction terms with MMI station type (simulation, interview, and group) in a hierarchical linear model. Second, we tested whether the prediction of OSCE overall and communication station performance in particular differed depending on sociodemographic background by adding interaction terms between MMI performance and gender, age and medical family background in a linear regression model. Results Young female candidates performed better than young male candidates both at interview and simulation stations. The gender difference was smaller (simulation) or non-significant (interview) in older candidates. There were no gender or age effects in MMI group station performance. All effects were very small, with the overall model explaining only 0.6% of the variance. MMI performance was not related to OSCE overall performance but significantly predicted OSCE communication station performance with no differences in the prediction for sociodemographic subgroups. Conclusions The Hamburg MMI is fair in its prediction of OSCE communication scores. Differences in MMI station performance for gender and age and their interaction with MMI station type can be related to the dimensions assessed at different station types and thus support the validity of the MMI. Rather than being threats to fairness, these differences could be useful for decisions relating to the design and use of an MMI. |
| first_indexed | 2024-12-11T06:09:39Z |
| format | Article |
| id | doaj.art-689e0c73ca4a4ea79adcf52b7ae8a13e |
| institution | Directory Open Access Journal |
| issn | 1472-6920 |
| language | English |
| last_indexed | 2024-12-11T06:09:39Z |
| publishDate | 2019-07-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Education |
| spelling | doaj.art-689e0c73ca4a4ea79adcf52b7ae8a13e2022-12-22T01:18:09ZengBMCBMC Medical Education1472-69202019-07-0119111210.1186/s12909-019-1674-zExploring sociodemographic subgroup differences in multiple mini-interview (MMI) performance based on MMI station type and the implications for the predictive fairness of the Hamburg MMIMirjana Knorr0Hubertus Meyer1Susanne Sehner2Wolfgang Hampe3Stefan Zimmermann4Institute of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf (UKE)Institute of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf (UKE)Institute of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf (UKE)Institute of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf (UKE)Institute of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf (UKE)Abstract Background Sociodemographic subgroup differences in multiple mini-interview (MMI) performance have been extensively studied within the MMI research literature, but heterogeneous findings demand a closer look at how specific aspects of MMI design (such as station type) affect these differences. So far, it has not been investigated whether sociodemographic subgroup differences imply that an MMI is biased, particularly in terms of its predictive validity. Methods Between 2010 and 2017, the University Medical Centre Hamburg-Eppendorf (UKE) tested 1438 candidates in an MMI who also provided sociodemographic data and agreed to participate in this study. Out of these, 400 candidates were admitted and underwent a first objective structured clinical examination (OSCE) after one and a half years, including one station assessing communication skills. First, we analyzed the relationship between gender, age, native language and medical family background and MMI station performance including interaction terms with MMI station type (simulation, interview, and group) in a hierarchical linear model. Second, we tested whether the prediction of OSCE overall and communication station performance in particular differed depending on sociodemographic background by adding interaction terms between MMI performance and gender, age and medical family background in a linear regression model. Results Young female candidates performed better than young male candidates both at interview and simulation stations. The gender difference was smaller (simulation) or non-significant (interview) in older candidates. There were no gender or age effects in MMI group station performance. All effects were very small, with the overall model explaining only 0.6% of the variance. MMI performance was not related to OSCE overall performance but significantly predicted OSCE communication station performance with no differences in the prediction for sociodemographic subgroups. Conclusions The Hamburg MMI is fair in its prediction of OSCE communication scores. Differences in MMI station performance for gender and age and their interaction with MMI station type can be related to the dimensions assessed at different station types and thus support the validity of the MMI. Rather than being threats to fairness, these differences could be useful for decisions relating to the design and use of an MMI.http://link.springer.com/article/10.1186/s12909-019-1674-zMultiple mini-interviewSociodemographic subgroup differencesPredictive fairnessGender differencesGender and age interactionNative language |
| spellingShingle | Mirjana Knorr Hubertus Meyer Susanne Sehner Wolfgang Hampe Stefan Zimmermann Exploring sociodemographic subgroup differences in multiple mini-interview (MMI) performance based on MMI station type and the implications for the predictive fairness of the Hamburg MMI BMC Medical Education Multiple mini-interview Sociodemographic subgroup differences Predictive fairness Gender differences Gender and age interaction Native language |
| title | Exploring sociodemographic subgroup differences in multiple mini-interview (MMI) performance based on MMI station type and the implications for the predictive fairness of the Hamburg MMI |
| title_full | Exploring sociodemographic subgroup differences in multiple mini-interview (MMI) performance based on MMI station type and the implications for the predictive fairness of the Hamburg MMI |
| title_fullStr | Exploring sociodemographic subgroup differences in multiple mini-interview (MMI) performance based on MMI station type and the implications for the predictive fairness of the Hamburg MMI |
| title_full_unstemmed | Exploring sociodemographic subgroup differences in multiple mini-interview (MMI) performance based on MMI station type and the implications for the predictive fairness of the Hamburg MMI |
| title_short | Exploring sociodemographic subgroup differences in multiple mini-interview (MMI) performance based on MMI station type and the implications for the predictive fairness of the Hamburg MMI |
| title_sort | exploring sociodemographic subgroup differences in multiple mini interview mmi performance based on mmi station type and the implications for the predictive fairness of the hamburg mmi |
| topic | Multiple mini-interview Sociodemographic subgroup differences Predictive fairness Gender differences Gender and age interaction Native language |
| url | http://link.springer.com/article/10.1186/s12909-019-1674-z |
| work_keys_str_mv | AT mirjanaknorr exploringsociodemographicsubgroupdifferencesinmultipleminiinterviewmmiperformancebasedonmmistationtypeandtheimplicationsforthepredictivefairnessofthehamburgmmi AT hubertusmeyer exploringsociodemographicsubgroupdifferencesinmultipleminiinterviewmmiperformancebasedonmmistationtypeandtheimplicationsforthepredictivefairnessofthehamburgmmi AT susannesehner exploringsociodemographicsubgroupdifferencesinmultipleminiinterviewmmiperformancebasedonmmistationtypeandtheimplicationsforthepredictivefairnessofthehamburgmmi AT wolfganghampe exploringsociodemographicsubgroupdifferencesinmultipleminiinterviewmmiperformancebasedonmmistationtypeandtheimplicationsforthepredictivefairnessofthehamburgmmi AT stefanzimmermann exploringsociodemographicsubgroupdifferencesinmultipleminiinterviewmmiperformancebasedonmmistationtypeandtheimplicationsforthepredictivefairnessofthehamburgmmi |