Medical education research study quality instrument: an objective instrument susceptible to subjectivity
ABSTRACTBackground: The medical education research study quality instrument (MERSQI) was designed to appraise medical education research quality based on study design criteria. As with many such tools, application of the results may have unintended consequences. This study applied the MERSQI to publ...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2024-12-01
|
| Series: | Medical Education Online |
| Subjects: | |
| Online Access: | https://www.tandfonline.com/doi/10.1080/10872981.2024.2308359 |
| _version_ | 1797347345259560960 |
|---|---|
| author | Scott Jaros Gary Beck Dallaghan |
| author_facet | Scott Jaros Gary Beck Dallaghan |
| author_sort | Scott Jaros |
| collection | DOAJ |
| description | ABSTRACTBackground: The medical education research study quality instrument (MERSQI) was designed to appraise medical education research quality based on study design criteria. As with many such tools, application of the results may have unintended consequences. This study applied the MERSQI to published medical education research identified in a bibliometric analysis. Methods: A bibliometric analysis identified highly cited articles in medical education that two authors independently evaluated using the MERSQI. After screening duplicate or non-research articles, the authors reviewed 21 articles with the quality instrument. Initially, five articles were reviewed independently and results were compared to ensure agreed upon understanding of the instrument items. The remainder of the articles were independently reviewed. Overall scores for the articles were analyzed with a paired samples t-test and individual item ratings were analyzed for inter-rater reliability. Results: There was a significant difference in mean MERSQI score between reviewers. Inter-rater reliability for MERSQI items labeled response rate, validity and outcomes were considered unacceptable. Conclusions: Based on these results there is evidence that MERSQI items can be significantly influenced by interpretation, which lead to a difference in scoring. The MERSQI is a useful guide for identifying research methodologies. However, it should not be used to make judgments on the overall quality of medical education research methodology in its current format. The authors make specific recommendations for how the instrument could be revised for greater clarity and accuracy. |
| first_indexed | 2024-03-08T11:46:26Z |
| format | Article |
| id | doaj.art-550e9e8239ef43a5aa5429303afc5647 |
| institution | Directory Open Access Journal |
| issn | 1087-2981 |
| language | English |
| last_indexed | 2024-03-08T11:46:26Z |
| publishDate | 2024-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | Medical Education Online |
| spelling | doaj.art-550e9e8239ef43a5aa5429303afc56472024-01-24T19:06:23ZengTaylor & Francis GroupMedical Education Online1087-29812024-12-0129110.1080/10872981.2024.2308359Medical education research study quality instrument: an objective instrument susceptible to subjectivityScott Jaros0Gary Beck Dallaghan1University of North Carolina School of Medicine, Chapel Hill, NC, USADepartment of Medical Education, University of Texas at Tyler School of Medicine, Tyler, TX, USAABSTRACTBackground: The medical education research study quality instrument (MERSQI) was designed to appraise medical education research quality based on study design criteria. As with many such tools, application of the results may have unintended consequences. This study applied the MERSQI to published medical education research identified in a bibliometric analysis. Methods: A bibliometric analysis identified highly cited articles in medical education that two authors independently evaluated using the MERSQI. After screening duplicate or non-research articles, the authors reviewed 21 articles with the quality instrument. Initially, five articles were reviewed independently and results were compared to ensure agreed upon understanding of the instrument items. The remainder of the articles were independently reviewed. Overall scores for the articles were analyzed with a paired samples t-test and individual item ratings were analyzed for inter-rater reliability. Results: There was a significant difference in mean MERSQI score between reviewers. Inter-rater reliability for MERSQI items labeled response rate, validity and outcomes were considered unacceptable. Conclusions: Based on these results there is evidence that MERSQI items can be significantly influenced by interpretation, which lead to a difference in scoring. The MERSQI is a useful guide for identifying research methodologies. However, it should not be used to make judgments on the overall quality of medical education research methodology in its current format. The authors make specific recommendations for how the instrument could be revised for greater clarity and accuracy.https://www.tandfonline.com/doi/10.1080/10872981.2024.2308359Medical educationresearchMERSQIqualitybibliometric analysis |
| spellingShingle | Scott Jaros Gary Beck Dallaghan Medical education research study quality instrument: an objective instrument susceptible to subjectivity Medical Education Online Medical education research MERSQI quality bibliometric analysis |
| title | Medical education research study quality instrument: an objective instrument susceptible to subjectivity |
| title_full | Medical education research study quality instrument: an objective instrument susceptible to subjectivity |
| title_fullStr | Medical education research study quality instrument: an objective instrument susceptible to subjectivity |
| title_full_unstemmed | Medical education research study quality instrument: an objective instrument susceptible to subjectivity |
| title_short | Medical education research study quality instrument: an objective instrument susceptible to subjectivity |
| title_sort | medical education research study quality instrument an objective instrument susceptible to subjectivity |
| topic | Medical education research MERSQI quality bibliometric analysis |
| url | https://www.tandfonline.com/doi/10.1080/10872981.2024.2308359 |
| work_keys_str_mv | AT scottjaros medicaleducationresearchstudyqualityinstrumentanobjectiveinstrumentsusceptibletosubjectivity AT garybeckdallaghan medicaleducationresearchstudyqualityinstrumentanobjectiveinstrumentsusceptibletosubjectivity |