The Reliability and Validity of a Paediatric Script Concordance Test with Medical Students, Paediatric Residents and Experienced Paediatricians
Background: The Script Concordance (SC) approach was used as an alternative test format to measure the presence of knowledge organization reflective in one’s clinical reasoning skills (i.e., diagnostic, investigation and treatment knowledge). Methods: The present study investigated the reliability...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Canadian Medical Education Journal
2010-07-01
|
| Series: | Canadian Medical Education Journal |
| Online Access: | https://journalhosting.ucalgary.ca/index.php/cmej/article/view/36531 |
| _version_ | 1818407192764612608 |
|---|---|
| author | Jean-Francois Lemay Tyrone Donnon Bernard Charlin |
| author_facet | Jean-Francois Lemay Tyrone Donnon Bernard Charlin |
| author_sort | Jean-Francois Lemay |
| collection | DOAJ |
| description | Background: The Script Concordance (SC) approach was used as an alternative test format to measure the presence of knowledge organization reflective in one’s clinical reasoning skills (i.e., diagnostic, investigation and treatment knowledge).
Methods: The present study investigated the reliability and validity of a 40-item paediatric version of the SC test with three groups representing 53 medical students (novices), 42 paediatric residents (intermediates) and 11 paediatricians (experts).
Results: A comparison between scoring techniques based on experts’ ratings of the items showed internal reliability coefficients from .74 for the one-best answer up to .78 for alternative scoring techniques. An ANOVA showed an increase in test performance from medical students through to expert paediatricians (F(2,103) = 84.05, p < .001), but did not differentiate between the postgraduate year 1 to 3 paediatric residents. A large effect size (Cohen’s d) difference of 1.06 was found between medical students and residents total SC test scores.
Conclusions: These results support other findings indicating the SC test format can be used to differentiate between the clinical reasoning skills of novices, intermediates and experts in paediatrics. An alternative scoring method that includes one best answer and partial marks was also supported for grading SC test items. |
| first_indexed | 2024-12-14T09:23:56Z |
| format | Article |
| id | doaj.art-5939ac792c58405eb0653f77a147ead8 |
| institution | Directory Open Access Journal |
| issn | 1923-1202 |
| language | English |
| last_indexed | 2024-12-14T09:23:56Z |
| publishDate | 2010-07-01 |
| publisher | Canadian Medical Education Journal |
| record_format | Article |
| series | Canadian Medical Education Journal |
| spelling | doaj.art-5939ac792c58405eb0653f77a147ead82022-12-21T23:08:13ZengCanadian Medical Education JournalCanadian Medical Education Journal1923-12022010-07-011223686The Reliability and Validity of a Paediatric Script Concordance Test with Medical Students, Paediatric Residents and Experienced PaediatriciansJean-Francois Lemay0Tyrone Donnon1Bernard Charlin2University of CalgaryUniversity of CalgaryUniversity of MontrealBackground: The Script Concordance (SC) approach was used as an alternative test format to measure the presence of knowledge organization reflective in one’s clinical reasoning skills (i.e., diagnostic, investigation and treatment knowledge). Methods: The present study investigated the reliability and validity of a 40-item paediatric version of the SC test with three groups representing 53 medical students (novices), 42 paediatric residents (intermediates) and 11 paediatricians (experts). Results: A comparison between scoring techniques based on experts’ ratings of the items showed internal reliability coefficients from .74 for the one-best answer up to .78 for alternative scoring techniques. An ANOVA showed an increase in test performance from medical students through to expert paediatricians (F(2,103) = 84.05, p < .001), but did not differentiate between the postgraduate year 1 to 3 paediatric residents. A large effect size (Cohen’s d) difference of 1.06 was found between medical students and residents total SC test scores. Conclusions: These results support other findings indicating the SC test format can be used to differentiate between the clinical reasoning skills of novices, intermediates and experts in paediatrics. An alternative scoring method that includes one best answer and partial marks was also supported for grading SC test items.https://journalhosting.ucalgary.ca/index.php/cmej/article/view/36531 |
| spellingShingle | Jean-Francois Lemay Tyrone Donnon Bernard Charlin The Reliability and Validity of a Paediatric Script Concordance Test with Medical Students, Paediatric Residents and Experienced Paediatricians Canadian Medical Education Journal |
| title | The Reliability and Validity of a Paediatric Script Concordance Test with Medical Students, Paediatric Residents and Experienced Paediatricians |
| title_full | The Reliability and Validity of a Paediatric Script Concordance Test with Medical Students, Paediatric Residents and Experienced Paediatricians |
| title_fullStr | The Reliability and Validity of a Paediatric Script Concordance Test with Medical Students, Paediatric Residents and Experienced Paediatricians |
| title_full_unstemmed | The Reliability and Validity of a Paediatric Script Concordance Test with Medical Students, Paediatric Residents and Experienced Paediatricians |
| title_short | The Reliability and Validity of a Paediatric Script Concordance Test with Medical Students, Paediatric Residents and Experienced Paediatricians |
| title_sort | reliability and validity of a paediatric script concordance test with medical students paediatric residents and experienced paediatricians |
| url | https://journalhosting.ucalgary.ca/index.php/cmej/article/view/36531 |
| work_keys_str_mv | AT jeanfrancoislemay thereliabilityandvalidityofapaediatricscriptconcordancetestwithmedicalstudentspaediatricresidentsandexperiencedpaediatricians AT tyronedonnon thereliabilityandvalidityofapaediatricscriptconcordancetestwithmedicalstudentspaediatricresidentsandexperiencedpaediatricians AT bernardcharlin thereliabilityandvalidityofapaediatricscriptconcordancetestwithmedicalstudentspaediatricresidentsandexperiencedpaediatricians AT jeanfrancoislemay reliabilityandvalidityofapaediatricscriptconcordancetestwithmedicalstudentspaediatricresidentsandexperiencedpaediatricians AT tyronedonnon reliabilityandvalidityofapaediatricscriptconcordancetestwithmedicalstudentspaediatricresidentsandexperiencedpaediatricians AT bernardcharlin reliabilityandvalidityofapaediatricscriptconcordancetestwithmedicalstudentspaediatricresidentsandexperiencedpaediatricians |