Inter-rater reliability of healthcare professional skills' portfolio assessments: The Andalusian Agency for Healthcare Quality model
This study aims to determine the reliability of assessment criteria used for a portfolio at the Andalusian Agency for Healthcare Quality (ACSA). Data: all competences certification processes, regardless of their discipline. Period: 2010-2011. Three types of tests are used: 368 certificates, 17,895 r...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Medwave Estudios Limitada
2014-07-01
|
| Series: | Medwave |
| Subjects: | |
| Online Access: | http://www.medwave.cl/link.cgi/Medwave/Estudios/Investigacion/5998 |
| _version_ | 1818465797704515584 |
|---|---|
| author | Antonio Almuedo-Paz Manuel Herrera-Usagre Begoña Buiza-Camacho José Julián-Carrión María del Pilar Carrascosa-Salmoral Sheila María Martín-García Rocío Salguero-Cabalgante |
| author_facet | Antonio Almuedo-Paz Manuel Herrera-Usagre Begoña Buiza-Camacho José Julián-Carrión María del Pilar Carrascosa-Salmoral Sheila María Martín-García Rocío Salguero-Cabalgante |
| author_sort | Antonio Almuedo-Paz |
| collection | DOAJ |
| description | This study aims to determine the reliability of assessment criteria used for a portfolio at the Andalusian Agency for Healthcare Quality (ACSA). Data: all competences certification processes, regardless of their discipline. Period: 2010-2011. Three types of tests are used: 368 certificates, 17,895 reports and 22,642 clinical practice reports (N = 3,010 candidates). The tests were evaluated in pairs by the ACSA team of raters using two categories: valid and invalid. Results: The percentage agreement in assessments of certificates was 89,9%, while for the reports of clinical practice was 85,1 % and for clinical practice reports was 81,7%. The inter-rater agreement coefficients (kappa) ranged from 0,468 to 0,711. Discussion: The results of this study show that the inter-rater reliability of assessments varies from fair to good. Compared with other similar studies, the results put the reliability of the model in a comfortable position. Among the improvements incorporated, progressive automation of evaluations must be highlighted. |
| first_indexed | 2024-04-13T20:50:29Z |
| format | Article |
| id | doaj.art-0d92140972b44367b02de045e7ac299a |
| institution | Directory Open Access Journal |
| issn | 0717-6384 |
| language | English |
| last_indexed | 2024-04-13T20:50:29Z |
| publishDate | 2014-07-01 |
| publisher | Medwave Estudios Limitada |
| record_format | Article |
| series | Medwave |
| spelling | doaj.art-0d92140972b44367b02de045e7ac299a2022-12-22T02:30:31ZengMedwave Estudios LimitadaMedwave0717-63842014-07-011406e5998e599810.5867/medwave.2014.06.5998Inter-rater reliability of healthcare professional skills' portfolio assessments: The Andalusian Agency for Healthcare Quality modelAntonio Almuedo-Paz0Manuel Herrera-Usagre1Begoña Buiza-Camacho2José Julián-Carrión3María del Pilar Carrascosa-Salmoral4Sheila María Martín-García5Rocío Salguero-Cabalgante6Agencia de Calidad Sanitaria de Andalucía, EspañaAgencia de Calidad Sanitaria de Andalucía, España; Departamento de Sociología, Universidad de Sevilla, España Agencia de Calidad Sanitaria de Andalucía, EspañaAgencia de Calidad Sanitaria de Andalucía, EspañaAgencia de Calidad Sanitaria de Andalucía, EspañaAgencia de Calidad Sanitaria de Andalucía, EspañaAgencia de Calidad Sanitaria de Andalucía, EspañaThis study aims to determine the reliability of assessment criteria used for a portfolio at the Andalusian Agency for Healthcare Quality (ACSA). Data: all competences certification processes, regardless of their discipline. Period: 2010-2011. Three types of tests are used: 368 certificates, 17,895 reports and 22,642 clinical practice reports (N = 3,010 candidates). The tests were evaluated in pairs by the ACSA team of raters using two categories: valid and invalid. Results: The percentage agreement in assessments of certificates was 89,9%, while for the reports of clinical practice was 85,1 % and for clinical practice reports was 81,7%. The inter-rater agreement coefficients (kappa) ranged from 0,468 to 0,711. Discussion: The results of this study show that the inter-rater reliability of assessments varies from fair to good. Compared with other similar studies, the results put the reliability of the model in a comfortable position. Among the improvements incorporated, progressive automation of evaluations must be highlighted.http://www.medwave.cl/link.cgi/Medwave/Estudios/Investigacion/5998clinical competencesportfolio assessmentcertificationaccreditationquality assurance |
| spellingShingle | Antonio Almuedo-Paz Manuel Herrera-Usagre Begoña Buiza-Camacho José Julián-Carrión María del Pilar Carrascosa-Salmoral Sheila María Martín-García Rocío Salguero-Cabalgante Inter-rater reliability of healthcare professional skills' portfolio assessments: The Andalusian Agency for Healthcare Quality model Medwave clinical competences portfolio assessment certification accreditation quality assurance |
| title | Inter-rater reliability of healthcare professional skills' portfolio assessments: The Andalusian Agency for Healthcare Quality model |
| title_full | Inter-rater reliability of healthcare professional skills' portfolio assessments: The Andalusian Agency for Healthcare Quality model |
| title_fullStr | Inter-rater reliability of healthcare professional skills' portfolio assessments: The Andalusian Agency for Healthcare Quality model |
| title_full_unstemmed | Inter-rater reliability of healthcare professional skills' portfolio assessments: The Andalusian Agency for Healthcare Quality model |
| title_short | Inter-rater reliability of healthcare professional skills' portfolio assessments: The Andalusian Agency for Healthcare Quality model |
| title_sort | inter rater reliability of healthcare professional skills portfolio assessments the andalusian agency for healthcare quality model |
| topic | clinical competences portfolio assessment certification accreditation quality assurance |
| url | http://www.medwave.cl/link.cgi/Medwave/Estudios/Investigacion/5998 |
| work_keys_str_mv | AT antonioalmuedopaz interraterreliabilityofhealthcareprofessionalskillsportfolioassessmentstheandalusianagencyforhealthcarequalitymodel AT manuelherrerausagre interraterreliabilityofhealthcareprofessionalskillsportfolioassessmentstheandalusianagencyforhealthcarequalitymodel AT begonabuizacamacho interraterreliabilityofhealthcareprofessionalskillsportfolioassessmentstheandalusianagencyforhealthcarequalitymodel AT josejuliancarrion interraterreliabilityofhealthcareprofessionalskillsportfolioassessmentstheandalusianagencyforhealthcarequalitymodel AT mariadelpilarcarrascosasalmoral interraterreliabilityofhealthcareprofessionalskillsportfolioassessmentstheandalusianagencyforhealthcarequalitymodel AT sheilamariamartingarcia interraterreliabilityofhealthcareprofessionalskillsportfolioassessmentstheandalusianagencyforhealthcarequalitymodel AT rociosalguerocabalgante interraterreliabilityofhealthcareprofessionalskillsportfolioassessmentstheandalusianagencyforhealthcarequalitymodel |