Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading Competence
In recent decades there have been significant changes in the conceptualization of reading as well as in the perception of how this activity should be assessed. Interest in the analysis of reading processes has led to the emergence of new explanatory models based primarily on the contributions of cog...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2018-08-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fpsyg.2018.01492/full |
| _version_ | 1828860825678381056 |
|---|---|
| author | Juan-José Navarro Juan-José Navarro Juan-José Navarro Catalina Mourgues-Codern Eduardo Guzmán Isabel R. Rodríguez-Ortiz Ricardo Conejo Claudia Sánchez-Gutiérrez Jesús de la Fuente Jesús de la Fuente Diana Martella Mahia Saracostti |
| author_facet | Juan-José Navarro Juan-José Navarro Juan-José Navarro Catalina Mourgues-Codern Eduardo Guzmán Isabel R. Rodríguez-Ortiz Ricardo Conejo Claudia Sánchez-Gutiérrez Jesús de la Fuente Jesús de la Fuente Diana Martella Mahia Saracostti |
| author_sort | Juan-José Navarro |
| collection | DOAJ |
| description | In recent decades there have been significant changes in the conceptualization of reading as well as in the perception of how this activity should be assessed. Interest in the analysis of reading processes has led to the emergence of new explanatory models based primarily on the contributions of cognitive psychology. In parallel, there have been notable advances in measurement procedures, especially in models based on Item Response Theory (IRT), as well as in the capacity and performance of specific software programs that allow data to be managed and analyzed. These changes have contributed significantly to the rise of testing procedures such as computerized adaptive tests (CATs), whose fundamental characteristic is that the sequence of items presented in the tests is adapted to the level of competence that the subject manifests. Likewise, the incorporation of elements of dynamic assessment (DA) as the prompts are gradually offered allows for obtaining information about the type and degree of support required to optimize the subject’s performance. In this sense, the confluence of contributions from DA and CATs offers a new possibility for approaching the assessment of learning processes. In this article, we present a longitudinal research developed in two phases, through which a computerized dynamic adaptive assessment battery of reading processes (EDPL-BAI) was configured. The research frame involved 1,831 students (46% girls) from 13 public schools in three regions of Chile. The purpose of this study was to analyze the differential contribution on reading competence of dynamic scores obtained in a subsample composed of 324 (47% girls) students from third to sixth grade after the implementation of a set of adaptive dynamic tests of morpho-syntactic processes. The results achieved in the structural equation modeling indicate a good global fit. Individual relationships show a significant contribution of calibrated score that reflects estimated knowledge level on reading competence, as well as dynamic scores based on the assigned value of graduated prompts required by the students. These results showed significant predictive values on reading competence and incremental validity in relation to predictions made by static criterion tests. |
| first_indexed | 2024-12-13T02:48:05Z |
| format | Article |
| id | doaj.art-86ae7c6e5eff42c584fb49f3061feb1e |
| institution | Directory Open Access Journal |
| issn | 1664-1078 |
| language | English |
| last_indexed | 2024-12-13T02:48:05Z |
| publishDate | 2018-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj.art-86ae7c6e5eff42c584fb49f3061feb1e2022-12-22T00:02:08ZengFrontiers Media S.A.Frontiers in Psychology1664-10782018-08-01910.3389/fpsyg.2018.01492406140Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading CompetenceJuan-José Navarro0Juan-José Navarro1Juan-José Navarro2Catalina Mourgues-Codern3Eduardo Guzmán4Isabel R. Rodríguez-Ortiz5Ricardo Conejo6Claudia Sánchez-Gutiérrez7Jesús de la Fuente8Jesús de la Fuente9Diana Martella10Mahia Saracostti11Department of Psychology and Sociology, Universidad de Zaragoza, Zaragoza, SpainFacultad de Educación, Universidad Autónoma de Chile, Santiago, ChileDepartamento de Psicología Evolutiva y de la Educación, Universidad de Sevilla, Seville, SpainLaboratoire Adaptations-Travail-Individu, Université Paris Descartes, Paris, FranceDepartamento de Lenguajes y Ciencias de la Computación, Universidad de Málaga, Málaga, SpainDepartamento de Psicología Evolutiva y de la Educación, Universidad de Sevilla, Seville, SpainDepartamento de Lenguajes y Ciencias de la Computación, Universidad de Málaga, Málaga, SpainSpanish and Portuguese Department, University of California, Davis, Davis, CA, United StatesDepartamento de Psicología Evolutiva y de la Educación, University of Almería, Almería, SpainFacultad de Psicología, Universidad Autónoma de Chile, Santiago, ChileFacultad de Psicología, Universidad Autónoma de Chile, Santiago, ChileCentro de Investigación sobre Procesos Socioeducativos, Familias y Comunidades, Universidad de La Frontera, Temuco, ChileIn recent decades there have been significant changes in the conceptualization of reading as well as in the perception of how this activity should be assessed. Interest in the analysis of reading processes has led to the emergence of new explanatory models based primarily on the contributions of cognitive psychology. In parallel, there have been notable advances in measurement procedures, especially in models based on Item Response Theory (IRT), as well as in the capacity and performance of specific software programs that allow data to be managed and analyzed. These changes have contributed significantly to the rise of testing procedures such as computerized adaptive tests (CATs), whose fundamental characteristic is that the sequence of items presented in the tests is adapted to the level of competence that the subject manifests. Likewise, the incorporation of elements of dynamic assessment (DA) as the prompts are gradually offered allows for obtaining information about the type and degree of support required to optimize the subject’s performance. In this sense, the confluence of contributions from DA and CATs offers a new possibility for approaching the assessment of learning processes. In this article, we present a longitudinal research developed in two phases, through which a computerized dynamic adaptive assessment battery of reading processes (EDPL-BAI) was configured. The research frame involved 1,831 students (46% girls) from 13 public schools in three regions of Chile. The purpose of this study was to analyze the differential contribution on reading competence of dynamic scores obtained in a subsample composed of 324 (47% girls) students from third to sixth grade after the implementation of a set of adaptive dynamic tests of morpho-syntactic processes. The results achieved in the structural equation modeling indicate a good global fit. Individual relationships show a significant contribution of calibrated score that reflects estimated knowledge level on reading competence, as well as dynamic scores based on the assigned value of graduated prompts required by the students. These results showed significant predictive values on reading competence and incremental validity in relation to predictions made by static criterion tests.https://www.frontiersin.org/article/10.3389/fpsyg.2018.01492/fulldynamic assessmentcomputerized adaptive testingitem response theorylearning potentialgraduated promptsreading processes |
| spellingShingle | Juan-José Navarro Juan-José Navarro Juan-José Navarro Catalina Mourgues-Codern Eduardo Guzmán Isabel R. Rodríguez-Ortiz Ricardo Conejo Claudia Sánchez-Gutiérrez Jesús de la Fuente Jesús de la Fuente Diana Martella Mahia Saracostti Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading Competence Frontiers in Psychology dynamic assessment computerized adaptive testing item response theory learning potential graduated prompts reading processes |
| title | Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading Competence |
| title_full | Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading Competence |
| title_fullStr | Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading Competence |
| title_full_unstemmed | Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading Competence |
| title_short | Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading Competence |
| title_sort | integrating curriculum based dynamic assessment in computerized adaptive testing development and predictive validity of the edpl bai battery on reading competence |
| topic | dynamic assessment computerized adaptive testing item response theory learning potential graduated prompts reading processes |
| url | https://www.frontiersin.org/article/10.3389/fpsyg.2018.01492/full |
| work_keys_str_mv | AT juanjosenavarro integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence AT juanjosenavarro integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence AT juanjosenavarro integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence AT catalinamourguescodern integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence AT eduardoguzman integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence AT isabelrrodriguezortiz integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence AT ricardoconejo integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence AT claudiasanchezgutierrez integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence AT jesusdelafuente integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence AT jesusdelafuente integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence AT dianamartella integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence AT mahiasaracostti integratingcurriculumbaseddynamicassessmentincomputerizedadaptivetestingdevelopmentandpredictivevalidityoftheedplbaibatteryonreadingcompetence |