Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015
Abstract Process data are becoming more and more popular in education research. In the field of computer-based assessments of collaborative problem solving (ColPS), process data have been used to identify students’ test-taking strategies while working on the assessment, and such data can be used to...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2023-11-01
|
| Series: | Large-scale Assessments in Education |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40536-023-00185-5 |
| _version_ | 1797557602531409920 |
|---|---|
| author | Areum Han Florian Krieger Francesca Borgonovi Samuel Greiff |
| author_facet | Areum Han Florian Krieger Francesca Borgonovi Samuel Greiff |
| author_sort | Areum Han |
| collection | DOAJ |
| description | Abstract Process data are becoming more and more popular in education research. In the field of computer-based assessments of collaborative problem solving (ColPS), process data have been used to identify students’ test-taking strategies while working on the assessment, and such data can be used to complement data collected on accuracy and overall performance. Such information can be used to understand, for example, whether students are able to use a range of styles and strategies to solve different problems, given evidence that such cognitive flexibility may be important in labor markets and societies. In addition, process information might help researchers better identify the determinants of poor performance and interventions that can help students succeed. However, this line of research, particularly research that uses these data to profile students, is still in its infancy and has mostly been centered on small- to medium-scale collaboration settings between people (i.e., the human-to-human approach). There are only a few studies involving large-scale assessments of ColPS between a respondent and computer agents (i.e., the human-to-agent approach), where problem spaces are more standardized and fewer biases and confounds exist. In this study, we investigated students’ ColPS behavioral patterns using latent profile analyses (LPA) based on two types of process data (i.e., response times and the number of actions) collected from the Program for International Student Assessment (PISA) 2015 ColPS assessment, a large-scale international assessment of the human-to-agent approach. Analyses were conducted on test-takers who: (a) were administered the assessment in English and (b) were assigned the Xandar unit at the beginning of the test. The total sample size was N = 2,520. Analyses revealed two profiles (i.e., Profile 1 [95%] vs. Profile 2 [5%]) showing different behavioral characteristics across the four parts of the assessment unit. Significant differences were also found in overall performance between the profiles. |
| first_indexed | 2024-03-10T17:19:11Z |
| format | Article |
| id | doaj.art-51fe8f5b076d484ea094505fe3f283c2 |
| institution | Directory Open Access Journal |
| issn | 2196-0739 |
| language | English |
| last_indexed | 2024-03-10T17:19:11Z |
| publishDate | 2023-11-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Large-scale Assessments in Education |
| spelling | doaj.art-51fe8f5b076d484ea094505fe3f283c22023-11-20T10:22:44ZengSpringerOpenLarge-scale Assessments in Education2196-07392023-11-0111112810.1186/s40536-023-00185-5Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015Areum Han0Florian Krieger1Francesca Borgonovi2Samuel Greiff3Department of Behavioral and Cognitive Sciences, University of LuxembourgDepartment of Rehabilitation Sciences, Technical University of DortmundInstitute of Education, University College LondonDepartment of Behavioral and Cognitive Sciences, University of LuxembourgAbstract Process data are becoming more and more popular in education research. In the field of computer-based assessments of collaborative problem solving (ColPS), process data have been used to identify students’ test-taking strategies while working on the assessment, and such data can be used to complement data collected on accuracy and overall performance. Such information can be used to understand, for example, whether students are able to use a range of styles and strategies to solve different problems, given evidence that such cognitive flexibility may be important in labor markets and societies. In addition, process information might help researchers better identify the determinants of poor performance and interventions that can help students succeed. However, this line of research, particularly research that uses these data to profile students, is still in its infancy and has mostly been centered on small- to medium-scale collaboration settings between people (i.e., the human-to-human approach). There are only a few studies involving large-scale assessments of ColPS between a respondent and computer agents (i.e., the human-to-agent approach), where problem spaces are more standardized and fewer biases and confounds exist. In this study, we investigated students’ ColPS behavioral patterns using latent profile analyses (LPA) based on two types of process data (i.e., response times and the number of actions) collected from the Program for International Student Assessment (PISA) 2015 ColPS assessment, a large-scale international assessment of the human-to-agent approach. Analyses were conducted on test-takers who: (a) were administered the assessment in English and (b) were assigned the Xandar unit at the beginning of the test. The total sample size was N = 2,520. Analyses revealed two profiles (i.e., Profile 1 [95%] vs. Profile 2 [5%]) showing different behavioral characteristics across the four parts of the assessment unit. Significant differences were also found in overall performance between the profiles.https://doi.org/10.1186/s40536-023-00185-5PISA2015Latent profile analysisCollaborative problem solvingProcess dataHuman-to-agent assessment |
| spellingShingle | Areum Han Florian Krieger Francesca Borgonovi Samuel Greiff Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015 Large-scale Assessments in Education PISA2015 Latent profile analysis Collaborative problem solving Process data Human-to-agent assessment |
| title | Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015 |
| title_full | Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015 |
| title_fullStr | Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015 |
| title_full_unstemmed | Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015 |
| title_short | Behavioral patterns in collaborative problem solving: a latent profile analysis based on response times and actions in PISA 2015 |
| title_sort | behavioral patterns in collaborative problem solving a latent profile analysis based on response times and actions in pisa 2015 |
| topic | PISA2015 Latent profile analysis Collaborative problem solving Process data Human-to-agent assessment |
| url | https://doi.org/10.1186/s40536-023-00185-5 |
| work_keys_str_mv | AT areumhan behavioralpatternsincollaborativeproblemsolvingalatentprofileanalysisbasedonresponsetimesandactionsinpisa2015 AT floriankrieger behavioralpatternsincollaborativeproblemsolvingalatentprofileanalysisbasedonresponsetimesandactionsinpisa2015 AT francescaborgonovi behavioralpatternsincollaborativeproblemsolvingalatentprofileanalysisbasedonresponsetimesandactionsinpisa2015 AT samuelgreiff behavioralpatternsincollaborativeproblemsolvingalatentprofileanalysisbasedonresponsetimesandactionsinpisa2015 |