Validation of newly developed tasks for the assessment of generic Critical Online Reasoning (COR) of university students and graduates
In recent decades, the acquisition of information has evolved substantially and fundamentally affects students’ use of information, so that the Internet has become one of the most important sources of information for learning. However, learning with freely accessible online resources also poses chal...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-11-01
|
| Series: | Frontiers in Education |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/feduc.2022.914857/full |
| _version_ | 1811330514969165824 |
|---|---|
| author | Marie-Theres Nagel Olga Zlatkin-Troitschanskaia Jennifer Fischer |
| author_facet | Marie-Theres Nagel Olga Zlatkin-Troitschanskaia Jennifer Fischer |
| author_sort | Marie-Theres Nagel |
| collection | DOAJ |
| description | In recent decades, the acquisition of information has evolved substantially and fundamentally affects students’ use of information, so that the Internet has become one of the most important sources of information for learning. However, learning with freely accessible online resources also poses challenges, such as vast amounts of partially unstructured, untrustworthy, or biased information. To successfully learn by using the Internet, students therefore require specific skills for selecting, processing, and evaluating the online information, e.g., to distinguish trustworthy from distorted or biased information and for judging its relevance with regard to the topic and task at hand. Despite the central importance of these skills, their assessment in higher education is still an emerging field. In this paper, we present the newly defined theoretical-conceptual framework Critical Online Reasoning (COR). Based on this framework, a corresponding performance assessment, Critical Online Reasoning Assessment (CORA), was newly developed and underwent first steps of validation in accordance with the Standards for Educational and Psychological Testing. We first provide an overview of the previous validation results and then expand them by including further analyses of the validity aspects “internal test structure” and “relations with other variables”. To investigate the internal test structure, we conducted variance component analyses based on the generalizability theory with a sample of 125 students and investigated the relations with other variables by means of correlation analyses. The results show correlations with external criteria as expected and confirm that the CORA scores reflect the different test performances of the participants and are not significantly biased by modalities of the assessment. With these new analyses, this study substantially contributes to previous research by providing comprehensive evidence for the validity of this new performance assessment that validly assesses the complex multifaceted construct of critical online reasoning among university students and graduates. CORA results provide unique insights into the interplay between features of online information acquisition and processing, learning environments, and the cognitive and metacognitive requirements for critically reasoning from online information in university students and young professionals. |
| first_indexed | 2024-04-13T16:02:52Z |
| format | Article |
| id | doaj.art-3acc34e2bd484968b03bf9614285d3a2 |
| institution | Directory Open Access Journal |
| issn | 2504-284X |
| language | English |
| last_indexed | 2024-04-13T16:02:52Z |
| publishDate | 2022-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Education |
| spelling | doaj.art-3acc34e2bd484968b03bf9614285d3a22022-12-22T02:40:28ZengFrontiers Media S.A.Frontiers in Education2504-284X2022-11-01710.3389/feduc.2022.914857914857Validation of newly developed tasks for the assessment of generic Critical Online Reasoning (COR) of university students and graduatesMarie-Theres NagelOlga Zlatkin-TroitschanskaiaJennifer FischerIn recent decades, the acquisition of information has evolved substantially and fundamentally affects students’ use of information, so that the Internet has become one of the most important sources of information for learning. However, learning with freely accessible online resources also poses challenges, such as vast amounts of partially unstructured, untrustworthy, or biased information. To successfully learn by using the Internet, students therefore require specific skills for selecting, processing, and evaluating the online information, e.g., to distinguish trustworthy from distorted or biased information and for judging its relevance with regard to the topic and task at hand. Despite the central importance of these skills, their assessment in higher education is still an emerging field. In this paper, we present the newly defined theoretical-conceptual framework Critical Online Reasoning (COR). Based on this framework, a corresponding performance assessment, Critical Online Reasoning Assessment (CORA), was newly developed and underwent first steps of validation in accordance with the Standards for Educational and Psychological Testing. We first provide an overview of the previous validation results and then expand them by including further analyses of the validity aspects “internal test structure” and “relations with other variables”. To investigate the internal test structure, we conducted variance component analyses based on the generalizability theory with a sample of 125 students and investigated the relations with other variables by means of correlation analyses. The results show correlations with external criteria as expected and confirm that the CORA scores reflect the different test performances of the participants and are not significantly biased by modalities of the assessment. With these new analyses, this study substantially contributes to previous research by providing comprehensive evidence for the validity of this new performance assessment that validly assesses the complex multifaceted construct of critical online reasoning among university students and graduates. CORA results provide unique insights into the interplay between features of online information acquisition and processing, learning environments, and the cognitive and metacognitive requirements for critically reasoning from online information in university students and young professionals.https://www.frontiersin.org/articles/10.3389/feduc.2022.914857/fullcritical online reasoningonline informationweb searchvalidationperformance assessmenthigher education |
| spellingShingle | Marie-Theres Nagel Olga Zlatkin-Troitschanskaia Jennifer Fischer Validation of newly developed tasks for the assessment of generic Critical Online Reasoning (COR) of university students and graduates Frontiers in Education critical online reasoning online information web search validation performance assessment higher education |
| title | Validation of newly developed tasks for the assessment of generic Critical Online Reasoning (COR) of university students and graduates |
| title_full | Validation of newly developed tasks for the assessment of generic Critical Online Reasoning (COR) of university students and graduates |
| title_fullStr | Validation of newly developed tasks for the assessment of generic Critical Online Reasoning (COR) of university students and graduates |
| title_full_unstemmed | Validation of newly developed tasks for the assessment of generic Critical Online Reasoning (COR) of university students and graduates |
| title_short | Validation of newly developed tasks for the assessment of generic Critical Online Reasoning (COR) of university students and graduates |
| title_sort | validation of newly developed tasks for the assessment of generic critical online reasoning cor of university students and graduates |
| topic | critical online reasoning online information web search validation performance assessment higher education |
| url | https://www.frontiersin.org/articles/10.3389/feduc.2022.914857/full |
| work_keys_str_mv | AT marietheresnagel validationofnewlydevelopedtasksfortheassessmentofgenericcriticalonlinereasoningcorofuniversitystudentsandgraduates AT olgazlatkintroitschanskaia validationofnewlydevelopedtasksfortheassessmentofgenericcriticalonlinereasoningcorofuniversitystudentsandgraduates AT jenniferfischer validationofnewlydevelopedtasksfortheassessmentofgenericcriticalonlinereasoningcorofuniversitystudentsandgraduates |