Predicting Cognitive Presence in At-Scale Online Learning: MOOC and For-Credit Online Course Environments
In this study, we work towards a strategy to measure and enhance the quality of interactions in discussion forums at scale. We present a machine learning (ML) model which identifies the phase of cognitive presence exhibited by a student’s post and suggest future applications of such a model to help...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Online Learning Consortium
2022-03-01
|
| Series: | Online Learning |
| Subjects: | |
| Online Access: | https://olj.onlinelearningconsortium.org/index.php/olj/article/view/3060 |
| _version_ | 1797327789377978368 |
|---|---|
| author | Jeonghyun Lee Farahnaz Soleimani India Irish John Hosmer, IV Meryem Yilmaz Soylu Roy Finkelberg Saurabh Chatterjee |
| author_facet | Jeonghyun Lee Farahnaz Soleimani India Irish John Hosmer, IV Meryem Yilmaz Soylu Roy Finkelberg Saurabh Chatterjee |
| author_sort | Jeonghyun Lee |
| collection | DOAJ |
| description |
In this study, we work towards a strategy to measure and enhance the quality of interactions in discussion forums at scale. We present a machine learning (ML) model which identifies the phase of cognitive presence exhibited by a student’s post and suggest future applications of such a model to help online students develop higher-order thinking. We collect discussion forum transcript data from two online courses: CS1301 (an introductory computer programming MOOC) offered by edX and CS6601 (a graduate course on artificial intelligence) which uses the Piazza online discussion tool. We manually code a random sample of students’ posts based on the Community of Inquiry coding scheme and explore trends in cognitive presence within and across the courses. We further use this coded data to analyze the relationship between students’ observed cognitive presence and course grades. In terms of testing and building an ML model, we use a Bidirectional Encoder Representations from Transformers model that uses a deep learning technique to train large text corpus and fine-tune the language model. Our results suggest that deeper cognitive engagement with course concepts, as expressed by higher cognitive presence, are associated with better learning outcomes for students in both course settings. Our ML approach achieves 92.5% accuracy on the classification task, motivating the use of ML for instructional interventions in online courses. We expect that our research study will not only contribute to extending the literature on cognitive presence but also have a beneficial impact on online instructors or curriculum developers in higher education.
|
| first_indexed | 2024-03-08T06:43:35Z |
| format | Article |
| id | doaj.art-b568bdc6a7db4b9db5e2e50265dafbd1 |
| institution | Directory Open Access Journal |
| issn | 2472-5749 2472-5730 |
| language | English |
| last_indexed | 2024-03-08T06:43:35Z |
| publishDate | 2022-03-01 |
| publisher | Online Learning Consortium |
| record_format | Article |
| series | Online Learning |
| spelling | doaj.art-b568bdc6a7db4b9db5e2e50265dafbd12024-02-03T08:25:22ZengOnline Learning ConsortiumOnline Learning2472-57492472-57302022-03-0126110.24059/olj.v26i1.3060Predicting Cognitive Presence in At-Scale Online Learning: MOOC and For-Credit Online Course EnvironmentsJeonghyun Lee0Farahnaz Soleimani1India Irish2John Hosmer, IV3Meryem Yilmaz Soylu4Roy Finkelberg5Saurabh Chatterjee6Georgia Institute of TechnologyGeorgia Institute of TechnologyGeorgia Institute of TechnologyGeorgia Institute of TechnologyGeorgia Institute of TechnologyGeorgia Institute of TechnologyGeorgia Institute of Technology In this study, we work towards a strategy to measure and enhance the quality of interactions in discussion forums at scale. We present a machine learning (ML) model which identifies the phase of cognitive presence exhibited by a student’s post and suggest future applications of such a model to help online students develop higher-order thinking. We collect discussion forum transcript data from two online courses: CS1301 (an introductory computer programming MOOC) offered by edX and CS6601 (a graduate course on artificial intelligence) which uses the Piazza online discussion tool. We manually code a random sample of students’ posts based on the Community of Inquiry coding scheme and explore trends in cognitive presence within and across the courses. We further use this coded data to analyze the relationship between students’ observed cognitive presence and course grades. In terms of testing and building an ML model, we use a Bidirectional Encoder Representations from Transformers model that uses a deep learning technique to train large text corpus and fine-tune the language model. Our results suggest that deeper cognitive engagement with course concepts, as expressed by higher cognitive presence, are associated with better learning outcomes for students in both course settings. Our ML approach achieves 92.5% accuracy on the classification task, motivating the use of ML for instructional interventions in online courses. We expect that our research study will not only contribute to extending the literature on cognitive presence but also have a beneficial impact on online instructors or curriculum developers in higher education. https://olj.onlinelearningconsortium.org/index.php/olj/article/view/3060cognitive presencediscussion forumsmachine learninghigher education |
| spellingShingle | Jeonghyun Lee Farahnaz Soleimani India Irish John Hosmer, IV Meryem Yilmaz Soylu Roy Finkelberg Saurabh Chatterjee Predicting Cognitive Presence in At-Scale Online Learning: MOOC and For-Credit Online Course Environments Online Learning cognitive presence discussion forums machine learning higher education |
| title | Predicting Cognitive Presence in At-Scale Online Learning: MOOC and For-Credit Online Course Environments |
| title_full | Predicting Cognitive Presence in At-Scale Online Learning: MOOC and For-Credit Online Course Environments |
| title_fullStr | Predicting Cognitive Presence in At-Scale Online Learning: MOOC and For-Credit Online Course Environments |
| title_full_unstemmed | Predicting Cognitive Presence in At-Scale Online Learning: MOOC and For-Credit Online Course Environments |
| title_short | Predicting Cognitive Presence in At-Scale Online Learning: MOOC and For-Credit Online Course Environments |
| title_sort | predicting cognitive presence in at scale online learning mooc and for credit online course environments |
| topic | cognitive presence discussion forums machine learning higher education |
| url | https://olj.onlinelearningconsortium.org/index.php/olj/article/view/3060 |
| work_keys_str_mv | AT jeonghyunlee predictingcognitivepresenceinatscaleonlinelearningmoocandforcreditonlinecourseenvironments AT farahnazsoleimani predictingcognitivepresenceinatscaleonlinelearningmoocandforcreditonlinecourseenvironments AT indiairish predictingcognitivepresenceinatscaleonlinelearningmoocandforcreditonlinecourseenvironments AT johnhosmeriv predictingcognitivepresenceinatscaleonlinelearningmoocandforcreditonlinecourseenvironments AT meryemyilmazsoylu predictingcognitivepresenceinatscaleonlinelearningmoocandforcreditonlinecourseenvironments AT royfinkelberg predictingcognitivepresenceinatscaleonlinelearningmoocandforcreditonlinecourseenvironments AT saurabhchatterjee predictingcognitivepresenceinatscaleonlinelearningmoocandforcreditonlinecourseenvironments |