Large-scale e-learning recommender system based on Spark and Hadoop
Abstract The present work is a part of the ESTenLigne project which is the result of several years of experience for developing e-learning in Sidi Mohamed Ben Abdellah University through the implementation of open, online and adaptive learning environment. However, this platform faces many challenge...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2019-01-01
|
| Series: | Journal of Big Data |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s40537-019-0169-4 |
| _version_ | 1819023063023353856 |
|---|---|
| author | Karim Dahdouh Ahmed Dakkak Lahcen Oughdir Abdelali Ibriz |
| author_facet | Karim Dahdouh Ahmed Dakkak Lahcen Oughdir Abdelali Ibriz |
| author_sort | Karim Dahdouh |
| collection | DOAJ |
| description | Abstract The present work is a part of the ESTenLigne project which is the result of several years of experience for developing e-learning in Sidi Mohamed Ben Abdellah University through the implementation of open, online and adaptive learning environment. However, this platform faces many challenges, such as the increasing amount of data, the diversity of pedagogical resources and a large number of learners that makes harder to find what the learners are really looking for. Furthermore, most of the students in this platform are new graduates who have just come to integrate higher education and who need a system to help them to take the relevant courses that take into account the requirements and needs of each learner. In this article, we develop a distributed courses recommender system for the e-learning platform. It aims to discover relationships between student’s activities using association rules method in order to help the student to choose the most appropriate learning materials. We also focus on the analysis of past historical data of the courses enrollments or log data. The article discusses particularly the frequent itemsets concept to determine the interesting rules in the transaction database. Then, we use the extracted rules to find the catalog of more suitable courses according to the learner’s behaviors and preferences. Next, we deploy our recommender system using big data technologies and techniques. Especially, we implement parallel FP-growth algorithm provided by Spark Framework and Hadoop ecosystem. The experimental results show the effectiveness and scalability of the proposed system. Finally, we evaluate the performance of Spark MLlib library compared to traditional machine learning tools including Weka and R. |
| first_indexed | 2024-12-21T04:32:56Z |
| format | Article |
| id | doaj.art-f1ad88279e0f407f946f3aae344c7144 |
| institution | Directory Open Access Journal |
| issn | 2196-1115 |
| language | English |
| last_indexed | 2024-12-21T04:32:56Z |
| publishDate | 2019-01-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Journal of Big Data |
| spelling | doaj.art-f1ad88279e0f407f946f3aae344c71442022-12-21T19:15:54ZengSpringerOpenJournal of Big Data2196-11152019-01-016112310.1186/s40537-019-0169-4Large-scale e-learning recommender system based on Spark and HadoopKarim Dahdouh0Ahmed Dakkak1Lahcen Oughdir2Abdelali Ibriz3Engineering Sciences Laboratory, FPT, Sidi Mohamed Ben Abdellah UniversityEngineering Sciences Laboratory, FPT, Sidi Mohamed Ben Abdellah UniversityEngineering Sciences Laboratory, FPT, Sidi Mohamed Ben Abdellah UniversityHigh School of Technology, Sidi Mohamed Ben Abdellah UniversityAbstract The present work is a part of the ESTenLigne project which is the result of several years of experience for developing e-learning in Sidi Mohamed Ben Abdellah University through the implementation of open, online and adaptive learning environment. However, this platform faces many challenges, such as the increasing amount of data, the diversity of pedagogical resources and a large number of learners that makes harder to find what the learners are really looking for. Furthermore, most of the students in this platform are new graduates who have just come to integrate higher education and who need a system to help them to take the relevant courses that take into account the requirements and needs of each learner. In this article, we develop a distributed courses recommender system for the e-learning platform. It aims to discover relationships between student’s activities using association rules method in order to help the student to choose the most appropriate learning materials. We also focus on the analysis of past historical data of the courses enrollments or log data. The article discusses particularly the frequent itemsets concept to determine the interesting rules in the transaction database. Then, we use the extracted rules to find the catalog of more suitable courses according to the learner’s behaviors and preferences. Next, we deploy our recommender system using big data technologies and techniques. Especially, we implement parallel FP-growth algorithm provided by Spark Framework and Hadoop ecosystem. The experimental results show the effectiveness and scalability of the proposed system. Finally, we evaluate the performance of Spark MLlib library compared to traditional machine learning tools including Weka and R.http://link.springer.com/article/10.1186/s40537-019-0169-4Big dataSparkHadoopE-learningOnline learningCourse recommender system |
| spellingShingle | Karim Dahdouh Ahmed Dakkak Lahcen Oughdir Abdelali Ibriz Large-scale e-learning recommender system based on Spark and Hadoop Journal of Big Data Big data Spark Hadoop E-learning Online learning Course recommender system |
| title | Large-scale e-learning recommender system based on Spark and Hadoop |
| title_full | Large-scale e-learning recommender system based on Spark and Hadoop |
| title_fullStr | Large-scale e-learning recommender system based on Spark and Hadoop |
| title_full_unstemmed | Large-scale e-learning recommender system based on Spark and Hadoop |
| title_short | Large-scale e-learning recommender system based on Spark and Hadoop |
| title_sort | large scale e learning recommender system based on spark and hadoop |
| topic | Big data Spark Hadoop E-learning Online learning Course recommender system |
| url | http://link.springer.com/article/10.1186/s40537-019-0169-4 |
| work_keys_str_mv | AT karimdahdouh largescaleelearningrecommendersystembasedonsparkandhadoop AT ahmeddakkak largescaleelearningrecommendersystembasedonsparkandhadoop AT lahcenoughdir largescaleelearningrecommendersystembasedonsparkandhadoop AT abdelaliibriz largescaleelearningrecommendersystembasedonsparkandhadoop |