A distributed real-time recommender system for big data streams
Recommender Systems (RS) play a crucial role in our lives. As users become continuously connected to the internet, they are less tolerant of obsolete recommendations made by an RS. Online RS has to address three requirements: continuous training and recommendation, handling concept drifts, and the a...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-08-01
|
| Series: | Ain Shams Engineering Journal |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2090447922003379 |
| _version_ | 1797826885233672192 |
|---|---|
| author | Heidy Hazem Ahmed Awad Ahmed Hassan Yousef |
| author_facet | Heidy Hazem Ahmed Awad Ahmed Hassan Yousef |
| author_sort | Heidy Hazem |
| collection | DOAJ |
| description | Recommender Systems (RS) play a crucial role in our lives. As users become continuously connected to the internet, they are less tolerant of obsolete recommendations made by an RS. Online RS has to address three requirements: continuous training and recommendation, handling concept drifts, and the ability to scale. Streaming RS proposed in the literature address the first two requirements only. That is because they run the training process on a single machine.To tackle the third challenge, we propose a Splitting and Replication mechanism for distributed streaming RS. Our mechanism is inspired by the shared-nothing architecture that underpins contemporary big data processing systems. We have applied our mechanism to two well-known approaches for online RS, namely, matrix factorization and item-based collaborative filtering. We conducted experiments comparing the performance with the baseline (single machine). Evaluating different data sets, experiments show online recall improvement by 40% with more than 50% less memory consumption. |
| first_indexed | 2024-04-09T12:39:26Z |
| format | Article |
| id | doaj.art-26a171572ce249a59fde70862aeae04f |
| institution | Directory Open Access Journal |
| issn | 2090-4479 |
| language | English |
| last_indexed | 2024-04-09T12:39:26Z |
| publishDate | 2023-08-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Ain Shams Engineering Journal |
| spelling | doaj.art-26a171572ce249a59fde70862aeae04f2023-05-15T04:14:13ZengElsevierAin Shams Engineering Journal2090-44792023-08-01148102026A distributed real-time recommender system for big data streamsHeidy Hazem0Ahmed Awad1Ahmed Hassan Yousef2Nile University, Giza, EgyptTartu University, Tartu, Estonia; Cairo University, Giza, Egypt; Corresponding author at: Narva Road 18 51009 Tartu City, Tartu City, Tartu County, Estonia.Nile University, Giza, EgyptRecommender Systems (RS) play a crucial role in our lives. As users become continuously connected to the internet, they are less tolerant of obsolete recommendations made by an RS. Online RS has to address three requirements: continuous training and recommendation, handling concept drifts, and the ability to scale. Streaming RS proposed in the literature address the first two requirements only. That is because they run the training process on a single machine.To tackle the third challenge, we propose a Splitting and Replication mechanism for distributed streaming RS. Our mechanism is inspired by the shared-nothing architecture that underpins contemporary big data processing systems. We have applied our mechanism to two well-known approaches for online RS, namely, matrix factorization and item-based collaborative filtering. We conducted experiments comparing the performance with the baseline (single machine). Evaluating different data sets, experiments show online recall improvement by 40% with more than 50% less memory consumption.http://www.sciencedirect.com/science/article/pii/S2090447922003379StreamingBig dataOnline Recommender Systems |
| spellingShingle | Heidy Hazem Ahmed Awad Ahmed Hassan Yousef A distributed real-time recommender system for big data streams Ain Shams Engineering Journal Streaming Big data Online Recommender Systems |
| title | A distributed real-time recommender system for big data streams |
| title_full | A distributed real-time recommender system for big data streams |
| title_fullStr | A distributed real-time recommender system for big data streams |
| title_full_unstemmed | A distributed real-time recommender system for big data streams |
| title_short | A distributed real-time recommender system for big data streams |
| title_sort | distributed real time recommender system for big data streams |
| topic | Streaming Big data Online Recommender Systems |
| url | http://www.sciencedirect.com/science/article/pii/S2090447922003379 |
| work_keys_str_mv | AT heidyhazem adistributedrealtimerecommendersystemforbigdatastreams AT ahmedawad adistributedrealtimerecommendersystemforbigdatastreams AT ahmedhassanyousef adistributedrealtimerecommendersystemforbigdatastreams AT heidyhazem distributedrealtimerecommendersystemforbigdatastreams AT ahmedawad distributedrealtimerecommendersystemforbigdatastreams AT ahmedhassanyousef distributedrealtimerecommendersystemforbigdatastreams |