Multi-objective deep reinforcement learning for recommendation systems
Most existing recommendation systems (RSs) are primarily concerned about the accuracy of rating prediction and only recommending popular items. However, other non-accuracy metrics such as novelty and diversity should not be overlooked. Existing multi-objective (MO) RSs employed collaborative filteri...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Published: |
IEEE Access
2022
|
| _version_ | 1825938676343898112 |
|---|---|
| author | Keat, Ee Yeo Mohd Sharef, Nurfadhlina Yaakob, Razali Kasmiran, Khairul Azhar Marlisah, Erzam Mustapha, Norwati Zolkepli, Maslina |
| author_facet | Keat, Ee Yeo Mohd Sharef, Nurfadhlina Yaakob, Razali Kasmiran, Khairul Azhar Marlisah, Erzam Mustapha, Norwati Zolkepli, Maslina |
| author_sort | Keat, Ee Yeo |
| collection | UPM |
| description | Most existing recommendation systems (RSs) are primarily concerned about the accuracy of rating prediction and only recommending popular items. However, other non-accuracy metrics such as novelty and diversity should not be overlooked. Existing multi-objective (MO) RSs employed collaborative filtering and combined with evolutionary algorithms to handle bi-objective optimization. Besides cold-start problem from collaborative filtering, it also vulnerable to highly sparse environment, while the evolutionary algorithm suffers from premature convergence and curse of dimensionality. These limitations have prompted this work to propose deep reinforcement learning (DRL) approaches for MO optimization in RSs. Several works in DRL are available but none has addressed MO RS problems. In this study, the performances of proposed DRL approaches that based on Deep Q-Network in MO recommendation problem were investigated. The approaches were evaluated with movie recommendation dataset by using three conflicting metrics, namely precision, novelty, and diversity. The results demonstrated that deep reinforcement learning approaches has superiority performance in MO optimization, and its capability of recommending precise item along with achieving high novelty and diversity against the benchmark that using probabilistic based multi-objective approach based on evolutionary algorithm (PMOEA). Although PMOEA algorithm secured higher average value in precision, it has lower values of novelty and diversity than the proposed DRL approaches. The DRL approaches surpassed the benchmark results in average of maximum novelty and the average of mean diversity metrics, the optimization between accuracy and non-accuracy metrics is inevitable. In addition, the experiments revealed that incorporation of user latent features enhanced the recommendation quality. |
| first_indexed | 2024-03-06T11:17:00Z |
| format | Article |
| id | upm.eprints-102257 |
| institution | Universiti Putra Malaysia |
| last_indexed | 2024-03-06T11:17:00Z |
| publishDate | 2022 |
| publisher | IEEE Access |
| record_format | dspace |
| spelling | upm.eprints-1022572023-08-18T23:38:31Z http://psasir.upm.edu.my/id/eprint/102257/ Multi-objective deep reinforcement learning for recommendation systems Keat, Ee Yeo Mohd Sharef, Nurfadhlina Yaakob, Razali Kasmiran, Khairul Azhar Marlisah, Erzam Mustapha, Norwati Zolkepli, Maslina Most existing recommendation systems (RSs) are primarily concerned about the accuracy of rating prediction and only recommending popular items. However, other non-accuracy metrics such as novelty and diversity should not be overlooked. Existing multi-objective (MO) RSs employed collaborative filtering and combined with evolutionary algorithms to handle bi-objective optimization. Besides cold-start problem from collaborative filtering, it also vulnerable to highly sparse environment, while the evolutionary algorithm suffers from premature convergence and curse of dimensionality. These limitations have prompted this work to propose deep reinforcement learning (DRL) approaches for MO optimization in RSs. Several works in DRL are available but none has addressed MO RS problems. In this study, the performances of proposed DRL approaches that based on Deep Q-Network in MO recommendation problem were investigated. The approaches were evaluated with movie recommendation dataset by using three conflicting metrics, namely precision, novelty, and diversity. The results demonstrated that deep reinforcement learning approaches has superiority performance in MO optimization, and its capability of recommending precise item along with achieving high novelty and diversity against the benchmark that using probabilistic based multi-objective approach based on evolutionary algorithm (PMOEA). Although PMOEA algorithm secured higher average value in precision, it has lower values of novelty and diversity than the proposed DRL approaches. The DRL approaches surpassed the benchmark results in average of maximum novelty and the average of mean diversity metrics, the optimization between accuracy and non-accuracy metrics is inevitable. In addition, the experiments revealed that incorporation of user latent features enhanced the recommendation quality. IEEE Access 2022-06-08 Article PeerReviewed Keat, Ee Yeo and Mohd Sharef, Nurfadhlina and Yaakob, Razali and Kasmiran, Khairul Azhar and Marlisah, Erzam and Mustapha, Norwati and Zolkepli, Maslina (2022) Multi-objective deep reinforcement learning for recommendation systems. Institute of Electrical and Electronics Engineers, 10 (1). 65011 - 65027. ISSN 2169-3536 https://ieeexplore.ieee.org/document/9791369 10.1109/ACCESS.2022.3181164 |
| spellingShingle | Keat, Ee Yeo Mohd Sharef, Nurfadhlina Yaakob, Razali Kasmiran, Khairul Azhar Marlisah, Erzam Mustapha, Norwati Zolkepli, Maslina Multi-objective deep reinforcement learning for recommendation systems |
| title | Multi-objective deep reinforcement learning for recommendation systems |
| title_full | Multi-objective deep reinforcement learning for recommendation systems |
| title_fullStr | Multi-objective deep reinforcement learning for recommendation systems |
| title_full_unstemmed | Multi-objective deep reinforcement learning for recommendation systems |
| title_short | Multi-objective deep reinforcement learning for recommendation systems |
| title_sort | multi objective deep reinforcement learning for recommendation systems |
| work_keys_str_mv | AT keateeyeo multiobjectivedeepreinforcementlearningforrecommendationsystems AT mohdsharefnurfadhlina multiobjectivedeepreinforcementlearningforrecommendationsystems AT yaakobrazali multiobjectivedeepreinforcementlearningforrecommendationsystems AT kasmirankhairulazhar multiobjectivedeepreinforcementlearningforrecommendationsystems AT marlisaherzam multiobjectivedeepreinforcementlearningforrecommendationsystems AT mustaphanorwati multiobjectivedeepreinforcementlearningforrecommendationsystems AT zolkeplimaslina multiobjectivedeepreinforcementlearningforrecommendationsystems |