PUC: parallel mining of high-utility itemsets with load balancing on spark
Distributed programming paradigms such as MapReduce and Spark have alleviated sequential bottleneck while mining of massive transaction databases. Of significant importance is mining High Utility Itemset (HUI) that incorporates the revenue of the items purchased in a transaction. Although a few algo...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
De Gruyter
2022-05-01
|
| Series: | Journal of Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1515/jisys-2022-0044 |
| _version_ | 1811236597865119744 |
|---|---|
| author | Brahmavar Anup Bhat Sheeranalli Venkatarama Harish Maiya Geetha |
| author_facet | Brahmavar Anup Bhat Sheeranalli Venkatarama Harish Maiya Geetha |
| author_sort | Brahmavar Anup Bhat |
| collection | DOAJ |
| description | Distributed programming paradigms such as MapReduce and Spark have alleviated sequential bottleneck while mining of massive transaction databases. Of significant importance is mining High Utility Itemset (HUI) that incorporates the revenue of the items purchased in a transaction. Although a few algorithms to mine HUIs in the distributed environment exist, workload skew and data transfer overhead due to shuffling operations remain major issues. In the current study, Parallel Utility Computation (PUC) algorithm has been proposed with novel grouping and load balancing strategies for an efficient mining of HUIs in a distributed environment. To group the items, Transaction Weighted Utility (TWU) values as a degree of transaction similarity is employed. Subsequently, these groups are assigned to the nodes across the cluster by taking into account the mining load due to the items in the group. Experimental evaluation on real and synthetic datasets demonstrate that PUC with TWU grouping in conjunction with load balancing converges mining faster. Due to reduced data transfer, and load balancing-based assignment strategy, PUC outperforms different grouping strategies and random assignment of groups across the cluster. Also, PUC is shown to be faster than PHUI-Growth algorithm with a promising speedup. |
| first_indexed | 2024-04-12T12:10:43Z |
| format | Article |
| id | doaj.art-dbeb8532ac5f413ba6c880224792ae6a |
| institution | Directory Open Access Journal |
| issn | 2191-026X |
| language | English |
| last_indexed | 2024-04-12T12:10:43Z |
| publishDate | 2022-05-01 |
| publisher | De Gruyter |
| record_format | Article |
| series | Journal of Intelligent Systems |
| spelling | doaj.art-dbeb8532ac5f413ba6c880224792ae6a2022-12-22T03:33:35ZengDe GruyterJournal of Intelligent Systems2191-026X2022-05-0131156858810.1515/jisys-2022-0044PUC: parallel mining of high-utility itemsets with load balancing on sparkBrahmavar Anup Bhat0Sheeranalli Venkatarama Harish1Maiya Geetha2Department of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, Karnataka, IndiaDepartment of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, Karnataka, IndiaDepartment of Computer Science and Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, Karnataka, IndiaDistributed programming paradigms such as MapReduce and Spark have alleviated sequential bottleneck while mining of massive transaction databases. Of significant importance is mining High Utility Itemset (HUI) that incorporates the revenue of the items purchased in a transaction. Although a few algorithms to mine HUIs in the distributed environment exist, workload skew and data transfer overhead due to shuffling operations remain major issues. In the current study, Parallel Utility Computation (PUC) algorithm has been proposed with novel grouping and load balancing strategies for an efficient mining of HUIs in a distributed environment. To group the items, Transaction Weighted Utility (TWU) values as a degree of transaction similarity is employed. Subsequently, these groups are assigned to the nodes across the cluster by taking into account the mining load due to the items in the group. Experimental evaluation on real and synthetic datasets demonstrate that PUC with TWU grouping in conjunction with load balancing converges mining faster. Due to reduced data transfer, and load balancing-based assignment strategy, PUC outperforms different grouping strategies and random assignment of groups across the cluster. Also, PUC is shown to be faster than PHUI-Growth algorithm with a promising speedup.https://doi.org/10.1515/jisys-2022-0044high utility itemset miningapache sparkbig data analyticsmapreduceload balancing68t0968t35 |
| spellingShingle | Brahmavar Anup Bhat Sheeranalli Venkatarama Harish Maiya Geetha PUC: parallel mining of high-utility itemsets with load balancing on spark Journal of Intelligent Systems high utility itemset mining apache spark big data analytics mapreduce load balancing 68t09 68t35 |
| title | PUC: parallel mining of high-utility itemsets with load balancing on spark |
| title_full | PUC: parallel mining of high-utility itemsets with load balancing on spark |
| title_fullStr | PUC: parallel mining of high-utility itemsets with load balancing on spark |
| title_full_unstemmed | PUC: parallel mining of high-utility itemsets with load balancing on spark |
| title_short | PUC: parallel mining of high-utility itemsets with load balancing on spark |
| title_sort | puc parallel mining of high utility itemsets with load balancing on spark |
| topic | high utility itemset mining apache spark big data analytics mapreduce load balancing 68t09 68t35 |
| url | https://doi.org/10.1515/jisys-2022-0044 |
| work_keys_str_mv | AT brahmavaranupbhat pucparallelminingofhighutilityitemsetswithloadbalancingonspark AT sheeranallivenkataramaharish pucparallelminingofhighutilityitemsetswithloadbalancingonspark AT maiyageetha pucparallelminingofhighutilityitemsetswithloadbalancingonspark |