New and Efficient Algorithms for Producing Frequent Itemsets with the Map-Reduce Framework
The Map-Reduce (MR) framework has become a popular framework for developing new parallel algorithms for Big Data. Efficient algorithms for data mining of big data and distributed databases has become an important problem. In this paper we focus on algorithms producing association rules and frequent...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2018-11-01
|
| Series: | Algorithms |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-4893/11/12/194 |
| _version_ | 1819160544974733312 |
|---|---|
| author | Yaron Gonen Ehud Gudes Kirill Kandalov |
| author_facet | Yaron Gonen Ehud Gudes Kirill Kandalov |
| author_sort | Yaron Gonen |
| collection | DOAJ |
| description | The Map-Reduce (MR) framework has become a popular framework for developing new parallel algorithms for Big Data. Efficient algorithms for data mining of big data and distributed databases has become an important problem. In this paper we focus on algorithms producing association rules and frequent itemsets. After reviewing the most recent algorithms that perform this task within the MR framework, we present two new algorithms: one algorithm for producing closed frequent itemsets, and the second one for producing frequent itemsets when the database is updated and new data is added to the old database. Both algorithms include novel optimizations which are suitable to the MR framework, as well as to other parallel architectures. A detailed experimental evaluation shows the effectiveness and advantages of the algorithms over existing methods when it comes to large distributed databases. |
| first_indexed | 2024-12-22T16:58:09Z |
| format | Article |
| id | doaj.art-c36fceb640174484bd1c372b63ba8a05 |
| institution | Directory Open Access Journal |
| issn | 1999-4893 |
| language | English |
| last_indexed | 2024-12-22T16:58:09Z |
| publishDate | 2018-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Algorithms |
| spelling | doaj.art-c36fceb640174484bd1c372b63ba8a052022-12-21T18:19:24ZengMDPI AGAlgorithms1999-48932018-11-01111219410.3390/a11120194a11120194New and Efficient Algorithms for Producing Frequent Itemsets with the Map-Reduce FrameworkYaron Gonen0Ehud Gudes1Kirill Kandalov2Department of Computer Science, Ben-Gurion University, Beer-Sheva 8410501, IsraelDepartment of Computer Science, Ben-Gurion University, Beer-Sheva 8410501, IsraelDepartment of Computer Science, Open University, Ra’anana 4353701, IsraelThe Map-Reduce (MR) framework has become a popular framework for developing new parallel algorithms for Big Data. Efficient algorithms for data mining of big data and distributed databases has become an important problem. In this paper we focus on algorithms producing association rules and frequent itemsets. After reviewing the most recent algorithms that perform this task within the MR framework, we present two new algorithms: one algorithm for producing closed frequent itemsets, and the second one for producing frequent itemsets when the database is updated and new data is added to the old database. Both algorithms include novel optimizations which are suitable to the MR framework, as well as to other parallel architectures. A detailed experimental evaluation shows the effectiveness and advantages of the algorithms over existing methods when it comes to large distributed databases.https://www.mdpi.com/1999-4893/11/12/194apriorimap reducebig datafrequent itemsetsclosed itemsetsincremental computation |
| spellingShingle | Yaron Gonen Ehud Gudes Kirill Kandalov New and Efficient Algorithms for Producing Frequent Itemsets with the Map-Reduce Framework Algorithms apriori map reduce big data frequent itemsets closed itemsets incremental computation |
| title | New and Efficient Algorithms for Producing Frequent Itemsets with the Map-Reduce Framework |
| title_full | New and Efficient Algorithms for Producing Frequent Itemsets with the Map-Reduce Framework |
| title_fullStr | New and Efficient Algorithms for Producing Frequent Itemsets with the Map-Reduce Framework |
| title_full_unstemmed | New and Efficient Algorithms for Producing Frequent Itemsets with the Map-Reduce Framework |
| title_short | New and Efficient Algorithms for Producing Frequent Itemsets with the Map-Reduce Framework |
| title_sort | new and efficient algorithms for producing frequent itemsets with the map reduce framework |
| topic | apriori map reduce big data frequent itemsets closed itemsets incremental computation |
| url | https://www.mdpi.com/1999-4893/11/12/194 |
| work_keys_str_mv | AT yarongonen newandefficientalgorithmsforproducingfrequentitemsetswiththemapreduceframework AT ehudgudes newandefficientalgorithmsforproducingfrequentitemsetswiththemapreduceframework AT kirillkandalov newandefficientalgorithmsforproducingfrequentitemsetswiththemapreduceframework |