Progressive Entity Matching via Cost Benefit Analysis
Entity matching (EM) is a fundamental problem in data preprocessing, and is a long running topic in big data analytics and mining communities. In big data era, (nearly) real-time data applications become popular, and call for progressive EM, which produces as many match pairs as possible in very lim...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9667382/ |
| _version_ | 1818504236731727872 |
|---|---|
| author | Chenchen Sun Zhijiang Hou Derong Shen Tiezheng Nie |
| author_facet | Chenchen Sun Zhijiang Hou Derong Shen Tiezheng Nie |
| author_sort | Chenchen Sun |
| collection | DOAJ |
| description | Entity matching (EM) is a fundamental problem in data preprocessing, and is a long running topic in big data analytics and mining communities. In big data era, (nearly) real-time data applications become popular, and call for progressive EM, which produces as many match pairs as possible in very limited time. Previous progressive EM focus on memory based solutions, but disk based solutions are necessary when dirty datasets cannot be fully loaded into memory. To this end, we propose a cost benefit analysis based progressive EM approach, which partitions data according to coarse clustering results and then iteratively schedules data partitions in a greedy way for high progressive resolution. At first, based on estimated record pair similarities, records are fast coarsely clustered; then, record clusters with near average similarities are greedily distributed to the same partitions, and data partitions are cached. After that, cost model is defined with time and space constrains, and benefit model is defined with expected resolution results. On the basis of the cost benefit model, a greedy approximate method is proposed to effectively schedule data for high progressiveness of EM. Finally, we implement extensive experiments over several datasets to evaluate our approach, and show its advantages over existing works. |
| first_indexed | 2024-12-10T21:34:27Z |
| format | Article |
| id | doaj.art-b56c7db254a64f368cb3e4297cf8f6e4 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-10T21:34:27Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-b56c7db254a64f368cb3e4297cf8f6e42022-12-22T01:32:41ZengIEEEIEEE Access2169-35362022-01-01103979398910.1109/ACCESS.2021.31399879667382Progressive Entity Matching via Cost Benefit AnalysisChenchen Sun0https://orcid.org/0000-0002-9990-0425Zhijiang Hou1Derong Shen2Tiezheng Nie3Key Laboratory of Computer Vision and System (Ministry of Education), Tianjin University of Technology, Tianjin, ChinaLibrary, Tianjin University of Technology, Tianjin, ChinaSchool of Computer Science and Engineering, Northeastern University, Shenyang, ChinaSchool of Computer Science and Engineering, Northeastern University, Shenyang, ChinaEntity matching (EM) is a fundamental problem in data preprocessing, and is a long running topic in big data analytics and mining communities. In big data era, (nearly) real-time data applications become popular, and call for progressive EM, which produces as many match pairs as possible in very limited time. Previous progressive EM focus on memory based solutions, but disk based solutions are necessary when dirty datasets cannot be fully loaded into memory. To this end, we propose a cost benefit analysis based progressive EM approach, which partitions data according to coarse clustering results and then iteratively schedules data partitions in a greedy way for high progressive resolution. At first, based on estimated record pair similarities, records are fast coarsely clustered; then, record clusters with near average similarities are greedily distributed to the same partitions, and data partitions are cached. After that, cost model is defined with time and space constrains, and benefit model is defined with expected resolution results. On the basis of the cost benefit model, a greedy approximate method is proposed to effectively schedule data for high progressiveness of EM. Finally, we implement extensive experiments over several datasets to evaluate our approach, and show its advantages over existing works.https://ieeexplore.ieee.org/document/9667382/Entity matchingprogressivecost benefit modeldata partitioningdata integration |
| spellingShingle | Chenchen Sun Zhijiang Hou Derong Shen Tiezheng Nie Progressive Entity Matching via Cost Benefit Analysis IEEE Access Entity matching progressive cost benefit model data partitioning data integration |
| title | Progressive Entity Matching via Cost Benefit Analysis |
| title_full | Progressive Entity Matching via Cost Benefit Analysis |
| title_fullStr | Progressive Entity Matching via Cost Benefit Analysis |
| title_full_unstemmed | Progressive Entity Matching via Cost Benefit Analysis |
| title_short | Progressive Entity Matching via Cost Benefit Analysis |
| title_sort | progressive entity matching via cost benefit analysis |
| topic | Entity matching progressive cost benefit model data partitioning data integration |
| url | https://ieeexplore.ieee.org/document/9667382/ |
| work_keys_str_mv | AT chenchensun progressiveentitymatchingviacostbenefitanalysis AT zhijianghou progressiveentitymatchingviacostbenefitanalysis AT derongshen progressiveentitymatchingviacostbenefitanalysis AT tiezhengnie progressiveentitymatchingviacostbenefitanalysis |