A BiLSTM cardinality estimator in complex database systems based on attention mechanism
Abstract An excellent cardinality estimation can make the query optimiser produce a good execution plan. Although there are some studies on cardinality estimation, the prediction results of existing cardinality estimators are inaccurate and the query efficiency cannot be guaranteed as well. In parti...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-09-01
|
| Series: | CAAI Transactions on Intelligence Technology |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/cit2.12069 |
| _version_ | 1811340306922078208 |
|---|---|
| author | Qiang Zhou Guoping Yang Haiquan Song Jin Guo Yadong Zhang Shengjie Wei Lulu Qu Louis Alberto Gutierrez Shaojie Qiao |
| author_facet | Qiang Zhou Guoping Yang Haiquan Song Jin Guo Yadong Zhang Shengjie Wei Lulu Qu Louis Alberto Gutierrez Shaojie Qiao |
| author_sort | Qiang Zhou |
| collection | DOAJ |
| description | Abstract An excellent cardinality estimation can make the query optimiser produce a good execution plan. Although there are some studies on cardinality estimation, the prediction results of existing cardinality estimators are inaccurate and the query efficiency cannot be guaranteed as well. In particular, they are difficult to accurately obtain the complex relationships between multiple tables in complex database systems. When dealing with complex queries, the existing cardinality estimators cannot achieve good results. In this study, a novel cardinality estimator is proposed. It uses the core techniques with the BiLSTM network structure and adds the attention mechanism. First, the columns involved in the query statements in the training set are sampled and compressed into bitmaps. Then, the Word2vec model is used to embed the word vectors about the query statements. Finally, the BiLSTM network and attention mechanism are employed to deal with word vectors. The proposed model takes into consideration not only the correlation between tables but also the processing of complex predicates. Extensive experiments and the evaluation of BiLSTM‐Attention Cardinality Estimator (BACE) on the IMDB datasets are conducted. The results show that the deep learning model can significantly improve the quality of cardinality estimation, which is a vital role in query optimisation for complex databases. |
| first_indexed | 2024-04-13T18:39:59Z |
| format | Article |
| id | doaj.art-9a353ab450ea42beb18fad6a03fb0156 |
| institution | Directory Open Access Journal |
| issn | 2468-2322 |
| language | English |
| last_indexed | 2024-04-13T18:39:59Z |
| publishDate | 2022-09-01 |
| publisher | Wiley |
| record_format | Article |
| series | CAAI Transactions on Intelligence Technology |
| spelling | doaj.art-9a353ab450ea42beb18fad6a03fb01562022-12-22T02:34:45ZengWileyCAAI Transactions on Intelligence Technology2468-23222022-09-017353754610.1049/cit2.12069A BiLSTM cardinality estimator in complex database systems based on attention mechanismQiang Zhou0Guoping Yang1Haiquan Song2Jin Guo3Yadong Zhang4Shengjie Wei5Lulu Qu6Louis Alberto Gutierrez7Shaojie Qiao8School of Information Science and Technology, Southwest Jiaotong University Chengdu ChinaSchool of Software Engineering Chengdu University of Information Technology Chengdu ChinaSchool of Energy Power and Mechanical Engineering North China Electric Power University Baoding ChinaSchool of Information Science and Technology, Southwest Jiaotong University Chengdu ChinaSchool of Information Science and Technology, Southwest Jiaotong University Chengdu ChinaDigital Media Art, Key Laboratory of Sichuan Province Sichuan Conservatory of Music Chengdu ChinaSchool of Software Engineering Chengdu University of Information Technology Chengdu ChinaDepartment of Computer Science Rensselaer Polytechnic Institute New York New York USASchool of Software Engineering Chengdu University of Information Technology Chengdu ChinaAbstract An excellent cardinality estimation can make the query optimiser produce a good execution plan. Although there are some studies on cardinality estimation, the prediction results of existing cardinality estimators are inaccurate and the query efficiency cannot be guaranteed as well. In particular, they are difficult to accurately obtain the complex relationships between multiple tables in complex database systems. When dealing with complex queries, the existing cardinality estimators cannot achieve good results. In this study, a novel cardinality estimator is proposed. It uses the core techniques with the BiLSTM network structure and adds the attention mechanism. First, the columns involved in the query statements in the training set are sampled and compressed into bitmaps. Then, the Word2vec model is used to embed the word vectors about the query statements. Finally, the BiLSTM network and attention mechanism are employed to deal with word vectors. The proposed model takes into consideration not only the correlation between tables but also the processing of complex predicates. Extensive experiments and the evaluation of BiLSTM‐Attention Cardinality Estimator (BACE) on the IMDB datasets are conducted. The results show that the deep learning model can significantly improve the quality of cardinality estimation, which is a vital role in query optimisation for complex databases.https://doi.org/10.1049/cit2.12069query processingdatabase management systemsrecurrent neural netsdeep learning (artificial intelligence)estimation theory |
| spellingShingle | Qiang Zhou Guoping Yang Haiquan Song Jin Guo Yadong Zhang Shengjie Wei Lulu Qu Louis Alberto Gutierrez Shaojie Qiao A BiLSTM cardinality estimator in complex database systems based on attention mechanism CAAI Transactions on Intelligence Technology query processing database management systems recurrent neural nets deep learning (artificial intelligence) estimation theory |
| title | A BiLSTM cardinality estimator in complex database systems based on attention mechanism |
| title_full | A BiLSTM cardinality estimator in complex database systems based on attention mechanism |
| title_fullStr | A BiLSTM cardinality estimator in complex database systems based on attention mechanism |
| title_full_unstemmed | A BiLSTM cardinality estimator in complex database systems based on attention mechanism |
| title_short | A BiLSTM cardinality estimator in complex database systems based on attention mechanism |
| title_sort | bilstm cardinality estimator in complex database systems based on attention mechanism |
| topic | query processing database management systems recurrent neural nets deep learning (artificial intelligence) estimation theory |
| url | https://doi.org/10.1049/cit2.12069 |
| work_keys_str_mv | AT qiangzhou abilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT guopingyang abilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT haiquansong abilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT jinguo abilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT yadongzhang abilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT shengjiewei abilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT luluqu abilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT louisalbertogutierrez abilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT shaojieqiao abilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT qiangzhou bilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT guopingyang bilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT haiquansong bilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT jinguo bilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT yadongzhang bilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT shengjiewei bilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT luluqu bilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT louisalbertogutierrez bilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism AT shaojieqiao bilstmcardinalityestimatorincomplexdatabasesystemsbasedonattentionmechanism |