Instance-Optimized Database Indexes and Storage Layouts
For any modern database system, its physical design, which is composed of both the storage layout of the data itself and auxiliary data structures such as indexes, is a critical piece of maintaining high performance in the face of increasing data volumes. Existing physical design components are gene...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2023
|
| Online Access: | https://hdl.handle.net/1721.1/147396 |
| _version_ | 1826199470384087040 |
|---|---|
| author | Ding, Jialin |
| author2 | Kraska, Tim |
| author_facet | Kraska, Tim Ding, Jialin |
| author_sort | Ding, Jialin |
| collection | MIT |
| description | For any modern database system, its physical design, which is composed of both the storage layout of the data itself and auxiliary data structures such as indexes, is a critical piece of maintaining high performance in the face of increasing data volumes. Existing physical design components are general-purpose: they achieve adequate performance for the average use case but don’t achieve optimal performance for any individual use case. These physical design components expose numerous configuration knobs that users must manually tune to achieve better performance for their individual use case, but tuning complex systems is labor-intensive, and poor tuning can result in degraded performance and increased costs.
In this thesis, we explore how database systems can maximize performance while minimizing manual effort through instance-optimization, which is the process of designing systems that are able to automatically self-adjust in order to achieve the best performance for a given use case. We leverage instance-optimization to introduce novel designs for database indexes and data storage layouts that outperform existing state-of-the-art indexes and data layouts by orders of magnitude. We also demonstrate how to incorporate multiple instance-optimized database components into an end-toend analytic database system that outperforms a well-tuned commercial cloud-based analytics system by up to 3×. |
| first_indexed | 2024-09-23T11:20:25Z |
| format | Thesis |
| id | mit-1721.1/147396 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T11:20:25Z |
| publishDate | 2023 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1473962023-01-20T03:40:00Z Instance-Optimized Database Indexes and Storage Layouts Ding, Jialin Kraska, Tim Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science For any modern database system, its physical design, which is composed of both the storage layout of the data itself and auxiliary data structures such as indexes, is a critical piece of maintaining high performance in the face of increasing data volumes. Existing physical design components are general-purpose: they achieve adequate performance for the average use case but don’t achieve optimal performance for any individual use case. These physical design components expose numerous configuration knobs that users must manually tune to achieve better performance for their individual use case, but tuning complex systems is labor-intensive, and poor tuning can result in degraded performance and increased costs. In this thesis, we explore how database systems can maximize performance while minimizing manual effort through instance-optimization, which is the process of designing systems that are able to automatically self-adjust in order to achieve the best performance for a given use case. We leverage instance-optimization to introduce novel designs for database indexes and data storage layouts that outperform existing state-of-the-art indexes and data layouts by orders of magnitude. We also demonstrate how to incorporate multiple instance-optimized database components into an end-toend analytic database system that outperforms a well-tuned commercial cloud-based analytics system by up to 3×. Ph.D. 2023-01-19T18:50:27Z 2023-01-19T18:50:27Z 2022-09 2022-10-19T19:08:09.217Z Thesis https://hdl.handle.net/1721.1/147396 In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Ding, Jialin Instance-Optimized Database Indexes and Storage Layouts |
| title | Instance-Optimized Database Indexes and Storage Layouts |
| title_full | Instance-Optimized Database Indexes and Storage Layouts |
| title_fullStr | Instance-Optimized Database Indexes and Storage Layouts |
| title_full_unstemmed | Instance-Optimized Database Indexes and Storage Layouts |
| title_short | Instance-Optimized Database Indexes and Storage Layouts |
| title_sort | instance optimized database indexes and storage layouts |
| url | https://hdl.handle.net/1721.1/147396 |
| work_keys_str_mv | AT dingjialin instanceoptimizeddatabaseindexesandstoragelayouts |