Partitioning Techniques for Fine-grained Indexing
URL to paper listed on conference site
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
International Conference on Data Engineering
2011
|
| Online Access: | http://hdl.handle.net/1721.1/63110 https://orcid.org/0000-0002-7470-3265 |
| _version_ | 1826215352398249984 |
|---|---|
| author | Wu, Eugene Madden, Samuel R. |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Wu, Eugene Madden, Samuel R. |
| author_sort | Wu, Eugene |
| collection | MIT |
| description | URL to paper listed on conference site |
| first_indexed | 2024-09-23T16:25:41Z |
| format | Article |
| id | mit-1721.1/63110 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T16:25:41Z |
| publishDate | 2011 |
| publisher | International Conference on Data Engineering |
| record_format | dspace |
| spelling | mit-1721.1/631102022-09-29T19:48:32Z Partitioning Techniques for Fine-grained Indexing Wu, Eugene Madden, Samuel R. Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Madden, Samuel R. Madden, Samuel R. Wu, Eugene URL to paper listed on conference site Many data-intensive websites use databases that grow much faster than the rate that users access the data. Such growing datasets lead to ever-increasing space and performance overheads for maintaining and accessing indexes. Furthermore, there is often considerable skew with popular users and recent data accessed much more frequently. These observations led us to design Shinobi, a system which uses horizontal partitioning as a mechanism for improving query performance to cluster the physical data, and increasing insert performance by only indexing data that is frequently accessed. We present database design algorithms that optimally partition tables, drop indexes from partitions that are infrequently queried, and maintain these partitions as workloads change. We show a 60× performance improvement over traditionally indexed tables using a real-world query workload derived from a traffic monitoring application 2011-05-25T15:08:52Z 2011-05-25T15:08:52Z 2011-04 Article http://purl.org/eprint/type/ConferencePaper http://hdl.handle.net/1721.1/63110 Wu, Eugene and Samuel Madden. "Partitioning Techniques for Fine-grained Indexing." International Conference on Data Engineering, ICDE 2011, Hannover, April 11-16, 2011. https://orcid.org/0000-0002-7470-3265 en_US http://www.icde2011.org/node/94 Proceedings of the International Conference on Data Engineering, ICDE 2011 Creative Commons Attribution-Noncommercial-Share Alike 3.0 http://creativecommons.org/licenses/by-nc-sa/3.0/ application/pdf International Conference on Data Engineering MIT web domain |
| spellingShingle | Wu, Eugene Madden, Samuel R. Partitioning Techniques for Fine-grained Indexing |
| title | Partitioning Techniques for Fine-grained Indexing |
| title_full | Partitioning Techniques for Fine-grained Indexing |
| title_fullStr | Partitioning Techniques for Fine-grained Indexing |
| title_full_unstemmed | Partitioning Techniques for Fine-grained Indexing |
| title_short | Partitioning Techniques for Fine-grained Indexing |
| title_sort | partitioning techniques for fine grained indexing |
| url | http://hdl.handle.net/1721.1/63110 https://orcid.org/0000-0002-7470-3265 |
| work_keys_str_mv | AT wueugene partitioningtechniquesforfinegrainedindexing AT maddensamuelr partitioningtechniquesforfinegrainedindexing |