Behavior-driven optimization techniques for scalable data exploration
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2017.
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | eng |
| Published: |
Massachusetts Institute of Technology
2017
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/111853 |
| _version_ | 1811078505490808832 |
|---|---|
| author | Battle, Leilani Marie |
| author2 | Michael Stonebraker. |
| author_facet | Michael Stonebraker. Battle, Leilani Marie |
| author_sort | Battle, Leilani Marie |
| collection | MIT |
| description | Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2017. |
| first_indexed | 2024-09-23T11:01:14Z |
| format | Thesis |
| id | mit-1721.1/111853 |
| institution | Massachusetts Institute of Technology |
| language | eng |
| last_indexed | 2024-09-23T11:01:14Z |
| publishDate | 2017 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1118532019-04-11T01:23:13Z Behavior-driven optimization techniques for scalable data exploration Battle, Leilani Marie Michael Stonebraker. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Electrical Engineering and Computer Science. Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2017. This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. Cataloged from student-submitted PDF version of thesis. Includes bibliographical references (pages 153-162). Interactive visualizations are a popular medium used by scientists to explore, analyze and generally make sense of their data. However, with the overwhelming amounts of data that scientists collect from various instruments (e.g., telescopes, satellites, gene sequencers and field sensors), they need ways of efficiently transforming their data into interactive visualizations. Though a variety of visualization tools exist to help people make sense of their data, these tools often rely on database management systems (or DBMSs) for data processing and storage; and unfortunately, DBMSs fail to process the data fast enough to support a fluid, interactive visualization experience. This thesis blends optimization techniques from databases and methodology from HCI and visualization in order to support interactive and iterative exploration of large datasets. Our main goal is to reduce latency in visualization systems, i.e., the time these systems spend responding to a user's actions. We demonstrate through a comprehensive user study that latency has a clear (negative) effect on users' high-level analysis strategies, which becomes more pronounced as the latency is increased. Furthermore, we find that users are more susceptible to the effects of system latency when they have existing domain knowledge, a common scenario for data scientists. We then developed a visual exploration system called Sculpin that utilizes a suite of optimizations to reduce system latency. Sculpin learns user exploration patterns automatically, and exploits these patterns to pre-fetch data ahead of users as they explore. We then combine data-prefetching with incremental data processing (i.e., incremental materialization) and visualization-focused caching optimizations to further boost performance. With all three of these techniques (pre-fetching, caching, and pre-computation), Sculpin is able to: create visualizations 380% faster and respond to user interactions 88% faster than existing visualization systems, while also using less than one third of the space required by other systems to store materialized query results. by Leilani Battle. Ph. D. 2017-10-18T14:42:15Z 2017-10-18T14:42:15Z 2017 2017 Thesis http://hdl.handle.net/1721.1/111853 1004859394 eng MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission. http://dspace.mit.edu/handle/1721.1/7582 162 pages application/pdf Massachusetts Institute of Technology |
| spellingShingle | Electrical Engineering and Computer Science. Battle, Leilani Marie Behavior-driven optimization techniques for scalable data exploration |
| title | Behavior-driven optimization techniques for scalable data exploration |
| title_full | Behavior-driven optimization techniques for scalable data exploration |
| title_fullStr | Behavior-driven optimization techniques for scalable data exploration |
| title_full_unstemmed | Behavior-driven optimization techniques for scalable data exploration |
| title_short | Behavior-driven optimization techniques for scalable data exploration |
| title_sort | behavior driven optimization techniques for scalable data exploration |
| topic | Electrical Engineering and Computer Science. |
| url | http://hdl.handle.net/1721.1/111853 |
| work_keys_str_mv | AT battleleilanimarie behaviordrivenoptimizationtechniquesforscalabledataexploration |