Implementing accelerated key-value store: From SSDs to datacenter servers
Efficient management of storage is a primary concern in all systems dealing with Big Data. In the modern era, flash-based solid-state drives (SSDs) are widely adopted in computer systems, slowly replacing hard disk drives. As many kinds of data generated and collected these days are not well-structu...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2023
|
| Online Access: | https://hdl.handle.net/1721.1/150217 |
| _version_ | 1826205017177063424 |
|---|---|
| author | Chung, Chanwoo |
| author2 | Arvind |
| author_facet | Arvind Chung, Chanwoo |
| author_sort | Chung, Chanwoo |
| collection | MIT |
| description | Efficient management of storage is a primary concern in all systems dealing with Big Data. In the modern era, flash-based solid-state drives (SSDs) are widely adopted in computer systems, slowly replacing hard disk drives. As many kinds of data generated and collected these days are not well-structured, a key-value store has become one of the most important building blocks widely used in datacenters thanks to its simple interface. Key-value stores are often used as a internal engine for other databases.
This thesis explores whether a modern flash-based solid-state drive (SSD) augmented with near-storage computations can be re-designed to provide cheaper and power-efficient solution to maintaining various key-value services in the cloud. The thesis explores a new type of storage device, called a key-value SSD (KV-SSD), that exposes a key-value interface instead of the legacy block interface to the host machine.
The two alternative power- and cost-efficient solutions that can replace existing KVS components are based on KV-SSDs, LightStore and PinK. LightStore is a new storage architecture based on a group of network-attached KV-SSDs without storage host servers. LightStore aims to primarily support large-sized objects and emulates other types of data stores using application-side adapters. Compared to existing storage server-based solutions, LightStore is up to 2.3X space- and 7.4X energy-efficient. PinK is a novel design of an LSM-tree for KV-SSDs with software and hardware techniques that provides bounded tail latency and design flexibility. PinK prototype reduces the read and 99th percentile latency by 22% and improves read throughput by 44% compared to LightStore prototype. The PinK prototype showed 42-73% better latency and 37% better throughput compared to commercial hash-based prototype. A proposed future design based on smart SSDs, a block-based SSD with an accelerator, shows how the smart SSDs can help existing software KVS on hosts. We believe these alternatives to running various types of key-value stores in datacenters would reduce storage management cost drastically. |
| first_indexed | 2024-09-23T13:05:27Z |
| format | Thesis |
| id | mit-1721.1/150217 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T13:05:27Z |
| publishDate | 2023 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1502172023-04-01T03:30:12Z Implementing accelerated key-value store: From SSDs to datacenter servers Chung, Chanwoo Arvind Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Efficient management of storage is a primary concern in all systems dealing with Big Data. In the modern era, flash-based solid-state drives (SSDs) are widely adopted in computer systems, slowly replacing hard disk drives. As many kinds of data generated and collected these days are not well-structured, a key-value store has become one of the most important building blocks widely used in datacenters thanks to its simple interface. Key-value stores are often used as a internal engine for other databases. This thesis explores whether a modern flash-based solid-state drive (SSD) augmented with near-storage computations can be re-designed to provide cheaper and power-efficient solution to maintaining various key-value services in the cloud. The thesis explores a new type of storage device, called a key-value SSD (KV-SSD), that exposes a key-value interface instead of the legacy block interface to the host machine. The two alternative power- and cost-efficient solutions that can replace existing KVS components are based on KV-SSDs, LightStore and PinK. LightStore is a new storage architecture based on a group of network-attached KV-SSDs without storage host servers. LightStore aims to primarily support large-sized objects and emulates other types of data stores using application-side adapters. Compared to existing storage server-based solutions, LightStore is up to 2.3X space- and 7.4X energy-efficient. PinK is a novel design of an LSM-tree for KV-SSDs with software and hardware techniques that provides bounded tail latency and design flexibility. PinK prototype reduces the read and 99th percentile latency by 22% and improves read throughput by 44% compared to LightStore prototype. The PinK prototype showed 42-73% better latency and 37% better throughput compared to commercial hash-based prototype. A proposed future design based on smart SSDs, a block-based SSD with an accelerator, shows how the smart SSDs can help existing software KVS on hosts. We believe these alternatives to running various types of key-value stores in datacenters would reduce storage management cost drastically. Ph.D. 2023-03-31T14:40:18Z 2023-03-31T14:40:18Z 2023-02 2023-02-28T14:39:37.455Z Thesis https://hdl.handle.net/1721.1/150217 0000-0002-2279-1806 In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Chung, Chanwoo Implementing accelerated key-value store: From SSDs to datacenter servers |
| title | Implementing accelerated key-value store: From SSDs to datacenter servers |
| title_full | Implementing accelerated key-value store: From SSDs to datacenter servers |
| title_fullStr | Implementing accelerated key-value store: From SSDs to datacenter servers |
| title_full_unstemmed | Implementing accelerated key-value store: From SSDs to datacenter servers |
| title_short | Implementing accelerated key-value store: From SSDs to datacenter servers |
| title_sort | implementing accelerated key value store from ssds to datacenter servers |
| url | https://hdl.handle.net/1721.1/150217 |
| work_keys_str_mv | AT chungchanwoo implementingacceleratedkeyvaluestorefromssdstodatacenterservers |