Cliffhanger: Scaling Performance Cliffs in Web Memory Caches
Web-scale applications are heavily reliant on memory cache systems such as Memcached to improve throughput and reduce user latency. Small performance improvements in these systems can result in large end-to-end gains. For example, a marginal increase in hit rate of 1% can reduce the application laye...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
USENIX Association
2017
|
| Online Access: | http://hdl.handle.net/1721.1/110700 https://orcid.org/0000-0002-0014-6742 |
| _version_ | 1826191644929556480 |
|---|---|
| author | Cidon, Asaf Eisenman, Assaf Katti, Sachin Alizadeh Attar, Mohammadreza |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Cidon, Asaf Eisenman, Assaf Katti, Sachin Alizadeh Attar, Mohammadreza |
| author_sort | Cidon, Asaf |
| collection | MIT |
| description | Web-scale applications are heavily reliant on memory cache systems such as Memcached to improve throughput and reduce user latency. Small performance improvements in these systems can result in large end-to-end gains. For example, a marginal increase in hit rate of 1% can reduce the application layer latency by over 35%. However, existing web cache resource allocation policies are workload oblivious and first-come-first-serve. By analyzing measurements from a widely used caching service, Memcachier, we demonstrate that existing cache allocation techniques leave significant room for improvement. We develop Cliffhanger, a lightweight iterative algorithm that runs on memory cache servers, which incrementally optimizes the resource allocations across and within applications based on dynamically changing workloads. It has been shown that cache allocation algorithms underperform when there are performance cliffs, in which minor changes in cache allocation cause large changes in the hit rate. We design a novel technique for dealing with performance cliffs incrementally and locally. We demonstrate that for the Memcachier applications, on average, Cliffhanger increases the overall hit rate 1.2%, reduces the total number of cache misses by 36.7% and achieves the same hit rate with 45% less memory capacity. |
| first_indexed | 2024-09-23T08:59:09Z |
| format | Article |
| id | mit-1721.1/110700 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T08:59:09Z |
| publishDate | 2017 |
| publisher | USENIX Association |
| record_format | dspace |
| spelling | mit-1721.1/1107002022-09-26T09:39:20Z Cliffhanger: Scaling Performance Cliffs in Web Memory Caches Cidon, Asaf Eisenman, Assaf Katti, Sachin Alizadeh Attar, Mohammadreza Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Alizadeh Attar, Mohammadreza Web-scale applications are heavily reliant on memory cache systems such as Memcached to improve throughput and reduce user latency. Small performance improvements in these systems can result in large end-to-end gains. For example, a marginal increase in hit rate of 1% can reduce the application layer latency by over 35%. However, existing web cache resource allocation policies are workload oblivious and first-come-first-serve. By analyzing measurements from a widely used caching service, Memcachier, we demonstrate that existing cache allocation techniques leave significant room for improvement. We develop Cliffhanger, a lightweight iterative algorithm that runs on memory cache servers, which incrementally optimizes the resource allocations across and within applications based on dynamically changing workloads. It has been shown that cache allocation algorithms underperform when there are performance cliffs, in which minor changes in cache allocation cause large changes in the hit rate. We design a novel technique for dealing with performance cliffs incrementally and locally. We demonstrate that for the Memcachier applications, on average, Cliffhanger increases the overall hit rate 1.2%, reduces the total number of cache misses by 36.7% and achieves the same hit rate with 45% less memory capacity. 2017-07-14T14:13:00Z 2017-07-14T14:13:00Z 2016-03 Article http://purl.org/eprint/type/ConferencePaper http://hdl.handle.net/1721.1/110700 Cidon, Asaf et al. "Cliffhanger: Scaling Performance Cliffs in Web Memory Caches." 13th USENIX Symposium on Networked Systems Design and Implementation, Santa Clara, California, 16-18 March, 2016. https://orcid.org/0000-0002-0014-6742 en_US https://www.usenix.org/node/194949 Proceedings of 13th USENIX Symposium on Networked Systems Design and Implementation Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf USENIX Association MIT Web Domain |
| spellingShingle | Cidon, Asaf Eisenman, Assaf Katti, Sachin Alizadeh Attar, Mohammadreza Cliffhanger: Scaling Performance Cliffs in Web Memory Caches |
| title | Cliffhanger: Scaling Performance Cliffs in Web Memory Caches |
| title_full | Cliffhanger: Scaling Performance Cliffs in Web Memory Caches |
| title_fullStr | Cliffhanger: Scaling Performance Cliffs in Web Memory Caches |
| title_full_unstemmed | Cliffhanger: Scaling Performance Cliffs in Web Memory Caches |
| title_short | Cliffhanger: Scaling Performance Cliffs in Web Memory Caches |
| title_sort | cliffhanger scaling performance cliffs in web memory caches |
| url | http://hdl.handle.net/1721.1/110700 https://orcid.org/0000-0002-0014-6742 |
| work_keys_str_mv | AT cidonasaf cliffhangerscalingperformancecliffsinwebmemorycaches AT eisenmanassaf cliffhangerscalingperformancecliffsinwebmemorycaches AT kattisachin cliffhangerscalingperformancecliffsinwebmemorycaches AT alizadehattarmohammadreza cliffhangerscalingperformancecliffsinwebmemorycaches |