SDCBench: A Benchmark Suite for Workload Colocation and Evaluation in Datacenters
Colocating workloads are commonly used in datacenters to improve server utilization. However, the unpredictable application performance degradation caused by the contention for shared resources makes the problem difficult and limits the efficiency of this approach. This problem has sparked research...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
American Association for the Advancement of Science (AAAS)
2022-01-01
|
| Series: | Intelligent Computing |
| Online Access: | https://spj.science.org/doi/10.34133/2022/9810691 |
| _version_ | 1797810743291150336 |
|---|---|
| author | Yanan Yang Xiangyu Kong Laiping Zhao Yiming Li Huanyu Zhang Jie Li Heng Qi Keqiu Li |
| author_facet | Yanan Yang Xiangyu Kong Laiping Zhao Yiming Li Huanyu Zhang Jie Li Heng Qi Keqiu Li |
| author_sort | Yanan Yang |
| collection | DOAJ |
| description | Colocating workloads are commonly used in datacenters to improve server utilization. However, the unpredictable application performance degradation caused by the contention for shared resources makes the problem difficult and limits the efficiency of this approach. This problem has sparked research in hardware and software techniques that focus on enhancing the datacenters’ isolation abilities. There is still lack of a comprehensive benchmark suite to evaluate such techniques. To address this problem, we present SDCBench, a new benchmark suite that is specifically designed for workload colocation and characterization in datacenters. SDCBench includes 16 applications that span a wide range of cloud scenarios, which are carefully selected from the existing benchmarks using the clustering analysis method. SDCBench implements a robust statistical methodology to support workload colocation and proposes a concept of latency entropy for measuring the isolation ability of cloud systems. It enables cloud tenants to understand the performance isolation ability in datacenters and choose their best-fitted cloud services. For cloud providers, it also helps them to improve the quality of service to increase their revenues. Experimental results show that SDCBench can simulate different workload colocation scenarios by generating pressures on multidimensional resources with simple configurations. We also use SDCBench to compare the latency entropies in public cloud platforms such as Huawei Cloud and AWS Cloud and a local prototype system FlameCluster-II; the evaluation results show FlameCluster-II has the best performance isolation ability over these three cloud systems, with 0.99 of experience availability and 0.29 of latency entropy. |
| first_indexed | 2024-03-13T07:13:27Z |
| format | Article |
| id | doaj.art-e51400411cd24287846f3c7ab7ff74ea |
| institution | Directory Open Access Journal |
| issn | 2771-5892 |
| language | English |
| last_indexed | 2024-03-13T07:13:27Z |
| publishDate | 2022-01-01 |
| publisher | American Association for the Advancement of Science (AAAS) |
| record_format | Article |
| series | Intelligent Computing |
| spelling | doaj.art-e51400411cd24287846f3c7ab7ff74ea2023-06-05T16:38:11ZengAmerican Association for the Advancement of Science (AAAS)Intelligent Computing2771-58922022-01-01202210.34133/2022/9810691SDCBench: A Benchmark Suite for Workload Colocation and Evaluation in DatacentersYanan Yang0Xiangyu Kong1Laiping Zhao2Yiming Li3Huanyu Zhang4Jie Li5Heng Qi6Keqiu Li71 Tianjin Key Lab. of Advanced Networking, College of Intelligence and Computing, Tianjin University, China2 School of Computer Science and Technology, Dalian University of Technology, Dalian, China1 Tianjin Key Lab. of Advanced Networking, College of Intelligence and Computing, Tianjin University, China1 Tianjin Key Lab. of Advanced Networking, College of Intelligence and Computing, Tianjin University, China1 Tianjin Key Lab. of Advanced Networking, College of Intelligence and Computing, Tianjin University, China1 Tianjin Key Lab. of Advanced Networking, College of Intelligence and Computing, Tianjin University, China2 School of Computer Science and Technology, Dalian University of Technology, Dalian, China1 Tianjin Key Lab. of Advanced Networking, College of Intelligence and Computing, Tianjin University, ChinaColocating workloads are commonly used in datacenters to improve server utilization. However, the unpredictable application performance degradation caused by the contention for shared resources makes the problem difficult and limits the efficiency of this approach. This problem has sparked research in hardware and software techniques that focus on enhancing the datacenters’ isolation abilities. There is still lack of a comprehensive benchmark suite to evaluate such techniques. To address this problem, we present SDCBench, a new benchmark suite that is specifically designed for workload colocation and characterization in datacenters. SDCBench includes 16 applications that span a wide range of cloud scenarios, which are carefully selected from the existing benchmarks using the clustering analysis method. SDCBench implements a robust statistical methodology to support workload colocation and proposes a concept of latency entropy for measuring the isolation ability of cloud systems. It enables cloud tenants to understand the performance isolation ability in datacenters and choose their best-fitted cloud services. For cloud providers, it also helps them to improve the quality of service to increase their revenues. Experimental results show that SDCBench can simulate different workload colocation scenarios by generating pressures on multidimensional resources with simple configurations. We also use SDCBench to compare the latency entropies in public cloud platforms such as Huawei Cloud and AWS Cloud and a local prototype system FlameCluster-II; the evaluation results show FlameCluster-II has the best performance isolation ability over these three cloud systems, with 0.99 of experience availability and 0.29 of latency entropy.https://spj.science.org/doi/10.34133/2022/9810691 |
| spellingShingle | Yanan Yang Xiangyu Kong Laiping Zhao Yiming Li Huanyu Zhang Jie Li Heng Qi Keqiu Li SDCBench: A Benchmark Suite for Workload Colocation and Evaluation in Datacenters Intelligent Computing |
| title | SDCBench: A Benchmark Suite for Workload Colocation and Evaluation in Datacenters |
| title_full | SDCBench: A Benchmark Suite for Workload Colocation and Evaluation in Datacenters |
| title_fullStr | SDCBench: A Benchmark Suite for Workload Colocation and Evaluation in Datacenters |
| title_full_unstemmed | SDCBench: A Benchmark Suite for Workload Colocation and Evaluation in Datacenters |
| title_short | SDCBench: A Benchmark Suite for Workload Colocation and Evaluation in Datacenters |
| title_sort | sdcbench a benchmark suite for workload colocation and evaluation in datacenters |
| url | https://spj.science.org/doi/10.34133/2022/9810691 |
| work_keys_str_mv | AT yananyang sdcbenchabenchmarksuiteforworkloadcolocationandevaluationindatacenters AT xiangyukong sdcbenchabenchmarksuiteforworkloadcolocationandevaluationindatacenters AT laipingzhao sdcbenchabenchmarksuiteforworkloadcolocationandevaluationindatacenters AT yimingli sdcbenchabenchmarksuiteforworkloadcolocationandevaluationindatacenters AT huanyuzhang sdcbenchabenchmarksuiteforworkloadcolocationandevaluationindatacenters AT jieli sdcbenchabenchmarksuiteforworkloadcolocationandevaluationindatacenters AT hengqi sdcbenchabenchmarksuiteforworkloadcolocationandevaluationindatacenters AT keqiuli sdcbenchabenchmarksuiteforworkloadcolocationandevaluationindatacenters |