Achieving Consistent Real-Time Latency at Scale in a Commodity Virtual Machine Environment Through Socket Outsourcing-Based Network Stacks
It is challenging to achieve a consistent real-time (RT) response time in commodity virtual machine (VM) environments because they have longer and more complex network protocol stacks. This paper analyzes such network stacks and proposes a method that achieves consistent latency in a Linux KVM-based...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2018-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8501912/ |
| _version_ | 1819174795768496128 |
|---|---|
| author | Oscar F. Garcia Yasushi Shinjo Calton Pu |
| author_facet | Oscar F. Garcia Yasushi Shinjo Calton Pu |
| author_sort | Oscar F. Garcia |
| collection | DOAJ |
| description | It is challenging to achieve a consistent real-time (RT) response time in commodity virtual machine (VM) environments because they have longer and more complex network protocol stacks. This paper analyzes such network stacks and proposes a method that achieves consistent latency in a Linux KVM-based hosted environment. The analysis identifies a priority inversion in the interrupt-first host kernel of vanilla Linux, and the proposed method addresses it by using the PREEMPT_RT patch. Subsequently, the analysis identifies another priority inversion in softirq handling of the host kernel. The proposed method addresses it by dividing softirq handling into RT and non-RT types. The analysis then identifies the cache pollution problem by co-located non-RT servers and the latter priority inversion in a guest kernel. The proposed method addresses them by socket outsourcing, in which a guest kernel delegates network processing to the host kernel. The proposed method achieved consistent latency. Compared to the threaded interrupt handling method, the proposed method reduced the standard deviation (SD) of the latencies of a simple RT server by a factor of 6, achieving 5.6% higher throughput and 32% lower CPU utilization. Compared to the exclusive CPU method, the proposed method reduced the SD by a factor of 2 and prevented underutilization of the exclusive CPU. The proposed method was more scalable in terms of the number of RT VMs. A four-CPU host was able to execute 40 RT VMs using the proposed method while maintaining the throughputs of non-RT servers. |
| first_indexed | 2024-12-22T20:44:39Z |
| format | Article |
| id | doaj.art-b4da723aa79346c999965ef91e674608 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-22T20:44:39Z |
| publishDate | 2018-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-b4da723aa79346c999965ef91e6746082022-12-21T18:13:16ZengIEEEIEEE Access2169-35362018-01-016699616997710.1109/ACCESS.2018.28772968501912Achieving Consistent Real-Time Latency at Scale in a Commodity Virtual Machine Environment Through Socket Outsourcing-Based Network StacksOscar F. Garcia0https://orcid.org/0000-0001-6865-4930Yasushi Shinjo1Calton Pu2Department of Computer Science, University of Tsukuba, Tsukuba, JapanDepartment of Computer Science, University of Tsukuba, Tsukuba, JapanCollege of Computing, Georgia Institute of Technology, Atlanta, GA, USAIt is challenging to achieve a consistent real-time (RT) response time in commodity virtual machine (VM) environments because they have longer and more complex network protocol stacks. This paper analyzes such network stacks and proposes a method that achieves consistent latency in a Linux KVM-based hosted environment. The analysis identifies a priority inversion in the interrupt-first host kernel of vanilla Linux, and the proposed method addresses it by using the PREEMPT_RT patch. Subsequently, the analysis identifies another priority inversion in softirq handling of the host kernel. The proposed method addresses it by dividing softirq handling into RT and non-RT types. The analysis then identifies the cache pollution problem by co-located non-RT servers and the latter priority inversion in a guest kernel. The proposed method addresses them by socket outsourcing, in which a guest kernel delegates network processing to the host kernel. The proposed method achieved consistent latency. Compared to the threaded interrupt handling method, the proposed method reduced the standard deviation (SD) of the latencies of a simple RT server by a factor of 6, achieving 5.6% higher throughput and 32% lower CPU utilization. Compared to the exclusive CPU method, the proposed method reduced the SD by a factor of 2 and prevented underutilization of the exclusive CPU. The proposed method was more scalable in terms of the number of RT VMs. A four-CPU host was able to execute 40 RT VMs using the proposed method while maintaining the throughputs of non-RT servers.https://ieeexplore.ieee.org/document/8501912/KVMLinuxpriority inversionreal-time latencysocket outsourcing |
| spellingShingle | Oscar F. Garcia Yasushi Shinjo Calton Pu Achieving Consistent Real-Time Latency at Scale in a Commodity Virtual Machine Environment Through Socket Outsourcing-Based Network Stacks IEEE Access KVM Linux priority inversion real-time latency socket outsourcing |
| title | Achieving Consistent Real-Time Latency at Scale in a Commodity Virtual Machine Environment Through Socket Outsourcing-Based Network Stacks |
| title_full | Achieving Consistent Real-Time Latency at Scale in a Commodity Virtual Machine Environment Through Socket Outsourcing-Based Network Stacks |
| title_fullStr | Achieving Consistent Real-Time Latency at Scale in a Commodity Virtual Machine Environment Through Socket Outsourcing-Based Network Stacks |
| title_full_unstemmed | Achieving Consistent Real-Time Latency at Scale in a Commodity Virtual Machine Environment Through Socket Outsourcing-Based Network Stacks |
| title_short | Achieving Consistent Real-Time Latency at Scale in a Commodity Virtual Machine Environment Through Socket Outsourcing-Based Network Stacks |
| title_sort | achieving consistent real time latency at scale in a commodity virtual machine environment through socket outsourcing based network stacks |
| topic | KVM Linux priority inversion real-time latency socket outsourcing |
| url | https://ieeexplore.ieee.org/document/8501912/ |
| work_keys_str_mv | AT oscarfgarcia achievingconsistentrealtimelatencyatscaleinacommodityvirtualmachineenvironmentthroughsocketoutsourcingbasednetworkstacks AT yasushishinjo achievingconsistentrealtimelatencyatscaleinacommodityvirtualmachineenvironmentthroughsocketoutsourcingbasednetworkstacks AT caltonpu achievingconsistentrealtimelatencyatscaleinacommodityvirtualmachineenvironmentthroughsocketoutsourcingbasednetworkstacks |