Lightweight kernel isolation with virtualization and VM functions

Vikram Narayanan, Yongzhe Huang, Gang Tan, Trent Ray Jaeger, Anton Burtsev

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Commodity operating systems execute core kernel subsystems in a single address space along with hundreds of dynamically loaded extensions and device drivers. Lack of isolation within the kernel implies that a vulnerability in any of the kernel subsystems or device drivers opens a way to mount a successful attack on the entire kernel. Historically, isolation within the kernel remained prohibitive due to the high cost of hardware isolation primitives. Recent CPUs, however, bring a new set of mechanisms. Extended page-Table (EPT) switching with VM functions and memory protection keys (MPKs) provide memory isolation and invocations across boundaries of protection domains with overheads comparable to system calls. Unfortunately, neither MPKs nor EPT switching provide architectural support for isolation of privileged ring 0 kernel code, i.e., control of privileged instructions and well-defined entry points to securely restore state of the system on transition between isolated domains. Our work develops a collection of techniques for lightweight isolation of privileged kernel code. To control execution of privileged instructions, we rely on a minimal hypervisor that transparently deprivileges the system into a non-root VT-x guest. We develop a new isolation boundary that leverages extended page table (EPT) switching with the VMFUNC instruction. We define a set of invariants that allows us to isolate kernel components in the face of an intricate execution model of the kernel, e.g., provide isolation of preemptable, concurrent interrupt handlers. To minimize overheads of virtualization, we develop support for exitless interrupt delivery across isolated domains. We evaluate our approach by developing isolated versions of several device drivers in the Linux kernel.

Original languageEnglish (US)
Title of host publicationVEE 2020 - Proceedings of the 16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments
PublisherAssociation for Computing Machinery, Inc
Pages157-171
Number of pages15
ISBN (Electronic)9781450375542
DOIs
StatePublished - Mar 17 2020
Event16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, VEE 2020 - Lausanne, Switzerland
Duration: Mar 17 2020 → …

Publication series

NameVEE 2020 - Proceedings of the 16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments

Conference

Conference16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, VEE 2020
CountrySwitzerland
CityLausanne
Period3/17/20 → …

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Software
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Lightweight kernel isolation with virtualization and VM functions'. Together they form a unique fingerprint.

  • Cite this

    Narayanan, V., Huang, Y., Tan, G., Jaeger, T. R., & Burtsev, A. (2020). Lightweight kernel isolation with virtualization and VM functions. In VEE 2020 - Proceedings of the 16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (pp. 157-171). (VEE 2020 - Proceedings of the 16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments). Association for Computing Machinery, Inc. https://doi.org/10.1145/3381052.3381328