RootkitDet: Practical end-to-end defense against kernel rootkits in a cloud environment

Lingchen Zhang, Sachin Shetty, Peng Liu, Jiwu Jing

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

In cloud environments, kernel-level rootkits still pose serious security threats to guest OSes. Existing defenses against kernel-level rootkit have limitations when applied to cloud environments. In this paper, we propose RootkitDet, an end-to-end defense system capable of detecting and diagnosing rootkits in guest OSes with the intent to recover the system modifications caused by the rootkits in cloud environments. RootkitDet detects rootkits by identifying suspicious code region in the kernel space of guest OSes through the underneath hypervisor, performs diagnosis on the code of the detected rootkit to categorize it and identify modifications, and reverses the modifications if possible to eliminate the effect of rootkits. Our evaluation results show that the RootkitDet is effective on detection of kernel-level rootkits and recovery modifications with less than 1% performance overhead to the guest OSes and the computation and network overhead is linear with the quantity of the VM instances being monitored.

Original languageEnglish (US)
Title of host publicationComputer Security, ESORICS 2014 - 19th European Symposium on Research in Computer Security, Proceedings
PublisherSpringer Verlag
Pages475-493
Number of pages19
EditionPART 2
ISBN (Print)9783319112114
DOIs
Publication statusPublished - Jan 1 2014
Event19th European Symposium on Research in Computer Security, ESORICS 2014 - Wroclaw, Poland
Duration: Sep 7 2014Sep 11 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 2
Volume8713 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other19th European Symposium on Research in Computer Security, ESORICS 2014
CountryPoland
CityWroclaw
Period9/7/149/11/14

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Zhang, L., Shetty, S., Liu, P., & Jing, J. (2014). RootkitDet: Practical end-to-end defense against kernel rootkits in a cloud environment. In Computer Security, ESORICS 2014 - 19th European Symposium on Research in Computer Security, Proceedings (PART 2 ed., pp. 475-493). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8713 LNCS, No. PART 2). Springer Verlag. https://doi.org/10.1007/978-3-319-11212-1_27