Detecting malicious exploit kits using tree-based similarity searches

Teryl Taylort, Xin Hut, Ting Wang, Jiyong Jang, Marc Ph Stoecklint, Fabian Monroset, Reiner Sailer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Scopus citations

Abstract

Unfortunately, the computers we use for everyday activities can be infiltrated while simply browsing innocuous sites that, unbeknownst to the website owner, may be laden with malicious advertisements. So-called malvertising, redirects browsers to web-based exploit kits that are designed to find vulnerabilities in the browser and subsequently download malicious payloads. We propose a new approach for detecting such malfeasance by leveraging the inherent structural patterns in HTTP traffic to classify exploit kit instances. Our key insight is that an exploit kit leads the browser to download payloads using multiple requests from malicious servers. We capture these interactions in a "tree-like" form, and using a scalable index of malware samples, model the detection process as a subtree similarity search problem. The approach is evaluated on 3800 hours of real-world traffic including over 4 billion flows and reduces false positive rates by four orders of magnitude over current state-of-the-art techniques with comparable true positive rates. We show that our approach can operate in near real-time, and is able to handle peak traffic levels on a large enterprise network - identifying 28 new exploit kit instances during our analysis period.

Original languageEnglish (US)
Title of host publicationCODASPY 2016 - Proceedings of the 6th ACM Conference on Data and Application Security and Privacy
PublisherAssociation for Computing Machinery, Inc
Pages255-266
Number of pages12
ISBN (Electronic)9781450339353
DOIs
StatePublished - Mar 9 2016
Event6th ACM Conference on Data and Application Security and Privacy, CODASPY 2016 - New Orleans, United States
Duration: Mar 9 2016Mar 11 2016

Publication series

NameCODASPY 2016 - Proceedings of the 6th ACM Conference on Data and Application Security and Privacy

Other

Other6th ACM Conference on Data and Application Security and Privacy, CODASPY 2016
CountryUnited States
CityNew Orleans
Period3/9/163/11/16

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Information Systems
  • Software

Fingerprint Dive into the research topics of 'Detecting malicious exploit kits using tree-based similarity searches'. Together they form a unique fingerprint.

  • Cite this

    Taylort, T., Hut, X., Wang, T., Jang, J., Stoecklint, M. P., Monroset, F., & Sailer, R. (2016). Detecting malicious exploit kits using tree-based similarity searches. In CODASPY 2016 - Proceedings of the 6th ACM Conference on Data and Application Security and Privacy (pp. 255-266). (CODASPY 2016 - Proceedings of the 6th ACM Conference on Data and Application Security and Privacy). Association for Computing Machinery, Inc. https://doi.org/10.1145/2857705.2857718