Cryptographic Function Detection in Obfuscated Binaries via Bit-Precise Symbolic Loop Mapping

Dongpeng Xu, Jiang Ming, Dinghao Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

31 Scopus citations

Abstract

Cryptographic functions have been commonly abused by malware developers to hide malicious behaviors, disguise destructive payloads, and bypass network-based firewalls. Now-infamous crypto-ransomware even encrypts victim's computer documents until a ransom is paid. Therefore, detecting cryptographic functions in binary code is an appealing approach to complement existing malware defense and forensics. However, pervasive control and data obfuscation schemes make cryptographic function identification a challenging work. Existing detection methods are either brittle to work on obfuscated binaries or ad hoc in that they can only identify specific cryptographic functions. In this paper, we propose a novel technique called bit-precise symbolic loop mapping to identify cryptographic functions in obfuscated binary code. Our trace-based approach captures the semantics of possible cryptographic algorithms with bit-precise symbolic execution in a loop. Then we perform guided fuzzing to efficiently match boolean formulas with known reference implementations. We have developed a prototype called CryptoHunt and evaluated it with a set of obfuscated synthetic examples, well-known cryptographic libraries, and malware. Compared with the existing tools, CryptoHunt is a general approach to detecting commonly used cryptographic functions such as TEA, AES, RC4, MD5, and RSA under different control and data obfuscation scheme combinations.

Original languageEnglish (US)
Title of host publication2017 IEEE Symposium on Security and Privacy, SP 2017 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages921-937
Number of pages17
ISBN (Electronic)9781509055326
DOIs
StatePublished - Jun 23 2017
Event2017 IEEE Symposium on Security and Privacy, SP 2017 - San Jose, United States
Duration: May 22 2017May 24 2017

Publication series

NameProceedings - IEEE Symposium on Security and Privacy
ISSN (Print)1081-6011

Other

Other2017 IEEE Symposium on Security and Privacy, SP 2017
CountryUnited States
CitySan Jose
Period5/22/175/24/17

All Science Journal Classification (ASJC) codes

  • Safety, Risk, Reliability and Quality
  • Software
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Cryptographic Function Detection in Obfuscated Binaries via Bit-Precise Symbolic Loop Mapping'. Together they form a unique fingerprint.

Cite this