BinSIM: Trace-based semantic binary diffing via system call sliced segment equivalence checking

Jiang Ming, Dongpeng Xu, Yufei Jiang, Dinghao Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Detecting differences between two binary executables (binary diffing), first derived from patch analysis, have been widely employed in various software security analysis tasks, such as software plagiarism detection and malware lineage inference. Especially when analyzing malware variants, pervasive code obfuscation techniques have driven recent work towards determining semantic similarity in spite of ostensible difference in syntax. Existing ways rely on either comparing runtime behaviors or modeling code snippet semantics with symbolic execution. However, neither approach delivers the expected precision. In this paper, we propose system call sliced segment equivalence checking, a hybrid method to identify fine-grained semantic similarities or differences between two execution traces. We perform enhanced dynamic slicing and symbolic execution to compare the logic of instructions that impact on the observable behaviors. Our approach improves existing semantics-based binary diffing by 1) inferring whether two executable binaries’ behaviors are conditionally equivalent; 2) detecting the similarities or differences, whose effects spread across multiple basic blocks. We have developed a prototype, called BinSim, and performed empirical evaluations against sophisticated obfuscation combinations and more than 1,000 recent malware samples, including now-infamous crypto ransomware. Our experimental results show that BinSim can successfully identify fine-grained relations between obfuscated binaries, and outperform existing binary diffing tools in terms of better resilience and accuracy.

Original languageEnglish (US)
Title of host publicationProceedings of the 26th USENIX Security Symposium
PublisherUSENIX Association
Pages253-270
Number of pages18
ISBN (Electronic)9781931971409
StatePublished - Jan 1 2017
Event26th USENIX Security Symposium - Vancouver, Canada
Duration: Aug 16 2017Aug 18 2017

Publication series

NameProceedings of the 26th USENIX Security Symposium

Conference

Conference26th USENIX Security Symposium
CountryCanada
CityVancouver
Period8/16/178/18/17

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Information Systems
  • Safety, Risk, Reliability and Quality

Fingerprint Dive into the research topics of 'BinSIM: Trace-based semantic binary diffing via system call sliced segment equivalence checking'. Together they form a unique fingerprint.

Cite this