Bindex 2.0 presented at Virus Bulletin 2011

by Tim Ebringer (Microsoft),

Tags: Security


Summary : "We present the algorithms, applications and new experiments based on the next generation of Bindex, our in-house binary
search engine. It enables binary queries on as little as four bytes, across terabytes of data. The latest version
successfully scaled up to a much bigger deployment, meeting or exceeding all of our performance goals. At present, we
index memory dumps of malware processes (to bypass obfuscation and packers), as well a clean file set.
Bindex is used to find related samples, name samples and avoid false positives. Its greatest feature is that it provides
instant feedback for malware researchers, who can perform several speculative queries in the time it takes to rebuild the
signatures. It is now ingrained into our research workflow, and we present several examples of unusual and successful
queries, such as a binary query against the bytes in the embedded GIF file used by a rogue.
Early Bindex results were presented at CARO 2010, but since then, the algorithms and data structures have changed
significantly to address scalability. We will present the new algorithms behind Bindex 2.0 as well as the workflows our
research team has adopted over the first production year of its life.
Finally, we will present a new, derived application, which can visually provide a 'heat map' in IDA, of the 'rareness' of
bytes. For library code, which has typically been indexed many times, we can provide a visual cue that this code is
common, and not suitable for a signature.