Industrial Bug Mining - Extracting, Grading And Enriching The Ore Of Exploits presented at NinjaCon 2010

by Ben Nagy,

Tags: Others Data Exfiltration


Summary : If bugs are the raw ore of exploits - Rootite, if you like - then we're mining in areas where the Rootite is rare and deeply buried. Industrial scale bug mining starts with very, very fast fuzzing. In contrast to the MS Fuzzing Botnet, we use a dedicated, single purpose cluster of virtual machines which is optimised for fuzzing. Last year we released some metrics, then MS released better ones. So, we rebuilt the whole system and made it faster and more scalable - can we outperform the Redmond Botnet in one small rack? After a fuzz run, we are left with massive piles of low-grade Rootite, full of impurities such as Nullpointium, which needs to be graded and enriched before it is valuable. After grading, We "enrich" our highest grade Rootite by using differential runtracing of crashes to assist root cause analysis. The runtraces are tens of millions of lines long, but we postprocess them using magic, funky graphs and compression before comparing them side by side with the clean run. Our diff files are plaintext, small enough for us to eyeball them, and allow us navigate to any point in the trace using any debugger we choose. Feel free to drop by for a guided tour of the mine. Bring a beer.