Auditing binaries for security vulnerabilities presented at BlackHatWindows 2001

by Havlar Flake,

Summary : Even with the advent of the open-source movement, many critical systems such as most firewalls and many high-performance web servers still run closed-source software. Few security architectures are not prone to failure if the security of critical software fails. As Joey__ demonstrated in his speech at Black Hat Singapore, reverse engineering can be used in order to find unknown vulnerabilities in closed-source software. The first half of the speech will give in-depth coverage on reverse engineering of compiled C/C++ code on the x86 platform with specific focus on its immediate application to network security. The speech will begin with a short note on the legality of reverse engineering for security purposes in both the US and EU. The following topics will include an thorough review of common C/C++ programming mistakes that can lead to illicit code execution, how these mistakes look when they have been compiled into assembly code and how one can go about finding these problems. The techniques which are explained will then be used to find a yet-undiscovered format string bug in a major web server.
The second half of the speech will focus on more "esoteric" topics in relation to reverse engineering, including reconstruction of C++ classes via the "this"-pointer (both manually and automatically) and ways to faciliate and automate the auditing process.
A solid knowledge of C and a decent understanding of x86 assembly language
will help in getting most out of this speech.
HalVar Flake is an independent reverse engineer specializing in application security evaluation and source reconstruction. With a background in copyprotection, he realized one day that reverse engineering was a very handy asset on closed-source platforms such as NT/2k. Fluent in various assembly languages and C/C++, he is furthering his research on his days off from his mandatory military service.
Previous work experiences include: Detection and Exploitation of Buffer Overflows and format string vulnerabilities under NT, Analysis of PE(Win32)-Virii, Polymorphic Engines, Trojans, CPU-Emulators and many other things that have been written to be annoying to reverse engineer.
Their Presentation! (PowerPoint 464k)