Proprietary network protocols - risky business on the wire. presented at BSidesUK 2015

by Jakub Kaluzny,

Summary : When speed and latency counts, there is no place for standard HTTP/SSL stack and a wise head comes up with a proprietary network protocol. How to deal with embedded software or thick clients using protocols with no documentation at all? Binary TCP connections, unlike anything, impossible to be adapted by a well-known local proxy. Without disassembling the protocol, pentesting the server backend is very limited. However, when you dive inside this traffic and reverse-engineer the communication inside, you are there. Welcome to the world full of own cryptography, revertible hash algorithms and no access control at all.
We would like to present our approach and a short guideline how to reverse engineer proprietary protocols. To demonstrate, we will show you few case-studies, which in our opinion are a quintessence of ""security by obscurity"" - the most interesting examples from real-life financial industry software, which is a particularly risky business regarding security.