Identification of Malicious SSL Networks by Subgraph Anomaly Detection presented at flocon 2018

by Dhia Mahjoub, Thomas Mathew,

Summary : Sophisticated attackers use SSL to secure communications to command-and-control domains or provide their clients with secure hosting infrastructure. The goal of this talk is to describe methods to automatically detect threats from SSL scan data without relying on prior seeds. We present a series of statistical graph techniques that allow us to discover botnet and bulletproof hosting IP space by examining SSL distribution patterns from open source data. SSL data obtained from scanning the entire IPv4 namespace can be represented as a 4 million node bipartite graphs where a x509 common name is connected to either an IP/CIDR/ASN via an edge. The challenge we face is to identify common names that are attached to a malicious subgraph of the larger ASN-CommonName graph. The identification of malicious subgraphs involves splitting the graph into its component pieces and then performing tests of similarity between the various subgraphs. The subgraph comparison requires constructing a distance metric. We use the concept of relative entropy to create a pairwise distance metric between any two common names and any two ASNs. The metric allows us to generalize the concept of regular and anomalous SSL distribution patterns. Consequently, by setting relative entropy thresholds we can identify anomalous SSL certificates. The measure of relative entropy is useful in identifying domains that have anomalous network structures. The domains we found in this case were related to the Zbot proxy network. The Zbot proxy network contains a structure similar to popular CDNs like Akamai, Google, fbcdn, etc but instead rely on compromised devices to relay their data. We provide evidence collected over a 5 month period that this anomalous network structure is unique botnets and can be used as a signal for identification. Layering these SSL signals with passive DNS data we create a pipeline that can extract Zbot domains with high accuracy.Attendees will learn:Attendees will learn about the current ways malicious operators use SSL to secure their command-and-control and IP infrastructure. This includes how bulletproof hosters use SSL to host carding websites and ZBot operators use SSL to protect their C2C servers. They will also learn techniques that are useful for identifying anomalous subgraphs found within a bipartite graph. The algorithms discussed in this talk are not unique to SSL and can be applied to other heavily network intensive datasets.