Network Awareness and Network Security presented at AUScert 2007

by John Mchugh,

Summary : Routine acquisition and aggregation of network data offers an opportunity to understand some of the forces that drive the internet. It also offers an opportunity to detect and understand a variety of phenomena that are related to overtly questionable or malicious activities on the part of network users and abusers. The initial observations analyzed by the US CERT were based on data observed at the borders of a very large network, and concentrated on characterizing network scale phenomena. Carried out on a smaller scale, if offers an opportunity to perform passive monitoring on the activities on your own network, including the detection of spyware and other forms of compromise. By monitoring the unoccupied portions of an organization's address space, scanning and other activities that are often precursors to attacks can be identified. Given cheap, fast, analysis machines and inexpensive mass storage, it is possible to maintain relatively complete activity records for all the hosts in a modest network for long periods of time. This allows us to characterize ``normal'' activity on a per host basis. The richness of the source means that it is always a source of new insights and observations.
In this talk, I will summarize a variety of large and small scale observations that have resulted from such monitoring activities. Key to this work is the choice of suitable abstractions for the representation of both data and analysis results. The talk will also consider some of the issues associated with the management of the quantities of the data involved as well as techniques for analyzing the data and presenting the analysis results. These techniques aid system managers in better understanding the activities that routinely occur on their networks and provide a baseline against which changes in behavior, whether benign or malicious can be evaluated.