Network data analysis for intrusion detection


Anna Sperotto


13th Dec. 2007






Intrusion Detection (ID) aims to recognise malicious activities in the monitored system through the analysis of massive quantities of data (audit data, log files, network packets, flows). In particular, in the past few year there has been an increasing interest in ID for IP
networks, in order to provide a secure network environment and services. The first phase of ID saw 10-100Mbps networks as protagonists: the wire speed, as well as the amount of data, permitted a deep inspection of all the traffic, alluring the researcher to explore the field of payload based analysis methodologies.

Nowadays, this scenario is, if not unrealistic, at least not exhaustive. Network infrastructures are fast moving towards 1-10Gbps connections with a consequent increase in both speed and amount of data passing through the wire. Under these conditions, ID in high speed network is challenging, involving different and heterogeneous data from different sources, such as packets, security resources log files and data summaries in the form of flows.

This presentation aims to describe the operational experience gained in network data collection and analysis, furnishing some preliminary observations about anomalous behaviours detection in large amount of data. In particular, our interest will focus on the possible correlation between small scale and large scale measurements. Starting from a local, but privileged, observation point (a honeypot based monitoring infrastructure) we want in the first place to investigate which is the impact of a locally detected malicious activity on the overall network traffic. On the second place, our goal is to see if a flow-based traffic analysis, the only one able to cope with the current amount of data, would be powerful enough to allow meaningful intrusion detection.