As cloud computing has become a popular service recent years, a number of big companies, such as Google, Yahoo!, Microsoft, Amazon and Apple, have constructed large datacenters to provide such services. Meanwhile, datacenter monitoring and network traffic analysis is important for planning, building and managing of datacenters. However, research in these areas has become challenging because of the large investment needed for building datacenter-scale testbeds....[ Read more ]

As cloud computing has become a popular service recent years, a number of big companies, such as Google, Yahoo!, Microsoft, Amazon and Apple, have constructed large datacenters to provide such services. Meanwhile, datacenter monitoring and network traffic analysis is important for planning, building and managing of datacenters. However, research in these areas has become challenging because of the large investment needed for building datacenter-scale testbeds.

In this thesis, based on the analysis of characteristics of the network consisting of different virtual machines on one single physical machine and that consisting of different physical machines, we propose to emulate the datacenter network environment based on the Xen architecture, on which we can host a number of virtual machines emulating physical machines residing in a datacenter network. Thus, the emulation environment can provide a good platform for planning, and deciding monitoring strategy without costly full implementation for large scale equipment. We have evaluated our emulation based on the comparison of network performance data under TCP workloads.

Meanwhile, the network analysis based on the monitoring traces generated on the emulation environment or real datacenters is also a grand technical challenge in large datacenters. It is also crucial since it provides evidences for anomaly detection which is important for the security of cloud computing services. In this thesis, we have studied the structural characteristics of IP address octets observed in large datacenters, and presented centroid based measures to capture the inherent IP structure in high-volume datacenter traffic, and subsequently designed a simple yet effective algorithm to detect abnormal traffic patterns caused by network attacks such as worms, viruses, and distributed denial of service (DDoS) attacks. We evaluate the effectiveness and efficiency of this algorithm with synthetic traffic that combines real datacenter traffic collected from a large Internet content provider with worm traces or DDos packets.