UC San Diego

Modern data centers host hundreds of applications with hundreds of thousands of flows and varied traffic patterns. Efficient management of the data center flows is critical in order to ensure high utilization of the network and also achieving desired application performance. Numerous proposals aim to improve data center and enterprise networks through better management of network traffic. Examples include load balancing using Equal Cost Multipathing (ECMP), traffic engineering, precise per-flow rate limiting, scalable flow prioritization and classification, and support for microsecond-scale TDMA. These proposals rely on (1) support from the switching hardware for in-network flow management and/or (2) require the end host data plane being sufficiently performant and high precision, in order to work in coordination with network switches and centralized controllers. While these flow management proposals meet the high resource utilization and performance requirement, often they rely on hardware modifications making them hard to deploy or cost- prohibitive, or, inextensible interfaces that make it hard for them to inter- operate with the existing software stack in the end hosts. To address these challenges, we evaluate and propose readily deployable, cost-effective mechanisms for flow management both within the data center network fabric, leveraging commodity switch hardware and at the end host stack by leveraging the high performance packet processing libraries with better interfacing with the applications and the network. We propose Weighted Cost Multipathing (WCMP) as an in-network flow management framework that allows network operators to manage flows and load balance traffic with existing commodity switches. WCMP requires no hardware modification and allows fair bandwidth sharing between flows in presence of failures and asymmetry deliver high utilization and application performance. For end host based flow management, we evaluate the existing state-of-art mechanisms for flow management and leverage the high performance packet processing frameworks such as Dataplane development kit (DPDK) to propose a software defined dataplane (SDD) that allows the end host stack to interface with a remote controller and provide fine-grain control over packet transmissions and flow management