With the popularity of Internet applications (e.g., web search, cloud computing, social networking, etc.), data centers [3–5] have been built at an unprecedented scale across the world. Among the applications hosted in data centers, some require ultra low round-trip latency, while others require large sustained throughput for bulk data transfers. This imposes stringent requirements on the data center network fabrics and protocols.

Generally, switches only provide forwarding services and have no credit to computation in distributed computing frameworks. The emerging programmable switches make in-network computing possible, and enable some computation to be offloaded to the switch data plane. In this thesis, we propose P4COM, a first attempt to implement MapReduce-like functions...[ Read more ]

With the popularity of Internet applications (e.g., web search, cloud computing, social networking, etc.), data centers [3–5] have been built at an unprecedented scale across the world. Among the applications hosted in data centers, some require ultra low round-trip latency, while others require large sustained throughput for bulk data transfers. This imposes stringent requirements on the data center network fabrics and protocols.

Generally, switches only provide forwarding services and have no credit to computation in distributed computing frameworks. The emerging programmable switches make in-network computing possible, and enable some computation to be offloaded to the switch data plane. In this thesis, we propose P4COM, a first attempt to implement MapReduce-like functions in programmable switches. P4COM consists of three modules. First, P4COM automatically translates application logic to switch data plane programs. Second, P4COM adopts a memory management policy to efficiently utilize the limited switch on-chip memory. Third, P4COM provides fault-tolerance service to handle packet loss. We have built a system prototype of P4COM on a testbed with a Barefoot Tofino switch and multiple commodity servers. With a combination of testbed experiments and large-scale simulations, we show that P4COM is able to achieve line-rate processing at 10Gbps links, and can significantly reduce the data shuffling traffic by 20–40%, depending on data skewness and the ratio between mappers and reducers.