The goal for this dissertation is to propose and evaluate IO consolidation as a cost-efficient approach to server design. In our context, IO consolidation is to replace network interface devices dedicated to each server and corresponding network switches with lower cost IO switches and fewer shared network interface devices. Our key contributions include: (a) a novel, general-purpose infrastructure (GPI) that seamlessly supports conventional and future interconnect protocols in server blade enclosures; and (b) a quantitative evaluation (and associated cost and performance models) of the efficiency of IO consolidation in relation to conventional network consolidation.

In the first part of the dissertation, we focus on the design of a GPI. We combine technology trends and insights into several fabrics' physical layer similarities and discuss tradeoffs in the design to address cost, density, flexibility, and availability issues. We then architect an interconnect infrastructure by volumetric positioning of switch bays and adaptive grouping of signal lanes. The resulting infrastructure can seamlessly support various fabric protocols and provides a mechanism to implement IO consolidation.

In the second part, we evaluate performance and cost efficiencies of IO consolidation solutions with respect to a network consolidation solution within a GPI. We choose two industry-standard protocols—PCI Express as our IO consolidation protocol since it is the IO interface in high-volume computers, and 10Gigabit-Ethernet as our reference network consolidation protocol since it has become the de-facto standard for high-performance networks. Currently, no full systems with IO consolidation exist. Hence, for the purpose of evaluation, we derive our own performance metrics and cost models based on our insights into the characteristics of the protocols, the constraints of the GPI, and the characteristics of datacenter applications. We use a hybrid evaluation method that consists of hardware emulators, a software simulator, and limited-function hardware system prototypes.

Our main result proves that IO consolidation can be cost- and performance-efficient within the GPI architecture, but requires careful consideration of tradeoffs. Our results also show that IO fabrics with relatively narrow links are compelling for their dramatic cost savings and scalability, while maintaining acceptable performance with well-balanced bandwidth utilization throughout the infrastructure.