Abstract

We investigate the advantages of using co-packaged optics for building low-diameter, large-scale high-performance computing (HPC) and data center networks. The increased escape bandwidth offered by co-packaged optics can enable high-radix switch implementations of more than 150 switch ports, which can be combined with data rates of up to 400 Gb/s per port. From the network architecture perspective, the key benefits of using co-packaged optics in future fat-tree networks include (a) the ability to implement large-scale topologies of ${\gt}{11}{,}{000}$ end points by eliminating the need for a third switching layer and (b) the ability to provide up to ${4\times}$ higher bisection bandwidth compared to existing solutions, reducing at the same time the number of required switch application-specific integrated circuits by ${\gt}{80}\%$. From the network operation perspective, both reduced energy consumption and lower packet delays can be achieved since fewer hops are required; i.e., packets need to traverse fewer serializer/deserializer lanes and fewer switch buffers, which reduces the probability of contending with other packets and improves the tolerance of network congestion. The performance of the proposed architecture is evaluated via discrete-event simulations for a wide range of representative HPC synthetic-traffic cases that include both hotspot and non-hotspot scenarios. The simulation results suggest that co-packaged optics form a promising solution to keep up with bandwidth scaling in future networks, while the reduced number of switching layers can lead to significant mean packet delay improvements that start from 30% and reach up to 74% for high-load conditions.

© 2020 Optical Society of America

Toward higher-radix switches with co-packaged optics for improved network locality in data center and HPC networks [Invited]

Pavlos Maniotis, Laurent Schares, Daniel M. Kuchta, and Bengi Karacali
J. Opt. Commun. Netw. 14(6) C1-C10 (2022)

Exploring the benefits of using co-packaged optics in data center and AI supercomputer networks: a simulation-based analysis [Invited]

Pavlos Maniotis and Daniel M. Kuchta
J. Opt. Commun. Netw. 16(2) A143-A156 (2024)

Intra-node high-performance computing network architecture with nanosecond-scale photonic switches [Invited]

Pavlos Maniotis, Nicolas Dupuis, Laurent Schares, Daniel M. Kuchta, Marc A. Taubenblatt, and Benjamin G. Lee
J. Opt. Commun. Netw. 12(12) 367-377 (2020)

Optics enabled networks and architectures for data center cost and power efficiency [Invited]

Marc Taubenblatt, Pavlos Maniotis, and Asser Tantawi
J. Opt. Commun. Netw. 14(1) A41-A49 (2022)

Network Architecture in the Era of Integrated Optics

Cyriel Minkenberg, Nick Kucharewski, and German Rodriguez
J. Opt. Commun. Netw. 11(1) A72-A83 (2019)