My notes on where AI/ML networking stands in October 2026. This area moves fast and there is a lot of active research going on, so parts of this will age quickly. Naming is NVIDIA’s (NVLink, NCCL, PXN), AMD and Intel have their own terms for similar things.

Traffic

For comparison, a normal DC Clos carries lots of small TCP flows. ECMP spreads them fine, TCP recovers from drops, 3:1 oversubscription is ok.

A training back-end has no TCP in the data path. GPUs use RDMA (RoCEv2 or InfiniBand) and NCCL collectives (all-reduce, all-to-all). All GPUs wait for each other, so the slowest flow sets the job speed. Few huge flows, ECMP can easily put two on the same link. RDMA recovery from drops is expensive, one drop can stall the whole collective.

Networks

  • scale-up: NVLink inside the server or rack (AMD has Infinity Fabric / UALink, Intel Gaudi uses Ethernet ports)
  • back-end: GPU to GPU, 1:1, mostly plain L3 with eBGP (RFC 7938), VXLAN (RFC 7348) when GPUs are shared between tenants
  • front-end: mgmt, tenant access, often EVPN-VXLAN (RFC 8365)
  • storage: often its own fabric

Rail-optimized

8 GPUs and 8 NICs per server. NIC1 of every server goes to leaf1, NIC2 to leaf2 and so on, each leaf is a rail. GPU n talks to GPU n on other servers and stays on one leaf.

Cross-rail with PXN: data is copied over NVLink to the local GPU3 and leaves on rail 3. Without PXN it goes leaf1 -> spine -> leaf3. Busy spines often mean all-to-all traffic (MoE) or PXN switched off.

Congestion control

InfiniBand uses credit-based flow control, so there are no congestion drops. The subnet manager runs the fabric.

RoCEv2 is RDMA over UDP 4791 on Ethernet and needs help with congestion. DCQCN was built by Microsoft and Mellanox around 2015 for RDMA in Microsoft’s datacenters, cloud storage being one of the main use cases. With PFC alone, congestion pauses the whole priority hop by hop, which causes head-of-line blocking and unfairness. With DCQCN the switch marks ECN, the receiver sends CNPs back and the sender slows down before queues fill up.

ECN thresholds (Kmin, Kmax, Pmax) often need tuning per fabric. Kmin too high and PFC fires first, too low and you lose throughput.

My view is that PFC should be the last resort, a safety net with the watchdog on that ideally never triggers. Pause storms and deadlocks happen in real life. Not everyone agrees, Meta turned DCQCN off on their 400G fabric and runs PFC only, with congestion handled in the collective library. A study of DCQCN, DCTCP, TIMELY and HPCC also found they made little difference for training workloads.

Research is very active here: delay-based congestion control, HPCC, packet spraying with NIC reordering, Ultra Ethernet. UEC 1.0 came out in 2025, 1.1 is just out with in-network collectives and smaller headers.

Low hanging fruit when troubleshooting

  • MTU mismatch somewhere in the path
  • PFC/ECN config not the same on all ports
  • DSCP trust missing
  • bad optic with high BER, job is slow and nothing alarms
  • counters: pause frames, ECN marks, CNPs, queue drops, FEC
  • nccl-tests baseline while the fabric is healthy

Videos

Great source of insights, Petr Lapukhov with Jeff Doyle and Jeff Tantsura on AI/ML data center design: