InfiniBand vs Ethernet for AI Clusters

First, get the layers straight, because most confusion here is a layer error.

• Scale-up is the interconnect inside a node — or, on rack-scale systems, inside a rack. That is NVLink, switched by NVSwitch. Fifth-generation NVLink runs at 1.8 TB/s per GPU.
• Scale-out is the network between nodes. That is InfiniBand or Ethernet, reached through the node’s NICs and external leaf and spine switches. A 400 Gb/s port is 50 GB/s; an 800 Gb/s port is 100 GB/s.

Compare those numbers and the design principle falls out immediately: per-GPU scale-up bandwidth is more than an order of magnitude above scale-out bandwidth. Every serious cluster layout is an attempt to keep traffic inside the NVLink domain and to make the traffic that must leave it as efficient as possible.

InfiniBand vs Ethernet is a question only about the scale-out layer. It has no bearing on what happens inside a node.

Distributed training is bulk-synchronous. GPUs compute, then exchange gradients or activations in a collective operation — an all-reduce, an all-gather, an all-to-all — and cannot proceed until the exchange completes. Three consequences follow, and they are why AI networks are designed differently from enterprise networks:

• The slowest path sets the pace. Every participant waits for the last one. Average bandwidth is nearly irrelevant; tail latency is what you feel.
• Traffic is bursty and synchronised. Thousands of GPUs transmit in the same instant, in predictable patterns — exactly the condition that produces congestion hot spots.
• Packet loss is catastrophic, not degrading. A dropped packet triggers a retransmission that stalls a collective, which stalls every GPU in the job.

So an AI fabric is judged on lossless behavior, congestion control and tail latency under synchronised load — not on headline port speed. That is the real subject of the InfiniBand vs Ethernet argument.

InfiniBand was purpose-built for HPC interconnect and carries several properties that map well onto AI collectives.

• Lossless by design. Credit-based flow control means a sender only transmits when the receiver has buffer space, so congestion produces back-pressure rather than drops.
• RDMA in hardware. The adapter moves data between GPU memory on different nodes without involving host CPUs, keeping latency low and predictable.
• Adaptive routing and congestion control are built into the fabric rather than bolted on, and some collective operations can be reduced inside the switches — cutting the volume that traverses the fabric at all.
• A centralized subnet manager gives one coherent view of the fabric: an operational simplification for people who know it, an unfamiliar model for people who do not.

Current generations run at NDR (400 Gb/s) and XDR (800 Gb/s) per port. The trade-offs are a narrower vendor ecosystem, a distinct skill set from the Ethernet operations most enterprises already have, and cost posture that generally sits above commodity Ethernet.

Ethernet reaches comparable behavior through RoCEv2 — RDMA over Converged Ethernet — which carries the same RDMA semantics over a routed Ethernet network. The catch has always been that Ethernet is lossy by default, so RoCE deployments depend on congestion management: priority flow control and explicit congestion notification, or newer AI-specific congestion-control schemes.

Two developments have changed the picture materially.

• AI-tuned Ethernet platforms. Switch silicon and NICs designed specifically for AI back-end networks — NVIDIA’s Spectrum-X among them — pair adaptive routing and congestion control with the NIC to reduce the hand-tuning that early RoCE deployments required.
• Ultra Ethernet. The Ultra Ethernet Consortium published its 1.0 specification in mid-2025, standardizing transport, congestion signaling and telemetry aimed squarely at AI and HPC traffic. That gives the Ethernet path a multi-vendor roadmap rather than a set of per-vendor extensions.

The honest strengths are ecosystem and operations: many switch vendors, familiar tooling, existing staff skills, and one network technology spanning back-end, front-end and storage. The honest weakness is that getting lossless behavior right takes deliberate design — a badly tuned RoCE fabric underperforms a well-built InfiniBand one by a wide margin.

Both are deployed at serious scale. Read the differences as trade-offs, not as a verdict.

Two spec-sheet details start making sense once you know the traffic pattern.

Rail-optimized topology. In an 8-GPU node, each GPU has its own NIC. A rail-optimized design connects GPU 0 in every node to one leaf switch, GPU 1 in every node to a second leaf switch, and so on — eight parallel "rails". Because collectives typically exchange data between GPUs occupying the same position across nodes, most of that traffic stays within a single rail and never touches the spine. The spine layer is still there to carry cross-rail traffic, but the common case is shorter, less congested and more predictable. Rails are deliberately aligned to the NVLink domains inside the nodes, so the two layers reinforce each other rather than fighting.

GPU:NIC ratio. This is how much scale-out bandwidth each GPU can actually claim. Large training clusters commonly run 1:1 — one high-speed NIC per GPU on the east-west fabric, plus separate ports for storage and management. Inference-oriented builds often run leaner ratios, because serving generates far less cross-node traffic than training does. The ratio appears on node spec sheets because it sets a hard ceiling: no topology recovers bandwidth a node cannot physically emit.

Honestly: it is contested, and both answers are defensible.

InfiniBand carried the large majority of AI back-end networks through 2023 and remains the conservative choice for very large, latency-sensitive training clusters — the behavior is well understood and the tooling is mature. Since then, analyst coverage has described Ethernet closing the gap quickly and taking a growing share of new AI back-end deployments, driven by AI-tuned Ethernet silicon, hyperscaler validation of RoCE at large scale, and the Ultra Ethernet 1.0 specification. Anyone telling you one of these is simply obsolete is selling. The gap in capability has narrowed enough that cluster size, workload mix and operator skill now dominate the decision more than raw fabric characteristics do.

A workable decision path:

• A few nodes. The fabric choice barely matters. Standard high-speed Ethernet is usually fine; do not over-engineer.
• Tens of nodes, training-heavy, tight collectives. InfiniBand is the low-risk default, particularly if you want a well-trodden reference design and predictable behavior out of the box.
• Inference-heavy or mixed estates. Ethernet usually wins on operations and on sharing one network technology across back-end, storage and front-end.
• You already run large Ethernet networks well. That skill is worth real money — an Ethernet-expert team will get more out of a well-designed RoCE fabric than out of an unfamiliar InfiniBand one.
• You are buying rack-scale. Much of the traffic that would have crossed the fabric now stays inside the rack’s NVLink domain, which changes the scale-out sizing — see GB300 NVL72 rack vs HGX nodes.

Switches, optics and NICs are also part of the facility load, so the fabric decision feeds the sizing arithmetic — how to size a GPU cluster walks it end to end. To compare NIC configurations by node, browse the GPU catalog.