AI Factory Networking in 2026: Scale-Across, RoCEv2 and CPO
AI Factory networking now spans three dimensions. How scale-across fabrics, RoCEv2 Ethernet and co-packaged optics decide multi-site GPU cluster economics.
AI Factory networking has become the binding constraint on frontier-scale compute — not GPU supply, and not model architecture. Three shifts landed within a year of each other: scale-across fabrics that stitch multiple buildings into one training domain, the maturation of RoCEv2 as the default lossless Ethernet, and co-packaged optics moving from research curiosity to procurement checklist.
Individually, each is an engineering optimisation. Together they redefine what an AI Factory actually is — and what it takes to plan capacity for the 2026-2028 window.
Scale-across networking: the third dimension of AI supercomputing
Single buildings have run out of room
AI clusters have outgrown the physical and power constraints of any single facility. Power density, cooling capacity and available floor space now impose hard ceilings — not soft ones you can engineer around with a better rack layout.
The only viable path forward is to treat several geographically distributed data centres as one logical supercomputer. That is the working definition of an AI Factory in 2026: a training domain that spans campuses, and sometimes cities.
Why ordinary long-haul Ethernet fails here
Deep-buffer long-haul switching works perfectly well for enterprise traffic and bulk transfers. It fails catastrophically for AI training.
Training is synchronous. Every GPU must exchange gradients at the same moment. Deep buffers absorb bursts by adding latency, and that added latency is unpredictable — so even microsecond-scale jitter forces the entire cluster into idle wait states. Utilisation collapses and time-to-train inflates.
The failure mode is worth stating plainly: in a synchronous collective, the slowest link sets the pace for every GPU in the job.
What scale-across fabrics do differently
Scale-across fabrics reuse the same high-performance switch silicon and SuperNIC hardware already proven for intra-facility scale-out. What is new is the control logic.
- Distance-aware congestion control — the algorithm adapts based on whether the two endpoints sit in the same hall or tens of kilometres apart.
- Telemetry-driven rate control at the NIC — injection rates are throttled before congestion materialises, rather than absorbed in deep buffers afterwards.
- Adaptive routing tuned for collective communication patterns rather than for flow-level fairness.
The measured result is up to 1.9x higher NCCL all-reduce collective bandwidth across 10 km compared with commodity Ethernet. Leading GPU cloud operators are already deploying these fabrics to stitch separate facilities into unified training domains.
The three dimensions, side by side
| Dimension | Scope | Typical distance | Technology class | Primary failure mode |
|---|---|---|---|---|
| Scale-Up | GPU-to-GPU inside the rack | Centimetres to metres | NVLink-class interconnect | Bandwidth ceiling per node |
| Scale-Out | Cluster-wide fabric in one facility | Metres to hundreds of metres | Lossless Ethernet or InfiniBand | Congestion and packet loss |
| Scale-Across | Facility-to-facility | Kilometres, metro to regional | Distance-aware Ethernet fabric | Latency jitter stalling collectives |
Why this matters
Any AI Factory roadmap that plans only for scale-up and scale-out has a hard scalability ceiling designed into it — one that arrives the day your first campus fills its power envelope. The third dimension has to be present from the start, because it changes site selection, fibre procurement and network topology — none of which can be retrofitted cheaply.
RoCEv2: Ethernet that behaves like InfiniBand
The old rule no longer holds
For years the industry operated on a simple split: InfiniBand for large-scale training, Ethernet for inference and general-purpose workloads. InfiniBand’s native lossless credit-based flow control and 1-2 microsecond latency made it the default for synchronous GPU collectives.
Standard Ethernet, by contrast, drops packets under congestion and relies on TCP retransmissions measured in tens or hundreds of milliseconds. That is untenable at scale — a 1% drop rate in a 1,024-GPU cluster can waste hundreds of thousands of dollars in idle GPU time every week.
How lossless Ethernet actually works
RDMA over Converged Ethernet version 2 (RoCEv2) encapsulates InfiniBand transport semantics inside ordinary UDP/IP packets. Operators keep the open, multi-vendor Ethernet ecosystem while achieving near-InfiniBand behaviour.
Two mechanisms make the fabric lossless, and they operate on different timescales:
- Priority Flow Control (PFC) — a hard pause signal that prevents buffer overflow. Blunt, effective, and dangerous if it propagates upstream.
- Explicit Congestion Notification (ECN) — an early, soft signal that tells endpoints to reduce injection rates before buffers fill.
DCQCN is the algorithm that orchestrates the two, using ECN as the primary control loop and holding PFC in reserve. Tuned properly, the resulting 2-5 microsecond latency is only marginally higher than native InfiniBand, and at the scale of today’s largest runs the difference in job completion time is often negligible. Major developers already train frontier models on RoCEv2 fabrics measured in tens of thousands of GPUs.
| Attribute | InfiniBand | RoCEv2 |
|---|---|---|
| Latency | 1-2 microseconds | 2-5 microseconds |
| Losslessness | Native credit-based flow control | PFC and ECN, orchestrated by DCQCN |
| Ecosystem | Narrow, largely single-vendor | Broad, multi-vendor |
| Operational familiarity | Specialist skill set | Standard Ethernet operations |
| Tuning burden | Low | High — PFC thresholds, ECN marking, buffers |
Two competing silicon strategies
The debate has moved on from which protocol to which flavour of lossless Ethernet.
- Enhanced proprietary stacks layer adaptive per-packet routing and faster congestion-control reactions on top of RoCEv2, delivering measured gains on the order of 1.6x better job completion versus baseline commodity Ethernet for large collective jobs.
- Open high-radix switches prioritise raw bandwidth — 100+ Tbps per chip — with pure standards-based DCQCN, preserving the freedom to mix NICs and switches from multiple vendors without lock-in.
The protocol war is effectively over. RoCEv2 won on cost, operational familiarity and ecosystem breadth. What remains is a genuine architectural trade between measured performance today and vendor freedom tomorrow.
For AI Factory operators
Treat high-performance RoCEv2 as the default scale-out fabric unless there is a specific, quantified reason to remain on InfiniBand. But budget for the tuning. PFC thresholds, ECN marking points, buffer allocation and traffic-class isolation must be tuned end to end — an untuned RoCEv2 fabric will underperform the InfiniBand it replaced, and the failure is silent until job completion times drift.
Co-packaged optics: power and density at the fabric layer
The pluggable optics tax
As AI factories approach millions of GPUs, the network stops being a passive pipe and becomes a first-order claim on the power budget itself.
Conventional pluggable optical transceivers carry three compounding costs:
- Roughly 30 W per interface.
- Up to 22 dB of electrical loss between the switch ASIC and the optic.
- Millions of discrete failure points across a large deployment — every pluggable is a connector, a seating tolerance and a field-replaceable unit.
What moving the optics inside the package buys
Co-packaged optics (CPO) places the optical engine inside the switch package, millimetres from the ASIC. Electrons travel only a short distance before light takes over. The gains are not incremental:
- Power per interface drops by a factor of roughly 3.5x, to around 9 W.
- Electrical loss improves by more than an order of magnitude.
- Component count and failure rates fall sharply.
- Switch density reaches 512 ports of 800 Gb/s in a single chassis.
Why this is a compute decision, not a networking one
At gigawatt scale, the power saved by eliminating high-power pluggables is redirected straight to GPUs. That is the entire argument: higher tokens per joule from the same grid connection.
Both InfiniBand and Ethernet variants of these photonic switches are entering production. The network is no longer a passive pipe but an integral, power-efficient extension of the AI computer — which is why CPO is moving from optional innovation to near-mandatory for facilities that must maximise compute density inside a fixed energy envelope.
Agentic AI: freight trains versus a swarm of drones
The traffic profile has changed
Training frontier models generates traffic that is enormous but highly predictable — scheduled, structured, engineerable. A freight train.
Autonomous agents change the equation entirely. A fleet of thousands of agents continuously spawns sub-agents, invokes external tools, reads and writes memory stores, and converses with specialist models. The result is an always-on storm of micro-transactions: east-west, low-latency, high-concurrency and highly unpredictable. A swarm of delivery drones.
A network engineered for the freight train will struggle badly when thousands of drone-style interactions arrive at once.
What agentic fabrics have to provide
- Microsecond-level latency for tool calls — an agent’s reasoning loop is gated by its slowest round trip, and the loop runs many times per task.
- Persistent high-bandwidth paths for state — memory and context retrieval are continuous rather than bursty.
- Burst absorption without collapse — concurrency spikes are the normal operating condition, not an anomaly to be smoothed away.
Copper interconnects reach their limits quickly under this profile. High-bandwidth optical fabrics become essential, and when agent fleets span regions, data-centre interconnect (DCI) optical networks rise to equal importance with intra-rack switching.
This is not theoretical for us. Running Yehia, our Arabic-first LLM, and Bayan, our retrieval platform, in agentic configurations produces exactly this signature — many small, latency-sensitive calls rather than a handful of large ones.
What this means for sovereign AI Factories and NeoCloud operators
These developments converge on one architectural reality: the modern AI Factory is no longer a monolithic building. It is a distributed system whose performance is gated by the predictability of its interconnects at every distance scale.
For anyone planning sovereign capacity in the Kingdom, four consequences follow.
1. Assume multi-facility operation from day one
Site selection and power planning must assume multi-facility operation from the outset. A 15-50 MW AI Factory may deliberately span two or three campuses linked by scale-across networking, rather than forcing all capacity into one constrained site.
In a market where grid connections, land and cooling capacity are the real bottlenecks, that flexibility is a strategic advantage rather than a compromise. It is the core planning assumption behind our data centre solutions practice.
2. Treat network design as a peer of compute and cooling
Telemetry-driven congestion control, precise latency budgeting and distance-aware routing are now mandatory engineering disciplines — not post-procurement tuning exercises handed to the network team after the racks land.
3. Avoid lock-in at the fabric layer
Favour open or semi-open Ethernet fabrics that still deliver InfiniBand-class collective performance. GPU and NIC roadmaps move faster than fabric refresh cycles, and a single-vendor fabric quietly constrains every future compute decision.
4. Invest in operational maturity before scale
Fabric observability, automated PFC and ECN policy management, and continuous NCCL validation under realistic multi-site traffic patterns. Validate against the traffic you will actually run, not synthetic benchmarks that never generate your real collective patterns.
The sovereignty angle
Sovereign AI is usually discussed in terms of data residency and model ownership. The harder constraint is physical. If a training domain cannot span multiple domestic sites with predictable interconnect performance, national-scale capacity is capped by whichever single site has the most available power — a much lower ceiling than most roadmaps assume.
The bottom line
The scarce resource in AI infrastructure is shifting. For three years it was GPUs. It is becoming predictable interconnect at every distance scale — inside the rack, across the hall, and between campuses.
Operators who internalise this now will design campuses that can grow. Those who do not will meet a wall the first time a single site fills its power envelope — and retrofitting a scale-across fabric costs far more than designing for one.
Navid AI tracks these primitives closely as we architect sovereign AI Factories and NeoCloud platforms optimised for Arabic and regional workloads. Scale-across capability, lossless Ethernet maturity, photonic switch roadmaps and agent-aware network design belong on the 2026 strategic priority list, not the 2028 one. If you are planning capacity for that window, talk to our infrastructure team.
