Scaling AI Clusters Through CXL Integration

Meta is currently collaborating with Panmnesia to pioneer a sophisticated artificial intelligence data center design. The primary goal of this initiative is to enable thousands of processors to function within a unified, coherent environment. By leveraging Compute Express Link (CXL) technology, the design aims to facilitate direct communication between CPUs, accelerators, and memory across multiple server racks, effectively bypassing the limitations of traditional network protocols.


Overcoming Bottlenecks in AI Training

As AI workloads continue to expand, the coordination of hundreds or even thousands of accelerators has become a significant technical hurdle. In large-scale training systems, every component must pass through synchronized computational phases. If even one device experiences a delay, the entire cluster must wait, leading to inefficiencies. Conventional networking solutions like Ethernet or InfiniBand, which typically bridge systems, introduce latency fluctuations due to the necessity of packet processing and software-level coordination.

Panmnesia’s approach shifts the paradigm by utilizing a coherent resource environment. By implementing specialized hardware—including high-fan-out switches, link acceleration units, and fabric controllers—the architecture ensures more consistent processing behavior across the infrastructure. These components are organized using structural principles inspired by semiconductor chip design, spanning trays, pods, and the broader fabric.


Significant Gains in Capacity and Speed

The proposed architecture offers a massive leap in scalability compared to existing standards. While current configurations, such as NVIDIA's GB200 NVL72, allow a single CPU to coordinate two accelerators via NVLink-C2C, the Panmnesia design enables one CPU to manage up to 16 accelerators. By grouping approximately 60 of these units, the system could reach a coherence domain of nearly 960 accelerators.

Beyond capacity, the performance improvements are substantial:

  • Reduced Latency: Cross-rack access times are expected to drop from the microsecond range to a few hundred nanoseconds, an improvement of roughly ten times.
  • Modular Maintenance: The architecture allows for the replacement of individual failed hardware without requiring the entire server to be taken offline.

Bridging the Distance Gap

Despite the innovation, physical constraints remain. Electrical CXL signaling is currently limited to approximately seven meters at 128 GT/s when using two retimers. To address this, Panmnesia has proposed utilizing optical CXL links to maintain coherence over greater distances. The company reports that it has already successfully completed a hardware proof-of-concept for this optical approach.

As Myoungsoo Jung, CEO of Panmnesia, stated: «CXL enables the entire datacenter to operate as a single computing system.» With silicon validation for key components already underway, the industry moves closer to realizing this vision of hyperscale, unified AI processing.