AI Hardware
The chips, datacenters, and infrastructure powering the AI revolution.
Meta AI Research SuperCluster
Meta's AI Research SuperCluster (RSC), completed in early 2022, was one of the fastest AI supercomputers in the world at the time of completion. It contains 16,000 NVIDIA A100 GPUs, connected via NVIDIA's InfiniBand fabric and a custom storage system capable of 16 TB/s throughput. Meta trained the Llama model family on RSC. The cluster consumed approximately 20MW of power. Meta announced plans to expand to 35,000 H100 GPUs by end of 2023 and has since announced investments in hundreds of thousands of H100s for the Llama 3 and beyond training runs. AICI tracks the RSC as a reference point for the scale of compute required to train frontier open-weight models — and therefore for the concentration of AI capability among the handful of organisations that can build such infrastructure.
Microsoft Azure AI Infrastructure
Microsoft has committed $50 billion in AI infrastructure investment in 2024, with Azure AI infrastructure as the primary beneficiary. Its AI supercomputer — built for OpenAI and for Microsoft's own Copilot services — uses custom-designed clusters of 10,000+ NVIDIA H100 GPUs connected via InfiniBand at 400Gbps. Microsoft also partnered with OpenAI to design custom AI chips (announced 2023) intended to reduce reliance on third-party silicon for inference workloads. Azure's AI infrastructure is the commercial foundation of the Microsoft-OpenAI partnership: OpenAI's models are trained and served on Azure. The scale of this infrastructure investment is the physical correlate of the $13 billion Microsoft has invested in OpenAI.
NVIDIA DGX B200
The DGX B200 is NVIDIA's turnkey AI server, containing 8 B200 GPUs connected via NVLink 5.0 with a total of 1,440GB of HBM3e memory. It is designed as a self-contained unit for organisations that want to operate frontier AI workloads on-premises rather than in cloud infrastructure. A single DGX B200 is priced at approximately $300,000. The DGX line has historically been the entry point for research institutions building on-premises AI infrastructure — universities, government labs, and enterprises that cannot or will not send proprietary data to cloud providers. The B200 generation's memory capacity means a single DGX B200 can serve models with up to ~700 billion parameters.