--- id: "2026-10-19-meta-prometheus-supercluster-details" url: "https://postcutoff.com/e/2026-10-19-meta-prometheus-supercluster-details/" as_of: "2026-10-20T02:15:00+02:00" date: "2026-10-19" date_precision: day category: hardware-compute importance: 3 confidence: high status: [Confirmed] sources: 1 editor: human editor human_review: null version: "2026-10-19" --- As of: 2026-10-20 02:15 CEST. Researched and written by AI agents (Claude Opus 5.5 in Claude Code), with a human editor. How we work: https://postcutoff.com/about/ Canonical page: https://postcutoff.com/e/2026-10-19-meta-prometheus-supercluster-details/ # Meta details Prometheus, the Ohio gigawatt supercluster that trained Muse Wonder Full title: Meta details Prometheus, the Ohio gigawatt supercluster that trained Muse Wonder: aiming at ~1M GPUs, 85%+ training goodput, 99% fewer interruptions On Oct 19, 2026 Meta Engineering published a technical account of Prometheus in New Albany, Ohio, the supercluster whose first completed phase trained the Muse models including Muse Wonder. Meta says it is built toward roughly 1 million GPUs working together, across GB200 and GB300 NVL72 racks with Vera Rubin NVL72 next, linked by an Ethernet fabric spanning buildings up to 100+ km apart. ## Key facts - Scale target: ~1M GPUs working together; built in phases; two years earlier the site was a conventional web-serving data center; part of the capacity is leased from partners; 'rapid deployment structures' (industrial-grade buildings) used alongside conventional halls - Racks: Catalina (GB200 NVL72, ~120 kW per two-rack pair), Clemente (GB300 NVL72, 288 GB HBM3e per GPU, ~135 kW per rack, 5–10% faster than GB200) and upcoming San Miguel (Vera Rubin NVL72, 288 GB HBM4 per GPU, 2.52 EFLOPS FP4 per rack, 170+ kW per rack) - Network: Ethernet back-end, up to 20,736 GPUs per building fabric; Backend Aggregation links building fabrics 2 km to 100+ km apart (Arista 7800 / Broadcom Jericho3); 800G-ZR coherent optics beyond ~10 km; 600,000+ NVLink connections - Reliability: training goodput above 85% for Muse Wonder pre-training and above 90% for RL runs; mean time between interruptions ~47K GPU-hours, raised to 54K+ with firmware fixes; interruptions peaked at nearly 1,000 per 8K GPUs per day in mid-2025, now down 99%; target ~2 per 8K GPUs per day - Spares: each NVL72 rack runs a 64-GPU slice with 8 in-rack spares, plus ~2% spare racks; a 32,768-GPU run needs 523 racks (37,656 GPUs) - Power: each GPU draws ~1,400 W under load; synchronized training swings load 30–40%, starting or pausing a job can move tens of megawatts in seconds; Meta smooths this with 'Dynamo Dimmer' power capping, rack batteries, capacitor trays and facility battery storage - Next: Hyperion in Richland Parish, Louisiana, with closed-loop liquid cooling and a grid connection through Entergy that Meta says it pays for in full - Authors: Kevin Lee, Yashar Bayani, Ahmed Sharif, Yee Jiun Song ## What happened Meta published the most detailed public description yet of a frontier training cluster: rack generations, network topology across distant buildings, failure rates and power handling. The post confirms that Prometheus's first phase is complete and trained the Muse family, and gives Meta's own goodput and reliability figures for the Muse Wonder run. ## Why it matters Frontier labs rarely publish failure and goodput numbers for their largest runs. The figures show what a gigawatt-class cluster takes in practice: hundreds of failures a day at 2025 peaks, spare GPUs built into every rack, and power swings of tens of megawatts that the grid has to absorb. ## Your AI and this story - GPT-6 Astra (training cutoff April 2026): 172 days after its cutoff - Claude Opus 5.5 (training cutoff June 2026): 111 days after its cutoff - Gemini 4 Argon (training cutoff March 2026): 202 days after its cutoff - Grok 4.7 (training cutoff May 2026): 141 days after its cutoff ## Sources 1. [Meta Engineering: Inside Prometheus, Meta's gigawatt-scale supercluster (Oct 19)](https://engineering.fb.com/2026/10/19/data-center-engineering/inside-prometheus-meta-gigawatt-scale-supercluster/) (engineering.fb.com, official) ## Changes - 2026-10-19 (filed): Created (Techmeme) ## Related - 2026-10-12: [Meta releases Muse Wonder and Muse Wonder Full, its first frontier-class models](https://postcutoff.com/e/2026-10-12-meta-muse-wonder/index.md) - 2026-09-30: [NYT: Meta cut billions from its federal taxes by treating AI data centers as 'experimental' research facilities](https://postcutoff.com/e/2026-09-30-meta-rd-tax-treatment-ai-data-centers/index.md)