AI at Meta· @AIatMeta · X·· 7 天前AI 评分31
AI 导读
Meta 在俄亥俄州新奥尔巴尼建立吉瓦级超级集群 Prometheus，用于训练包括 Muse Wonder 在内的 Muse 基础模型。该集群部署了数十万块 GPU，在预训练中实现了 85% 以上的可用吞吐率，在强化学习中达到 90% 以上。其在 Muse Wonder 训练期间的平均中断间隔时间约为 4.7 万 GPU 小时。
正文 · 原文
We trained our Muse foundational models, including Muse Wonder, at Prometheus, our gigawatt-scale supercluster in New Albany, Ohio.
Across a supercluster with hundreds of thousands of GPUs, even one underperforming host can slow an entire training run. We engineered Prometheus for resilience at scale, achieving 85%+ goodput on pretraining, 90%+ on reinforcement learning and ~47K GPU hours mean time between interruptions during Muse Wonder training.
Take a look at the tech powering Prometheus: https://go.meta.me/4yXXApZ
来源：AI at Meta · x.com