来自 @microagi 关于传输 PB 级机器人数据文章中的一个有趣统计数据: 他们对一个 239 GB 的数据集进行了微小修改后重新上传到 Hugging Face。只有 2.6 MB 的数据通过网络传输,大约一分钟就完成了(!) 而且这在规模化应用中不断带来回报,因为机器人录像高度重复(相同的场景、相同的设备)。然后 @Cloudflare workers 将其分发到任何云端,超过 1 PB 的数据只需约 500 美元 该团队写了一篇很棒的文章:https://www.microagi.ai/articles/delivery-platform
fun stat from @microagi’s write-up on shipping petabytes of robot data:
they re-uploaded a 239 GB dataset with small changes to Hugging Face. only 2.6 MB went over the network, done in about a minute (!)
and it keeps paying off at scale, because robot recordings are super repetitive (same scenes, same rigs). then @Cloudflare workers fan it all out to any cloud, >1 PB for ~$500
great write-up by the team: https://www.microagi.ai/articles/delivery-platform
来源:Thomas Wolf · x.com