跳到正文
Larsen Cundric· @larsencc · X·· 9 天前AI 评分61
AI 导读

长程智能体的整体可靠性随步骤增加而急剧下降,99%的单步成功率在100步任务中成功率仅剩37%。虽然单次重试能将概率提升至约99%,但实际重试并非独立事件,还面临超时重复执行和状态改变等复杂问题。这表明长程智能体不仅依赖更强的模型,同样高度依赖周边的系统工程设计。

正文 · 原文

99% reliability sounds great but you should really do the math. If each action succeeds 99% of the time, a 10-step task has a 90% chance of finishing without a single failure. At 100 steps, that drops to 37%. Now give each failed action one retry. Assuming independent failures, that 37% jumps to ~99%. Thats a huge difference. But retries aren’t always that simple. What if an action succeeded but the response timed out? Retrying could execute it twice. What if the failure left the system in a different state? They are probably also not independent. You need to know what actually happened before deciding what to do next. This is why long-horizon agents aren’t just an LLM problem. Better models are definitely needed, but the systems around them matter just as much!

来源:Larsen Cundric · x.com