跳到正文
Rohan Paul· @rohanpaul_ai · X·· 6 天前AI 评分60
AI 导读

Meta 推出了名为 ProPerMem 的基准测试,用于评估具备长时间记忆的主动式 AI 助手在自主决定何时打断及回答上的表现。研究表明,在包含约 618K token 的一年期个人记忆中,现有基准和前沿模型在自主选择时机时表现较差,准确率最高不超过 15%,而内容与时机兼顾并减少冗余触发可将准确率提升至 42.4%。

正文 · 原文

New Meta paper shows that deciding on its own when to interrupt, not just what to say, is where proactive assistants fail.

Proactive assistants, like AI glasses, get no query. They must decide when to speak and what to say by finding the justifying fact in a year of personal memory, not handed to them.

They built ProPerMem: egocentric video paired with a distractor-rich, year-long memory averaging 618K tokens, with human-verified trigger windows. A clip counts only if the assistant speaks just when due inside the window with grounded content.

Every baseline scored at most 15%, even frontier models with retrieval-augmented memory. With oracle evidence in the window GPT-5.3-Chat reached 96%, but choosing its own timing dropped it to 38%, and using retrieval dropped it to 15%.

For proactive assistants, get both the content and the timing right and fire sparingly, which lifted accuracy from below 10% to 42.4%.

来源:Rohan Paul · x.com