非营利组织 AItonomy Foundation 推出 Science's Last Exam (SLE) 科研智能体基准,旨在通过 9 个学科的 113 项真实研究任务评估 AI 的独立科研能力。该基准通过对比模型训练截止日期后发表的研究来测试 AI 假设,其中 DeepSeek V4.1 Flash 表现领先,有 21.28% 的假设与后续研究相符。
Lean gave AI math a grader nobody can argue with, and empirical science is now getting its own version of that idea.
Science’s Last Exam (SLE), a benchmark, grades research agents on artifacts that can be opened and re-run, with checks that domain experts seal when each task is built.
Led by the US nonprofit AItonomy Foundation, the benchmark spans 113 tasks in 9 disciplines and grades submissions with checks hidden from agents.
Most AI science tests ask questions with known answers; SLE hands an AI agent a research goal, data and lab software, then checks its finished work with tests the agent never sees.
It scores AI hypotheses against studies published after the model’s training cutoff, so memory cannot help. DeepSeek V4.1 Flash led, with 21.28% of its hypotheses anticipating later research.
来源:Rohan Paul · x.com