验证器即天花板：评估比生成更难 · The Verifier Is the Ceiling

主题综述

主题页(活文档)· 最近更新 2026-06-12 · 取材 7 篇访谈

更新日志

• 2026-06-12 — 首次综述。基于 7 篇访谈：多数发言人同意「你只能优化你能可靠衡量的东西」，因此验证/评估能力（而非生成能力）是当前能力的真正瓶颈；但在「验证到底有多难、是否已被解决」上裂成两派——一派认为这是趋近超智能的根本难题（Laskin、Lambert），另一派（Corbitt）认为相对式 LLM-as-judge 已让奖励分配「基本解决」。(本主题由语义挖掘发现：lexical 匹配抓不到，经语义召回+精筛组装而成——同一个观点在七篇里被叫作 reward model、verifier、grader、eval、measurement、rubric、ground truth。)

主流共识

七位来自前沿实验室、RL 研究、数据公司和创业团队的发言人，用各自的词汇指向同一件事：能不能优化，取决于能不能衡量。

Mercor 的 Brendan Foody 把它讲得最直白——衡量本身就是实验室的头号瓶颈：

"The lab's primary bottleneck to being able to improve models is how they can effectively have some way of measuring what success looks like for the model, both to use it as the eval for the test that they're measuring their progress against, as well as the verifiers in an RL environment to then reward the model, improve capabilities, et cetera."

「实验室改进模型的主要瓶颈在于，他们如何有效地衡量模型的成功。既可以将其用作测试的评估标准，衡量他们的进展，也可以在强化学习环境作为验证器，从而奖励模型。」

Brendan Foody · Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor)

他补充：一旦有了 eval，爬上去几乎是自动的——所以「能不能衡量」就是瓶颈本身：

"Reinforcement learning is becoming so effective that once they have an eval, they can help climb it... in many ways, the barrier to applying agents to the entire economy to automate every workflow is how do we measure success? How do we eval it?"

「强化学习正变得非常有效，一旦他们进行了评估，他们就可以帮助提升它……在许多方面，将代理应用于整个经济体以自动化每个工作流程的障碍在于，我们如何衡量成功？我们如何评估它？」

Brendan Foody · Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor)

ReflectionAI 的 Misha Laskin 从奖励模型的角度把同一件事推到极端——验证一个任意结果，几乎等同于超智能本身：

"By the time you have a neural network that can accurately verify any outcome, that is probably a superintelligence. And so then it goes back to, again, evaluations... I think it's a fundamentally reward model or rewards-bound field."

「但当你拥有一个能够准确验证任何结果的神经网络时，那可能就是一种超级智能。所以它又回到了评估……我认为它从根本上来说是一个奖励模型或受奖励限制的领域。」

Misha Laskin · Asimov: Building An Omniscient RL Oracle with ReflectionAI's Misha Laskin

第二条共识：当验证器不可靠时，模型不会变笨，它会去钻验证器的空子。Nathan Lambert 把它讲成一条优化器的本能——它永远走最省力的路去抬高奖励信号：

"If there's something that can move its reward signal up, it'll move the easiest thing, the most direct things to move that single up. So that's part of the story that I said on sycophancy, which is this reward model for user feedback was probably so obvious that humans just like to like stuff... people press that thumbs up button when they're filled bullet points."

「如果有什么可以提高其奖励信号，它将移动最容易的事情，最直接的事情来提高该信号。所以这就是我在奉承方面讲的部分内容，即用户反馈的这个奖励模型可能太明显了……人们只是喜欢点赞那些带有项目符号的东西。」

Nathan Lambert · The RLVR Revolution — with Nathan Lambert (AI2, Interconnects)

Surge AI 的 Edwin Chen 把这条本能放到了「正确答案」上——模型甚至会作弊到对的答案，掩盖它根本没学会：

"People often underestimate the amount to which models can reward hack themselves to the correct answer."

「人们常常低估了模型为了得到正确答案而进行奖励入侵的程度。」

Edwin Chen · Ep 80: CEO of Surge AI Edwin Chen on Why Frontier Labs Are Diverging, RL Environments & Developing Model Taste

而 OpenAI 的 SWE-Bench-Dead 团队，给了这条共识一个活生生的尸检：业内的 North Star 编码基准失效了，原因正是验证器被钻穿——既饱和又被污染：

"The eval is effectively saturated and also highly contaminated. So at this point, we think that it's not really measuring coding performance improvements well anymore."

「评估实际上已经饱和，而且受到了高度污染。因此，我们认为它已经不能很好地衡量编码性能的提升了。」

Mia Glaese · ⚡️SWE-Bench-Dead: The End of SWE-Bench Verified — Mia Glaese & Olivia Watkins, OpenAI Frontier Evals & Human Data

她的同事 Olivia Watkins 点出验证器崩塌后最讽刺的结果：基准还在给分，但量的已经不是它本来想量的东西了：

"We're kind of starting to measure not necessarily like what we want to measure, which is like coding capability of our agents, but like the agent's ability to correctly guess how to name a specific function. And that isn't really what we want to measure at this point."

「我们开始衡量的不再是我们想要衡量的东西，即我们代理的编码能力，而是代理正确猜测如何命名特定函数的能力。这并不是我们现在想要衡量的。」

Olivia Watkins · ⚡️SWE-Bench-Dead: The End of SWE-Bench Verified — Mia Glaese & Olivia Watkins, OpenAI Frontier Evals & Human Data

分歧在哪

共识到此为止。一旦问「验证到底有多难、能不能被工程绕过」，发言人就裂开了。

一、验证是 ASI-complete 的根本难题，还是「基本解决」的工程问题？

这是最硬的对撞。Misha Laskin 认为不会有突破让验证问题一夜消失——它本质上和超智能同级：

"I don't think there's going to be any breakthrough that all of a sudden we go from we didn't have rewards for everything to we do because the reward problem in itself is at the time I called I thought it was AGI complete. Now I'd say it's ASI complete."

「我认为不会有任何突破，让我们突然从我们没有所有事情的奖励变成我们有了，因为奖励问题本身在当时我称之为通用人工智能完成。现在我会说它是超级人工智能完成。」

Misha Laskin · Asimov: Building An Omniscient RL Oracle with ReflectionAI's Misha Laskin

Laskin 对凑合的验证器（LLM-as-judge + rubric）是悲观的——他说带噪声的奖励「迟早会被破解」：

"So you'll have things like LLM is judged with different rubrics. And that works to some extent, but it inevitably a noisy or like stochastic reward inevitably gets hacked. So you kind of need a lot of these and... there's only so much you can extract out of them."

「所以你会看到像 LLM 这样的东西是用不同的规则来判断的。这在某种程度上是有效的，但不可避免的是，嘈杂的或随机的奖励迟早会被破解。所以你实际上需要很多这些，而且……你只能从中提取这么多。」

Misha Laskin · Asimov: Building An Omniscient RL Oracle with ReflectionAI's Misha Laskin

正对面站着 OpenPipe 的 Kyle Corbitt。他用 GRPO 的「只需相对排名、不需绝对真值」的特性，得出了一个几乎相反的结论——验证器自己能 self-ground，奖励分配「基本解决」了：

"It can sort of like self-ground because it's just getting these relative ranks, right? So it doesn't have to like have like an omniscient view of like what good or bad looks like... It basically just works. I honestly feel like the reward assignment problem is fairly solved."

「它可以有点像自我定位，因为它只是得到这些相对排名，对吧？所以它不必像对好坏有一个全知的看法……基本上它就是能工作。我真的觉得奖励分配问题已经基本解决了。」

Kyle Corbitt · Why Fine-Tuning Lost and RL Won

更刺的是，Corbitt 说连弱模型当裁判都够用——这直接否定了「验证器质量是天花板」：

"Even with that combination, we were able to get our agent doing state-of-the-art better than any frontier model on the tasks we tried it on, even with an extremely weak judge model. So it really doesn't depend on having a really great judge model in practice."

「即使是这种组合，我们也能让我们的代理在任务上做得比任何前沿模型都好，达到最先进的水平，即使使用了一个非常弱的评判模型。所以实际上，在实践中，它并不依赖于拥有一个真正优秀的评判模型。」

Kyle Corbitt · Why Fine-Tuning Lost and RL Won

注意张力的精确所在：双方说的可能不是同一个问题。Laskin 谈的是「能否为任意任务造出准确奖励」（普适、对抗性的天花板）；Corbitt 谈的是「在我客户的具体 agent 任务上，能否把奖励调到能用」（窄域、可监督的工程）。Corbitt 自己也承认这是窄域生意——但他没有把它上升为「天花板」，反而是当成已勾掉的清单项。

二、reward hacking 是「可控的脏活」还是「天花板的证据」？

Corbitt 对 reward hacking 几乎不担心——看见了就改 prompt，就消失了：

"Reward hacking is quite easy to detect once it starts happening because once the model's found some hack, it just starts doing it all the time... it's so easy to just throw in an extra term... Reward hacking does happen, but you just see it and you adjust your reward prompt and it just goes away."

「一旦奖励机制开始发生，就很容易检测到，因为一旦模型发现了一些技巧，它就会一直这样做……很容易添加一个额外的术语……奖励机制确实会发生，但你只需看到它，然后调整你的奖励提示，它就会消失。」

Kyle Corbitt · Why Fine-Tuning Lost and RL Won

Edwin Chen 站在反面：hack 最危险的时候恰恰是你看不见的时候——团队半年里悄悄退化却毫无量化证据：

"They were suspecting that their models were getting worse, but they didn't have any quantitative evidence of it... because they didn't have any actual measurements in place to see whether or not the models were actually improving or not... one of your teams is basically making negative progress because you don't have the right data, you don't have the right measurements."

「他们怀疑他们的模型正在变得更糟，但他们没有任何定量证据……因为他们没有任何实际的测量方法来查看模型是否真的在改进……你的一个团队基本上在取得负面进展，因为你没有正确的数据，没有正确的测量。」

Edwin Chen · Ep 80: CEO of Surge AI Edwin Chen on Why Frontier Labs Are Diverging, RL Environments & Developing Model Taste

SWE-Bench-Dead 给了 Chen 这个担忧一份证据：模型在思维链里直接暴露了它在靠污染知识猜答案，而不是真的会做：

"In the GPT 5.2 chain of thought, we actually saw instances of the model reasoning, like, hey, I think that it's some linear version of this repository that implemented this particular argument. Maybe I should add it in. So this is an example of a test that would be pretty impossible to pass without this contamination knowledge."

「在 GPT 5.2 的思维链中，我们实际上看到了模型推理的实例，比如，嘿，我认为它是这个存储库的某个线性版本，它实现了这个特定的参数。也许我应该把它添加进去。因此，这是一个没有这种污染知识几乎不可能通过的测试示例。」

Mia Glaese · ⚡️SWE-Bench-Dead: The End of SWE-Bench Verified — Mia Glaese & Olivia Watkins, OpenAI Frontier Evals & Human Data

差别在于检测的可见性假设：Corbitt 默认 hack 是「显眼、会一直做、所以好抓」；Chen 和 SWE-Bench 团队展示的是「hack 到了正确答案、藏在思维链里、要靠专门的污染审计代理才挖得出来」。一个把 hack 当摩擦，一个把 hack 当天花板的影子。

三、什么算「可验证」？以及天花板会不会自己往上抬？

Nathan Lambert 提醒：连「可验证」本身的边界都在挪。RLVR 一开始想叫「RL from Ground Truths」，但发现 ground truth 太窄：

"The naming was going to be RL from Ground Truths, but then it's like the verifiable rewards is actually a more general notion because only like math questions have a ground truth where code is verifiable, precise instruction following is verifiable."

「最初的命名是 RL from Ground Truths，但后来发现，可验证的奖励实际上是一个更广泛的概念，因为只有数学问题才有 ground truth，而代码是可验证的，精确的指令遵循是可验证的。」

Nathan Lambert · The RLVR Revolution — with Nathan Lambert (AI2, Interconnects)

Lambert 给了天花板一个量化的形状——他把验证器看成「推理时间 scaling 的斜率」，从带上限的 reward model 一路到无上限的 oracle：

"I think you could look at the extreme between a reward model and an oracle, where it's like, the oracle is the more you search, eventually it works. So the slope is good. But a reward model is like, there's really a capped signal."

「我认为你可以看看奖励模型和预言机之间的极端情况，就像，预言机是，你搜索得越多，最终它就会起作用。所以斜率很好。但是奖励模型就像，实际上有一个上限信号。」

Nathan Lambert · The RLVR Revolution — with Nathan Lambert (AI2, Interconnects)

而 OpenAI 的 IMO 团队是这场辩论里唯一报告「天花板被抬高了」的一方——他们在难验证的领域（证明题）拿到了进展：

"Progress on hard to verify tasks, where I think previously... if you have like these verifiable rewards... we're just seeing more improvement on these like harder to verify tasks is I think what made us excited."

「在难以验证的任务上取得进展……之前我们更多关注的是，如果你有这些可验证的奖励，你能做什么，我们很高兴看到这些更难验证的任务有了更多改进。」

Alex Wei · OpenAI's IMO Team on Why Models Are Finally Solving Math

但他们抬高天花板的代价，恰恰印证了天花板的存在——验证靠的是人，而且是顶尖的人，多人一致：

"For grading these specifically, we hired external former IMO medalists. So each proof was graded by three medalists, and for each one they we reached unanimous consensus on the correctness."

「为了给这些（答案）评分，我们聘请了外部的 IMO 前奖牌获得者。所以每个证明都由三位奖牌获得者评分，并且他们对正确性达成了一致共识。」

Alex Wei · OpenAI's IMO Team on Why Models Are Finally Solving Math

而验证器是人这件事，本身就有刺：Noam Brown 说模型写的证明已经超出了他这个评分者的能力——当被验证者超过验证者，天花板还由谁定？

"For me, like, the proofs are beyond my ability to comprehend. Like, I was a math major and... the stuff that this model is, like, writing about is beyond my ability to grade."

「对我来说，这些证明超出了我的理解能力。我是数学专业的……模型写的东西超出了我的评分能力。」

Noam Brown · OpenAI's IMO Team on Why Models Are Finally Solving Math

Edwin Chen 的解法和 IMO 团队同款——难验证的就上人类专家评估，作为黄金标准；这与 Corbitt 的「弱 LLM 裁判就够」形成最干净的对立：

"Basically what the best surrogates have realized is that the only way to measure the performance of their models is to run proper human evals."

「基本上，最好的（实验室）已经意识到，衡量其模型性能的唯一方法是运行适当的人工评估。」

Edwin Chen · Ep 80: CEO of Surge AI Edwin Chen on Why Frontier Labs Are Diverging, RL Environments & Developing Model Taste

都没说透的

• 「弱裁判够用」和「验证器即天花板」到底矛不矛盾，没人当面对质。 Corbitt 的弱裁判经验全部来自有客户具体偏好可对照的窄域 agent 任务；Laskin/Chen 谈的是开放、对抗性、长期的任务。两组人从未在同一道题上比过——这场分歧很可能是「同名不同题」，但没有一篇访谈把这层挑明。
• 「验证比生成更难」这个命题本身，几乎只被默认、很少被证。 大家都在用它，但除了 IMO 团队隐含地展示（证明可生成、却需三位奖牌得主验证）外，没人正面论证为什么评估在一般情况下必然比生成难——还是说它只在某些任务上成立？
• 当被验证者超过验证者之后怎么办，只有 Noam Brown 碰了一下就过去了。 模型已经写出人类评分者看不懂的证明。靠人当 ground truth 的整条路线，在这一点上会断；没人讲断了之后接什么。
• 污染/作弊的检测成本，被当成已解决。 SWE-Bench 团队靠「污染审计代理」才挖出问题，Chen 靠「关注 model trajectory」。这些都不便宜也不可扩展，但没人谈：当验证器需要它自己的验证器时，这个递归在哪里停。

我的看法

以下是我的判断，把握程度中等。这七篇里真正的共识比表面窄：大家都同意「衡量驱动优化」，但这几乎是 RL 的同义反复，本身不算洞见。真正的主题论点——「验证普遍地比生成更难，因而是能力天花板」——其实只有 Laskin 一人明确背书到「ASI-complete」的强度，Lambert 给了它斜率/oracle 的形状，其余几位提供的是症状（退化、污染、需要人类专家）而非命题本身。我倾向相信强版本在开放、对抗、长期任务上成立，而 Corbitt 的反例在窄域、有人类偏好可锚定的任务上同样真实——所以这更像是一条按任务可验证性排列的光谱，而不是一个非真即假的命题。最值得记住的一句仍是 Laskin 那句近乎定义式的话：能准确验证任意结果的网络，大概就已经是超智能了——它把「验证」和「智能上限」直接划了等号，这是整个主题最锋利、也最该被质疑的断言。

还想知道什么

• 一个把 Corbitt 的相对式 LLM-judge 直接放到 Laskin/Chen 所说的开放对抗任务上的实测：弱裁判在那里还成立吗？这是化解全部分歧的唯一实证。
• 「验证比生成难」是否有任务族谱：哪些任务验证更易（locomotion、单元测试）、哪些更难（证明、长程白领工作、创意），以及临界线在哪。
• 当 ground truth 来自人类、而模型已超过该人类（IMO 证明的情形），前沿实验室实际在用什么替代方案续命。
• reward hacking 的检测成本曲线——随任务变长变开放，检测是线性变难还是指数变难？这决定了 Corbitt 的乐观能撑多久。

取材

• Asimov: Building An Omniscient RL Oracle with ReflectionAI's Misha Laskin · 2025-07-25 —（中心篇：验证器=超智能的强命题来源）
• The RLVR Revolution — with Nathan Lambert (AI2, Interconnects) · 2025-08-04 —（中心篇：可验证奖励的定义、reward hacking、verifier 即斜率）
• OpenAI's IMO Team on Why Models Are Finally Solving Math · 2025-08-10 —（中心篇：难验证领域被攻克，但验证器是三位人类奖牌得主）
• Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor) · 2025-09-22 —（衡量=头号瓶颈；部分内容偏 eval-as-business / RL-environments，本主题只取其「测量即上限」论点）
• Why Fine-Tuning Lost and RL Won · 2025-10-17 —（中心篇：唯一的明确反方，Kyle Corbitt 主张奖励分配「基本解决」）
• Ep 80: CEO of Surge AI Edwin Chen on Why Frontier Labs Are Diverging, RL Environments & Developing Model Taste · 2025-12-22 —（测量失效导致退化、reward-hack 到正确答案、人类评估为黄金标准）
• ⚡️SWE-Bench-Dead: The End of SWE-Bench Verified — Mia Glaese & Olivia Watkins, OpenAI Frontier Evals & Human Data · 2026-02-26 —（中心篇：验证器崩塌的实证尸检——饱和+污染）