Agent harness 会消亡吗？从 Harness-Zero 看 harness 蒸馏改变了什么
文章探讨了 Harness-Zero 等研究带来的 harness 蒸馏现象，分析了哪些外壳逻辑能被大模型吸收以及哪些特定功能仍需保留。研究表明，通过在模型力所能及的修正轨迹上进行微调，可以显著提升任务成功率，促使开发者将通用脚手架下沉至模型内部。随着新模型不断吸收原本由外壳承担的通用逻辑，外壳的职责正转向处理工具、数据和环境等个性化要素。
- Explore Skills
View all * Data Engineering * Data Warehouse * SQL * Apache Spark * Microsoft SQL Server * MySQL * Kafka * Data Lake * Streaming & Messaging * NoSQL Databases * Relational Databases
View all * Data Science * Pandas * R * MATLAB * SAS * D3 * Power BI * Tableau * Statistics * Exploratory Data Analysis * Data Visualization
View all * AI & ML * Generative AI * Machine Learning * Artificial Intelligence (AI) * Deep Learning * Reinforcement Learning * Natural Language Processing * TensorFlow * Scikit-Learn * Hyperparameter Tuning * MLOps
View all * Programming Languages * Java * JavaScript * Spring * Python * Go * C# * C++ * C * Swift * Rust * Functional Programming
View all * Software Architecture * Object-Oriented * Distributed Systems * Domain-Driven Design * Architectural Patterns
View all * IT/Ops * Kubernetes * Docker * GitHub * Terraform * Continuous Delivery * Continuous Integration * Database Administration * Computer Networking * Operating Systems * IT Certifications
View all * Security * Network Security * Application Security * Incident Response * Zero Trust Model * Disaster Recovery * Penetration Testing / Ethical Hacking * Governance * Malware * Security Architecture * Security Engineering * Security Certifications
View all * Design * Web Design * Graphic Design * Interaction Design * Film & Video * User Experience (UX) * Design Process * Design Tools
View all * Business * Agile * Project Management * Product Management * Marketing * Human Resources * Finance * Team Management * Business Strategy * Digital Transformation * Organizational Leadership
View all * Soft Skills * Professional Communication * Emotional Intelligence * Presentation Skills * Innovation * Critical Thinking * Public Speaking * Collaboration * Personal Productivity * Confidence / Motivation
Platform
Search for books, courses, events, and more
- Explore Skills
Explore Skills
* Cloud Computing
Cloud Computing
* [Microsoft Azure](https://oreilly.com/search/skills/microsoft-azure/)
* [Amazon Web Services (AWS)](https://oreilly.com/search/skills/amazon-web-services-aws/)
* [Google Cloud](https://oreilly.com/search/skills/google-cloud/)
* [Cloud Migration](https://oreilly.com/search/skills/cloud-migration/)
* [Cloud Deployment](https://oreilly.com/search/skills/cloud-deployment/)
* [Cloud Platforms](https://oreilly.com/search/skills/cloud-platforms/)
Back
* Data Engineering
Data Engineering
* [Data Warehouse](https://oreilly.com/search/skills/data-warehouse/)
* [SQL](https://oreilly.com/search/skills/sql/)
* [Apache Spark](https://oreilly.com/search/skills/apache-spark/)
* [Microsoft SQL Server](https://oreilly.com/search/skills/microsoft-sql-server/)
* [MySQL](https://oreilly.com/search/skills/mysql/)
* [Kafka](https://oreilly.com/search/skills/kafka/)
* [Data Lake](https://oreilly.com/search/skills/data-lake/)
* [Streaming & Messaging](https://oreilly.com/search/skills/streaming-messaging/)
* [NoSQL Databases](https://oreilly.com/search/skills/nosql-databases/)
* [Relational Databases](https://oreilly.com/search/skills/relational-databases/)
Back
* Data Science
Data Science
* [Pandas](https://oreilly.com/search/topics/pandas/)
* [R](https://oreilly.com/search/topics/r/)
* [MATLAB](https://oreilly.com/search/topics/matlab/)
* [SAS](https://oreilly.com/search/topics/sas/)
* [D3](https://oreilly.com/search/topics/d3/)
* [Power BI](https://oreilly.com/search/topics/power-bi/)
* [Tableau](https://oreilly.com/search/topics/tableau/)
* [Statistics](https://oreilly.com/search/topics/statistics/)
* [Exploratory Data Analysis](https://oreilly.com/search/topics/exploratory-data-analysis/)
* [Data Visualization](https://oreilly.com/search/topics/data-visualization/)
Back
* AI & ML
AI & ML
* [Generative AI](https://oreilly.com/search/skills/generative-ai/)
* [Machine Learning](https://oreilly.com/search/skills/machine-learning/)
* [Artificial Intelligence (AI)](https://oreilly.com/search/skills/artificial-intelligence-ai/)
* [Deep Learning](https://oreilly.com/search/skills/deep-learning/)
* [Reinforcement Learning](https://oreilly.com/search/skills/reinforcement-learning/)
* [Natural Language Processing](https://oreilly.com/search/skills/natural-language-processing/)
* [TensorFlow](https://oreilly.com/search/skills/tensorflow/)
* [Scikit-Learn](https://oreilly.com/search/skills/scikit-learn/)
* [Hyperparameter Tuning](https://oreilly.com/search/skills/hyperparameter-tuning/)
* [MLOps](https://oreilly.com/search/skills/mlops/)
Back
* Programming Languages
Programming Languages
* [Java](https://oreilly.com/search/skills/java/)
* [JavaScript](https://oreilly.com/search/skills/javascript/)
* [Spring](https://oreilly.com/search/skills/spring/)
* [Python](https://oreilly.com/search/skills/python/)
* [Go](https://oreilly.com/search/skills/go/)
* [C#](https://oreilly.com/search/skills/c-sharp/)
* [C++](https://oreilly.com/search/skills/c-plus-plus/)
* [C](https://oreilly.com/search/skills/c/)
* [Swift](https://oreilly.com/search/skills/swift/)
* [Rust](https://oreilly.com/search/skills/rust/)
* [Functional Programming](https://oreilly.com/search/skills/functional-programming/)
Back
* Software Architecture
Software Architecture
* [Object-Oriented](https://oreilly.com/search/skills/object-oriented-architecture/)
* [Distributed Systems](https://oreilly.com/search/skills/distributed-systems/)
* [Domain-Driven Design](https://oreilly.com/search/skills/domain-driven-design/)
* [Architectural Patterns](https://oreilly.com/search/skills/architectural-patterns/)
Back
* IT/Ops
IT/Ops
* [Kubernetes](https://oreilly.com/search/topics/kubernetes/)
* [Docker](https://oreilly.com/search/topics/docker/)
* [GitHub](https://oreilly.com/search/topics/github/)
* [Terraform](https://oreilly.com/search/topics/terraform/)
* [Continuous Delivery](https://oreilly.com/search/topics/continuous-delivery/)
* [Continuous Integration](https://oreilly.com/search/topics/continuous-integration/)
* [Database Administration](https://oreilly.com/search/topics/database-administration/)
* [Computer Networking](https://oreilly.com/search/topics/computer-networking/)
* [Operating Systems](https://oreilly.com/search/topics/operating-systems/)
* [IT Certifications](https://oreilly.com/search/topics/it-certifications/)
Back
* Security
Security
* [Network Security](https://oreilly.com/search/skills/network-security/)
* [Application Security](https://oreilly.com/search/skills/application-security/)
* [Incident Response](https://oreilly.com/search/skills/incident-response/)
* [Zero Trust Model](https://oreilly.com/search/skills/zero-trust-model/)
* [Disaster Recovery](https://oreilly.com/search/skills/disaster-recovery/)
* [Penetration Testing / Ethical Hacking](https://oreilly.com/search/skills/penetration-testing-ethical-hacking/)
* [Governance](https://oreilly.com/search/skills/governance/)
* [Malware](https://oreilly.com/search/skills/malware/)
* [Security Architecture](https://oreilly.com/search/skills/security-architecture/)
* [Security Engineering](https://oreilly.com/search/skills/security-engineering/)
* [Security Certifications](https://oreilly.com/search/skills/security-certifications/)
Back
* Design
Design
* [Web Design](https://oreilly.com/search/skills/web-design/)
* [Graphic Design](https://oreilly.com/search/skills/graphic-design/)
* [Interaction Design](https://oreilly.com/search/skills/interaction-design/)
* [Film & Video](https://oreilly.com/search/skills/film-video/)
* [User Experience (UX)](https://oreilly.com/search/skills/user-experience-ux/)
* [Design Process](https://oreilly.com/search/skills/design-process/)
* [Design Tools](https://oreilly.com/search/skills/design-tools/)
Back
* Business
Business
* [Agile](https://oreilly.com/search/skills/agile/)
* [Project Management](https://oreilly.com/search/skills/project-management/)
* [Product Management](https://oreilly.com/search/skills/product-management/)
* [Marketing](https://oreilly.com/search/skills/marketing/)
* [Human Resources](https://oreilly.com/search/skills/human-resources/)
* [Finance](https://oreilly.com/search/skills/finance/)
* [Team Management](https://oreilly.com/search/skills/team-management/)
* [Business Strategy](https://oreilly.com/search/skills/business-strategy/)
* [Digital Transformation](https://oreilly.com/search/skills/digital-transformation/)
* [Organizational Leadership](https://oreilly.com/search/skills/organizational-leadership/)
Back
* Soft Skills
Soft Skills
* [Professional Communication](https://oreilly.com/search/skills/professional-communication/)
* [Emotional Intelligence](https://oreilly.com/search/skills/emotional-intelligence/)
* [Presentation Skills](https://oreilly.com/search/skills/presentation-skills/)
* [Innovation](https://oreilly.com/search/skills/innovation/)
* [Critical Thinking](https://oreilly.com/search/skills/critical-thinking/)
* [Public Speaking](https://oreilly.com/search/skills/public-speaking/)
* [Collaboration](https://oreilly.com/search/skills/collaboration/)
* [Personal Productivity](https://oreilly.com/search/skills/personal-productivity/)
* [Confidence / Motivation](https://oreilly.com/search/skills/confidence-motivation/)
Back
Back
- Platform
Platform
* [All features](https://oreilly.com/online-learning/features.html)
* [Verifiable skills](https://oreilly.com/online-learning/verifiable-skills.html)
* [Academies](https://oreilly.com/online-learning/academies.html)
* [Courses](https://oreilly.com/online-learning/courses.html)
* [Certifications](https://oreilly.com/online-learning/feature-certification.html)
* [Interactive learning](https://oreilly.com/online-learning/intro-interactive-learning.html)
* [Live events](https://oreilly.com/online-learning/live-events.html)
* [Virtual conferences](https://oreilly.com/products/superstreams.html)
* [Answers](https://oreilly.com/online-learning/feature-answers.html)
* [Insights reporting](https://oreilly.com/online-learning/insights-dashboard.html)
Back
Search for books, courses, events, and more
Toggle dark mode
AI & MLBusinessDataInnovationResearchSecurity
Try the O’Reilly learning platform
With the O’Reilly learning platform, you get the resources and guidance to keep your skills sharp and stay ahead. Try it free for up to 14 days.
Try a course for free
Join a live online event on the O’Reilly platform to learn from the experts shaping tech.
Get the Radar Trends newsletter
Your email
Country
Subscribe
Please read our privacy policy.
Thank you for subscribing to the O’Reilly Radar Trends to Watch newsletter.
Are agent harnesses dying? What harness distillation changes.
Harness distillation can train general scaffolding into a model. The harness that remains is the part tied to your tools, data, users, and environment.
By Laurie VossOctober 27, 2026 • 10 minute read
Share
CloseLinkedInXFacebookThreadsBlueskyReddit
The following article originally appeared onThe Arize Blogand is being republished here with the author’s permission.
Every time you think you’ve just got the hang of this AI thing, another thing gets declared “dead.” This week, it’s agent harnesses.
Back in ancient history, aka June, I wrote a piece called “Frameworks are dying, harnesses are winning” and I don’t take that back. But this week a research team took a specialized agent harness, trained its behavior into a model and then took the harness away. The trained model handily beat the original model+harness combination. So are harnesses done now?
Betteridge’s Law applies, so the answer is “no.” But the job of the agent harness is changing. Parts of a harness that are sufficiently generalizable can be trained into the model. That leaves your harness holding the things that make your application unique: your tools, your data, your users, and your environment. You still have to build a harness, but you have to be prepared to regularly throw it away and rebuild it to keep up with evolving models.
This blog is about why that’s true, and how to turn “constantly rebuilding your harness” into an automatic process, not a giant time suck.
How Harness-Zero trained an agent harness into a model
First, the paper that kicked off this train of thought for me: it’s called “Harness-Zero,” from Haoran Ye and colleagues at Peking University. Its subject is harness distillation, i.e., training a model to reproduce behavior that used to be the result of the harness. The methodology is a little involved, so stick with me.
First, the team built a specialized harness for a small model, Qwen3.5-9B. An automated loop, run with Kimi K3, analyzed where the small model kept failing on training tasks and added the usual components to fix those failures: custom tools, middleware rules that block risky actions, skills files and memory. They called this the “evolved harness.”
Then they used that harness to generate training data. They put the small model through training tasks inside a minimal harness while a stronger model, GPT-5.6 Sol, reviewed every step the small model proposed. Sol used a copy of the evolved harness to tell it whether that step was a good idea. Sol passed good ideas from the small model through unchanged, and whenever the small model and the guidance disagreed, Sol made the smallest fix needed before the step ran.
That gave them a bunch of training runs and, crucially, data about what needed to be corrected. Then they created the distilled model: They fine-tuned the small model on the corrected runs. Then they removed the evolved harness, the reviewer, and everything else.
They then tested both the evolved harness and the distilled model head-to-head on spreadsheet editing, multi-app tool use, and chemistry. The distilled model’s average task success rose 21 points versus its untrained predecessor. That’s more than the untrained model gained with the full evolved harness attached. So the distilled model wasn’t just as good as the model+harness, it was noticeably better.
What harness distillation can and cannot move into the model
It’s not that the distilled model ran with no harness whatsoever: It still ran inside a minimal loop with one Bash tool, and the authors say that their method narrows what a harness has to provide rather than eliminating it entirely. The distilled model also didn’t win every competition: In chemistry, where the task is predicting which molecules combine to make a target compound, the distilled model trailed the harnessed version by eight points. The harness there supplied chemistry knowledge, candidate-generation logic, and a molecule validation tool. The authors’ explanation is that step-by-step procedures, like inspecting before editing and verifying before finishing, are much easier to train in than deep domain knowledge. Context management didn’t fully transfer either. So there are still some things harnesses definitely need to do.
Why training on corrected runs made the difference
But the most important finding, in my opinion, is that they showed what kind of training made the difference:
- Fine-tuning on the stronger model’s own successful runs left the small model’s score exactly where it started. Just knowing somebody else is smarter isn’t enough.
- So did fine-tuning on runs the small model completed inside the evolved harness. Again, all that teaches the model is “somebody is better than you at this.”
- Giving the reviewer the correct answers produced nearly perfect training runs and a model that barely improved, this time because it overfit and learned nothing.
- What worked was training on the corrected runs: The model was able to see runs that looked a lot like what it would have done, but with small changes still within its capacity.
Remember that part, because it’s going to be important.
Three training signals left the score flat or overfit: the stronger model’s own successes, runs completed inside the evolved harness, and a reviewer given the right answers. Training on corrected runs, where each fix was still within the small model’s reach, raised task success by 21 points.
Why models are absorbing parts of the agent harness
In Rich Sutton’s 2019 essay The Bitter Lesson, he looked back over 70 years of AI research and found the same pattern in chess, Go, speech recognition, and computer vision: Researchers built their own knowledge of the problem into their systems. That helped in the short term, then plateaued, and eventually lost by a wide margin to general methods, search and learning, that improve as computation gets cheaper.
A harness is that kind of built-in knowledge, and model training is the learning that ends up absorbing it. Sutton’s second point is also relevant: He argued we should build in “only the meta-methods that can find and capture this arbitrary complexity,” rather than the things we’ve already figured out. For harnesses that means: don’t just build it, but capture the process you used to build it, and automate that process.
But that’s a general prediction. More recently people have been saying very specifically that “models are going to eat the harness”:
- In January, Jesus Rodriguez at TheSequence wrote “The Great Absorption,” arguing that hand-coded agent scaffolding keeps losing to capabilities models learn directly.
- In February, Boris Cherny, who created Claude Code, told Y Combinator that scaffolding might buy you 10% to 20% on a task, and then “the gain is wiped out with the next model.”
- In March, Vivek Trivedy at LangChain predicted that planning, self-verification, and long-running coherence would move from the harness into the model.
- And so did lotsofother people, probably more than I’ve been able to find.
Almost everyone on my list builds harnesses for a living; they’re not really arguing that harnesses will go away, just that they have to be able to evolve rapidly.
What still belongs in an agent harness
As noted already, some types of knowledge don’t seem to transfer well from distillation. In ARC Prize’s results for ARC-AGI-3, Gemini 3.8 Flash scores 3.4-times higher with one harness than with another. The better harness keeps the model’s reasoning state between requests and compacts long conversations. That’s context management, one of the things Harness-Zero said they couldn’t fully train in.
Anthropic’s Opus 5.5 prompting guide, released this week, also shows evidence of shrinking harnesses. It suggests deleting “think carefully” instructions because the model now sets its own amount of thinking, and it tells you to retest scaffolding you built for charts and screenshots because the model reads them better without that help now. But at the same time, Anthropic added new harness pieces: a system prompt paragraph that keeps unattended agents from stopping early, a reminder your harness sends when a long run goes quiet, and elapsed-time budgets for multi-agent setups.
So harnesses aren’t going to shrink to nothing, but what’s in them is going to need to evolve with every new model generation, and the parts that survive are the ones tied to your specific tools, data, and environment.
How to evolve an agent harness as models improve
If your harness has to change every generation, the natural move is to automate the changes, but Google’s RRSI paper shows how that can go wrong. Unconstrained automatic harness evolution overfits, meaning the harness gets better at the tasks it’s tuned on without getting better at the general job. In their tests, the unconstrained version scored highest of any variant on the tasks it evolved against, and landed within a point of the original, unevolved harness on benchmarks it hadn’t seen.
RRSI’s fix is mostly bookkeeping: For every candidate change it records which component the change touches, what it was supposed to fix, the diff, how score and cost moved, and whether the change was kept. Components that stop producing gains get removed.
A practical agent harness maintenance loop
You don’t need an automated evolution loop to borrow that discipline. Here’s what it looks like by hand:
When a model ships, tag traces with the harness version and the model version, log what the harness changed, turn off one component at a time, and rerun evals. Keep what still helps and delete the rest, then repeat on the next model.
- **Tag every trace**with the harness version as well as the model version. When either one changes, you want to know which one moved your results, and reading the full traces tells you far more than the scores alone.
- Log each harness intervention along with what the model did before the harness changed it. That’s data you have right now and are probably just throwing away. That’s what produced the record Harness-Zero trained on, and it’s the same shape Perplexity uses to train its Computer agent: learning from good steps in real sessions and correcting failed tool calls, which cut tool-call failures by 21% between its earlier and later trained versions.
- When a new model ships, turn off each harness component one at a time and rerunyour evals. Keep the components that still improve your scores and delete the rest, which is what the Claude Code team does to its own system prompt.
If your traces live in Arize AX, you can run the same agent evals across harness and model versions and compare the results side by side.
So don’t delete your harness, but don’t treat any version of it as finished either. Keep the record of what it corrected, get rid of what the latest model already does without help, and put your effort into the parts that are really specific to your problem, because nobody else has the traces to train those into a model for you.
Post topics: AI & ML
Explore related articles
article AI & ML ## Agent Skills Work but the Research Shows Most Teams Are Building Them Wrong By undefined | undefined
article AI & ML ## What OpenClaw Reveals About the Next Phase of AI Agents By undefined | undefined
Follow us
About O'Reilly
- Teach/Write/Train
- Careers
- O'Reilly News
- Media Coverage
- Community Partners
- Affiliate Program
- Diversity
- Content Sponsorship
Support
International
Download the O'Reilly App
Take O'Reilly with you and learn anywhere, anytime on your phone and tablet.
Do not sell or share my personal information.
© 2026, O'Reilly Media, Inc. All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners.
Close Previous
Next
Privacy Preference Center
When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.
More information
Enable All
Manage Consent Preferences
Strictly Necessary Cookies
Always Active
These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work. These cookies do not store any personally identifiable information.
View Vendor Details
Performance Cookies
Always Active
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.
View Vendor Details
Functional Cookies
Always Active
These cookies enable the website to provide enhanced functionality and personalisation. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.
View Vendor Details
Targeting Cookies
Always Active
These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.
View Vendor Details
Vendors List
Clear
- checkbox label label
Apply Cancel
Consent Leg.Interest
checkbox label label
checkbox label label
checkbox label label
Reject All Confirm
来源：O'Reilly Radar · oreilly.com