Markus Eisele 实测智能体编码后提出按验证难度与学习价值决定委托多少代码
Markus Eisele 在实测智能体编码构建社区助手 Task Viewer 后提出，应当根据验证难度与学习价值来决定向 AI 委托多少代码编写工作。
- Explore Skills
View all * Data Engineering * Data Warehouse * SQL * Apache Spark * Microsoft SQL Server * MySQL * Kafka * Data Lake * Streaming & Messaging * NoSQL Databases * Relational Databases
View all * Data Science * Pandas * R * MATLAB * SAS * D3 * Power BI * Tableau * Statistics * Exploratory Data Analysis * Data Visualization
View all * AI & ML * Generative AI * Machine Learning * Artificial Intelligence (AI) * Deep Learning * Reinforcement Learning * Natural Language Processing * TensorFlow * Scikit-Learn * Hyperparameter Tuning * MLOps
View all * Programming Languages * Java * JavaScript * Spring * Python * Go * C# * C++ * C * Swift * Rust * Functional Programming
View all * Software Architecture * Object-Oriented * Distributed Systems * Domain-Driven Design * Architectural Patterns
View all * IT/Ops * Kubernetes * Docker * GitHub * Terraform * Continuous Delivery * Continuous Integration * Database Administration * Computer Networking * Operating Systems * IT Certifications
View all * Security * Network Security * Application Security * Incident Response * Zero Trust Model * Disaster Recovery * Penetration Testing / Ethical Hacking * Governance * Malware * Security Architecture * Security Engineering * Security Certifications
View all * Design * Web Design * Graphic Design * Interaction Design * Film & Video * User Experience (UX) * Design Process * Design Tools
View all * Business * Agile * Project Management * Product Management * Marketing * Human Resources * Finance * Team Management * Business Strategy * Digital Transformation * Organizational Leadership
View all * Soft Skills * Professional Communication * Emotional Intelligence * Presentation Skills * Innovation * Critical Thinking * Public Speaking * Collaboration * Personal Productivity * Confidence / Motivation
Platform
Search for books, courses, events, and more
- Explore Skills
Explore Skills
* Cloud Computing
Cloud Computing
* [Microsoft Azure](https://oreilly.com/search/skills/microsoft-azure/)
* [Amazon Web Services (AWS)](https://oreilly.com/search/skills/amazon-web-services-aws/)
* [Google Cloud](https://oreilly.com/search/skills/google-cloud/)
* [Cloud Migration](https://oreilly.com/search/skills/cloud-migration/)
* [Cloud Deployment](https://oreilly.com/search/skills/cloud-deployment/)
* [Cloud Platforms](https://oreilly.com/search/skills/cloud-platforms/)
Back
* Data Engineering
Data Engineering
* [Data Warehouse](https://oreilly.com/search/skills/data-warehouse/)
* [SQL](https://oreilly.com/search/skills/sql/)
* [Apache Spark](https://oreilly.com/search/skills/apache-spark/)
* [Microsoft SQL Server](https://oreilly.com/search/skills/microsoft-sql-server/)
* [MySQL](https://oreilly.com/search/skills/mysql/)
* [Kafka](https://oreilly.com/search/skills/kafka/)
* [Data Lake](https://oreilly.com/search/skills/data-lake/)
* [Streaming & Messaging](https://oreilly.com/search/skills/streaming-messaging/)
* [NoSQL Databases](https://oreilly.com/search/skills/nosql-databases/)
* [Relational Databases](https://oreilly.com/search/skills/relational-databases/)
Back
* Data Science
Data Science
* [Pandas](https://oreilly.com/search/topics/pandas/)
* [R](https://oreilly.com/search/topics/r/)
* [MATLAB](https://oreilly.com/search/topics/matlab/)
* [SAS](https://oreilly.com/search/topics/sas/)
* [D3](https://oreilly.com/search/topics/d3/)
* [Power BI](https://oreilly.com/search/topics/power-bi/)
* [Tableau](https://oreilly.com/search/topics/tableau/)
* [Statistics](https://oreilly.com/search/topics/statistics/)
* [Exploratory Data Analysis](https://oreilly.com/search/topics/exploratory-data-analysis/)
* [Data Visualization](https://oreilly.com/search/topics/data-visualization/)
Back
* AI & ML
AI & ML
* [Generative AI](https://oreilly.com/search/skills/generative-ai/)
* [Machine Learning](https://oreilly.com/search/skills/machine-learning/)
* [Artificial Intelligence (AI)](https://oreilly.com/search/skills/artificial-intelligence-ai/)
* [Deep Learning](https://oreilly.com/search/skills/deep-learning/)
* [Reinforcement Learning](https://oreilly.com/search/skills/reinforcement-learning/)
* [Natural Language Processing](https://oreilly.com/search/skills/natural-language-processing/)
* [TensorFlow](https://oreilly.com/search/skills/tensorflow/)
* [Scikit-Learn](https://oreilly.com/search/skills/scikit-learn/)
* [Hyperparameter Tuning](https://oreilly.com/search/skills/hyperparameter-tuning/)
* [MLOps](https://oreilly.com/search/skills/mlops/)
Back
* Programming Languages
Programming Languages
* [Java](https://oreilly.com/search/skills/java/)
* [JavaScript](https://oreilly.com/search/skills/javascript/)
* [Spring](https://oreilly.com/search/skills/spring/)
* [Python](https://oreilly.com/search/skills/python/)
* [Go](https://oreilly.com/search/skills/go/)
* [C#](https://oreilly.com/search/skills/c-sharp/)
* [C++](https://oreilly.com/search/skills/c-plus-plus/)
* [C](https://oreilly.com/search/skills/c/)
* [Swift](https://oreilly.com/search/skills/swift/)
* [Rust](https://oreilly.com/search/skills/rust/)
* [Functional Programming](https://oreilly.com/search/skills/functional-programming/)
Back
* Software Architecture
Software Architecture
* [Object-Oriented](https://oreilly.com/search/skills/object-oriented-architecture/)
* [Distributed Systems](https://oreilly.com/search/skills/distributed-systems/)
* [Domain-Driven Design](https://oreilly.com/search/skills/domain-driven-design/)
* [Architectural Patterns](https://oreilly.com/search/skills/architectural-patterns/)
Back
* IT/Ops
IT/Ops
* [Kubernetes](https://oreilly.com/search/topics/kubernetes/)
* [Docker](https://oreilly.com/search/topics/docker/)
* [GitHub](https://oreilly.com/search/topics/github/)
* [Terraform](https://oreilly.com/search/topics/terraform/)
* [Continuous Delivery](https://oreilly.com/search/topics/continuous-delivery/)
* [Continuous Integration](https://oreilly.com/search/topics/continuous-integration/)
* [Database Administration](https://oreilly.com/search/topics/database-administration/)
* [Computer Networking](https://oreilly.com/search/topics/computer-networking/)
* [Operating Systems](https://oreilly.com/search/topics/operating-systems/)
* [IT Certifications](https://oreilly.com/search/topics/it-certifications/)
Back
* Security
Security
* [Network Security](https://oreilly.com/search/skills/network-security/)
* [Application Security](https://oreilly.com/search/skills/application-security/)
* [Incident Response](https://oreilly.com/search/skills/incident-response/)
* [Zero Trust Model](https://oreilly.com/search/skills/zero-trust-model/)
* [Disaster Recovery](https://oreilly.com/search/skills/disaster-recovery/)
* [Penetration Testing / Ethical Hacking](https://oreilly.com/search/skills/penetration-testing-ethical-hacking/)
* [Governance](https://oreilly.com/search/skills/governance/)
* [Malware](https://oreilly.com/search/skills/malware/)
* [Security Architecture](https://oreilly.com/search/skills/security-architecture/)
* [Security Engineering](https://oreilly.com/search/skills/security-engineering/)
* [Security Certifications](https://oreilly.com/search/skills/security-certifications/)
Back
* Design
Design
* [Web Design](https://oreilly.com/search/skills/web-design/)
* [Graphic Design](https://oreilly.com/search/skills/graphic-design/)
* [Interaction Design](https://oreilly.com/search/skills/interaction-design/)
* [Film & Video](https://oreilly.com/search/skills/film-video/)
* [User Experience (UX)](https://oreilly.com/search/skills/user-experience-ux/)
* [Design Process](https://oreilly.com/search/skills/design-process/)
* [Design Tools](https://oreilly.com/search/skills/design-tools/)
Back
* Business
Business
* [Agile](https://oreilly.com/search/skills/agile/)
* [Project Management](https://oreilly.com/search/skills/project-management/)
* [Product Management](https://oreilly.com/search/skills/product-management/)
* [Marketing](https://oreilly.com/search/skills/marketing/)
* [Human Resources](https://oreilly.com/search/skills/human-resources/)
* [Finance](https://oreilly.com/search/skills/finance/)
* [Team Management](https://oreilly.com/search/skills/team-management/)
* [Business Strategy](https://oreilly.com/search/skills/business-strategy/)
* [Digital Transformation](https://oreilly.com/search/skills/digital-transformation/)
* [Organizational Leadership](https://oreilly.com/search/skills/organizational-leadership/)
Back
* Soft Skills
Soft Skills
* [Professional Communication](https://oreilly.com/search/skills/professional-communication/)
* [Emotional Intelligence](https://oreilly.com/search/skills/emotional-intelligence/)
* [Presentation Skills](https://oreilly.com/search/skills/presentation-skills/)
* [Innovation](https://oreilly.com/search/skills/innovation/)
* [Critical Thinking](https://oreilly.com/search/skills/critical-thinking/)
* [Public Speaking](https://oreilly.com/search/skills/public-speaking/)
* [Collaboration](https://oreilly.com/search/skills/collaboration/)
* [Personal Productivity](https://oreilly.com/search/skills/personal-productivity/)
* [Confidence / Motivation](https://oreilly.com/search/skills/confidence-motivation/)
Back
Back
- Platform
Platform
* [All features](https://oreilly.com/online-learning/features.html)
* [Verifiable skills](https://oreilly.com/online-learning/verifiable-skills.html)
* [Academies](https://oreilly.com/online-learning/academies.html)
* [Courses](https://oreilly.com/online-learning/courses.html)
* [Certifications](https://oreilly.com/online-learning/feature-certification.html)
* [Interactive learning](https://oreilly.com/online-learning/intro-interactive-learning.html)
* [Live events](https://oreilly.com/online-learning/live-events.html)
* [Virtual conferences](https://oreilly.com/products/superstreams.html)
* [Answers](https://oreilly.com/online-learning/feature-answers.html)
* [Insights reporting](https://oreilly.com/online-learning/insights-dashboard.html)
Back
Search for books, courses, events, and more
Toggle dark mode
AI & MLBusinessDataInnovationResearchSecurity
Try the O’Reilly learning platform
With the O’Reilly learning platform, you get the resources and guidance to keep your skills sharp and stay ahead. Try it free for up to 14 days.
Try a course for free
Join a live online event on the O’Reilly platform to learn from the experts shaping tech.
Get the Radar Trends newsletter
Your email
Country
Subscribe
Please read our privacy policy.
Thank you for subscribing to the O’Reilly Radar Trends to Watch newsletter.
Cheap Code, Expensive Confidence
Choosing what to automate, and developing the expertise to check it
By Markus EiseleOctober 29, 2026 • 11 minute read
Share
CloseLinkedInXFacebookThreadsBlueskyReddit
I recently used a coding agent to build Task Viewer, a small community helper for IBM Bob that we use internally at IBM. It’s a convenience plug-in to Bob, with no critical production role. Despite helping some users internally, for me it’s a real-world experiment on a project to find out how much of the implementation I can delegate. How much to delegate also depends on what the person doing the task still needs to learn. Someone familiar with the application may already know how to check the result, while a developer learning the system may need to work through part of the implementation to develop that understanding.
Even when I hand all the coding to an agent, my basic requirements for myself are to check functionality and the design manually. I want to use the application and see whether it behaves the way I intended. Does the design make sense when I actually have to find something? Is the information presented in a way that I can follow? Those are questions I still need to spend time with, even when the application is small and the implementation was largely delegated.
For Task Viewer, I also created plenty of UI tests to check that the data appeared correctly. That gave me a way to repeat some of the checks as the application changed. There was still a separate question about the calculations, though. Seeing a number in the right place on the screen does not tell me whether it’s the right number. I ran SQL queries against the underlying data to check the math, going back to the records the application was supposed to represent.
There is quite a bit of understanding involved in that seemingly ordinary step. To check a calculation, you need to know what the result is supposed to mean and which records should contribute to it. You need some expectation against which to compare what the application displays. Thankfully, I had help from the team in uncovering relationships in the data that I would not have been able to derive or even explain on my own.
For this particular application, I could inspect both its behavior and the data behind it, and I had people to turn to when my own understanding was incomplete. Its limited role also gave me room to experiment without making the result part of a production system. Building that confidence took time, because I had to check the behavior, establish what the numbers should mean, and resolve the questions I could not answer myself. All of that, for me, belongs in the assessment of how much coding I could delegate.
This is why I find descriptions such as “The agent got me 80% of the way there” increasingly difficult to interpret. I understand the feeling behind them. You have something running, most of the interface exists, and it looks as though the remaining effort should be the smaller part. But a calculation based on the wrong records may look perfectly reasonable until you find a way to check it. Before correcting anything, you have to discover where your understanding and the application’s behavior diverge. The agent may help with that investigation too, although the amount of code it has already produced tells you very little about how long it will take.
DORA’s analysis of 1,110 responses from Google software engineers describes some of the time saved during code generation being used for auditing and verification, alongside productivity benefits. I would expect to check an application I had implemented myself too, so any comparison needs to include the time it takes to reach a result we are prepared to use. Otherwise, we can make agentic development look unusually expensive by counting checks we leave out of the human version, or unusually cheap by stopping the clock when the agent says it has finished.
Some of that effort can make later changes easier to accept. Once a UI test exercises an important behavior, it can keep doing so as the application changes, and a calculation checked against a small set of records can become a regression test. I wrote about this in my earlier article on the right amount of specification. Turning an expectation into an executable check lets us reuse some of the thinking that went into it. We still have to establish that expectation, though. If the application and its tests share the same mistaken interpretation, running the tests more often will keep confirming the same mistake.
For Task Viewer, those expectations depended on relationships in the data that colleagues helped me understand. I knew how to run a query, but I still needed their explanations before I could judge what it should return. Looking only at the time I spent checking would leave out the knowledge they brought to those conversations, just as looking only at the generated code would leave out my checks. The confidence we need to accept a change can depend on several people understanding different parts of the system.
When someone is still learning one of those parts, the implementation itself may be how they begin to understand it. Following a value from the database to the screen gives them reasons to ask why certain records are selected and how the application uses them. If an agent completes that whole path for them, they may reach a working feature without having answered those questions themselves. The same task can be a sensible delegation for an experienced engineer and an important learning opportunity for someone else.
Shen and Tamkin’s study about how AI impacts skill formation reports on a randomized experiment with 52 developers learning an unfamiliar Python library. Participants with AI assistance scored lower on an immediate assessment of their understanding. But the researchers also observed that those participants who stayed actively engaged with the AI had better learning outcomes, although that observation does not establish which way of using AI led to better learning. A short exercise cannot tell us how an engineer develops over several years, but it gives us a reason to look beyond whether the assigned feature was completed when we assess what someone gained from a task.
We can make a decision about autonomy before the agent starts, agreeing on which parts it should carry through on its own and which parts a developer should attempt. Sometimes the reason to limit autonomy is that we do not yet have a convincing way to check the result. Sometimes we can check the result quite easily, but we want the person doing the task to develop a particular skill. In that second case, deliberately keeping part of the implementation with the developer can be the right choice even when the agent could produce acceptable code.
The right choice can probably be made by asking these questions: How much effort will the team need to establish that this result is correct for its intended use? And what does this developer need to learn by doing this part of the task? A feature may be easy for a colleague to verify because they already know the application, while the person implementing it still needs to learn how it works before taking responsibility for future changes. We do not need to turn every routine change into an exercise, but we can choose tasks that give someone a chance to develop understanding they will need again.
Table 1 applies to a particular task, or a part of one, with a particular developer. “Straightforward to verify” means we have a credible way to assess correctness for the intended use. A passing test suite does not settle that question if we are unsure what the tests should assert. When checking is difficult or we do not yet know how to do it, the first step is to make the uncertainty smaller, with help from someone who understands the area. Giving the whole task back to an inexperienced developer would leave that uncertainty unresolved.
Learning value for this developerStraightforward to verifyDifficult or uncertain to verify Little new learning needed Delegate within scopeAgree on acceptance checks and let the agent implement.Establish evidence firstNarrow the task and resolve unknowns. Delegate the parts you can assess. Important learning opportunity Keep the learning stepThe developer attempts the relevant reasoning or implementation before seeing a generated solution.Investigate togetherUse a smaller example and an experienced colleague. Limit the agent to specific questions and supporting tasks.
Table 1. Choosing agent autonomy for a specific task
Imagine a new hire adding a “show only open issues” filter to an issue list in a small Java application. An agent could generate the request parameter, the query condition, and the changes to the screen. For someone who already knows the application, that may be a sensible delegation. Someone learning the system also needs to find out which statuses count as open, where the application selects the records, and how the team expects the change to be tested. Discussing those questions with a colleague is part of learning how to contribute to this application.
For this assignment, the developer could follow the existing request through the application and write the first test before asking the agent to implement the filter. Suppose the list shows twenty issues at a time, and the test data contains twenty closed issues followed by five open ones. Selecting “show only open issues” should display those five. If the application first loads twenty issues and then filters them in the browser, the screen will be empty. Working through that difference with a colleague gives the developer an opportunity to understand where filtering and pagination happen. The agent can help with unfamiliar framework code while the developer investigates why the expected issues are missing.
Once the developer can explain the behavior and has a test that catches the mistake, the agent can handle more of the surrounding implementation. A later request to filter by assignee gives them a chance to use what they learned. Can they work out which issues should appear and whether the existing tests cover the combination of filters? Their explanation during review gives the team a basis for deciding how much of that next change to delegate. The first assignment has also given the new hire experience asking about requirements and discussing a change with another developer.
An experienced engineer can also have good reason to limit the agent’s autonomy in an unfamiliar area. I still needed help with Task Viewer’s data, despite being comfortable writing the queries. When a task takes us into a part of the system we do not yet understand, it makes sense to reconsider how much to hand over. A team can make verification easier by improving its tests and sharing what its members know. It can also choose to preserve a learning opportunity after verification has become easy, because another person is going to need that understanding on future changes.
A developer who has been asked to learn through a task needs time to follow the behavior through the application and someone to discuss it with. If we agree on that purpose and then judge the result only by how quickly the feature appeared, we make it difficult for them to keep the learning step for themselves. We may also leave more of the checking with the same experienced colleagues, while giving others fewer chances to develop the knowledge needed to share it.
Before starting, the developer and the colleague supporting them can agree on what the agent will handle, what the developer will do and why, and what would justify changing that division of labor on a later task. For the status-filter example, this means allowing time for the developer to trace the request and write the first test before the agent implements the filter. As that understanding develops, they can revisit the division on a related change.
With Task Viewer, I could delegate much of the coding because I had ways to check the result and colleagues who could help where my understanding was incomplete. I want us to keep creating those conditions as we ask agents to do more. Better checks can reduce the effort needed to accept a change, while time spent learning can help another developer become able to make that judgment. Sometimes that will mean deliberately leaving the first test and investigation with a person, even though the agent could finish them. When we expect that person to take responsibility for the next change, giving them the opportunity to understand this one belongs in our decision about how much to delegate.
Follow us
About O'Reilly
- Teach/Write/Train
- Careers
- O'Reilly News
- Media Coverage
- Community Partners
- Affiliate Program
- Diversity
- Content Sponsorship
Support
International
Download the O'Reilly App
Take O'Reilly with you and learn anywhere, anytime on your phone and tablet.
Do not sell or share my personal information.
© 2026, O'Reilly Media, Inc. All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners.
Privacy Preference Center
When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.
More information
Enable All
Manage Consent Preferences
Strictly Necessary Cookies
Always Active
These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work. These cookies do not store any personally identifiable information.
View Vendor Details
Performance Cookies
Always Active
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.
View Vendor Details
Functional Cookies
Always Active
These cookies enable the website to provide enhanced functionality and personalisation. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.
View Vendor Details
Targeting Cookies
Always Active
These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.
View Vendor Details
Vendors List
Clear
- checkbox label label
Apply Cancel
Consent Leg.Interest
checkbox label label
checkbox label label
checkbox label label
Reject All Confirm
来源：O'Reilly Radar · oreilly.com