跳到正文
O'Reilly Radar·· 3 小时前AI 评分55

Tim O'Reilly 撰文探讨 AI 经济中的新知识产物:代码之外的价值转移

AI 导读

出版人 Tim O'Reilly 撰文探讨人工智能时代软件开发价值的转移,指出随着代码生成变得廉价,源代码正被提示词、上下文包、智能体技能和评估基准等新型知识产物所取代。文章通过回顾开源和网络革命的历史,分析了未来开发者需要分享和保留的新对象,以及建立信任与协作机制的新需求。

正文 · 原文

Skip to main content

View all * Data Engineering * Data Warehouse * SQL * Apache Spark * Microsoft SQL Server * MySQL * Kafka * Data Lake * Streaming & Messaging * NoSQL Databases * Relational Databases

View all * Data Science * Pandas * R * MATLAB * SAS * D3 * Power BI * Tableau * Statistics * Exploratory Data Analysis * Data Visualization

View all * AI & ML * Generative AI * Machine Learning * Artificial Intelligence (AI) * Deep Learning * Reinforcement Learning * Natural Language Processing * TensorFlow * Scikit-Learn * Hyperparameter Tuning * MLOps

View all * Programming Languages * Java * JavaScript * Spring * Python * Go * C# * C++ * C * Swift * Rust * Functional Programming

View all * Software Architecture * Object-Oriented * Distributed Systems * Domain-Driven Design * Architectural Patterns

View all * IT/Ops * Kubernetes * Docker * GitHub * Terraform * Continuous Delivery * Continuous Integration * Database Administration * Computer Networking * Operating Systems * IT Certifications

View all * Security * Network Security * Application Security * Incident Response * Zero Trust Model * Disaster Recovery * Penetration Testing / Ethical Hacking * Governance * Malware * Security Architecture * Security Engineering * Security Certifications

View all * Design * Web Design * Graphic Design * Interaction Design * Film & Video * User Experience (UX) * Design Process * Design Tools

View all * Business * Agile * Project Management * Product Management * Marketing * Human Resources * Finance * Team Management * Business Strategy * Digital Transformation * Organizational Leadership

View all * Soft Skills * Professional Communication * Emotional Intelligence * Presentation Skills * Innovation * Critical Thinking * Public Speaking * Collaboration * Personal Productivity * Confidence / Motivation

View all

View all

Search for books, courses, events, and more


  • Explore Skills

Explore Skills

*   Cloud Computing

Cloud Computing

    *   [Microsoft Azure](https://www.oreilly.com/search/skills/microsoft-azure/)
    *   [Amazon Web Services (AWS)](https://www.oreilly.com/search/skills/amazon-web-services-aws/)
    *   [Google Cloud](https://www.oreilly.com/search/skills/google-cloud/)
    *   [Cloud Migration](https://www.oreilly.com/search/skills/cloud-migration/)
    *   [Cloud Deployment](https://www.oreilly.com/search/skills/cloud-deployment/)
    *   [Cloud Platforms](https://www.oreilly.com/search/skills/cloud-platforms/)

Back

*   Data Engineering

Data Engineering

    *   [Data Warehouse](https://www.oreilly.com/search/skills/data-warehouse/)
    *   [SQL](https://www.oreilly.com/search/skills/sql/)
    *   [Apache Spark](https://www.oreilly.com/search/skills/apache-spark/)
    *   [Microsoft SQL Server](https://www.oreilly.com/search/skills/microsoft-sql-server/)
    *   [MySQL](https://www.oreilly.com/search/skills/mysql/)
    *   [Kafka](https://www.oreilly.com/search/skills/kafka/)
    *   [Data Lake](https://www.oreilly.com/search/skills/data-lake/)
    *   [Streaming & Messaging](https://www.oreilly.com/search/skills/streaming-messaging/)
    *   [NoSQL Databases](https://www.oreilly.com/search/skills/nosql-databases/)
    *   [Relational Databases](https://www.oreilly.com/search/skills/relational-databases/)

Back

*   Data Science

Data Science

    *   [Pandas](https://www.oreilly.com/search/topics/pandas/)
    *   [R](https://www.oreilly.com/search/topics/r/)
    *   [MATLAB](https://www.oreilly.com/search/topics/matlab/)
    *   [SAS](https://www.oreilly.com/search/topics/sas/)
    *   [D3](https://www.oreilly.com/search/topics/d3/)
    *   [Power BI](https://www.oreilly.com/search/topics/power-bi/)
    *   [Tableau](https://www.oreilly.com/search/topics/tableau/)
    *   [Statistics](https://www.oreilly.com/search/topics/statistics/)
    *   [Exploratory Data Analysis](https://www.oreilly.com/search/topics/exploratory-data-analysis/)
    *   [Data Visualization](https://www.oreilly.com/search/topics/data-visualization/)

Back

*   AI & ML

AI & ML

    *   [Generative AI](https://www.oreilly.com/search/skills/generative-ai/)
    *   [Machine Learning](https://www.oreilly.com/search/skills/machine-learning/)
    *   [Artificial Intelligence (AI)](https://www.oreilly.com/search/skills/artificial-intelligence-ai/)
    *   [Deep Learning](https://www.oreilly.com/search/skills/deep-learning/)
    *   [Reinforcement Learning](https://www.oreilly.com/search/skills/reinforcement-learning/)
    *   [Natural Language Processing](https://www.oreilly.com/search/skills/natural-language-processing/)
    *   [TensorFlow](https://www.oreilly.com/search/skills/tensorflow/)
    *   [Scikit-Learn](https://www.oreilly.com/search/skills/scikit-learn/)
    *   [Hyperparameter Tuning](https://www.oreilly.com/search/skills/hyperparameter-tuning/)
    *   [MLOps](https://www.oreilly.com/search/skills/mlops/)

Back

*   Programming Languages

Programming Languages

    *   [Java](https://www.oreilly.com/search/skills/java/)
    *   [JavaScript](https://www.oreilly.com/search/skills/javascript/)
    *   [Spring](https://www.oreilly.com/search/skills/spring/)
    *   [Python](https://www.oreilly.com/search/skills/python/)
    *   [Go](https://www.oreilly.com/search/skills/go/)
    *   [C#](https://www.oreilly.com/search/skills/c-sharp/)
    *   [C++](https://www.oreilly.com/search/skills/c-plus-plus/)
    *   [C](https://www.oreilly.com/search/skills/c/)
    *   [Swift](https://www.oreilly.com/search/skills/swift/)
    *   [Rust](https://www.oreilly.com/search/skills/rust/)
    *   [Functional Programming](https://www.oreilly.com/search/skills/functional-programming/)

Back

*   Software Architecture

Software Architecture

    *   [Object-Oriented](https://www.oreilly.com/search/skills/object-oriented-architecture/)
    *   [Distributed Systems](https://www.oreilly.com/search/skills/distributed-systems/)
    *   [Domain-Driven Design](https://www.oreilly.com/search/skills/domain-driven-design/)
    *   [Architectural Patterns](https://www.oreilly.com/search/skills/architectural-patterns/)

Back

*   IT/Ops

IT/Ops

    *   [Kubernetes](https://www.oreilly.com/search/topics/kubernetes/)
    *   [Docker](https://www.oreilly.com/search/topics/docker/)
    *   [GitHub](https://www.oreilly.com/search/topics/github/)
    *   [Terraform](https://www.oreilly.com/search/topics/terraform/)
    *   [Continuous Delivery](https://www.oreilly.com/search/topics/continuous-delivery/)
    *   [Continuous Integration](https://www.oreilly.com/search/topics/continuous-integration/)
    *   [Database Administration](https://www.oreilly.com/search/topics/database-administration/)
    *   [Computer Networking](https://www.oreilly.com/search/topics/computer-networking/)
    *   [Operating Systems](https://www.oreilly.com/search/topics/operating-systems/)
    *   [IT Certifications](https://www.oreilly.com/search/topics/it-certifications/)

Back

*   Security

Security

    *   [Network Security](https://www.oreilly.com/search/skills/network-security/)
    *   [Application Security](https://www.oreilly.com/search/skills/application-security/)
    *   [Incident Response](https://www.oreilly.com/search/skills/incident-response/)
    *   [Zero Trust Model](https://www.oreilly.com/search/skills/zero-trust-model/)
    *   [Disaster Recovery](https://www.oreilly.com/search/skills/disaster-recovery/)
    *   [Penetration Testing / Ethical Hacking](https://www.oreilly.com/search/skills/penetration-testing-ethical-hacking/)
    *   [Governance](https://www.oreilly.com/search/skills/governance/)
    *   [Malware](https://www.oreilly.com/search/skills/malware/)
    *   [Security Architecture](https://www.oreilly.com/search/skills/security-architecture/)
    *   [Security Engineering](https://www.oreilly.com/search/skills/security-engineering/)
    *   [Security Certifications](https://www.oreilly.com/search/skills/security-certifications/)

Back

*   Design

Design

    *   [Web Design](https://www.oreilly.com/search/skills/web-design/)
    *   [Graphic Design](https://www.oreilly.com/search/skills/graphic-design/)
    *   [Interaction Design](https://www.oreilly.com/search/skills/interaction-design/)
    *   [Film & Video](https://www.oreilly.com/search/skills/film-video/)
    *   [User Experience (UX)](https://www.oreilly.com/search/skills/user-experience-ux/)
    *   [Design Process](https://www.oreilly.com/search/skills/design-process/)
    *   [Design Tools](https://www.oreilly.com/search/skills/design-tools/)

Back

*   Business

Business

    *   [Agile](https://www.oreilly.com/search/skills/agile/)
    *   [Project Management](https://www.oreilly.com/search/skills/project-management/)
    *   [Product Management](https://www.oreilly.com/search/skills/product-management/)
    *   [Marketing](https://www.oreilly.com/search/skills/marketing/)
    *   [Human Resources](https://www.oreilly.com/search/skills/human-resources/)
    *   [Finance](https://www.oreilly.com/search/skills/finance/)
    *   [Team Management](https://www.oreilly.com/search/skills/team-management/)
    *   [Business Strategy](https://www.oreilly.com/search/skills/business-strategy/)
    *   [Digital Transformation](https://www.oreilly.com/search/skills/digital-transformation/)
    *   [Organizational Leadership](https://www.oreilly.com/search/skills/organizational-leadership/)

Back

*   Soft Skills

Soft Skills

    *   [Professional Communication](https://www.oreilly.com/search/skills/professional-communication/)
    *   [Emotional Intelligence](https://www.oreilly.com/search/skills/emotional-intelligence/)
    *   [Presentation Skills](https://www.oreilly.com/search/skills/presentation-skills/)
    *   [Innovation](https://www.oreilly.com/search/skills/innovation/)
    *   [Critical Thinking](https://www.oreilly.com/search/skills/critical-thinking/)
    *   [Public Speaking](https://www.oreilly.com/search/skills/public-speaking/)
    *   [Collaboration](https://www.oreilly.com/search/skills/collaboration/)
    *   [Personal Productivity](https://www.oreilly.com/search/skills/personal-productivity/)
    *   [Confidence / Motivation](https://www.oreilly.com/search/skills/confidence-motivation/)

Back

Back

  • Platform

Platform

*   [All features](https://www.oreilly.com/online-learning/features.html)
*   [Verifiable skills](https://www.oreilly.com/online-learning/verifiable-skills.html)
*   [Academies](https://www.oreilly.com/online-learning/academies.html)
*   [Courses](https://www.oreilly.com/online-learning/courses.html)
*   [Certifications](https://www.oreilly.com/online-learning/feature-certification.html)
*   [Interactive learning](https://www.oreilly.com/online-learning/intro-interactive-learning.html)
*   [Live events](https://www.oreilly.com/online-learning/live-events.html)
*   [Virtual conferences](https://www.oreilly.com/products/superstreams.html)
*   [Answers](https://www.oreilly.com/online-learning/feature-answers.html)
*   [Insights reporting](https://www.oreilly.com/online-learning/insights-dashboard.html)

Back


Search for books, courses, events, and more

Toggle dark mode

AI & MLBusinessDataInnovationResearchSecurity

Try the O’Reilly learning platform

With the O’Reilly learning platform, you get the resources and guidance to keep your skills sharp and stay ahead. Try it free for up to 14 days.

Start trial

Try a course for free

Join a live online event on the O’Reilly platform to learn from the experts shaping tech.

See what’s coming soon

Get the Radar Trends newsletter

Your email

Country

Subscribe

Please read our privacy policy.

Thank you for subscribing to the O’Reilly Radar Trends to Watch newsletter.

Radar>Topics>AI & ML

The New Knowledge Artifacts of the AI Economy

Every technology revolution creates new valuable knowledge artifacts that are worth sharing.

Image 1: Tim O’Reilly By Tim O’ReillyNovember 5, 2026 • 21 minute read

Share

CloseLinkedInXFacebookThreadsBlueskyReddit

At our recent Foo Camp, a number of people said something like, “I don’t want someone else’s code any more. I may have my agent take a look at it, but I want to extract just the parts that I want, and may rewrite even those in Rust. It’s just too easy to create just the code that I want.”

This topic was hot enough that Jesse Vincent held an entire session about it. I did not attend, but Jesse gave me a report from his Plaude AI notetaker. Here’s a relevant section of the summary:

Several participants argue that software development’s primary artifact may be shifting from executable code toward natural-language prompts, generation processes, and developer skills. One speaker compares Python to assembly language, suggesting that writing conventional code may become optional as natural-language generation takes precedence. Another argues that open source can survive such a transition because its core values—peer production, capacity building, and shared specifications—do not depend on who or what writes the code.

At the same time, Jesse warns that maintainers are overwhelmed by floods of uninvited, automated, and often low-quality contributions. Participants say that traditional milestones such as pull requests, repositories, and conferences have helped contributors connect and gain recognition; as AI makes code replication easier, communities may need new ways to acknowledge participation and create paths for future developers. Some community interaction has moved to spaces such as Discord and direct messages, but structured opportunities are still needed.

Scientific computing raises particular concerns. One panelist warns that AI-generated forks and agent attacks could threaten carefully verified foundations such as NumPy and SciPy. Scientific systems depend on battle-tested libraries and reproducibility, so thousands of unreviewed alternatives could undermine reliability. Participants suggest bridging traditional open-source communities and modern AI development, which currently may use different tools and artifacts.

Other proposals include treating code as “liquid” and focusing open-source governance on durable infrastructure and specifications; maintaining agent storage independently of ephemeral container lifecycles; and using corporate compliance and safety-monitoring architectures to aggregate organizational traces.

These are important observations, all pointing in the same direction: AI is transforming the mechanisms and the artifacts of software development. For an increasing number of use cases, the software source code itself may no longer be the most important shareable knowledge artifact. This isn’t a new phenomenon. Every major change in technology calls for new mechanisms, new artifacts, and new institutions.

Originally, software developers mostly distributed binaries, then open source developers made sharing source code the norm. Now we need to distribute and share something else: context and patterns for generating code on the fly. Knowledge is being unbundled and can be shared in new forms. A repo is now a source of context for solving the exact problem I have, rather than the bundle of problems that the original developer(s) encoded into their library or packaged solution. Some people will keep their context and patterns private, either because they see proprietary advantage in keeping them to themselves or because they are too busy or too lazy to share. But part of the problem is that we haven’t yet figured out what needs to be shared!

The challenge for the software development community is to identify the new knowledge artifacts of the AI economy, and to build new mechanisms and incentives for sharing them.

We’ve all had the experience of seeing a bespoke agentically generated app, wanting something similar, and wishing we could ask for a starting point so we didn’t have to vibe code an equivalent from scratch, with all the attention, false starts, and token costs that comes with that. Even in a world where software is generated by agents, it’s still not costless, and most of us like a good place to start.

Of course, this won’t be all software. There are lots of areas where “sort of alike” software is a recipe for disaster. Imagine a sales and marketing group where everyone had their own vibe-coded CRM tools, with different metrics and capabilities. This is not good. So there will still be big SaaS applications, distributed binary software, and distributed source code. But there’s also a need for something new.

The purpose of this article is to start a conversation about exactly what it is that developers should be preserving, sharing, and building on from others, and what mechanisms are still missing to make that virtuous circle work more effectively. I can’t see into the future; I can only speculate. So, as is often the case, I take a historical view. The poet W. S. Merwin wrote, “We are the echo of the future.” I have found that using history to frame the present can be a productive way to get our minds going in the right direction.

Note that while this article focuses on software development, the questions I’m asking here apply to virtually every field. Let me know if you have insights from your work in any area touched by AI.

From code to context

The ease with which LLMs can generate code is paradoxically getting people to focus too much on how easy it is not to produce code. That may be blinding us to the fact that code itself is being displaced as a valuable, tradeable, shareable knowledge artifact. Maybe “displaced” is too strong a word. Different species of value overlap for a long time. Eventually, though, old ones fade and new ones become dominant. This has happened before. Every technology revolution draws not only on new mechanisms but also creates new valuable knowledge artifacts. I talked about this in my recent short keynote at AGNTCon + MCPCon.

Video 1 The open source revolution depended in part on mechanisms like modular design and source code control, even more than it depended on free software licenses. It birthed new coordinating institutions like GitHub. But it also created a valuable new class of knowledge artifact in the form of the source code repo, and subsidiary valuable artifacts like commit history and reputation. A developer’s public repos became a kind of résumé, and a project’s commit log became a record of who had done what, and how well. This replaced a previous world where binaries were the tradeable objects of the software economy. Microsoft’s fortune was built on selling them, and to anyone who measured the software economy in binaries sold, free software looked like value being destroyed. (Steve Ballmer called GPL’d free software “a cancer” and Microsoft VP Jim Allchin called it “an intellectual property destroyer.”)

There were two ways that I like to think about the effects of open source software. First, value that was based on scarcity was replaced by value that was created by abundance. Back in 1975, Steve Baer described this as “the clothesline paradox,” explaining that when you put your clothes in the dryer, the energy you use is measured and counted, but when you hang your clothes on the line, it is ignored by accounting, despite doing the same job. Open source software was like sunshine, free for the taking, and its value unmeasured. Everyone compared the revenue of free software companies like Red Hat and MySQL to the vastly larger revenues of proprietary software companies like Microsoft and Oracle, noted the fact that they hadn’t even survived as independent companies, and concluded that open source was a less effective business model because its creators captured less of the value. They missed the point that far more value had been created than had been captured. Open source went on to power most of the internet economy, with the value captured by other means than selling the software itself.

Some people might think of this as a kind of failure, but they would be wrong. It is the system working exactly as its creators intended. I remember talking with Bob Scheifler, one of the creators of the X Window System, about whether it was OK that we were building commercial versions of their documentation. He told me that was exactly what he had intended. They put things out with a very permissive license precisely because they wanted others to build on top of them. I remember him saying “We want to raise the floor.” The creators of Berkeley Unix, Apache, and others felt exactly the same way. They didn’t want to win the existing game. They wanted to play a different game.

And they succeeded, big time. As Clayton Christensen noted in 2004, when one thing becomes a commodity, modular and fungible, something else (usually something adjacent) becomes valuable. He called this “The Law of Conservation of Attractive Profits.” At about the same time, I had stumbled onto this principle in explaining the implications of open source software in giving rise to the new paradigm that I was shortly to start calling Web 2.0. Clay and I first bonded when we gave our respective keynotes about essentially the same thing at the 2004 Open Source Business Conference. I later wrote about the conjunction of our respective ideas in a piece called “The Open Source Paradigm Shift.”

New artifacts are often the first visible sign of this value transition. You can see this at work in the evolution of the web. The web was born from the new mechanism of internet-enabled hypertext via HTTP, HTML, and the modular, replaceable software (the web browser and the web server) that made it useful.

At first, the new valuable knowledge artifact was the website with its domain name, associated URLs, and HTML-formatted web pages. DNS was the coordinating institution. People paid fortunes for domain names on the premise that the site was the thing that mattered. And it did matter, and still does. But less than it used to. Over time, Google became the new coordinating institution, and its ability to point deep within websites made the web page, not just the site itself, a valuable knowledge artifact. Social media made “posts” and short-form video into other knowledge artifacts. New kinds of value creation and value capture tumbled out of that unbundling. Packaged software was displaced as the key monetizable artifact and replaced by web applications, which were information- and software-based frontends to vast backend databases.

The click became a new and incredibly valuable knowledge artifact, the unit in which attention and intent were measured, and then priced. First pay per click advertising (Google), then engagement maxing (social media) became the basis of vast fortunes. The people who notice the new artifacts early, while everyone else is still focused on the old ones, then invent new business models based on them, often play a bigger game than those who were still trying to play by the new rules.

When I summed up these ideas in “What Is Web 2.0” in 2005, I talked about the key principles being “harnessing collective intelligence,” “software above the level of a single device,” and “data as the next Intel Inside.” (That is, the next valuable source of lock-in.) The companies that survived the dot-com bust were the ones that understood that the data their users generated through their activity on the internet was the asset that mattered, not just the software that served it. That was an argument about a new knowledge artifact, though I didn’t use those words at the time. PageRank understood that a link was a vote, and a new mechanism to improve organic search birthed new dimensions of the internet economy. Links were bought, sold, and gamed accordingly. SEO (both “white hat” and “black hat”) and search quality became a battlefield.

All those disciplines have had to change now that content has been absorbed into AI models. Books, music, video, web content, sites like Wikipedia, GitHub, and Stack Overflow were all grist for the mill. In the open source era, a voted-on answer to a well-posed question on Stack Overflow was a valuable artifact, and the reputation that came with writing good ones was a subsidiary artifact, just as it was on GitHub. The number of new questions asked on Stack Overflow peaked at roughly 200,000 a month around 2014 and was already sliding before ChatGPT, but the decline accelerated sharply after November 2022. The knowledge in those answers didn’t go away. It was absorbed into the models, which now give their own answers synthesized from all that they have consumed. In that way, they have also consumed much of the traffic that would have gone to the original knowledge creators. New mechanisms to create a circulatory economy of value creation need to be developed.

Software development also changed radically because of the internet. Another of my web 2.0 principles was what I called “the perpetual beta.” It’s easy to forget that in the PC era, software had major releases only every couple of years, with the new version distributed on CDs (and long before that, on 9-track magnetic tape.) I remember talking with Mark Lucovsky, who’d been a developer on Windows NT at Microsoft, shortly after he arrived at Google in 2005. He was exulting at the experience of being able to push a single change live worldwide soon after it had been tested. Eventually, we saw the formalization of new mechanisms like test-driven development (TDD) and continuous integration and continuous delivery/deployment (CI/CD), which we now take for granted, as if they’d always been the norm.

The story isn’t over yet, though. The old knowledge artifacts don’t go away all at once, and they don’t go down without fighting. The new artifacts are only half-formed, and have weaknesses that can be exploited. We’re in the middle of that fight right now. But I suspect the winners will be those who first understand what the sources of deep value will be five years from now.

When code is cheap, what becomes valuable?

When code is cheap to generate, the valuable thing may be the prompts, context, and other artifacts that tell the model what code to generate and how to do it well. I’ve started using the generic term “context bundles” to describe the artifacts that you hand the model to do its work.

In the opening to this piece, I talked about software patterns extracted from code as one kind of new shareable knowledge artifact. But these are only one of many kinds of “context bundles” that are starting to be shared. Spec-driven programming is all about developing context bundles. The spec becomes the thing that is reviewed, versioned, and kept, and the code is regenerated from it as needed. (A lot of people are worried that this might become a return to waterfall development. And it could, if done badly. But done well, with rapid turns? Maybe just right.)

An Agent Skill is another kind of context bundle, a folder of instructions, scripts, and reference material that an agent loads only when a task calls for it. Anthropic released the format as an open standard, and it has been adopted by a growing number of other agent products. An MCP server is yet another mechanism for presenting context that will shape the work of an LLM.

There’s also what I call “magic words.”

But that’s not all. The agent trace, the record of what an agent tried, which tools it called, and which context it drew on, is the closest thing we have to commit history for work that doesn’t get formalized as code. The log of which context bundles were invoked, and by whom, could also be the raw material for attribution and payment, much as the click was the raw material for advertising. And the stream of what people ask their agents to do may become to this era what the click was to the last, the signal of intent around which the coordinating institutions of the AI economy get built. I’ve often wondered what the agentic equivalent to PageRank will be. Will agents leave “pheromone trails” that grow stronger or fade over time, in the way that links do?

Even a well-formed question or conjecture could be a kind of context bundle. Math and science already value and exchange these, and I’ve started to think that there is an opportunity for building infrastructure for sharing and crediting them. After all, arXiv.org has already pushed the boundary for credit further upstream by publishing preprints rather than just reviewed papers. Why not go all the way to giving priority credit for well-formed conjectures?

The eval, a set of test cases that encodes what good work looks like for a particular task, may also turn out to be one of the most valuable things a domain expert can create, withhold, or share. But there’s a serpent in the Garden of Eden. At Foo Camp, Nick Vincent observed to me that there’s a real risk to evals as a context object, because there is a race going on to acquire massive amounts of data (say on corporate workflows) and sell it to the labs to train on it. But an eval only works if the model being judged hasn’t already seen the answers. Once the test cases leak into the training data, the eval measures what the model memorized rather than what it can do. How do we protect the holdout set for evals? That may be a critical mechanism that is still needed, and one that the people who build evals, and the companies whose workflows they describe, have every reason to develop. IMO, it would be good if there were more transparency into this process, and perhaps even supervision by a neutral body.

How do we come to trust new artifacts?

Part of the work of every revolution is building the mechanisms that help market participants to trust its new artifacts. In his book Nature’s Metropolis, William Cronon tells how Chicago became the center of the world’s grain trade in the 1850s. The mechanisms that enabled this were the railroad, the steam-powered grain elevator, and the telegraph. The coordinating institution became the Chicago Board of Trade. In 1856 the board adopted uniform grading standards. Before that, a farmer loaded his grain into individual sacks, and every buyer along the way had to inspect each lot or trust the seller. Afterward, a load of wheat was inspected by a middleman, dumped into a bin with other grain of the same grade, and its owner was handed a receipt for, say, a thousand bushels of No. 2 spring wheat. The grading standards became a new mechanism enabling trust and the grade receipt the new knowledge artifact certifying it. Together they separated the ownership of grain from the grain itself. Once merchants started trading receipts for grain, it didn’t take long before they were trading promises of grain not yet delivered, and the futures market was born. (That raises an intriguing question: What will be the derivatives of the AI era?)

Cronon’s history also observes that the receipts were only as good as the warehousemen who issued them. Some of them issued receipts backed by no grain at all, or spread rumors that stored grain was spoiling so they could buy receipts cheaply. It took years of fights inside the Board of Trade, and finally state regulation that the Supreme Court upheld in Munn v. Illinois, to make the receipts trustworthy. New knowledge artifacts invite counterfeits.

Even when there’s no fraud involved, there are often missteps in the early days of a new technology. Consider the rise of spreadsheet models that could be filled in with the user’s own data. A model was a set of assumptions and formulas that let anyone ask “what if?” and get an answer in seconds. Magic at the time! And eventually completely taken for granted. The model became an artifact that could be passed around, and used to make business decisions at companies other than the one that had created it. In 1984, a few years into the spreadsheet revolution, Steven Levy wrote “A Spreadsheet Way of Knowledge” about what VisiCalc and Lotus 1-2-3 were doing to American business. Levy warned that a spreadsheet model is only as good as its assumptions. Back then, people taking a poorly-executed spreadsheet model as gospel was the equivalent of some of today’s AI usage fails.

Feeling our way into the future

When people stop valuing the thing they used to value, it pays to ask what they are coming to value instead. Being alert to valuable new knowledge artifacts is a superpower. The new artifacts tend to show up well before the institutions that organize them, and long before anyone has worked out how to measure them. GitHub was founded more than a decade after open source software became mainstream, and Google didn’t become dominant till more than a decade after the first websites. My bet is that we are at a similar moment with context, with the artifacts starting to appear and the institutions that will coordinate them still unbuilt.

There is a lot of work that I’m following eagerly, looking to see where, whether, and how it gains traction.

  • Duncan Davidson is doing fascinating work on what it would mean to have a home for personal context. I can’t wait till he makes more of this work public and reusable. Relatedly, the open memory protocol initiative is an attempt to define a useful set of mechanisms for exchanging context.
  • Sam Schillace has started building a repository of what he calls agent-building patterns. One of those patterns, “Anchor on Canonical Names,” echoes the idea that I wrote about last year in “Magic Words,” and that we are also exploring with the O’Reilly Expert Intelligence MCP. As Sam puts it, canonical names are a way of referencing “a whole coherent methodology the model already learned in training, far more reliably than a paragraph paraphrasing the same idea.”
  • There are numerous attempts to reinvent GitHub for the agentic era, including Cursor Origin and Entire, founded by former GitHub CEO Thomas Dohmke. Entire describes itself as “new distributed Git forge that stores your intent alongside your code, built for the scale and throughput of agentic coding.” Instead of traditional pull requests, it uses Trails, an agent-driven review system that logs session traces and automated evaluations so developers can merge code without reading every single line of a diff. Origin is an agent-first Git hosting and collaboration platform designed around structured semantic diffs, machine-readable merge states, and automated agent routing. As commercial products, both of these are examples of existing software distribution mechanisms and artifacts.
  • Perhaps most interestingly for the point I’m exploring in this piece, Anthropic is building a proprietary, internal-only replacement for Git itself for its own use. The company published a very detailed description of the system, which it calls Scion. It might even be detailed enough for someone with a big team and a ton of compute to reproduce, using the description as a starting point. But you can see at the scale of what they have built the limits of the “distribute context and specs and rebuild on the fly” model.
  • But there are also attempts to build systems that enable developers to share lightweight pieces in a new architecture of participation. Common Fabric is building an entire system premised on disposable agent-generated software expressing repeatable patterns, using data that is associated with users rather than with applications and that carries with it its own rules for how it can be used. In my conversation with cofounder and CEO Alex Komoroske, he described these patterns as small pieces of code that can be fetched from anywhere on the internet or written by an LLM on the fly. Agents will mostly write them, not human developers. But they will mostly write them on users’ behalf, and once someone’s agent has built a pattern, anyone else can use it. Their solution may not be the winning one, but it is definitely a bold leap into what may be a very different future for software.

Don’t get trapped in the past of software development. Root yourself in its history, the soil from which it grows, and imagine its future. What will we want to share, what will we want to reuse, and what mechanisms are needed for us to do so effectively?

Post topics: AI & ML

Explore related articles

Image 2 article AI & ML ## The Missing Mechanisms of the Agentic Economy By Tim O’Reilly | March 23, 2026Image 3 article AI & ML ## The Missing Half of the AI Economy By Tim O’Reilly | January 7, 2026

Follow us

About O'Reilly

Support

International

Download the O'Reilly App

Take O'Reilly with you and learn anywhere, anytime on your phone and tablet.

  • Image 4: Apple app store
  • Image 5: Google play store

Image 6: O'Reilly home

Do not sell or share my personal information.

© 2026, O'Reilly Media, Inc. All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners.

Image 7

Image 10: O'Reilly Logo

Privacy Preference Center

When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.
More information

Enable All

Manage Consent Preferences

Strictly Necessary Cookies

Always Active

These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work. These cookies do not store any personally identifiable information.

View Vendor Details‎

Performance Cookies

Always Active

These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.

View Vendor Details‎

Functional Cookies

Always Active

These cookies enable the website to provide enhanced functionality and personalisation. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.

View Vendor Details‎

Targeting Cookies

Always Active

These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

View Vendor Details‎

Vendors List

Clear

  • checkbox label label

Apply Cancel

Consent Leg.Interest

  • checkbox label label

  • checkbox label label

  • checkbox label label

Reject All Confirm

Image 11: Powered by Onetrust

Image 12

来源:O'Reilly Radar · oreilly.com