What the hell is a harness?

RUN CLAUDE OPUS 4.5 against SWE-bench Pro, the industry's hardest public test of whether an AI can fix real software bugs, and it scores 45.9%. Run the same model, unchanged, against the same test, and it scores 55.4%. Not a single parameter was retrained between the two runs; what changed was the software wrapped around the model. That software, a layer most people outside the frontier labs have never heard named, now moves more of the score than the model does.

The layer is called a harness. A large language model, reduced to essentials, does one thing: it takes text in and produces text out, and then it stops. It has no memory between calls, no way to run the code it writes, no hands to run it with. Ask it to fix a bug and it will describe the fix in fluent prose, but it cannot open the file, run the test, read the error, and try again. The harness is what gives it hands: the loop that hands the model a task, catches whatever it decides to do, carries that action out against the world, and feeds the result back so the model can decide what to do next—reason, act, observe, and around again, until the task is done or the budget runs out.

Everything that is not the model is the harness. The system prompt that sets its behavior; the catalog of tools it can reach for, from a shell to a file system to a test runner; the rules for what it keeps as the context window fills and what gets thrown away; the retry logic when a tool call fails; the ceiling on how many times it may loop before giving up. Claude Code, Anthropic's coding agent, calls itself the harness around Claude in its own documentation; Codex is OpenAI's version; Nous Research, an open-source AI lab, ships a popular one called Hermes, the second Hermès to make its name in harnesses, the first being the Parisian saddler that opened in 1837 and has since moved on to handbags.

Beast of burden

For most of the AI boom, the leap that mattered came from a bigger model, and each release (more parameters, more training compute, a higher benchmark) was the event the industry organized itself around. Those gains have compressed: at the top of the standard coding benchmarks the leading models now sit within a few points of one another, and a genuine model upgrade buys a lab two or three points over its predecessor if it is lucky. Hold the model fixed and change only the harness, and independent analyses find swings of ten to twenty points on the same test. In one such run, a scaffold built by researchers at Meta and Harvard took Claude Sonnet 4.5, the cheaper model, and edged out Claude Opus 4.5, the dearer one, on Anthropic's own harness. The weaker model, better harnessed, came out ahead.

The labs have noticed, and their spending has followed. When Anthropic gathered developers in San Francisco on May 6th for Code with Claude, its flagship coding event, it released no new model, a deliberate choice, and showed instead a run of features that all live in the harness: scheduled memory review, an automated grading loop, multi-agent orchestration. The grading loop by itself reportedly raised output quality by about a tenth, with no change to the model beneath it. A month earlier, within seven days of each other, Anthropic and OpenAI had both shipped managed-agent-harness infrastructure—sandboxed execution, checkpointing, credential scoping—as though each had reached the same conclusion about where the contest now runs. The lock-in points the same way. A team's accumulated Claude Code workflows, skills, and memory do not port to Codex, and that portability, not the raw weights, is where the switching costs are quietly gathering.

Anthropic's own engineers say as much. "Harness design is key to performance at the frontier of agentic coding," one wrote on the company's engineering blog, in a post about breaking through performance ceilings that prompt engineering alone could not. For an industry that has raised two years of capital on the premise that the model is the asset, that is an awkward thing to have to concede. A frontier training run costs hundreds of millions of dollars and takes months; a harness change costs a few engineers and a fortnight, and can move delivered performance further. Labs have begun spending on the second because it is where the marginal dollar now goes furthest.

None of this is new; it is only newly visible. The loop at the heart of every harness (observe, act, observe again) comes straight out of reinforcement learning, where an agent has for decades been defined as a function that reads its environment and returns an action, and repeats until it reaches a goal. The pattern it belongs to is older still. When the microprocessor became a commodity, the value moved to the operating system; when the smartphone's silicon commoditized, it moved to the app store. Abundance in one layer pushes the returns up to the layer that decides how the abundant thing gets used. Model capability is becoming abundant, and the return is moving to the harness that puts it to work.

For the buyer, the arithmetic is uncomfortable. A frontier model and the harness around it are sold, benchmarked, and priced as one object, and the harness is the half that costs least to build and moves the result most. The labs already understand which half they are competing on. The slides at the launch still lead with the other one.