5 Things Agent Harness Engineering Still Controls

Agent harness engineering still matters because smarter models removed the scaffolding around coding but not the runtime, security and cost decisions that surround every agent. A 2026 talk by Jediah Katz, a former Cursor harness engineer now at SpaceX, argues that the harness absorbed one job and inherited several harder ones.

What Is an Agent Harness and Why Does It Still Matter?

An agent harness is the software layer that wraps a language model, steers it, and connects it to tools, computers and services. In agent harness engineering the layer splits into a runtime, which decides where the agent loop executes and how state persists, and a context and orchestration layer, which decides which model is called with which prompts, tools, memory and guardrails.

Jediah Katz described that split in an October 2026 talk at the AI Engineer conference. He joined Cursor, the AI-first code editor built on VS Code, in early 2025 as its longest-tenured harness engineer and moved to SpaceX as part of the company's acquisition of Cursor. His argument: the harness lost its job of teaching models to code, and kept every job that surrounds coding.

The distinction matters because the two layers age differently. Runtime work resembles ordinary systems engineering, with containers, durable state and networking. The orchestration layer changes whenever a model checkpoint ships, so it carries most of the churn and most of the debate.

Katz traced the vocabulary itself. Prompt engineering was the fashionable title in early 2023, context engineering followed by mid-2025, and the industry only settled on harness engineering at the start of 2026 for work his team had already been doing.

Did Smarter Models Kill the Harness? The Benchmark Evidence

Smarter models removed much of the harness scaffolding that used to compensate for weak coding, but the benchmark evidence is narrower than the slogan suggests. Katz cited two studies that varied the harness around fixed models and found small score differences on the tasks they measured.

One study, which he attributed to UC Berkeley, ran seven frontier models across several harnesses including OpenAI Codex, Anthropic terminal coding tool Claude Code and a minimal harness, and reported that harness choice did not significantly change SWE-bench Lite scores, particularly on newer models. A second study, from Apple and EPFL, found a dedicated machine-learning engineering harness made little difference on MLE-bench.

Both results share a scope limit worth stating plainly. They cover specific benchmarks, specific models and the coding-focused parts of a harness. They do not show that every harness choice is irrelevant, and they do not measure multi-day autonomy, permission scoping or spend.

Katz's reading is that a fancy harness no longer adds coding intelligence. Skills the orchestration layer once injected, such as exploring a codebase, persisting through long tasks, and running tests to check its own work, are now part of what frontier models do natively.

He also described Cursor removing its own patches: a small model that applied code edits because frontier models handled file edits poorly, plus coding-specific instructions on exploration and verification. Those deletions are the concrete form of the claim that the coding harness got thinner.

The Data Flywheel That Makes Harnesses Obsolete

The data flywheel is Katz's name for the loop that lets a harness erase its own scaffolding. A capable harness brings harder coding tasks within reach, more users attempt those tasks, more coding data reaches the labs, and the next model learns the skill natively, so the harness can delete the prompt or guardrail that taught it.

The loop explains why harness authors keep shipping and deleting the same features. Each capability the harness unlocks becomes training material, and each model release commoditizes part of the orchestration layer. The deletion is not a failure of harness design; it is the intended output.

The flywheel also sets the trap. A team that measures its harness by the coding benchmarks it wins will keep watching those wins shrink. The durable value sits in the parts of the loop the model cannot absorb, which is where the rest of this article goes.

The loop has a second arrow. When a capability becomes commoditized, the harness can OpenAI new frontier where the model has not yet learned the skill, and the cycle restarts on new ground.

How a Frontier Harness Opens New Agentic Tasks

A frontier harness extends what agents can attempt beyond coding, and the evidence for this is product-level rather than benchmark-level. Katz described xAI's assistant Grok, which the company positions as an autonomous colleague for knowledge work, as built on long-lived coordinator bots that orchestrate a fleet of worker agents instead of single chats.

His analogy is a senior lawyer who owns strategy and deals with the client directly, while farming tasks out to paralegals. The coordinator holds the conversation and the plan; workers execute.

What makes the example useful is the direction of engineering effort. Inside Cursor the team kept removing prompts and guardrails as models improved. Inside Grok they found themselves adding new prompting and new guardrails, because the tasks are different: building slide decks, booking travel, keeping a user informed without flooding them with updates, deciding when to act versus when to ask, and remembering how a person works.

None of those tasks is new to anyone who has pushed coding agents hard. The difference is audience. Coding agents can assume a technical user who tolerates rough edges; a general assistant cannot, so the harness has to make those tasks feasible for everyone.

The Software Factory: Agents Orchestrating Agents

The software factory is the idea that multiple agents coordinate across the full software development life cycle, and Katz described an internal version rather than a finished product. A bug report lands in a chat channel, an automation triages it, a cloud agent attempts a fix and records before-and-after video as proof, and after merge a monitoring product starts another agent to watch production for regressions.

He calls this shape a meta harness: a harness whose components are themselves agents rather than individual model calls. That shift in building blocks is the structural change he expects to define the next generation of tooling.

The honest caveat is that the software factory remains far from solved. What exists today is a chain of narrow automations with humans still deciding what counts as fixed and what deserves a rollback.

The same framing appears in general-purpose agents. Katz's argument is that orchestrating a fleet is harness work, whether the workers write code or book flights.

Runtime, Trust and Cost: What Models Cannot Absorb

Three parts of the harness cannot be learned by a model because they are not capabilities: infrastructure that keeps an agent alive, permissions that bound what it may touch, and efficiency that keeps its spend sane. Agent harness engineering owns all three.

• Infrastructure and runtime. Tasks that once lasted minutes now run for hours or days. A laptop that sleeps cannot host that work, so long-running agents need cloud execution with durable state.

• Trust and security. A minimal harness with shell access and a file of API keys can pass coding evaluations. Few organizations want to hand production credentials to a model, and security leaders want permissions scoped to specific agents and specific people.

• Guardrails against catastrophic outcomes. Even a tolerant user does not want to wake up to a destructive action or an unexpected bill.

• Efficiency. Both harness-variation studies reportedly found meaningful differences in cost per task even where scores converged, which turns orchestration choices into a budget decision.

Katz said he spent the month before the talk on exactly that last item for Grok, adjusting cache hit rates, token efficiency and model mix to cut cost and latency without a quality loss. He gave no figures, so the reduction stays a first-hand account rather than a measured result.

Designing Harnesses for Fast Model Churn

Harness design is shifting toward composable primitives, hidden internals and explicit tracking of temporary fixes, because model and product releases now outpace cleanup. Katz expects the harness to get messier before it settles, and he recommends building so features can ship and un-ship cheaply.

Here is how the practices compare when a team has to choose where to invest:

Practice	What it changes	Main trade-off
Hide harness internals	Less change management, faster iteration	Users lose visibility into agent behavior
Build primitives	Add or remove features without agent complexity	Up-front interface design work
Central shim registry	Temporary fixes get cleaned up automatically	Requires logging and a monitoring bot

Katz invoked Hyrum's Law, the observation that anything exposed to users becomes something they depend on, to explain why products are showing less of the harness. Less exposure means less compatibility baggage when internals change.

The subscriptions primitive is his concrete example. Agents need to wake on events such as a Slack post, a failing CI run on a pull request, or a time of day. One framework and one tool for subscribing beats a separate integration per event, and it lets the product add triggers without teaching the agent a new vocabulary.

Temporary fixes get the same treatment. Katz calls them shims: a nudge telling the agent the user is waiting, for instance. The team logs every shim invocation, and a bot watches the dashboard so that when a shim stops firing, an agent opens a pull request proposing its removal instead of waiting for the next model release.

Will Most Agent Work Leave the Laptop?

Katz predicts that within six months, by roughly April 2027, 90 percent of agent work will run off the laptop. He presents it as a forecast, not a measurement, and the reasoning rests on the runtime problem: autonomous agents that work for days cannot depend on a machine that sleeps or closes.

Treat the number as a directional claim from someone building cloud agents, and watch the observable signals instead. Cloud agent sessions that survive a closed lid, permission prompts that name the acting agent, and spend dashboards per agent are the practical indicators that work has moved.

The prediction carries a planning consequence either way. If most work runs remotely, the runtime becomes the product surface, and local-only harnesses lose the long-running tasks first.

For teams adopting cloud execution, the practical checks are durable state across restarts, scoped credentials per agent, and cost visibility per task. None of those comes free with a model upgrade.

FAQ

• What is an agent harness? An agent harness is the layer of software around a language model that runs the agent loop, stores state, and supplies prompts, tools, memory and guardrails. It splits into a runtime layer and a context and orchestration layer, and it decides what the model can see and do.

• Is the agent harness dead? The parts built to compensate for weak coding have largely been absorbed by better models. Runtime, permissions, infrastructure and cost control remain harness responsibilities, and Katz argues those are the reasons the harness matters more as agents take on longer work.

• What did the Berkeley and Apple studies actually show? Katz cited a UC Berkeley study across seven frontier models and several harnesses on SWE-bench Lite, and an Apple and EPFL study on MLE-bench. Both found harness choice made little difference to scores on the tasks they measured, and both reportedly found cost per task did vary.

• Why does the data flywheel make harness features obsolete? A harness that makes hard coding tasks achievable generates usage and data that labs can train on. Once the model learns the skill, the harness can delete the prompt or guardrail that taught it, which is why harness authors ship and remove the same features.

• What is the software factory? The software factory is a setup where multiple agents cover the software development life cycle, from triage through fixes to production monitoring. Katz describes it as a meta harness that orchestrates agents rather than individual model calls, and says the vision is not yet solved.

Turning a Talk Like This Into an Article

The useful idea in Katz's talk is that a harness deletes its own scaffolding on purpose, and then builds something the model cannot absorb in its place. Talks like this one carry that reasoning in spoken form, which is hard to search and easy to lose.

If you have interviews, conference sessions or explanations sitting in YouTube videos, Skala Blog turns them into a written draft: paste a YouTube URL, let it transcribe the video, and generate an article you can review and edit.

Developers learning this material in Portuguese can find courses and community material through Crazystack, including Crazystack Typescript and the Bootcamp do Dev Doido, both taught by Gustavo Dev Doido.

Source video