Tech Matters: From AI Assistance to AI Delivery: The Shift Towards Harness Engineering
This is the second article in a two-part series on moving from isolated AI experimentation to AI-enabled software delivery. In the first part, we covered how to optimize your operating model for AI-driven software delivery. In this article, we will focus on how to engineer agent harnesses to reduce human bottlenecks and make AI-delivered code more reliable.
Code generation got cheap. Verification and human attention did not – and that, in one sentence, is the bottleneck in AI-augmented delivery. This article is about how to shrink it. The agent harness becomes the main thing you develop under human supervision, rather than the product itself. You build it out of the codebase you already have, and every improvement moves more work into the lane where agents run unsupervised – the lane where humans are no longer the constraint.
1. The bottleneck moved
Teams that run coding agents seriously are now merging pull requests faster than they can read them. Not a few more. An order of magnitude more. Work that would have been sized at several senior-engineer days lands in an afternoon – provided the agent has an environment it can actually work in. Because the merge queue is not the constraint. The reviewers are. Once an agent can open a correct-looking PR in twenty minutes, the question stops being can we produce this? and becomes how do we know it's right without a human reading every line? We should name the scarce resource explicitly, because every practice below exists to protect it: human time and attention. The harness – everything you wrap around the model to guide it and check its work – is what converts agent throughput into shipped software. Without one, more agents just means a longer review queue which in turn stagnates the efficiency gains of adopting AI.
The teams getting the most out of agents in large, legacy, high-stakes codebases did not start over. They got there because they already had things that worked for humans: reliable dev environments, a real test suite, linters, fast CI. The agents inherited an environment rather than a blank page. The harness is largely made of things you were already supposed to have – and you are about to find out that discipline compounds.
2. Agent = Model + Harness
Some vocabulary before we argue anything. The harness is everything in the agent except the model. Part of it ships with your tooling; the part you own is the outer harness: rules files, tools, linters, tests, environments, review agents. It has two jobs – raise the odds the agent is right the first time, and let it self-correct before a human ever sees the output.
Two axes make everything else legible. Controls that act before the agent works we call guides: codemods, bootstrap scripts, a type system, rules files, reference implementations. Controls that observe after and feed back we call sensors: linters, type checkers, tests, structural checks, AI code review. Each can be computational (deterministic, milliseconds, reliable) or inferential (model-driven, slower but powerful, semantically rich). You need both guides and sensors – guides alone, and the agent never learns whether your rules worked; sensors alone, and it repeats the same mistake forever. Computational controls are cheap enough to run on every change; reserve inferential ones for where semantic judgment earns its higher cost.
One more term: harnessability. Strong types give you a sensor for free, clear module boundaries let you write architectural rules, fast tests make loops cheap. But harnessability is not a property of your repository – it is a property of a module. You do not harness a monolith. You harness one bounded context at a time, and the moment you do, agents can work inside it with far less supervision than outside it. Section 7 turns that into a routing rule.
3. The new job: designing environments, not typing code
When the agent fails, the fix is never "try harder," and never "fine, I'll write it myself." It is: what capability is missing, and how do I make it legible and enforceable? Then have the agent build that capability. Expect the first weeks to feel slower, not faster – not because the model is weak, but because your environment is underspecified. Fortunately, that cost is only paid once.
Some teams commit to never touching the keys at all. Read that as a forcing function rather than an ideology, because the logic is airtight: The moment you are allowed to fix code by hand, you stop building the harness. Every hand-fix is a lesson the environment doesn't learn. So pick one scope – one module, one sprint – and forbid yourself from editing code directly. You will discover your missing capabilities within hours. That list is your harness backlog.
Consider, meanwhile, what a coding agent actively rewards: automated tests (with a stable suite, agents fly; without one, they report success on things they never tested), planning before implementing, comprehensive documentation, good version control habits, effective automation, a culture of code review, an instinct for edge cases, a safe preview environment, judgment about what to delegate. Every item on that list is already a characteristic of a strong engineer. Agents amplify existing expertise. The discipline you already have just became your competitive advantage.
4. Make the repo the system of record
The governing principle: What the agent cannot see in context does not exist. The chat thread where you agreed the architectural pattern is, to the agent, exactly as unknown as it would be to a new hire joining three months later. Tickets, wikis, tacit knowledge – invisible. Only repo-local, versioned artifacts count.
A hierarchical knowledge base, not a giant rules file
The single enormous rules file fails predictably: it crowds out the task and the code (context is scarce), it turns guidance into noise (when everything is important, nothing is), it decays over time if not maintained well, and it resists mechanical verification. The fix is progressive disclosure. Keep the root rules file to roughly a hundred lines and make it a table of contents; the real knowledge lives in a structured docs tree – design docs with an index, execution plans split into active, completed and known debt, generated schema references, product specs, condensed third-party material, and a handful of topic files.
Scope rules to where they apply: global rules sparingly, the rest attached to subdirectories and file patterns, in the format your agent tools already read. This is also the most legacy-friendly practice here – you don't need a rules file describing your whole system. Start with the modules your agents will touch most.
The legacy on-ramp: have the agent write the map
The reason nobody documented your monolith is that it was never worth a human month. It is now worth an agent afternoon. Point an agent at a module and have it produce the architecture overview, the dependency graph, the real data flow, the conventions it can infer – and, crucially, a list of the places where those conventions contradict each other. That last artifact is gold, and no human was ever going to write it. Hence a rule we now apply on every engagement: The first thing you point an agent at in an existing codebase is not a feature. It's the harness.
Plans, enforcement, and technology choices
For complex work, commit execution plans with progress and decision logs to the repo, so the agent never depends on context living outside it. For multi-session work, express the task list as structured data – JSON, not prose. Models are measurably less likely to corrupt a structured file; let the implementing agent flip a status field, nothing more.
Then enforce the knowledge base mechanically to prevent natural decay into staleness: CI checks for freshness and cross-links, plus a recurring doc-gardening agent that finds documentation drifting from real behaviour and opens fix-up PRs.
With respect to technology choices: Favour dependencies that can be fully internalised in the repo. Human developers prefer time-saving libraries and frameworks because they can reduce the amount of code to be written, abstracting boiler plate code into reusable elements someone has already developed before and published to a package repository. Useful for humans, less so for agents. Be more conservative about external dependencies than before, not less: Agents work best when application code is predictable and all logic is accessible, rather than hidden in external libraries. Free bonus: Less use of external dependencies decreases the risk of being impacted by software supply chain attacks, an increasingly popular attack vector.
5. Encode taste and architecture mechanically
The principle: enforce invariants, don't micromanage implementations. Constrain the what, not the how – left free within a clear constraint, agents choose sensibly. Remember that the future audience of your codebase is not the next developer joining the team – it’s agents. Code rules and architectural principles that merely serve to keep code beautiful and consistent in nature for the sake of human readability - are often unnecessary or counter-productive for agents.
A layering discipline that works: each business domain divided into fixed layers with strictly validated dependency directions – types, config, repository, service, runtime, shared utilities, UI – with cross-cutting concerns entering only through one explicit interface. Everything else disallowed, enforced by custom linters and structural tests. Not by convention. Not by code review.
These are often architectural rules you wouldn't normally document in much detail, as they are common sense to experienced human developers. With agents it becomes an early prerequisite – documenting constraints are precisely what buy you speed without decaying the code base.
The single highest-leverage detail – and it is free
Write your lint and structural-test error messages to inject remediation instructions directly into the agent's context. The rules are yours, so the messages are yours. Don't emit "module boundary violation." Emit what was violated, why it matters, exactly how to fix it, and where the rule is documented. A benign prompt injection: a sensor that doesn't merely detect, but coaches. The cost is a string edit. The effect is that the agent stops making that class of mistake – everywhere, permanently. Sounds complex to write good lints like that? Remember, you merely write down the expectations, an agent writes the actual lint code and config.
The same logic covers taste invariants: structured logging, naming conventions, file size limits, reliability requirements. In a human workflow these feel pedantic. With agents they are multipliers – encoded once, applied everywhere, forever. And the steering loop tells you where the rules come from: whenever an issue happens more than once, improve the guide or the sensor until it becomes less probable, or impossible. When documentation isn't enough, promote the rule into lints.
You don't need the whole architecture. You need one rule.
Nobody is retrofitting a six-layer architecture onto a decade-old system. Pick one invariant you already believe in and have never been able to enforce – "the web layer never touches the database directly," "every new endpoint carries a permission check." Have the agent write the structural test. Apply it to new code only, if that's what it takes to get it green. You have just converted an argument you have been losing in code review for three years into something that fails the build. That is the broader unlock: agents made custom static analysis cheap, and it applies most to the messiest codebases.
Accept, in return, that the output will not always match your stylistic preferences. That's fine. The bar is: correct, maintainable, legible to future agent runs. Enforce boundaries centrally; allow autonomy locally. At the code level: have the agent do the thing that works, use plain SQL (agents write it excellently and can match it against your logs, which beats watching them struggle with an ORM), and keep permission checks local and visible – hide them in a distant config and the agent will forget them when it adds a route. That last one is in fact a security bullet, not a style bullet.
6. Give the agent the means to check its own work
If the agent cannot observe something, it cannot fix it. It can only guess – and then report success. Sections 4 and 5 make code maintainable. This one makes it correct, which is why it carries the most leverage.
Let it drive the application
Make the app bootable per working copy, and wire browser automation into the agent's runtime – DOM snapshots, screenshots, navigation – so it reproduces bugs and validates fixes directly instead of inferring behaviour from source code. Left alone, an agent will change code, run a unit test, poke the server with an HTTP request, and still never notice the feature doesn't work end to end. Instruct it to test the way a user would, and results change dramatically. The generalisation: Every capability you give the agent to observe reality eliminates a category of "it looked right but wasn't."
Give it observability
Set up a throwaway logging, metrics and tracing stack alongside each agent's instance, torn down when the task ends, and let the agent query it directly. The prompts this unlocks are the return on this investment: "ensure service startup completes in under 800 milliseconds." "No span in these four critical user journeys exceeds two seconds." Such non-functionals stop being wishes and become validated requirements. With this in place, a single agent run can work one task productively for many hours, unattended.
Better still, aim for observability as a byproduct of the agent writing the code, rather than a debug loop bolted on after failure. An example every developer can copy today: in debug mode, log any outbound emails (e.g. sign-up confirmation) to stdout or an observability tool running as a sidecar, and say so in the rules file. That single choice lets the agent complete a full registration-and-sign-in flow with a remote-controlled browser, unassisted – it reads the log for the link to click. Ask, for each flow: what does the agent need to see to finish this without me? Then log exactly that.
Reuse the environment you already have
Do not build a bespoke agent sandbox. Run agents on the same development environments your engineers use – pre-warmed, standardised, disposable. Because those environments are already isolated from production data and open networks, agents can run with full permissions and no confirmation prompts; the blast radius of a mistake is one disposable machine. And auto-approve matters more than it sounds: approving every command materially reduces the agent's ability to brute-force its way to a solution. Enable it with a dev container and a network allowlist, or a burnable cloud VM. Neither requires a platform team. Agents running inside secure and isolated environments is good practice anyway.
Tools and context
The question is never "tool server or command-line tool?" It is "how many tokens does this cost per turn, and how few tools can I show the agent for this task?" The teams running hundreds of tools successfully never expose them all at once – they present a curated subset per task type. The teams who replaced a tool server with a thin CLI and six worked examples did the same thing by another route. Both are progressive disclosure: show the agent what this task needs, and nothing else. The same logic governs sub-agents – let them do the noisy searching in a throwaway context and return a condensed summary, keeping the parent's window clean. It is also the cheapest token optimisation on the table.
7. Route the work: decision gates, review, CI and merge
Deterministic rails around the creative parts
Structure agent workflows as a sequence of nodes, some agentic, some hardcoded. "Implement the feature" gets the full loop. "Run the linters" and "push the branch" do not – those steps must never be subject to the agent's judgment. Hardcoding them saves tokens, removes a failure mode, and guarantees the step happens every time. Find every step that currently depends on "the agent will probably remember to," and make it deterministic.
Shift feedback left, then budget CI hard
The expensive anti-pattern: Push broken code, wait for CI, read the error, retry – a loop that burns tokens and wall-clock. The ladder that fixes it: a fast local check under five seconds before every push, autofixes applied automatically, so only real failures go back to the agent. Add a hard cap of two CI rounds – fail twice, and it goes to a human. That cap is the underrated one. Unlimited autonomy is a cost bug, not a feature.
The decision gate: route the change, don't blanket the policy
You will see two opposite policies in the wild: every agent PR is reviewed by a human, and human review is optional. These are not competing philosophies. They are two teams setting a single global threshold for what should be a per-change decision. "Always review" makes humans the bottleneck by construction – you have automated generation and left the constraint exactly where it was. That is not a harness. It's a queue. "Never review" is a policy nobody consciously chooses – it simply arrives: across the industry, the share of PRs merging with no review of any kind climbs sharply as agent adoption rises. Nobody decides that. It happens.
The decision is best made during requirement refinement, before a line is written. You are already refining the requirement; add one question: can an agent take this hands-off, or does it need a human in the loop, and at which stage? Deciding up front changes the economics entirely. At merge time, "this needs review" costs a human reading two thousand lines that already exist. At refinement time, the same verdict costs a human reading a two-hundred-line plan – and it steers the agent rather than auditing it afterwards. It also frees effort for where it now matters: a hands-off change should get sharper acceptance criteria, not fewer, because the spec has become the only human artifact in the loop.
A starting point for a checklist. Route to human review if any of these hold:
- Blast radius. Payments, authentication, permissions, personal data, migrations – anything irreversible.
- Weak sensors. Thin coverage, no structural constraints, no observability the agent can query. If the harness can't see it, a human must.
- Novel surface. No existing pattern to imitate – the agent is about to invent one that every future run will copy.
- Fuzzy acceptance criteria. No harness catches "correct" when nobody defined correct.
- CI or infrastructure touched. Non-negotiable.
Route hands-off only if all hold: a well-trodden pattern, strong deterministic sensors, contained blast radius, sharp criteria, and an end-to-end check the agent can run itself.
Now look at those triggers again. Nearly every one is a harness gap – thin coverage, no structural rule, no observability, a vague spec. Close any one, and an entire category of change moves out of the review lane and into the hands-off lane. Your decision gate is therefore also your harness backlog. They are the same document, and the gate tells you precisely which gap to close next. In an existing codebase, this is the whole strategy: you are not harnessing the monolith. You are growing an island where agents work unsupervised, one closed gap at a time.
Keep the gate alive. Version it in the repo; when a hands-off change bites you, work out which trigger should have caught it and add it. Calibrate with data, not vibes: Track escaped defects by lane, and if hands-off changes are not defecting more often than reviewed ones, your gate is too conservative – you are spending attention you didn't need to spend. Expect the boundary to move in both directions: Better models push work toward hands-off, a nasty incident pushes a category back. Every rule encodes an assumption about what the model can't do on its own, and those assumptions go stale. Adopting AI coding practices in an organisation is, in large part, the ongoing work of refining these gates. It is not a policy memo you write once.
Parallel agents: yes – but start with one
Start with one agent and a good harness; you will learn more in a week than from six agents and no sensors. Then parallelise, because it is the natural end state – five small issues piled up overnight, five agents, one review session, rather than five context switches. The governing constraint follows directly from our opening: Parallelism is bounded by your verification bandwidth, not your compute. Where verification is automated, parallelise freely; where a human must read the diff, you can only run as many agents as you can read. Which means your hands-off lane is your parallel lane, and every gap you close in the gate raises your parallelism ceiling.
8. Keep the harness healthy
Agents replicate the patterns already in the repository – including the uneven ones. In an existing codebase, that means they will faithfully reproduce your worst decisions from years ago, at scale and at speed. Drift is guaranteed. The predictable first response is a standing Friday spent tidying up after the agents. It doesn't scale.
Instead, use background garbage-collection agents on a cadence, scanning for deviations and opening small targeted refactoring PRs, most reviewable in under a minute, many auto-mergeable. A quality document grading each domain and layer over time, so documentation drift becomes measurable rather than a feeling. The argument underneath: Technical debt is a high-interest loan.
Test the harness
Everyone asks whether their harness is any good. Almost nobody measures it. Logging which sensors fire tells you something – a sensor that never fires is probably unnecessary – but not whether they *would catch* what they should. So test them: Deliberately inject a known violation and check the sensor catches it. Commit a layering breach – does the structural test fail? Add an endpoint with no permission check – does anything notice? Delete a test – does the gate block the PR? Remember, the agent harness becomes the most crucial DevOps tool to maintain, so keep it to high standards and validate it pro-actively against edge cases.
We have seen teams run a scheduled adversarial agent whose only job is to smuggle known-bad patterns past your sensors and report which got through. Cheap to build, and it converts "I think our harness is working" into a number. Every escape is a harness gap – which means an entry in your decision gate, which means a backlog item. The loop closes.
The harness is never done, and that is not a failure. It is the job. Three cheap signals, checked regularly: sensors that never fire (delete them, or find out why they're blind); issues that recur (encode them); and the most common reason changes get routed to human review (close that gap first – it has the largest cost attached to it).
9. Where to start
Ordered by leverage – and note that the top three cost nothing and don't depend on scale. A team of three can do everything in the first list on a Wednesday afternoon.
This week – roughly an afternoon each:
1. Take the three mistakes your agent keeps repeating, and turn them into lint rules. Write the error message so it tells the agent exactly how to fix the problem and where the rule is documented. Cheapest and highest-leverage change in this article.
2. Make your logs readable by the agent. Log to a file as well as stdout, say where in the rules file, and log whatever the agent needs to complete a flow unattended – confirmation links, debug tokens, seeded login credentials, etc.
3. Cut your rules file down to a map. Over roughly 150 lines and it's an encyclopedia. Move the content into a docs tree and point at it. Your token bill drops on every turn.
This month:
4. Give the agent a pre-work smoke test. A startup script, a booted app, one critical path end to end. If the app is broken, it fixes that first. Nothing else here changes behaviour as much per line of code written.
5. Point an agent at your least-understood module and have it write the map – architecture, data flow, inferred conventions, and where those conventions contradict each other. Read the contradictions.
6. Have an agent write characterization tests for one untested module. Review them. Freeze them.
7. Move one human review from the diff to the plan. One non-trivial ticket. Research, plan, implement. Review the plan.
8. Add a deterministic pre-push gate – a lint subset under five seconds – and cap CI retries at two.
9. Write v1 of your decision gate. Five triggers is enough. Commit it to the repo. Expect it to be wrong, and revise it monthly.
This quarter:
10. Encode one architectural invariant you have been losing in code review for years. Have the agent write the structural test. New code only, if that's what it takes.
11. Schedule one garbage-collection agent against a single class of drift. Start with docs that no longer match the code.
12. Test your harness. Inject known violations. Count the escapes. Fix the biggest one.
13. Work the gate as a backlog. Close the most common reason changes get routed to a human, watch a category of work move hands-off. Repeat.
And measure, so this doesn't become faith-based: the share of agent PRs passing CI first try; review time per PR, which should fall as sensors improve; escaped defects split by lane, which calibrates the gate; and the share of changes eligible for the hands-off lane over time, the real measure of harness maturity.
Closing
The discipline didn't go away. It moved. It used to live in the code; now it lives in the scaffolding – the tooling, the abstractions, the feedback loops that keep a codebase coherent. Building software still demands rigour. The hardest problems are simply no longer about writing the code, but about designing the environments and control systems that let agents build and maintain reliable software at scale.
And here is why you can start on Monday, in the codebase you already have. Every practice in this article – tests, docs, version control, fast CI, clean boundaries, good logs, sharp specifications – was already a good idea. The reason you didn't is that the payoff was slow, diffuse, and always someone else's problem next quarter. That changed. These practices compound now – automatically, on every line of code, forever. The engineers who already had discipline are about to discover it was an investment. And the ones who didn't have finally been handed both a reason to build it, and an agent that will do most of the work of building it.