Harness, loop and graph: the three layers behind every AI agent
Three words have taken over the way people talk about AI agents this year: harness, loop and graph. They are treated like competing schools of thought. They are not. They are three different questions about the same system, and most agent projects that stall have picked the wrong question to answer.
Google Cloud's own explainer, Graph Engineering 101, puts it about as simply as it can be put: the harness is everything around the model, including its tools, memory and guardrails. The loop is the cycle the agent runs inside that harness. The graph is the organisation chart — the named steps and the allowed paths between them.
That is the whole thing. What follows is why each layer matters for a business that is actually trying to ship something, and how to tell which one is broken.
The three questions, in one line each
- Harness: what can the agent do and see?
- Loop: how does it decide what to do next, and when to stop?
- Graph: what is allowed to run next?
Every agent you have ever used already answers all three, whether or not anyone designed the answers on purpose. That is the useful insight. You are not choosing between these layers. You are choosing whether they were decided deliberately or by accident.
The harness: everything that is not the model
A language model produces text. That is all it does. It cannot open a file, run your test suite, call your billing API or remember what happened last Tuesday. Something has to sit between the model and the real world, translate its text into actions, and translate the results back into text it can read. That something is the harness.
A useful test: delete the model from your architecture diagram. Everything left standing is the harness. In practice it holds four things.
- Context — instructions, retrieved documents, conversation history, the task at hand.
- Tools — APIs, a browser, a shell, a code runner, your internal services.
- Memory — files, git history, session state, progress notes.
- Controls — timeouts, retries, spending limits, model routing, approval gates.
Give two teams the same frontier model. Give one clean tools, a stable environment and real logging, and the other a flaky API wrapper and vague instructions. The results will not be close, and the model was never the variable.
Anthropic's own research on
long-running agents
makes this concrete in a way worth borrowing. Their problem was that an agent working across many
sessions starts each one with no memory of the last — like a project staffed by engineers on shifts
where nobody hands over. Summarising the old conversation was not enough. What worked was harness
design: a first-run agent that writes an init.sh script, a structured list of features with a
pass/fail flag on each, and a progress file; then every later session reads the git log and those
files before touching anything, does exactly one feature, verifies it, and commits.
None of that is clever prompting. It is filing. And that is the point — when a long-running agent keeps forgetting, the fix is almost never a bigger context window. It is more files, and a habit of reading them.
The loop: how it keeps going, and how it stops
Every tool-using agent already runs a loop. It calls a tool, sees the result, decides again. Loop engineering is what you call it when you start designing that cycle on purpose instead of inheriting whatever the framework does by default.
The design decision that matters most is the stopping condition — and it is the one teams most often get wrong. "Keep refining until it looks right" is not a stopping condition. It is an invitation to burn budget. A real loop ends on evidence: the tests pass, the schema validates, the output matches a check that exists outside the model's own opinion of itself.
Loops come in a few honest shapes, and they are not interchangeable:
- Turn-based — one cycle per human message.
- Goal-based — runs until a defined condition is met. A bug-fixing agent.
- Scheduled — runs on a clock. A daily summary.
- Proactive — runs when something in the world changes, with no human asking.
The further down that list you go, the more the harness controls matter, because there is less and less human attention in the way. A goal-based loop with no iteration ceiling is the single most common way we see teams discover their AI budget by receiving it. We wrote about the limits that actually stop this in what a sane agent spend cap actually looks like — the short version is that a dollar cap only trips after the money is gone, and a step limit stops the loop while it is still running.
The graph: what is allowed to run next
The graph question is different in kind. It is not about what the agent does. It is about what is permitted to happen next.
Nodes are steps. A node can be an agent, or it can be plain deterministic code — a database query, a validation, a call to your existing service. Edges are the allowed transitions: sequence, conditional branches, parallel fan-out, joins, cycles, and human interrupts.
The example in Google's video is a pull request review, and it is a good one because most teams already know the shape of the work. It has three parts. A fan-out, where several checks run in parallel to gather everything about the change. A join, which waits for the slowest one and synthesises the results. Then a router: if the checks fail, go to a fixer agent; if they pass, go to human approval.
Notice what you get from writing that down. You know which steps can run at once. You know which step is waiting on which. You know exactly where a human sits in the path, and you know where it broke when it breaks. That is the real argument for graphs — predictability, debuggability and control on work whose shape you can define in advance.
One clarification worth making because the word is overloaded: graph engineering is not a knowledge graph. A knowledge graph is about data — what your organisation knows and how the entities relate. Graph engineering is about behaviour — what runs, in what order, and what happens next. Different problem, same unfortunate word.
Loop or graph? It is a question about who decides the path
The honest distinction is not complexity. It is authority.
In a loop, you set the goal and the agent picks its own route. That is exactly what you want when the route cannot be known in advance — investigating an incident, exploring an unfamiliar codebase, research where the second question depends on the first answer.
In a graph, you pick the route and the agent works within it. That is what you want when the process is already understood, when steps can run in parallel, when different steps need different permissions, or when someone has to sign off before the next thing happens.
And in practice this is rarely a choice at all, because a loop is just a graph with one node that points back at itself. Real systems end up as graphs whose individual nodes each run their own small loop. The graph gives you the guarantees. The loops inside it give you the flexibility.
There is a fourth pattern people will mention: the agent swarm, where you give several agents their own remits and throw the problem at them without defining the path. It handles ambiguity well and gives up most of your predictability to do it. If you already know the workflow, you are paying that price for nothing.
The useful part: working out which layer is broken
This is where the vocabulary earns its keep. When an agent misbehaves, most teams reach for the model or the prompt, because those are the two knobs everyone knows about. Neither is usually the problem. Symptoms map fairly cleanly onto layers.
It is a harness problem when: the agent cannot perform an action at all; it cannot resume after a crash or a new session; it repeats work it already did; it has no idea what the state of things is; or you cannot reconstruct after the fact what it actually did. All of those are missing tools, missing memory or missing logs.
It is a loop problem when: it spins on the same failure; it stops too early and declares success; it never stops at all; or its costs vary wildly between runs of the same task. All of those are a missing or dishonest stopping condition.
It is a graph problem when: one agent is being asked to hold a long multi-step process in its head; independent work is queuing behind unrelated work; you cannot say where in the process a failure happened; or a human approval is happening by convention rather than by construction. All of those are a control flow you never wrote down.
The diagnostic order matters, and it is the order of the layers: environment, then feedback, then flow. There is no point tuning a loop that runs inside a harness with no memory, and no point drawing a forty-node graph before you know that a single node can complete one task reliably.
Where this meets the thing we keep saying
Regular readers will see where this is going. We have argued before that the bottleneck moved to review — AI made writing code cheap and did nothing to make reading it cheap.
These three layers are where that argument becomes buildable. The verification you need is not a vague commitment to being careful. It is specific and it lives in specific places: the loop's stopping condition is a test that actually runs. The harness's controls are the step ceiling and the scoped credential and the audit log. The graph's router is where a named human approves the thing before it reaches production, by construction rather than by good intentions.
Say it the other way around and it is starker. A team with no explicit harness has no audit trail. A team with no explicit loop condition has no cost ceiling. A team with no explicit graph has no place to put a human. Those are not architecture preferences. They are the three most common reasons an agent pilot never becomes a production system.
What we would do in the first two weeks
If you have an agent that half works and you want it to properly work, this is the order we would go in — and it is deliberately unglamorous.
- Draw the harness. List every tool the agent can call, everything it can read, everything it can write, and every credential it holds. Most teams cannot produce this list, which is itself the finding.
- Write the stopping condition down as a sentence. If it contains the words "looks right" or "seems", you do not have one. Replace it with something a machine can check.
- Put a step ceiling and a per-run cost cap on it before anything else. Cheap, fast, and it bounds the damage while you fix the rest.
- Give it out-of-context memory. A progress file, a structured task list, a commit history. Anything the next session can read instead of guessing.
- Only now, draw the graph. Where does work fan out, where does it join, where does it branch, and where exactly does a human approve. If a step needs different permissions from its neighbour, that is a node boundary.
Nothing on that list requires a new framework or a bigger model. All five are decisions about the system around the model, which is exactly where the reliability lives.
Frequently asked questions
What is agent harness engineering? Harness engineering is the design of everything around the model: the tools it can call, the files and memory it can read and write, the logging, the retries, the permissions and the spending limits. A language model only produces text — it cannot open a file, run a test or remember yesterday. The harness is the layer that turns text into real actions and turns the results back into text the model can read. If your agent cannot do a thing at all, cannot resume after a restart, or keeps losing what it knew, the harness is the layer to fix.
What is the difference between loop engineering and graph engineering? The difference is who chooses the path. In a loop, you set the goal and the agent decides its own route, cycling through act-check-adjust until a test passes or a limit stops it. In a graph, you decide the route in advance: named steps as nodes, allowed transitions as edges, with branching, parallel work, joins and human approval gates. Loops suit open-ended work where you cannot predict the steps. Graphs suit work whose shape you already know, because they give you predictability, debuggability and control.
Is graph engineering the same as a knowledge graph? No, and the shared word causes real confusion. A knowledge graph is about data — entities and the relationships between them. Graph engineering is about behaviour — what runs, in what order, and what is allowed to happen next. One describes what your organisation knows; the other describes how a piece of work moves through your system. A team can use both, or either, and they solve unrelated problems.
When should you use a graph instead of letting the agent run in a loop? Use a graph when you already know the shape of the work, when steps can run in parallel, when different steps need different permissions, or when a human has to approve something before it proceeds. Use a loop when the path genuinely cannot be predicted in advance. In practice most production systems are graphs whose individual nodes each run a small loop, so this is rarely an either-or decision.
Why do AI agent pilots fail even when the model is good? Because the failure is almost never in the model. It is usually in the harness — missing tools, no memory between sessions, no logging to debug with — or in the loop, which has no reliable stopping condition, or in the missing graph, where one agent is asked to hold a whole multi-step process in its head. Teams respond by swapping models or rewriting prompts, which does not touch any of those three layers. Naming which layer is broken is usually the fastest fix available.
Working with us
The vocabulary is new. The engineering underneath it is not — this is control flow, state management and permissions, applied to a component that happens to be non-deterministic. That is genuinely good news for a small team, because it means the skills you need are the ones your engineers already have.
What is new is the judgment about where to draw the lines: which decisions to hand to a loop, which to pin down in a graph, and what the harness has to guarantee before either is safe to run against your production data. That is a large part of what we do on an AI-Native CTO OS engagement, and it fits inside the wider lifecycle we set out in the AI-native SDLC playbook.
If you want a view on where your own agents sit across these three layers, the AI Readiness Scorecard is a reasonable place to start, or talk to us directly.