This series is about the discipline that made a chain of language-model agents trustworthy, and it opens with the layer everything else stands on: the harness. By default, such a pipeline gives you no way to see where or why it went wrong. Each stage passes the next a slab of text, the next stage interprets it however it likes, and when the final answer comes out wrong you get to re-read a seventeen-minute transcript looking for the moment it went sideways. This note is about the harness that replaced all of that with something legible, and the single idea underneath it: wherever there was an implicit assumption, I put an explicit contract.
A strict schema at every seam
Between two stages, an unvalidated blob of JSON is just a rumor. The next stage believes it, but nobody checked. So every hand-off got a strict schema as its contract: required fields, exact types, value patterns, and the part that earns its keep: additionalProperties: false, so an unexpected field is an error, not a shrug. A stage's output is validated the instant it's produced, against the contract the next stage relies on.
# stage_3.output.json — written, validated, then handed on { "summary": { "net": 48200, "unit": "CHF" }, "items": [ { "kind": "A", "value": 31000 }, ... ], "checks": { "balanced": true } } ✓ valid — stage_3.schema.json · strict: unknown fields rejected
The value isn't the validation for its own sake, it's where the failure lands. Without a contract, a slightly-wrong field slips through, three stages compound on it, and the damage surfaces at the very end as a plausible-looking wrong answer with no return address. With one, the run stops at the boundary that produced the fault, loudly, pointing at itself.
Between two agents, an unvalidated JSON is just a rumor.
Shape is necessary, but not sufficient
A schema proves each hand-off is well-formed. It can't prove two well-formed hand-offs agree. One stage can emit a perfectly valid number that a later stage never expected and no per-stage schema will say a word. So the contracts don't stop at shape. Before the final stage assembles the report, a preflight re-checks that the stages actually reconcile: that a figure computed in one place matches the one consumed in another; that the balances carried forward are the projected ones, not today's. These are checks on meaning, not form. They catch the bug a schema is blind to: everyone honored the contract, and they still disagree.
A schema says each hand-off is well-formed. It can't say two well-formed hand-offs agree.
Every boundary writes to disk
Because each stage emits its contract-checked JSON to a file, a run isn't an event that happens and vanishes, it leaves a trail of savepoints. When a final answer looked wrong, I stopped re-reading transcripts. I opened the intermediate artifacts and diffed them against a known-good run, boundary by boundary, until I found the first one that diverged.
That stage owns the bug. Everything downstream of it is just faithfully compounding an error it inherited. The search collapses from "somewhere in seventeen minutes of reasoning" to "the first artifact that's wrong." It's a move that runs through this whole series: localize the blame by construction.
The first artifact that's wrong is the stage that's wrong.
The model decides; the code computes
The largest source of silent variation wasn't reasoning at all, it was letting the agent re-derive deterministic calculations on the fly. Ask a model to redo the same arithmetic on each run and it will write it a little differently each time: a rounding choice here, a reordered step there, and the number wanders for reasons no transcript will confess.
So every deterministic calculation was written down once, as a tested function in a library. The agent's job shrank to the part it's genuinely good at, which is reading messy input, gathering the right values, deciding what to compute, and the arithmetic moved to the part of the system that's good at it: code that returns the same answer every time and lives in one place you can read, test, and fix.
The model as calculator
The agent re-derives each calculation per run. It phrases the math a little differently every time, the number drifts, and nothing tells you which run was right.
The model as caller
Each calculation written once, tested, and called. Same inputs, same output, every run, plus a single place to inspect and correct the logic.
It's the same contract idea again, pointed inward: a tested function is a promise about the math, the way a schema is a promise about the shape. The agent orchestrates; it doesn't improvise the parts that were never supposed to vary.
Judgment is the agent's job. Arithmetic is the code's.
None of the other notes work without this
The harness is the least glamorous thing I built and the one everything else stands on. The stability eval this series ends on can only diff two runs because each stage leaves a machine-readable artifact behind. The complexity-vector curriculum can only blame one axis once it can first blame one stage. Reproducibility — "same input, same output," the precondition for measuring anything — exists only because the math lives in seeded, tested code instead of in the model's head.
Contracts made the seams visible. Savepoints made failures addressable. Moving the computation into code made the numbers mean something.
What I'd wire into any agent pipeline
Put a strict schema on every hand-off
Required fields, exact types, and no unknown fields. The contract catches malformed output at the boundary that produced it, not three stages later.
Fail loud upstream, not quiet downstream
A schema violation at the seam beats a plausible-looking wrong answer at the end. Loud-and-early has a return address; quiet-and-late doesn't.
Check that stages agree, not just that each is valid
Shape contracts catch a malformed hand-off; a cross-stage consistency pass catches two well-formed stages that quietly contradict each other. Validate meaning, not only form.
Persist every stage's output
Write each contract-checked artifact to disk. A run then leaves a trail of savepoints instead of vanishing when it finishes.
Debug by diffing artifacts, not re-reading transcripts
Compare intermediate outputs against a known-good run. The first artifact that diverges names the stage that owns the bug.
Move deterministic math into tested code
Write each calculation once, as a function. The agent gathers inputs and calls it. It never re-derives the parts that were never meant to vary.
Let the model decide, let code compute
Give judgment and messy-input reading to the agent; give repeatable arithmetic to the function. Each does the half it's good at.
The harness is the foundation, not the feature
Contracts, savepoints, and computation-in-code are unglamorous. But they're what make stability measurable and failures localizable at all.