← Field notes yingzhiva.github.io
Field Notes · A Series

Making a Multi-Agent Pipeline Reliable

Five notes on taking a chain of language-model agents from "a different answer every run" to reproducible. They build on each other: construct an inspectable, deterministic machine, feed it graded test cases, then measure whether it holds. But each stands on its own.

  1. 01

    Contracts All the Way Down

    Strict JSON schemas as hand-off contracts, cross-stage consistency checks, inspectable per-stage artifacts, and the math pinned in code the agent only calls.

  2. 02

    Say It Once

    Give every fact one home: code, prompt, or knowledge file, so nothing drifts out of sync, bloats the context, or costs tokens twice.

  3. 03

    Never Let the Model Type a Number

    Render every figure and table in code; let the model fill only the prose. Never trust it to transcribe a value.

  4. 04

    One Axis at a Time

    Break the difficulty of a case into complexity vectors and grow the test set one dimension at a time, behind a regression gate.

  5. 05

    Teaching an Agent to Hold Still

    Measure stability before accuracy: a drift eval that turns "it's flaky" into a located, fixable cause. The capstone that rests on the four before it.

New here? Read in order: the machine gets built in 01–03, tested in 04, and measured in 05. In a hurry? 05 is the one that ties it together.