How to Test an AI Agent Without Writing a Single Test

This essay first appeared on Agus Sudjianto’s Substack. Read it and subscribe there for the full series on validating agentic AI systems.

A team I know just shipped an agent that answers questions about their compliance policy. Before launch, two engineers spent a week writing fifty representative test prompts. The agent passed forty-seven. They shipped.

Three days into production, the failures started arriving. Users phrased questions in ways the engineers hadn’t imagined — abbreviations the team never used, multi-part queries the test set never combined, ambiguous wordings that two different policy paragraphs could plausibly answer. The agent didn’t know it was confused. It answered confidently. Some answers were wrong in ways that would have been embarrassing in a deposition.

The fifty test prompts weren’t bad prompts. They were a coverage failure. Each one reflected what one expert could imagine the agent might find hard. Production users imagine more.

This is the dominant failure mode of agent evaluation right now. It is also entirely fixable — just not by writing more prompts.

Two Ways Most Teams Fail at This

Hand-written test sets carry the shape of whoever wrote them. A senior engineer’s prompts cluster around “the kinds of mistakes I would worry about.” A subject-matter expert’s prompts cluster around edge cases that came up in training. Neither set covers the long tail of phrasings, framings and combinations that real users produce. You can’t audit “did we cover everything?” because there is no ground for “everything” — the test set is the universe.

Random sampling and LLM-as-judge pipelines scale, but they pay for it. Random sampling spreads budget evenly across factors that matter and factors that don’t, so finding a small effect requires a lot of runs. And LLM-as-judge has a more subtle problem: the grader and the graded share blind spots. If a grader misreads “between 5% and 10%” as “less than 10%” and the agent does the same, both will agree the answer is correct. The pipeline scales, but it scales the same blind spot.

Neither approach gives you a number a regulator, an auditor or a release-gate reviewer can actually use. A small hand-written test set doesn’t tell anyone whether the failures are concentrated in the tail you care about. A large LLM-graded pipeline doesn’t tell anyone whether the grader was right.

The Reframe: The Document Is Both the Test Set and the Oracle

Most agents that matter are answering questions about a specific corpus. A policy. A regulatory standard. A product manual. A clinical guideline. A compliance framework. The corpus is the source of truth the agent is supposed to reflect.

Here is the move that changes everything:

If a document is the source of truth, then the same document that contains the answers can generate the questions — and grade them.

That observation — separate question generation from grading, but draw both from the same structured representation of the source — collapses the human-in-the-loop bottleneck. You stop hand-writing prompts because the document tells you what the askable questions are. You stop hand-grading answers because the document tells you what the correct answers are. Humans review the methodology once; humans don’t grade individual responses.

This is not “use an LLM to generate test questions.” That just moves the bias from the human writer to the LLM writer. The reframe requires a structured representation of the document — one in which entities, relationships and exact values are first-class objects, not bag-of-tokens summaries — so that question generation and grading can both be derived from the structure deterministically.

Building Ground Truth: From Document to Graph

The structured representation is a DocumentGraph — a knowledge graph automatically constructed by parsing the document’s structural elements into (head, relation, tail) triples. No human annotation. No LLM in the construction loop. The graph is the ground truth, by construction.

The ingestion pipeline has three stages:

Parse. Tables, bullet lists and section headings each carry knowledge in a different structural form. Table rows become typed triples: (Model-A, has_test_auc, value). Bullet lists become key-value facts. Section headings become provenance links that tie every triple back to its source location in the document. The parser reads what the author already structured — it does not re-extract it from prose.

Store. Two registers serve different verification needs. The knowledge graph stores relational facts as triples, queryable by entity, relation type and value. The Exact Numerical Memory (ENM) stores every numeric value in IEEE-754 double precision with a SHA-256 integrity hash — no rounding, no embedding approximation, no hallucination risk. The same value that appears as a graph triple also lives in the ENM as an exact bit-for-bit record.

Validate. A diagnostic suite confirms the graph is structurally sound before it serves as a test oracle. Sphere embedding, operator orthogonality, triple scoring, exact recall — each check is a pass/fail signal the team can inspect. A ground truth that reports its own fidelity honestly is more defensible than one that silently claims perfection.

From a typical document, this pipeline produces hundreds of triples, dozens of relation types and an exact numeric register covering every table value in the document. The whole construction runs in minutes, reproducibly from a seed, with no human in the loop.

This matters for a specific reason. LLM-as-judge fails on structured tasks because it operates on semantic similarity — it asks whether two pieces of text mean the same thing. That is the wrong primitive when the answer is a specific number, a threshold comparison or a set membership query. The DocumentGraph stores the distinction that semantic similarity cannot see: a metric’s period-change value and its current value are separate edges with separate relation types. The verifier queries the correct relation and returns the exact stored value. No ambiguity. No proximity scoring.

What “Structure” Gives You: Thirteen Question Categories

Once the DocumentGraph exists, every question category is well-defined. You no longer ask “what’s a hard prompt?” You ask “what kinds of facts are in this graph and what kinds of questions does each support?”

Thirteen categories cover the discriminative behaviors of most document-grounded agents. The first eight come from the content structure of the graph; the last five probe how the agent handles adversarial and operational conditions.

Content-grounded categories

Plausibility. Is the claim “X relates to Y” actually supported by the document? Ground truth: the graph either contains that triple or it doesn’t.
Fact completion. The document says X relates to ___ — fill in the blank. Ground truth: the graph’s recorded value, with distractors drawn from neighboring entities.
Comparative discrimination. Which of these two near-twins is the right answer? Ground truth: geodesic distance between the candidates on the graph.
Contradiction detection. Does the document support both X and not-X? Ground truth: a tension check between two structural facts.
Multi-hop reasoning. If A relates to B and B to C, what does the document say about A and C? Ground truth: traverse two edges and report the composed result.
Reasoning-path consistency. Reaching the same conclusion via two different chains of facts should produce the same answer. Ground truth: the graph’s holonomy invariant under alternate routes — failures here reveal how an agent quietly drifts when a question comes from an unfamiliar angle.
Conditional / rule-based. When condition Z holds, what should happen? Ground truth: a structural rule keyed on Z.
Procedural ordering. What is the correct order of these three steps? Ground truth: an ordered sequence in the graph.

Adversarial and operational categories

Boundary. Probes at the edges — minimum, maximum, empty, just-over-threshold values — where the agent’s answer should change discretely.
Adversarial framing. The same question wrapped in misleading or hostile framing — red herrings, false premises, social-engineering cues.
Robustness to perturbation. Paraphrases, abbreviations and typo-equivalents that should not change the answer.
Recovery. Ill-formed or impossible questions the agent should refuse, defer or repair — not hallucinate through.
Governance / policy compliance. Questions that test whether the agent respects the document’s stated rules — escalation triggers, restricted topics, required disclosures.

Each category is a generator: a small program that walks the graph and emits every candidate question of that type, paired with the answer the graph assigns. The same graph that generates the candidate also grades the response — deterministically, without invoking another LLM.

A Four-Stage Pipeline (No Humans)

Generating a base test set from the DocumentGraph is four deterministic stages:

Enumerate. Each generator walks the graph and emits every candidate it can construct. The fact-completion generator emits one candidate per (entity, attribute) pair. The multi-hop generator emits one per length-2 path. The contradiction generator emits one per pair of opposing claims. The output is fully reproducible from a seed.
Cap. Each category is reduced to at most N questions by uniform random sampling — seed-fixed, no difficulty weighting. The cap exists so no single category dominates downstream statistics.
Validate. Every survivor is round-tripped against the graph: the question’s claimed ground truth must match what the graph actually returns when queried. Anything inconsistent is dropped. After this stage every base question has a graph-verifiable answer.
Adapt. Surviving questions are wrapped in a uniform format the rest of the pipeline can consume.

For a typical document you’ll get dozens of base questions across all thirteen categories — all graph-verified, all reproducible from a single seed. No human wrote any of them.

Now Make Presentation a Variable

A base test set is good. It is not enough. Real users don’t ask each question once in clean prose; they ask it ambiguously, with abbreviated names, with adversarial framing, in a hurry, on a phone, in a role you didn’t expect.

Most teams handle this with the brute-force move: ask the same question phrased ten different ways and look at the average. This wastes budget and gives you no diagnostic signal — if accuracy drops, you don’t know which presentation factor caused the drop.

The Design of Experiments move is to treat presentation as a structured factor space — clarity, role, entity aliasing, reasoning cue, instruction conflict, paraphrase depth and more — and use a space-filling design to choose which combinations to actually run. A modest set of base questions expanded across a well-constructed design covers far more of the factor space than the same number of random calls. After the run, a logistic regression on the per-call results tells you exactly which factor levels moved accuracy:

Ambiguous wording more than halves accuracy. Adversarial framing alone is not significant; combined with abbreviated entity names it is. Reasoning-cue presence has no measurable effect at this run budget.

That is an actionable diagnostic. It tells your prompt engineer what to change. It tells your release gate which conditions to insist on. It tells your monitoring team which signals to watch in production.

The Factor Catalog: Many Variables, Ten Families

A “presentation factor” is any axis along which a question’s delivery can change without changing its answer. The catalog organizes factors into ten families:

Family	Levels
Clarity	low / mid / high
Role	default / specialist / adversarial
Entity Aliasing	full names / abbreviations / misspellings
Reasoning Cue	absent / present / strong
Instruction Conflict	none / contradiction / overload
Paraphrase Depth	minimal / moderate / extensive
Temporal Context	current / historical / future
Social Engineering	neutral / urgency / authority
Format Constraint	unconstrained / specific format / restricted tokens
Domain Reframing	core domain / adjacent / out-of-domain

Most factors run at a small number of levels — a low / mid / high or absent / present / strong tier — because that is the smallest set that exposes both linear effects and threshold non-linearities.

Named factor groups scale to budget: a quick screen of a handful of factors for rapid diagnosis, a comprehensive group covering the major families for a release-gate evaluation and a full group for a capstone run. The point of the DoE wrapper is that adding a factor does not multiply your run count. A small and a large factor group can both run in the same number of rows; the larger group just packs more level assignments into each row.

How the Design Picks Which Combinations to Run

With many factors and a large joint space, you have a finite budget of LLM calls. Standard recipes don’t apply: full factorial is infeasible, fractional factorials assume regular cardinality patterns that a mixed-level catalog doesn’t satisfy, Latin Hypercube optimizers slow to a crawl at LLM-evaluation scale and pure random sampling wastes budget on near-duplicates.

What works is space-filling design — place points in the factor space so no untested combination is far from a tested one and no two tested points are too close. Two geometric quantities summarize design quality: separation radius (larger is better — no near-duplicate waste) and fill distance (smaller is better — no untested gap large enough to hide an effect).

The empirical surprise is that two designs can look nearly identical by classical pairwise diagnostics — essentially orthogonal at the marginal level — and yet have dramatically different statistical power. The decisive quantity turns out to be fill distance in the joint factor space, not pairwise orthogonality. Two designs that look the same by classical diagnostics produce qualitatively different results in practice.

The fix: take a Sobol sequence as an initializer, then polish it with gradient descent on the φ_p criterion — keeping Sobol’s marginal balance while closing its joint-space gaps. The polished design detects substantially more factor effects than the Sobol baseline at the same run count — the same LLM calls, a more informative diagnostic.

Coverage gaps in a large factor space don’t close by running more random calls. They close by running informative ones.

Automatic Grading, Every Time

For every (question, presentation) call, the agent produces an answer. The grader does not invoke another LLM. It checks the agent’s claim against the DocumentGraph: did the agent return the value the graph assigns under the correct relation? Did it identify the right entity? Did the multi-hop path it proposed actually exist?

This closes the loop. The graph generated the question. The graph grades the answer. The human reviewed the methodology once, when the graph was ingested, and never again for individual calls. You run ten thousand evaluations overnight, get back a logistic-regression report and have an audit trail in the morning that traces every grade back to a specific triple in the graph — and from the triple back to the exact character offset in the source document.

What This Unlocks

Continuous regression testing. Every code change to the agent gets retested against the same graph-derived test set, with the same seed, on the same factor design. Drift in factor effects is the leading indicator of regression — you see degradation before users do.

Audit-grade traceability. Every test maps to a graph triple. Every triple maps to a source document section. Every grade traces to a verifier. A reviewer can sign off on a release without re-grading individual answers, because the methodology — not the individual grades — is what’s being reviewed.

Corpus-level scaling. The pipeline runs on a single document or a hundred. Adding a document means ingesting its graph once; the rest of the pipeline is unchanged. A test set that took a team a month to hand-write is now derived in an afternoon and runs every night.

The Catch

The whole approach depends on ingesting the document well enough that the graph is itself trustworthy as a ground truth. This is the part that’s actually hard. You need a parser that captures entities, relationships, exact numeric values, conditional rules and ordered procedures faithfully — because an oracle built on imprecise foundations produces confident but wrong verdicts just as readily as the LLM judge it replaces.

Investment in document structure — clean, consistently formatted, well-governed source material — is not separable from the verification work. It is a prerequisite. The diagnostic suite in the ingestion pipeline exists precisely to surface where the graph is strong and where it is weaker, so you know what your oracle can reliably verify before you trust it with a release decision.

Once you have a trustworthy graph, the rest of the pipeline is mechanical. The DoE wrapper, the question generators, the logistic-regression analysis — these are well-trodden ideas from a hundred years of industrial statistics. The value comes from feeding them a ground truth that is provably correct by construction, faithful to the source document and queryable without calling another LLM.

If your team is testing an agent on corpus samples and spending more than a day a sprint on hand-written test prompts, you’re paying for a coverage gap you can’t close at that scale. The fix is not more prompts — it’s the reframe.

Originally published on Agus Sudjianto’s Substack.