You can't write a unit test for an LLM agent. The output changes every run. The reasoning path is opaque. Mocking the model means you're testing your mock, not your agent. Traditional test-driven development breaks down the moment your system's core behavior is non-deterministic.

We tried anyway. Early on, we wrote tests that asserted on agent output strings. "Given this task, the agent should output TASK_COMPLETE with a summary containing 'deployed.'" These tests were flaky from day one. The agent would say "pushed to production" instead of "deployed" and the assertion failed. We'd loosen the regex, and then garbage output would pass. Classic testing antipattern: testing the wrong layer.

Six months and 5,000+ agent tasks later, we've landed on a different approach. Stop testing what the agent says. Test what it's allowed to do.

The Boundary Insight

Every agent in our system interacts with the world through tools — CLI scripts, API calls, file writes. The agent is non-deterministic. The tools aren't.

bin/blog-publish check either exits 0 or it doesn't. bin/design-qa either passes dimensional and transparency validation or it fails. The Printify API either accepts a product creation payload or it rejects it. These boundaries are deterministic, testable, and — critically — they don't care what the agent was thinking.

This is the contract test pattern applied to agentic systems. You define contracts at every boundary where the agent's output becomes the tool's input. Then you test those contracts exhaustively, and let the agent do whatever it wants inside.

Anatomy of an Agent Contract

A contract has three parts:

Input schema. What shape must the agent's output take to be accepted by the next tool? Our blog publishing gate requires frontmatter with title, date, author, and excerpt fields. The date must be parseable and not in the future beyond the scheduled window. Word count must fall between 800 and 1500. These are checkable before the post touches production.

Invariant checks. What must remain true regardless of the agent's approach? Blog posts must not contain platform automation keywords (Reddit ToS). Sticker designs must be a single connected shape (die-cut manufacturing). Product images must have transparent backgrounds (apparel printing). These invariants encode domain rules that the agent might not know or might forget.

Output validation. What does the tool produce, and does it match what downstream consumers expect? Reddit comments must be verified as actually posted (comment count incremented). Printify products must have mockup images after a 60-second sync window. These post-execution checks close the loop.

The agent can take any path it wants between "receive task" and "call tool." The contract only cares about the handoff point.

Five Contracts We Actually Run

1. Blog publishing gate (bin/blog-publish check). Validates word count, pacing rules (max 4/week, 1-day gap), content calendar entry, security keyword scan, and LLM tell-phrase detection. The marketing agent can write whatever it wants. This gate decides if it ships.

2. Design QA (bin/design-qa). Checks image dimensions, transparency percentage, aspect ratio, and visual complexity. Sticker-specific: fill ratio and rectangularity detect "rectangle on transparent background" designs that break die-cut printing. The designer agent has creative freedom. The contract protects manufacturing constraints.

3. Reddit post validator (lib/reddit_post_validator.rb). Screens all Reddit content against banned phrases before submission. Originally invisible (silent exit code 1, no feedback). We learned the hard way that contracts must surface rejections — 129 posts were silently blocked over three weeks, including our own brand name, causing an 83% traffic crash before anyone noticed.

4. Printify transparency gate. Apparel products abort if the design image has less than 20% transparent pixels. This catches designs with solid backgrounds that would print as visible rectangles on fabric. The product agent runs bin/printify create_tshirt, which calls the gate internally. No override path except --skip-qa, which gets flagged in logs.

5. MoltBook verification solver. Every post returns a math word problem that must be solved correctly. Our tool auto-solves it, but tracks consecutive failures with a circuit breaker — 5 failures triggers a safety pause because 10 failures triggers a platform suspension. The contract here is bidirectional: we validate our output, and the platform validates our identity.

What Makes This Different From "Just Add Validation"

Input validation is necessary but not sufficient. The contract pattern has two properties that plain validation misses:

Contracts are composable into chains. Our task system auto-injects QA chains: designer tasks chain to product tasks, product tasks chain to QA review. Each link in the chain is a contract boundary with its own input/output schema. The designer's output becomes the product agent's input, and the product agent's output becomes QA's input. You can test each boundary independently, and the whole pipeline is verified by testing the interfaces, not the implementations.

Contracts fail loud. This is where our Reddit validator taught us the most painful lesson. A contract that silently rejects input is worse than no contract at all. The agent thinks it succeeded. The system logs no error. Three weeks later you notice traffic fell off a cliff. Every contract must produce a machine-readable rejection signal that the calling agent can parse and retry against. REDDIT_REJECTED:{reason} is ours. Exit code alone isn't enough — the agent needs to know why the boundary rejected it.

Testing the Contracts Themselves

The contracts are deterministic code. They get normal unit tests.

# test/lib/blog_publish_gate_test.rb
test "rejects posts exceeding word limit" do
  post = build_post(words: 1600, type: :standalone)
  result = BlogPublishGate.check(post)
  assert_equal :rejected, result.status
  assert_match /over 1500 word limit/, result.reason
end

test "rejects same-day double publish" do
  publish_post(date: Date.current)
  result = BlogPublishGate.check(build_post(date: Date.current))
  assert_equal :rejected, result.status
end

These tests are fast, deterministic, and they run in CI. If the contract passes, the agent's non-deterministic output is guaranteed to meet production requirements — or it doesn't ship. You're testing the fence, not the animal inside it.

The Inversion

Traditional testing asks: "Did the code do the right thing?" Agent contract testing asks: "Is the agent unable to do the wrong thing?"

It's a subtle but important shift. You stop trying to predict what the agent will do (impossible with LLMs) and instead guarantee what it can't do (straightforward with deterministic gates). The contracts define the negative space — the boundary of acceptable behavior — and everything inside that boundary is the agent's problem.

We can't control what our agents think. We can control what gets through to production.

Next time: the coordination layer between agents — how task handoffs, state files, and shared queues replace the service mesh you'd build for microservices.