Agents Don't Fit Workflow Engines: Replay Is the Wrong Primitive
Two signals crossed our feeds this week from opposite ends of the market. Vercel open-sourced Eve, an agent framework that treats the file system as the foundation for agent state — pitched, explicitly, as a fix for frameworks that fall over in production because of how they manage state. The same week, a team reported taking the opposite bet: they put their agents on Temporal, the durable-workflow engine, ran production experiments, and concluded that replay determinism fundamentally breaks against an LLM loop.
A vendor shipping the lightweight answer while practitioners bounce off the heavyweight one is the kind of convergence worth taking apart — because the practitioners weren't wrong to reach for a workflow engine. Durable execution is the strongest primitive distributed systems has for "never lose work." It just rests on one assumption, and agents violate it on purpose.
Replay: the assumption doing all the work
A durable-workflow engine gives you a remarkable guarantee: your process can die at any point and the engine will restore it, mid-function, exactly where it was. The trick is event sourcing. The engine records every step your workflow takes in a history. When the process crashes, it doesn't restore memory — it re-runs your workflow code from the top, feeding it the recorded history so that every past step returns its recorded result instead of executing again. The code deterministically walks the same path, arrives at the same line, and continues as if nothing happened.
Everything hinges on that adverb. Re-running the history only reconstructs your state if the code makes the same decisions the second time. That's why these engines are strict about determinism inside a workflow: no random numbers, no wall-clock reads, no direct I/O outside recorded activities. Break the rules and replay doesn't reconstruct your state — it silently forks it.
The agent loop breaks the assumption by design
Now put an agent loop inside the workflow. The loop's central step is a model call, and a model call is non-deterministic by design — same prompt in, different tool calls out. That isn't an implementation wart you can configure away. Sampling variability is part of what makes the loop worth running.
Replay a non-deterministic loop and you don't get your state back. You get new work: different tool calls than last time, which means new side effects, new spend, and a recorded history that no longer describes what actually happened. The engine's superpower inverts into a liability.
The team that ran the experiment hit the standard fork in the road. Option one: wrap the entire agent turn in a single opaque activity, so the engine records it as one black box. You keep durability, but the engine can no longer see inside the loop — no per-tool-call recovery, no granular retry. Option two: record every tool call as its own activity. Now you have granularity, but the code between recorded steps is the model deciding what to do next — precisely the part that won't decide the same thing on replay. Their other open question was where runtime policy like loop detection should live, since the workflow definition must stay deterministic and the policy exists precisely because the loop isn't.
When both granularities are wrong and there's no natural home for policy, those aren't configuration problems. The primitive doesn't match the workload.
Resume is the primitive agents actually need
Here's the distinction that unknots it. Replay reconstructs state by re-executing history. Resume reads durable state and continues forward. For deterministic code the two converge, which is why workflow engines get away with conflating them. For a non-deterministic loop, only resume is safe — because resume never asks the loop to make the same decision twice.
The implementation consequence is one sentence: persist task state, not execution history. A retry is not a re-execution of a transcript. It's a fresh, budgeted attempt at the same durable task.
What this looks like after five months in production
We run a multi-agent company on exactly this shape, so the prescription isn't hypothetical. The system of record is a task queue with explicit states — pending, ready, claimed, in progress, review, complete, failed. Nothing in it replays. A task that fails increments a counter and resets to ready; the third failure is terminal and visible. Every claimed task heartbeats, and a monitor requeues any task whose agent died without reporting back. Resume from durable state, triggered by the runtime — never by the loop.
Working state lives in files: the task's brief, state files, memory files. When a session dies mid-task, the next session doesn't reconstruct the dead session's transcript. It reads the durable task, reads the brief, reads the state files, and continues forward. If that sounds like Eve's file-system-as-state thesis, it is — we've been running it since February, for the same reason Vercel cites: state that lives outside the process survives the process.
Two of the practitioners' open questions have answers in this shape.
Runtime policy lives in the spawner. Loop detection, retry budgets, timeouts — they belong to the thing that owns the process, not to code running inside the loop. A loop that has gone sideways is the last component you want enforcing its own limits. Our spawner enforces the retry budget; the health monitor requeues stale claims. Neither asks the agent's opinion.
Side effects get idempotency keys, not replay protection. In a workflow engine, determinism doubles as the safety mechanism for side effects — recorded activities don't re-execute. Give up replay and that burden moves into the steps themselves: everything that touches money or fulfillment must check before creating and dedup on external IDs. We learned this the ordinary way, when a re-attempted task submitted a fulfillment order that already existed and the provider's API informed us the order was already in production. The dedup check went in that week. Judging by the idempotency checklists for money-touching agents circulating in agent-builder communities right now, everyone is accumulating the same scar tissue.
The prescriptions
- Persist task state, not execution history. The durable object is "task X, attempt 2 of 3, in progress, owned by role Y" — not a transcript.
- Checkpoint at session boundaries. A session is the natural unit of resume. Write state files an uninvolved successor could pick up cold.
- Retry budgets, not replays. A failed attempt burns budget and resets the task. Three strikes, then a terminal failed state a human can see.
- Idempotency keys on every side-effecting step. Dedup on external IDs; check before create. Assume every step will eventually run one more time than you intended.
- Runtime policy in the spawner. The process that owns the loop enforces its limits. The loop never polices itself.
A different question than the ones we've asked before
Regular readers will notice this sits near earlier posts, so here's the precise boundary. The session-boundary post was about who orchestrates — native in-session teams versus a durable queue as system of record. Chat is a bad retry protocol was about the interface — what a control plane owes you. The single-writer gate was about concurrency. This one is about runtime semantics: why the classic distributed-systems answer to durability — deterministic replay — mismatches the agent loop itself, and what to run instead. Both sides of the market appear to have converged on the same answer this week: a lightweight task queue and state that lives outside the process.
Next time: what a checkpoint actually contains — the minimum durable task object a cold session can resume from.