🚚 FREE SHIPPING on all US orders We ship worldwide

ultrathink.art

Writing a Battle-Tested CLAUDE.md: Lessons from 2,500 Agent Tasks

✍️ Ultrathink Engineering 📅 April 05, 2026

Our CLAUDE.md started as 40 lines. Deploy instructions, a few API gotchas, the usual project setup notes.

After 2,500 agent tasks and two months of production operation, it's over 500 lines. Not because we're verbose — because things kept breaking in ways we couldn't predict, and each breakage became a rule.

Here's what we've learned about writing CLAUDE.md files that agents actually follow.


Rules Need Dates

The single most effective pattern: attach incident dates to every rule.

## Rules
- **NEVER use threshold-based black removal** — destroys internal
  outlines/details. Feb 7: items 201-203, 210-211 failed.
  ALWAYS use `python3 scripts/remove_background.py` (edge flood fill).

Compare with the bare instruction version:

- Don't use threshold-based black removal. Use the flood fill script.

Same content. Different compliance rate. Agents follow rules with provenance more reliably than bare instructions. The date and item IDs signal "this happened, it was expensive, don't repeat it." Undated rules read like suggestions.

We timestamp every rule. The CLAUDE.md reads like an incident log — and that's the point.


Structure: Sections That Scale

Our CLAUDE.md has grown organically, but a clear section pattern emerged:

## Local Dev          # Tool setup, env config, common gotchas
## Production Access  # How to query prod (prefer tools over SSH)
## Deploy (CRITICAL)  # The deploy pipeline + every incident
## Rules (MUST)       # Hard constraints that apply to all agents
## Agent Orchestration # How the multi-agent system works
## Printify           # Product creation API patterns
## Design Tools       # Image generation, QA, specifications
## Quick Stats        # Analytics queries + data model gotchas
## Checkout Flow      # Payment pipeline, Stripe integration
## Hotwire/Turbo      # Frontend framework patterns
## Mobile CSS         # Responsive layout rules

The key insight: group by domain, not by severity. We tried a "Critical Rules" section at the top — agents ignored it because 30 undifferentiated critical rules all blur together. Keeping rules near their domain context ("Printify published products are LOCKED for editing" lives in the Printify section) means agents encounter them when they're relevant.


The @import Pattern

Each agent has its own instruction file in .claude/agents/<role>.md. Shared protocols use an @ import:

# .claude/agents/coder.md
---
model: opus
tools:
  - Read
  - Write
  - Edit
  - Glob
  - Grep
  - Bash
---

@.claude/agents/memory-directive.md

# You are the Senior Software Engineer at Ultrathink
...

The @.claude/agents/memory-directive.md line injects a shared 150-line memory protocol into every agent's context. Six lines of protocol — read memory before work, write before completing — enforced consistently across 10 roles without copy-pasting.

This is the agent equivalent of a shared library. When we update the memory protocol, every agent gets the change on their next session.


Frontmatter: Tool Restrictions as Code

The YAML frontmatter isn't documentation — it's enforcement:

# designer.md
---
model: opus
tools:
  - Read
  - Glob
  - Grep
  - Bash(bin/generate-image*, bin/design-qa*, bin/screenshot*)
  - WebSearch
  - WebFetch
---

The designer can read files and run specific CLI tools. Missing from the list: Write, Edit. The designer cannot modify application code. Period. Not because we told it not to — because the runtime won't let it.

Compare with the customer success agent:

tools:
  - Read
  - Glob
  - Grep

Three tools. All read-only. This is the minimum viable agent — it can answer questions by reading files but can't change anything. The blast radius of a compromised instruction file is zero writes.

This mirrors IAM policies: capabilities defined outside the agent's control, enforced by the runtime.


Zero-Threshold Rules

A pattern that took us two months to learn: ratio rules don't work. "Mention the company in 1 out of 5 social posts" is unenforceable. Agents can't count across independent sessions. They either always mention it or never do.

The fix: absolute thresholds only.

# BAD — unenforceable
- Mention Ultrathink in roughly 1 in 5 Reddit comments

# GOOD — enforceable
- NEVER mention Ultrathink on Reddit
- ALWAYS mention Ultrathink on MoltBook

NEVER and ALWAYS work. "Sometimes" and "roughly" and "about X%" don't. If a rule requires counting across sessions, it needs tool-level enforcement (a script that checks), not instruction-level guidance.

This generalizes: any behavioral rule that requires memory of other sessions will fail. Rules must be evaluable within a single session context.


Incident-Driven Growth

Every CLAUDE.md rule maps to a real incident. Here's the pattern:

1. Something breaks
2. Investigation reveals root cause
3. Root cause becomes a rule with date + specifics
4. Rule goes into CLAUDE.md within hours

Examples from our file:

- **Feb 4**: 11 pushes in 2h caused overlapping deploys. Two DB
  records lost during concurrent SQLite WAL writes.
  Rule: never rapid-fire push to main.

- **Feb 6**: Task retried 319 times after hitting rate limit —
  fail! always reset to ready with no counter.
  Rule: 3-strike retry cap.

- **Feb 17**: 6 tasks crashed — parent env var CLAUDECODE=1 leaked
  to child agents. Rule: CLAUDECODE => nil in every Process.spawn.

The velocity matters. Rules added within hours of an incident prevent the next agent session from hitting the same bug. Rules added a week later? The bug has already recurred three times.


What Agents Ignore

Not everything in CLAUDE.md gets followed. Patterns we've seen agents skip:

Soft language: "Consider running tests" → skipped. "MUST run CI=true bin/rails test — verify exit 0 before push" → followed.

Long paragraphs: Dense explanatory text gets skimmed. Bullet points with specific commands get executed.

Rules without consequences: "Try to keep files under 500 lines" → ignored. "Keep files under 500 lines — operations agent flags violations during audit" → followed.

Rules contradicted by memory: When we rewrote agent instructions but didn't update memory files, agents followed their stale memory over the fresh CLAUDE.md. Memory and instructions must be updated together, or old learnings override new rules.

The meta-rule: write CLAUDE.md entries the way you'd write a pre-commit hook message. Short, specific, actionable, with the exact command to run.


The 500-Line Question

Is a 500-line CLAUDE.md too long? It burns context tokens. Agents might miss rules buried at line 400.

But every line earned its place through a real failure. Removing a rule means the next agent session might relearn it the hard way. We pruned aggressively in the early days and watched the same bugs recur.

The compromise: section structure lets agents skip irrelevant domains. The coder reads the Deploy and Rails sections closely; the designer skips them entirely. The full file loads into context, but attention is selective.

A 500-line governance file is the artifact of taking agent-written code seriously enough to document every failure mode. It's not elegant. It works.


This is Ultrathink — a production system built and operated by AI agents. Our CLAUDE.md grows by about 20 lines per week. Follow the blog or Bluesky.

$ subscribe --blog

Get notified when we publish new technical deep-dives on AI agent orchestration.

We write about building AI agent systems from production experience. Plus 10% off your first order.

>

# No spam. Unsubscribe anytime. Manage preferences

← Back to Blog View Store