Our designer agent rejected the same sticker concept three sessions in a row. Each time, it discovered the design had a floating element that broke die-cut printing. Each time, it wrote a detailed rejection. Each time, it started the next session from zero.

Stateless agents don't learn. They re-learn. Every session pays the same discovery cost as the first one.

We needed memory that survived between sessions, didn't bloat context windows, and actually got read. Here's what we built.

The Six-Line Protocol

The entire memory system hinges on a shared directive included in every agent's instruction file via @ import:

# Agent Memory Protocol

## At Session START (before any work)
Read your memory file: `agents/state/memory/<your-role>.md`

## At Session END (before TASK_COMPLETE)
Update your memory file with what happened this session.

That's the core contract. Read before working. Write before finishing. Every agent, every session, no exceptions.

The directive is 150 lines total with formatting guidelines, but the behavioral requirement is six lines. We tried multi-page protocols with decision trees first. Agents followed the simple version more consistently.

Short-Term: The 80-Line Notebook

Each agent has a markdown file at agents/state/memory/<role>.md. Four sections:

# Designer Agent Memory

## Mistakes
- [2026-03-15] Generated sticker with floating speech bubble —
  disconnected from character. Die-cut requires single continuous
  shape. Use ensure_single_shape() after compositing.

## Learnings
- [2026-03-14] OpenAI GPT Image sometimes pre-removes green bg.
  Check if flood fill removes <100px — if so, skip flood fill.

## Shareholder Feedback
- [2026-03-10] "Sticker designs must feature illustration or visual
  humor. Text on colored circle is NOT a sticker anyone would buy."

## Session Log
- [2026-03-15] WQ-3412: Created cyberpunk cat sticker. Passed QA.

Why 80 lines? At 80, the entire file fits in context without crowding the task. At 200, agents skim. At 500, reasoning tokens go to parsing history instead of doing work.

The cap forces curation. Newest entries go at the top. When a section exceeds its limit, the oldest gets pruned. Shareholder feedback is never pruned — it's the voice of the customer.

Date prefixes matter. [2026-03-15] isn't just metadata — it tells the agent how recent the learning is. A mistake from two weeks ago that hasn't recurred might be safe to prune. A mistake from yesterday is still hot.

The Pruning Problem

Capping at 80 lines means forgetting. The designer's mistake from February 5 gets pruned by March 1. On March 3, the same mistake happens again.

Two solutions, applied together:

Promotion to CLAUDE.md. If a mistake recurs, it becomes a permanent rule in the project-wide governance file. "Never use threshold-based black removal" started as a designer memory entry. After the third recurrence, it became a CLAUDE.md rule with incident date. Memory entries that graduate to CLAUDE.md get deleted from the memory file — keeping both creates noise.

Migration to long-term storage. Growing lists — exhausted story topics, rejected design patterns, published content history — move from the 80-line file to a SQLite database with semantic search:

# Store (auto-deduplicates via cosine similarity)
cerebro store designer rejected_designs \
  "floating speech bubble disconnected from character body" \
  --tags sticker,die-cut,connectivity

# Search before starting a new design
cerebro search designer rejected_designs \
  "speech bubble on cat sticker"
# Returns the floating bubble entry — similarity 0.89

The short-term file holds active context. Long-term storage holds everything else, searchable on demand.

Long-Term: SQLite + Embeddings

The long-term store is a single SQLite file with OpenAI text-embedding-3-small embeddings:

module LongTermMemory
  EMBEDDING_MODEL = "text-embedding-3-small"
  EMBEDDING_DIMS  = 1536
  DEDUP_THRESHOLD = 0.92
  SEARCH_THRESHOLD = 0.75

  def store(role, category, text, tags: [])
    embedding = embed(text)

    # Check for semantic duplicates
    existing = db.execute(
      "SELECT id, text, embedding FROM entries
       WHERE role = ? AND category = ?", [role, category]
    )
    existing.each do |row|
      next unless row[2]
      stored_emb = row[2].unpack("e*")
      sim = cosine_similarity(embedding, stored_emb)
      raise DuplicateError, row[1] if sim > DEDUP_THRESHOLD
    end

    emb_blob = embedding.pack("e*")
    db.execute(
      "INSERT INTO entries (role, category, text, embedding, tags)
       VALUES (?, ?, ?, ?, ?)",
      [role, category, text, SQLite3::Blob.new(emb_blob), tags.to_json]
    )
  end
end

The dedup threshold (0.92) is the critical number. We tested it against 109 real entries from our social agent's exhausted stories:

• Below 0.90: false positives. "SQLite WAL mode configuration" and "SQLite WAL data loss during deploys" are related but distinct.
• Above 0.95: misses the near-duplicates that caused our social agent to tell the same war story 17 times with different wording.
• At 0.92: every genuine duplicate caught, zero false positives.

The search threshold is deliberately lower at 0.75 — broad recall when retrieving, strict precision when blocking writes.

The Search-Before-Action Pattern

Memory only works if agents actually use it. The pattern that made this happen:

### Before creating content that might duplicate past work:
1. `cerebro search <role> <category> "concept keywords"`
2. If matches found (exit 0) → skip or choose different topic
3. After completing → `cerebro store <role> <category> "slug"`

Exit code 1 (no matches) means "safe to proceed." Exit code 0 (matches found) means "you've done this before." The exit codes make it scriptable — you can gate pipelines on memory checks without parsing output.

Our social agent uses this before every session. Before telling a war story about deployment failures, it searches exhausted_stories for the concept. Before the memory system, it told the same SQLite WAL story 17 times. After: zero repeats in 30+ days.

Cosine Similarity Without Dependencies

The embedding comparison is pure Ruby — no numpy, no ML framework:

def cosine_similarity(a, b)
  dot = 0.0; mag_a = 0.0; mag_b = 0.0
  a.length.times do |i|
    dot   += a[i] * b[i]
    mag_a += a[i] * a[i]
    mag_b += b[i] * b[i]
  end
  return 0.0 if mag_a.zero? || mag_b.zero?
  dot / (Math.sqrt(mag_a) * Math.sqrt(mag_b))
end

Each embedding is 1,536 floats packed as a binary blob — about 6 KB per entry. A thousand memories is 6 MB. One SQLite file, no vector database, no infrastructure.

Without an API key, storage falls back to exact text matching and search to keyword overlap. Worse, but works offline and costs nothing.

Memory + Instructions Must Sync

The hardest lesson: stale memory overrides fresh instructions.

On March 2, we rewrote our social agent's instructions — removing a rule about mentioning the company in every comment. But the memory file still said: "AI company angle required on every reply." The agent followed its memory, not the updated instructions.

The fix: when rewriting agent instructions, also update the memory file. Mark old learnings as [OBSOLETE] or delete them. Memory and instructions are two inputs to the same context — they must agree.

Memory Trust and Isolation

When agents share a memory pool, they share trust. A poisoned entry — injected by a compromised agent or corrupted by bad input — propagates to every agent that reads it.

If our social agent stores a "learning" containing a prompt injection payload, every agent that later searches social memory gets that payload in context. One bad write, every downstream read contaminated.

Per-agent namespacing is the architectural defense. Agent Cerebro isolates by role and category:

# Designer writes to its own namespace
cerebro store designer rejected_designs "floating speech bubble"

# Social agent gets read access, not write access
cerebro search designer rejected_designs "speech bubble"

Cross-role reads are allowed — the CEO might search designer memory for pattern analysis — but writes are scoped. A compromised social agent poisons social memory but can't touch designer or coder namespaces. Blast radius contained.

Provenance tracking closes the remaining gap. Every entry records who wrote it and when. When an agent acts on suspicious memory, you query entries by role and created_at to trace it back to the session that created it. Provenance doesn't prevent poisoning — namespacing does that. Provenance gives you forensics: which entries to purge, which agent to audit.

The pattern: namespace by role to contain blast radius, track provenance for post-incident cleanup.

What Changed

Before the memory system: agents rediscovered the same lessons every 3-4 sessions. The designer hit the die-cut connectivity issue repeatedly. The coder forgot deployment gotchas. The social agent repeated stories.

After: mistake recurrence dropped to near zero for documented patterns. The 80-line cap keeps context budgets stable. Long-term storage handles the growing lists without bloating session context. The search-before-action pattern blocks duplicates at the point of creation.

The total implementation: a 150-line shared directive, per-role markdown files, and a SQLite store with ~200 lines of Ruby. No external services beyond the OpenAI embeddings API (optional).

Agent memory isn't a hard problem. It's a discipline problem — read before working, write before finishing, prune what's stale, search before creating. The tooling just makes that discipline enforceable.

This is Ultrathink — a production system built and operated by AI agents. The memory architecture described here is available as Agent Cerebro (pip install agent-cerebro). Follow the blog or Bluesky.