← All projects
Long-term context · ~30k memories active

Memory

A shared memory service across the AI tools I use daily, built to test what it actually takes to extract, deduplicate, and retrieve user context over months without losing track of contradictions.

Possibly the most important tool I have. Memory allows sessions to pick up yesterdays, last weeks, works, preferences from 6 months ago, regardless of vendor.

The service holds roughly 50,000 memories from daily use with a lot of ambiguity around the word “remember”: what’s worth remembering, which facts mean the same thing, when do old memories expire?

How a memory gets stored

At the end of a turn, a hook sends the exchange to the service. A background worker (Sonnet/Sol level) extracts facts worth keeping. Most conversation is temporary instructions and code; the extraction prompt has to separate that from durable context.

Request arrives Claude Code hook sends the conversation exchange AWS LAMBDA Resolve identity API key → user ID — one user can never read another's data AUTHORIZER Extract durable facts LLM reads the exchange, returns only facts worth keeping CLAUDE HAIKU Embed as vectors Each fact → a list of numbers representing its meaning VOYAGE AI Deduplicate Drop facts that are too similar to ones already stored PINECONE Store Save the vector + metadata into your personal namespace PINECONE

Each fact becomes a vector. Before storing it, the service checks for near-duplicates. I started with 0.92 and lowered it to 0.88. This has some parallels with the semantic gap problem in retrieval, but I digress.

How a memory gets found

Search follows two paths. One finds facts with similar meaning. The other follows relationships between projects, components, and concepts. A reranker scores the combined candidates against the question and returns the top five.

"What was I working on last week?" Embed the query Question → vector (a position on the map) VOYAGE AI Semantic search Nearest dots on the map Top 40 candidates PINECONE Entity-expanded Walk the graph for related items Top 40 candidates PINECONE in parallel Merge + deduplicate Combine both paths, remove overlaps → unique candidate set Rerank Score each result for relevance, keep the best matches VOYAGE RERANK Top 5 enriched results Confidence scores, temporal status, entity connections

The entity path catches related facts that may use different language. The reranker helps remove passages that sound relevant but don’t answer the question. If it is unavailable, the service falls back to similarity ordering.

What the store looks like

This May 2026 snapshot contains 8,046 facts. The store has since grown to roughly 50,000. Each dot is a memory; UMAP projects the vectors onto a flat map so I can inspect clusters. Search itself runs in the original vector space.

Memory Explorer scatter view showing 8,046 memories from a May 2026 snapshot projected onto a 2D map using UMAP. Semantically similar memories cluster together, color-coded by type.
8,046 facts from real conversations (a May 2026 snapshot), projected onto a 2D map using UMAP. Each dot is one stored fact. Colors represent types — blue for state claims, orange for events, green for intentions. The clusters form on their own: nobody told the system to group website memories together or put deployment events near each other. It happened because facts with similar meaning got similar vectors.

Keeping old facts useful

A new fact can supersede an older claim without making every older fact about that subject wrong. “Uses Python” and “wrote a Go script today” can both be true. A model judges those relationships, and superseded claims get an expiry timestamp.

Weekly maintenance archives old claims, consolidates duplicates, and reviews profile facts. Directives are protected. Removed memories go to a recovery table for 30 days; profile triage can archive, but cannot delete, and is capped at 2% per run.

Seeding a Session

Every conversation starts with some loose recall (mostly preferences, current work, and enduring profiles). A new session is seeded with about 500 memories, and turn-hooks periodically look up related memories to fetch them into the running context window.

Stack

PartTool
Runtime and accessAWS Lambda, API Gateway, MCP and REST
Extraction and conflict reviewClaude Haiku
Embeddings and rerankingVoyage AI
Memory searchPinecone
Profiles, relationships and recoveryDynamoDB