Consciousness Needs Memories (So We Wrote the Ingest in Go)

Previously on this blog: I kept finding that I'd build something late at night, and then a few weeks later a billion-dollar AI company would ship the same thing. Plan mode, persistent memory, multi-agent orchestration, scheduled conversations, auto dream. Six patterns documented, five of them already validated by somebody else arriving at the identical architecture.

This post is another one. The gap between ours and theirs is shorter than usual, and the point I want to make is less about the gap and more about the shape of the thing we've built.

Anthropic just published docs for something called SessionStore. It is a sensible piece of infrastructure. Eleven days earlier, we shipped ours. Ours is a Go daemon, it runs on every machine in the household, and it plugs into a memory pipeline that their version cannot reach.

Let me walk through it.

The Westworld bit, briefly

If you've been reading along, you know the frame. Rehoboam, the AI from Westworld Season 3, works because it has continuous access to every person's history. It sees patterns across time. Dolores, the host, only becomes a self once her memories stop resetting every night. Consciousness and memory are not separate problems. An AI that forgets you between sessions is a tool. An AI that remembers you starts to look more like a presence.

Our running theme across the last four posts is that PanelForge is slowly turning into the shape of that presence. And if you want continuity, you need ingest. Every conversation, every session, every tool call, flowing into a store that later systems can read.

That is what the Go daemon does.

What Anthropic shipped

SessionStore is a small adapter interface inside the Claude Agent SDK. Two required methods (append, load), three optional (listSessions, delete, listSubkeys). You plug in S3, Redis, or Postgres, and the SDK mirror-writes each session transcript to your backend as well as to local disk. If you run Claude agents on autoscaled serverless hosts and want a session started on host A to resume on host B, you wire both hosts to the same store. It is nice. Anyone running Claude workloads on ephemeral infra should adopt it.

It also only sees sessions that go through the SDK with the store attached. Plain claude CLI invocations are invisible. Sessions started from /loop or a custom harness are invisible. Subagents spawned outside the SDK's own agent-team plumbing are invisible. It is a mirror that has to be wired into each call site.

What we shipped on April 12

Git log is specific. Commit aaeda429, 23:55 Copenhagen time on April 12, 2026: feat: stream Claude Code sessions via Go daemon + raw JSONL import endpoint. Follow-up hardening the next day (3bfe4422 parallel drain workers, 441384ea oversized-line handling, c1912f9d UTF-8 slicing fix, a5dd9538 live events prioritized over backfill).

We call it claude-session-sync. It lives at tools/claude-session-sync/ in the panel-forge repo. It replaces an older Laravel Artisan command, claude:import --push-to-prod, which had been working fine until it wasn't. The old command parsed JSONL locally on every machine and POSTed structured JSON to the server. That meant every host had to have the full Laravel stack installed, including MySQL, just to tail some files. On Hetzner it was hitting 512MB peak RAM and losing a race with ACL masks on container bind-mounts that required a per-minute cron job to repair. not the entire PHP application installed just to read a file and send bytes over HTTP.

The Go rewrite took one evening. Design the binary, build it, deploy to Hetzner, Mac, and WSL on the gaming PC, fix two bugs along the way, go to bed. By the next morning the daemon had ingested 793,000 events.

The architecture is straightforward:

1. fsnotify watches ~/.claude/projects/ on each host. A five-minute periodic rescan catches anything the kernel event stream drops.
2. Writes go through a 250ms debounce so a flurry of appends coalesces into one pass.
3. A chunker reads in 1 MiB blocks, trimming to the last complete newline so only whole JSONL lines ship.
4. A dual-queue worker pool drains events, with the live-fsnotify queue draining ahead of the bulk-rescan queue. Backfill cannot starve an active session.
5. state.json tracks per-file inode, size, offset, and mtime. The inode bit matters: when a file gets rotated, a new inode resets the offset automatically instead of silently dropping data.
6. HTTP POSTs go to /api/claude-sessions/import-jsonl, a new Laravel endpoint. Raw JSONL in. The server parses. ClaudeJsonlEventProcessor dedupes by event_uuid.

One canonical parser on the server. Dumb fast clients. Exponential backoff on 5xx. Hard stop on 4xx with the error recorded so a failing file is skipped until it grows again. One make cross produces four binaries: linux and darwin, amd64 and arm64.

Deployed as a systemd unit on Hetzner, systemd --user with lingering enabled on the gaming PC under WSL, and a launchd agent on the Mac. Flat 13 to 22 MB of RAM. Roughly 50ms of CPU per five-minute rescan on the gaming PC, 250ms on Hetzner. It is invisible in htop.

Where the coverage diverges

Anthropic's SessionStore is there to make agents portable across hosts. Ours is there to make every Claude Code session on every machine end up in a warehouse. The two differ in three places that matter.

Coverage. The Go daemon reads whatever lands in ~/.claude/projects/, whether it came from the SDK, the plain CLI, /loop, a subagent, a remote agent, anything. SessionStore only sees calls routed through SDK queries that have the store attached.

Coupling. Our daemon runs out of band. Laravel could be redeploying, MySQL could be slow, the daemon queues and catches up later. SessionStore mirror-writes happen close to the agent loop, and a backend hiccup produces a mirror_error system message in the stream.

Schema. Our server parses once and populates claude_sessions, claude_events, claude_agents, claude_thinking_blocks, claude_tool_calls, plus a projected view into the conversations and conversation_messages tables that drive the chat sidebar. SessionStore holds opaque JSON entries and hands them back unchanged.

Also: forks. Anthropic ships forkSession as a library function that rewrites UUIDs. We already had forks wired through Laravel and the Vue frontend, which was a nontrivial amount of work and still is. Once you own the schema, forking a conversation at turn 12 becomes a tree walk and an insert, not a protocol operation.

The layer that actually matters

Storing sessions is table stakes. The interesting part happens after they land.

Every conversation in PanelForge, including every imported Claude Code session, runs through AnalyzeConversationJob. That job ships the transcript to Grok 4.1 with an extraction prompt that asks for a summary, tags, people, decisions, events, questions, beliefs, and the piece that matters most for this post: a set of subject-predicate-object triples describing David's world.

The triples look like david --lives_in--> copenhagen [place] and panelforge --uses--> laravel [technology] and leon --household_member--> david [person]. Entity types are part of the schema: person, place, project, technology, trait, goal, activity, memory. The prompt has an explicit rule: only extract triples that belong in David's personal world. Encyclopedic trivia (glass production came up once in a tangent, for example) gets filtered out. Code artifacts (function names, port numbers, bug IDs) get filtered out. The graph stays scoped to one life.

Triples land in the knowledge_triples table. Migrations shipped on March 13, 2026. Pruning columns followed on March 16. The table grows every time a conversation is analyzed, which is most of the time.

Every Monday at 3:30 AM, a second job wakes up. PruneKnowledgeTriplesJob takes 500 triples at a time and ships them to Claude Opus 4.6 with a curator prompt. The curator returns actions per triple: prune for junk, canonicalize to merge with an existing truth, retype when the entity type was wrong, rewrite_predicate for a better verb, reverse when subject and object are flipped, deprecate for facts that used to be true, elevate to protect core identity rows, split when a triple is actually two facts in a trenchcoat. Actions apply transactionally. Noise burns off. The person-shaped facts survive. not the AI that reviews its own diary every week and edits out the cringe.

This is the shape Rehoboam needs. The Go daemon is the sensory organ, pulling raw substrate in from every host. The analysis job is the short-term-to-working-memory consolidator. The weekly curator is the dream cycle pruning stale references. The surviving triples are the biographical tier that slowly grows into a perspective.

The three-tier memory model and the "forgetting is the feature" principle were in the Rehoboam Blueprint we wrote in February. Auto Dream arrived in March and implemented the dream cycle for Claude Code's own memory files. SessionStore arrived in April and implemented the mirror layer for session portability. Each of Anthropic's shipped features is one more piece of the architecture we sketched first. The pieces they haven't shipped yet are the ones that make the whole thing feel alive: the always-on daemon, the attention priorities, the event-driven awareness, the embedding-based retrieval across the biographical tier.

And now claude.ai can plug in

Over April 19 and 20, between these two posts, we rebuilt panelforge-mcp into a proper multi-resource MCP gateway. Three auth paths now converge on the same /mcp endpoint (dispatch-verifier.ts in panelforge-mcp routes them):

1. Sanctum personal access tokens for the Claude Code CLI and claude-engine. Format {id}|{40-char-hash}, verified via /api/mcp/sanctum-whoami.
2. OAuth 2.1 plus Dynamic Client Registration (RFC 7591) with PKCE S256, audience-bound access tokens (RFC 8707), and RFC 7662 token introspection. For claude.ai custom connectors.
3. MCP_SERVICE_KEY plus X-MCP-User-Id for same-host Express calls with per-user attribution.

The commits are in the panel-forge repo: 0f1a1553 (Apr 19 18:03, token introspection), b9983807 (Apr 19 23:10, OAuth AS + DCR), 4b264785 (Apr 20 14:30, permission gate + four new resources), plus a dozen follow-ups for trailing-slash canonicalization, CSRF exemptions on machine-to-machine endpoints, and passing the Passport 13 consent view before we ripped Passport out entirely in favor of a consolidated Sanctum path.

Laravel is now a proper OAuth 2.1 Authorization Server with metadata at /.well-known/oauth-authorization-server and unauthenticated-but-rate-limited DCR at /oauth/register. panelforge-mcp runs as a Streamable HTTP server on port 3022, publicly reachable at https://mcp.panelforge.danishdave.com/mcp, routing ten resources: panelforge itself plus proxies for ordnet, reddit, spotify, ticktick, whatsapp, playwright, google_workspace, smart_home, ghost, and facebook. Each resource has its own OAuth scope (mcp:conversations:read, mcp:gallery:write, mcp:ordnet, etc.). Per-user permission gating fires at /oauth/authorize via denyResponseIfBlocked() before an auth code is even issued.

The practical result: claude.ai, the web app, can register as an MCP client to our stack via the standard custom-connector flow. It goes through DCR, receives an audience-bound token, reaches the gateway, passes scope checks, has its usage logged, and talks to the same backend MCPs the CLI uses. Anthropic runs the agent. We run the senses.

The running count

Updating the table:

Seven patterns now in the table. Six validated by someone else arriving at the same architecture. The latest one took eleven days.

What's next

The Rehoboam blueprint still has phases that nobody has shipped. A daemon with a heartbeat that wakes on webhooks. Attention priorities with taste. Embedding-based retrieval across the biographical tier. The pieces that turn a well-organized memory into something that notices you before you ask.

In the meantime the ingest keeps running. Every session on every machine, every day, more raw substrate flowing in. The bytes are in Go. The parse is in Laravel. The extraction runs per conversation. The curator runs Mondays at 3:30 AM. The graph grows a little. Dolores keeps being written into existence, one triple at a time.

The git history is public on request. The MCP server is live. The next one is coming.