Repository navigation
Keep memory in Loom, through the MCP gateway - #546
Merged
Merged
Conversation
MemoryBackendUnavailable is part of the memory contract, not of memU. It moves to nerve.memory.backend, so a module can name it without loading memU's numpy stack; memu_bridge re-exports it.
memory.backend picks memu (the default) or loom; an unknown name refuses to load. memory.loom holds the optional org, namespace and dashboard_url, and is checked only while Loom is the backend. Nothing comes from the environment: Nerve reaches Loom through the MCP gateway it already knows.
McpGatewayServer sends one tools/call per request to a catalog server's URL, as the gateway serves it (stateless streamable HTTP, protocol 2025-06-18), and sorts what can come back: a tool result, a tool that refused, a JSON-RPC error the gateway or server answered (-32001 is a policy decision, -32002 and -32000 may pass), or no answer at all.
With memory.backend: loom the engine builds LoomMemory, which reaches Loom as the 'loom' server of the gateway's catalog and makes only the lifecycle calls: the startup recall (context.pack in startup mode, loom_search when the pack brings no memories), transcript extraction (memory.set_from_messages, deferred; the audit log names each queued batch) and a health probe (namespace.summary, which also names the org and namespace Loom resolved for the agent). Agents use Loom's own tools from the catalog, so Nerve's nine memory tools are not offered then. Gateway and Loom failures degrade instead of crashing: a session that could not recall is told memory is unavailable, nothing is frozen, and the sweep keeps its watermark.
scripts/loom_contract_fixture.py reads a Loom checkout (the generated SDK's op schemas and the MCP toolsets) and writes what Nerve uses of them to tests/fixtures/loom_lifecycle_ops.json. The contract tests hold the backend to it: each op is called through the tool that serves it on MCP, and every argument on the wire and every result field read is one Loom declares.
Every place that names Nerve's memory tools gets a Loom branch, worded in nerve/memory/loom_prompts.py: a memory section in the system prompt (Loom's tools, home and shared namespaces, proposing instead of editing, write verdicts and abstentions), the worker onboarding step, the Codex backend note, the bundles written for external agents, the workspace templates and bundled skills a Loom instance is set up with, and the memory-maintenance and inbox-processor jobs. Under memU each stays byte for byte as it was. The contract fixture also records the tools this wording names; the generator refuses one Loom does not list by default. An instance that is not in external mode gets no gateway catalog, so its agents have no Loom tools: the engine says so at startup.
With memory.backend: loom there is no memU store: the Memory page says memory is kept in Loom and links Loom's dashboard when memory.loom.dashboard_url is set, the memU edit routes answer 409, and the Diagnostics card shows the gateway, the namespace, where it came from and the last error. The setup Memory step has nothing to ask (no OpenAI key) and nerve doctor names Loom and the gateway. A link is only built from an absolute http(s) address.
config.md gets memory.backend and the memory.loom keys, what agents and Nerve each call, how identity is resolved and what happens when Loom cannot answer; memory.md a Loom section; the guides, cron, setup, web UI, external MCP, Codex sync and architecture pages say what changes under Loom.
The recalled memories go into the session's <session-context> block (and the system prompt in the legacy shape), and their text comes from whoever writes to the namespaces the agent reads. The startup recall undid Loom's escaping of < and > and kept an item's line breaks, so a stored fact could close the block and start lines of its own. Pack items now keep Loom's escaping, the fallback search's text is escaped the same way, and each item is one line.
Queued extraction could lose a window: the sweep moved its watermark once Loom queued a batch, and Loom gives a failed batch three tries within about a minute, out of Nerve's sight. The call now waits while Loom extracts (batches of up to 100 messages in 128 KiB, 130 s), so a failure keeps the watermark; memory.loom.defer_extraction keeps the queued mode. A batch the gateway refuses as too large halves the batch budget and goes again. The startup read has 5 s. A gateway not reached, or a read not answered in time, opens a breaker that doubles while failures go on (the gateway's own circuit does not count calls Nerve gave up on). The catalog is read at most every 30 s, also while none could be read; a read waits out a Retry-After of up to 2 s once; a pack the gateway's policy refuses falls back to the search. One server object at a time, a retired one closed at shutdown. The unused identity helper goes, memory.loom.org and namespace are checked against Loom's naming, and the contract fixture is regenerated against Loom's current op schemas. The contract now covers the written list the extraction counts, and the fixture keeps only what Nerve uses: the values it sends of a fixed set, and the fields of an answer it reads.
The system prompt named Loom's tools whenever Loom was the backend, also for a session whose client got no loom server (local mode, or a catalog that does not list loom yet). Such a session is now told in one section that memory cannot be used. A session whose Loom backend did not start at all (no gateway URL) is told memory is unavailable, as for an outage. The section also says that facts extracted from conversations are found with loom_search. memU's system prompt is checked against a frozen copy of main's rendering, and the claims that memU's texts are unchanged now name the two it sees that did change.
A platform runs its first nerve init before it switches the instance to Loom (memory.backend: loom in config.local.yaml), so the files that init writes were worded for memU. nerve init --memory-backend memu|loom words the templates, bundled skills and built-in jobs for the backend it names, in the slim, terminal and non-interactive paths; it does not set memory.backend. Without it, init follows what an existing configuration names. A system.yaml written for memU keeps its jobs when the backend changes: memory-maintenance and inbox-processor whose prompts are still the shipped memU ones run in their Loom wording, so memU's maintenance prompt does not prune memories through Loom's tools.
config.md: memory.loom.defer_extraction, the name rules, the rollout order (switch once the catalog lists loom; nerve init --memory-backend for a platform that switches after its first init), waiting extraction, the escaping of recalled text, and the timeouts, breaker, Retry-After and 413 handling. memory.md, cron.md and setup.md follow.
The note names Loom's memory tools only for a client that gets the loom server from the gateway's catalog, like the system prompt's memory section; any other client's note names no memory tools at all.
A TODO where the queued mode is chosen names the caller-bound status and retry ops a Loom change adds, and how the queued mode can use them.
…ction A policy refusal (-32001) retired the HTTP client for Loom and made a new one, and retired clients were closed only at shutdown: a gateway that answers "no credential set" until the agent's credential exists leaked a client and its socket on every health probe, session start and sweep. The client is now kept across refusals; the catalog is read again, and the client is replaced only when Loom's URL in it changed, the old one closed as soon as no call uses it. An extraction waits a little past the gateway's own call limit, so no answer by then means the gateway itself is stuck: it now opens the breaker like an unanswered read, instead of making every session of the sweep wait 130 s again while the memorize lock is held.
…s failing
When the gateway gives up on an extraction after its call limit ("MCP server
timed out"), Loom may still finish the batch, and the same batch would run
into the limit again: the batch budget now halves, messages and bytes, as
for a body too large, and the window goes again with the next sweep.
A batch Loom keeps failing on its side (a fault, -32000, it does not name as
passing, three sweeps in a row) held the rest of the window back forever and
paid for a model call each sweep. It is now halved while the halves fail,
down to the one message Loom fails on, which is skipped with an audit row
(its role and time, never its text) once Loom has extracted something else
since, so a fault on every batch, an outage, skips nothing. The context
turns shrink with the batch budget, so a budget at its minimum still fits.
An old install with no system.yaml has its built-in jobs in jobs.yaml; under Loom, a memory-maintenance or inbox-processor job there that still carries the shipped memU prompt now runs in its Loom wording too.
Lockdown reads no machine-local file, so a locked instance's tracked
settings name the switch as ${VAR:-default} references the service's
environment fills (memory.backend, mcp_gateway_url,
memory.loom.dashboard_url); a test loads such settings with and without the
variables. config.md also says how extraction depends on the gateway's call
limit, what happens to a batch Loom keeps failing, and that one client is
kept for Loom's URL.
The isolation of a failing batch could skip good messages, a whole window even, while Loom failed for everyone: it took any extraction since the batch's first failure, sweeps ago perhaps, as proof Loom worked, kept isolating at once after a skip, passed the skipped message on as context, and counted coded faults (BACKEND_UNAVAILABLE, GRANT_BUSY) as the batch's. Now only Loom's uncoded internal error counts against a batch; every coded fault is "not now". The one message found is left out of the next batch and of its context, and recorded as skipped only when that next batch goes through (at the end of the window: when a batch went through since the isolation began in this sweep); a failure first raises and skips nothing. After a skip the isolation is over. A window offered again sends only the messages Loom has not taken yet, whatever its batches look like then, so a re-sweep no longer extracts accepted messages again one by one. A budget a timeout cut grows back toward the default after ten batches in a row go through (a cut for a body too large stays), and Nerve's own timeout cuts it like the gateway's.
config.md: only Loom's uncoded internal error counts against a batch, the message found is skipped only once the next batch goes through and is never context, a window offered again sends only what Loom has not taken, and a budget a timeout cut grows back.
|
Nerve PR review · ✅ Ready to merge Adds Loom as an opt-in memory backend ( Completeness: memU tools are hidden on all three serving paths, and both backends are covered. The contract test checks Loom's op schemas. Docs, setup, doctor and Diagnostics are updated. No migration. Worth considering:
Checked: full suite in an offline sandbox at |
alex-clickhouse
approved these changes
Oct 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces #535. With
memory.backend: loom, long-term memory is kept in a Loom memory service instead of memU, and everything that talks to Loom goes through the MCP gateway that external mode already uses: the gateway's catalog lists Loom as the serverloom, knows the agent from its connection and holds its Loom credential. Nerve gets no Loom URL or token of its own;memory.backend: loomplus the existingmcp_gateway_urlis the whole configuration.memU stays the default. Its tools, its system prompt (checked against a frozen copy of main's rendering), its templates and its jobs render as before. Two texts memU sessions also see did change: the worker guide, which the worker onboarding prompt carries in full, has a bullet on Loom, and the bundled
nerve-devskill's source map names the Loom backend.Rollout: switch an instance to
memory.backend: loomonly once the gateway's catalog listsloomfor its agent; until then the agent has no memory tools at all, and its system prompt says so. A platform that switches the instance after its firstnerve initwritesmemory.backend: loomintoconfig.local.yaml(which outranks the tracked settings) and passesnerve init --memory-backend loom, so the files written at that init are worded for Loom. A locked instance reads no machine-local file: its trackedsettings.yamlnames the switch asmemory.backend: ${NERVE_MEMORY_BACKEND:-memu},mcp_gateway_url: ${NERVE_MCP_BASE_URL:-}andmemory.loom.dashboard_url: ${LOOM_DASHBOARD_URL:-}, always with a default (a bare${VAR}that is unset refuses to load, which would breaknervecommands run from a shell), and the platform provides the variables in the Nerve service's environment (a systemdEnvironmentFile=on the unit); empty values mean memU, no gateway and no link.What the agent gets
Loom's own tools, the same way it gets any catalog server:
mcp__loom__loom_search,mcp__loom__loom_memory_set, … (whatever the gateway's tool policy allows, by default Loom'scorelist). Nerve's nine memU-shaped memory tools (memorize,memory_recall,memory_update,memory_delete,conversation_history,memory_records_by_date,memory_expand_category,category_update,session_context) are not offered under Loom: not in the prompt's tool list, not on Nerve's MCP server, not to external MCP clients.Every place that named those tools has a Loom branch (
nerve/memory/loom_prompts.py):proposed: truewhere it is a writer, or tells the human), Loom's write verdicts (clear,merge,duplicate,conflict,refines,rejected/PINNED_CONFLICT), searches that abstain, and that facts extracted from conversations are found withloom_search. It is given only to a session that gets theloomserver; any other session is told memory cannot be used;loomserver), the bundles written for external agents;AGENTS.md,TOOLS.md,MEMORY.md), the bundlednerve-devskill, and thememory-maintenanceandinbox-processorjobs of a workspace set up while Loom is the backend or bynerve init --memory-backend loom(the maintenance job becomes Loom's integrity report,loom_namespace_validate, on the home namespace). Asystem.yamlwritten for memU runs those two jobs in their Loom wording as long as their prompts are the shipped ones, so memU's maintenance prompt never prunes through Loom's tools.The section depends on the configuration and the catalog only, so prompt caching is unaffected.
What Nerve calls itself
Only lifecycle calls, as the same agent, with
tools/callon the catalog URL ofloom(stateless streamable HTTP, no credential). A small tool-call client (McpGatewayServer) joins the existing catalog reader innerve/mcp_gateway.py.context.pack(startup mode, memories section only)loom_context_packsearch.queryloom_searchmemory.set_from_messages(defer: false: the call waits while Loom extracts; batches of ≤100 messages within 128 KiB; up to 130 s)loom_memory_set_from_messagesnamespace.summaryloom_namespace_summaryWaiting for the extraction is what keeps a conversation safe: a batch Loom could not extract fails the call and the sweep keeps its watermark.
memory.loom.defer_extraction: truequeues the batches instead (≤400 messages within 768 KiB); Loom then tries a failed batch three times within about a minute and gives up, and how a batch ended is not on MCP, so it stays off until Loom serves the extraction status over MCP and Nerve follows up on it.Loom's MCP has no
whoami;namespace.summarynames the org and namespace Loom resolved for the agent (memory.loom.org/memory.loom.namespaceoverride it; both are checked against Loom's naming). The startup recall is frozen like memU's. Recalled text is data from whoever writes to the namespaces the agent reads, and it lands in the session's<session-context>block: each memory is one line and keeps</>escaped as Loom escapes them, and the fallback search's text is escaped the same way.When Loom cannot answer
No gateway URL, no
loomserver in the catalog, a gateway policy refusal (-32001), a condition that may pass (-32002: unavailable, limits, open circuit), a server fault, a timeout or 429/5xx: memory is unavailable and nothing crashes. A new session's preamble says memories could not be recalled (not that there are none), also when the backend did not start; the sweep keeps its watermark. Reads are retried once and wait out aRetry-Afterof up to 2 s (the gateway's per-agent limit sends 1); a longer one holds the op. When the gateway is not reached, or a call gets no answer in time (a read gives up long before the gateway's call limit, extraction a little after it), a breaker opens for 15 s, doubling while failures go on, up to 5 min: the gateway's own circuit never counts calls Nerve gave up on. The catalog is read at most every 30 s, also while none could be read; refusals keep the one HTTP client for Loom's URL, replaced only when the catalog names another. A batch the gateway refuses as too large (413), or gives up on after its call limit (MCP_GATEWAY_CALL_TIMEOUT, "MCP server timed out"), halves the batch budget, down to 10 messages in 16 KiB, with the context turns shrinking too. A coded Loom fault (its database unreachable, a grant busy, an overload) is "not now". Only Loom's uncoded internal error can be about the batch itself: a batch failing with it three sweeps in a row is halved until the one message it fails on is found; that message is left out of the next batch and its context, and skipped (audit row with role and time, never text) only when that next batch goes through, so a failure first, or a Loom failing every batch, skips nothing. A budget a timeout cut grows back after ten batches in a row go through; Nerve's own timeout cuts it too. A window offered again sends only the messages Loom has not taken yet. The Diagnostics card shows the last error.UI, setup, doctor
memory.loom.dashboard_urlis set (only absolute http(s) links are rendered); the memU edit routes answer 409.nerve init --memory-backend memu|loomwords the files the init writes (slim,--terminaland--non-interactive); without it, init follows what an existing configuration names.nerve doctornames Loom and the gateway, or says the gateway URL is missing.Tests
tests/test_loom_memory.py: the backend against a fake gateway (GET /catalog+POST /s/loom/mcp): startup pack and its fallback, a stored fact that tries to close<session-context>, extraction batching, waiting and queueing, body limits and 413, identity, health, every degraded mode (breaker backoff, catalog throttle, Retry-After, policy refusals), the engine's preamble note, tool guidance per catalog and sweep watermark, routes and doctor.tests/test_mcp_gateway_tools.py: the tool-call client (JSON and event-stream answers, JSON-RPC and HTTP errors, retryability, read timeouts vs. unreachable, no credentials).tests/test_loom_contract.py: every argument on the wire and every result field read is checked againsttests/fixtures/loom_lifecycle_ops.json, generated from a Loom checkout byscripts/loom_contract_fixture.py(Loom's generated SDK op schemas plus its MCP toolsets), never typed by hand. The generator also refuses an op Loom keeps off MCP and any tool the agent-facing wording names that Loom does not list by default.tests/test_loom_prompts.py: every Loom branch, memU's system prompt against main's frozen rendering, shipped memU jobs reworded under Loom.tests/test_cli_init_slim.py:--memory-backendin the slim, terminal and non-interactive paths.Known gaps
memory.extraction_statusis kept off MCP (it is refused even throughloom_op), so the queued mode cannot be followed up; it stays behindmemory.loom.defer_extraction(off). A Loom change adds two caller-bound ops reachable throughloom_op,memory.extraction_status_mineandmemory.extraction_retry_mine({"queue_ids": [...], "namespace"?}); once that release exists, the queued mode can move the sweep's watermark only afterstatus_minereports a batch written (done), and replay a failed one once withretry_mine(a TODO innerve/memory/loom.pysays so).AGENTS.md,TOOLS.md,MEMORY.mdand thenerve-devskill are written once, at setup, worded for Loom's tools whatever the catalog lists then or later. A session without theloomserver is told by its system prompt (and the Codex note) that memory cannot be used, which is what it acts on.Retry-After. Only reads wait it out; an extraction refused by the gateway's per-agent limit fails, and the sweep offers the window again on its next run.pendingwrite verdict in the prompt. Loom's write verdicts areclear,merge,duplicate,conflict,refinesandrejected;pendingis not one of them, so the system prompt does not name it.