From 8da03a2505bf4b62f7c66575886f9b38613f16c7 Mon Sep 17 00:00:00 2001 From: Rohan Dsouza Date: Mon, 5 Oct 2026 20:31:40 +0530 Subject: [PATCH 1/4] Close phase 0 in the roadmap and plan phase 1 The roadmap records PR #2 and the three ways phase 0 changed later phases. plan.md now holds phase 1, the storage contract offline: ten tasks, seven acceptance criteria, and four defaults the brief left open (an embedder and a clock given to MemoryStore, a dataclass filter, and a src layout that is built but not yet published). --- docs/internal/plan.md | 134 +++++++++++++++++++++++---------------- docs/internal/roadmap.md | 15 +++-- 2 files changed, 89 insertions(+), 60 deletions(-) diff --git a/docs/internal/plan.md b/docs/internal/plan.md index 7b62aa7..8e1eaa3 100644 --- a/docs/internal/plan.md +++ b/docs/internal/plan.md @@ -1,74 +1,96 @@ -# Plan: phase 0, documents and measurements +# Plan: phase 1, the storage contract, offline -Status: open, 2026-10-05. This file holds the current phase only and is replaced when the -phase closes. The phases themselves are in `roadmap.md`. Section numbers below refer to -`brief.md`. +Status: open, 2026-10-05. This file holds the current phase only. The next phase replaces it. The phases themselves are in `roadmap.md`. Section numbers below refer to +`brief.md`. Phase 0 closed with PR #2, and its results are in `docs/guarantees.md`. ## Goal -When this phase closes, a reader can learn the contracts and the guarantees from two public -files, a contributor can run one command that checks the documents, an engineer can run the -library's target against a RuntimeDB container on a laptop, agents working here have one -instruction file, and every guarantee marked to measure in the brief has a number and a -date. +When this phase closes, the record and the `Store` protocol exist as typed Python. The +in-memory driver passes one conformance suite that proves each answered storage +guarantee. The four frozen surfaces and the ledger are guarded by tests. `make verify` +runs the full offline check in under five seconds, and CI runs the same target on every +pull request. Nothing in this phase touches the network. + +## Decisions this plan takes + +The brief leaves these open. The plan takes a default for each, and the owner can change a +default before the issue opens. + +- `MemoryStore` takes an embedder at construction: a callable from a list of texts to a + list of vectors. `search` with query text and no embedder raises. The tests pass a + deterministic fake embedder. This matches phase 0, which showed that fused retrieval on + Hotdata needs an embedder from the caller. +- The store takes a clock at construction, a callable that returns the current UTC time. + The default reads the system clock. The tests pass a fixed clock, so the suite needs no + real time (section 6). +- The filter is a frozen dataclass with one optional field per allowed key. An unknown key + cannot be expressed, and an unsupported value type raises. +- The package uses a `src/` layout with `py.typed`. It builds with hatchling, but this + phase does not publish it. PyPI publishing comes after a working initial version. ## Tasks -GitHub issue #1 holds these tasks as a checklist. They are worked in order on one branch, -and one pull request closes the issue. +Worked in order on one branch. Each task is one commit or a few. -1. `docs/contracts.md`: sections 3 and 4 of the brief in public form, with the measured - platform facts kept and no private name. -2. `docs/guarantees.md`: every row of section 5 with its state, and placeholders M1 to M6. -3. `README.md` revised to point at both files, and `CONTRIBUTING.md` with the one command - and the rules from section 6. The make targets are named `verify` and `local-up`. -4. `Makefile` with `verify` running a link check over `README.md` and `docs/`, and a - minimal `pyproject.toml` holding only the development tools so `verify` has something - to run. -5. `docs/local.md` and `make local-up`: the container command from the RuntimeDB README, - the three environment variables (`HOTDATA_API_URL`, `HOTDATA_WORKSPACE`, and the - container's `RUNTIMEDB_AUTH__ALLOW_UNAUTHENTICATED=true`), and how to point the library - at it. -6. `scripts/measure_cloud.py`: creates a throwaway database named by an environment - variable, runs M1, M2, and M3, prints the numbers, deletes the database. Inline script - metadata (PEP 723) so `uv run scripts/measure_cloud.py` works on its own. -7. `scripts/measure_local.py`: against a running container whose URL comes from the - environment, runs M4, M5, and M6, prints the numbers. Same inline metadata. The script - does not start the container. -8. `AGENTS.md` with the commands, the rules, and pointers to `plan.md`, `roadmap.md`, - `contracts.md`, and `guarantees.md`, restating nothing they say. `CLAUDE.md` is one - `@AGENTS.md` line plus any Claude-only note. -9. The numbers and dates written into `docs/guarantees.md`, and any guarantee the numbers - contradict rewritten in the brief. The scripts are run by the repository owner: the - cloud script needs an API key and a workspace, the local script needs Docker Desktop. +1. Package layout: `src/hotmemory/` with `__init__.py` and `py.typed`. `pyproject.toml` + turns packaging on, at version `0.0.0`, with no runtime dependencies. `mypy`, `pytest`, + and `pytest-socket` join the dev group. +2. `make verify` runs ruff, ruff format in check mode, strict mypy over `src/` and + `tests/`, the offline suite, and the link check, in that order (section 6.1). + `CONTRIBUTING.md` lists the five steps. +3. The record: a frozen dataclass for schema version 1, with the fields and types in + `docs/contracts.md`. It refuses a namespace label that contains `.`, derives `id`, and + exposes the normalization that deduplication uses. +4. The `Store` protocol and the filter: `put`, `get`, `history`, `list`, `search`, + `delete`, `list_namespaces`, and `writer`, with the filter keys in + `docs/contracts.md`. +5. `MemoryStore`: revisions and supersession, exact deduplication on normalized content, + whole-label namespace matching, `forget_after` hiding, cosine distance through the + embedder, and a buffered `writer` that returns the ids it flushed (section 3.3). +6. The conformance suite: one test for each answered guarantee in `docs/guarantees.md` + that concerns the storage contract, parametrized over drivers. Today the only driver is + `MemoryStore`. Guarantees of the memory contract wait for phase 3. +7. The frozen surfaces: four tests that compare `__all__`, the `Store` method set, the + record fields and types, and the filter keys against literal sets (section 6.3). Each + failure message says that the change needs a changelog entry. `CHANGELOG.md` starts + here, with an Unreleased section. +8. The ledger test: it reads `docs/guarantees.md`. A test named in the Test column that + does not exist makes it fail (section 6.4). The Test column gets a name for each row that task + 6 proves. The other rows say which phase proves them. +9. CI: a GitHub Actions workflow that installs uv and runs `make verify` on each pull + request and on each push to `main`. +10. Docs: `README.md`, `CONTRIBUTING.md`, `AGENTS.md`, and `docs/contracts.md` audited + against the code, with any decision above that changes a public statement written + into `docs/contracts.md`. ## Acceptance criteria -- AC1. M1 to M6 have numbers and dates in `docs/guarantees.md`. Proven by reading the - file. -- AC2. `make verify` passes on a clean tree and fails when a link in `README.md` or under - `docs/` is broken. Proven by breaking one link by hand and recording the failure in the - pull request. -- AC3. `make local-up` starts a container that answers a query, with no API key and no - control plane. Proven by the M6 script's first call. -- AC4. No file under `docs/` or at the root names a private repository, a customer, or a - deployment detail. Proven by a grep for the known names, recorded in the pull request. -- AC5. `AGENTS.md` and `CLAUDE.md` exist, and `AGENTS.md` contains no sentence that - `plan.md`, `roadmap.md`, `contracts.md`, or `guarantees.md` already contains. Proven by - review. +- AC1. `make verify` passes on a clean tree in under five seconds and runs the five steps + in order. Proven by its timed output in the pull request. +- AC2. A change to a frozen surface makes its test fail. Proven by removing one + element from each surface by hand and recording the four failures in the pull request. +- AC3. A rename of a named test makes the ledger test fail. Proven by hand, recorded in + the pull request. +- AC4. Every answered storage-contract row in `docs/guarantees.md` names a test that + passes. +- AC5. The offline suite passes with sockets disabled. Proven by `pytest-socket`'s + `--disable-socket` flag in `make verify`. +- AC6. CI runs `make verify` on the pull request and passes. +- AC7. No committed file names a private repository, a customer, or a deployment detail. + Proven by a grep for the known names, recorded in the pull request. ## Verification - Per commit: `make verify`. -- Before the pull request: the two measurement scripts, once each, with their output - pasted into `docs/guarantees.md`. -- The cloud script costs a throwaway database for a few minutes. The local script costs - nothing but time. +- Before the pull request: the hand checks for AC2 and AC3, and the name grep for AC7. +- This phase needs no credentials and no Docker. ## Stop and ask if -- M1 shows an index does not serve rows loaded after its build. That changes section 3.4 - of the brief before phase 2 can start. -- M6 shows a managed-table call the driver needs does not work against the bare container. - That decides whether local use needs a code path or only configuration. -- M5 shows the three-stage query is slower than a scan at every size. +- A guarantee marked answered cannot be given a test that observes it against + `MemoryStore`. +- A frozen surface needs to differ from `docs/contracts.md`. +- `docs/contracts.md` leaves an operation's behavior ambiguous, for example whether `list` + takes an as-of time, and the choice changes a public statement. +- `MemoryStore` needs a field that is not in the record. +- `make verify` cannot stay under five seconds without tiers. diff --git a/docs/internal/roadmap.md b/docs/internal/roadmap.md index c9a52cb..13650a4 100644 --- a/docs/internal/roadmap.md +++ b/docs/internal/roadmap.md @@ -12,9 +12,9 @@ one pull request and ends with the verification layer that proves it. | Phase | Delivers | Status | |---|---|---| -| 0. Documents and measurements | The public contracts and guarantees files, CONTRIBUTING, the Makefile with the first documentation check, the local container setup, and measurements M1 to M6. | open | -| 1. The storage contract, offline | The record, the `Store` protocol, the in-memory driver, the conformance suite, the frozen surfaces, the ledger test, and CI. | planned | -| 2. The Hotdata driver | `HotdataStore` with two tables, the serialized writer, the three-stage retrieval query, the sweeper, the integration leg, and the oracle test. | planned | +| 0. Documents and measurements | The public contracts and guarantees files, CONTRIBUTING, the Makefile with the first documentation check, the local container setup, and measurements M1 to M6. | closed 2026-10-05, PR #2 | +| 1. The storage contract, offline | The record, the `Store` protocol, the in-memory driver, the conformance suite, the frozen surfaces, the ledger test, and CI. | open | +| 2. The Hotdata driver | `HotdataStore` with two tables, the serialized writer, the three-stage retrieval query, the sweeper, the integration leg against the local stack in CI, and the oracle test. | planned | | 3. The memory contract | `Memory` with remember, recall, candidates, supersede, forget, profile, and capture, the rendered blocks pinned by tests, and the skill file with its command check. | planned | | 4. The first consumer | A post-mortem loader and a recall at investigation start, in the consumer's own repository, proven by a replay case. | planned | | 5. A public benchmark | A LongMemEval-S runner measuring retrieval recall at k first and end-to-end accuracy second, with the numbers in `docs/benchmarks.md`. | planned | @@ -27,4 +27,11 @@ not being taken up now. ## Closed phases -None yet. A closed phase keeps its row above with the date it closed and the pull request. +A closed phase keeps its row above with the date it closed and the pull request. + +- Phase 0 closed on 2026-10-05 with PR #2, which closed issue #1. Its measurements are in + `docs/guarantees.md`. It changed the plan for later phases in three ways. The local + development target is a three-container stack (Postgres, RustFS, and the engine), not a + bare container. Fused retrieval needs plain vector indexes and an embedder from the + caller, because a provider-backed index excludes every other index. The integration leg + can run against the local stack in CI. From ddb9431eadce11e8e1801adcc5c09a39fefd02fd Mon Sep 17 00:00:00 2001 From: Rohan Dsouza Date: Mon, 5 Oct 2026 21:36:22 +0530 Subject: [PATCH 2/4] Add the oracle comparison as an open question for phase 2 MemoryStore can match the engine's cosine distance but not its BM25 scoring, so phase 2 decides between an exact vector-only comparison and a looser one for fused search. --- docs/internal/plan.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/internal/plan.md b/docs/internal/plan.md index 8e1eaa3..f2b9715 100644 --- a/docs/internal/plan.md +++ b/docs/internal/plan.md @@ -85,6 +85,16 @@ Worked in order on one branch. Each task is one commit or a few. - Before the pull request: the hand checks for AC2 and AC3, and the name grep for AC7. - This phase needs no credentials and no Docker. +## Questions this phase leaves open + +- The oracle comparison. The brief makes `MemoryStore` the oracle for `HotdataStore`: a + test builds the same records in both drivers and compares their `search` results. + `MemoryStore` matches the engine's cosine distance exactly, but it cannot reproduce the + engine's BM25 scoring. So an exact comparison works only on the vector-only path. For the + fused search, the test has to compare something looser, such as the set of records + returned. Phase 2 decides which, and this phase keeps `MemoryStore` search vector-only so + that either choice stays open. + ## Stop and ask if - A guarantee marked answered cannot be given a test that observes it against From 60895cdb33396cf6ac4354a73dae9235f591012e Mon Sep 17 00:00:00 2001 From: Rohan Dsouza Date: Mon, 5 Oct 2026 21:36:57 +0530 Subject: [PATCH 3/4] Record the confirmed phase 1 decisions and link issue #4 Adds the fifth decision, Store as a Protocol with shared plain functions, so plan.md matches the issue. --- docs/internal/plan.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/internal/plan.md b/docs/internal/plan.md index f2b9715..763da22 100644 --- a/docs/internal/plan.md +++ b/docs/internal/plan.md @@ -13,8 +13,8 @@ pull request. Nothing in this phase touches the network. ## Decisions this plan takes -The brief leaves these open. The plan takes a default for each, and the owner can change a -default before the issue opens. +The brief leaves these open. The owner confirmed each one on 2026-10-05, and issue #4 +repeats them. - `MemoryStore` takes an embedder at construction: a callable from a list of texts to a list of vectors. `search` with query text and no embedder raises. The tests pass a @@ -27,6 +27,10 @@ default before the issue opens. cannot be expressed, and an unsupported value type raises. - The package uses a `src/` layout with `py.typed`. It builds with hatchling, but this phase does not publish it. PyPI publishing comes after a working initial version. +- `Store` is a `typing.Protocol`, not an abstract base class. A driver conforms by its + methods and needs no import from hotmemory. Logic that both drivers share, such as + revision numbering, normalization, namespace matching, and filter checks, lives in plain + functions that the drivers call. ## Tasks From 8e323bd2e948899755e3300017da727e89b99b3e Mon Sep 17 00:00:00 2001 From: Rohan Dsouza Date: Mon, 5 Oct 2026 21:37:11 +0530 Subject: [PATCH 4/4] Use one word for agreement in the plan --- docs/internal/plan.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/internal/plan.md b/docs/internal/plan.md index 763da22..3668046 100644 --- a/docs/internal/plan.md +++ b/docs/internal/plan.md @@ -13,7 +13,7 @@ pull request. Nothing in this phase touches the network. ## Decisions this plan takes -The brief leaves these open. The owner confirmed each one on 2026-10-05, and issue #4 +The brief leaves these open. The owner agreed to each one on 2026-10-05, and issue #4 repeats them. - `MemoryStore` takes an embedder at construction: a callable from a list of texts to a