Phase 1: the storage contract, offline

.github/workflows/verify.yml

@@ -0,0 +1,20 @@
+name: verify
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+permissions:
+  contents: read
+
+jobs:
+  verify:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v5
+      - uses: astral-sh/setup-uv@v6
+        with:
+          python-version: "3.11"
+      - run: make verify

AGENTS.md

@@ -24,6 +24,8 @@ uv run --env-file .env scripts/measure_local.py --cloud   # M4 to M6 in the clou
 - [docs/guarantees.md](docs/guarantees.md): what a consumer can rely on, and the proof.
 - [docs/internal/brief.md](docs/internal/brief.md): the design and its reasons.
 - [CONTRIBUTING.md](CONTRIBUTING.md): the checks inside `make verify` and the test rules.
+- [CHANGELOG.md](CHANGELOG.md): each change to a public surface.
+- `src/hotmemory/`: the library. `tests/test_conformance.py` is the conformance suite.
 
 ## Rules
 
@@ -37,3 +39,4 @@ uv run --env-file .env scripts/measure_local.py --cloud   # M4 to M6 in the clou
   workspace id, or a database id in a file.
 - If a change alters a behavior, update `docs/contracts.md` and `docs/guarantees.md` in the
   same commit.
+- If a change alters a frozen surface, add an entry to `CHANGELOG.md` in the same commit.

CHANGELOG.md

@@ -0,0 +1,19 @@
+# Changelog
+
+Each change to a public surface of hotmemory has an entry here. The frozen surfaces are the
+names in `hotmemory.__all__`, the method set of `Store`, the fields and field types of
+`Record` for each schema version, and the filter keys of `list` and `search`. A test guards
+each one, and its failure message points to this file.
+
+## Unreleased
+
+### Added
+
+- `Record`, the frozen dataclass for schema version 1, and `normalize`, the form that
+  deduplication compares.
+- `Store`, the protocol of the storage contract, with `put`, `get`, `history`, `list`,
+  `search`, `delete`, `list_namespaces`, and `writer`. `Writer` is the protocol of the
+  buffer that `writer` returns.
+- `Filter` and `TimeRange`, the exact filters of `list` and `search`, and `Hit`, one search
+  result with its distance.
+- `MemoryStore` and `MemoryWriter`, the in-process driver.

CONTRIBUTING.md

@@ -17,16 +17,16 @@ make verify
 `make verify` is the full check. CI runs the same target. If it passes on your machine, it
 passes in CI. It has no tiers, because the full check takes less than five seconds.
 
-Today, `make verify` runs these checks, in this order:
+`make verify` runs these checks, in this order:
 
 1. `ruff check` over the Python files.
 2. `ruff format --check` over the Python files and the Python code blocks in Markdown.
-3. The link check. It reads each relative link in the Markdown files at the root and
+3. Strict `mypy` over `src/` and `tests/`.
+4. The offline test suite, with `pytest --disable-socket`. A test that opens a network
+   socket fails.
+5. The link check. It reads each relative link in the Markdown files at the root and
    under `docs/`. A link to a file that does not exist makes it fail.
 
-When the library exists, strict `mypy` and the offline test suite join `make verify`
-before the link check.
-
 ## The other targets
 
 | Target | What it does |
@@ -38,37 +38,43 @@ before the link check.
 
 ## Rules of the harness
 
-The library is deterministic. The offline suite needs no clock, no network, and no model.
-These rules keep it that way.
-
-- One conformance suite runs against every driver. Each answered guarantee in
-  [docs/guarantees.md](docs/guarantees.md) is one test. A driver that fails a conformance
-  test is not a driver.
-- The in-memory driver is the reference for the Hotdata driver. A test builds the same
-  records in both drivers and compares the results of `search`.
+The library is deterministic. The offline suite needs no real clock, no network, and no
+model. The tests pass a fixed clock and a fake embedder to the store, from
+`tests/conftest.py`. These rules keep the suite deterministic.
+
+- One conformance suite, `tests/test_conformance.py`, runs against every driver through
+  the `store` fixture. Each answered guarantee of the storage contract in
+  [docs/guarantees.md](docs/guarantees.md) has its tests there. A driver that fails a
+  conformance test is not a driver. To add a driver, add it to `DRIVERS` in
+  `tests/conftest.py`.
+- The in-memory driver is the reference for the Hotdata driver. From phase 2, a test
+  builds the same records in both drivers and compares the results of `search`.
 - Four surfaces are frozen: the names in `__all__`, the method set of the `Store`
   protocol, the fields and field types of the record for each schema version, and the
   filter keys of `list` and `search`. A test compares each surface against a literal set.
-  A change to a frozen surface is a public contract change, and it needs a changelog
-  entry.
+  These tests are in `tests/test_frozen.py`. A change to a frozen surface is a public
+  contract change, and it needs an entry in [CHANGELOG.md](CHANGELOG.md).
 - A frozen surface is compared against a literal set, never against the thing that it
   protects. A test that iterates over the protected thing turns a deletion into one test
   fewer and not into a failure.
-- Each row in the guarantees ledger names the test that proves it. A test reads the ledger.
-  A named test that does not exist makes it fail.
-- Tests marked `hotdata` run the Hotdata driver against a real database. They need
-  `HOTMEMORY_TEST_DB` to name a throwaway database. Without it, they skip. They
+- Each row in the guarantees ledger names the conformance tests that prove it, or the phase
+  that will prove it. `tests/test_ledger.py` reads the ledger. A named test that the
+  conformance suite does not define makes it fail. A row with no test and no phase also
+  makes it fail.
+- From phase 2, tests marked `hotdata` run the Hotdata driver against a real database.
+  They need `HOTMEMORY_TEST_DB` to name a throwaway database. Without it, they skip. They
   run once for each pull request. They are the only tests that use the network.
-- No test calls a model. `capture` takes a callable, and the tests pass a fake extractor
-  that returns fixed facts.
+- No test calls a model. The tests pass a fake embedder. From phase 3, `capture` takes a
+  callable, and the tests pass a fake extractor that returns fixed facts.
 - There is no coverage gate, no mutation-testing gate, and no report generator. The output
   of `make verify` is the report.
 
 ## Documents
 
 Documents are checked like code. If you change a behavior, update
 [docs/contracts.md](docs/contracts.md) and [docs/guarantees.md](docs/guarantees.md) in the
-same pull request.
+same pull request. If you change a public surface, add an entry to
+[CHANGELOG.md](CHANGELOG.md).
 
 No file in this repository names a private repository, a customer, or a deployment detail.
 This rule includes `docs/internal/`.

Makefile

@@ -8,6 +8,8 @@ STORAGE_URL := http://127.0.0.1:9000
 verify:
 	uv run --group dev ruff check .
 	uv run --group dev ruff format --check .
+	uv run --group dev mypy
+	uv run --group dev pytest --disable-socket -q
 	uv run --no-project python scripts/check_links.py
 
 local-up:

README.md

@@ -8,8 +8,10 @@ Hotdata table, so an agent can join its memory to its own data in one SQL query.
 the memory systems that we surveyed stores memory as typed columns in the same engine as
 the data of the consumer.
 
-Status: design. No code exists yet. The contracts and the guarantees are written, and
-phase 0 measured each guarantee that was open.
+Status: version 0.0.0, not published. The storage contract exists in Python with one
+driver, `MemoryStore`, which runs in process memory. The Hotdata driver and the memory
+contract do not exist yet. [docs/internal/roadmap.md](docs/internal/roadmap.md) lists the
+phases.
 
 The library has two layers:
 
@@ -24,6 +26,31 @@ of the consumer. There is no hotmemory server. The first consumer is an incident
 investigator built on Hotdata. The library is not specific to it. A plain Python agent, a
 LangGraph agent, or any process with a Hotdata API key can use it.
 
+## Try the storage contract
+
+`MemoryStore` needs an embedder for a search with query text. An embedder is a callable
+that turns a list of texts into a list of vectors. This example uses a toy embedder that
+counts two words:
+
+```python
+from hotmemory import Filter, MemoryStore
+
+
+def embed(texts):
+    return [[text.count("disk") + 0.1, text.count("cpu") + 0.1] for text in texts]
+
+
+store = MemoryStore(embedder=embed)
+store.put(("team", "alerts"), "disk", kind="fact", content="The disk fills at night.")
+store.put(("team", "alerts"), "disk", kind="fact", content="The disk fills at noon.")
+store.put(("team", "alerts"), "cpu", kind="fact", content="The cpu spikes after a deploy.")
+
+print(store.get(("team", "alerts"), "disk").id)  # team/alerts/disk@2
+print([r.revision for r in store.history(("team", "alerts"), "disk")])  # [1, 2]
+hits = store.search("disk", [("team",)], Filter(kind="fact"), k=1)
+print(hits[0].record.content)  # The disk fills at noon.
+```
+
 ## Documents
 
 - [docs/contracts.md](docs/contracts.md): the record, the storage operations, the memory
@@ -34,5 +61,6 @@ LangGraph agent, or any process with a Hotdata API key can use it.
   container.
 - [CONTRIBUTING.md](CONTRIBUTING.md): the one command that checks a change, and the rules
   of the test suite.
+- [CHANGELOG.md](CHANGELOG.md): each change to a public surface.
 - [docs/internal/](docs/internal/brief.md): the design brief, the survey, the roadmap, and
   the plan for the current phase.

docs/contracts.md

@@ -5,8 +5,9 @@ implements. The memory contract is the surface that an agent calls, and it is bu
 store. This file states both. The behavior that each contract guarantees, and the proof for
 each guarantee, are in [guarantees.md](guarantees.md).
 
-Status: design. No code exists yet. This file describes schema version 1 as the library
-will ship it.
+Status: the storage contract exists in Python, with `MemoryStore` as its only driver.
+`HotdataStore` and the memory contract are design, and this file describes them as the
+library will ship them. This file describes schema version 1.
 
 ## The platform under the store
 
@@ -76,16 +77,16 @@ A record is the unit that the store holds. Schema version 1 fixes these fields.
 
 | Field | Type | Meaning |
 |---|---|---|
-| `namespace` | tuple of strings | Where the record lives. The store keeps it as one path string joined with `/`. A match is on whole labels and never on a string prefix. No label contains `.`. |
-| `key` | string | The stable identifier of the caller inside the namespace. |
+| `namespace` | tuple of strings | Where the record lives. The store keeps it as one path string joined with `/`. A match is on whole labels and never on a string prefix. A namespace has at least one label. A label is not empty and contains no `.` and no `/`. |
+| `key` | string | The stable identifier of the caller inside the namespace. A key is not empty and contains no `/` and no `@`. |
 | `revision` | integer | 1 for the first put under a key. Each later put adds 1. |
 | `kind` | string | One of `fact`, `profile`, `procedure`, `episode`. |
 | `subject` | string | What the record is about, for example an alert key, a person, or a service. Empty if unknown. |
-| `content` | string | The text that a model reads. |
-| `cues` | list of strings | Questions or phrases that this record answers. Optional. The driver embeds them apart from `content`. |
+| `content` | string | The text that a model reads. It is not empty after normalization. |
+| `cues` | tuple of strings | Questions or phrases that this record answers. Optional. The driver embeds them apart from `content`. |
 | `payload` | JSON object | Structured data that the consumer defines. The store never reads it. |
-| `tags` | list of strings | Free labels. You can filter on them. |
-| `sources` | list of strings | References to the origin of the record: a thread id, a document path, a run id, an episode key. The length of the list is the corroboration count. |
+| `tags` | tuple of strings | Free labels. You can filter on them. |
+| `sources` | tuple of strings | References to the origin of the record: a thread id, a document path, a run id, an episode key. The length of the list is the corroboration count. |
 | `actor` | string | Who wrote this revision: a user id, an agent name, or an extractor name. |
 | `created_at` | timestamp | When the store wrote this revision. System clock. |
 | `observed_at` | timestamp or null | The time of the source. A post-mortem that you load a year later keeps the incident date here. |
@@ -97,6 +98,12 @@ A record is the unit that the store holds. Schema version 1 fixes these fields.
 | `forget_reason` | string | The reason for `forget_after`. Empty if `forget_after` is null. |
 | `id` | string | `namespace/key@revision`. Derived. The load key. |
 
+In Python, the record is a frozen dataclass. The list fields are tuples, so a record
+cannot change after the store writes it. The store copies `payload` when it writes a
+record and when it returns one, so a change to a dict that a caller holds never reaches the
+store. Every timestamp carries a time zone. The record
+refuses a value that the table above does not allow.
+
 The public record has no embedding field. If the caller supplies an embedder, the Hotdata
 driver adds embedding columns. If the caller uses a provider-backed index, the driver adds
 none. The driver configuration selects one of the two.
@@ -110,18 +117,23 @@ records the moment that the store found out. To ask what memory held on a given
 
 | Operation | Arguments | Behavior |
 |---|---|---|
-| `put` | namespace, key, record fields | Writes a new revision. If the key exists, the new row gets the next revision, and the previous current row gets `superseded_by`. Returns the id. If the normalized content is equal to the content of the current revision, it writes nothing and returns the current id. |
+| `put` | namespace, key, record fields | Writes a new revision. If the key exists, the new row gets the next revision, and the previous current row gets `superseded_by`. Returns the id. If the normalized content is equal to the content of the current revision, it writes nothing and returns the current id. This rule compares content only. It does not apply when the current revision is past its `forget_after`, so a `put` of the same content brings the fact back as a new revision. |
 | `get` | namespace, key, optional revision | Returns the current revision, or the named revision. Returns None if the record does not exist. |
 | `history` | namespace, key | Returns every revision, oldest first. |
-| `list` | namespace prefix, optional filter, optional since, limit | Returns current revisions under the prefix, newest first. It uses no model and no embedding. |
-| `search` | query text or none, namespace prefixes, optional filter, k | Returns up to k current revisions in order of relevance, closest first, each with a distance. It takes the same filter as `list`. With no query text, it is `list`. |
+| `list` | namespace prefix, optional filter, optional since, limit | Returns current revisions under the prefix, newest first, with ties in `created_at` ordered by id. `since` keeps the revisions whose `created_at` is at or after it. It uses no model and no embedding. |
+| `search` | query text or none, namespace prefixes, optional filter, k | Returns up to k current revisions in order of relevance, closest first, each with a distance. It takes the same filter as `list`. With no query text, it is `list`, and each distance is None. |
 | `delete` | namespace, key | Removes every revision of the key. This is a hard delete. |
-| `list_namespaces` | optional prefix | Returns the distinct namespaces under the prefix. |
-| `writer` | none | A context manager. It buffers every `put` inside it. The buffer flushes on exit, at a row count, or at an interval, and returns the ids that it flushed. |
+| `list_namespaces` | optional prefix | Returns the distinct namespaces under the prefix that hold a record, sorted. |
+| `writer` | optional row count, optional interval | A context manager. It buffers every `put` inside it. The buffer flushes when the block exits, when it reaches the row count, and on the first `put` after the interval passes. The writer records the ids that it flushed. If the block raises an error, the writer drops the buffer. |
 
 The filter accepts equality on `kind`, `subject`, `tags`, and `actor`. It accepts a range on
 `valid_from`, `valid_until`, `created_at`, and `expired_at`. Any other filter raises an
-error.
+error. In Python, the filter is a frozen dataclass with one optional field for each key, so
+an unknown key cannot be written, and a value of the wrong type raises an error.
+
+- A `tags` filter matches a record that holds every tag that the filter names.
+- A range includes its start and excludes its end. A side that is not given is open.
+- A null timestamp on a record never matches a range. This is the SQL rule for null.
 
 `list` and `search` never return a deleted revision, a superseded revision, or a record
 past `forget_after`. `history` returns superseded revisions. No operation returns a deleted
@@ -139,7 +151,11 @@ Version 1 ships two drivers.
 - `MemoryStore` runs in the process, in memory. It is a real driver and not a mock. It
   computes relevance with the same cosine distance that the engine uses, and it refuses a
   filter that it does not model. The offline test suite runs against it, and it is the
-  reference for the other driver.
+  reference for the other driver. It takes two optional arguments. The embedder is a
+  callable that turns a list of texts into a list of vectors. Without it, a `search` with
+  query text raises an error. The clock is a callable that returns the current time, and
+  the default reads the system clock. `MemoryStore` ranks by the cosine distance between
+  the query and `content` only. It does not rank by BM25 or by `cues`.
 - `HotdataStore` uses one managed database, two tables per schema version, keyed loads, a
   serialized writer, and the retrieval query below. With the local RuntimeDB stack, it is also
   the development driver.