Repository navigation
Phase 1: the storage contract, offline - #5
Conversation
src/hotmemory holds an empty package and py.typed. pyproject.toml builds it with hatchling at version 0.0.0, with no runtime dependencies. mypy, pytest, and pytest-socket join the dev group. Refs #4
make verify now runs five steps in order: ruff, ruff format in check mode, strict mypy over src/ and tests/, pytest with sockets disabled, and the link check. CONTRIBUTING.md lists the five steps. Refs #4
Record is a frozen dataclass with the fields in docs/contracts.md. It derives id as namespace/key@revision and refuses a value the contract does not allow. A namespace label cannot be empty or contain . or /, and a key cannot be empty or contain / or @, so the stored path and the id stay unambiguous. cues, tags, and sources are tuples. normalize() gives the form that deduplication compares. contracts.md states the new refusals and the tuple types. Refs #4
Store is a typing.Protocol with put, get, history, list, search, delete, list_namespaces, and writer. Writer is the protocol of the buffer that writer() returns. Filter is a frozen dataclass with one optional field per allowed key, and TimeRange is a half-open range in which a null timestamp never matches. Hit pairs a record with its distance. contracts.md states how the filter matches tags and ranges, the order of list and its since argument, the distance when search has no query text, and the arguments and flush rules of writer. Refs #4
MemoryStore implements Store: revisions and supersession, exact deduplication on normalized content, whole-label namespace matching, forget_after hiding, cosine distance on content through the embedder, and a buffered writer that records the ids it flushed. It takes an embedder and a clock. The rules that the Hotdata driver will share (prefix matching, revision numbering, deduplication, visibility, filter matching, list order, cosine distance) are plain functions in hotmemory._rules. tests/conftest.py adds a fixed clock, a deterministic fake embedder, and a store fixture parametrized over drivers. Refs #4
tests/test_conformance.py holds one test per answered storage-contract guarantee in docs/guarantees.md, run against every driver through the store fixture. Today the only driver is MemoryStore. Refs #4
tests/test_frozen.py compares __all__, the Store method set, the Record fields and types with the Kind values, and the Filter keys against literal sets. Each failure message says the change needs a CHANGELOG.md entry. CHANGELOG.md starts with an Unreleased section. Refs #4
tests/test_ledger.py reads docs/guarantees.md. It fails when the Test column names a test that tests/test_conformance.py does not define, or when a row names neither a test nor a phase. The Test column now names the conformance test for each storage-contract row. The row on two processes waits for phase 2, because only the Hotdata driver has a load lock, and the memory-contract rows wait for phase 3. Refs #4
A GitHub Actions workflow installs uv with Python 3.11, the lowest version that pyproject.toml allows, and runs make verify on each pull request and on each push to main. Refs #4
README and contracts.md no longer say that no code exists. README gets an example of MemoryStore, run as written. CONTRIBUTING.md names the test files, the fake clock and embedder, and the ledger rule for rows that a later phase proves, and marks the Hotdata tests and the oracle test as phase 2. AGENTS.md points to CHANGELOG.md and the library, and adds the changelog rule. local.md says the Hotdata driver comes in phase 2. The brief and the ledger carry the record rules and the writer wording from contracts.md. plan.md records the supersede question for phase 3. Refs #4
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| slot = (draft.namespace, draft.key) | ||
| revisions = self._revisions.get(slot, []) | ||
| current = revisions[-1] if revisions else None | ||
| if current is not None and is_duplicate(current, draft.content): |
There was a problem hiding this comment.
nit: Skip deduplication when the current revision is past forget_after, or state the rule in docs/contracts.md (not blocking).
Today a put with the same normalized content returns the id of a hidden record. Example: put(content="X", forget_after=t), the clock passes t, then put(content="X"). The second put returns …@1 and writes nothing. list and search never return …@1, so the caller cannot bring the fact back. The same rule also drops a change to tags, sources, kind, or forget_after when the content is the same. docs/contracts.md:89 makes the length of sources the corroboration count, and this rule blocks adding a source to an existing fact.
| subject: str = "" | ||
| content: str | ||
| cues: tuple[str, ...] = () | ||
| payload: dict[str, JSONValue] = field(default_factory=dict) |
There was a problem hiding this comment.
nit: Freeze payload too, for example with a deep copy on the way in and out, or types.MappingProxyType (not blocking).
MemoryStore.get returns the stored Record object. A caller that runs store.get(ns, k).payload["x"] = 1 changes the stored revision in place. docs/contracts.md:101-102 says a record cannot change after the store writes it. The tuple change for cues, tags, and sources does not cover payload.
| query_vector = self._embed_query(query) | ||
| self._embed_missing(records) | ||
| hits = [ | ||
| Hit(record, cosine_distance(query_vector, self._vectors[record.id])) |
There was a problem hiding this comment.
nit: Give a record with a zero vector a distance of None, or put it last, and do not raise (not blocking).
cosine_distance raises ValueError for a zero vector. One record with empty content (the record allows it), or with text that the embedder maps to zero, makes every search with query text under that prefix fail. The conformance fake_embedder returns a zero vector for content="".
There was a problem hiding this comment.
The full diff was omitted from the review prompt. Reviewed from the checkout: src/hotmemory/memory.py, _rules.py, record.py, filter.py, store.py, tests/test_conformance.py, tests/conftest.py, docs/contracts.md. Not read: the other test files, CI config, and the remaining docs. CI was still pending when this review started. Only non-blocking nits are left inline.
A put whose content equals a current revision past its forget_after now writes a new revision, so a forgotten fact can come back. The record deep-copies payload, and MemoryStore returns copies, so a caller cannot change a stored record. A record refuses content that is empty after normalization, so no record gets a zero vector from its own text. Each rule has a conformance test that the ledger names, and each test was checked to fail with its fix undone. contracts.md, the brief, and the ledger state the rules. plan.md records for phase 3 that deduplication on content alone blocks adding a source. Refs #4
1 participant
This pull request adds the storage contract as typed Python, with
MemoryStoreas its first driver. It also adds the conformance suite, the frozen-surface tests, the ledger test, and CI. Nothing in this phase uses the network. Closes #4.How
src/hotmemory/record.py:Record, a frozen dataclass for schema version 1. It derivesidand refuses values that the contract does not allow.normalize()gives the form that deduplication compares.src/hotmemory/store.py:StoreandWriterastyping.Protocol, andHit(a record with its distance).src/hotmemory/filter.py:Filter, a frozen dataclass with one optional field per allowed key, andTimeRange.src/hotmemory/_rules.py: plain functions that both drivers call. They cover prefix matching, revision numbering, deduplication, visibility, filter matching, list order, and cosine distance.src/hotmemory/memory.py:MemoryStore, which takes an embedder and a clock, andMemoryWriter.tests/test_conformance.py: one test per answered storage-contract guarantee. Each test runs against every driver inDRIVERSintests/conftest.py.tests/test_frozen.pyfreezes the four surfaces.tests/test_ledger.pyreadsdocs/guarantees.md.make verifyruns ruff, ruff format, strict mypy, pytest with sockets disabled, and the link check, in that order..github/workflows/verify.ymlruns it on Python 3.11.Decisions taken during the phase
The owner answered three stop conditions:
cues,tags, andsourcesaretuple[str, ...], so a record cannot change in place..or/. A key cannot be empty or contain/or@. With these rules, the stored path and the id stay unambiguous.Where the contract was silent, this pull request fixes the following rules in
docs/contracts.md:tagsfilter matches records that hold every tag that it names.sinceincludes a record created at exactly that time.forget_afteris at or before the clock's current time.searchwith no query text returns the order oflist, with a distance of None.MemoryStoreranks by cosine distance overcontentonly.withblock of awriterraises an error, the writer drops its buffer.Verification
make verifypassed in 0.98 s after all caches were deleted. All five steps ran in order, and 70 tests passed.normalizefrom__all__,Store.delete,Record.forget_reason,Filter.actor). Each frozen test failed with this message:Frozen surface changed: the filter keys of list and search. This is a public contract change. Add an entry to CHANGELOG.md under Unreleased, then update the literal in this test.test_last_writer_wins_on_one_key.test_every_named_test_existsfailed withdocs/guarantees.md names tests that tests/test_conformance.py does not define.--disable-socket.git grep -i -w -Efor the known private names over the whole tracked tree found nothing.Not verified: nothing in this pull request runs against Hotdata or the local RuntimeDB stack. That is phase 2.
Notes
remember, as-of, scope, sources, turns) name phase 3. The row on two processes names phase 2.plan.mdhas one open question that the issue does not have.supersedemust setvalid_untilandexpired_aton the old record, but noStoreoperation can change those fields. Phase 3 decides howMemorysets them.