feat: add temporal claims to wiki pages

[Amidwestnoob]

Add temporal claims to OpenKB pages

Closes #222

Problem

OpenKB stores factual state as prose. A page cannot identify the date of a claim. A page cannot separate assistant output from source evidence. A page also cannot show how new evidence corrects an old claim.

Approach

This change adds claims to summary, concept, and entity pages.

• Store the claims field as a one-line JSON array in YAML frontmatter.
• Require a non-empty text field.
• Require an ISO-8601 as_of field.
• Require a non-empty source_anchor field.
• Require the status field to have a value of proposed, validated, superseded, or abandoned.
• Create the id field in OpenKB code.
• Ignore each model-supplied id field.
• Keep old claims as page history.
• Apply page-local supersession links after OpenKB adds new claims.
• Accept candidate supersession links in the supersedes field.
• Validate each candidate supersession link before OpenKB applies the link.
• Create the superseded_by supersession links for the applied links.
• Use read_current_page_claims() for strict current reads.
• Return claims that have a status value of proposed only after an explicit request.

Authority rules

The optional authority field can have a value of first_party, document, or assistant.

• If an assistant claim has a status value of validated, OpenKB changes the value to proposed.
• Assistant claims do not enter strict current reads.
• An assistant claim cannot supersede a claim that has a status value of validated.
• Only a claim with an authority value of first_party can supersede another claim with an authority value of first_party.
• If a stored claim has the same id value as an incoming duplicate claim, OpenKB keeps the stored claim.
• OpenKB ignores every field and every supersession link from the incoming duplicate claim.
• The document compiler sets the authority field to document before it writes a model claim.
• The document compiler sets the authority field to document for summary, concept, and entity pages.
• An external adapter can verify source authority. The adapter can then set the authority field to first_party or assistant through the public claims API.
• Claims with an authority value of document use the same as_of and status rules as claims without an authority field.

Synthetic regression case

The tests use a synthetic device.

• One assistant claim says that the device uses revision A.
• A second assistant claim says that the operating limit is 40 units.
• Both assistant claims have a status value of proposed.
• A later claim has an authority value of first_party.
• The first-party claim says that the device uses revision B.
• The first-party claim says that the operating limit is 100 units.
• Without the later claims, strict current reads exclude claims that have a status value of proposed.
• With the later claims, current_claims() returns the corrections from the claims that have an authority value of first_party.
• OpenKB keeps the old assistant claims with a status value of superseded.

The tests also cover direct authority changes. The tests cover duplicate-identity authority changes. The tests cover supersession link injection through a duplicate id value.

Aggregate production case study

A conversation-ingestion deployment supplied these aggregate lessons. The deployment filtered private data before model inference.

• The external source remained the source of truth.
• The compiled wiki remained a read-only projection.
• The materializer cleaned messages before model inference.
• The materializer removed duplicate synthetic payloads before model inference.
• The materializer kept source, speaker, authority, and evidence identifiers.
• One verified export contained 280 conversations.
• The materializer created 537 documents from the 280 conversations.
• Initial materialization used a document limit of 240,000 characters.
• Recovery after a context overflow used a document limit of 40,000 characters.
• A later materializer used a message limit of 40,000 characters.
• The later materializer used a document limit of 120,000 characters.
• The materializer did not change a window after it closed the window.
• Content hashes and idempotent manifests prevented duplicate knowledge.
• Replay-safe compilation also prevented duplicate knowledge.
• One complex document exceeded a timeout of 600 seconds.
• The document completed in 885 seconds during a later run.
• Tests set the timeout to 1,200 seconds for each document.
• Each batch contained no more than two documents.
• The scheduler guard remained 3,600 seconds.
• The scheduler processed the oldest pending documents first.
• Caller code applied privacy filters before model inference.
• OpenKB core does not contain a project-specific regular expression for privacy.
• Caller code controls the closed-window delay.
• Caller code controls the list of transport sources.

Public documentation

• README.md explains the claims field and strict current reads.
• README.md explains the authority field.
• openkb/schema.py defines each JSON field and each allowed field value.
• docs/golden-principles.md defines the history, authority, and duplicate-ID rules.

Tests

The deterministic tests cover these cases:

• OpenKB creates the id field.
• OpenKB validates the text, as_of, status, source_anchor, and authority fields.
• OpenKB rejects a backward supersession link.
• OpenKB limits supersession links to one page.
• Repeated input gives the same result.
• OpenKB records the supersedes and superseded_by supersession links.
• Strict current reads return claims that have a status value of validated.
• An assistant claim cannot supersede a claim with an authority value of first_party.
• An assistant claim with a status value of abandoned cannot supersede a claim with a status value of validated.
• A document claim cannot supersede a claim with an authority value of first_party.
• A duplicate id value cannot change the stored authority or status field.
• A duplicate id value cannot add a supersession link.
• The document compiler sets the authority field to document before each summary, concept, or entity write.
• A document claim can supersede another document claim when the as_of and status fields permit the supersession link.
• A claim without an authority field can supersede another claim without an authority field when the as_of and status fields permit the supersession link.
• Frontmatter keeps the claims JSON array.
• read_current_page_claims() rejects a path outside the wiki root.

Run these commands:

python -m pytest
python -m ruff check .
python -m ruff format --check .
python -m mypy openkb/claims.py
git diff --check

The last run gave these results:

• Full pytest: 1,286 passed.
• ruff check .: passed.
• ruff format --check .: 122 files already formatted.
• mypy openkb/claims.py: passed.
• git diff --check: passed.

Risks and non-goals

• OpenKB cannot prove that source evidence is true.
• An external adapter must verify source authority before it sets the authority field to first_party.
• A raw page read includes claim history.
• A consumer must use current_claims() for strict current reads.
• The long-document path keeps existing summary claims.
• The long-document path does not extract new claims from the PageIndex overview.
• This change does not compact claim history.
• This change does not add a transport adapter.
• This change does not add scheduler rules.
• This change does not add privacy rules.
• This change does not add live-ingestion code.

Relation to existing work

• This change does not depend on OpenKB PR Add pluggable ingest bundle pipeline #179.
• This change does not modify OpenKB PR Add pluggable ingest bundle pipeline #179.
• ChatIndex is not a dependency.
• ChatIndex conversation trees do not provide page-local authority rules or supersession links.
• OpenKB chat persistence stores OpenKB query chats.
• OpenKB chat persistence does not ingest external conversations.

Follow-up

Create a separate conversation adapter that does not depend on a transport. Create a separate materialization interface. The adapter and the interface can provide deterministic cleaning. The adapter and the interface can provide source identifiers. The adapter and the interface can provide speaker identifiers. The adapter and the interface can provide authority values. The adapter and the interface can provide evidence identifiers. The adapter and the interface can provide bounded documents. The adapter and the interface can provide recovery splits. The adapter and the interface can provide closed windows. The adapter and the interface can provide content hashes. The adapter and the interface can provide idempotent manifests. The adapter and the interface can provide timeouts. The adapter and the interface can provide replay-safe compilation.

Caller code must control transport lists. Caller code must control closed-window delays. Caller code must control privacy filters. Caller code must control credentials. Caller code must control scheduler limits. The adapter design and the interface design must not depend on PR #179.