Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,12 @@ public CLI/import or serialized contracts. Retain public behavior tests; remove
only characterization scaffolding whose retired implementation has no consumer.
Deletion is code retirement, not deletion of users' state, receipts or backups.

### Proposed T4 slice: unused Python lease/handoff facades
### Merged T4 slice: unused Python lease/handoff facades

Caller audit at `e240730ec` finds the following internal crossings unused by
production. Native decision and transaction owners remain; this is independent
of D2 qualification and default-entry adoption.
The caller audit at `e240730ec` led to #5395, merged at `8474c8d86`.
The following unused internal crossings are retired. Native decision and
transaction owners remain; this is independent of D2 qualification and
default-entry adoption.

| Removed boundary | Last caller / replacement | Compatibility and validation |
| --- | --- | --- |
Expand All @@ -65,9 +66,11 @@ of D2 qualification and default-entry adoption.
`LeaseModeGateCommand` also remain because the semantic-vocabulary registry
explicitly retains that input contract until its M4 review. This slice does not
lower semantic coverage floors to discard a declared compatibility obligation.
Old facade-only tests retire with their implementation; public/native behavior
Old facade-only tests retired with their implementation; public/native behavior
tests remain. Reverting this slice restores the internal crossing without a data
conversion. Maintainer review is required; this proposal is not installed behavior.
conversion. Local CLI adoption at `db3672f3c` verifies a clean source manifest,
qualified SQLite runtime, current known authority formats and healthy canonical
contract readback. This does not certify every installed Host or D2.

## Next delivery order

Expand Down Expand Up @@ -373,3 +376,24 @@ microbenchmark with fingerprint memoization explicitly cleared regressed from
costs more when all bytes are already hot. Neither workload establishes a fleet
latency guarantee. Whole-Goal payload/consumer work and sustained operation
remain open; this increment authorizes no legacy-writer deletion or UI truncation.

### File recovery receipt batches

Archive restore and audit already use the provider-neutral 1–64 operation
receipt batch contract. File now implements that contract with one exact-byte
and store-identity proof per batch instead of rereading its envelope for each
receipt. Caller order, duplicates, missing results and original receipt bodies
remain intact; each returned body is detached. Invalid input or corrupt retained
history rejects the batch. Array holes are rejected before storage access,
including through the shared helper. Single-receipt error projection stays unchanged.

On the same detached, restored 1,287-commit history, nine warm samples per arm
on macOS arm64 / Node 24.21.0 reduce a 16-receipt File batch median from
346.2 to 21.3 ms; the unchanged SQLite control measures 111.3 and 111.1 ms.
Receipt results and authority heads match within each provider. These are warm
component timings, not equivalent provider-integrity work, whole-restore latency,
cold-read or D2/default qualification. File still rewrites the retained envelope
on each restored commit; a prior full-history restore exceeded its caller's
300-second timeout and later published an exact matching acknowledgement.
That remaining recovery cost is not closed by this receipt-read optimization.
The #4224 soak was started; its final evidence and applicability remain pending.
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,11 @@ owner、持久兼容义务、正反例证据及回退方式,和不可变基线
内部删除必要但不充分,不能忽略公开 CLI/import 和序列化契约。保留公共行为测试,
只删没有消费者的旧实现专属 characterization。删的是代码,不是用户状态、回执和备份。

### 提议的 T4 切片:已无调用方的 Python lease/handoff facade
### 已合入的 T4 切片:已无调用方的 Python lease/handoff facade

在 `e240730ec` 核对调用方后,下列内部跨界已无生产消费者。原生决策和事务 owner
保留;这项删除不等待 D2 资格或默认入口接入,也不宣称完成它们。
在 `e240730ec` 核对调用方后,#5395 已于 `8474c8d86` 合入,退役了下列
无生产消费者的内部跨界。原生决策和事务 owner 保留;这项删除不等待 D2 资格或
默认入口接入,也不宣称完成它们。

| 删除边界 | 最后调用方/替代 owner | 兼容与验证 |
| --- | --- | --- |
Expand All @@ -58,8 +59,9 @@ owner、持久兼容义务、正反例证据及回退方式,和不可变基线
`authority_core.py` 仍是活跃 Todo bridge。`LeaseAction`、`LeaseModeGateCommand`
也保留:semantic-vocabulary 注册表明确将该输入契约保留到 M4 评审。本切片不通过
降低语义覆盖下限丢弃已有兼容义务。仅服务旧 facade 的测试随实现退役,公共/原生
行为测试保留。回退该切片可恢复内部跨界,无需转换数据。须由维护者评审;这是提议,
不代表已安装行为。
行为测试保留。回退该切片可恢复内部跨界,无需转换数据。本机 CLI 已采用
`db3672f3c`,验证了干净源码清单、具备资格的 SQLite runtime、已知权威格式均为
当前版本及健康的 canonical 合同读回。这不证明所有已安装 Host 或 D2 已验收。

## 下一轮交付顺序

Expand Down Expand Up @@ -280,3 +282,19 @@ Node 24.21.0。启动预热一次后,每组交替运行九个独立 CLI 进程
请求仍复用缓存。字节已经在热缓存中时,线程调度成本更高。两种负载都不能外推为
用户群体的延迟保证。整 Goal 大包/消费者与持续运行仍待推进,本增量不授权删除
旧 writer 或裁剪 UI 数据。

### File 恢复的批量回执读取

Archive restore/audit 已使用 provider 通用的 1–64 个操作批量回执合同。File 现在
也实现该合同:每批只做一次完整字节与 store identity 证明,不再为每条回执重读
整个文件。调用顺序、重复 ID、缺失结果、原始回执内容均保留,每个返回内容独立。
非法输入或损坏历史会拒绝整批;数组空位在访问存储前被拒绝,公共 helper 入口也
遵循该规则。单条回执查询的错误投影不变。

在同一份已恢复、隔离的 1,287 笔历史上,macOS arm64/Node 24.21.0 每组九次暖读
样本,File 查询 16 条回执的中位数从 346.2 降至 21.3 毫秒;未改动的 SQLite 对照为
111.3/111.1 毫秒。每个 provider 的回执结果与权威 head 均保持一致。这是暖读组件
测量,不是相同完整性工作下的 provider 比较、整次恢复延迟、冷读或 D2/默认项
验收。File 每恢复一笔仍重写保留的文件;此前一次完整历史恢复超出调用方的
300 秒超时,随后才发布精确匹配的确认。批量回执优化没有闭合这项恢复成本。
#4224 的 soak 已启动;其最终证据和对当前候选的适用性仍待核对。
32 changes: 21 additions & 11 deletions loopx/control_plane/coordination/file_authority_store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import type {
AuthorityStoreHead,
AuthorityStoreReadFailure,
AuthorityStoreReceiptResult,
AuthorityStoreReceiptBatchResult,
AuthorityStoreScanResult,
} from "./authority_store.ts";
import {
Expand Down Expand Up @@ -467,19 +468,28 @@ export class FileAuthorityStore implements AuthorityStore {
reason: error instanceof Error ? error.message : "invalid operation id",
};
}
const batch = await this.readReceipts([normalized]);
return batch.status === "receipts" ? batch.results[0]! : batch;
}

/** One exact-byte/identity proof per bounded batch. Look up every requested
* operation in that verified index; do not replace receipt proof with a scan.
* Returned items own their bodies, including duplicate operation IDs. */
async readReceipts(operationIds: readonly string[]): Promise<AuthorityStoreReceiptBatchResult> {
try {
const transaction = (await this.readVerified())?.receipts.get(normalized);
return transaction
? {
status: "found",
cursor: transaction.cursor,
if (!Array.isArray(operationIds) || operationIds.length < 1 || operationIds.length > 64) {
throw new AuthorityStoreProtocolError("receipt batch requires 1..64 operations");
}
const normalized = Array.from({length: operationIds.length}, (_, i) =>
requireAuthorityStoreId(operationIds[i], "operation id"));
const verified = await this.readVerified();
return {status: "receipts", results: normalized.map((id): AuthorityStoreReceiptResult => {
const transaction = verified?.receipts.get(id);
return transaction ? {status: "found", cursor: transaction.cursor,
provider_revision: transaction.providerRevision,
receipts: structuredClone([...transaction.receipts]),
}
: { status: "missing" };
} catch (error) {
return readFailure(error);
}
receipts: structuredClone([...transaction.receipts])} : {status: "missing"};
})};
} catch (error) { return readFailure(error); }
}

async scanCommitted(afterCursor: string | null, limit: number): Promise<AuthorityStoreScanResult> {
Expand Down
99 changes: 99 additions & 0 deletions tests/control_plane_ts/authority_store.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { FileAuthorityStore } from "../../loopx/control_plane/coordination/file_
import {
AUTHORITY_STORE_PROVIDER_PROFILES,
AUTHORITY_STORE_REQUIRED_GUARANTEES,
readAuthorityReceipts,
} from "../../loopx/control_plane/coordination/authority_store.ts";
import {
authorityStoreCommitFixture as commit,
Expand All @@ -31,6 +32,104 @@ registerAuthorityOperationReplayConformance("file provider", async (t) => {
return { store, contender: new FileAuthorityStore(root, "goal-a") };
});

test("file receipt batches retain caller order, original receipts and detached results", async t => {
const {store} = await fixture(t);
assert.equal(typeof store.readReceipts, "function");
let revision: string | null = null;
const expected = [];
for (let i = 1; i <= 3; i++) {
const request = commit(revision, `operation-${i}`, i, i);
request.receipts = [{original: i, metadata: {absent: null, enabled: false}}];
const result = await store.commitAuthority(request);
assert.equal(result.status, "applied");
if (result.status !== "applied") throw new Error("seed failed");
revision = result.provider_revision;
expected.push({status: "found", cursor: String(i), provider_revision: revision, receipts: request.receipts});
}
const before = await readFile(store.path);
const ids = ["operation-3", "absent", "operation-1", "operation-3"];
const result = await readAuthorityReceipts(store, ids);
assert.deepEqual(result, {status: "receipts", results: [expected[2], {status: "missing"}, expected[0], expected[2]]});
if (result.status !== "receipts" || result.results[0].status !== "found") throw new Error("batch failed");
result.results[0].receipts[0].original = "caller edit";
assert.deepEqual(result.results[3], expected[2], "duplicate results must not share mutable receipt bodies");
assert.deepEqual(await store.readReceipt("operation-3"), expected[2]);
assert.deepEqual(await readFile(store.path), before);

// Readonly types do not freeze a caller's array while filesystem IO yields.
// Force a fresh proof and mutate that array inside the existing decode seam.
await writeFile(store.path, Buffer.concat([before, Buffer.from("\n")]));
class MutatingCallerStore extends FileAuthorityStore {
protected override decodeStoredDocument(value: unknown, identity: string) {
ids.splice(0, ids.length, "");
return super.decodeStoredDocument(value, identity);
}
}
assert.deepEqual(await readAuthorityReceipts(new MutatingCallerStore(store.directory, "goal-a"), ids),
{status: "receipts", results: [expected[2], {status: "missing"}, expected[0], expected[2]]});
});

test("file receipt batches reject invalid input and corrupt historical proof as a whole", async t => {
const {store} = await fixture(t);
const first = await store.commitAuthority(commit(null, "old", 1, 1));
assert.equal(first.status, "applied");
if (first.status !== "applied") throw new Error("seed failed");
assert.equal((await store.commitAuthority(commit(first.provider_revision, "current", 2, 2))).status, "applied");
const original = await readFile(store.path, "utf8");
for (const ids of [[], Array(65).fill("old"), ["old", ""], ["old", null]]) {
// Exercise direct provider input too, rather than only the common length guard.
const batch = await store.readReceipts(ids as string[]);
assert.equal(batch.status, "failed");
assert.deepEqual(await readFile(store.path, "utf8"), original);
}
const invalidSingle = await store.readReceipt("");
assert.equal(invalidSingle.status, "failed");
if (invalidSingle.status === "failed") assert.equal(invalidSingle.reason_code, "invalid_operation_id");
const forged = JSON.parse(original);
forged.committed[0].receipts = [{forged: true}];
await writeFile(store.path, JSON.stringify(forged));
assert.equal((await readAuthorityReceipts(store, ["current", "absent"])).status, "failed",
"unchanged current receipts and missing IDs cannot bypass corrupt older history");
await writeFile(store.path, original);
assert.equal((await readAuthorityReceipts(store, ["old", "current"])).status, "receipts");
await writeFile(store.identityPath, `file:${"f".repeat(32)}`);
assert.equal((await readAuthorityReceipts(store, ["current"])).status, "failed");
});

test("missing File receipt batches stay read-only", async t => {
const {store} = await fixture(t);
assert.deepEqual(await readAuthorityReceipts(store, Array(64).fill("absent")),
{status: "receipts", results: Array(64).fill({status: "missing"})});
await assert.rejects(readFile(store.identityPath), {code: "ENOENT"});
});

test("file receipt batches reject array holes before reading storage", async t => {
const {root, store} = await fixture(t);
const first = await store.commitAuthority(commit(null, "present", 1, 1));
assert.equal(first.status, "applied");
const original = await readFile(store.path);
// Sparse arrays are valid string[] values in TypeScript. Every request slot
// must be validated, even when an array method would skip an absent property.
const empty = new Array<string>(1);
const mixed = ["present", "hole", "absent", "present"];
delete mixed[1];
const masked = new Array<string>(1);
masked[Symbol.iterator] = () => ["present"].values();
const unreadable = new FileAuthorityStore(root, "unreadable");
await mkdir(unreadable.path);
assert.equal((await unreadable.readReceipts(["present"])).status, "unavailable");
for (const ids of [empty, mixed, masked]) {
for (const provider of [store, unreadable]) {
for (const result of [await provider.readReceipts(ids),
await readAuthorityReceipts(provider, ids)]) {
assert.equal(result.status, "failed", "holes cannot produce successful undefined receipt items");
if (result.status === "failed") assert.equal(result.reason_code, "provider_protocol_violation");
}
}
}
assert.deepEqual(await readFile(store.path), original);
});

test("file provider persists object keys in deterministic Unicode order", async (t) => {
const { store } = await fixture(t);
const ordered = commit(null, "operation-order", 1, 1);
Expand Down
Loading