From dbc1a2a5ba99e697dd725b1344dd8abbba4fe47a Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Thu, 27 Aug 2026 16:20:04 +0100 Subject: [PATCH 001/192] Add the integration provider seam design spec The five-PR series (#1043 to #1047) opens the identity, device and geo seams. The nine vendor integrations already in core sit behind the integration registry instead, which is a private table, so none of them can move out until that table is opened. This spec defines the one core change that opens it: public registration builders with a second input on IntegrationRegistry, browser JavaScript carried on the registration, startup validation as a hook, the same treatment for auction providers and the bid renderer contract, and neutral replacements for the two places where a vendor reaches into core. It then sets out the migration of all nine existing integrations, one PR each. The change is complete in itself: after it, no vendor move needs a core change. Written against the series' tree with the file and line references for every claim about the current code. Documentation only. --- ...-08-27-integration-provider-seam-design.md | 207 ++++++++++++++++++ 1 file changed, 207 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md new file mode 100644 index 0000000000..6dbfcb7eea --- /dev/null +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -0,0 +1,207 @@ +# Design Spec: The Integration Provider Seam + +**Status:** Proposed, 2026-08-27. Sixth PR in the provider series. It adds +this document only and targets `main` directly; it reads alongside the +series' specs, which land with PR #1047. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-07-30-pluggable-providers-design.md`, +`2026-07-30-provider-migration-rollout-design.md`, +`provider-code-registry.md` +**Related PRs:** #1043, #1044, #1045, #1046, #1047, #1054 +**Last updated:** 2026-08-27 + +> **Why this spec exists.** PRs #1043 to #1047 open the identity, device and +> geo seams, so a vendor can ship an Edge Cookie provider in its own crate +> and an adapter injects it. The nine vendor integrations already inside +> `trusted-server-core` do not sit behind those seams. They hang off the +> integration registry, which is a private table in core, so none of them +> can move out until that table is opened. This spec defines the one core +> change that opens it, so the migration of every existing vendor is a +> single defined piece of work rather than an open question repeated once +> per vendor. + +## 1. The problem, with the code that causes it + +Every claim here was read from the tree at `split/5-response-hook-docs`, +which is `main` plus the five PRs. + +1. **The registry is closed.** `IntegrationRegistry::new` takes only + `&Settings` and iterates a fixed table + (`crates/trusted-server-core/src/integrations/registry.rs:797`, table at + `crates/trusted-server-core/src/integrations/mod.rs:290`). Both + `IntegrationBuilder` and `builders()` are `pub(crate)` with private + fields, so no adapter and no external crate can add to the list. The + payload type `IntegrationRegistration` + (`registry.rs:586`) is already public, so an outside crate can build a + registration but has nowhere to hand it. +2. **Browser JavaScript is fixed at build time.** `trusted-server-js` + discovers `lib/src/integrations/*/index.ts`, builds one file per + integration, and `build.rs` writes a fixed array of `include_str!` + entries consumed by `bundle.rs`. `IntegrationRegistry::js_module_ids` + only serves a module when + `trusted_server_js::module_bundle(id).is_some()` + (`registry.rs:1169`), so an integration outside that compile-time map + gets no script however it registers. +3. **Startup validation names every vendor.** `validate_enabled_integrations` + imports and calls each vendor's config type by name + (`crates/trusted-server-core/src/config.rs:136` to `:166`). +4. **Auction providers are a second closed table.** + `crates/trusted-server-core/src/auction/mod.rs:49` lists Prebid, APS and + the ad server mock. +5. **Two vendors reach further into core.** DataDome drives cache privacy + and the origin fetch decision through a marker type + (`html_processor.rs:303`, `publisher.rs:4367` to `:4387`, + `publisher.rs:2653`), and GPT diagnostics is called by name from all four + adapters (for example + `crates/trusted-server-adapter-fastly/src/app.rs:584`). + +The result is that the project carries nine vendors as core code (ten +registered integrations, since GPT registers a proxy and a diagnostics +integration). Tech Lab engineering time is spent on named commercial vendors, and +every new vendor is another core change, as PR #1054 shows. + +## 2. Principle + +A vendor integration is a provider like any other. Core owns the seam and +owns nothing behind it. Concretely: + +- Core defines the registration contract and the request pipeline. It names + no vendor. +- A vendor integration ships as its own crate with its Rust, its browser + JavaScript, its configuration type, its startup validation and its tests. +- An adapter composes the deployment by injecting the registrations it was + built with, exactly as it already injects the geo and device providers. +- Tech Lab engineering assesses and reviews vendor crates. It does not + maintain them. + +## 3. Design + +### 3.1 Opening the registry + +Make the builder contract public and give the registry a second input. + +- `IntegrationBuilder` becomes `pub` with a public constructor, and + `builders()` stays as the built-in set. +- `IntegrationRegistry::new` gains a companion, + `IntegrationRegistry::with_registrations(settings, extra)`, where `extra` + is a slice of externally supplied builders. `new` keeps its signature and + calls the companion with an empty slice, so no existing caller changes + behavior. +- Duplicate integration ids are a startup error, naming both sources, so a + vendor crate cannot silently shadow a built-in. + +### 3.2 Carrying browser JavaScript on the registration + +Add one optional field to `IntegrationRegistration`, next to `js_deferred` +and `js_disabled`: the module source and its hash, both `&'static str`, so a +crate can `include_str!` its own built bundle. + +`js_module_ids` keeps serving built-in ids from the compile-time map and +serves a carried module from the registration. `publisher.rs` composes the +served script and its hash from both sources rather than calling +`trusted_server_js::concatenate_modules` alone. The hash rule is unchanged, +so the served bundle stays cacheable and its integrity attribute stays +correct. + +### 3.3 Startup validation on the registration + +Replace the named list in `config.rs` with a validation hook on the +registration, so a vendor validates its own configuration and a missing +vendor cannot silently stop being validated. The existing test that asserts +every registered integration is covered by deploy validation +(`config.rs:408`) is rewritten against the hook, so the guarantee survives +in a vendor-neutral form. + +### 3.4 Auction providers + +Give `AuctionOrchestrator` the same treatment as the registry: a public +provider-builder type and a second input, so an auction-side vendor such as +APS can register from its own crate. Prebid stays in core as protocol +support rather than as a vendor integration. + +The bid renderer contract is generalized in the same change. Today +`BidRenderer` is an enum with one variant, `Aps(ApsRendererV1)` +(`crates/trusted-server-core/src/auction/types.rs:216`), serialized into the +OpenRTB response extension under a `type` tag +(`crates/trusted-server-core/src/auction/formats.rs:377`, +`crates/trusted-server-core/src/openrtb.rs:183`). It becomes an open +descriptor, a type tag and a payload the auction provider supplies, with +the same serialized form, so the response a page receives does not change +and the APS renderer type moves with APS. The ad server mock in core uses +the neutral form. + +### 3.5 The two neutral hooks + +- **Response shaping.** DataDome's marker becomes a neutral request + extension meaning "this response is personalized to the request, do not + share it", set by any integration. Core keeps the behavior, being the full + body buffer and private cache, and stops naming a vendor. +- **Request prepare and finalize.** The direct GPT diagnostics calls in the + four adapters move behind hooks on the registration, so an adapter runs + whatever its registrations declare. + +## 4. Migration of the nine existing integrations + +One vendor per PR, after this change lands. Each moves its Rust, its +TypeScript, its config type and its tests into +`crates/integrations/`, and the adapter that wants it depends on it. + +| Vendor | What it needs | +| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | +| Didomi, Google Tag Manager, Lockr, Osano, Permutive, Sourcepoint | Move as they are. Coupled only through the builder table, deploy validation and the JS map. | +| APS | Also needs the auction provider seam and the generalized renderer contract in §3.4, both of which this change delivers. | +| GPT (the `gpt` proxy and `gpt_diagnostics`) | The proxy moves as it is. The diagnostics half needs the prepare and finalize hooks in §3.5. | +| DataDome | Needs the neutral response-shaping hook in §3.5, and about forty test literals move with it. | + +`[integrations.]` configuration tables need no change, because +`IntegrationSettings` is a flattened map that already accepts unknown vendor +keys (`crates/trusted-server-core/src/settings.rs:166`). + +## 5. What does not change + +The request pipeline, the hook traits and their order, the served script +format and its hash, every `[integrations.*]` table, the permission model, +and the Edge Cookie, device and geo seams from PRs #1043 to #1046. No +integration changes behavior. A deployment that lists the same integrations +gets the same responses. + +## 6. Acceptance + +1. **A round trip with a non-default implementation.** A test integration + defined outside `trusted-server-core`, carrying its own JavaScript, + registers through an adapter, appears in the served bundle with the + right hash, runs its hooks in the right order, and is rejected on a + duplicate id. A seam is only proven by an implementation that is not the + built-in one. +2. **Parity.** The existing integration and parity suites pass unchanged, + because the built-in set still registers through the same path. +3. **No vendor left behind.** The rewritten deploy-validation test shows + every registered integration validates its configuration. +4. All CI gates in `CLAUDE.md`, on all four adapters. + +## 7. Risk + +The change is wide but shallow. It touches the registry, the served script +path, deploy validation and four adapter entry points, and it changes no +integration's behavior. The largest risk is the served script, where a +mistake shows up as a wrong hash or a missing module, so §6's round trip +covers both. Doing this once is what removes the per-vendor core change +that the project pays for today, most recently in PR #1054. + +## 8. Sign-off + +| # | Decision | Status | +| --- | ------------------------------------------------------------------------------- | -------------------- | +| 1 | Vendor integrations belong outside core, behind the registration contract | Proposed | +| 2 | Tech Lab engineering reviews vendor crates, and does not maintain them | Proposed, governance | +| 3 | A registration may carry its own browser JavaScript | Proposed | +| 4 | Deploy validation moves onto the registration | Proposed | +| 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | +| 6 | This change is complete in itself: after it, no vendor move needs a core change | Proposed | + +## Revision record + +| Date | Change | +| ---------- | ----------------------------------------------------------------------------------------------------------------------------- | +| 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | +| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | From 12ad16c7ebed0b3e0373a8a2e895bb04d5c9d489 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 28 Aug 2026 10:14:34 +0100 Subject: [PATCH 002/192] Revise the integration seam spec to carry identity, geo and device as registration capabilities Adopt the registration shape asked for in the review of PR 1043 and apply its rule consistently, so a module declares its identity, geo and device providers alongside its JavaScript and hooks (new section 3.6). Record the relationship to the pluggable-providers spec in PR 986, reorder the series so this spec and its core implementation precede PR 1043, and add the capabilities round trip to the acceptance list. --- ...-08-27-integration-provider-seam-design.md | 210 ++++++++++++++---- 1 file changed, 172 insertions(+), 38 deletions(-) diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 6dbfcb7eea..b86730f443 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -1,14 +1,18 @@ # Design Spec: The Integration Provider Seam -**Status:** Proposed, 2026-08-27. Sixth PR in the provider series. It adds -this document only and targets `main` directly; it reads alongside the -series' specs, which land with PR #1047. +**Status:** Proposed, 2026-08-27, revised 2026-08-28. This PR adds this +document only and targets `main` directly. Following the review of #1043 +(27 August) the seam it defines is a precondition for the provider series +rather than a follow-up to it, so the order is now this spec, then its +implementation in a seventh PR against `main` (51Degrees), then PRs #1043 +to #1047 reworked onto it. It reads alongside the series' specs, which land +with PR #1047. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-07-30-pluggable-providers-design.md`, `2026-07-30-provider-migration-rollout-design.md`, `provider-code-registry.md` -**Related PRs:** #1043, #1044, #1045, #1046, #1047, #1054 -**Last updated:** 2026-08-27 +**Related PRs:** #986, #1043, #1044, #1045, #1046, #1047, #1054 +**Last updated:** 2026-08-28 > **Why this spec exists.** PRs #1043 to #1047 open the identity, device and > geo seams, so a vendor can ship an Edge Cookie provider in its own crate @@ -20,14 +24,30 @@ series' specs, which land with PR #1047. > single defined piece of work rather than an open question repeated once > per vendor. +> **Relationship to #986 and the #1043 review.** The pluggable-providers +> spec in #986 (31 July) defines identity, device and geo as providers +> selected by `[ec] provider`, `[device] provider` and `[geo] provider` and +> wired by each adapter through a composition root. #1043 and #1044 +> implement that. The review of #1043 on 27 August asks instead that a +> vendor's identity provider be a capability declared on its integration +> registration, because a vendor ships its browser JavaScript and its +> identity function together. This revision adopts that end state (§3.6) +> and applies its rule consistently, so geo and device providers attach the +> same way. The lifecycle contract, the identifier envelope, the permission +> gating and the validation rules in #986 are unchanged. What changes is +> only where a vendor's provider is constructed and selected from. The +> registration shape needs a registry a vendor crate can register with, +> which is what §3.1 opens, so this spec precedes #1043 rather than +> following it. + ## 1. The problem, with the code that causes it -Every claim here was read from the tree at `split/5-response-hook-docs`, -which is `main` plus the five PRs. +Every claim here was read from `main` at b7fcb5d4c (28 August), which the +seventh PR targets; the five series PRs do not touch these files. 1. **The registry is closed.** `IntegrationRegistry::new` takes only `&Settings` and iterates a fixed table - (`crates/trusted-server-core/src/integrations/registry.rs:797`, table at + (`crates/trusted-server-core/src/integrations/registry.rs:792`, table at `crates/trusted-server-core/src/integrations/mod.rs:290`). Both `IntegrationBuilder` and `builders()` are `pub(crate)` with private fields, so no adapter and no external crate can add to the list. The @@ -40,7 +60,7 @@ which is `main` plus the five PRs. entries consumed by `bundle.rs`. `IntegrationRegistry::js_module_ids` only serves a module when `trusted_server_js::module_bundle(id).is_some()` - (`registry.rs:1169`), so an integration outside that compile-time map + (`registry.rs:1155`), so an integration outside that compile-time map gets no script however it registers. 3. **Startup validation names every vendor.** `validate_enabled_integrations` imports and calls each vendor's config type by name @@ -50,10 +70,10 @@ which is `main` plus the five PRs. the ad server mock. 5. **Two vendors reach further into core.** DataDome drives cache privacy and the origin fetch decision through a marker type - (`html_processor.rs:303`, `publisher.rs:4367` to `:4387`, + (`html_processor.rs:303`, `publisher.rs:4361` to `:4381`, `publisher.rs:2653`), and GPT diagnostics is called by name from all four adapters (for example - `crates/trusted-server-adapter-fastly/src/app.rs:584`). + `crates/trusted-server-adapter-fastly/src/app.rs:564`). The result is that the project carries nine vendors as core code (ten registered integrations, since GPT registers a proxy and a diagnostics @@ -88,7 +108,10 @@ Make the builder contract public and give the registry a second input. calls the companion with an empty slice, so no existing caller changes behavior. - Duplicate integration ids are a startup error, naming both sources, so a - vendor crate cannot silently shadow a built-in. + vendor crate cannot silently shadow a built-in. There is no such check + today, only a per-route conflict check and a debug-only assertion, and + `AuctionOrchestrator::register_provider` silently keeps the last writer, + so the builder carries a source label and both tables get the check. ### 3.2 Carrying browser JavaScript on the registration @@ -97,11 +120,22 @@ and `js_disabled`: the module source and its hash, both `&'static str`, so a crate can `include_str!` its own built bundle. `js_module_ids` keeps serving built-in ids from the compile-time map and -serves a carried module from the registration. `publisher.rs` composes the -served script and its hash from both sources rather than calling -`trusted_server_js::concatenate_modules` alone. The hash rule is unchanged, -so the served bundle stays cacheable and its integrity attribute stays -correct. +serves a carried module from the registration. The composition of the +served script moves from `trusted-server-js` into core, because every hop +after `js_module_ids` today re-enters `trusted-server-js` by id and silently +drops an id it does not know (`bundle.rs`, `concatenated_module_ids` and +`visit_concatenated_module_parts`), and the hash memo is keyed on the id +list alone. Core composes body and hash from (id, source, hash) triples +drawn from both sources, keeping the exact byte rule of today (core first, +`;\n` separator) so every existing `?v=` hash is unchanged. Three consumers +follow the registry rather than the compile-time list: the standalone +module route `parse_single_module_filename` (`publisher.rs`), the +`GPT_DIAGNOSTICS_INTEGRATION_ID` standalone special case, which becomes a +registration property, and `template_fingerprint`, which must cover carried +modules so a vendor crate rebuild invalidates the server-side template +cache. The served script keeps its cache rule, being the `?v=` query +matched at serve time (there is no integrity attribute on the tag today, and +this change adds none). ### 3.3 Startup validation on the registration @@ -109,8 +143,14 @@ Replace the named list in `config.rs` with a validation hook on the registration, so a vendor validates its own configuration and a missing vendor cannot silently stop being validated. The existing test that asserts every registered integration is covered by deploy validation -(`config.rs:408`) is rewritten against the hook, so the guarantee survives -in a vendor-neutral form. +(`config.rs:688` on `main`) is rewritten against the hook, so the guarantee +survives in a vendor-neutral form. Two details the map of `main` adds. The +enumeration the test needs is independent of which integrations a +configuration enables, so the registry exposes the full set of registrations +it was built from, not only the enabled ones. And `adserver_mock` is +validated today without being a registration (it exists only as an auction +provider), so auction-side registrations carry the same validation hook and +the test covers both tables. ### 3.4 Auction providers @@ -140,6 +180,67 @@ the neutral form. four adapters move behind hooks on the registration, so an adapter runs whatever its registrations declare. +### 3.6 Identity, geo and device as registration capabilities + +The rule. Things the host supplies are platform services, being the KV +store, the HTTP client, the host geo lookup, and the host TLS and HTTP/2 +signals. Things a vendor supplies are capabilities of that vendor's module. +An identity provider, a geo provider and a device provider are supplied by +vendors, with or without any host involved, so all three are module +capabilities, and the same registration carries them alongside the module's +JavaScript and hooks. + +- The registration builder gains three optional capabilities, at most one + of each per registration (names indicative, the shape is normative): + `.with_ec_provider(Arc)`, + `.with_geo_provider(Arc)` and + `.with_device_provider(Arc)`. The traits are the ones + #1043 and #1044 define, unchanged. +- Selection keeps the select-exactly-one semantics of #986. `[ec] provider`, + `[geo] provider` and `[device] provider` each name either a built-in (the + names #986 and #1043/#1044 already define, for example `hmac` and `none` + for identity) or the id of a registered module that declares the matching + capability. A selector that names a module which is registered but does + not declare the capability, or that names nothing registered, is a + startup error. A module that declares a capability the selector does not + name is inert for that capability and its other hooks still run, and + startup logs a warning naming the module and the unused capability, so an + operator can see a module shipping script for a provider that is not + selected. +- No provider is built into core. Everything goes through one method, so + the HMAC identity provider from #1043 and the User-Agent-only device + provider from #1044 become Tech Lab-owned modules in their own crates + under `crates/integrations/`, configured under `[integrations.]` and + validated through §3.3 like any other module, and the adapters register + them by default. Core keeps only the seam and the `none` state for each + capability (no identity, no location, unknown device signals). A + deployment that registers no identity module is stateless, as #986's + `provider = "none"` already means. +- Composition. The composition root resolves the selected provider for + each capability from the registry once at startup and places it in the + per-request services, so the request path is unchanged from #1043 and + #1044. Adapters stop injecting vendor providers directly (the + `ec_provider` slot on the runtime services builder and the injected + closures in `build_device_provider` and `build_geo_provider` go). Host + defaults are still supplied by the adapter as platform services and are + consumed by a built-in or a module through the request evidence and host + signal abstractions, exactly as now. A provider that needs a host signal + the running adapter does not expose is rejected at startup, as #986 + requires. +- A module that declares all three capabilities may share one backend call + per request across them, which is the shared-backend principle in + `CLAUDE.md`, and is the case that a split between a registry-attached + identity provider and platform-attached geo and device providers would + have made impossible. +- The host-signal device provider that #1044 ships as a separate crate is a + provider built on platform signals, so it registers as a module too. The + signals it reads stay platform. + +Effect on the series. #1043 and #1044 rework their construction and +selection path onto this section, move the HMAC and User-Agent-only +providers into module crates, and keep everything else. #1045, #1046 +and #1047 are unaffected beyond the rebase. + ## 4. Migration of the nine existing integrations One vendor per PR, after this change lands. Each moves its Rust, its @@ -157,13 +258,31 @@ TypeScript, its config type and its tests into `IntegrationSettings` is a flattened map that already accepts unknown vendor keys (`crates/trusted-server-core/src/settings.rs:166`). +Two more places every move must touch, found by mapping `main`: + +- `crates/trusted-server-core/src/migration_guards.rs` embeds every core + source file by relative path with `include_str!`, the thirteen vendor + files included, so a vendor move that leaves its entry behind breaks the + build rather than a test. The guard cannot derive its list from the + registrations, because `include_str!` paths are fixed at compile time, so + this change drops the nine vendors' files from the guard instead: a module + crate is outside the core neutrality guarantee, and a move then deletes + nothing there. +- The `ts audit` command carries its own vendor table (detection patterns + and configuration section names in + `crates/trusted-server-cli/src/commands/audit/analyzer.rs` and + `commands/audit/mod.rs`). It is outside the registry and outside this + change. Each vendor move takes its `ts audit` rows with it, and how the + CLI learns a vendor's detection pattern from a crate is a follow-up this + spec records but does not solve. + ## 5. What does not change The request pipeline, the hook traits and their order, the served script format and its hash, every `[integrations.*]` table, the permission model, -and the Edge Cookie, device and geo seams from PRs #1043 to #1046. No -integration changes behavior. A deployment that lists the same integrations -gets the same responses. +and the identity lifecycle, envelope and validation contracts from #986 as +implemented in PRs #1043 to #1046. No integration changes behavior. A +deployment that lists the same integrations gets the same responses. ## 6. Acceptance @@ -173,11 +292,17 @@ gets the same responses. right hash, runs its hooks in the right order, and is rejected on a duplicate id. A seam is only proven by an implementation that is not the built-in one. -2. **Parity.** The existing integration and parity suites pass unchanged, +2. **Capabilities round trip.** The same test integration declares an + identity, a geo and a device provider. With the three selectors naming + it, a request is served by all three (the minted identifier carries its + code, the resolved country and the device signals are its). With a + selector naming a module that lacks the capability, startup fails with + an error that names the module and the capability. +3. **Parity.** The existing integration and parity suites pass unchanged, because the built-in set still registers through the same path. -3. **No vendor left behind.** The rewritten deploy-validation test shows +4. **No vendor left behind.** The rewritten deploy-validation test shows every registered integration validates its configuration. -4. All CI gates in `CLAUDE.md`, on all four adapters. +5. All CI gates in `CLAUDE.md`, on all four adapters. ## 7. Risk @@ -185,23 +310,32 @@ The change is wide but shallow. It touches the registry, the served script path, deploy validation and four adapter entry points, and it changes no integration's behavior. The largest risk is the served script, where a mistake shows up as a wrong hash or a missing module, so §6's round trip -covers both. Doing this once is what removes the per-vendor core change +covers both, and the existing hash round-trip tests in `bundle.rs`, +`publisher.rs` and `tsjs.rs` pin every current `?v=` value. The second risk +is the renderer contract, where `BidRenderer::as_aps` is an exhaustive +single-arm match with eight test sites constructing the variant directly, +and the wire shape `{"type":"aps", ...}` must survive byte for byte. Doing this once is what removes the per-vendor core change that the project pays for today, most recently in PR #1054. ## 8. Sign-off -| # | Decision | Status | -| --- | ------------------------------------------------------------------------------- | -------------------- | -| 1 | Vendor integrations belong outside core, behind the registration contract | Proposed | -| 2 | Tech Lab engineering reviews vendor crates, and does not maintain them | Proposed, governance | -| 3 | A registration may carry its own browser JavaScript | Proposed | -| 4 | Deploy validation moves onto the registration | Proposed | -| 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | -| 6 | This change is complete in itself: after it, no vendor move needs a core change | Proposed | +| # | Decision | Status | +| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | +| 1 | Vendor integrations belong outside core, behind the registration contract | Proposed | +| 2 | Tech Lab engineering reviews vendor crates, and does not maintain them | Proposed, governance | +| 3 | A registration may carry its own browser JavaScript | Proposed | +| 4 | Deploy validation moves onto the registration | Proposed | +| 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | +| 6 | This change is complete in itself: after it, no vendor move needs a core change | Proposed | +| 7 | Identity, geo and device providers are capabilities of a module registration (§3.6), the #1043 review's rule applied to all three | Proposed | +| 8 | No provider is built into core: HMAC and the User-Agent-only device provider are Tech Lab-owned modules configured under `[integrations.]`, and core keeps only `none` | Proposed | +| 9 | This spec and its core implementation precede #1043; 51Degrees implements the core seam, the nine vendor moves in §4 stay one PR each | Proposed | ## Revision record -| Date | Change | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------- | -| 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | -| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | +| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | +| 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | +| 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | From 216596a7172f979cb56323520073ff4d9c56690c Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 28 Aug 2026 21:32:36 +0100 Subject: [PATCH 003/192] Record what implementing the integration seam found A probe integration built outside core and registered through an adapter exercised every seam end to end. Four things surfaced that reading the code did not: the operator CLI reaches deploy validation through a type that supplies no builders, so a vendor's own rules are skipped on the path an operator uses; a carried browser module's hash literal is hand-maintained and breaks on a checkout that rewrites line endings; a provider is resolved more than once per request, which the shared-backend principle needs a per-request context to fix; and one core reader still reads the APS renderer payload directly. Also records the construction-time verification of a carried module's declared hash. --- ...-08-27-integration-provider-seam-design.md | 85 ++++++++++++++++++- 1 file changed, 82 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index b86730f443..3736273928 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -133,8 +133,13 @@ module route `parse_single_module_filename` (`publisher.rs`), the `GPT_DIAGNOSTICS_INTEGRATION_ID` standalone special case, which becomes a registration property, and `template_fingerprint`, which must cover carried modules so a vendor crate rebuild invalidates the server-side template -cache. The served script keeps its cache rule, being the `?v=` query -matched at serve time (there is no integrity attribute on the tag today, and +cache. The registry verifies a carried module's declared hash against its source +when it is built, so a stale literal is a startup error rather than a +stale script served under a valid-looking URL. Covering carried modules in +the template cache fingerprint means the publisher entry point needs the +registry, so it takes the configuration and the registry as one argument +rather than two. The served script keeps its cache rule, being the +`?v=` query matched at serve time (there is no integrity attribute on the tag today, and this change adds none). ### 3.3 Startup validation on the registration @@ -317,7 +322,80 @@ single-arm match with eight test sites constructing the variant directly, and the wire shape `{"type":"aps", ...}` must survive byte for byte. Doing this once is what removes the per-vendor core change that the project pays for today, most recently in PR #1054. -## 8. Sign-off +## 8. What implementing this found + +A probe integration built outside `trusted-server-core` and registered +through an adapter exercised every seam end to end. Four things surfaced +that reading the code did not, and they are recorded here rather than left +for each vendor to rediscover. + +1. **A vendor's own deploy rules do not run through the operator CLI.** + `ts config validate` and `ts config push` reach validation through + `TrustedServerAppConfig`, which supplies no builders, so a vendor's + `[integrations.]` rules are skipped on exactly the path an operator + uses. The validation hook in §3.3 is only real once that path can carry + the builders a deployment was composed with. This needs a decision: + either the CLI is built per deployment with its vendor crates, or the + adapter validates at startup and the CLI checks only what core owns. +2. **A carried module's hash is hand-maintained and line-ending fragile.** + Core's own modules get their hashes generated at build time. A vendor + crate keeps a literal beside its `include_str!`, and on a checkout that + rewrites line endings the embedded file changes and the literal stops + matching, which fails startup on that machine only. A generated helper + or a documented build-script recipe removes the trap; the probe pins the + file's line endings and tests the literal, which every vendor would + otherwise have to reinvent. +3. **A provider is resolved more than once per request.** A proxy that + wants the resolved location calls the geo provider itself while the + request path has already called it. `CLAUDE.md`'s principle that a + vendor sharing one backend makes a single call per request needs a + per-request provider context to hang that on, which this change does not + introduce. +4. **One core reader still reaches into a vendor's payload.** The + `hb_adid` fallback in the publisher reads the APS renderer's fields, so + the APS migration needs a neutral answer for it rather than only the + seam in §3.4. + +5. **Request preparation covers different routes on each host.** Every + adapter runs preparers before routing, but not on the same set of + routes: one runs them on every route but the health check, one skips a + batch endpoint and its admin diagnostics deliberately, one covers three + of its paths, and one skips its inline admin stubs. A module that strips + its own reserved query or cookie is therefore protected on a different + set of routes depending on the host it is deployed to. Making that + uniform means routing each adapter's hand-written handlers through one + wrapper, which is worth doing before a vendor depends on it. + +6. **A module's validate function runs nowhere in a real deployment.** + Building the registry calls only a builder's build function, and the + operator CLI calls deploy validation without the builders, so a vendor + whose checks live in `validate` has them enforced on no path at all. The + probe works around it by repeating its check inside its build function, + which every vendor would have to copy. Either the registry runs + `validate` when it builds, or the operator path carries the builders, + and the second is item 1. This is the same gap as item 1 seen from the + other side, and together they mean §3.3 is not yet delivered in + practice even though the hook exists. +7. **The Fastly adapter cannot take a vendor crate at all.** Its + `build_state_with_registrations` is crate-private and it has no library + target, so composing a module into a Fastly deployment means editing the + adapter. The other three adapters expose both entry points. Fastly is + the primary deployment target, so this one decides whether the seam is + usable in production or only in the dev server. + +Items 1, 6 and 7 are the ones a vendor meets on its first day, and item 7 +decides whether any of this is reachable on the platform most deployments +use. Item 5 is the one that produces a bug report nobody can reproduce, +because whether it appears depends on which host the reporter runs. + +Taken together these say the seam is proven but not yet finished. A vendor +can register a module, ship its browser code, declare a geo provider and +serve a route, all from its own crate and proven end to end. It cannot yet +do that on Fastly, and its own configuration rules are not enforced +anywhere. Both are small changes against what this document already +defines, and both should land before the first vendor is asked to use it. + +## 9. Sign-off | # | Decision | Status | | --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | @@ -338,4 +416,5 @@ that the project pays for today, most recently in PR #1054. | 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | | 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | | 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | +| 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | From 62a01b654944657a8e070a179ea50d68cf02a246 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 30 Aug 2026 08:29:01 +0100 Subject: [PATCH 004/192] Move the provider series design specs ahead of their implementation The review of #1043 asked that spec changes land before the code that implements them, so a divergence is a decision taken in review rather than a ratification of something already merged. PRs #1043 to #1047 each carried the design document for their own step, and #1043 carried a 607-line spec describing device providers, geo providers, the permission model and the browser resolve endpoint, none of which is in that PR. Move all six series documents here, so this PR carries the complete normative set and no code: - 2026-07-30-pluggable-providers-design.md (from #1043) - provider-code-registry.md (from #1043) - 2026-07-30-permission-model-design.md (from #1045) - 2026-07-30-client-cycle-ec-resolve-design.md (from #1046, later revised by #1047) - 2026-07-30-integration-response-header-hook-design.md (from #1047) - 2026-07-30-provider-migration-rollout-design.md (from #1047) Each file is taken verbatim at the tip of the stack, so the later revisions are preserved: the provider-switching continuity section, the geo requires-signal floor, and the code-envelope paragraph #1047 added to the client-cycle spec. The revision-record tables are unchanged. No document's substance was edited. The only edits are to this spec's own status line, which said the PR adds one document and that the series specs land with #1047, and a revision-record row recording the move. --- ...26-07-30-client-cycle-ec-resolve-design.md | 260 ++++ ...integration-response-header-hook-design.md | 1219 +++++++++++++++++ .../2026-07-30-permission-model-design.md | 826 +++++++++++ .../2026-07-30-pluggable-providers-design.md | 641 +++++++++ ...07-30-provider-migration-rollout-design.md | 516 +++++++ ...-08-27-integration-provider-seam-design.md | 10 +- .../specs/provider-code-registry.md | 30 + 7 files changed, 3498 insertions(+), 4 deletions(-) create mode 100644 docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md create mode 100644 docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md create mode 100644 docs/superpowers/specs/2026-07-30-permission-model-design.md create mode 100644 docs/superpowers/specs/2026-07-30-pluggable-providers-design.md create mode 100644 docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md create mode 100644 docs/superpowers/specs/provider-code-registry.md diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md new file mode 100644 index 0000000000..6949889a38 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -0,0 +1,260 @@ +# Design Spec: Client-Cycle Edge Cookie Providers and the Resolve Endpoint + +**Status:** Implemented (hardened v1) in PR #1046, on the threat model below. +The full anti-replay reservation machinery (§3.9) and the vendor envelope +verification it serves are deliberately not in v1: they land with the first +real vendor scheme, which brings the concrete envelope format the +reservation design must fit. §8 records exactly what v1 implements, what it +defers, and why the feature is normative rather than deferred. +**Author:** Engineering (revised against the implementation, 2026-08-25) +**Issue references:** #778 (series), successor spec of the 2026-07-31 draft +**Related specs:** `2026-07-30-pluggable-providers-design.md` +**Last updated:** 2026-08-25 + +> **Context.** PR #838 shipped, undeclared and unspec'd, a second provider +> _type_: a "client-cycle" EC provider whose identifier is established by a +> browser POST to a new public endpoint (`POST /_ts/api/v1/ec/resolve`), +> plus a demo provider (`client-fixed`) and a JS bundle. Review found the +> endpoint accepted cross-origin identity-setting posts with no origin +> check, minted cookies with no identity-graph row (violating an invariant +> the organic path enforces explicitly), was registered on only one of four +> adapters, and could never round-trip because the core did not recognize +> non-HMAC identifiers. None of that is an argument the feature is a bad +> idea. Vendor identity systems with a browser leg (for example +> signed-envelope schemes) are a real integration target, and for this +> project the client-side path is the preferred route for the first vendor +> integration. It is an argument that the feature needs a threat model +> before an implementation. This spec is that threat model; PR #1046 is the +> implementation measured against it. + +--- + +## 1. Overview + +A **client-cycle** EC provider establishes the identifier via a browser +round trip: server-injected first-party JS obtains or derives a value in the +page (typically a signed envelope from a vendor identity system), posts it to +a Trusted Server endpoint, and the endpoint, after provider-specific +verification, sets the first-party `ts-ec` cookie. + +This differs from server-side providers in one security-critical way: **the +identifier is attacker-influenceable input**, not server-derived evidence. +Everything in this spec follows from that. + +Why this feature is normative in the series rather than deferred: the first +vendor integration this project targets works client-side by design, since the +page script talks to the vendor's identity system and hands the result to +the edge, so the server-side path alone cannot carry it. The 2026-07-31 +draft deferred the feature for lack of a consumer; the consumer now exists +as a planned vendor provider, and v1 builds the endpoint that provider will +verify against. + +## 2. Threat model + +| Threat | Vector | Consequence if unmitigated | +| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Cross-site identity fixation** | `text/plain` POST is a CORS-simple request: any page on the web can `fetch(resolveUrl, {method: "POST", credentials: "include", body: payload})` with no preflight | An attacker pins a chosen identity onto a victim's first-party cookie jar, a login-CSRF for the ad-identity layer, and the victim's activity accretes to an attacker-controlled ID | +| **Replay** | A captured valid payload (from the attacker's own session or a leak) replayed against another browser | Same as fixation, without needing to mint payloads | +| **Phantom identity** | Endpoint sets the cookie without an identity-graph row | Later requests carry an EC that the KV graph has never seen; downstream sync and withdrawal logic operate on an identity that half-exists (the organic generation path explicitly refuses to write a cookie when the graph write fails, for exactly this reason) | +| **Un-tombstoneable identity** | Core does not recognize the provider's identifier shape | Withdrawal cannot expire or tombstone the identity, which is a compliance failure and not only a bug | +| **Amplification** | The page script cannot observe an HttpOnly cookie, so it cannot know the cookie is already set | A POST on every page view of every session (PR #838's JS gated on reading a cookie its own server marked HttpOnly, making the guard permanently false) | + +## 3. Requirements on the endpoint + +`POST /_ts/api/v1/ec/resolve` MUST: + +1. **Reject cross-site requests with an origin check.** v1 authorizes a + request only when its `Origin` header names the publisher's configured + domain or a subdomain of it; a missing or foreign `Origin` is rejected + with `403`. Browsers always send `Origin` on POST `fetch`, so its + absence means a non-browser caller, which has no business on a + page-script endpoint. The 2026-07-31 draft asked for an exact origin + allowlist as new configuration plus an optional session-bound CSRF + token; v1 derives the allowed set from `publisher.domain` (existing, + operator-trusted configuration) and admits the publisher's own + subdomains, because the publisher controls their subdomain namespace, and the + draft's sibling-subdomain concern applies to `Sec-Fetch-Site: +same-site` (which v1 does not consult at all), not to a suffix match on + the publisher's own apex. An explicit multi-origin allowlist remains + open (§7.5) for publishers whose pages run on domains other than the + configured apex. +2. **Verify the payload per provider.** The endpoint hands the posted + payload to the selected provider's `resolve_from_client` and mints only + what the provider returns; whether the payload is trustworthy is the + provider's responsibility, stated on the trait. A real vendor provider + verifies a signed, audience-bound, expiring envelope; the draft's + session-binding and replay analysis (see §3.9) is the bar that + verification must clear when the vendor scheme lands. The demo provider + verifies a fixed constant and is compiled out of production builds + (§5). `resolve_from_client` is normative in v1, with a no-op default so + server-side providers are untouched. +3. **Preserve the identity-graph invariant.** Implemented: the graph row is + written before the cookie is set, keyed by the provider's + `normalize_id_for_kv` canonical form, exactly like the organic mint + path. No graph available → no mint (`204`), same as organic generation; + a graph write failure → `503`, no cookie. +4. **Round-trip through the lifecycle contract.** Implemented: read-back + goes through the selected provider's `accepts_id`, the KV key through + `normalize_id_for_kv`, and withdrawal reaches the row like any other + identity. A round-trip test drives an opaque client identifier through + organic deferral, resolve, cookie set, and verbatim read-back. +5. **Exist on every adapter, where parity means identical behavior, + including identical refusal.** Partially implemented, documented: the + Fastly adapter routes the endpoint (passing the same bot-gated identity + graph as organic generation, so unrecognized clients cannot mint + through resolve either). The Axum, Cloudflare, and Spin adapters + deliberately do not route it, matching `identify` and `batch-sync`, + which need the same platform KV wiring those adapters do not have; the + route list in the Spin adapter documents all three together. The + draft's stronger ask, identical startup rejection of the client-cycle + selection on adapters that cannot serve it, is the agreed follow-up + when the portability adapters gain KV (§7.4). +6. **Be uncacheable and permission-gated on the provider's full + declaration.** Implemented: every response carries `Cache-Control: +no-store`, and the gate is the selected provider's complete + `required_permissions()` through the same resolved permission state as + organic minting, not a hard-coded storage check. +7. **Bound every input.** Implemented: request body at most 65,536 bytes, where + an advertised `Content-Length` over the limit answers `413` before the + read, and the read body is re-checked so a missing or false length does + not bypass the bound. `Content-Type` allowlist: `text/plain` and + `application/json`, matched on the media type alone, case-insensitively, + ignoring parameters (the browser's default `text/plain;charset=UTF-8` + passes); anything else → `415`. The minted identifier must fit the + global identifier bounds (at most 256 bytes, cookie-safe alphabet, + shared with every other mint path); violation → `400`, never a rewrite. + Core then applies the provider's registered code envelope + (`provider-code-registry.md`): the cookie and the identity-graph key + carry `{code}~value`, so a client-set identity is namespaced to its + provider exactly like an edge-minted one (the demo's cookie value is + `cfix~an-ec`). Status codes are part of the contract: `400` out-of-bounds identifier, + `403` origin rejection, `409` different-identity conflict, `413` body, + `415` content type, `503` graph-write failure, `204` closed gate / no + provider / no graph / unverified payload. Tests exercise each rejection. +8. **Define behavior against an existing identity, with no silent + replacement.** Implemented: resolving to the same identity refreshes + idempotently; resolving to a different identity while the request + carries a recognized EC is rejected with `409`. Any legitimate + re-identification flow (account link, vendor migration) is an explicit + linking design this spec does not authorize (§7). +9. **Replay consumption and graph persistence as one idempotent + sequence.** Not in v1, by decision rather than omission. The draft's + reservation design (atomic single-key CAS reservation with owner hash, + lease epochs, fenced transitions, and family-epoch revocation + linearization) presupposes a payload with a unique id, a session + binding, and a validity window, which are properties of a concrete vendor + envelope format that does not exist yet, and a CAS-class storage + primitive no production adapter exposes today. Designing the + reservation against a hypothetical envelope would repeat the mistake + this series exists to fix. v1's stance: the endpoint is safe without it + for the providers v1 ships (the demo mints a constant, feature-gated + out of production), and the reservation lands with the first vendor + scheme, designed against its real envelope, with the draft's §3.9 as + the starting bar. Until then the draft's text is preserved below as the + agreed requirement. + +The 2026-07-31 draft's §3.9 reservation requirement is retained verbatim as +the bar for the vendor-scheme implementation: + +> Neither naive order works: consume-the-nonce-first makes a subsequent +> graph failure unretryable (the token is spent, the identity never +> existed); graph-first lets the losers of a replay race leave residual +> rows. Required shape: consumption is an atomic single-key reservation +> (CAS) keyed by the payload's unique id, with explicit states: `pending` → +> `committed` | `failed`, each carrying an owner hash (the session binding) +> and a monotonic lease epoch. Takeover of an expired `pending` lease +> increments the epoch, and every state transition is a fenced CAS on +> (state, epoch). `failed` is retryable by the same owner at a higher +> epoch. The graph write happens under the reservation and is deterministic +> under its key. A duplicate must never receive the cookie unless it proves +> the original session binding; a requester matching the reservation's +> owner hash has the `Set-Cookie` re-emitted (lost-response recovery), +> anyone else gets a terminal response with no cookie. The same-identity +> no-op first checks the family revocation record, and "revocation wins" is +> enforced by a CAS conditioned on the family epoch read at the start. + +## 4. Requirements on the page script + +- **The re-post guard must not depend on reading an HttpOnly cookie.** + Implemented as the draft's first option: the resolve response sets a + non-HttpOnly companion marker cookie (`ts-ecr=1`) carrying no identity, + which is the only signal the page has that a resolve succeeded. The + marker shares the Edge Cookie's scope and lifetime and is expired + together with it on withdrawal, so a visitor who later re-establishes + the permission can resolve again. +- **The page leg is permission-gated before vendor contact.** The demo + module contacts no vendor (it posts a constant), so the draft's + injection-time and live-CMP gating requirements bind the first vendor + module, not v1: a vendor module must not derive identity or contact the + vendor for a visitor whose resolved permissions do not satisfy the + provider's declaration, must re-check immediately before vendor contact + (consent can change between document delivery and asynchronous vendor + contact, including BFCache restoration), and its injection is keyed off + the provider selection exactly as the demo's is today. +- The JS module ships through the standard integration bundle mechanism, + loaded only when a client-cycle provider is the selected EC provider. + The interaction between provider-keyed bundle content and content-hash / + SRI pinning remains open (§7.6). +- **Any constant shared between Rust and TS is asserted equal by a test.** + Implemented: a Rust test reads the page-script source and asserts the + fixed word and the marker cookie name match their Rust constants, so a + rename on either side fails the build instead of silently breaking the + round trip. + +## 5. Demo providers + +Implemented as required: the `client-fixed` demonstration provider (fixed +identifier, constant-equality verification) is compiled only behind the +`client-fixed-demo` cargo feature. In a production build the settings +validator rejects the selection at startup with a direct message, and the +provider builder rejects it again as defense in depth. A fixed shared word +is not an identity; the demo exists to exercise verify-before-mint end to +end in tests and demonstrations. + +## 6. Testing + +- Endpoint unit tests cover: origin rejection (missing and foreign), + subdomain acceptance, content-type rejection, the body bound, the + identifier bound, the different-identity conflict, the no-graph refusal, + the closed permission gate, the unverified payload, and the marker, + cache-control, and graph-row effects of a success. +- A round-trip test drives a client identifier through organic deferral, + resolve, cookie set, and verbatim read-back recognition. +- The cross-language constant test pins the endpoint's shared constants to + the page-script source. +- Remaining for the vendor scheme: a real-browser integration round trip + (JS → POST → Set-Cookie → next request recognized) in the browser suite, + and the §3.9 reservation tests (crash-between-steps, lease takeover, + concurrent duplicates, owner-hash recovery, resolve-vs-revocation + races). + +## 7. Open questions, carried to the vendor-scheme issue + +0. The commit path spans keys (reservation, identity row, family-epoch + CAS): the cross-key atomicity or saga/compensation design is undefined, since + the single-key CAS steps are specified, their composition is not. +1. The first vendor scheme's envelope format, measured against §3.2 and + §3.9 (audience binding, expiry, unique id, session binding). +2. Whether the resolve flow needs consent-state echo in its response, and + the minimal disclosure if so. +3. Rate limiting / abuse posture at the edge for an unauthenticated POST. +4. Startup rejection of the client-cycle selection on adapters that cannot + route the endpoint, once the portability adapters gain platform KV. +5. An explicit multi-origin allowlist for publishers whose pages run on + domains other than the configured apex (v1 authorizes the apex and its + subdomains). +6. How JS module selection keyed off EC provider configuration coexists + with content-hashed/SRI-pinned bundles, meaning per-config hashes, cache + keying, and the config-push story for them. + +## 8. Revision record vs the 2026-07-31 draft + +| Draft position | v1 (PR #1046) | Why | +| ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | +| Feature deferred, `resolve_from_client` de-normalized | Feature normative; trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | +| Origin check via new allowlist config + CSRF token | Publisher apex + subdomains from existing `publisher.domain`; missing/foreign `Origin` → `403` | Uses existing operator-trusted configuration; explicit allowlist stays open for multi-domain publishers (§7.5) | +| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme; draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | +| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph); portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes; startup rejection follow-up recorded (§7.4) | +| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option; testable and observable | +| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | +| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md new file mode 100644 index 0000000000..0182495fbd --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -0,0 +1,1219 @@ +# Design Spec: Integration Response-Header Hook + +**Status:** Not implemented in the series. PR #1047 carries the +documentation set only; the hook was removed from the earlier draft of that +PR because it has no consumer, which is this spec's own §-rule for +speculative surface. The spec is retained as the design bar for the hook +when its first consumer arrives (an integration that must set response +headers such as Accept-CH or detection results). Revised 2026-08-25. +**Author:** Engineering +**Issue references:** #782 +**Related specs:** `2026-07-30-pluggable-providers-design.md` and the +baseline DataDome design +`2026-06-11-datadome-server-side-protection-design.md` +**Last updated:** 2026-08-25 + +> **Context.** Issue #782 already specifies this feature well; its done-when +> is the contract. PR #838 shipped the trait and registry wiring with **no +> adapter call site**. `apply_response_headers` had zero production +> callers, so the feature existed only in its own unit test. This short spec +> restates the contract plus the two details the issue left open (ordering +> and collision policy), and adds the rule that prevents a repeat, which is +> that the hook lands with a consumer or not at all. The implemented series +> (PRs #1043-#1047) applied that rule to this spec itself, see the Status +> above and §7. + +--- + +## 1. Overview + +The pre-existing DataDome design remains unchanged. For implementations +governed by this spec, §4a is the normative PR-specific delta and supersedes +that baseline wherever its generic request/effect API, header ordering, +session-by-header behavior, endpoint/request surface, cookie lifecycle, +challenge transport, or response-pointer behavior conflicts. The baseline +continues to provide historical context; it is not a second normative source +for those surfaces. + +Integrations can today rewrite request-path behavior (proxies, attribute +rewriters, head injectors), and a request filter can name response headers +to set through `RequestFilterEffects.response_headers`, but no integration +can inspect a response and decide its headers from it. The hook will add +that capability. An integration registers a response-header mutator +via its `IntegrationRegistration` builder, and every adapter applies all +registered mutators to the outbound response for HTML document responses it +processed. + +## 2. Contract + +- `IntegrationRegistration::builder(ID).with_response_mutator(...)` registers + a mutator; `IntegrationRegistry::apply_response_headers(...)` applies all + registered mutators in registration order. +- Every registration carries a nonzero `behavior_revision: u32`, bumped for + any change to its response decision, operation semantics, declared read set, + or security field list. Integration IDs match + `[a-z0-9][a-z0-9-]{0,63}` and are unique. The registry revision hashes the + ordered registration list. Order is behavior, so the array is never sorted. +- **The mutator API is structured operations, not header-map access.** A + mutator returns (or is handed a recorder for) typed operations, + `append(name, value)` and `replace(name, value)` (v1 is headers-only; + the cookie operation arrives with the deferred cookie surface, §3), + which **core validates and applies**, + attributing each to its integration id. PR #838's shape handed the + integration an unrestricted `&mut HeaderMap`, which makes §3's collision + policy unenforceable by construction: core cannot validate or attribute + writes it never sees. An API that cannot express a violation beats one + that promises to catch it. +- **Every adapter calls the apply point** on its outbound-response path for + processed documents. The call site lives in shared response-finalization + code where one exists; where adapters finalize independently, each adapter + gains the call and a test proving it. +- **Ordering is three stages, and the last one is inviolable:** core + response-header handling (EC Set-Cookie emission, EC header clearing, + privacy headers) → integration operations → **final cache/privacy + invariant enforcement**, which no integration operation can override. + Running the hook dead-last would be wrong: current `main` deliberately + runs cookie-cache protection _after_ arbitrary header changes, stripping + surrogate caching and forcing private/no-store on any response that sets + a cookie, a hook applied after that recheck could combine an appended + `Set-Cookie` with a replaced public `Cache-Control` into a + **shared-cacheable cookie response**. The invariant pass therefore runs + after all mutations, unconditionally, and it enforces more than the + cookie rule. Core **snapshots the complete pre-hook cache restriction + state**, whether the restriction came from core's own classification + (processed auction HTML is marked private even when no cookie is + emitted; today's final helper returns early without `Set-Cookie`) **or + from the origin** (an origin-supplied `private, no-store` that core + merely passed through), and the post-hook response may only be + **equal or stronger** on the privacy axis: integrations can tighten + caching, never loosen it, regardless of which header they replaced. + "Equal or stronger" is a defined merge over **independent sticky + directives, not a totally ordered lattice**, because `no-cache` and `private` + are orthogonal constraints (RFC 9111: `no-cache` permits shared + storage subject to revalidation; `private` forbids shared storage), so + "replace `private` with the stronger `no-cache`" would make a + personalized response shared-storable. The merge: each of the **six sticky directives**, `no-store`, `no-cache`, + `private`, `must-revalidate`, `proxy-revalidate`, `no-transform`, is + present-in-snapshot-or-mutation ⇒ present-in-final (this + "snapshot or mutation ⇒ final" rule scopes to exactly these six), with two refinements. First, `must-understand` is the deliberate + exception: **mutation-introduced `must-understand` is rejected** + (snapshot-present survives untouched), because under RFC 9111 + §5.2.2.3 a cache that understands the status may then ignore an + accompanying `no-store`, "adding" it weakens a stored `no-store`, so + it is not additive. Second, stickiness is **form-preserving, not + name-presence-only**: an **unqualified directive never becomes + field-qualified, and a qualified field set never shrinks**, where + `private` → `private="Set-Cookie"` keeps the directive name while + authorizing shared storage of everything but one field (RFC 9111: + qualified `private`/`no-cache` have materially weaker semantics), so + a mutation supplying a qualified form where the snapshot is + unqualified keeps the snapshot's bare form, and qualified snapshot + sets may only grow; `public` is + dropped whenever any restriction is present; **request-side authority + is part of the invariant**. If the request carried `Authorization` + and the origin did not itself authorize shared reuse (no `public`, + `must-revalidate`, or `s-maxage` from the origin), an integration may + not introduce `public`, `must-revalidate`, or `s-maxage` (RFC 9111 + §3.5 makes those the very directives that unlock shared caching of + authenticated responses); the invariant pass forces `private, +no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may + only shrink relative to the snapshot; `stale-while-revalidate`/ + `stale-if-error` may appear only if the snapshot had them **and their + durations may only shrink** (present-at-1s must not become + present-at-1y); CDN-specific cache fields are **reserved outright, by enumerated + name in the field registry**, `Surrogate-Control`, + `CDN-Cache-Control`, `Cloudflare-CDN-Cache-Control`, and + `Edge-Control`, each individually tested ("host equivalents" was not + a matching rule four adapters would implement identically). + Merging them per-directive on unrestricted responses was a hole (an + unrestricted `CDN-Cache-Control: max-age=60` could become a year), and + they are additionally stripped from any restricted response; and the + final `Vary` is the **union of the complete snapshot `Vary` set**, + origin-supplied members included, not only core-required ones, and + the mutation. Ordering is normative so the final `Vary` reaches TS's + own cache key, not just the wire: **mutation/invariant → final `Vary` + computation → cache-key construction → body/metadata commit**, with + three identity rules. Cache matching uses the **exact final publisher + request** (post-overlay view; keying from the redacted view would + collapse personalized variants). + + The final `Vary` name list has one cross-adapter grammar. Core collects every + `Vary` response field line from the snapshot plus mutation, parses each as an + HTTP comma-list, trims optional whitespace around every member, and requires + each non-empty member to be an RFC field-name token. It lowercases ASCII, + removes duplicates, and sorts unique names by unsigned ASCII byte order. An + empty member or invalid token introduced by a mutation invalidates that + batch. If the reverted snapshot itself is malformed, the invariant replaces + the final value with `Vary: *`, forces `no-store`, and writes no cache + artifact. `*` in either source likewise dominates every other member: the + normalized result is the single `*` and is uncacheable. For an ordinary + list, the wire response emits one lowercase `, `-joined value, while the + variant descriptor stores `vary_names` as the exact sorted JSON array of + lowercase strings. Thus field-line grouping, case, and input ordering cannot + produce different cache identities. + + A normalized name nominates one request header. Digest construction obtains + **all** values for that header from the exact final publisher request after + overlay, preserving received field-line order and value octets; it does not + comma-fold, trim, or split those request values. The `` bytes in the + HMAC input below are always the normalized lowercase ASCII name, and a name + is digested once even if it appeared repeatedly in `Vary`. Known-answer + normalization fixture: snapshot/mutation lines `Vary: X-Tenant , + Accept-Encoding` and `Vary: accept-encoding` produce wire value + `accept-encoding, x-tenant` and descriptor value + `["accept-encoding", "x-tenant"]`, then digest each nominated request + field's original instances in their received order. Fixtures also cover + differently grouped lines, case variants, duplicate names, empty members, + invalid tokens, `*`, absent versus present-empty request fields, and one value + containing a comma. + + **Every** `Vary`-nominated request value is stored **only as a keyed + digest**, every value, not a sensitivity classification an unknown + credential field could slip past: HMAC-SHA-256 with domain tag + `tsvry1|`, over an input that encodes **presence, instance count, and + length-prefixed octets**, since the earlier comma-join grammar had + deterministic collisions (absent hashed identically to + present-but-empty, violating RFC 9111 §4.1's absence-matches-only- + absence, and two members `a`,`b` collided with one member `a,b`), + which could select a representation built under a different + credential or tenant. Absent field → `tsvry1||a`; present → + `tsvry1||p|` with `count` as ASCII decimal, then, per member in received order, + `|:` with `len` the ASCII-decimal byte count. Output + lowercase hex (64 chars). **The key is a deployable contract, not an + implementation detail**: the setting + `[cache] vary_digest_key_secret_name` names a platform-secret-store + entry containing one versioned keyring JSON object: + `{ "schema_version": 1, "current_key_id": "", "keys": + [{ "id": "", "key_base64url": "" }] }`. `keys` is + sorted by `id`, contains 1..4 unique entries, rejects unknown fields, + and every value decodes to exactly 32 CSPRNG bytes. An id is derived, + not operator-invented: the first 16 lowercase hex characters of + SHA-256 over the raw key bytes; a supplied mismatch or one id bound to + different bytes is fatal. The current id must exist in the array. + Every cache entry stores its id; raw keys never enter config, cache + artifacts, logs, or metrics. Startup **fails** when response caching is + enabled and the keyring/current key does not resolve (digests are never + computed unkeyed). + + Lookup uses a stable per-representation **variant index** so the key ID is + discoverable before the variant artifact is addressed. The base index key is + the cache tuple excluding `Vary` values. Each bounded index descriptor stores + only the normalized final `Vary` name list, key ID, corresponding keyed + digests, artifact key, artifact expiry, and artifact revision tuple, never + raw request values. A reader loads the index, groups descriptors by key ID, + resolves each referenced key (one atomic keyring refresh if an ID is + unknown), recomputes the digests from the exact final publisher request, and + fetches only a descriptor whose complete name/digest tuple matches. A + still-unknown ID, malformed descriptor, missing artifact, expiry, or revision + mismatch is a miss for that descriptor, never a comparison under another + key. Multiple matching descriptors are corruption and make the entire base + lookup a miss with a metric; index order never chooses a winner. + + Publication writes and verifies the immutable artifact first, then CAS-adds + or replaces its complete descriptor in the index. Rekeying or a changed + `Vary` set inserts the new artifact/descriptor before removing the old + descriptor; a crash may leave a safely unreachable artifact or two + nonmatching descriptors but cannot point to a partial artifact. Index + capacity eviction removes expired descriptors first and otherwise the + least-recently-used complete descriptor; it never rewrites a digest under a + new key ID. This is the one meaning of “insert-new-then-index-update rekey” in + the capability matrix and 304 rules. + + Rotation atomically replaces the secret entry with a new valid keyring + containing the new current key **and all still-live previous keys**. Fleet + propagation may be mixed only in the safe direction: a process with the old + keyring can write/read the old id; the variant index exposes that id before + lookup, so a process seeing an unknown descriptor id refreshes the keyring + once, then treats a still-unknown descriptor as a cache miss. It never + guesses, probes with the current key, or computes unkeyed. A previous key may + be retired only after no unexpired index descriptor references it **and** the + maximum processed-artifact lifetime plus the adapter's qualified keyring + refresh bound has elapsed since it stopped being current. Key IDs are never + reused. Secret replacement atomicity, maximum propagation/refresh time, and + unknown-id refresh are explicit adapter capability cells and startup gates; + an unqualified adapter disables response caching rather than weakening the + grammar. Cross-adapter fixtures cover old→mixed→new propagation, unknown-id + refresh/miss, premature retirement rejection, and id/material mismatch. + Known-answer vectors under the all-zero 32-byte test key: + `tsvry1|authorization|p|1|10:Bearer abc` → + `c880c5e8c36febc0b1581c92f1d598fded34391626e67372ed63b2857d8a7b6b`; + absent-field form `tsvry1|x-tenant|a` → + `a2ae26cf529a5843a25f1448acc4e90016d4c1dce0ffda5662e3ac459433e1ab`. And + a response derived from a request carrying an **identity-bearing TS + overlay** (the DataDome ClientID overlay) is forced `private, no-store` + unless an explicit per-overlay contract says otherwise, because + the `Authorization` rule protects origin credentials and this rule + protects the identity TS itself injected. Parsing itself is a **shared core + parser with fail-closed normalization**, not four adapter interpretations: + a `Cache-Control` value that fails the shared grammar has an + **enumerated result, not a "most restrictive reading"** (restrictions + are independent axes, so no single most-restrictive point exists): + the response is treated as **uncacheable for the storage decision** + (`no-store`-equivalent in the invariant) and any mutation batch + merging against the malformed value is rejected whole; among + well-formed values, duplicate directives keep the strongest; quoted and unquoted + forms are equivalent; conflicting `max-age` values keep the smallest; + unknown extension directives are dropped **from mutations only, while + unknown directives already in the snapshot are preserved verbatim** (a + downstream cache may honor a restrictive extension TS does not + recognize; dropping it would weaken origin policy, RFC 9111 §5.2.3); + `Expires` participates in the freshness bound via **RFC 9111 §4.2.1's + freshness-lifetime algorithm, referenced directly**: the + `Expires`-derived lifetime is `Expires − Date` (absent `Date` → + response receipt time), invalid or duplicate date values are treated + as already expired (the RFC's conservative option, chosen + normatively), and `Age` is handled per the RFC, effective freshness + is the minimum across `max-age`, `s-maxage`, and that derived + lifetime and a mutation may **not introduce `max-age`/`s-maxage` + where the snapshot supplied no upper bound**, HTTP prefers `max-age` + over `Expires` (RFC 9111 §5.3), so introducing one would override an + origin's shorter or already-expired `Expires`; and `Vary: *` is + treated as uncacheable-by-shared-caches (no-store-equivalent for the + invariant). Conformance fixtures cover each rule. Middle-stage placement also keeps + the earlier property: an integration mutation is not silently stripped + by ordinary core handling, only by the invariant pass, which logs the + downgrade it applies. + +## 3. Collision policy + +- Mutators may not touch **reserved surface**, which is defined at two + granularities because `Set-Cookie` is multi-valued: (a) reserved header + _names_, HTTP framing and hop-by-hop headers (`Content-Length`, + `Transfer-Encoding`, `Connection`, `Trailer`, `Upgrade`, `TE`, + `Keep-Alive`), **freshness metadata** (`Age`, `Date`, `Expires`, since replacing `Age: 59` + with `Age: 0` on a cached `max-age=60` response, or pushing `Expires` + into the future, extends downstream freshness in exactly the way the + monotonic merge forbids for `max-age`, so these are reserved outright + rather than merged), **representation headers coupled to body bytes + the hook cannot see** (`Content-Encoding`, `Content-Range`, + `Content-Type`, `ETag`, `Last-Modified`, `Accept-Ranges`, and digest + headers, since + relabeling uncompressed bytes as Brotli, or advertising a validator or + digest for bytes the hook never saw, corrupts responses or poisons + caches), the `x-ts-*` namespace, and the + consent/privacy headers core emits; (b) reserved cookie _names_ within `Set-Cookie`, `ts-ec`, + `ts-eids`, and the other `ts-*` cookies core owns. **Cookie operations are deferred out of the v1 hook, headers only.** + The write-side gate alone ("persistent cookies require P1") was shown + insufficient: it never modeled reading, using, forwarding, or + withdrawing the cookie, so a P1-granted-then-withdrawn integration + cookie would keep arriving on every request with nothing required to + expire, hide, or stop egressing it, and an advertising-identifier + cookie needs P4 the contract never expressed. Rather than ship + "inside the permission model" as a claim the model does not back, + `append_set_cookie` and the typed cookie builder are **deferred** to a + follow-up spec whose entry bar is: declared per-cookie required + permissions, a typed authorized request-side view, stripping from + unauthorized integration/proxy inputs, mandatory expiry on destructive + P1 withdrawal, and startup-unique (name, domain, path) ownership. + Integration IDs are **startup-unique, enforced**: registry + construction rejects a duplicate ID (current code silently coalesces, + which corrupts attribution and budgets), with a duplicate-ID test in + the done-when. The registration's `behavior_revision` follows §2's bump + contract; configuration-dependent behavior is captured separately by the + effective-config digest. Model-only activation is separate from both, so the + **one cache revision tuple** contains exactly + `integration_registry_revision`, `effective_config_revision`, + `active_policy_digest`, `active_policy_ordinal`, `model_epoch`, + `activation_generation`, and `hook_invariant_revision` from the strong active + tuple at publication. Every processed artifact, mutation IR/read-set bundle, + variant descriptor, and variant-index update stores that complete tuple; a + lookup, local conditional, HEAD update, or 304 replay requires byte-for-byte + equality with current active. In particular, the `permissions_v2` model CAS + misses every `pre_epic_v1` artifact even though config/policy bytes did not + change. Core also carries a nonzero + `HOOK_INVARIANT_REVISION: u32`, bumped for every parser, merge, budget, + cache-artifact, or invariant semantic change. The cache tuple stores all + fields above, so a deploy cannot silently reuse old finals while a new + restriction waits for cache expiry. Until then the + operation set is headers-only, and `Set-Cookie` is fully reserved. + Violations are rejected at the operation layer (§2) and + logged at `warn` with the integration id. The reserved lists are single + constants next to the definitions they protect, not duplicated in the + hook. +- For non-reserved headers, the mutator API distinguishes **append** from + **replace** explicitly; append/replace legality comes from a **core-owned field registry**, + not adapter judgment, and the v1 registry admits **inert fields + only**: "headers-only" is not automatically permission-neutral, since + `Link` (preload/prefetch), `Reporting-Endpoints`/NEL, CSP report + directives, and `Refresh` cause browser-initiated vendor contact on + requests that granted nothing. Fields with active egress side effects + are **rejected in v1**; a follow-up may admit them behind declared + required permissions gated at mutation time. Within the inert set, + each field is classified append-legal (genuinely list-valued), + replace-only (true singletons, e.g. `Content-Location`, `Retry-After`; + an earlier draft miscited `Content-Language`, which is list-valued), + or rejected; **unknown extension fields are rejected entirely in v1** + (neither append nor replace, their side-effect class is unknowable). + The v1 registry is enumerated here, not delegated: **admitted**, + `Cache-Control` (monotonic merge per this section), `Vary` (union + merge), `Content-Language` (append), `X-Robots-Tag` (append), + `Retry-After` (replace-only), `Content-Location` (replace-only); + everything else known is classified reserved or rejected by the rules + above, and growing the admitted set is a spec change to this list (cookies: see the §3 deferral above). Replacing a + header the origin set is a deliberate act, visible in the mutator's code. +- Later registrations see earlier mutations (order = registration order, + which is deterministic). +- **Operation-layer hygiene:** generic `append`/`replace` reject the + `Set-Cookie` header name outright, cookies go only through + the deferred cookie builder (when it exists), so cookie validation + cannot be bypassed by spelling + the header name in a generic op. Per-integration limits bound total + operations (≤ 32), added headers (≤ 16), and added bytes (≤ 8 KiB), and + a **cumulative final-response budget** (≤ 128 headers / ≤ 32 KiB total, + counting `name: value` plus separators, within any lower adapter + ceiling, and those ceilings are the enumerated capability cells **below**, + with the counting rule fixed exactly: counted bytes = Σ over emitted + fields of `len(name) + 2 + len(value) + 2` (the `": "` and CRLF + separators), validated against core's budget at startup so a batch + that passes core can never fail only on one adapter; a snapshot + already **over** the core budget before any mutation rejects every + ordinary batch, the budget bounds additions and never bricks an + over-budget origin response, which passes through and is counted; + security follows the replacement/reserve rule below) + bounds the sum across integrations. The budget has normative priority + partitions: ordinary mutators may consume at most **112 headers / 24 KiB**, + reserving 16 headers / 8 KiB for the core-owned security channel. Ordinary + batches remain registration-ordered within their partition. A security + `Continue` batch uses the reserve and, if necessary, evicts whole accepted + ordinary batches in reverse registration order until it fits; it never + removes half a batch and never drops origin fields. Ordinary output can + therefore never crowd out security effects. If the immutable origin head + plus the security batch alone exceeds the full budget, the security batch + is rejected atomically and the request follows the documented security + fail-open path with a dedicated metric, origin fields are not silently + sacrificed. A security `Respond` owns a replacement + response: all ordinary mutation batches are discarded and the challenge + is validated against the full 128-header / 32-KiB budget. A base publisher + response already over the full budget still passes unchanged, but no + ordinary mutation applies; security `Continue` applies only if the final + response fits after all ordinary batches are removed. These are separate + outcomes and metrics. + + | Adapter | Header-count / total-bytes ceiling (capability cell) | + | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | + | Axum | no platform ceiling below the core budget (native HTTP stack), cell fixed at ≥ 128 headers / ≥ 32 KiB | + | Fastly | **qualification-pending**: the measured platform ceiling is recorded in this cell by the adapter-qualification commit; unrecorded ⇒ hook startup fails | + | Cloudflare | **qualification-pending**: same rule | + | Spin | **qualification-pending**: same rule | + + A recorded cell below core's 128-header / 32 KiB budget is a startup + error (shrink the core budget or raise the ceiling, never a silent + per-adapter divergence). + + Hook/cache eligibility has the following concrete adapter cells; any + `qualification-pending` cell fails startup when the depending feature is + selected: + + | Capability | Fastly | Axum (dev) | Cloudflare | Spin | + | -------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------- | ------------------------------ | -------------------------------- | + | Runtime secret lookup for Vary-HMAC/DataDome | wired secret store; qualify key-rotation behavior | dev secret binding required | qualify Workers secret binding | qualify component secret binding | + | Persisted processed artifact + mutation IR/read sets | qualification-pending | in-process dev implementation required; non-durable | qualification-pending | qualification-pending | + | Atomic artifact/metadata entry commit | qualification-pending | implementation required | qualification-pending | qualification-pending | + | `Vary` variant index + insert-new-then-index-update rekey | qualification-pending | implementation required | qualification-pending | qualification-pending | + | DataDome field-line order, trusted IP/port, fixed HTTPS backend/no-redirect, and exact form limits | qualification-pending | qualification-pending | qualification-pending | qualification-pending | + | SecurityUse JA4 request evidence | platform value available; exact-field/payload qualification and sign-offs 23/28 pending | unavailable | unavailable | unavailable | + + The qualification commit records storage lifetime, maximum object size, + concurrency semantics, torn-write behavior, and fault-injection evidence; + “platform has KV” is not a qualifying cell. + `expose_host_fingerprints_to_vendor = true` also requires a qualified + SecurityUse JA4 cell; unsupported or pending is a startup error, while the + default `false` remains portable. + + Each mutator receives an **immutable, redacted snapshot of the + response head** (status and headers as of its turn, prior integrations' + accepted operations applied) as its read context; it never holds a + mutable reference (§2). Redaction is a security boundary, not + tidiness: the hook runs after core queues the EC `Set-Cookie`, so an + unredacted view would hand a mutator the raw EC to copy into + `X-Vendor-Identity` or its own cookie, walking around + `AuthorizedIdentity` entirely. The snapshot therefore + **excludes every `Set-Cookie` value and every reserved identity, + consent, and privacy header value** (names may be listed as present; + values are withheld). + Each registration also declares the complete set of response fields its + decision may read, including status as a distinguished input. Core records + the union with the accepted operation batch. Undeclared reads are a hard + conformance failure in tests; an integration unable to declare a complete + read set marks itself `revalidation = "refetch"`, which forbids IR replay + after any origin metadata change. + Operations arrive as **attributed batches bound to a registration + ID**, one batch per integration per response, ordered by + registration, with the security channel's batch (§4a) ordered **after** + ordinary response mutators, one global order, core finalization → + ordinary mutators → security effects → invariant pass, so the + security layer's precedence over publisher-facing mutations holds through + both position and its reserved/response-owning budget rule; the current flat effects vector satisfies neither + attribution nor budgets and will be restructured accordingly. Validation + and budgeting are **atomic per batch**: a batch that exceeds its + budget is rejected whole (logged, attributed), never partially + applied, since item-by-item rejection could apply a security 302's + `Set-Cookie` while dropping its `Location`. The response itself is + never rejected. A mutator that returns an error is skipped in full, its + operations are all-or-nothing, and the response proceeds without it. + **Panics are forbidden and fatal, not recoverable**: the primary target + (`wasm32-wasip1`) has no unwinding support, so there is no unwind + boundary to catch at, a spec that promised panic recovery would be + unimplementable there. Mutators are infallible-by-construction or + return `Result`; a panic is a bug that takes the instance down, same as + anywhere else in the request path. + +## 3a. Response eligibility, normative + +Which responses the hook runs on, enumerated so two implementations cannot +diverge silently: + +In this table, **persisted post-hook finals** means the cache-safe ordinary +artifact only: origin metadata plus accepted ordinary mutation IR and the +cache/privacy invariant result, with `Set-Cookie`, core request-specific +identity fields, security-channel effects, and origin validators excluded. +The security request filter evaluates every request before cache selection; a +fresh `Respond` bypasses the artifact, while a fresh `Continue` batch is +applied to the persisted ordinary artifact and the invariant pass reruns before +emission. This applies equally to ordinary hits, local conditionals, +origin-revalidation 304s, and HEAD. Therefore "`Set-Cookie` is never replayed" +means never replayed from storage; a freshly validated per-request typed cookie +operation may still emit on that response. Security `Respond` outputs are +always `private, no-store` and never become artifacts. + +An **unconditional recovery fetch** first saves the client's preconditions for +later local evaluation, then removes every upstream conditional/range field +(`If-Match`, `If-None-Match`, `If-Modified-Since`, `If-Unmodified-Since`, +`If-Range`, and `Range`) so origin must return full bytes. TS processes that +200 under current revisions, constructs the processed validator, and only then +evaluates the saved client preconditions as the authoritative server for the +transformed representation. Another bodyless origin 304 can never satisfy +recovery. + +The 304 **safe-update set** is exactly `Cache-Control`, `Expires`, `Date`, +`Age`, `Vary`, `Surrogate-Control`, `CDN-Cache-Control`, +`Cloudflare-CDN-Cache-Control`, and `Edge-Control`. The phrase +"registry-admitted mutable fields" in the matrix denotes an empty set in v1; +adding any name requires a reviewed spec/registry revision and conformance +fixture, never a runtime wildcard. + +| Response | Hook runs? | +| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Processed HTML document (rewritten by TS) | Yes | +| Streamed processed document | Yes, operations apply to the header block before first byte | +| Pass-through proxy response (not processed) | No, TS is a transparent proxy for it | +| Served-from-cache processed document | Serves the **persisted post-hook finals** captured at cache fill (revision-versioned; mismatch = miss), mutators run at fill/processing time only, so a normal hit and a conditional hit return identical policy metadata **by construction**, with no purity or determinism assumption about mutators | +| Redirect (3xx) | No | +| Error responses TS itself generates (4xx/5xx) | No | +| `304 Not Modified` for a processed representation | **Persisted-metadata pass**: a cached processed 200 stores its final post-hook headers, accepted mutation-operation batches, and the union of every mutator's declared response-field read set (the persisted mutation IR), versioned by §3's complete cache revision tuple, including model epoch and logical activation generation. A local conditional hit re-emits persisted finals only when the complete tuple matches current active. An origin-revalidation 304 is staged and diffed against separately stored origin-side metadata. (a) Any byte-coupled field changed (`Content-Encoding`, `Content-Type`, validators, digests) → invalidate and fetch/process a full 200. (b) A changed metadata field that intersects any persisted mutator read set, or an artifact/mutator lacking a complete read-set declaration, is also unsafe → full 200 refetch and ordinary hook execution; deterministic replay of old operations cannot stand in for re-evaluating a decision made from changed inputs. (c) If every changed field is outside every declared read set and belongs to the enumerated safe-update set (`Cache-Control`, reserved CDN cache fields, `Expires`, `Date`, `Age`, `Vary`, registry-admitted mutable fields), replay the persisted deterministic operations over updated origin metadata and rerun invariants. Updated origin metadata, finals, IR, read sets, and complete revision tuple publish in one atomic entry commit; changed `Vary` uses insert-new-entry-then-index-update ordering. (d) No change → re-emit persisted finals. Artifact absence or any tuple mismatch triggers an unconditional recovery fetch so TS obtains bytes. For processed-document GET/HEAD routes, TS is explicitly the authoritative server for the transformed representation: after processing the full 200 it evaluates RFC 9110 §13 preconditions against **processed** validators; this is not evaluation of origin validators by an intermediary cache. Other methods are never eligible for this recovery path. `Set-Cookie` and origin validators are never replayed. Absent metadata → cache miss | +| `HEAD` of a processed document | **Serves the persisted GET artifact when one exists**, parity with GET/304 by construction, since RFC 9111 §4.3.5 lets HEAD metadata update a stored GET response and mutators are not required to be deterministic; with no persisted artifact, HEAD is processed like a GET (headers only) and its finals stored as a **distinct head-only artifact type that never satisfies a later GET**, when a stored GET artifact exists a HEAD may **update** it only when the comparison, made against the separately stored **origin-side** metadata, never the processed artifact's (origin and rewritten lengths differ by construction, so the processed length is never the comparand and HEAD metadata never touches processed-side headers), finds validators and origin `Content-Length` matching and no byte-coupled representation field changed (RFC 9111 §4.3.5); qualifying updates land origin-side under the same atomic-commit discipline as the 304 rules, and any mismatch **invalidates** the stored GET artifact rather than updating it | +| Informational `1xx`, `204`, `205`, `206` | No, enumerated so adapters do not infer independently | + +This deliberately narrows #782's general "outbound response" phrasing to +processed documents (§6). + +## 4a. The security channel, normative closed boundary + +The security channel (today: DataDome) runs under a distinct, typed +`SecurityUse` authority rather than the advertising permissions P1/P4. +`SecurityUse` permits bot/fraud evaluation and challenge continuity only; +it never authorizes TS-controlled advertising identity, graph linkage, partner egress, +other integrations, or general raw-value observability. Request-scoped raw +security evidence may be disclosed only to the fixed DataDome Protection API +endpoint and only from that integration's explicit field allowlist; it is not +persisted in the identity graph, exposed to publisher origin or other +integrations, or emitted in logs. It carries its own configured retention and +deletion path. An advertising opt-out does not erase a +strictly security-scoped identifier, while an authenticated deletion request +or expiry under the security retention policy does. This is not a general +exception. The path-only `Request` remains publisher-originated data and is +explicitly covered by vendor retention/DSR sign-off; query strings and full +referrers are never in the security view. Every degree of freedom is closed: + +- **Host evidence is not a back door to the device provider.** + `[device] provider = "fastly"` is an explicit opt-in selection (#1044) and + the hook does not widen it: no JA4-derived classification is stored by a + mutator. If DataDome's Protection API is allowed to receive + request-scoped `TlsProtocol`/`JA4` evidence, its registration enumerates each field, + proves vendor necessity and payload bounds, and keeps it ephemeral under + `SecurityUse`; those exact consumers and fields are part of product/vendor + sign-offs 23/28. `TlsCipher` is omitted because the platform value has + different semantics from the vendor field, `H2Fingerprint` is not in the + vendor contract, and all other host evidence is omitted. +- **Deletion and retention name the system boundary honestly.** TS stores no + server-side DataDome identifier mapping. On an authenticated TS deletion + request it excludes the route from vendor validation, emits a typed + `datadome` cookie deletion, and sends no ClientID to DataDome for that + request; re-presentation retries deletion. A lost browser response can leave + the cookie until its configured `security_cookie_max_age`, which is the + bounded residual sign-off 23 accepts. This operation does **not** claim to + erase data already held by DataDome: vendor-side retention and data-subject + deletion require a named contractual/API procedure in the decision record. + If no such vendor procedure exists, operator documentation says so and may + not describe TS cookie deletion as vendor-data deletion. + +- **Typed security-cookie operation with a concrete lifecycle, not + header strings.** The channel emits cookies only through a typed + operation, and the registration is not a placeholder, for DataDome + it pins, **aligned to documented vendor behavior where hardening was + not intended**: cookie name exactly `datadome`; one configured ownership + tuple for every set and deletion: path exactly `/` and + `security_cookie_domain = "host-only"` (default) or one explicit normalized + ASCII domain. Host-only mode requires the vendor `Domain` attribute to be + absent; explicit-domain mode requires it to equal the configured domain + exactly, TS never accepts a different domain and never rewrites one scope + into another. The explicit value cannot exceed the registrable domain, + computed against the **vendored + Mozilla PSL snapshot in §4a.1**, + ICANN + private sections, IDNA-mapped; IP-literal or single-label + hosts fall back to host-only), path `/`; `Secure` mandatory; + `SameSite` configurable `Lax` (default) / `Strict` / `None` + (`None` requires `Secure`), matching the vendor's endpoint options; + the configured/returned `Domain` must additionally **domain-match the current + request host** per RFC 10025, the current cookie specification obsoleting RFC 6265 (domain-match and the PSL boundary check + are separate requirements) and a vendor cookie using `Expires` is + normalized to its Max-Age equivalent (both present → `Max-Age` wins, + per RFC 10025); a normalized lifetime exceeding the ceiling rejects + the whole operation batch; and the parser is total: repeated `Cookie` + request fields are joined with `"; "` (semicolon-space, the + order-preserving join of RFC 10025) before parsing, and + **duplicate `datadome` pairs after the join make the request-side + identity ambiguous: treated as cookie-absent for the vendor call and + counted**, while cookies under other names pass through untouched; + `Set-Cookie` fields are **never combined**, and a vendor response + carrying more than one `datadome` `Set-Cookie` field, a `Set-Cookie` + for any other name (the typed operation pins the name exactly), an + **`HttpOnly` attribute** (the vendor requires script readability, so + its presence is a protocol anomaly), duplicate attributes, duplicate + cookies in one field, an unparseable `Expires`, or unknown attributes + each reject the operation batch (the vendor cookie is well-formed; + strictness is safe). `Expires` normalizes as + `Max-Age = max(0, floor(expires − now))` whole seconds on the server + wall clock at parse time (the shared skew-bounded basis; a result of + 0 is a deletion), and the 512-byte limit measures the **normalized** + serialized `name=value` plus attributes in bytes; + `Max-Age` is capped by required operator configuration + `security_cookie_max_age` in the vendor-supported range 7 days through + **31,536,000 seconds** (one year); the returned cookie may be shorter but + never longer. There is no silent one-year default. Size ≤ **512 bytes** + (DataDome's current Fastly-module limit; 4 KiB was ours, not theirs). + Where the contract **is** deliberately narrower than the vendor, the + spec-pinned pointer allowlist starting at ClientID-only against + DataDome's mandatory response-directed mapping set, that reduction + needs explicit product **and vendor** acceptance: sign-off item 28; + a violating operation is rejected whole (the batch rule). **Both + sessionByHeader is startup-rejected in v1, one state, not three**: + TS never sends `X-DataDome-X-Set-Cookie: true`; a vendor + `X-Set-Cookie` **invalidates the batch → Continue** (its matrix cell, since + a session mode the fleet never requested must not half-apply); and an incoming + browser `X-DataDome-ClientID` is **not forwarded to the vendor** + (cookie-only session identity, an earlier revision translated + `X-Set-Cookie` into an ordinary cookie, which is not equivalent: + header-session clients expect JavaScript to receive `X-Set-Cookie` + and `X-DD-B`, and the higher-priority header session would never see + a cookie update; the older DataDome spec's "always send + X-DataDome-X-Set-Cookie when the header ID is used" is superseded for + v1 by this section). Supporting header mode later means the full vendor + protocol, typed owner-scoped `X-Set-Cookie`/`X-DD-B` forwarding, + CORS exposure, and a JavaScript/local-storage identifier observer, + as an explicit opt-in under **sign-off 23**. Every `ts-*` + name is rejected. **Read is owner-only across every _server-side_ surface, and the + browser side is explicitly not ownable**: DataDome requires the + cookie to be readable by its JavaScript and warns against `HttpOnly`, + so **every same-origin page script can observe it**, and a Respond + serves vendor-owned HTML under the publisher origin (vendor scripts + with same-origin access to cookies, storage, and APIs; publisher CSP + can conversely break the challenge). Those browser-side observers, + same-origin vendor code, CSP interaction, and challenge redirects + enter **sign-offs 23/28** for ratification (both decision records + still open), not implied. The server-side strip + inventory is exhaustive, not integration-scoped, because the browser sends `datadome` in the ordinary + `Cookie` header, so it is removed from **every non-DataDome surface**: + other integrations' request views, publisher-origin proxy forwarding, + proxy/click/Testlight upstreams, auction/page-bids request + serialization, and logs (redaction list), each surface a tested row + of the inventory; only the security channel itself observes it; vendor + egress goes only to the fixed DataDome Protection API authority and path; + redirects are not followed; deletion is always possible + through the `SecurityUse` lifecycle; and advertising withdrawal never + grants access to or reuses the identifier. No other request filter + inherits the cookie capability. +- **Cookie ownership makes deletion total for the scope TS creates.** While + DataDome is enabled, `datadome` is a security-owned name across the final + response: before the security batch applies, core removes and meters every + origin, core, cached, or ordinary-mutator `Set-Cookie` for that name; + unrelated cookie names remain separate field lines. Only the typed security + operation may emit it. Authenticated deletion emits the same configured + `(name, domain mode/domain, path)` tuple with `Max-Age=0`; it does not guess a + scope from the request cookie, whose wire form carries no Domain or Path. + Candidate validation rejects a change of domain mode/domain while the + previous active DataDome configuration can still have a live cookie. The + supported migration is disable + wait at least the previous + `security_cookie_max_age` + activate the new scope; a faster scope change + requires a separate bounded deletion-fan-out design. The permission spec + §5.5 whole-settings serve fence applies before cookie processing. A bounded + old-generation admission validation may survive only during the pre-promotion + drain; the register's promotion-not-before plus member quiescence proves it + and every admitted effect ended before the activation CAS. After that CAS, + no instance may emit, refresh, or delete a `datadome` cookie until it has + loaded and leased the exact new active tuple. A stale instance stops at serve + admission rather than extending the old scope. Fixtures cover origin + collision, host-only vs explicit-domain set/delete, attempted domain change, + and duplicate request cookies. The deletion claim therefore covers every + cookie this contract can create, not arbitrary pre-contract scopes. +- **The incoming `X-DataDome-ClientID` request header is owner-only, + like the cookie.** DataDome prioritizes the header over the cookie, + so leaving it in the shared request would hand other integrations and + upstream routing the same identifier the cookie boundary strips: core + **removes it from the shared request** before integrations and + upstream routing run, it joins `RedactedRequestView`'s enumerated + strip set (providers spec), **and in v1 it is stripped for the + vendor too: the Protection API request's ClientID derives only from + the `datadome` cookie, never from the incoming header** (DataDome's + contract requires `X-DataDome-X-Set-Cookie: true` whenever a + header-supplied ClientID is forwarded, so forwarding the header under + cookie-only mode would misdeclare the session mode); observed header + occurrences are counted, and a **fixture pins the path**: a request + carrying both cookie and header produces a vendor payload whose + ClientID equals the cookie value, with no header-derived identity + sent. Only DataDome-returned overlay data reaches the publisher, + never the raw browser-supplied header. +- **The pointer protocol has a total parser contract**, adapters + cannot differ where malformed batches fail open: the pointer list is + tokenized by the vendor's documented space separation, repeated + pointer header fields are concatenated with a single SP before + tokenizing, tokens split on runs of SP/HTAB, empty tokens ignored, + then names are ASCII-lowercased before duplicate detection; duplicate + names after normalization, invalid names, more than 16 pointers, or + more than 4 KiB of pointer payload render the batch invalid + (→ Continue, the vendor's fail-open). **Pointed-field multiplicity is + closed**: for singleton fields (`Location`, `Content-Type`, + `X-DataDome`, `X-DD-B`, `X-Set-Cookie`) more than one instance in the + vendor response invalidates the batch atomically, never a + first/last/join choice an adapter makes; list-valued fields + (`Cache-Control`, `Pragma`) are joined per RFC 9110 §5.3 before their + matrix outcome applies; `Set-Cookie` multiplicity follows the typed + cookie rule (exactly one `datadome` field, above). No both-source + priority rule exists in v1: the header session form (`X-Set-Cookie`) + is matrix-governed as batch-invalid, so "header form wins" is + unreachable and deleted. +- **Request-header pointers are a positive, enumerated allowlist with no + default publisher-origin identifier exposure.** + "Documented enrichment headers" is not enforceable; the registration + enumerates the exact names from the **inline DataDome field contract in + §4a.2**, spec-pinned + today to exactly **`X-DataDome-ClientID`**, admitted only when the + operator explicitly sets + `[integrations.datadome] expose_client_id_to_origin = true` (default + `false`); every other `X-DataDome-*` + field is rejected until a reviewed commit adds it to §4a.2 + ("documented enrichment set, listed one by one" without an actual list + was a wildcard whose contents could change outside the spec), + resolving what was a contradiction. When the opt-in is false, the + vendor-returned ClientID is discarded and the publisher origin is not + an identifier observer. When true, it applies only to an owner-scoped + publisher-upstream overlay, never the shared request; startup logs the + additional consumer, operator documentation must disclose its purpose + and retention, and a fixture proves no other surface can read it. + Everything else, authentication, + `Cookie`, `Forwarded`/`X-Forwarded-*`, other identity, consent, and + routing-authority fields, is rejected by name and by class: a + compromised endpoint must not replace origin credentials, inject + `ts-ec`, or spoof client location. +- **Browser-response headers are decision-scoped through the single + matrix, and only there.** This spec carries **no per-decision field + list**: every pointer's outcome per decision and session mode, + `Location`'s Respond-only 3xx admission, `Pragma`'s + drop-individually middle path (response `Pragma: no-cache` has no + standardized meaning, RFC 9111 §5.4), and the batch-invalidation + default for unlisted names, is exactly one cell of the matrix in + §4a.2.3. The fail-open consequence of batch + invalidation stays within sign-off 28's scope, and both documented + vendor responses are verbatim fixtures at the matrix. +- **Representation rules are decision-scoped and narrow.** A _Respond_ + decision (challenge/deny) owns its body but may describe it with + **`Content-Type` only**, encoding and validator fields + (`Content-Encoding`, `ETag`, `Last-Modified`, digests) stay reserved + even for Respond: challenge bodies are simple and uncacheable, the + allowlist does not admit those fields, and ambiguity here decides + whether a challenge enforces or silently fails open (batch rejection → + Continue). If the vendor ever requires more, it arrives as a reviewed + §4a.2 field-contract addition. A _Continue_ decision may not touch + representation metadata of publisher bytes. +- **Respond transport is bounded, with exact measurement points.** The + challenge body has a maximum size (64 KiB) and a **complete-response + deadline of 3000 ms on the instance's monotonic clock, measured from + immediately before vendor-backend acquisition/dispatch to the final + body byte**, meaning connection setup and request send are inside the + window; the 1500 ms first-byte bound (the older spec's figure, now + first-byte only) runs from the same origin on the same clock. At + expiry the decision is final (batch fails → Continue) and the vendor + request is canceled; cancellation and resource cleanup complete + asynchronously and never delay the response. TS sends + `Accept-Encoding: identity`, and because that does not _guarantee_ + identity coding, a response arriving with any `Content-Encoding` is + itself batch-invalid; `Content-Length` is recomputed from the actual + bytes before Respond commits. **On a HEAD request, Respond validates + the challenge body exactly as for GET (size, deadline, encoding) but + emits no body**: the outward response **omits `Content-Length`** and + carries no content, RFC 9110 §9.3.2 permits `Content-Length` on + HEAD only when it equals the equivalent GET body's length, and the + vendor does not guarantee method-invariant challenge bodies, so the + validated HEAD bytes cannot establish the GET length (a + vendor-guaranteed equivalence, if ever ratified, may restore the + field as a reviewed change; the older DataDome spec's HEAD handling + remains superseded). Exceeding + size, first-byte, or total deadline fails the batch → Continue. +- **One pointer contract, one place.** The single normative + decision × session-mode × pointer matrix lives in **§4a.2.3**, this + spec's earlier inline + decision-scoped list and outcome list are deleted in its favor + (duplicated lists disagreed about `X-Set-Cookie`, `X-DataDome`, + `X-DD-*`, and `Pragma`, letting one conforming implementation accept + the vendor's documented `Set-Cookie X-DD-B` allow-example while + another invalidated the whole batch). No `X-DD-*` wildcard exists: + every name is enumerated, `X-DD-B` included and forwarded exactly once + as the vendor's documented cookie-mode browser-response signal; it is + never copied into publisher-upstream or another integration. The + documented vendor responses (both the challenge example and the + `Set-Cookie X-DD-B` allow example) are **verbatim fixtures asserting + the decision survives** and exactly the mapped fields emit. +- **Every security Respond ends uncacheable, unconditionally.** After + the decision's fields are applied, the invariant pass forces + `Cache-Control: private, no-store` and strips all CDN cache fields on + **every** Respond regardless of status, vendor pointers, or cookie + emission, a cookie-less `301` challenge with no effective vendor + cache header could otherwise be heuristically stored and served to + unrelated clients. +- **One global order:** core finalization → ordinary mutators → + security effects → **final cache/privacy invariant pass, + unconditionally last**. Security precedence over publisher-facing + mutations comes from its position, not a "wins" rule; nothing outranks + the invariant pass, or a challenge could combine `Set-Cookie` with + public caching. The older DataDome spec's "applies last, after + finalization" wording is **superseded by this order**. That baseline remains + unchanged; this PR-specific section prevents an implementation from placing + DataDome after the invariant pass and reopening the + public-cache-plus-cookie bug. +- The channel adopts the shared layers: structured attributed batches + (§3, atomic per batch, a 302 must never lose `Location` to a budget + while keeping its cookie), reserved header names, budgets, and the + invariant pass, with one sequencing rule fail-open depends on: the + complete challenge batch is **validated and budgeted before the + Respond decision commits**, so a rejection converts to Continue while + the publisher route is still available; discovering the rejection + after Respond has short-circuited routing would leave nothing to fail + open _to_. + +### 4a.1 Public Suffix List snapshot (normative) + +The vendored Mozilla PSL revision used for registrable-domain +computation in §4a: the implementation PR vendors the list file +and records its upstream commit hash here. Rules: ICANN **and** private +sections apply; hostnames are IDNA-mapped before matching; IP literals +and single-label hosts have no registrable domain (cookie falls back to +host-only). Updating the snapshot is a reviewed spec change. + +| Field | Value | +| ---------------------- | -------------------------------------------------------------------- | +| Upstream repository | `publicsuffix/list` | +| Upstream commit | `e1b8015c3b2f0f4f8c18659c2480fc1a22c07b20` | +| Upstream source path | `public_suffix_list.dat` | +| Required vendored path | `crates/trusted-server-core/data/public_suffix_list.dat` | +| Required hash path | `crates/trusted-server-core/data/public_suffix_list.sha256` | +| Required provenance | `crates/trusted-server-core/data/public_suffix_list.provenance.json` | + +The implementation copies the source bytes at that commit without editing +and writes the lowercase 64-hex SHA-256 plus one trailing LF (no filename or +other fields) to the required hash path. CI verifies the bytes, hash, and +commit reference together; updating any one without the others fails. The +provenance file is canonical JSON with exactly +`{upstream_repository, upstream_commit_oid, upstream_commit_tree_oid, +source_path, source_blob_oid, source_sha256_hex}`. The vendoring PR description +quotes the same commit/tree/blob values and the independent command output +that verified the raw upstream SHA-256 and byte-for-byte vendored copy. A +commit OID without its tree and source-blob witness does not satisfy the +release gate. + +### 4a.2 DataDome field contract (normative) + +Adding or changing any name here is a reviewed spec change. This subsection +holds the **only** normative Protection API request-field and response-pointer +lists; §4a carries no duplicate or per-decision lists of its own. + +#### 4a.2.1 Protection API request fields (browser request → DataDome only) + +`SecurityUse` admits only the fields below to the configured DataDome +Protection API endpoint. They are request-scoped and are never persisted in +the identity graph, copied to publisher upstream or another integration, or +logged as raw values. + +The endpoint is the fixed core-owned +`https://api-fastly.datadome.co/validate-request`; “configured endpoint” in +this subsection means that DataDome protection is enabled, not that an operator may +supply an authority. Redirect following is disabled. No TS-controlled +advertising identifier, consent-store key, graph value, request query, or full +referrer is admitted. The normalized publisher URL path remains disclosed and +may itself contain publisher-chosen data; sign-offs 23/28 must classify that +surface, its retention, and DSR handling rather than calling the entire URL +identity-free. + +Core-derived fields: + +- `Key`, `IP`, `Method`, `Protocol`, `Host`, `ServerHostname`, `Request` +- `RequestModuleName`, `ModuleVersion`, `TimeRequest`, `Port` +- `ServerName`, `ServerRegion` +- `ClientID` from the single unambiguous `datadome` cookie only. The form key + is always present because the Protection API declares it mandatory; its + value is the empty string when no unambiguous cookie exists +- `CookiesLen`, `AuthorizationLen`, and `PostParamLen` as lengths only +- `HeadersList`, containing only the source header names admitted by the + next list plus `authorization`, `content-length`, and `cookie` (whose values + remain length-only/ClientID-only above). Names are lowercased, comma-separated, + and retain received field-line order, including repeated admitted names; + arbitrary/custom header names are excluded. An adapter that cannot preserve + received header order does not qualify this integration until DataDome + approves a canonical replacement order in sign-off 28 + +Core derives those fields identically on every qualified adapter: + +- `Key` is the resolved DataDome server secret and is never obtained from + request/config text; `IP` and `Port` are the remote address and TCP source + port from trusted connection metadata. Missing `Key`, `IP`, or `Port` skips + the call through the metered fail-open path; no sentinel is synthesized +- `Method` is the validated HTTP method token; `Protocol` is exactly `http` or + `https` from the adapter request URI; `Host` is the normalized ASCII request + authority with a non-default port retained; `ServerHostname` is trusted TLS + SNI/local-host metadata, omitted when unavailable +- `Request` is only the URL path. Empty path becomes `/`; dot segments are + removed, percent escapes are preserved without percent-decoding and + normalized to uppercase hex, and the complete query and fragment are + discarded before the security view exists +- `RequestModuleName` is the literal `trusted-server`; `ModuleVersion` is the + build's checked-in Trusted Server version; `TimeRequest` is the request-ingress + Unix timestamp in decimal microseconds, captured once before integration + processing and constrained to `0..=2^53-1` +- `ServerName` is the adapter-qualified deployment/service name and + `ServerRegion` is its adapter-qualified region code; either is omitted when + the platform cannot supply it without request input +- `CookiesLen`, `AuthorizationLen`, and `PostParamLen` are decimal byte counts + of the received field/body surfaces before redaction. Overflow beyond an + unsigned 64-bit count skips the call; it never wraps or truncates + +Exact request-header value mappings: + +- `Accept` ← `accept`; `AcceptCharset` ← `accept-charset`; + `AcceptEncoding` ← `accept-encoding`; `AcceptLanguage` ← `accept-language` +- `CacheControl` ← `cache-control`; `Connection` ← `connection`; + `ContentType` ← `content-type`; `From` ← `from`; `Origin` ← a successfully + parsed `origin` serialized as scheme + ASCII host + non-default port only; + `Pragma` ← `pragma`; `Referer` ← a successfully parsed `referer` reduced to + scheme + ASCII host + non-default port only; `UserAgent` ← `user-agent`; + `Via` ← `via` +- `SecCHDeviceMemory` ← `sec-ch-device-memory`; `SecCHUA` ← `sec-ch-ua`; + `SecCHUAArch` ← `sec-ch-ua-arch`; `SecCHUAFullVersionList` ← + `sec-ch-ua-full-version-list`; `SecCHUAMobile` ← `sec-ch-ua-mobile`; + `SecCHUAModel` ← `sec-ch-ua-model`; `SecCHUAPlatform` ← + `sec-ch-ua-platform` +- `SecFetchDest` ← `sec-fetch-dest`; `SecFetchMode` ← `sec-fetch-mode`; + `SecFetchSite` ← `sec-fetch-site`; `SecFetchStorageAccess` ← + `sec-fetch-storage-access`; `SecFetchUser` ← `sec-fetch-user` +- `X-Requested-With` ← `x-requested-with` + +Request-field multiplicity is normalized **before** parsing, truncation, and +form encoding, and adapters expose every received field line rather than a +preselected first/last value. Every admitted value line must contain valid HTTP +field-value octets **and** valid UTF-8 after OWS removal; otherwise the vendor +call is skipped through the metered fail-open path, because adapter-specific +byte-to-string replacement is forbidden: + +- The list-valued source fields are exactly `accept`, `accept-charset`, + `accept-encoding`, `accept-language`, `cache-control`, `connection`, + `pragma`, `via`, `sec-ch-ua`, and `sec-ch-ua-full-version-list`. Core removes + leading and trailing optional whitespace from each field value, rejects a + value containing invalid field-value octets, and combines all field lines + (including empty values) in received order with the two literal bytes `, `. + This one normalized value is then parsed where the mapping above requires + parsing and is then bounded. Commas inside an individual value are not split + and reserialized. +- Every other admitted value-bearing source header in the exact mapping above + is singleton. Zero lines means omit the DataDome field. Exactly one valid + line is OWS-normalized and processed. Two or more lines, even identical, + are ambiguous and skip the vendor call through the metered fail-open path; + core never chooses first, last, or comma-joined. In particular this applies + to `origin`, `referer`, `user-agent`, `content-type`, `from`, every remaining + `sec-ch-*`/`sec-fetch-*` field, and `x-requested-with`. +- `authorization` and `content-length` are security singletons for this view. + Repetition skips the vendor call before either length or `HeadersList` is + constructed. `AuthorizationLen` is the byte length of the one + OWS-normalized value. `PostParamLen` is always the byte length of the body + actually presented to core, not the numeric `content-length` value; a + malformed or body-inconsistent `content-length` is rejected by the shared + HTTP request boundary before integrations run. +- Multiple `cookie` field lines are permitted. Core OWS-normalizes them and + joins them in received order with the literal bytes `; ` for the shared RFC + cookie parser. `CookiesLen` is the byte length of that canonical joined + value. `ClientID` is populated only when the parsed result contains exactly + one syntactically valid `datadome` pair; malformed cookie syntax or duplicate + `datadome` pairs produces the required empty `ClientID` value without + exposing another cookie. The original cookie values never enter the vendor + payload. +- After successful normalization, `HeadersList` records the lowercased name of + every admitted received field line in original line order, so repeated list + fields and cookie lines remain repeated. A rejected request produces no + `HeadersList` and no vendor call. Per-field caps apply to the single + normalized value; the 24,576-byte cap applies after complete form encoding. + +For an optional mapped value, zero received lines omits both source and mapped +field; one or more lines whose OWS-normalized values are all empty omits the +mapped form field but retains each received source name in `HeadersList`. If at +least one list-valued line is nonempty, empty siblings remain represented in +the exact received-order `, ` join. Mandatory `ClientID` and the three length +fields follow their explicit rules instead of this optional-field omission. + +Adapter qualification fixtures feed the same ordered repeated-field corpus to +every host and assert byte-identical form fields, lengths, `HeadersList`, and +reject/omit outcomes. The corpus includes repeated list fields, identical and +different singleton duplicates, multiple cookies, duplicate `datadome` +cookies, empty values, invalid octets, and headers whose individual values +contain commas; invalid UTF-8 is a skip, never replacement decoding. + +`true-client-ip`, `x-forwarded-for`, and `x-real-ip` are not admitted in v1. +The trusted `IP` field already supplies connection provenance; copying raw +forwarding headers would let a client or unqualified proxy manufacture vendor +evidence. A future adapter-normalized forwarding chain requires a separately +named typed field and vendor sign-off, never reuse of the raw header mapping. + +Platform host evidence: + +- `TlsProtocol`, capped by TS at 32 bytes +- `JA4`, capped by TS at 128 bytes, only when the operator explicitly sets + `[integrations.datadome] expose_host_fingerprints_to_vendor = true`; + the default is `false`, omission is represented by absence rather than an + empty field, and startup logs the additional vendor disclosure +- `TlsCipher` is omitted in v1: DataDome defines it as the ordered list of + cipher suites offered by the client, while `RuntimeServices::client_info()` + exposes only the negotiated cipher. Substituting that value would silently + change the field's meaning +- `H2Fingerprint` is omitted in v1 because the current Protection API contract + does not define such a request field + +`X-DataDome-ClientID` is never a Protection API source in cookie-mode v1. +No wildcard (`Sec-CH-*`, `Sec-Fetch-*`, `X-*`, or otherwise) expands this +list. + +The following limits are bytes of the decoded field value before form +encoding. Truncation is UTF-8-boundary-safe. `XForwardedForIP` alone truncates +from the end; every other bounded field retains its prefix: + +- 8 bytes: `SecCHDeviceMemory`, `SecCHUAMobile`, + `SecFetchStorageAccess`, `SecFetchUser` +- 16 bytes: `SecCHUAArch` +- 32 bytes: `SecCHUAPlatform`, `SecFetchDest`, `SecFetchMode`, and the TS cap + on `TlsProtocol` +- 64 bytes: `ContentType`, `SecFetchSite`, and the TS cap on `ServerRegion` +- 128 bytes: `AcceptCharset`, `AcceptEncoding`, `CacheControl`, `Connection`, + `From`, `Pragma`, `SecCHUA`, `SecCHUAModel`, `X-Requested-With`, and the TS + cap on opt-in `JA4` +- 256 bytes: `AcceptLanguage`, `SecCHUAFullVersionList`, `Via` +- 512 bytes: `Accept`, `ClientID`, `HeadersList`, `Host`, `Origin`, + origin-only `Referer`, `ServerHostname`, and `ServerName` +- 768 bytes: `UserAgent` +- 2,048 bytes: path-only `Request` + +`Key`, `AuthorizationLen`, `CookiesLen`, `IP`, `Method`, `ModuleVersion`, +`Port`, `PostParamLen`, `Protocol`, `RequestModuleName`, and `TimeRequest` are +unbounded per-field by the vendor table but remain subject to the total bound. +The complete `application/x-www-form-urlencoded` body, including field names, +`=`/`&` separators, and percent-encoding expansion, must be at most **24,576 +bytes**. Core constructs and measures the whole payload before issuing the +request. It does not silently drop optional fields to fit: overflow skips the +vendor call and takes the same metered fail-open `Continue` path as a transport +failure. + +This is deliberately narrower than DataDome's currently documented required +surface: notably, it withholds `CookiesList` and omits empty source-header +fields. Product/vendor sign-off 28 therefore requires written confirmation +that this exact reduced profile is supported. Until that confirmation and +adapter conformance fixtures exist, the DataDome integration is not +release-qualified. + +#### 4a.2.2 Request-direction pointer (vendor response → publisher-upstream overlay) + +The complete set of vendor-response header pointers the security +channel (hook spec §4a) may copy into the owner-scoped +publisher-upstream overlay. Every `X-DataDome-*` name not listed here +is rejected. + +| Header | Direction | Scope | +| --------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `X-DataDome-ClientID` | response → upstream overlay | Disabled by default; admitted only with `expose_client_id_to_origin = true`. Owner-scoped publisher overlay only, never the shared request view or another integration | + +#### 4a.2.3 The single pointer matrix (normative, decision × session mode × pointer) + +This is the one authoritative browser-response contract. Session mode +is **cookie** in v1 (sessionByHeader is startup-rejected; a header-mode +column is added by the sign-off-23 opt-in, never implicitly). No +wildcard rows exist, every accepted name is enumerated, and **every +cell terminates in exactly one outcome**. + +| Pointer | Respond (cookie mode) | Continue (cookie mode) | +| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------- | +| `Set-Cookie` | typed `datadome` cookie operation (hook spec §4a parser; exactly one `datadome` field) | typed `datadome` cookie operation | +| `X-Set-Cookie` | **invalidate the batch → Continue** (mode mismatch: TS never requests header sessions in v1; a vendor response asserting one must not half-apply) | **invalidate the batch** → effects dropped | +| `Location` | forward on a 3xx Respond, replace; **on a non-3xx Respond → invalidate the batch → Continue** (a redirect field without a redirect status is a malformed decision) | **invalidate the batch** (effects dropped) | +| `Content-Type` | forward (owns its body), replace | **invalidate the batch** (effects dropped) | +| `Cache-Control` | restricted merge; invariant pass last | **invalidate the batch** (effects dropped) | +| `Pragma` | drop-individually, logged (response `Pragma` has no standardized meaning, RFC 9111 §5.4) | drop-individually, logged | +| `X-DataDome` | forward, owner-scoped typed telemetry | forward, owner-scoped typed telemetry | +| `X-DD-B` | forward as a browser-response security signal; never copy to publisher-upstream or another integration | forward as a browser-response security signal | +| anything not listed | invalidate the batch → Continue | invalidate → effects dropped | + +Singleton-field multiplicity (`Location`, `Content-Type`, `X-DataDome`, +`X-DD-B`, `X-Set-Cookie` appearing more than once) invalidates the +batch atomically; list-valued fields (`Cache-Control`, `Pragma`) join +per RFC 9110 §5.3 before their cell applies (hook spec §4a). + +`X-DD-B` is security-owned when DataDome is enabled. Before applying the fresh +security batch, core removes every pre-existing instance from the origin, +cached ordinary artifact, 304 metadata update, core response, or ordinary +mutator. A valid pointed vendor value then uses **replace-all** and the final +response cardinality must be exactly one; if the fresh vendor batch does not +point to it, final cardinality is zero. Append is never allowed. Fixtures cover +origin collision, cache-hit collision, 304 collision, repeated vendor fields, +and one valid fresh value, proving “exactly once” at final emission rather than +merely inside the vendor batch. + +**Fixtures**: DataDome's documented challenge response (`Set-Cookie`, +`Pragma`, `X-DataDome`, `Cache-Control`) asserts the decision stays +**Respond** with exactly the mapped fields; the documented allow example +(`Set-Cookie X-DD-B`) asserts Continue proceeds with the cookie applied +and `X-DD-B` forwarded exactly once, neither fixture may fail open. + +### 4a.3 DataDome configuration delta (normative) + +The canonical v1 effective configuration is the pre-existing DataDome schema +plus this exact PR-specific delta. The configuration parser rejects unknown +security/session fields rather than ignoring them as compatibility toggles, +and the materialized values below participate in permission §5.5's complete +effective-config digest. + +```toml +[integrations.datadome] + +# Existing timeout_ms remains the 1500 ms first-byte bound. +complete_response_timeout_ms = 3000 +challenge_body_max_bytes = 65536 + +# Required when enable_protection = true; there is no silent lifetime default. +security_cookie_max_age = 2592000 # example; allowed 604800..=31536000 +security_cookie_domain = "host-only" # or one exact normalized ASCII domain +security_cookie_same_site = "Lax" # Lax | Strict | None + +expose_client_id_to_origin = false +expose_host_fingerprints_to_vendor = false +``` + +The Protection API authority is not configurable: it is the core-owned +`https://api-fastly.datadome.co/validate-request` endpoint defined in §4a.2.1, +with redirects disabled. The baseline `protection_api_origin` field is a +startup error under this delta, as are `sessionByHeader`, `session_by_header`, +any equivalent session-mode spelling, and every other unknown +security/session field. Supporting another region or a publisher proxy is a +reviewed endpoint-registry and product/security decision, never a free-form +URL setting. + +`complete_response_timeout_ms` defaults to and may not exceed 3000; +`challenge_body_max_bytes` defaults to and may not exceed 65,536. +`security_cookie_max_age` is mandatory when protection is enabled and must be +in `604800..=31536000` seconds. `security_cookie_domain` defaults to +`host-only`; an explicit value must pass §4a's exact-domain, domain-match, PSL, +and active-scope-change rules. `security_cookie_same_site` accepts exactly +`Lax`, `Strict`, or `None`; `None` is valid only with the unconditionally +emitted `Secure` attribute, and the cookie never carries `HttpOnly`. + +Both exposure booleans default to `false`. ClientID-to-origin additionally +requires the owner-scoped overlay capability. Host fingerprints additionally +require qualified JA4 availability and sign-offs 23/28. A selected adapter +that cannot preserve admitted request-header field-line order or enforce the +request/body limits fails startup for protection rather than synthesizing +different evidence. + +## 4. Done-when (from #782, sharpened) + +1. Trait + builder + registry application, each public item documented. +2. **The old generic `RequestFilterEffects.response_headers` channel is + removed.** DataDome uses the separate sealed, core-registered + `DataDomeSecurityRequestFilter` and typed `DataDomeSecurityEffects` defined + by §4a's PR-specific delta. Generic request filters + receive only `RedactedRequestView` and ordinary attributed effects and + cannot express the security view, owner overlay, cookie operation, or + reserved security header. The dedicated channel is necessary because + DataDome sets headers **and cookies** on 200, 301/302, 401, 403, and 429 + responses, response classes (§3a) the ordinary response hook never runs + on, with cookie emission v1 reserves. +3. **At least one real consumer ships in the same PR**, an existing + integration registering a mutator for a real need (or, failing a real + need, the feature waits; scaffolding with only self-referential tests is + dead code and will be removed). +4. Every adapter applies mutations on its outbound path, with a per-adapter + route test asserting an integration-set header appears in the response. +5. A parity-suite case asserts identical mutation behavior across adapters. +6. Reserved-surface, append/replace, operation-limit, and erroring-mutator + semantics covered by unit tests. +7. **Every row of the §3a eligibility matrix has a test**, streaming, + cache-hit, pass-through, redirect, error, and 304 each proven to run or + not run the hook, not merely one positive header test per adapter. +8. Cache/privacy invariant tests, one per restriction source and shape: + a **core-owned** cookie already queued before the hook + an + integration's public `Cache-Control` replacement → private/no-store, + surrogate stripped; **core-private cookieless** processed HTML + + public replacement → restriction preserved; **origin-private cookieless** processed HTML that retained the + origin's cache restrictions + public replacement → restriction + preserved (pass-through responses never run the hook, §3a); a cache-hit serve re-applying mutations without + weakening the stored classification; a `Vary` mutation neither + dropping core-required values nor bypassing the snapshot; the §3 + normalization fixture proves field-line grouping/case/order collapse to one + sorted lowercase descriptor, repeated names digest once, request value + instances retain octet/order boundaries, malformed/empty mutation members + reject, a malformed snapshot becomes `Vary: *` plus `no-store`, and `*` + writes no artifact; a `pre_epic_v1` artifact/index descriptor is a miss + immediately after the model-only `permissions_v2` activation CAS even when + config and policy digests are unchanged; each of the four enumerated CDN fields (`Surrogate-Control`, + `CDN-Cache-Control`, `Cloudflare-CDN-Cache-Control`, `Edge-Control`) + individually stripped; and a rejected `Content-Encoding` mutation. + +## 5. Size and sequencing + +The ordinary mutator API in §§2–3 is a modest headers-only feature with no +provider or permission-model coupling. It lands only with the real consumer +required by §4 item 3; without one, scaffolding does not ship. The separately +typed security channel in §4a is already that consumer's PR-specific contract: +it owns DataDome cookie operations and intentionally couples configuration +activation to permission §5.5. Those capabilities never become part of the +ordinary mutator API. + +## 6. Divergences from issue #782 + +This spec supersedes #782 on the following points; the issue is updated to +reference this spec when the implementing PR (the hook's return with its +first consumer, §7) merges: + +| #782 says | This spec says | Why | +| -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | +| Mutations apply to the outbound response generally | Eligibility is the explicit §3a matrix, centered on processed documents | Pass-through/error/304 mutation has different semantics per adapter; enumerating beats implying | +| Ship the trait + registry + adapter application | Additionally: structured operations API (§2), reserved surface (§3), and a real consumer in the same PR (§4, item 3) | PR #838 shipped the trait with zero call sites; an unrestricted `&mut HeaderMap` cannot enforce any collision policy | + +## 7. Disposition (2026-08-25) + +- The earlier draft of PR #1047 carried an `IntegrationResponseMutator` + response-header hook. The hook was removed from that PR before the series + was presented, so PR #1047 ships the documentation set only and no hook + trait, registration builder, or adapter call site exists in the series' + tree. This is exactly the outcome §4 item 3 and §5 require for a feature + with no consumer, applied to the feature's own spec. +- The hook returns together with its first consumer, meaning an integration + that must set response headers, for example `Accept-CH` client-hint + requests or detection results such as §4a's security channel. The + returning PR implements this spec, not PR #838's shape. +- Recorded for that future design, from review of the earlier draft: the + `&mut HeaderMap` shape concern stands. Handing a mutator a mutable + header map makes §3's collision policy unenforceable by construction, + because core cannot validate or attribute writes it never sees, so the + structured, attributed operations API of §2 is the required shape and the + `&mut HeaderMap` API must not return with the hook. diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md new file mode 100644 index 0000000000..ec98c28442 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -0,0 +1,826 @@ +# Design Spec: Jurisdiction Permission Model + +**Status:** Implemented in PR #1045; revised against the implementation, 2026-08-25. +**Author:** Engineering +**Issue references:** #779 +**Related specs:** `2026-07-30-pluggable-providers-design.md`, +`2026-07-30-provider-migration-rollout-design.md` +**Last updated:** 2026-08-25 + +> **Context.** PR #838 proposed a permission model whose review surfaced two +> classes of defect this spec exists to prevent: (1) silent behavioral +> inversions of consent-signal precedence, most seriously a present TCF string +> short-circuiting GPC/GPP/US-Privacy opt-outs, and (2) fail-open jurisdiction +> resolution when geolocation is disabled. Both defect classes are closed in +> the implementation. The precedence rules (§4) and the failure-mode matrix +> (§6) are normative and are now backed by pinning tests in +> `crates/trusted-server-core/src/ec/consent.rs`. One structural position of +> the 2026-07-31 draft was not adopted: policy remains a build-time-embedded +> `permissions.yaml`, not a `[permissions]` section of `trusted-server.toml`, +> because the runtime config push and activation apparatus the draft assumed +> does not exist yet (§3.1). Every other draft position that was narrowed, +> simplified, or deferred is recorded in §11. + +--- + +## 1. Overview + +The permission model replaces the hard-wired jurisdiction gate +(`allows_ec_creation` and its companions, now removed) with a single resolved +**permission set** per request. The data decisions Trusted Server itself makes +through this model are EC provider execution, EC creation and withdrawal, EID +transmission into the bidstream, and sharing of the EC identifier beyond the +edge (§7). Server-side auction dispatch is not yet a consumer (§7.4). + +The set is resolved from three inputs: + +1. **Jurisdiction**, the country and optional region the request resolves to + (§5). +2. **Policy**, a declarative map from jurisdiction to a baseline acquisition + rule per permission, plus a declared signal policy (§3). +3. **Signals**, the request's privacy signals, being TCF, GPP, GPC, and US + Privacy (§4). + +These are the initial sources. Issues #777 and #779 also envision publisher +interaction and external services as permission sources. That source interface +remains **explicitly deferred**, not silently dropped. §10 records the +divergence, and the documentation (`docs/guide/permission-model.md`) already +frames consent as one source among several so a later source plugs into the +same mechanism. Core code resolves permissions through a per-permission +`ConsentSignal` closure (`Grant`, `Revoke`, `Neutral`), so a new source is a +new producer of that signal, not a new resolution algorithm. + +Scope: the model governs decisions Trusted Server makes. A downstream protocol +receives the full regulatory context only where that protocol defines fields +for it (OpenRTB consent fields, and proxy-mode forwarding of raw strings). +The draft's stronger rule, that identity rows carry normalized per-permission +provenance and a digest and never a raw string, is **not yet true**: the +identity-graph entry (`KvEntry` in `ec/kv_types.rs`) stores the raw TCF and +GPP strings alongside the row today. The normalized provenance model travels +with the providers-spec storage work and is recorded as deferred (§11). + +## 2. Vocabulary: the IAB Privacy Taxonomy Data Uses + +Permissions are named by **IAB Privacy Taxonomy Data Uses**, mapped from the +IAB TCF Europe purposes and used strictly as technical identifiers. No CMP or +TCF policy is implemented by naming them. This replaces the draft's +TCF-purpose-identifier vocabulary (`store-on-device`, +`select-personalised-ads`). The taxonomy adoption postdates the 2026-07-31 +draft and follows the joint taxonomy work with the IAB Tech Lab. + +The implementation (`crates/trusted-server-core/src/permissions.rs`) models +the vocabulary in two tiers: + +- **Eleven named permissions**, one per TCF purpose 1 through 11, each with a + Data Use identifier. All eleven are resolved against the session signal + (§4): a present TCF record grants or revokes each mapped purpose. Two + purposes have no published Data Use yet, so purpose 1 uses a proposed + `necessary.operations.storage` key and purpose 11 keeps its TCF identifier + `select-basic-content`, both flagged for an upstream taxonomy addition. +- **Fifty-three additional taxonomy Data Uses**, carried so `permissions.yaml` + can declare a policy flag for the whole taxonomy. No provider gates on them + today and no signal maps to them, so their configured baseline stands. + They exist for completeness, testing, and demonstration, and where no + informed policy decision has been made the shipped file sets them `denied`. + +The two Data Uses that carry enforcement weight today are +`necessary.operations.storage` (TCF Purpose 1, storage) and +`advertising_marketing.first_party.targeted` (TCF Purpose 4, personalized-ad +selection). Provider execution gates on whatever a provider declares (the +built-in Edge Cookie providers declare storage), and sharing beyond the edge +gates on the storage plus personalized-ad pair (§7). + +This is a deliberate departure from the draft's rule that a permission appears +only when it has both a signal mapping and an enforcement point. The eleven +named Data Uses all have the signal mapping, and the fifty-three baseline-only +Data Uses are declared policy rather than enforced policy. The file header of +`permissions.yaml` states this plainly, and the `denied` default means an +undeployed flag cannot silently authorize anything. Policy validation still +**rejects** any rule or flag that references an identifier outside the modeled +vocabulary, so a policy cannot name a Data Use the code does not know. + +The eleven named Data Uses, with the TCF purpose each maps from: + +| # | Data Use identifier | TCF purpose | +| --- | ----------------------------------------------- | ----------------------------------------------- | +| 1 | `necessary.operations.storage` | Store and/or access information on a device | +| 2 | `advertising_marketing.first_party.contextual` | Use limited data to select advertising | +| 3 | `advertising_marketing.profiling` | Create profiles for personalised advertising | +| 4 | `advertising_marketing.first_party.targeted` | Use profiles to select personalised advertising | +| 5 | `advertising_marketing.personalize.profiling` | Create profiles to personalise content | +| 6 | `advertising_marketing.personalize.content` | Use profiles to select personalised content | +| 7 | `analytics.ad_reporting.measure_ad_performance` | Measure advertising performance | +| 8 | `analytics.ad_reporting.content_performance` | Measure content performance | +| 9 | `analytics.ad_reporting.market_research` | Understand audiences through statistics | +| 10 | `necessary.operations.improve` | Develop and improve services | +| 11 | `select-basic-content` | Use limited data to select content | + +(The purpose names are the IAB names verbatim, including their original +spelling.) + +## 3. Policy + +### 3.1 Location: `permissions.yaml`, compiled into the build + +Policy lives in a human-editable `permissions.yaml` at the repository root, +compiled into the binary with `include_str!` and parsed once per instance +(cached behind a `OnceLock` in `PermissionMaps::standard`). A deployer edits +or replaces the file and rebuilds to change policy. The file is not read at +runtime. + +This keeps the mechanism the draft rejected, for a reason the draft's own +premise no longer supports: the draft required policy to flow through the +runtime config pipeline (`ts config push`, staged activation, §5.5), and that +activation apparatus does not exist. Publishing a `[permissions]` TOML section +with no activation protocol would reintroduce exactly the mixed-revision and +lazy-validation hazards the draft cataloged. Until the runtime pipeline +exists, the embedded file is the safer home, and moving policy to runtime +configuration is recorded as deferred follow-up (§11), not abandoned. + +Within the embedded model, the draft's specific complaints are answered: + +- The parse runs once per instance, and the embedded file is a build-time + constant covered by unit tests, so a malformed committed file fails the + test suite rather than surfacing as a per-request failure. The documented + panic on a malformed embedded file is a build defect signal, not a runtime + condition. +- Unknown fields on a detailed rule are rejected (`deny_unknown_fields`), so + a misspelled override key fails loudly instead of being swallowed. +- Two rule keys naming the same location in different case are rejected at + parse, so one spelling cannot silently overwrite another. +- Auditability lives where the draft placed it, in version control: the file + ships in the repository, and its history is the change log. + +The `include_str!` path still reaches above the crate root, so the crate +packaging concern the draft raised remains open and moves with the runtime +follow-up. + +**Fallback posture.** The file is always present, so there is no "no policy" +state. A location that resolves no rule and has no configured default, and a +failed geo lookup, both resolve every permission to the **requires-signal +floor**: nothing is set without a signal that grants it. Absence of an +applicable rule is always safe, and there is no fail-open default. The +draft's `regime = "gdpr"` component of the protective profile has no +implemented counterpart because no regime concept exists (§3.2). + +### 3.2 Format + +Named **groups** (baselines) and **rules** mapping a country (`FR`) or +country/region pair (`US/CA`) to a group, plus a **signals** section that +declares how each session signal maps onto Data Uses. Each permission resolves +to an **acquisition rule**: + +- `granted`, set without any signal, +- `requires_signal`, set only when a signal grants it (opt-in), +- `denied`, never set, even when a signal grants it. + +```yaml +# permissions.yaml (abbreviated). The shipped file lists every Data Use in +# every group so each group's meaning is fully explicit. +groups: + gdpr-eu: + necessary.operations.storage: requires_signal + advertising_marketing.first_party.targeted: requires_signal + # ... every remaining Data Use, requires_signal or denied + gdpr-uk: + necessary.operations.storage: granted + # ... the other mapped purposes requires_signal, the rest denied + us-opt-out: + necessary.operations.storage: granted + advertising_marketing.first_party.targeted: granted + # ... the other mapped purposes granted, the rest denied + +rules: + FR: gdpr-eu + GB: gdpr-uk + US: us-opt-out + AU: us-opt-out + # A country/region key takes precedence over its country. A detailed rule + # applies explicit per-permission acquisitions on top of its group: + # US/CA: + # group: us-opt-out + # permissions: + # advertising_marketing.first_party.targeted: requires_signal + +signals: + tcf: + authoritative: true + purposes: + 1: necessary.operations.storage + 4: advertising_marketing.first_party.targeted + # ... purposes 2, 3, 5..11 likewise + us_opt_out: + sources: [gpc, gpp_sale_opt_out, us_privacy_opt_out] + revokes: all +``` + +Format rules, as implemented: + +- A group is a flat map of Data Use to acquisition flag, with an optional + `default` key covering any permission the group omits. A group without + `default` must list **every** modeled permission exactly once, or the + parse fails (`IncompleteGroup`). The shipped groups list every Data Use. +- A detailed rule is `{group, permissions}` where `permissions` maps a Data + Use to an explicit acquisition (`granted`, `requires_signal`, or + `denied`), overriding the group baseline for exactly that Data Use. This + adopts the draft's requirement that overrides name explicit target states. + The earlier `+`/`-` sigil scheme, which could not express + `requires_signal`, is gone. +- A rule key is a bare country or a `country/region` pair. Keys are matched + case-insensitively, and a region entry takes precedence over its country + entry. +- The **signals** section is new relative to the draft: the TCF purpose to + Data Use mapping, the opt-out source list, and the opt-out revoke set are + data in the file, so no signal-to-permission policy lives in the code. The + `signals.tcf.authoritative` flag governs only whether a present TCF + record's own grants and revokes apply. It never lets a TCF record override + an opt-out (§4). The `us_opt_out.revokes` value is `all` or an explicit + list of Data Uses, so a deployer bounds what an opt-out drops. + +The draft's required per-group **`regime`** class (`gdpr`, `us-privacy`, +`none`) is **not implemented**. Its intended consumer, server-side auction +dispatch, was not migrated to the permission model (§7.4), so the field would +be inert today. It returns with the dispatch migration. + +### 3.3 Validation + +Validation runs where the policy actually enters the system: + +- **At parse**, meaning the unit tests and any `PermissionMaps::from_yaml` + caller, the file is rejected for: malformed YAML, a rule referencing an + undefined group, an unknown Data Use identifier anywhere (group flag, + detailed-rule entry, signals purpose map, or revoke list), an acquisition + value outside `granted | requires_signal | denied`, a group without + `default` that does not list every permission, duplicate rule keys under + case-insensitive comparison (`us` and `US`), unknown fields on a detailed + rule, and a `revokes` keyword other than `all`. +- **At startup**, `[geo] default_country` must be set and must resolve to a + rule in the compiled `permissions.yaml` + (`GeoConfig::validate_default_country`), and the no-geo acknowledgment + must be present where required (§5.3). Both are settings-construction + failures, never per-request failures. + +Not implemented from the draft's list, and recorded as future hardening +(§11): checking rule-key country parts against the assigned ISO 3166-1 list, +checking region parts against assigned ISO 3166-2 subdivisions, and the group +identifier grammar. A mistyped country key (`DL` for `DK`) therefore still +parses. For the shipped table the EU and EEA coverage test (§3.5) closes the +consequence the draft cared about, a member state silently dropping to the +fallback. + +### 3.4 One source of jurisdiction truth (deferred) + +Not implemented. `detect_jurisdiction`, driven by the runtime lists +`consent.gdpr.applies_in` and `consent.us_states.privacy_states`, remains the +jurisdiction source for the auction consent gate and for +`ConsentContext.jurisdiction`, while the permission model resolves against +`permissions.yaml` independently. The drift risk the draft named is real and +stands recorded: adding a country to one source has no effect on the other, +and no CI test asserts consistency between the legacy lists and the policy +table. Unifying the two, with the auction gate reading a policy regime class, +travels with the dispatch migration (§7.4) as deferred follow-up. + +### 3.5 Shipped-table coverage + +Implemented as a unit test +(`every_eu_and_eea_member_requires_a_signal_for_storage` in +`permissions.rs`): every one of the 27 EU member states plus the three EEA +members (IS, LI, NO), 30 codes in all, must have a rule, and each must +resolve `necessary.operations.storage` as `requires_signal`. A mistyped +member-state key fails this test rather than silently diverting the country +to the deployer default. The shipped table maps the EU 27 and EEA to +`gdpr-eu`, the UK to `gdpr-uk` (storage `granted`, a baseline the rollout +ledger row 4 asks the task force to confirm with its citation, everything +else opt-in), and the US and Australia to +`us-opt-out`. Countries with no rule fall to the deployer's +`[geo] default_country` (§5.4). + +## 4. Signal precedence (normative, implemented) + +Precedence is **fixed in code** +(`permission_signal` in `crates/trusted-server-core/src/ec/consent.rs`), not +in policy, and runs most restrictive first. The policy file decides which +sources count and what they revoke or grant. The code decides only the order. + +1. **Policy `denied`** is never set, regardless of any signal. (Enforced in + the resolver, `PermissionMaps::resolve_with`.) +2. **A US-style opt-out always suppresses the Data Uses the policy revokes**, + regardless of any consent record present. The opt-out sources are the + `Sec-GPC` header, a GPP US sale opt-out, and a US Privacy sale opt-out, + as declared in `signals.us_opt_out.sources`. A GPC header suppresses the + revoked Data Uses even when an accompanying TCF string consents to them. + An explicit opt-out is never overridden by another signal, and the + `signals.tcf.authoritative` flag cannot change that. (This is the rule + PR #838 inverted. Three pinning tests now hold it in place, one per + opt-out source against a consenting TCF record.) +3. **A consent record present but undecodable revokes everything.** A + malformed record is a preference that could not be read, so it fails + closed rather than degrading to the no-signal baseline, which under a + `granted` baseline would turn garbage into a grant. It never withdraws + (§4.2). An **expired** TCF record is deliberately a distinct state, not + malformed: the decoded record is cleared, the raw string is kept for + proxy forwarding, and acquisition proceeds as if the record were absent, + so the baseline applies. +4. **Only then does a present TCF record decide the mapped Data Uses**, when + `signals.tcf.authoritative` is true: granted where the record consents to + the mapped purpose, revoked where it does not, neutral where no purpose + maps. The effective record is the standalone TC string or the EU TCF + section of a GPP string (`effective_tcf`). A TCF refusal of a mapped + purpose is a revoke at this step, which drops a `granted` baseline and + leaves a `requires_signal` baseline unset. +5. **No signal leaves the baseline standing**: `granted` sets the + permission, `requires_signal` leaves it unset. + +Two simplifications against the draft's taxonomy, both recorded in §11: + +- **TCF is the only grant-class signal.** The draft's grant class also + admitted explicit GPP/USP non-opt-out values, regime-scoped, so a US rule + could be `requires_signal` yet grant on signal-carrying traffic. + The implementation instead expresses the US posture as a `granted` + baseline that opt-outs revoke, so no-signal US traffic is allowed rather + than blocked pending a signal. Explicit non-opt-out GPP/USP values grant + nothing on their own. +- **Malformed-present blocks everything, not per family.** Any present but + undecodable record (TCF, GPP, or US Privacy) revokes every Data Use for + the request, rather than blocking only the permissions mapped to the + malformed source. This is strictly more restrictive than the draft's + per-family rule. + +### 4.1 Decision matrix + +For each permission, with baseline _B_ from the resolved rule: + +| Signal state (per §4 order) | B = granted | B = requires_signal | B = denied | +| ------------------------------------------- | ----------- | ------------------- | ---------- | +| Opt-out present, Data Use in the revoke set | unset | unset | unset | +| Any record present but undecodable | unset | unset | unset | +| TCF present, consents to the mapped purpose | set | set | unset | +| TCF present, refuses the mapped purpose | unset | unset | unset | +| No signal (or neutral for this Data Use) | set | unset | unset | + +An expired TCF record resolves as the "no signal" row. Whether an unset +outcome is also a **withdrawal** is a separate, narrower question (§4.2). + +### 4.2 Withdrawal vs. absence + +Withdrawal (destructive: expire the `ts-ec` cookie, write the identity-graph +tombstone) and non-grant (the permission is simply unset, EC response +headers stripped, nothing egressed) are distinct outcomes, never conflated. +"Baseline" below means the resolved acquisition rule for +`necessary.operations.storage` in the request's jurisdiction, resolved once +at `EcContext` construction (`storage_acquisition`), never a group label. + +The implemented trigger, exhaustively (nothing else withdraws): + +1. **A TCF record refusing storage (Purpose 1) withdraws iff the baseline is + not `granted`, and only when the refusal is carried by the live + request.** Under a `requires_signal` (or `denied`) baseline the refusal + is the visitor declining the very signal storage depends on, so it + withdraws. Where the baseline is `granted`, storage never depended on + the record, so the refusal suppresses use without destroying the + identifier. Tombstones are irreversible, and PR #838 wrote them for + visitors in unregulated jurisdictions whose global CMP emitted a + purpose-refusing string. The `EcContext` consent pipeline runs without + the persisted-KV inputs, so the withdrawal decision sees live request + signals only, satisfying the draft's live-request constraint by + construction. +2. **US-style opt-outs never withdraw.** GPC and sale opt-outs are use + restrictions: they suppress the permissions the policy revokes (EC + headers stripped, nothing egressed) but never trigger destruction, so + lifting the opt-out restores the identity. +3. **A malformed record never withdraws.** It suppresses only (§4, step 3). + Destruction requires an affirmative, decodable signal. +4. **Absence of signal never destroys identity.** A visitor who has not yet + made a choice is never stripped of an existing identity. +5. **A policy change is not a user signal.** There are no runtime policy + edits (§3.1), and a rebuild that tightens a baseline does not itself + tombstone: withdrawal still requires the affirmative refusal above on a + live request. + +The draft's additional trigger, an explicit storage-withdrawal or +authenticated deletion request honored in every jurisdiction, has no +implemented carrier: no such endpoint exists. It is recorded as deferred +(§11), and when it arrives it joins this list as a global trigger. + +`ec_storage_withdrawn` (in `ec/consent.rs`, surfaced as +`EcContext::storage_withdrawn`) has direct unit coverage for every arm +above: refusal under `requires_signal` withdraws, refusal under `granted` +does not, consent does not, GPC alone does not, sale opt-outs do not, no +signal does not, malformed does not. + +### 4.3 Withdrawal durability (largely deferred) + +Implemented behavior (`ec/finalize.rs`): when the request carries the +withdrawal signal and the client presented a cookie, the response expires +the EC cookie, and the identity-graph tombstone is written for each +presented identifier the provider accepts (the incoming cookie value and +the active value). The tombstone is the authoritative revocation marker for +subsequent EC behavior. A tombstone write failure is logged at error level +and the request completes, so the write is best effort. + +The draft's durability protocol is **not implemented** and is recorded as +deferred follow-up in full: the family ID with deterministic derivation for +legacy rows, the family revocation record written before the cookie +expires, the permission-exempt suppression and authority-state records with +CAS fencing, evidence-recency comparison and anti-replay pinning, the +durable negative-intent outbox, the global identity safety breaker, and the +associated consistency and retention contracts. That machinery depends on +storage primitives (linearizable per-key CAS, independent durability +domains) the current adapters do not qualify. The 2026-07-31 draft remains +the reference design for that work. Until it lands, the known gaps the +draft called out stand: cookie expiry is not fenced on the tombstone +commit, and revocation durability is bounded by the KV store's behavior. + +### 4.4 Signal normalization + +The consent subsystem (`consent/mod.rs`) remains the decoder and +normalizer. The permission layer consumes its output only through the +per-permission `ConsentSignal` closure. The implemented pipeline: + +1. Extract raw signals from cookies and headers, and decode TCF v2, GPP, + and US Privacy. A decode failure keeps the raw string and leaves the + decoded field empty, which the permission layer reads as + malformed-present (§4, step 3). +2. Resolve standalone-TCF vs GPP-embedded-TCF conflicts per the configured + mode (`restrictive`, `permissive`, `newest`), preserving the pre-epic + selection algorithm. +3. Apply the expiry check: a TCF record older than the configured maximum + age has its decoded form cleared, the `expired` flag set, and its raw + string preserved. Expiry is its own state, excluded from + malformed-present, and resolves as absent for acquisition. +4. Construct a US Privacy string from GPC for US privacy states with no + explicit USP cookie, so the opt-out also travels in transport fields. + +The draft's declared reordering, expiry filtering **before** conflict +resolution, was **not implemented**: conflict resolution still runs first, +so the pre-epic order stands. Recorded in §11. + +**Persisted-KV consent.** When the pipeline runs with an EC ID and a KV +store (not on the `EcContext` construction path), a request carrying no +consent signals falls back to the consent persisted for that EC ID, with +the jurisdiction re-derived from the current request's geo. Staleness is +enforced by the store: entries are written with a TTL equal to +`max_consent_age_days`, so an entry older than a live record's allowed age +has expired out of the store. A live signal always wins because the +fallback is consulted only when the request carries none. The draft's +declared change, running the loaded record through the full normalization +pipeline, is not implemented: the loaded record substitutes directly. The +narrow read is permission-exempt by construction, since determining +storage cannot itself require storage. + +**Proxy mode.** Proxy mode still skips semantic decoding entirely. The +draft's minimal opt-out extraction was not implemented, but the fail-open +consequence the draft feared does not arise under the permission model: a +present record in proxy mode is present-but-undecoded, which blocks every +baseline grant (§4, step 3), and the GPC header needs no decoding, so the +GPC opt-out is honored directly. No grants are ever derived in proxy mode. +The net posture is equal to or more restrictive than the draft's row. +Absent records resolve to the baseline. + +### 4.5 US signal field mapping + +Implemented sources, as declared in the shipped `signals` section: + +| Source | Effect | +| ---------------------------------- | --------------------------------------- | +| `Sec-GPC` request header | US-style opt-out | +| GPP US section sale opt-out | US-style opt-out | +| US Privacy `opt_out_sale = Y` | US-style opt-out | +| US Privacy `opt_out_sale = N` | Nothing (no grant class exists for USP) | +| Any explicit Not Applicable value | Nothing | +| Absent / unknown / reserved values | Nothing | + +An opt-out revokes the Data Uses the policy's `revokes` value names. The +shipped file says `revokes: all`, so an opt-out drops **every** granted +Data Use, including storage. That is deliberately broader than the draft's +mapping, which scoped sale opt-outs to personalized-ad selection only, and +a deployer narrows it by listing specific Data Uses instead. No +sale-family opt-out is destructive (§4.2). + +The remainder of the draft's §4.5 is **not implemented** and is recorded +as deferred: `SharingOptOut` and `TargetedAdvertisingOptOut` as distinct +inputs, grant-class non-opt-out values, embedded-GPC detection inside GPP +sections, the per-section applicability and aggregation algorithm with its +state-over-national precedence, the mapped-section malformed blocker at +per-section granularity, the derived OpenRTB `gpp_sid` construction with +the `__gpp_sid` consistency companion, and the complete pinned section +map. The current GPP decoder surfaces the EU TCF section, the section ID +list, and a US sale opt-out. Extending it to the full pinned registry is +its own project. + +#### 4.5.1 GPP registry snapshot (deferred) + +Not implemented. The vendored registry snapshot, the pinned per-section +accepted versions, the provenance manifest, and the fixture corpus travel +with the §4.5 decoder work. The 2026-07-31 draft's §4.5.1, including the +pinned upstream commit, remains the reference for that effort. + +## 5. Jurisdiction resolution + +### 5.1 Order + +Geo resolution runs **before** permission resolution. Jurisdiction is an +input to the permission set, which is why geo providers cannot themselves +be gated on it (providers spec §5). The selected geo provider resolves a +country and optional region, and rules match `country/region` first, then +`country`, case-insensitively. Implemented in +`EcContext::read_from_request_resolving_geo` and +`PermissionMaps::rules_for`. + +### 5.2 Lookup failure + +Implemented, with the failure state carried explicitly: +`GeoStatus { Located, NoLocation, Failed }` (in `ec/consent.rs`) separates +"the provider resolved no location" from "the lookup errored". A **failed** +lookup resolves every permission to the **requires-signal floor**, never +the deployer's `[geo] default_country`. The failure is logged at error +level so it is visible. The storage-withdrawal baseline follows the same +floor, so a failed lookup cannot widen destructive withdrawal either. The +rule is proven through the seam by +`a_geo_provider_failure_resolves_permissions_at_the_requires_signal_floor` +in `ec/mod.rs`, which drives a `PlatformGeo` that returns an error rather +than constructing the status by hand. + +**Which providers can reach it.** The floor is the contract for a geo +provider that performs its own fallible lookup, and none of the providers +shipped in this workspace is one. Fastly's `geo_lookup` returns an +`Option` and the SDK collapses every hostcall, buffer and parse failure +into `None` before it reaches the caller; the Cloudflare provider reads +request headers, which cannot error; the Axum and Spin providers resolve +nothing at all; and `DisabledGeo`, the default whenever +`[geo] provider` is not `"platform"`, returns nothing by construction. So +**a host geo outage today does not reach this floor.** It surfaces as +`Ok(None)`, which is `NoLocation`, and falls back to the deployer's +`[geo] default_country` like any other unmatched request. A deployer +choosing a permissive default country is therefore choosing what a geo +outage does, and should read §5.3 and the default-country guidance with +that in mind. The floor becomes reachable when a vendor geo crate under +`crates/geo/` does a real lookup that can fail. + +At the floor, an explicit valid grant still counts: a TCF record consenting +to a mapped purpose sets that permission under `requires_signal`, exactly +the divergence-from-deny-all the draft declared for this row. Absent, +malformed, or refusing evidence sets nothing. + +Not implemented: a lookup-failure metric (the error log is the signal +today) and the draft's `regime = "gdpr"` component of the failure profile, +since no regime concept exists. The draft's capability check, that an +adapter whose geo implementation can never resolve anything must not accept +the selection, is part of the providers-spec provider qualification rather +than this model. + +### 5.3 No geo provider selected + +Every request resolves to `[geo] default_country`, so jurisdiction becomes +a static constant, which is only honest when the operator can genuinely +assert single-jurisdiction traffic. + +Constraint, implemented in +`GeoConfig::validate_jurisdiction_acknowledgment`: **startup fails** when +an Edge Cookie provider is configured and no geo provider is selected, +unless the operator sets `[geo] assume_single_jurisdiction = true`. This +makes the dangerous migration config (a permissive `default_country` with +geo unset) an explicit operator decision rather than an accident, closing +the highest-severity finding of the PR #838 review. + +The guard's consumer list is narrower than the draft's: the draft enumerated +every jurisdiction consumer (EC provider, regime-gated auction dispatch, +raw-EC and EID egress). In the implementation the EC provider is the only +consumer whose behavior the policy gates, because auction dispatch was not +migrated (§7.4), so the guard fires on the EC provider selection alone. When +dispatch joins the model, the guard's trigger list grows with it. + +### 5.4 Defaults: one deployer fallback plus a protective floor + +`[geo] default_country` is **required in every mode** and is validated at +startup: it must be set, and it must resolve to a rule in +`permissions.yaml`. It accepts a country (`FR`) or a country/region key +(`US/CA`), matched case-insensitively, so a no-geo single-state deployment +can select its state rule. It covers two states the draft kept separate: + +- the geo provider resolved no location (or none is configured), and +- the resolved country or country/region matches no rule. + +The draft's `rules.default` policy entry does not exist, so +"resolved-but-unmatched" falls to the same deployer default as +"unresolved". The separation the draft treated as safety-critical is the +one the implementation does keep: a **failed** lookup never reaches the +deployer default and resolves at the requires-signal floor instead (§5.2). +With no default configured startup fails, and in the unreachable +belt-and-braces case where resolution still finds no rule, the floor +applies. + +### 5.5 Policy revision activation (deferred) + +Not implemented, and currently moot: policy is compiled into the binary +(§3.1), so the deployed artifact is the policy identity and there is no +runtime activation to coordinate. The draft's activation design, covering +the JCS-canonical policy digest and ordinal pair, the activation register +and candidate protocol, fleet readiness and quiescence, admission leases, +the hash-linked activation journal with store-clock retention, and the +model-epoch transition, is the reference design for the runtime-policy +follow-up recorded in §11. Its guarantee that mixed-revision irreversible +behavior is prohibited is honored today by construction, because a +deployment runs exactly one embedded policy and destructive withdrawal +requires a live user signal (§4.2), never a policy change. + +## 6. Failure-mode matrix (normative, implemented) + +| Condition | Resolution behavior | +| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Geo lookup reports a failure | Requires-signal floor for every permission, never the deployer default; error logged. No provider shipped today can report one, so a host geo outage lands on the row above instead (§5.2) | +| No geo provider configured | `default_country` baseline, guarded by `assume_single_jurisdiction` (§5.3) | +| Country resolved, no matching rule | `default_country` baseline (§5.4) | +| Region resolved, no region rule | Country rule | +| `default_country` unset or names no rule | Startup failure (§3.3) | +| EC provider configured, no geo, no acknowledgment | Startup failure (§5.3) | +| Malformed `permissions.yaml` | Parse error at settings load, once per instance; the embedded file is a build-time constant, never per-request | +| Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition); never withdraws; opt-outs still honored | +| Expired TCF record | Distinct state, not malformed; treated as absent, so the baseline applies | +| Signals contradict (opt-out plus consent) | Opt-out wins (§4) | +| No EC provider selected | Identity fails closed: nothing minted, an incoming cookie value never used or egressed (§7) | + +The posture is fail-closed. Every ambiguous state resolves to the +configured baseline or more restrictive. + +## 7. Enforcement points + +Consumers of the resolved set in the implementation: + +1. **EC provider execution.** The provider declares + `required_permissions()`, core resolves a `PermissionState` once per + request at `EcContext` construction, and the provider executes only when + every declared permission is set (`ec_allowed`). A provider that + requires nothing always runs. **Geo** is ungated because gating it is + circular, jurisdiction being an input to permission resolution. + **Device** is ungated by a separate, deliberate decision: its + security-classification role must run for traffic that has granted + nothing, and operator selection is the recorded authorization (providers + spec §5). The built-in Edge Cookie providers declare + `necessary.operations.storage`. + +2. **EC lifecycle.** Creation requires the provider's declared permissions + through the gate above. Withdrawal follows §4.2. Recognition and + revocation of an existing identifier are never permission-gated: the + withdrawal path runs precisely when `ec_allowed` is false, reading the + raw cookie value kept for that purpose. + +3. **Sharing beyond the edge.** One predicate, + `EcContext::ec_sharing_allowed`, requires the provider gate plus + **both** `necessary.operations.storage` and + `advertising_marketing.first_party.targeted`. A storage-only grant + therefore keeps first-party use while withholding partner sharing. The + implemented inventory: + + | Path | Gate | + | -------------------------------------------- | ------------------------------------------------------------- | + | Bidstream EIDs (every auction path) | `gate_eids_by_permissions`, storage plus personalized ads | + | OpenRTB `user.id` on the `/auction` endpoint | `ec_sharing_allowed` | + | Identify endpoint (partner-facing) | `ec_sharing_allowed` | + | Pull sync (browser-request-scoped) | `ec_sharing_allowed`, from the live request resolution | + | Batch sync (context-free S2S) | Authenticated; withdrawn or missing rows are ineligible | + | KV EID resolution for auctions | `ec_allowed`, then the EID pair gate on the result | + | Publisher navigation and page-bids `user.id` | `ec_sharing_allowed` (the storage plus personalised-ads pair) | + + With **no EC provider configured**, identity fails closed: the gate is + closed rather than open by default (`ec_allowed` is false), so a cookie + value present on the request is treated as absent and never used or + egressed. This replaces PR #838's vacuously-true `is_none_or` check. + + **Follow-up note (recorded, not silent).** The publisher navigation and + page-bids paths attach the EC-derived request ID and `user.id` under + the provider gate (`ec_allowed`) rather than the sharing pair, while + their EIDs are pair-gated. With the built-in providers (storage-only + requirement), a storage-only grant can therefore still place the EC in + `user.id` on those paths. Aligning them with the `/auction` endpoint's + pair gate is recorded follow-up (§11). The draft's fuller inventory + (proxy/click/Testlight forwarding gates, the observability denylist + with typed redaction boundaries, the raw-regulatory-transport + destination allowlist, integration response cookies) is deferred with + it. Today EC values are truncated (`log_id`) before logging as the + observability mitigation. + +4. **Server-side auction dispatch (not migrated).** Dispatch is still gated + by the consent subsystem (`consent_allows_server_side_auction`), not by + the permission model: when the jurisdiction is GDPR or unknown, or an EU + TCF signal is present, dispatch requires an effective TCF record + consenting to Purpose 1, and otherwise no bid request leaves (a no-bid + response, with no PBS/APS call and no UA/IP/geo forwarding). Known + non-GDPR jurisdictions without an EU TCF signal dispatch freely. The + draft's regime-keyed dispatch table, and the `ContextualAuctionView` + positive projection for dispatch with personalized-ad selection unset, + are **not implemented**. When personalized-ad selection is unset today, + EIDs and the pair-gated identifiers are stripped but dispatch is the + ordinary request, not a contextual projection. Recorded as deferred + (§11) together with §3.4. + +The client-cycle resolve endpoint (`/_ts/api/v1/ec/resolve`) is a further +consumer: a provider-derived identifier posted by the page is accepted only +through the same provider and permission gates. + +### 7.1 Contextual OpenRTB v1 allowlist (deferred) + +Not implemented. The machine-readable projection manifest, its path +grammar, cardinalities, cross-field rules, and derivation vocabulary, and +the conformance walker over final encoded bytes, belong to the dispatch +migration (§7.4) and remain specified by the 2026-07-31 draft for that +work. + +## 8. Testing strategy + +Implemented, in `permissions.rs`, `ec/consent.rs`, `ec/mod.rs`, +`ec/finalize.rs`, and `consent/mod.rs`: + +- **Signal precedence pinning.** Three opt-out-beats-TCF tests, one per + opt-out source against a consenting TCF record + (`gpc_suppresses_storage_even_with_a_consenting_tcf_record` and + companions). These reinstate the + behavior PR #838 inverted. +- **Withdrawal scoping.** One test per §4.2 arm: refusal under + `requires_signal` withdraws, refusal under `granted` suppresses without + destroying, consent is not a withdrawal, GPC alone never withdraws, sale + opt-outs never withdraw, no signal never withdraws, malformed never + withdraws. +- **Fail-closed acquisition.** Malformed records block baseline grants; + each undecodable record family is detected; an expired TCF record is not + treated as malformed and resolves at the baseline. +- **Geo status.** A failed lookup resolves at the requires-signal floor + (permissions and the storage baseline both), driven through a + `PlatformGeo` that returns an error rather than from a hand-built status, + and no-location falls back to the configured default. +- **Policy parsing and validation.** Groups, rules, detailed-rule + acquisition maps (including `requires_signal` per rule), rejection of + unknown groups, unknown permissions, unknown acquisitions, incomplete + groups, case-insensitive duplicate rule keys, unknown revoke keywords, + and the signals-section round trip. +- **Shipped-table coverage.** All 30 EU/EEA codes lock storage to + requires-signal (§3.5). +- **Vocabulary breadth.** A TCF record grants or revokes every one of the + eleven mapped purposes, not only storage and personalized ads. +- **Gates.** Provider execution blocked without its required permissions, + the sharing pair withheld on a storage-only grant, EIDs stripped when + either permission of the pair is unset, and the no-provider stateless + posture. + +The draft's fuller matrices (the complete normalization matrix as +table-driven tests, the per-row egress inventory with a denylist check, the +dispatch regime-by-signal matrix, contextual-serializer poisoning, S2S +authority denial reasons, and the §4.3 fault-injection suite) travel with +their deferred features. + +## 9. Out of scope + +- The §4.3 durability protocol, §4.5 full field mapping and registry + snapshot, §5.5 activation apparatus, §7.4 dispatch migration, and §7.1 + contextual projection: deferred follow-ups, recorded in §11 with the + 2026-07-31 draft as their reference design. Deferred, not rejected. +- Runtime policy configuration (a `[permissions]` config section): + deferred until the config push and activation pipeline exists (§3.1). +- Per-signal jurisdiction scoping (honoring GPC only where a law defines + it): rejected, as in the draft. The opt-out signal layer is + jurisdiction-free in the implementation, and the country baseline decides + only what a revoke has to drop. +- An authenticated deletion or explicit storage-withdrawal endpoint: + deferred (§4.2). + +## 10. Divergences from issue #779 + +This spec supersedes #779 on the following points, so there is one +acceptance contract, not two: + +| #779 says | This spec says | Why | +| -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | +| Unmatched countries fall to `default_country` | Adopted: unmatched and unresolved requests both fall to the required `[geo] default_country`; a **failed** lookup floors instead (§5) | The failure state is the one that must never reach a permissive default; the draft's `rules.default` split was not kept | +| The full TCF purpose vocabulary is modeled | Adopted and extended: all eleven purposes are signal-resolved, and the full Privacy Taxonomy is carried as declared baseline (§2) | The joint taxonomy work made whole-taxonomy declaration the goal; `denied` defaults keep undeclared uses inert | +| Policy is an embedded file | Adopted: `permissions.yaml` is compiled into the build (§3.1); runtime configuration is deferred follow-up | The runtime push and activation pipeline does not exist; version control is the audit trail meanwhile | +| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signals; further sources are deferred, and the `ConsentSignal` closure is their seam (§1) | Shipping an interface with no second source repeats the inert-surface mistake; the extension seam is defined | + +## 11. Revision record vs the 2026-07-31 draft + +One row per divergence between the draft and the implementation this +revision was verified against (branch `split/5-response-hook-docs`, +PR #1045). + +| Draft position | Implemented position | Why | +| --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Vocabulary is two TCF-purpose identifiers, enforced permissions only (§2) | IAB Privacy Taxonomy Data Uses: eleven named purposes all signal-resolved, plus 53 taxonomy Data Uses carried as declared but unenforced baseline flags | The joint taxonomy adoption postdates the draft; whole-taxonomy declaration serves completeness and demonstration, with `denied` defaults keeping unenforced flags inert | +| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Policy is `permissions.yaml`, compiled into the build with `include_str!`, parsed once and covered by tests | The runtime config push and activation apparatus does not exist; publishing runtime policy without it would recreate the hazards the draft cataloged; runtime policy is deferred follow-up | +| Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated; the field returns with that work | +| Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted: a detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema; `requires_signal` is now expressible per rule | +| Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated; a failed lookup floors separately | One deployer knob is simpler; the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | +| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first; the EU/EEA coverage test guards the shipped table against the typo class; ISO-assignment checks are future hardening | +| Three-class signal taxonomy with regime-scoped grant acceptance; the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source; the US posture is a `granted` baseline that opt-outs revoke; explicit non-opt-out values grant nothing | A simpler two-signal model without regimes; the cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | +| Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification; per-family scoping needs the full §4.5 decoder work | +| Normalization runs expiry before conflict resolution, a declared change (§4.4) | Conflict resolution still runs before the expiry check | The reordering was not implemented; the expired state itself (distinct from malformed, absent for acquisition) was adopted | +| Persisted-KV consent flows through the full normalization pipeline with an explicit TTL comparison (§4.4) | The loaded record substitutes directly when the request carries no signals, jurisdiction re-derived; staleness is enforced by the store TTL (`max_consent_age_days`) | The store-level TTL delivers the staleness bound without a second normalization pass | +| Proxy mode gains minimal opt-out extraction (§4.4) | Proxy mode still skips decoding; a present record blocks all grants via the malformed-present rule and the GPC header opt-out is honored without decoding | The permission-layer outcome is equally or more restrictive with no new decode paths; revisit with the §4.5 decoder work | +| Withdrawal has four triggers including an explicit storage-withdrawal or authenticated deletion request (§4.2) | The TCF Purpose 1 refusal under a non-granted baseline is the only trigger; opt-outs, malformed records, absence, and policy changes never withdraw (adopted) | No deletion endpoint exists to carry the extra trigger; the narrowest destructive surface shipped first | +| §4.3 durability protocol: family records first, suppression and authority-state records, outbox, breaker, strong reads | Cookie expiry plus best-effort identity-graph tombstones per presented identifier, with failures logged | The protocol requires storage primitives (linearizable CAS, independent durability domains) the adapters do not yet qualify; deferred with the providers-spec storage work | +| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out; the revoke set is policy-declared, shipped as `all` (which also drops storage) | The full decoder and registry vendoring are their own project; the policy-declared revoke set gives deployers the scoping lever meanwhile | +| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists; the built binary is the policy identity | With no runtime policy there is nothing to activate; the draft remains the reference design for the runtime-config follow-up | +| §3.4 single jurisdiction truth, and §7 dispatch gated on the policy regime with a contextual projection | Auction dispatch keeps the consent-subsystem gate (effective TCF Purpose 1 for GDPR or unknown jurisdictions); `detect_jurisdiction` and its lists remain; no contextual view | Dispatch migration is follow-up; the legacy-list drift risk the draft named still stands and is recorded rather than resolved | +| Every raw-EC egress path is pair-gated, with per-row tests and a denylist check (§7) | Pair gating is centralized in `ec_sharing_allowed` (auction endpoint `user.id`, publisher navigation and page-bids `user.id`, identify, pull sync) and `gate_eids_by_permissions` (EIDs everywhere); batch sync checks row state only | Partial adoption; aligning the remaining paths, the S2S stored-provenance authority, and the inventory tests is recorded follow-up | +| Identity rows never store raw consent strings, only normalized provenance and a digest (§1) | The identity-graph entry stores the raw TCF and GPP strings with the row | The normalized provenance schema belongs to the providers-spec storage work; until then rows carry the raw strings | +| No signals block in policy; the signal mapping is fixed in the spec | New: a `signals` section in `permissions.yaml` declares the TCF purpose map, opt-out sources, and revoke set, with `tcf.authoritative` governing only TCF's own effect | Moves signal policy from code into deployer-editable data; the flag can never let a TCF record override an opt-out, preserving §4 precedence | +| The §5.3 no-geo guard covers every jurisdiction consumer | The guard fires when an Edge Cookie provider is configured with no geo provider | The EC provider is the only policy-gated consumer today; the trigger list grows when dispatch and further egress paths join the model | +| `default_country` is required only in the acknowledged static no-geo mode (§5.4) | Required always and startup-validated against `permissions.yaml` | It is the baseline for unmatched requests in every mode, so it must always exist | diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md new file mode 100644 index 0000000000..78aca06846 --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -0,0 +1,641 @@ +# Design Spec: Pluggable Edge Cookie, Device, and Geo Providers + +**Status:** Implemented in PR #1043 (Edge Cookie provider seam) and PR #1044 +(device and geo selection); revised against the implementation, 2026-08-25. +**Author:** Engineering +**Issue references:** #777, #778, #780, #781 +**Related specs:** `2026-07-30-permission-model-design.md`, +`2026-07-30-provider-migration-rollout-design.md`, +`2026-07-30-client-cycle-ec-resolve-design.md` +**Last updated:** 2026-08-25 + +> **Context.** PR #838 proposed a first implementation of this epic in a single +> change. Review of that PR surfaced design gaps this spec exists to close +> before a second implementation pass: an identity abstraction that owned +> minting but not recognition, per-adapter divergence in provider selection, +> silent misconfiguration modes, and speculative trait surface with no +> production caller. This spec is the authoritative statement of what the +> provider architecture must do; where it contradicts PR #838, this spec wins. +> The second pass has now landed (PR #1043 and PR #1044, with permission +> enforcement in PR #1045), and this revision restates the spec to match the +> implemented code. A final section records every divergence from the +> 2026-07-31 draft. + +--- + +## 1. Overview and goals + +Trusted Server makes three per-request data decisions that were previously +hard-wired: whether to create or keep an Edge Cookie (EC) identity, how to +classify the requesting device, and whether to resolve geolocation. Each is +now a **provider**, a selectable component chosen in operator configuration, +with a deliberately neutral default. + +Goals, as implemented: + +- A deployment picks an implementation per concern (including none) without a + code change to Trusted Server core. +- Defaults are neutral. With no configuration, no EC is created, device + classification uses only the User-Agent, and no geolocation is performed. A + default deployment makes no third-party or host-specific call. +- An **EC provider declares** the permissions its data use requires + (`required_permissions` on the trait), and **core enforces** that + declaration before minting or using an identity. A provider cannot + authorize itself. The enforcement machinery is the permission model's + subject and lands with it in PR #1045 (see the permission model spec). + Geo and device carry the same declaration method with an empty default, + for the reasons spelled out in section 5. +- All adapters (Fastly, Axum, Cloudflare, Spin) route selection through the + same core builders, so identical configuration selects identical providers + everywhere. A selection the deployment cannot satisfy fails loudly rather + than degrading. The EC API routes (identify, batch-sync, ec/resolve) are + registered by the Fastly entry point only today, because the portability + adapters do not yet wire a platform KV store. The Spin adapter's route + list documents that gap explicitly rather than leaving those paths silent. + +Non-goals: + +- No vendor provider ships in this epic beyond the host-platform + implementations named below. The `crates/edgecookie/` directory holds a + README describing where vendor EC crates will live. +- The client-cycle (browser round-trip) provider type has its own spec. The + trait ships the seam for it (`resolve_from_client`, a no-op by default) + and a demonstration provider (`client-fixed`) compiled only into test and + demonstration builds. Production selection of the demo provider is a + startup error. + +## 2. Provider taxonomy + +| Concern | Trait | Built-in default | Opt-in implementations | +| ----------- | -------------------- | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | +| EC identity | `EdgeCookieProvider` | none (stateless) | `hmac` (in core, HMAC over client IP, preserves today's identity), `host-signals` (in core, see below), and `client-fixed` (demo builds only) | +| Device | `DeviceProvider` | `builtin` (User-Agent only) | `fastly` (TLS JA4 and HTTP/2 signals through an injected `HostSignals` service) | +| Geo | `PlatformGeo` | none (no location) | `platform` (host geo lookup) | + +The geo trait is the existing `PlatformGeo` in `platform/traits.rs` rather +than a new `GeoProvider` name. The EC trait lives in `ec/provider.rs` and the +device trait in `ec/device.rs`. + +Selection keys are strings in operator configuration +(`trusted-server.example.toml` carries the commented template): + +```toml +[ec] +provider = "hmac" + +[ec.providers.hmac] +passphrase = "replace-with-32-plus-byte-random-secret" + +[device] +provider = "builtin" # default. "fastly" opts into TLS/H2 signal evidence + +[geo] +provider = "platform" # default is none (no location, no host call) +default_country = "FR" # required (section 6) +# assume_single_jurisdiction = true # required when EC runs with no geo +``` + +**The `host-signals` EC provider** (identity from HMAC over the host TLS JA4 +and HTTP/2 fingerprints plus the client IP) was deliberately dropped from the +2026-07-31 draft. It has since shipped in PR #1044 as an opt-in built-in +(`[ec.providers.host-signals]`), implemented against the host-agnostic +`HostSignals` capability rather than a Fastly API, so any host that supplies +the fingerprints can run it and a host that supplies none cannot build it. +When the host supplies no fingerprint at all the provider defers with a +warning instead of degrading to an IP-only identifier under the host-signals +name. **An open review question stands on whether this provider should ship +in the series at all**, because its identifier shape shares the built-in +HMAC grammar and a sign-off row defers host fingerprint processing. The +question is flagged for the series review and this spec does not present +the provider as settled either way. + +## 3. The identity lifecycle contract + +This is the section PR #838 lacked. Its trait abstracted **minting** an +identifier but left **recognition** and **KV key normalization** hard-coded +to the built-in HMAC shape, so a provider whose identifiers did not match +`{64hex}.{6alnum}` minted cookies that the very next request discarded. + +The implemented contract routes every lifecycle operation core performs on +an EC value through the selected provider: + +| Lifecycle operation | Where core uses it | Contract | +| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Mint** | EC generation on first eligible request, and the client-cycle resolve endpoint | The provider returns the identifier (`generate` server-side, `resolve_from_client` for the client cycle) and only core writes the cookie, after enforcing the global bounds below. | +| **Recognize** | Reading `ts-ec` back from the request, deciding `ec_was_present`, withdrawal checks, and every path that hands the value onward: the origin URL in `append_ec_id`, the click-target URL in `handle_first_party_click`, and the proxied body an integration builds | `accepts_id` answers whether a value is a well-formed identifier the provider issues. A value the selected provider does not recognize is treated as absent, so it is never used or egressed, while the raw cookie value stays visible to withdrawal handling. The egress paths reach the same answer through `edge_cookie::recognized_ec_id`, and a deployment with no provider selected recognizes nothing and so egresses nothing. | +| **KV key** | Identity-graph row reads and writes | `normalize_id_for_kv` returns the key form. The default lowercases the built-in HMAC hash segment and preserves the suffix, keeping today's keys. An opaque or case-sensitive provider overrides to the identity function so distinct identifiers never collapse into one row. | +| **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader: it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | + +**Invariant:** for every provider `P` and every identifier `id` minted by +`P`, `id` round-trips read-back byte for byte. A test in `ec/mod.rs` proves +the round-trip with a non-default provider whose identifiers are opaque, and +a second test in `ec/resolve.rs` proves the client-cycle value survives the +full scenario verbatim. + +The draft's richer lifecycle surface, a canonicalizing `parse` with +per-provider equivalence fixtures, a core-constructed graph key built from a +provider `graph_key_suffix`, a declared cluster-prefix capability, declared +namespace descriptors with a startup disjointness proof, and a reusable +conformance suite driven by fixtures, is **not implemented in these PRs**. +Recognition plus KV normalization proved sufficient for the operations core +actually performs today, and each deferred piece is tracked as follow-up +work rather than silently dropped (see the revision record). Until the key +grammar lands, the KV key is the provider's normalized identifier verbatim, +which keeps every pre-epic HMAC row reachable. + +The pre-epic IP-cluster prefix listing runs unchanged, but the key space it +lists over does not. A fresh mint is keyed `hmac~.`, so the +prefix `evaluate_cluster` derives is `hmac~` for a coded row while a +legacy bare row still lists under `` on its own. Prefix matching is +anchored at the start of the key, so two rows for the same client IP that +straddle the envelope never count each other, and `cluster_size` under-reports +for as long as both populations coexist. + +That undercount is accepted rather than bridged, for three reasons. +`cluster_size` is reported in the identify response and gates nothing: the +only place its value is read at all is a cache short circuit in +`evaluate_cluster` that tests whether a value is stored, not what it is, and +the `cluster_trust_threshold` and `cluster_recheck_secs` settings that a +reader might expect to gate on it have no readers in the code. The undercount +is bounded by the legacy bare-identifier read window in section 6.1 and ends +when the last pre-epic cookie expires. And bridging it would mean a second +prefix scan on every identify request for the whole of that window. It becomes +a real fault only if a later change makes the count gate something, and that +is the change that has to build the bridge. + +One global rule sits above every provider, and it is implemented: + +- **Identifier bounds.** A minted identifier obeys a global cookie-safe + alphabet (normatively `[A-Za-z0-9._~-]`, valid cookie octets with no + separators, whitespace, or control characters) and a global maximum of + **256 bytes**, stated here so dependent documents reference one number. + The bound applies to the identifier itself, not only its key form. Core + enforces the bound wherever an identifier enters the system, at mint + (both `generate` and the resolve endpoint), at cookie read-back, and at + cookie write. The constant is `MAX_EC_ID_LEN` in `ec/cookies.rs`. A + violating value is rejected outright and logged. No sanitizing rewrite + exists anywhere on the path, so an identifier survives byte for byte or + not at all, and the cookie value and the identity-graph key can never + silently diverge. + +## 4. Trait surface: minimalism rule + +Every trait method must have at least one production (non-test) caller in +the same change that introduces it. How the surface observed in PR #838 +resolved in the implementation: + +- `keys_equal`: **not shipped.** Its legitimate purpose (equivalent-envelope + comparison, #778) is served structurally, because read-back acceptance and + KV normalization both route through the provider, so no comparison method + exists to leave uncalled. +- `GeneratedEdgeCookie::response_headers`: **shipped, with a production + caller.** EC finalization applies provider-requested headers to the + outbound response, and the client-cycle resolve path returns them, which + is how a client-side provider requests further evidence from the page. + The draft banned the field when nothing consumed it. The consumer landed + in the same series, satisfying the rule the ban enforced. +- `IdentityInput.permissions` / `IdentityInput.consent`: **shipped and + populated.** The organic mint path passes the request's resolved + permission state and consent context so a provider can read them for + behavior beyond gating. The gate itself has already run before `generate` + is called, so a provider cannot use the fields to authorize itself. +- `required_permissions` on `DeviceProvider` and `PlatformGeo`: **present, + with an empty default and no enforcement point.** The draft removed the + method from both traits because PR #838's copies were decorative. The + implementation keeps one uniform declaration seam across all three traits + instead. The built-in device and geo providers declare empty sets, and + core enforces the declaration only for the EC provider (section 5), so + nothing reads as a gate that is not one. The geo circularity argument + stands unchanged and is restated in section 5. + +The implemented `EdgeCookieProvider` surface (`ec/provider.rs`): + +```rust +pub trait EdgeCookieProvider: Send + Sync + core::fmt::Debug { + /// Stable configuration key ("hmac"). + fn id(&self) -> &'static str; + /// Registered four-character code (provider-code-registry.md), the + /// `{code}~` namespace of every identifier the provider mints. + /// Mandatory, no default: a provider cannot exist without a unique + /// code, so identifiers from different providers can never collide. + fn code(&self) -> ProviderCode; + /// Derives an identifier from the provider's injected services and the + /// request evidence passed at call time. A client-side provider defers + /// here (returns no id) and mints later in resolve_from_client. + fn generate( + &self, + request_info: &dyn RequestInfo, + input: &IdentityInput<'_>, + ) -> Result>; + /// Whether `value` is a well-formed identifier this provider issues. + /// Default: the built-in HMAC shape (`<64 hex>.<6 alphanumeric>`). + fn accepts_id(&self, value: &str) -> bool { /* built-in shape */ } + /// The KV-key form of `value`. Default: lowercase the HMAC hash + /// segment, preserve the suffix. Opaque providers return the value + /// unchanged. + fn normalize_id_for_kv(&self, value: &str) -> String { /* ... */ } + /// Permissions this provider's data use requires. Default: none, so a + /// vendor-neutral provider requires no permission. + fn required_permissions(&self) -> PermissionSet { /* none */ } + /// Client-cycle counterpart to generate: mints from a value the page + /// posted to the resolve endpoint, after verifying it. Default: no-op, + /// so a server-side provider does not participate. See the + /// client-cycle spec. + fn resolve_from_client( + &self, + input: &ClientResolveInput<'_>, + ) -> Result> { /* ... */ } +} +``` + +Core owns the code envelope. At mint it prefixes the provider's value with +`{code}~`, at read-back it strips and checks the code before the provider's +`accepts_id` sees the value part, and the identity graph key preserves the +code verbatim around the provider's canonical form. A cookie carrying +another provider's code is treated as absent, never adopted, so switching +providers cannot silently mix identity populations, and a withdrawal always +acts on a key that can only belong to one provider. The built-in HMAC +provider mints `hmac~<64 hex>.<6 alphanumeric>` and dual-reads its +pre-envelope bare form for one release cycle so deployed cookies keep +working; the bare form belongs to hmac alone. Codes are allocated +append-only in `provider-code-registry.md`, and a leading digit is valid +(`51dd`). + +The draft's alternative shape (`parse` returning a typed `EcId`, +`graph_key_suffix`, `cluster_prefix`, `verify`, and a version-carrying +`GeneratedIdentity`) was not adopted. `verify`, provider versions, and +`mint_version` are tracked follow-up work with the migration spec. +Request data reaches a provider through injected services and the +`RequestInfo` passed at call time, not through a fixed parameter struct. +`RequestInfo` carries the evidence a provider in this workspace reads today, +which is the normalized client IP. Further evidence (headers, cookies, client +hints, the URL) is added to it as a defaulted accessor in the change that first +reads it, so an existing implementation keeps compiling and no accessor lands +ahead of the caller that consumes it. + +## 5. Permission enforcement is core's job, for EC providers + +Before minting through an EC provider, core resolves the request's +permission state and refuses when the provider's `required_permissions()` +are not all set. The gate is implemented in `EcContext`. The selected +provider is built once at request read time, its declaration is checked +against the resolved state, and generation is skipped (with a log line +naming the jurisdiction) when the requirement is not met. With no provider +selected, nothing may mint or use an identifier, so the gate is closed +rather than open by default. The enforcement point lands with the +permission model in PR #1045, and the permission model spec governs the +resolution machinery (country and region baselines, signals, and the +requires-signal floor). + +**Recognition and withdrawal always run**, permissions or not. Read-back +acceptance and withdrawal eligibility go through `accepts_id` with no +permission check, and withdrawal handling keeps the raw cookie value even +when the identifier is treated as absent, so an opt-out can always reach +the identity it revokes. A blanket execution gate would refuse to run the +provider in exactly the state an opt-out produces. + +The draft additionally specified an identity activation protocol (a +two-record commit point before any egress), rowless-cookie classification +and per-prefix withdrawal records, negative-record admission rules, and a +typed egress boundary (`AuthorizedIdentity`, +`RedactedRequestView`). **None of that is implemented in these PRs.** +Those positions remain recorded in the draft and are tracked as follow-up +work with the permission model spec, which owns identity-state persistence +and egress typing. The revision record lists them as deferred. + +The gate applies to EC providers **only**. Geo and device are ungated for +two different reasons, stated separately because only one of them is +structural: + +- **Geo: circularity.** The permission set is resolved from jurisdiction, + which is resolved by the geo provider. Gating geo on the resolved set is + unsatisfiable. `PlatformGeo::required_permissions` exists with an empty + default for interface uniformity, and nothing consults it on the lookup + path. +- **Device: host evidence is an explicit opt-in, not authorized by + selection defaults.** Device classification is not an input to permission + resolution. The neutral `builtin` classifier reads only the User-Agent + and makes no host call. The draft went further and made selecting a + fingerprint-reading device provider a startup error pending a separate + security design. The implementation instead ships `[device] provider = +"fastly"` as a selectable opt-in. The Fastly adapter injects a + `HostSignals` service carrying the TLS JA4 and HTTP/2 fingerprints, and + the provider uses them to strengthen the browser/bot gate that guards EC + writes. Identity rows persist the derived classification fields (the JA4 + class segment and a 12-hex-character hash prefix of the HTTP/2 SETTINGS + fingerprint), not raw fingerprints, and the neutral default persists + neither because the builtin provider produces no such fields. + +## 6. Selection, validation, and failure modes + +All configuration validation happens at **settings construction** +(`Settings::finalize_deserialized` runs every check below), so a +misconfiguration expressible in configuration alone is a startup error, +never a silent behavior change. A selection that only the running host can +satisfy (an injected vendor provider, or host fingerprints) fails loudly +when the provider is built, stopping the request rather than degrading. + +| Configuration state | Behavior | +| ------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `[ec] provider` set, its `[ec.providers.]` block missing | Startup error naming the missing block. There is no closed key list in core for EC, because a vendor key is legitimate when its block is present, so an unknown key with no block fails this same check. | +| `[ec.providers.]` block present, `provider` unset | **Startup error.** (In PR #838 this silently ran stateless. The half-migrated config becomes a production identity outage detected by revenue drop. Rejecting it is the fix.) An operator who genuinely wants stateless deletes the block. | +| `provider = "none"` (explicit stateless) | Valid, and means exactly what omitting the selector means. Any configured provider block alongside it is a startup error, the same stray-block rule as below. | +| A configured `[ec.providers.]` block that is not the selected one | **Startup error** (checked for the `hmac` block and every vendor block). An unreferenced block is almost always a mistyped selector or a stale block, and accepting it silently invites configuration drift. | +| A selected vendor key whose provider the adapter did not inject | Loud failure when the provider is built, naming the key, so the deployment never silently runs stateless. | +| `provider = "host-signals"` on a host that supplies no fingerprints | Loud failure when the provider is built. A host that cannot produce `HostSignals` cannot run the provider. | +| `provider = "client-fixed"` in a production build | Startup error. The demonstration provider is compiled only behind the `client-fixed-demo` cargo feature. | +| No `provider`, no providers block | Valid, the neutral default for that concern. | +| Deprecated `[ec] passphrase` | Migrated to `provider = "hmac"` with the passphrase in `[ec.providers.hmac]`, with a deprecation warning naming the new location. Both forms together are rejected so a half-edited file fails loudly instead of one form silently winning. | +| Any unknown key in `[ec]`, `[device]`, `[geo]`, or a built-in provider block | Startup error. `deny_unknown_fields` is on `Ec`, `DeviceConfig`, `GeoConfig`, and both built-in provider config structs, so a typo like `providr`, or a key from a deferred feature (`legacy_providers`, `rewrite_legacy`, `versions`), fails loudly. | +| `[device] provider` names an unknown key | Startup error. Valid keys are `builtin` (default) and `fastly`. | +| `[geo] provider` names an unknown key | Startup error. Valid states are unset (default, no geolocation), `none` (the same, spelled out), and `platform`. | +| `[geo] default_country` unset, or matching no `permissions.yaml` rule | **Startup error.** The value is the permission baseline for a request the geo provider leaves unmatched, so there must always be one and the value must resolve to a real rule. | +| An EC provider configured, no geo provider, `assume_single_jurisdiction` unset | **Startup error.** With geolocation off, every request resolves as `default_country`, so a visitor from any other jurisdiction silently receives the default jurisdiction's rules. That is acceptable only as an explicit operator decision. | + +One draft row was not adopted, the startup error for a minting provider +with no identity-graph store. `[ec] ec_store` remains optional, because the +portability adapters run without platform KV. The client-cycle resolve +endpoint refuses to mint when no graph is available (a cookie without a row +could never be withdrawn through the graph), and the organic path persists +the row whenever the graph is configured. Whether configuration should +force the pairing is follow-up work with the migration spec. + +Vendor provider blocks deserve their own note. Any `[ec.providers.]` +block whose key is not a built-in is captured in core as raw values (a +flattened map), and the adapter that injects the vendor provider +deserializes its own block into the vendor crate's config type. Core never +names a vendor, so a new provider adds nothing to core. The vendor crate +applies its own `deny_unknown_fields` when it deserializes. + +### 6.1 Provider switching: what a switch actually does + +Switching `[ec] provider` **retires every identity the previous provider +minted**. This section says exactly what that means, because a deployer has +to plan around it rather than discover it. + +The draft specified an ordered `legacy_providers` reader list, provider +`versions` with `mint_version` rotation, provenance tagging, and retirement +evidence rules, as the mechanism that would carry identities across a +switch. **None of that is implemented in these PRs.** The keys are rejected +as unknown, and the design is tracked follow-up work with the migration +spec. Until it lands there is no continuity across a switch of any kind. + +An earlier version of this section claimed shape-based continuity, that old +cookies stay recognized when the newly selected provider accepts their +shape. That is not what the code does and never was once core took +ownership of the `{code}~` envelope (§5). Ownership is decided on the code +prefix **before** any provider is asked about the shape, so a newly selected +provider rejects every identifier the previous one minted, whatever its +shape, because the code differs. + +What a switch does, precisely: + +- **Read-back.** Every identifier carrying the retired provider's code is + treated as absent. It never becomes the request's active identity, never + egresses to a partner, and is rejected on the pull-sync, batch-sync and + admin paths too. This is the §5 guarantee and it is the half of the + behavior that matters most: two providers' identity populations can never + mix. +- **The browser cookie.** A later withdrawal still expires the `ts-ec` + cookie, because that path keys off the raw cookie being present rather + than off who owns it. The browser stops carrying the retired identifier. +- **The identity-graph rows.** A later withdrawal does **not** tombstone the + retired provider's rows. Core cannot derive their canonical keys, because + the canonical form is the owning provider's own normalization and the + owning provider is no longer configured. Those rows stay as they are until + their one-year entry TTL expires. + +**What a deployer must do about revocation.** Treat a provider switch as a +one-way retirement of the identity population, and deal with the previous +provider's rows before or alongside the switch, not after. A withdrawal +that arrives after the switch clears the browser but leaves the row. Every +row a retired provider wrote shares that provider's `{code}~` key prefix, so +the set is identifiable and can be listed and cleared with the platform's +own KV tooling. Either clear it at the switch, or accept that the rows +persist until the one-year TTL expires and that withdrawals arriving in the +meantime are recorded only in the browser. Do not switch providers while +identities are live that the deployment may still be obliged to revoke in +the identity graph. + +The `cluster_fallback` degradation policy from the draft is likewise +deferred with the cluster capability itself. + +### 6.2 Runtime failure modes + +Startup validation covers configuration. This covers a healthy +configuration meeting an unhealthy runtime. Implemented behavior, each row +logged, none silent: + +| Failure | Behavior | +| -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `generate` returns an error | No identity this request. The organic caller logs at error level and the request proceeds stateless. No cookie is written. | +| A provider mints an identifier outside the global bounds | Rejected at mint, never rewritten. The organic path yields no identity. The resolve endpoint returns 400. | +| Identity-graph write fails at mint | The mint is undone (no identifier, no cookie), with the error logged. The resolve endpoint returns 503. The next eligible request retries. | +| The host-signals provider finds no TLS/HTTP-2 fingerprints | Defers with a warning. No identity this request, and no degraded IP-only identifier is minted under the host-signals name. | +| Geo lookup **fails** (the provider errors) | Every permission resolves to the requires-signal floor, and the failure is logged at error level. The failure is **not** papered over with the `default_country` baseline. | +| Geo resolves **no location**, or a country/region with no rule | The `[geo] default_country` baseline applies. This is the configured-default case, deliberately distinct from the failure row above (`GeoStatus` in `ec/consent.rs`). | +| An incoming cookie value fails the bounds at read-back | Treated as absent, with a warning naming the source. | + +The distinction between a failed lookup and no location is resolved in +core, where `EcContext::read_from_request_resolving_geo` runs the +configured geo provider itself and classifies the outcome, so every +adapter reports the two states identically. The draft's remaining matrix rows (rowless +withdrawal records, promotion, the negative-intent outbox, the identity +safety breaker, cluster-listing degradation) belong to the deferred +material of sections 5 and 6.3. + +### 6.3 Storage contract + +The draft specified a delimiter-free physical key grammar with fixed-width +segments, a provider-code registry, record classes for family revocation, +authority state, negative-intent outbox, rowless withdrawal, and deployment +metadata, wire schemas with known-answer vectors, and a per-field graph-row +contract. The provider-code registry is now implemented: codes are +allocated in `provider-code-registry.md`, carried as the `{code}~` prefix +of every minted identifier, and therefore present in every graph key. The +key grammar differs from the draft in one deliberate way, a tilde separator +instead of delimiter-free fixed width, because pre-envelope bare +identifiers remain deployed and a code such as `51dd` is valid hex, so +delimiter-free parsing could misread a legacy identifier during the +migration window. The remainder (record classes, family revocation, +authority state, outbox, rowless withdrawal, wire schemas, per-field +contract) is not implemented in these PRs and stands as recorded design for +the follow-ups. + +The implemented storage today keys the identity graph by the selected +provider's `normalize_id_for_kv` output verbatim. For the built-in HMAC +provider that is the identifier with the hash segment lowercased, which is +today's key, so every pre-epic row stays reachable and the pre-epic +cluster prefix listing stays intact. For an opaque provider the identifier +itself is the key. Rows carry the same JSON envelope as before the epic, +extended with the derived device-classification fields noted in section 5. + +## 7. Composition root and adapter parity + +Provider construction happens in one place per concern, in core, called by +every adapter. No adapter wires a concrete implementation directly into the +request path: + +- `build_provider` (`ec/provider.rs`) constructs the selected EC provider, + injecting the host's `HostSignals` when supplied and matching an + adapter-injected vendor provider by its `id()`. The provider is built + once per request during `EcContext` construction and reused for + read-back, the permission gate, and minting, so the per-request + triple-build observed in PR #838 (cloning the secret into a fresh box up + to three times per request) is gone. +- `build_device_provider` (`ec/device.rs`) returns the builtin classifier + unless `fastly` is selected, in which case the adapter's closure builds + the host-evidence provider. +- `build_geo_provider` (`platform/mod.rs`) returns `DisabledGeo` unless + `platform` is selected, in which case the adapter's host geo + implementation is used. All four adapters (Fastly, Axum, Cloudflare, + Spin) route their host geo through this selector when they assemble + their runtime services, verified in each adapter's platform wiring. + +All four adapters construct the EC request state through the same core +constructors (`EcContext::read_from_request_resolving_geo` and its +variants), so selector behavior and the geo failure classification are +identical everywhere. The cross-adapter parity suite +(`trusted-server-integration-tests`) asserts geo response parity across +adapters. The EC API routes are Fastly-only today, as section 1 notes, and +the Spin adapter's route list records why. + +The draft's adapter capability matrix (declared per-record-class +consistency semantics, durability and retention proofs, activation and +lease qualification) is **not implemented in these PRs** and is tracked +follow-up work. The matrix's motivating rule is preserved for that +follow-up, which is that "has KV" says nothing about whether a revocation +is observable, so eligibility for identity features must eventually be +declared and checked, not assumed. + +## 8. Crate layout and CI + +Host and vendor provider crates live in nested directories grouped by +capability, with flat package names following the existing convention: + +- `crates/device/fastly` is package `trusted-server-device-fastly`. +- `crates/geo/fastly` is package `trusted-server-geo-fastly`. +- `crates/edgecookie/` is the documented home for vendor EC + crates. The directory currently holds only a README, because the + built-in providers live in core and no vendor crate exists yet. + +The draft mandated flat directories (`crates/trusted-server-geo-fastly`) +and banned placeholder directories. The implementation diverges on both +points. Nested directories scale per vendor as providers multiply, package +names already carry the flat convention, and the README stakes out the +location before the first vendor crate lands. Both divergences are +recorded in the revision table. + +Every new crate is in the `.cargo/config.toml` aliases (`check-fastly`, +`clippy-fastly`, `test-fastly`, `build-fastly`), so the provider crates are +linted with `-D warnings` and tested by the same gates as every other +workspace member, closing the PR #838 gap where new crates compiled only +transitively. + +## 9. Behavior preservation notes + +Two defaults chosen for neutrality change effective behavior on existing +Fastly deployments. Both are called out in the migration spec and must be +prominent in release notes: + +- **Bot gate.** The pre-provider EC bot gate required JA4 and platform + class. The default `builtin` classifier is User-Agent only, so the gate + is weaker by default. The stronger gate is available as `[device] +provider = "fastly"` rather than being startup-rejected as the draft + specified. Release notes call out the weaker default rather than + presenting selection alone as authorization. +- **Geo.** With no geo provider, jurisdiction resolution falls to the + required `[geo] default_country`. The permission model constrains the + combination so it cannot silently grant permissions to mis-attributed + traffic. The default must resolve to a real `permissions.yaml` rule, a + deployment running an EC provider without geo must set + `assume_single_jurisdiction = true`, and a failed lookup resolves to the + requires-signal floor instead of the default. The default flip landed in + the same series as those constraints, honoring the draft's sequencing + requirement that the constraint exist before the flip. + +## 10. Testing strategy + +Implemented, in the crates named: + +- Round-trip tests with a non-default provider, proving an opaque + identifier survives read-back byte for byte (`ec/mod.rs`) and the + client-cycle value survives the full scenario as cookie and KV key + (`ec/resolve.rs`). +- Delegation tests proving the injected-provider wrapper forwards + `accepts_id` and `normalize_id_for_kv` to the inner provider, so a + vendor identifier is never dropped by the built-in defaults. +- Gate tests proving the HMAC provider's declared requirement blocks + generation until the permission is set, and that a provider declaring + nothing requires nothing. +- Settings validation tests covering the section 6 table, including the + missing block, the block without a selector, explicit `none`, the stray + block, unknown selector keys for all three concerns, unknown fields in + every section, the deprecated passphrase migration with its both-forms + rejection, `default_country` validation, and the jurisdiction + acknowledgment. +- Geo builder tests showing the default selects no geo, `none` selects no + geo explicitly, and `platform` selects the host implementation. +- Host-signals provider tests covering minting from fingerprints, + deferring without them, and the loud failure of a selected but + uninjected vendor provider. + +Deferred with their features are the fixture-driven provider conformance +suite, legacy-reader tests, and the parity cases for capability-mismatch +startup failures. + +## 11. Implementation order + +As landed: + +1. **PR #1043, the Edge Cookie provider seam.** The trait with recognition + and KV normalization, the global identifier bounds, selection and + validation, the vendor block capture, the deprecated-passphrase + migration, and the round-trip proof with a non-default provider. +2. **PR #1044, device and geo selection.** `DeviceProvider` with the + builtin default and the opt-in Fastly host-evidence provider, + `PlatformGeo` selection with the no-geo default, all four adapters + routed through the shared builders, and the opt-in host-signals EC + provider (carrying the open review question of section 2). +3. **PR #1045, the permission model.** The enforcement point for + `required_permissions`, the `default_country` requirement and + jurisdiction acknowledgment, and the failed-lookup floor. That change + has its own spec, which this document cross-references rather than + restates. + +The draft's step 4 warning (do not flip the geo neutral default before the +permission model exists) was honored. The flip and its constraints landed +together in the permission model change. + +## 12. Divergences from issue #778 + +This spec supersedes #778 on the following points, so implementation has +one acceptance contract: + +| #778 says | This spec says | Why | +| ------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Identifier comparison is a provider operation (`keys_equal`) | Comparison is structural. Read-back acceptance and KV normalization route through the provider, so no comparison method exists (§3, §4) | Satisfies the same requirement with no method to leave uncalled | +| A provider can return response headers | Kept, with a production consumer. EC finalization applies them, and the client-cycle path uses them (§4) | The caller the minimalism rule demands landed in the same series | +| One built-in provider (HMAC) preserving today's behavior | HMAC preserved verbatim, plus the opt-in host-signals built-in (open question, §2) and the demo client-cycle provider | A switch retires the previous provider's identity population outright (§6.1); carrying identities across a switch (`legacy_providers`) remains follow-up work with the migration spec | + +## 13. Revision record vs the 2026-07-31 draft + +One row per divergence between the 2026-07-31 draft and the implementation +this revision describes. + +| Draft position | Implemented position | Why | +| ----------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Trait surface is a canonicalizing `parse` returning a typed id, plus `graph_key_suffix`, `cluster_prefix`, and `verify` | `accepts_id` (recognition) plus `normalize_id_for_kv` (KV key form), defaults matching the built-in shape. No typed id, key suffix, cluster capability, or `verify`. `keys_equal` stays out, as the draft required. | Recognition and KV keying are the two operations core performs today. A byte-for-byte round-trip test with a non-default provider pins the contract. | +| `GeneratedEdgeCookie::response_headers` and `IdentityInput.permissions` / `.consent` banned as speculative surface | Shipped with production consumers. Finalization applies provider headers, the resolve path returns them, and the organic mint path populates the input fields. | The client-cycle resolve path landed in the same series and is their caller, satisfying the minimalism rule the ban enforced. | +| Identifier bounds enforced at mint and parse | Enforced at mint (`generate` and the resolve endpoint), cookie read-back, and cookie write. Violations rejected outright, never rewritten. `MAX_EC_ID_LEN` in `ec/cookies.rs`. | Every identifier entry point is covered, and the pre-epic sanitizing rewrite was removed as a silent-divergence hazard. | +| `provider = "none"` is valid alongside `legacy_providers` blocks | `none` (or an omitted selector) with any configured provider block is a startup error. | No `legacy_providers` exists in these PRs, so a block alongside statelessness can only be a mistake. | +| Every selection key is closed and unknown keys are startup errors | Device and geo keys are closed. EC vendor keys are open. Unknown blocks are captured as raw values in core, the adapter deserializes its own block, and a selected key with no injected provider fails loudly. | Core never names a vendor, so a vendor provider adds no core change. | +| Capability mismatch is a startup error at adapter wiring time | Configuration coherence fails at startup. A host-capability mismatch (missing `HostSignals`, uninjected vendor) fails loudly when the provider is built, stopping the request. | The adapter capability declaration that would move the check to startup is deferred with the capability matrix. | +| A minting provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to mint without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | +| `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw fingerprints. | Selection is an explicit operator opt-in and the neutral default makes no host fingerprint call. | +| The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host fingerprint processing, so the review decides whether the provider ships in the series. | +| Geo default flip sequenced into the later permission-model step, with an acknowledgment guard | Landed as specified in the same series, with the default of none, `default_country` required and validated against `permissions.yaml`, the `assume_single_jurisdiction` acknowledgment, and a failed lookup resolving to the requires-signal floor with error logging (`GeoStatus`, resolved in core so all adapters agree). | The permission model shipped in PR #1045, so the constraints exist where the draft required them. | +| All adapters serve the full EC feature set identically | Selector behavior is identical through the shared builders and core constructors. The EC API routes (identify, batch-sync, ec/resolve) are Fastly-only, documented in the Spin route list. | The portability adapters do not yet wire platform KV, and the gap is documented rather than silent. | +| Conformance suite, adapter capability matrix, delimiter-free key grammar, `verify`, `legacy_providers`, `versions` / `mint_version` | None of these are in PR #1043 or #1044. All are tracked follow-up work, deferred, not silently dropped. | The shipped seam did not need them, and each returns with the feature that gives it a production caller, per the spec's own minimalism rule. | +| `required_permissions` removed from the device and geo traits, added to the EC trait only at the permission-model step | Present on all three traits from the start, with empty defaults. Core enforces the EC declaration (gate in `EcContext`, landing in PR #1045). No device or geo enforcement point exists. | One uniform declaration seam, with an empty default that gates nothing, avoids the decorative-gate hazard while keeping the interface stable. The geo circularity stands. | +| Flat crate directories (`crates/trusted-server-geo-fastly`), no placeholder directories | Nested directories per capability (`crates/device/fastly`, `crates/geo/fastly`, `crates/edgecookie/`), flat package names. `crates/edgecookie` ships a README before its first crate. | Nested directories scale per vendor, package names already carry the naming convention, and the README stakes out the vendor location. | diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md new file mode 100644 index 0000000000..efe904a96d --- /dev/null +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -0,0 +1,516 @@ +# Design Spec: Provider and Permission Model, Migration and Rollout + +**Status:** Revised against the implemented series (PRs #1043-#1047), +2026-08-25. The §8 sign-off rows remain the series' decision ledger; rows the +implementation now satisfies are marked with their PR so the task force can +ratify rather than re-litigate. +**Author:** Engineering +**Issue references:** #777-#781 (epic) +**Related specs:** `2026-07-30-pluggable-providers-design.md`, +`2026-07-30-permission-model-design.md`, +`2026-07-30-integration-response-header-hook-design.md` (not implemented in +the series, see its status) +**Last updated:** 2026-08-25 + +> **Context.** The provider/permission epic is a breaking change to a live +> identity system. PR #838's review showed that the riskiest part of such a +> change is not the new code but the transition, meaning silent +> misconfiguration modes, undeclared behavior changes discovered by deleted +> tests, and no written statement of which pre-change behaviors were +> guaranteed to survive. This spec is that statement. The epic has now been +> implemented as five stacked PRs (#1043-#1047, the "series" below), and this +> revision reconciles the spec against that series, keeping §2's matrix and +> §8's ledger as the record the task force ratifies. Any further +> implementation PR must reconcile its diff against §2's matrix and list +> every deliberate divergence in its description. + +## The implemented series + +Five stacked PRs, verified against the tree at +`split/5-response-hook-docs` (the head of PR #1047; the branch is rebuilt on +each rebase, so the PR is the stable reference). A sixth PR, #1084, adds the +integration provider seam spec on top of the series and changes no code: + +1. **PR #1043, the Edge Cookie provider seam.** The `EdgeCookieProvider` + trait (`id`, `generate`, `accepts_id`, `normalize_id_for_kv`, + `required_permissions`, `resolve_from_client` in + `crates/trusted-server-core/src/ec/provider.rs`) routes identifier + creation, cookie read-back, and identity-graph keying through the + selected provider. Global identifier bounds are enforced by core at every + entry point (`MAX_EC_ID_LEN = 256` bytes and the cookie-safe alphabet + `[A-Za-z0-9._~-]` in `ec/cookies.rs`), rejecting loudly and never + rewriting, so the cookie value and the graph key cannot silently + diverge. `[ec] provider = "none"` spells explicit statelessness. A + configured `[ec.providers.*]` block with no selector, an unreferenced + block, and a selector with no block are each startup errors. The + deprecated `[ec] passphrase` form still starts for one release cycle, + mapping to `provider = "hmac"` with a deprecation warning, and a + configuration carrying both forms is rejected. +2. **PR #1044, device and geo selection.** `[device] provider` selects + `builtin` (User-Agent only, the default) or the opt-in `fastly` + classifier (`crates/device/fastly`, TLS JA4 and HTTP/2 signals). + `[geo] provider` defaults to no geolocation, with `"platform"` opting in + to the host lookup (`crates/geo/fastly`) and `"none"` spelling the + opt-out. All four adapters route geo through the one + `build_geo_provider` selector. The provider configuration structs carry + `deny_unknown_fields`, so a mistyped key fails startup. The host-signal + Edge Cookie provider (`[ec.providers.host-signals]`) ships opt-in. + Whether the host-signal surface stays is an **open review question** + (sign-off row 22), not a settled decision. +3. **PR #1045, the permission model.** Permission names follow the IAB + Privacy Taxonomy Data Uses (`permissions.yaml`, compiled into the + build). Signal precedence is fixed in code, most restrictive first, + meaning an opt-out (Sec-GPC, GPP sale opt-out, US Privacy) suppresses + the Data Uses the policy revokes even against a consenting TCF record, + a present but undecodable record blocks baseline grants (fail-closed), + and only then does a TCF record decide its mapped Data Uses. + Destructive withdrawal is narrow, where only a TCF record refusing + storage in a jurisdiction whose baseline did not grant storage expires + the cookie and writes the identity-graph tombstone, and opt-outs never + destroy an issued identifier. Sharing beyond the edge (bidstream + `user.id`, the identify response, partner pull sync) requires storage + plus personalised-ad selection, the same pair that gates bidstream + EIDs. `[geo] default_country` becomes required, a failed geo lookup + resolves at the requires-signal floor instead of the default, and a + no-geo deployment running an Edge Cookie provider must set + `[geo] assume_single_jurisdiction = true`. +4. **PR #1046, the hardened client-cycle resolve endpoint.** + `POST /_ts/api/v1/ec/resolve` enforces publisher-origin `Origin` (403), + a `text/plain` or `application/json` content-type allowlist (415), a + body size cap (413), the global identifier bounds on the created + identifier (400), and refusal to replace a different identity already on + the request (409). The identity-graph row is written before the cookie + is set, with no graph meaning no identifier is created and a graph + write failure returning 503. Every response carries + `Cache-Control: no-store`. A non-HttpOnly `ts-ecr` marker cookie tells + the page script a resolve succeeded, fixing the re-post loop the + HttpOnly Edge Cookie would otherwise cause, and a Rust test pins the + marker name and the demo's fixed word against the page script source. + The client-fixed demonstration provider compiles only under the + `client-fixed-demo` cargo feature and production builds reject + selecting the demo at startup. +5. **PR #1047, the documentation set.** Configuration reference for + `[ec]`, `[device]`, and `[geo]`, the Edge Cookie guide rewritten around + providers and the permission model, and the example configuration + documenting every selector. The integration response-header hook was + **stripped from the series** because the hook had no consumer, which is + that spec's own rule for speculative surface. The hook spec is retained + as the design bar for when its first consumer arrives. + +--- + +## 1. Scope + +Covers the transition of existing deployments from the hard-wired EC / +device / geo behavior to the provider architecture and permission model. +Applies to every implementation PR in the epic (now the implemented series +PRs #1043-#1047 and any follow-up), and to the operator-facing migration +guide that ships with the last of them (still outstanding, §7). + +## 2. Behavior-preservation matrix + +For each decision the system makes today, the target behavior after the +epic, and whether that is a preservation or a declared change. **Silent +changes are defects.** PR #838 changed six of these without declaring any. +The Status column now also records where the implemented series stands, +naming the PR. Rows the series did not touch keep their design-target text +for the follow-up work that will implement them. + +| # | Decision (today) | After epic | Status | +| --- | -------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved. Implemented (#1045), where `gdpr-eu` requires a signal for storage, with an EU-27 plus EEA coverage test | +| 2 | US-state request, sale opt-out → no EC, existing EC expired and tombstoned even beside a consenting TCF string | Opt-out suppresses use but does not delete the identity. Lifting the opt-out restores the identity. The positive contextual-auction projection (permission §7's `ContextualAuctionView`) remains the design for what may still dispatch | **Changed deliberately.** The separation is implemented (#1045), with opt-outs stripping egress and never tombstoning. The contextual projection is not implemented (sign-off 4 open), and today EIDs and `user.id` are withheld | +| 3 | US-state request, no signals at all → no EC (fail-closed) | The draft's recipe kept this via a `requires_signal` US rule with an extended grant-signal class | **Changed by the shipped default (#1045), flagged.** `permissions.yaml` ships US as a `granted` baseline whose uses drop on any opt-out, so a no-signal US request receives an EC. A deployer edits the yaml to differ | +| 3a | US-state request, explicit not-opted-out GPP/USP value → EC allowed | No US signal grants anything. N/A, absent, reserved, and unknown values grant nothing | Implemented (#1045) in a stricter form than drafted, where signals only revoke, so the drafted "explicit not-opted-out may grant" class does not exist (sign-offs 3, 17) | +| 3b | US-state request, TCF record refusing Purpose 1, no US opt-out → no EC | Refusal beats coexisting non-TCF grant signals | Implemented (#1045), where an authoritative TCF refusal revokes its mapped uses. Under the shipped `granted` US baseline the refusal suppresses without tombstoning | +| 3c | Consent-record conflict modes, expiry, KV fallback, proxy mode | Per the permission spec §4.4 matrix, whose changed row is that malformed-present blocks acquisition | Malformed-present fail-closed is implemented (#1045). The rest of the consent normalization pipeline is carried forward unchanged by the series | +| 3d | Valid plus expired consent records: conflict resolution can select the expired record | Expired sources drop before conflict resolution | Open. Not addressed by the series | +| 3e | Only the GPP sale field (and USP) is consulted, with sharing/targeted opt-outs ignored | Sale, sharing, and targeted-advertising opt-outs deny the personalised-ads uses, and none affects storage or destroys identity | Partially implemented (#1045), where sale, USP, and Sec-GPC are honored and never destructive. The sharing and targeted-advertising GPP fields are not yet decoded (sign-off 3 open) | +| 3f | Non-privacy-state US traffic (for example Wyoming) is non-regulated → EC allowed | Country-level `US` is a protective floor, region rules may be stricter, and regionless traffic never degrades to non-regulated | Implemented (#1045), where `permissions.yaml` maps country `US` to `us-opt-out`, with state rows able to override | +| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | The draft discontinued these for new rows | **Contradicted by the series, flagged.** Rows still persist device signals, including fingerprint hash prefixes, when the opt-in providers run (#1044/#1046). Tied to the open host-signal question (sign-off 22) | +| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB with citation and sign-off | **The shipped default adopts `granted` storage for GB (#1045), flagged.** The citation and sign-off the draft required do not exist yet. The task force owns this row | +| 5 | No country resolvable (geo failure) → no EC (fail-closed) | Protective failure profile, where permissions resolve at the requires-signal floor and `default_country` is reserved for unmatched requests in acknowledged static-jurisdiction mode | Implemented (#1045), where a failed lookup resolves at the requires-signal floor, logged at error level, and never falls back to `default_country` (sign-off 18) | +| 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, identity never tombstoned | Refusal blocks new grants everywhere, and existing identity is never tombstoned where the baseline is `granted` | Implemented (#1045), where an authoritative refusal revokes its mapped uses everywhere and withdrawal is scoped to non-granted baselines | +| 7 | Country resolved but in no regulation list → EC created, EIDs pass through | Governed by the deployment's default rule. The implementation expresses this as the required `[geo] default_country`, naming the `permissions.yaml` rule for unmatched requests | Implemented (#1045) with a changed mechanism, since no `rules.default` entry exists and `default_country` is required and validated at startup | +| 8 | Opt-out signal outside US states → ignored today | Mapped use restrictions are honored globally, and opt-outs never tombstone identity | Implemented (#1045), where the signal mapping is jurisdiction-free and suppresses even TCF-consented uses, without destruction (sign-off 1) | +| 9 | Fastly bot gate requires JA4 plus platform class before KV-backed EC writes | The draft deferred host fingerprinting and startup-failed `[device] provider = "fastly"` | **Contradicted by the series, flagged.** The `fastly` device provider ships opt-in with `builtin` (UA-only) as the default (#1044). Whether the host-signal surface stays is sign-off 22, open | +| 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral default flips only together with the permission model's jurisdiction guard, never in an intermediate step | Implemented as sequenced (#1044 kept the platform default, and #1045 flipped geo off by default together with `default_country` and the acknowledgment guard) | +| 11a | Raw EC egress on jurisdiction-gated paths today (`user.id`, EIDs, identify, pull sync) | Gated by the sharing pair (storage plus personalised-ads), at least as strict as today for every path | Implemented (#1045), where `ec_sharing_allowed` gates `user.id`, the identify response, and pull sync, and `gate_eids_by_permissions` gates EIDs, all on the same pair | +| 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header without today's jurisdiction gate | Gated by the egress inventory (both purposes) | Open. Not implemented by the series (sign-off 8) | +| 11c | Batch sync only authenticates the S2S caller and checks row state | Gated by stored-provenance recompute, with legacy rows failing closed until backfilled | Open. Not implemented. Batch sync still checks caller and row state, with missing or withdrawn rows collapsing to ineligible (sign-offs 7, 13, 25) | +| 12 | EC generation succeeds without a configured identity-graph store | The draft required an openable graph store at startup for a creating provider | **Changed from the draft (#1043/#1046).** No startup requirement ships. The implemented rule is per request, where no available graph means no identifier is created (no phantom cookies), organic and resolve paths alike | +| 13 | Cookies created by graphless deployments have no graph row | Rowless proof under the graphless-migration flag, capped `w` records, disclosed cookie-only expiry | Open. None of the rowless machinery is implemented. A cookie without a row is simply never shared (sign-offs 21, 29, 30) | + +Rows 3, 4, and 7 are policy decisions, not code decisions. In the +implemented series they live in `permissions.yaml`, which is compiled into +the build precisely so the policy stays visible and reviewable in version +control, made explicitly by maintainers rather than implied by an +implementation. The shipped defaults for rows 3 and 4 are flagged above for +the task force. + +## 3. Identity stability guarantee + +Today's EC identifier is `{64-hex}.{6-char}` where the 64-hex part is +deterministic, `HMAC-SHA256(passphrase, normalized_ip)`, and the 6-char +suffix is random per creation (an existing test asserts two creations +differ). Full identifiers are therefore not reproducible by design, and no +test may pretend otherwise. What stability means, precisely, for a +deployment that selects `provider = "hmac"` and carries its passphrase over +verbatim, with where the series stands on each: + +- **The deterministic prefix is stable per inputs.** Implemented and + tested (#1043, + `generate_hmac_ec_id_is_stable_per_parts_and_collision_resistant`). + The committed known-answer vector the draft required (fixed passphrase + plus IP → exact expected 64-hex prefix, failing CI on divergence) is + **not yet committed** and remains open work. +- **Existing cookies stay parseable.** The `is_valid_ec_id` shape check is + unchanged and the HMAC provider's `accepts_id` delegates to that check, + so pre-series `ts-ec` values keep resolving and their graph rows (keyed + through `normalize_id_for_kv`, which lowercases the hash exactly as the + pre-series normalization did) remain reachable. +- **The hash prefix keeps its semantics.** `ec_hash` remains the 64-hex + prefix, preserving both its stability and its deliberate collision + across identifiers created from the same IP. +- **Cookie name, attributes, and max-age are unchanged.** `ts-ec`, + `Domain` derived from the publisher domain, `Path=/`, `Secure`, + `SameSite=Lax`, `HttpOnly`, one-year `Max-Age` (`ec/cookies.rs`). +- **New global bounds (#1043), a declared addition.** Every identifier, + whatever provider created the value, must fit 256 bytes and + `[A-Za-z0-9._~-]`. Out-of-bounds identifiers are rejected loudly and + never rewritten. + +## 4. Configuration migration + +Old shape: + +```toml +[ec] +passphrase = "replace-with-32-plus-byte-random-secret" +``` + +New shape: + +```toml +[ec] +provider = "hmac" + +[ec.providers.hmac] +passphrase = "replace-with-32-plus-byte-random-secret" +``` + +Requirements, each marked with its implementation state: + +1. **The transition has a dual-read release; loud rejection comes one + release later.** Implemented (#1043) in the simple form, where the + current release accepts the old shape, mapping `[ec] passphrase` to the + `hmac` provider internally and logging a deprecation warning per + startup, and accepts the new shape. A configuration mixing the two + forms is rejected, not reconciled. Ordering remains strictly + reader-first, because pre-series binaries reject the new `[ec]` keys + and the new `[device]` / `[geo]` sections as unknown fields, so a fleet + converges binaries first and flips configuration second. The follow-on + release that rejects `[ec] passphrase` outright with a message naming + the new location is scheduled, not yet in the tree. + + The draft's full N+1/N+2 interim (the negative-record read/write + matrix, the `permissions_v2` model promotion, the `m00` mirror, and the + rollback-floor machinery) is **deferred** together with the durable + suppression design that motivates all of that machinery (sign-off rows + 11, 16, 19, 20). The series needs none of that machinery because the + only durable negative artifact today is the identity-graph withdrawal + tombstone the pre-series lifecycle already wrote, so a binary rollback + strands no new state. The interim matrix stays recorded in row 20 as + the design bar for when durable suppression ships. + +2. **Adapter qualification is a pre-ratification prerequisite.** + **Deferred.** No machine-readable capability matrix exists. The + series' de facto posture is that the identity endpoints (resolve, + identify, batch sync) route only on the Fastly adapter, and the other + adapters run without them, which is the stateless-identity path of + requirement 3. PSL vendoring and the pinned GPP corpus (draft + requirement text) remain prerequisites of the response-hook and GPP + follow-ups respectively (sign-offs 23, 28, 32). +3. **Revocation-eligible storage is a per-adapter gate, and ungated + adapters migrate stateless.** Direction implemented, since + `provider = "none"` spells explicit statelessness (#1043) and + non-Fastly adapters deliberately do not route the identity endpoints + (#1046), matching identify and batch sync. The formal per-adapter + capability gate is deferred with requirement 2 (sign-off 12). +4. **The graphless migration.** **Superseded in the series.** No + graphless flag, stub backfill, or rowless classification exists, and + the draft's 4b startup requirement (a creating provider must have an + openable graph store at boot) did not ship. The implemented rule is + request-scoped, where no available identity graph means no identifier + is created, on the organic path and on resolve alike, so a graphless + deployment runs identity-less rather than failing startup (§2 row 12). + The rowless design returns, if at all, with sign-offs 21, 29, and 30. +5. **The graph schema change is expand-contract.** **Deferred.** Rows + carry the existing schema. No provider/version field, per-permission + provenance, policy revision, family ID, model epoch, or rollback floor + ships in the series. This work belongs to the durable-suppression and + provenance follow-up (sign-offs 11, 16, 19, 20, 25). +6. **Half-migrated fails loud.** Implemented (#1043). An + `[ec.providers.hmac]` block with no `provider = "hmac"` selector is a + startup error, as is a selector whose block is absent and an + unreferenced block alongside a different selection. The exact state + that validated green and silently created zero ECs in PR #838 now + refuses to start. +7. **PR #838-era keys.** **Revised.** The draft required rejecting + `provider = "host-signals"` and `provider = "client-fixed"` as unknown + keys. The series instead ships both deliberately. `host-signals` is a + supported opt-in selection (#1044) pending the sign-off 22 review, and + `client-fixed` exists only under the `client-fixed-demo` cargo feature + with production builds rejecting the selection at startup (#1046). + Genuinely unknown keys still fail loud through the unknown-selector + error and `deny_unknown_fields`. +8. **Provider switches go through legacy readers.** **Deferred.** No + `legacy_providers` mechanism exists. Today a provider switch on a + deployment with live identities strands the outgoing provider's + cookies (the new provider's `accepts_id` rejects them and identity + restarts). The reader-chain design remains the bar for when a second + server-side provider makes switching real. +9. **The example config ships the migrated shape.** **Revised.** The + example ships Edge Cookie identity off by default, with each selector + and its block documented together and commented together, and + `default_country = "FR"` uncommented. The draft wanted the happy path + uncommented. The series instead closes PR #838's silent-stateless trap + by validation (requirement 6), so a half-uncommented configuration + refuses to start rather than running stateless. Static-geo examples + pair `default_country` with the commented + `assume_single_jurisdiction` acknowledgment, which startup enforces + whenever an EC provider runs with no geo provider (#1045). +10. Every misconfiguration in the providers spec §6 table fails at + **startup**. Implemented for configuration errors (#1043-#1045: + selector/block mismatches, unknown keys, unknown selector values, + missing `default_country`, missing acknowledgment, demo provider in a + production build). The pre-series passphrase minimum and placeholder + rejection carry over unchanged. One residual is declared. A + selected vendor or host provider the running adapter does not inject + fails per request, loudly, because adapter injection is a build fact + the settings layer cannot see. Closing that residual belongs to the + capability-matrix follow-up (requirement 2). +11. Validation is split into two named layers. Partially implemented. + Structural validation runs at `ts config push` / `ts config validate` + (the CLI deserializes and validates the typed settings, with a + regression test covering environment overlays) and again at startup, + and startup additionally validates deployment facts only startup can + see (provider availability in the build, `default_country` against + the compiled `permissions.yaml`, the acknowledgment rule). The + machine-readable adapter capability profile for push-time deployment + pre-checks is deferred with requirement 2. + +## 5. Minimal-divergence migration recipe (operator-facing) + +"Keep exactly today's behavior" is not fully achievable, and the recipe's +name says so. The divergences that actually shipped in the series, each a +matrix row: global honoring of mapped opt-outs (row 8), an authoritative +TCF refusal blocking new grants everywhere (row 6), the protective +geo-failure floor (row 5), country-wide protective US handling (row 3f), +malformed-present blocking acquisition (row 3c), and the shipped `granted` +US and GB storage baselines (rows 3 and 4, flagged for ratification). +Divergences the draft listed that have **not** shipped and remain deferred: +the sharing/targeted GPP fields (row 3e), the grant-signal class (row 3a, +now stricter instead), and the batch-sync provenance gate (row 11c). + +Policy now lives in `permissions.yaml`, compiled into the build. There is +no `[permissions]` block in `trusted-server.toml`, so the draft's +partial-policy trap (a TOML table containing only a permissive default) +cannot be written at all. A deployer edits the yaml and rebuilds, which +keeps every policy change reviewable in version control. The shipped +default policy maps the EU-27 plus the EEA states to `gdpr-eu` (a signal +required for every modeled use), GB to `gdpr-uk` (storage `granted`, +flagged in row 4), and US and AU to `us-opt-out` (a `granted` baseline +where all granted uses drop on any opt-out signal), leaves every unmodeled +Data Use `denied`, and sends unmatched requests to the rule named by the +required `[geo] default_country`. + +The migrated operator configuration is: + +- `[ec] provider = "hmac"` with its `[ec.providers.hmac]` block (32-plus + character passphrase), carried over verbatim for identity stability + (§3); +- `[device]` left at the `builtin` default, with `fastly` as the opt-in + documented alongside the open sign-off 22 question; +- `[geo] provider = "platform"` where the adapter supplies a host lookup + (Fastly and Cloudflare in production, the Axum dev server in + development, while Spin resolves nothing either way). A selected + provider's lookup failure resolves at the requires-signal floor and + never at `default_country`. A static deployment instead sets + `default_country` together with `assume_single_jurisdiction = true`; +- `[geo] default_country` naming the fallback rule (required). + +The draft's committed per-adapter fixture files +(`docs/guide/fixtures/migration-preserving-.toml`, CI-validated +and pinned against the §2 preservation rows) are **not yet written** and +remain the bar for the migration guide (§7). The guide must also state +that no recipe preserves row 8, because the global honoring of opt-out +signals is unconditional in the shipped model. + +## 6. Rollout sequence and observability + +1. Implementation PRs land in the epic's order. **Done as specified.** + The series landed providers first with the geo default held at today's + behavior (#1044), and the permission model PR flipped the default + together with its jurisdiction guard (#1045). Each PR's description + states what that PR changes. +2. **Adapter qualification as a release gate.** **Deferred** with §4 + requirement 2. The series' posture is the declared stateless-identity + path for adapters without the identity endpoints. The response hook + remains outside the series entirely (#1047). +3. **Staged activation for policy publication.** **Deferred** (sign-off + 19). `ts config push` today publishes the operator configuration as a + blob envelope and validates the configuration. The immutable + `push_sequence` envelope, prepare/commit activation, admission lease, + quiescence barrier, and scheduled-unavailability protocol are not + implemented. Policy in the shipped series changes by rebuilding + (`permissions.yaml`) or by pushing configuration, both taking effect + on restart/reload rather than through a fleet-wide activation CAS. +4. Before/after deploy, operators watch **EC issuance rate** and EID + attachment rate as the canary metrics, because the failure mode of a + bad migration is a silent drop to zero (or a silent grant to + everyone), not an error rate. **The named metric set with thresholds, + windows, and actions is not yet built.** The requirement stands for + the migration guide. The retirement-readiness bar for a legacy + provider (legacy-reader hits at zero for a quiet period no shorter + than the maximum cookie/row lifetime plus rollout skew) transfers to + the deferred `legacy_providers` design (§4 requirement 8). +5. Startup logs. Partially implemented. Startup logs the effective + default baseline and the exact list of permissions granted without a + signal (`Permission baseline: [geo] default_country = ...; granted +without a signal: [...]`), plus the passphrase deprecation warning. + The single greppable line naming the selected provider per concern and + whether geo is live remains to add. +6. **The batch-sync coverage dip.** **Deferred** with the provenance + gate (row 11c, sign-offs 7, 13, 25). Today batch sync authenticates + the caller and checks row state, and rows that are missing or + withdrawn collapse to ineligible. There is still no fail-open + shortcut, and grandfathering pre-epic identities past the permission + model remains rejected. +7. Rollback is config-only where possible. Implemented in the simple + sense, where selection is configuration, so reverting to the previous + configuration on the previous binary restores the previous behavior, + and the same compiled binary switches providers through the + `TRUSTED_SERVER__ec__provider` / `TRUSTED_SERVER__device__provider` / + `TRUSTED_SERVER__geo__provider` overrides applied when the operator + publishes configuration through `ts config push`. The durable + artifacts today are exactly the **identity-graph withdrawal + tombstones** written by explicit storage withdrawal (no recovery, that + is their purpose). The draft's longer durable list (model epoch tuple, + use-opt-out suppressions, negative outbox, safety breaker) describes + the deferred durable-suppression design (sign-offs 11, 16, 19, 20). + The two documented-not-automated procedures (cleanup after a policy + tightening, legacy-reader retirement) remain for the migration guide. + +### 6.1 Operator CLI delta for this epic + +The pre-existing CLI design remains unchanged and is what the series +uses. `ts config push` validates and publishes the operator configuration +(with typed environment overlays covered by a CLI regression test), and +`ts config validate` runs the same structural validation standalone. + +The draft's normative delta to `ts config push` (never-reused +`push_sequence`, immutable envelope identity, candidate CAS, §5.5 +promotion protocol) and the `ts config gc` command are **not +implemented**. They are deferred together with sign-off 19 and remain the +design bar for the durable-suppression and staged-activation follow-up. +Nothing in the series authorizes a generic raw metadata command or a +model-transition command. + +## 7. Documentation deliverables + +- Migration guide page (§5), linked from `CHANGELOG.md` and the release + notes. **Outstanding.** The series updated `configuration.md`, + `edge-cookies.md`, `ec-setup-guide.md`, and `error-reference.md` + (#1047), and the example configuration documents every selector, but + the dedicated migration page with per-adapter fixtures and the + CHANGELOG link do not exist yet. +- `configuration.md` documents **every** valid `provider` value for all + three concerns. **Done** (#1047), including the required + `default_country`, the acknowledgment flag, and the requires-signal + floor on a failed lookup. The environment-variable overrides are now + real in production, applied as typed EdgeZero app-config overlays when + the operator publishes through `ts config push`, with a CLI regression + test. (In PR #838 the documented override existed only under + `#[cfg(test)]`. The core test-only helper survives as legacy, and the + runtime path does not use that helper.) +- The permission model page states the precedence rules the code + implements. **Done** (#1045/#1047). The permission-model guide is in + the docs navigation, and the precedence is fixed in code (opt-out over + TCF, malformed-present fail-closed, then TCF), with pinning tests per + opt-out source against a consenting TCF record. Operator docs and + normative spec must not diverge on precedence. + +## 8. Product decisions requiring explicit sign-off + +These are product decisions this spec set needs that #838 had not already +made (or made differently). The table records the recommended resolution +approved for this spec revision, not a final product decision. +**Implementation is blocked while any row is `open`.** Each row is a +decision, not an assignment, since who decided is captured inside the +record itself (`docs/superpowers/specs/decisions/NN-title.md`, the +decision, the deciders, the date). The Decision-record column holds the +link (`(none)` while open). An unratified row reverts to open, not to +silently implemented. **The implemented series (PRs #1043-#1047) is +presented to the task force for ratification of the rows marked below as +implemented or bearing on the series**, so the task force can ratify +rather than re-litigate. Ratifying such a row creates its decision record +and closes the row. Rejecting one reverts the implementation, never keeps +the code with the row open. + +| # | Recommended resolution | Where | Decision record | Status | +| --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | Honor mapped use opt-outs globally. Destructive identity effects are limited to explicit storage withdrawal, authenticated deletion, or a qualifying live TCF Purpose 1 refusal. | permission §4, §4.2 | (none) | open. Implemented by #1045 (jurisdiction-free suppression, withdrawal only on a live TCF storage refusal under a non-granted baseline). Ratify | +| 2 | GPP/USP sale opt-outs suppress the personalised-ads uses only; they neither revoke storage nor delete the identity. | permission §4.5 | (none) | open. Partially implemented by #1045 (no opt-out deletes or destructively revokes). The shipped default maps opt-outs to `revokes: all` rather than personalised-ads only, a `permissions.yaml` choice to ratify | +| 3 | Sharing/targeted-advertising opt-outs suppress the personalised-ads uses; an explicit applicable not-opted-out value may grant them. Neither affects storage. | permission §4.5 | (none) | open. Not implemented (the sharing and targeted-advertising GPP fields are not decoded, and no US signal grants anything in the shipped model) | +| 4 | US auction dispatch may continue while personalised-ads is unset only through permission §7's positive `ContextualAuctionView` and its sole normative inline manifest. | permission §7.1 | (none) | open. The strip half is implemented by #1045 (EIDs and `user.id` withheld when the pair is unset). The positive contextual projection is not implemented | +| 5 | Country-only and regionless US traffic use a protective country-wide `us-opt-out` floor; state rules may be stricter. | permission §3.4 | (none) | open. Implemented by #1045 (`permissions.yaml` maps country `US` to `us-opt-out` with state overrides available). Ratify | +| 6 | Raw regulatory strings reach only the positively registered OpenRTB field that requires each source; all other destinations default deny. Identity rows retain normalized provenance/digests, not raw consent snapshots. | permission §7; providers §6.3 | (none) | open. Not addressed by the series | +| 7 | Reject legacy batch-sync traffic until live-browser provenance backfill makes the row re-evaluable. | rollout §6 item 6; permission §7 | (none) | open. Not implemented (no provenance exists to recompute) | +| 8 | Gate proxy, click, and Testlight identity forwarding on the sharing pair (storage plus personalised-ads). | §2 row 11b | (none) | open. Not implemented by the series | +| 9 | Defer integration-owned cookie operations from the v1 response hook; require a complete read/use/withdraw lifecycle before admission. | hook §3 | (none) | open. Overtaken (#1047 removed the whole hook from the series, so no cookie surface shipped). The deferral returns with the hook's first consumer | +| 10 | Do not create a blanket session-cookie exemption; every cookie must be covered by an approved permission or narrowly defined security-use authority. | hook §3 | (none) | open. Hook not shipped, unaffected | +| 11 | Require a durable per-family negative-intent outbox in a failure domain independent of its strong target and checked freshly by every identity consumer, with a globally visible breaker over positive identity operations when neither can commit. | permission §4.3 | (none) | open. Not implemented (part of the durable-suppression follow-up) | +| 12 | Adapters that cannot meet the revocation-storage contract migrate stateless rather than weakening the contract. | rollout §6 item 2; recipe §5 | (none) | open. Direction implemented (`provider = "none"` spells stateless in #1043; the identity endpoints were already Fastly-only before the series, and #1046 keeps `resolve` on the same footing). The formal capability gate is deferred. Ratify the direction | +| 13 | Keep batch sync fail-closed at cutover; stage partner communication and cleanup using explicit coverage thresholds, windows, and pause actions. | rollout §6 item 6 | (none) | open. Not implemented (no provenance cutover exists yet) | +| 14 | Policy tightening does not reinterpret historical refusal as a destructive event; destructive withdrawal requires fresh, live qualifying evidence. | permission §4.2 trigger 2 | (none) | open. Implemented by #1045 (withdrawal evaluates the live request's TCF record only, and historical records are never reinterpreted). Ratify | +| 15 | Descope the client cycle and `rewrite_legacy`; ship the v1 integration hook as headers-only. | client spec status; providers §6.1; hook §3 | (none) | open. Overtaken (the client cycle shipped hardened as #1046 instead of descoped, `rewrite_legacy` does not exist, and the hook shipped not at all per #1047). Re-decide against the shipped shape | +| 16 | Persist use-opt-out suppression until ordered explicit authorization for that use or identity deletion, with TCF `LastUpdated` or an authenticated monotonic revision proving order. | permission §4.3 | (none) | open. Not implemented (suppression in the series is request-scoped with no durable record) | +| 17 | N/A, absent, reserved, unknown, and unsupported values never grant processing. | permission §4.5 | (none) | open. Implemented by #1045 in a stricter form (signals only revoke, so no US-signal value grants anything). Ratify | +| 18 | A selected geo provider's lookup failure uses the compiled-in protective profile; `default_country` is only for acknowledged static-jurisdiction mode. | permission §5.2 | (none) | open. Implemented by #1045 (failed lookups resolve at the requires-signal floor, logged at error level, never `default_country`). Ratify | +| 19 | Use immutable version-addressed whole-config publication plus prepare/commit activation of the complete tuple, with authenticated fleet membership, bounded admission lease, quiescence, and an activation journal; a second unanimous model transition advances model epoch, minimum binary generation, and row schema floor atomically. | permission §5.5; rollout §6.1 | (none) | open. Not implemented (`ts config push` publishes and validates without the activation protocol) | +| 20 | N+1 keeps v1 creation and pre-epic live gating, reads/enforces N+2 negative state for rollback safety, and never originates durable use suppression; new-shape settings alone do not activate the new writer/model. | migration §4.4 | (none) | open. Overtaken in part (the shipped migration is a one-release dual-read of `[ec] passphrase` in #1043 with mixed forms rejected and nothing durable added, so the full interim waits for the durable design) | +| 21 | Expire and re-create rowless legacy cookies without continuity; a prefix match cannot authenticate the cookie suffix. | providers §5 | (none) | open. Not implemented (no rowless classification exists, and a cookie with no row is never shared) | +| 22 | Defer host JA4/H2 fingerprint processing to a separate approved design; reject `[device] provider = "fastly"` at startup and do not persist fingerprint-derived classifications. | providers §5 | (none) | open. **Contradicted by the series and flagged for review** (#1044 ships the `fastly` device provider and the host-signal EC provider opt-in, and device signals including fingerprint hash prefixes persist in rows) | +| 23 | Permit a narrow `SecurityUse` authority for DataDome only, with the exact bounded surface the hook spec defines. | hook §4a; permission §7 | (none) | open. Hook not shipped, unaffected | +| 24 | Malformed/absence suppression overrides a permissive baseline but clears on newer valid evidence; it is not sticky like an explicit use opt-out. | permission §4.3, §4.1 | (none) | open. Implemented by #1045 by construction (the fail-closed block is request-scoped, so newer valid evidence re-resolves). Ratify together with row 16's durable design | +| 25 | Enforce a stored-jurisdiction/provenance horizon for batch sync: moves into stricter regimes fail closed by the horizon, and moves out require a live visit. | permission §7 | (none) | open. Not implemented | +| 26 | Aggregate embedded GPP GPC with `Sec-GPC` by OR as a global, non-destructive use opt-out. | permission §4.5 | (none) | open. Partially implemented by #1045 (`Sec-GPC` is an honored non-destructive source). The embedded GPP GPC subfield is not read | +| 27 | In proxy mode, decode only mapped opt-out fields and derive no grants. | permission §4.4 | (none) | open. Partially (the shipped model derives no grants from any US signal anywhere). The proxy-mode decode restriction is not separately implemented | +| 28 | Require product and written vendor conformance approval for the reduced DataDome surface the hook spec pins. | hook §4a.2 | (none) | open. Hook not shipped, unaffected | +| 29 | Accept rowless roaming-cookie expiry as a bounded residual only with telemetry, an explicit maximum lifetime, operator documentation, and a removal/sunset criterion. | providers §5 | (none) | open. Not implemented | +| 30 | Saturation blocks rowless admission for that prefix but never revokes an authenticated real row without its exact suffix; monitor NAT-cohort pressure. | providers §5 | (none) | open. Not implemented | +| 31 | Keep replay history bounded by evicting expired/grant entries first and retaining restrictive state for its full horizon; saturation never shortens a later opt-out. | permission §4.3; providers wire schema | (none) | open. Not implemented | +| 32 | Accept official GPP sections 24-27 version 1, pin their layouts to the vendored IAB commit, and treat complete decoder/fixture support as a release prerequisite, with full vendoring evidence. | permission §4.5.1 | (none) | open. Not implemented (the shipped decoder consults the GPP sale field, with no vendored IAB corpus) | +| 33 | Treat any malformed or unsupported-version **mapped** GPP section as a global blocker for grants to the permissions its schema maps, while still honoring decodable opt-outs elsewhere and never deriving withdrawal from malformed bytes; unknown unmapped section IDs remain non-contributing. | permission §4.5 | (none) | open. Partially implemented by #1045 (a present undecodable record blocks baseline grants). The per-section mapped-blocker rule is not implemented | +| 34 | Permit providers whose canonical identifiers cannot fit an injective graph suffix to use the `sha256-detect` mode: domain-separated collision resistance plus stored canonical-identifier comparison, fail-closed collision handling, no overwrite/join. | providers §2, §6.3 | (none) | open. Premise revised by #1043 (identifiers are globally bounded at 256 bytes and the graph is keyed by `normalize_id_for_kv`). No `sha256-detect` mode exists | + +## Revision record vs the 2026-07-31 draft + +| Draft position | Revised position | Why | +| -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Epic #777-#781 pending sign-off, no implementation | Implemented as five stacked PRs #1043-#1047, with §8 staying the ledger and rows marked for ratification | The series shipped the seam, selection, permission model, resolve endpoint, and docs, so the task force ratifies rather than re-litigates | +| `[device] provider = "fastly"` startup-fails pending a separate fingerprinting design (§2 row 9, row 22) | The `fastly` device provider and host-signal EC provider ship opt-in, and device signals persist in graph rows. The identifier-collision defect the review found (host-signal identifiers shared the HMAC grammar and keyspace) is fixed by the mandatory provider-code envelope, so host-signal identifiers are `hs00~` namespaced; the policy question of row 22 is unchanged by the collision fix | Implementation choice, deliberately flagged as the open review question rather than presented as settled | +| US recipe: `requires_signal` with an extended grant-signal class (§2 rows 3/3a) | Shipped default: `granted` US baseline, `revokes: all` on any opt-out, and no US signal grants anything | The shipped signal model is revoke-only, which is simpler and stricter on grants, and the baseline choice is deployer-editable yaml, flagged in rows 3 and 5 | +| `[permissions]` TOML policy published at runtime | `permissions.yaml` compiled into the build, no `[permissions]` block | Policy stays reviewable in version control and the partial-policy trap cannot be written | +| `rules.default` worldwide default entry (§2 row 7) | Required `[geo] default_country` naming the fallback rule, validated at startup | Same role, one mechanism, loud when missing | +| Graph store mandatory at startup for creating providers (§2 row 12, §4.4b) | No startup requirement, and with no graph no identifier is created, per request | The phantom-cookie rule holds without a breaking startup change, and graphless deployments run identity-less | +| `legacy_providers` reader chain for provider switches (§4.8) | Not implemented, so a switch restarts identity | Deferred until a second server-side provider makes switching real | +| N+1/N+2 negative-record machinery, model epochs, `m00` mirror (§4.1, row 20) | Deferred with rows 11, 16, 19, 20. The shipped dual-read is one release of `[ec] passphrase` mapping with mixed forms rejected | Nothing durable beyond pre-existing withdrawal tombstones ships, so binary rollback strands no new state | +| Staged activation, `push_sequence`, quiescence, `ts config gc` (§6.3, §6.1) | Basic `ts config push` / `ts config validate` only | The activation protocol belongs to the deferred durable-suppression rollout (row 19) | +| Migration guide with committed per-adapter fixtures and a full gated metric set (§5, §6.4) | `configuration.md` / `edge-cookies.md` document the migrated shape, while fixtures, the guide page, and metrics remain outstanding | Documentation shipped for configuration, and the operational guide is the remaining deliverable before a release | +| Client cycle descoped and hook shipped headers-only (row 15) | The client cycle shipped hardened (#1046) and the hook shipped not at all (#1047) | Hardening replaced descoping, and the hook had no consumer, which is the hook spec's own admission rule | +| Example config ships the migrated happy path uncommented (§4.9) | Identity off by default with selector and block commented together, and validation closes the silent-stateless trap | Loud startup validation, not an uncommented default, is what prevents PR #838's silent-stateless state | +| Environment override documented but `#[cfg(test)]`-only in PR #838 (§7) | Override applied as typed EdgeZero app-config overlays at `ts config push`; the existing CLI overlay test covers the mechanism, and a provider-specific override test is still to write | The same compiled binary switches providers at deployment through the published configuration | +| Pinned known-answer HMAC vectors committed (§3) | Stability tested per inputs, and the pinned vector is still to commit | The cross-version CI pin remains open work under §3 | +| GB storage baseline change only with citation and sign-off (§2 row 4) | The shipped `permissions.yaml` adopts `granted` storage for GB without a recorded citation | Flagged in row 4, since the task force owns the decision and its record | diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 3736273928..cc53032a01 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -1,12 +1,13 @@ # Design Spec: The Integration Provider Seam -**Status:** Proposed, 2026-08-27, revised 2026-08-28. This PR adds this -document only and targets `main` directly. Following the review of #1043 +**Status:** Proposed, 2026-08-27, revised 2026-08-28. This PR adds design +documents only and targets `main` directly. Following the review of #1043 (27 August) the seam it defines is a precondition for the provider series rather than a follow-up to it, so the order is now this spec, then its implementation in a seventh PR against `main` (51Degrees), then PRs #1043 -to #1047 reworked onto it. It reads alongside the series' specs, which land -with PR #1047. +to #1047 reworked onto it. It reads alongside the series' specs, which this +PR now carries too, so the whole normative set is reviewable before any of +the code lands. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-07-30-pluggable-providers-design.md`, `2026-07-30-provider-migration-rollout-design.md`, @@ -418,3 +419,4 @@ defines, and both should land before the first vendor is asked to use it. | 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | | 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | +| 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged; only this status line and this row are new. | diff --git a/docs/superpowers/specs/provider-code-registry.md b/docs/superpowers/specs/provider-code-registry.md new file mode 100644 index 0000000000..0a2014c54b --- /dev/null +++ b/docs/superpowers/specs/provider-code-registry.md @@ -0,0 +1,30 @@ +# Provider-code registry (normative, append-only, never reused) + +Four-character codes (`[a-z0-9]`, zero-padded) that namespace Edge Cookie +identifiers. Every EC provider MUST allocate a code here before it can +exist: the `EdgeCookieProvider::code()` trait method is mandatory, and core +applies the code as the `{code}~` prefix of every identifier the provider +mints, checks it at read-back, and keys the identity graph with it. A +provider only ever sees its own value part, so identifiers from different +providers can never collide in the cookie, the graph, or a withdrawal, and +every identifier records which provider created it. + +Allocation is a reviewed commit to this file; codes are immutable and never +recycled, including for retired providers. A leading digit is valid. The +tilde separator keeps parsing exact while pre-envelope identifiers remain +deployed: a legacy bare identifier contains no tilde and dual-reads under +the built-in HMAC provider only. + +The class of provider expected to grow this table is one that consumes a +web-browser-supplied unique identifier, arriving either as a new web +platform feature or from a user-installed extension, delivered to the edge +through the client-cycle resolve path and verified by the provider before +minting. + +| Code | Provider | Allocated | Status | +| ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------------- | +| `hmac` | Built-in HMAC EC provider. Mints `hmac~<64 hex>.<6 alnum>`; dual-reads its pre-envelope bare form so deployed cookies keep working (retirement condition in `provider_owns_id`) | 2026-08-02 | active | +| `hs00` | Built-in host-signal EC provider (opt-in; TLS JA4 plus HTTP/2 signals plus client IP) | 2026-08-25 | active | +| `cfix` | Client-fixed demonstration provider (compiled only behind the `client-fixed-demo` cargo feature) | 2026-08-25 | active, test and demo only | +| `51dd` | 51Degrees Identifier (51Did) vendor provider | 2026-08-25 | reserved | +| `t0..` | Prefix family reserved for in-tree test providers (`t0cc`, `t0op`, and similar); never valid in configuration | 2026-08-25 | reserved | From 4ee2ab1bfd8222c174e467462dd9619ebf57452f Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 30 Aug 2026 22:27:03 +0100 Subject: [PATCH 005/192] Carve evidence interfaces out of the trait minimalism rule The rule that every trait method needs a production caller in the change that introduces it is right for a behavioural trait, where a method nothing calls is dead weight. It is wrong for an evidence interface, and we applied it there and should not have. An evidence interface describes what a request carries, not what today's code happens to read. Held to the caller rule it grows a method each time a vendor arrives, so a vendor cannot write against it and it cannot stay stable across a release. It also puts the boundary in the wrong place. What a provider may see is not the control, because what a provider may do with what it sees is the control, and that is the permission model's job. So RequestInfo carries what the request carries. The rule stands everywhere else. Addresses: docs/superpowers/specs/2026-07-30-pluggable-providers-design.md section 4, a rule applied to an interface it does not suit --- .../2026-07-30-pluggable-providers-design.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 78aca06846..4640d1fc06 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -178,10 +178,22 @@ One global rule sits above every provider, and it is implemented: not at all, and the cookie value and the identity-graph key can never silently diverge. -## 4. Trait surface: minimalism rule +## 4. Trait surface: minimalism rule, and where it does not apply Every trait method must have at least one production (non-test) caller in -the same change that introduces it. How the surface observed in PR #838 +the same change that introduces it. **This rule does not apply to an evidence +interface, and applying it there was a mistake we made and are correcting.** + +An evidence interface describes what a request carries, not what today's code +happens to read. Holding it to the caller rule produces an interface that grows +a method each time a vendor arrives, which is not something a vendor can write +against and cannot be stable across a release. It also puts the boundary in the +wrong place, because what a provider may see is not the control. What a provider +may do with what it sees is the control, and that is the permission model. + +So `RequestInfo` carries everything the request carries, whether or not code in +this repository reads it yet. The rule stands for behavioural traits, where a +method with no caller really is dead weight. How the surface observed in PR #838 resolved in the implementation: - `keys_equal`: **not shipped.** Its legitimate purpose (equivalent-envelope From a8bf6751d35c5ff206463319e472dbcb6eea3f8b Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 30 Aug 2026 22:30:08 +0100 Subject: [PATCH 006/192] Say create rather than mint throughout the provider specs "Mint" is jargon for something ordinary, which is making an identifier. It was defined in the glossary, so it was at least explained, but a reader should not need a glossary for the word that describes the main operation the specs are about. Fifty-one uses across five files, including the glossary row itself, now read create, creates, created or creating. Nothing else changed, so a reader comparing against the previous revision sees one word substituted and no argument altered. Addresses: plain language in documents others read --- ...26-07-30-client-cycle-ec-resolve-design.md | 24 ++++---- .../2026-07-30-permission-model-design.md | 2 +- .../2026-07-30-pluggable-providers-design.md | 60 +++++++++---------- ...-08-27-integration-provider-seam-design.md | 2 +- .../specs/provider-code-registry.md | 18 +++--- 5 files changed, 53 insertions(+), 53 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 6949889a38..4abab609e9 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -16,7 +16,7 @@ defers, and why the feature is normative rather than deferred. > browser POST to a new public endpoint (`POST /_ts/api/v1/ec/resolve`), > plus a demo provider (`client-fixed`) and a JS bundle. Review found the > endpoint accepted cross-origin identity-setting posts with no origin -> check, minted cookies with no identity-graph row (violating an invariant +> check, created cookies with no identity-graph row (violating an invariant > the organic path enforces explicitly), was registered on only one of four > adapters, and could never round-trip because the core did not recognize > non-HMAC identifiers. None of that is an argument the feature is a bad @@ -54,7 +54,7 @@ verify against. | Threat | Vector | Consequence if unmitigated | | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Cross-site identity fixation** | `text/plain` POST is a CORS-simple request: any page on the web can `fetch(resolveUrl, {method: "POST", credentials: "include", body: payload})` with no preflight | An attacker pins a chosen identity onto a victim's first-party cookie jar, a login-CSRF for the ad-identity layer, and the victim's activity accretes to an attacker-controlled ID | -| **Replay** | A captured valid payload (from the attacker's own session or a leak) replayed against another browser | Same as fixation, without needing to mint payloads | +| **Replay** | A captured valid payload (from the attacker's own session or a leak) replayed against another browser | Same as fixation, without needing to create payloads | | **Phantom identity** | Endpoint sets the cookie without an identity-graph row | Later requests carry an EC that the KV graph has never seen; downstream sync and withdrawal logic operate on an identity that half-exists (the organic generation path explicitly refuses to write a cookie when the graph write fails, for exactly this reason) | | **Un-tombstoneable identity** | Core does not recognize the provider's identifier shape | Withdrawal cannot expire or tombstone the identity, which is a compliance failure and not only a bug | | **Amplification** | The page script cannot observe an HttpOnly cookie, so it cannot know the cookie is already set | A POST on every page view of every session (PR #838's JS gated on reading a cookie its own server marked HttpOnly, making the guard permanently false) | @@ -79,7 +79,7 @@ same-site` (which v1 does not consult at all), not to a suffix match on open (§7.5) for publishers whose pages run on domains other than the configured apex. 2. **Verify the payload per provider.** The endpoint hands the posted - payload to the selected provider's `resolve_from_client` and mints only + payload to the selected provider's `resolve_from_client` and creates only what the provider returns; whether the payload is trustworthy is the provider's responsibility, stated on the trait. A real vendor provider verifies a signed, audience-bound, expiring envelope; the draft's @@ -90,8 +90,8 @@ same-site` (which v1 does not consult at all), not to a suffix match on server-side providers are untouched. 3. **Preserve the identity-graph invariant.** Implemented: the graph row is written before the cookie is set, keyed by the provider's - `normalize_id_for_kv` canonical form, exactly like the organic mint - path. No graph available → no mint (`204`), same as organic generation; + `normalize_id_for_kv` canonical form, exactly like the organic create + path. No graph available → no create (`204`), same as organic generation; a graph write failure → `503`, no cookie. 4. **Round-trip through the lifecycle contract.** Implemented: read-back goes through the selected provider's `accepts_id`, the KV key through @@ -101,7 +101,7 @@ same-site` (which v1 does not consult at all), not to a suffix match on 5. **Exist on every adapter, where parity means identical behavior, including identical refusal.** Partially implemented, documented: the Fastly adapter routes the endpoint (passing the same bot-gated identity - graph as organic generation, so unrecognized clients cannot mint + graph as organic generation, so unrecognized clients cannot create through resolve either). The Axum, Cloudflare, and Spin adapters deliberately do not route it, matching `identify` and `batch-sync`, which need the same platform KV wiring those adapters do not have; the @@ -113,20 +113,20 @@ same-site` (which v1 does not consult at all), not to a suffix match on declaration.** Implemented: every response carries `Cache-Control: no-store`, and the gate is the selected provider's complete `required_permissions()` through the same resolved permission state as - organic minting, not a hard-coded storage check. + organic creating, not a hard-coded storage check. 7. **Bound every input.** Implemented: request body at most 65,536 bytes, where an advertised `Content-Length` over the limit answers `413` before the read, and the read body is re-checked so a missing or false length does not bypass the bound. `Content-Type` allowlist: `text/plain` and `application/json`, matched on the media type alone, case-insensitively, ignoring parameters (the browser's default `text/plain;charset=UTF-8` - passes); anything else → `415`. The minted identifier must fit the + passes); anything else → `415`. The created identifier must fit the global identifier bounds (at most 256 bytes, cookie-safe alphabet, - shared with every other mint path); violation → `400`, never a rewrite. + shared with every other create path); violation → `400`, never a rewrite. Core then applies the provider's registered code envelope (`provider-code-registry.md`): the cookie and the identity-graph key carry `{code}~value`, so a client-set identity is namespaced to its - provider exactly like an edge-minted one (the demo's cookie value is + provider exactly like an edge-created one (the demo's cookie value is `cfix~an-ec`). Status codes are part of the contract: `400` out-of-bounds identifier, `403` origin rejection, `409` different-identity conflict, `413` body, `415` content type, `503` graph-write failure, `204` closed gate / no @@ -147,7 +147,7 @@ no-store`, and the gate is the selected provider's complete primitive no production adapter exposes today. Designing the reservation against a hypothetical envelope would repeat the mistake this series exists to fix. v1's stance: the endpoint is safe without it - for the providers v1 ships (the demo mints a constant, feature-gated + for the providers v1 ships (the demo creates a constant, feature-gated out of production), and the reservation lands with the first vendor scheme, designed against its real envelope, with the draft's §3.9 as the starting bar. Until then the draft's text is preserved below as the @@ -208,7 +208,7 @@ identifier, constant-equality verification) is compiled only behind the `client-fixed-demo` cargo feature. In a production build the settings validator rejects the selection at startup with a direct message, and the provider builder rejects it again as defense in depth. A fixed shared word -is not an identity; the demo exists to exercise verify-before-mint end to +is not an identity; the demo exists to exercise verify-before-create end to end in tests and demonstrations. ## 6. Testing diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index ec98c28442..0cedf6f630 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -638,7 +638,7 @@ requires a live user signal (§4.2), never a policy change. | Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition); never withdraws; opt-outs still honored | | Expired TCF record | Distinct state, not malformed; treated as absent, so the baseline applies | | Signals contradict (opt-out plus consent) | Opt-out wins (§4) | -| No EC provider selected | Identity fails closed: nothing minted, an incoming cookie value never used or egressed (§7) | +| No EC provider selected | Identity fails closed: nothing created, an incoming cookie value never used or egressed (§7) | The posture is fail-closed. Every ambiguous state resolves to the configured baseline or more restrictive. diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 4640d1fc06..b8000564e2 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -12,7 +12,7 @@ > **Context.** PR #838 proposed a first implementation of this epic in a single > change. Review of that PR surfaced design gaps this spec exists to close > before a second implementation pass: an identity abstraction that owned -> minting but not recognition, per-adapter divergence in provider selection, +> creating but not recognition, per-adapter divergence in provider selection, > silent misconfiguration modes, and speculative trait surface with no > production caller. This spec is the authoritative statement of what the > provider architecture must do; where it contradicts PR #838, this spec wins. @@ -40,7 +40,7 @@ Goals, as implemented: default deployment makes no third-party or host-specific call. - An **EC provider declares** the permissions its data use requires (`required_permissions` on the trait), and **core enforces** that - declaration before minting or using an identity. A provider cannot + declaration before creating or using an identity. A provider cannot authorize itself. The enforcement machinery is the permission model's subject and lands with it in PR #1045 (see the permission model spec). Geo and device carry the same declaration method with an empty default, @@ -111,22 +111,22 @@ the provider as settled either way. ## 3. The identity lifecycle contract -This is the section PR #838 lacked. Its trait abstracted **minting** an +This is the section PR #838 lacked. Its trait abstracted **creating** an identifier but left **recognition** and **KV key normalization** hard-coded to the built-in HMAC shape, so a provider whose identifiers did not match -`{64hex}.{6alnum}` minted cookies that the very next request discarded. +`{64hex}.{6alnum}` created cookies that the very next request discarded. The implemented contract routes every lifecycle operation core performs on an EC value through the selected provider: | Lifecycle operation | Where core uses it | Contract | | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Mint** | EC generation on first eligible request, and the client-cycle resolve endpoint | The provider returns the identifier (`generate` server-side, `resolve_from_client` for the client cycle) and only core writes the cookie, after enforcing the global bounds below. | +| **Create** | EC generation on first eligible request, and the client-cycle resolve endpoint | The provider returns the identifier (`generate` server-side, `resolve_from_client` for the client cycle) and only core writes the cookie, after enforcing the global bounds below. | | **Recognize** | Reading `ts-ec` back from the request, deciding `ec_was_present`, withdrawal checks, and every path that hands the value onward: the origin URL in `append_ec_id`, the click-target URL in `handle_first_party_click`, and the proxied body an integration builds | `accepts_id` answers whether a value is a well-formed identifier the provider issues. A value the selected provider does not recognize is treated as absent, so it is never used or egressed, while the raw cookie value stays visible to withdrawal handling. The egress paths reach the same answer through `edge_cookie::recognized_ec_id`, and a deployment with no provider selected recognizes nothing and so egresses nothing. | | **KV key** | Identity-graph row reads and writes | `normalize_id_for_kv` returns the key form. The default lowercases the built-in HMAC hash segment and preserves the suffix, keeping today's keys. An opaque or case-sensitive provider overrides to the identity function so distinct identifiers never collapse into one row. | | **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader: it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | -**Invariant:** for every provider `P` and every identifier `id` minted by +**Invariant:** for every provider `P` and every identifier `id` created by `P`, `id` round-trips read-back byte for byte. A test in `ec/mod.rs` proves the round-trip with a non-default provider whose identifiers are opaque, and a second test in `ec/resolve.rs` proves the client-cycle value survives the @@ -144,7 +144,7 @@ grammar lands, the KV key is the provider's normalized identifier verbatim, which keeps every pre-epic HMAC row reachable. The pre-epic IP-cluster prefix listing runs unchanged, but the key space it -lists over does not. A fresh mint is keyed `hmac~.`, so the +lists over does not. A fresh create is keyed `hmac~.`, so the prefix `evaluate_cluster` derives is `hmac~` for a coded row while a legacy bare row still lists under `` on its own. Prefix matching is anchored at the start of the key, so two rows for the same client IP that @@ -165,12 +165,12 @@ is the change that has to build the bridge. One global rule sits above every provider, and it is implemented: -- **Identifier bounds.** A minted identifier obeys a global cookie-safe +- **Identifier bounds.** A created identifier obeys a global cookie-safe alphabet (normatively `[A-Za-z0-9._~-]`, valid cookie octets with no separators, whitespace, or control characters) and a global maximum of **256 bytes**, stated here so dependent documents reference one number. The bound applies to the identifier itself, not only its key form. Core - enforces the bound wherever an identifier enters the system, at mint + enforces the bound wherever an identifier enters the system, at create (both `generate` and the resolve endpoint), at cookie read-back, and at cookie write. The constant is `MAX_EC_ID_LEN` in `ec/cookies.rs`. A violating value is rejected outright and logged. No sanitizing rewrite @@ -207,7 +207,7 @@ resolved in the implementation: The draft banned the field when nothing consumed it. The consumer landed in the same series, satisfying the rule the ban enforced. - `IdentityInput.permissions` / `IdentityInput.consent`: **shipped and - populated.** The organic mint path passes the request's resolved + populated.** The organic create path passes the request's resolved permission state and consent context so a provider can read them for behavior beyond gating. The gate itself has already run before `generate` is called, so a provider cannot use the fields to authorize itself. @@ -227,13 +227,13 @@ pub trait EdgeCookieProvider: Send + Sync + core::fmt::Debug { /// Stable configuration key ("hmac"). fn id(&self) -> &'static str; /// Registered four-character code (provider-code-registry.md), the - /// `{code}~` namespace of every identifier the provider mints. + /// `{code}~` namespace of every identifier the provider creates. /// Mandatory, no default: a provider cannot exist without a unique /// code, so identifiers from different providers can never collide. fn code(&self) -> ProviderCode; /// Derives an identifier from the provider's injected services and the /// request evidence passed at call time. A client-side provider defers - /// here (returns no id) and mints later in resolve_from_client. + /// here (returns no id) and creates later in resolve_from_client. fn generate( &self, request_info: &dyn RequestInfo, @@ -249,7 +249,7 @@ pub trait EdgeCookieProvider: Send + Sync + core::fmt::Debug { /// Permissions this provider's data use requires. Default: none, so a /// vendor-neutral provider requires no permission. fn required_permissions(&self) -> PermissionSet { /* none */ } - /// Client-cycle counterpart to generate: mints from a value the page + /// Client-cycle counterpart to generate: creates from a value the page /// posted to the resolve endpoint, after verifying it. Default: no-op, /// so a server-side provider does not participate. See the /// client-cycle spec. @@ -260,14 +260,14 @@ pub trait EdgeCookieProvider: Send + Sync + core::fmt::Debug { } ``` -Core owns the code envelope. At mint it prefixes the provider's value with +Core owns the code envelope. At create it prefixes the provider's value with `{code}~`, at read-back it strips and checks the code before the provider's `accepts_id` sees the value part, and the identity graph key preserves the code verbatim around the provider's canonical form. A cookie carrying another provider's code is treated as absent, never adopted, so switching providers cannot silently mix identity populations, and a withdrawal always acts on a key that can only belong to one provider. The built-in HMAC -provider mints `hmac~<64 hex>.<6 alphanumeric>` and dual-reads its +provider creates `hmac~<64 hex>.<6 alphanumeric>` and dual-reads its pre-envelope bare form for one release cycle so deployed cookies keep working; the bare form belongs to hmac alone. Codes are allocated append-only in `provider-code-registry.md`, and a leading digit is valid @@ -287,13 +287,13 @@ ahead of the caller that consumes it. ## 5. Permission enforcement is core's job, for EC providers -Before minting through an EC provider, core resolves the request's +Before creating through an EC provider, core resolves the request's permission state and refuses when the provider's `required_permissions()` are not all set. The gate is implemented in `EcContext`. The selected provider is built once at request read time, its declaration is checked against the resolved state, and generation is skipped (with a log line naming the jurisdiction) when the requirement is not met. With no provider -selected, nothing may mint or use an identifier, so the gate is closed +selected, nothing may create or use an identifier, so the gate is closed rather than open by default. The enforcement point lands with the permission model in PR #1045, and the permission model spec governs the resolution machinery (country and region baselines, signals, and the @@ -364,10 +364,10 @@ when the provider is built, stopping the request rather than degrading. | `[geo] default_country` unset, or matching no `permissions.yaml` rule | **Startup error.** The value is the permission baseline for a request the geo provider leaves unmatched, so there must always be one and the value must resolve to a real rule. | | An EC provider configured, no geo provider, `assume_single_jurisdiction` unset | **Startup error.** With geolocation off, every request resolves as `default_country`, so a visitor from any other jurisdiction silently receives the default jurisdiction's rules. That is acceptable only as an explicit operator decision. | -One draft row was not adopted, the startup error for a minting provider +One draft row was not adopted, the startup error for a creating provider with no identity-graph store. `[ec] ec_store` remains optional, because the portability adapters run without platform KV. The client-cycle resolve -endpoint refuses to mint when no graph is available (a cookie without a row +endpoint refuses to create when no graph is available (a cookie without a row could never be withdrawn through the graph), and the organic path persists the row whenever the graph is configured. Whether configuration should force the pairing is follow-up work with the migration spec. @@ -382,7 +382,7 @@ applies its own `deny_unknown_fields` when it deserializes. ### 6.1 Provider switching: what a switch actually does Switching `[ec] provider` **retires every identity the previous provider -minted**. This section says exactly what that means, because a deployer has +created**. This section says exactly what that means, because a deployer has to plan around it rather than discover it. The draft specified an ordered `legacy_providers` reader list, provider @@ -397,7 +397,7 @@ cookies stay recognized when the newly selected provider accepts their shape. That is not what the code does and never was once core took ownership of the `{code}~` envelope (§5). Ownership is decided on the code prefix **before** any provider is asked about the shape, so a newly selected -provider rejects every identifier the previous one minted, whatever its +provider rejects every identifier the previous one created, whatever its shape, because the code differs. What a switch does, precisely: @@ -441,9 +441,9 @@ logged, none silent: | Failure | Behavior | | -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `generate` returns an error | No identity this request. The organic caller logs at error level and the request proceeds stateless. No cookie is written. | -| A provider mints an identifier outside the global bounds | Rejected at mint, never rewritten. The organic path yields no identity. The resolve endpoint returns 400. | -| Identity-graph write fails at mint | The mint is undone (no identifier, no cookie), with the error logged. The resolve endpoint returns 503. The next eligible request retries. | -| The host-signals provider finds no TLS/HTTP-2 fingerprints | Defers with a warning. No identity this request, and no degraded IP-only identifier is minted under the host-signals name. | +| A provider creates an identifier outside the global bounds | Rejected at create, never rewritten. The organic path yields no identity. The resolve endpoint returns 400. | +| Identity-graph write fails at create | The create is undone (no identifier, no cookie), with the error logged. The resolve endpoint returns 503. The next eligible request retries. | +| The host-signals provider finds no TLS/HTTP-2 fingerprints | Defers with a warning. No identity this request, and no degraded IP-only identifier is created under the host-signals name. | | Geo lookup **fails** (the provider errors) | Every permission resolves to the requires-signal floor, and the failure is logged at error level. The failure is **not** papered over with the `default_country` baseline. | | Geo resolves **no location**, or a country/region with no rule | The `[geo] default_country` baseline applies. This is the configured-default case, deliberately distinct from the failure row above (`GeoStatus` in `ec/consent.rs`). | | An incoming cookie value fails the bounds at read-back | Treated as absent, with a warning naming the source. | @@ -464,7 +464,7 @@ authority state, negative-intent outbox, rowless withdrawal, and deployment metadata, wire schemas with known-answer vectors, and a per-field graph-row contract. The provider-code registry is now implemented: codes are allocated in `provider-code-registry.md`, carried as the `{code}~` prefix -of every minted identifier, and therefore present in every graph key. The +of every created identifier, and therefore present in every graph key. The key grammar differs from the draft in one deliberate way, a tilde separator instead of delimiter-free fixed width, because pre-envelope bare identifiers remain deployed and a code such as `51dd` is valid hex, so @@ -492,7 +492,7 @@ request path: injecting the host's `HostSignals` when supplied and matching an adapter-injected vendor provider by its `id()`. The provider is built once per request during `EcContext` construction and reused for - read-back, the permission gate, and minting, so the per-request + read-back, the permission gate, and creating, so the per-request triple-build observed in PR #838 (cloning the secret into a fresh box up to three times per request) is gone. - `build_device_provider` (`ec/device.rs`) returns the builtin classifier @@ -588,7 +588,7 @@ Implemented, in the crates named: acknowledgment. - Geo builder tests showing the default selects no geo, `none` selects no geo explicitly, and `platform` selects the host implementation. -- Host-signals provider tests covering minting from fingerprints, +- Host-signals provider tests covering creating from fingerprints, deferring without them, and the loud failure of a selected but uninjected vendor provider. @@ -638,12 +638,12 @@ this revision describes. | Draft position | Implemented position | Why | | ----------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Trait surface is a canonicalizing `parse` returning a typed id, plus `graph_key_suffix`, `cluster_prefix`, and `verify` | `accepts_id` (recognition) plus `normalize_id_for_kv` (KV key form), defaults matching the built-in shape. No typed id, key suffix, cluster capability, or `verify`. `keys_equal` stays out, as the draft required. | Recognition and KV keying are the two operations core performs today. A byte-for-byte round-trip test with a non-default provider pins the contract. | -| `GeneratedEdgeCookie::response_headers` and `IdentityInput.permissions` / `.consent` banned as speculative surface | Shipped with production consumers. Finalization applies provider headers, the resolve path returns them, and the organic mint path populates the input fields. | The client-cycle resolve path landed in the same series and is their caller, satisfying the minimalism rule the ban enforced. | -| Identifier bounds enforced at mint and parse | Enforced at mint (`generate` and the resolve endpoint), cookie read-back, and cookie write. Violations rejected outright, never rewritten. `MAX_EC_ID_LEN` in `ec/cookies.rs`. | Every identifier entry point is covered, and the pre-epic sanitizing rewrite was removed as a silent-divergence hazard. | +| `GeneratedEdgeCookie::response_headers` and `IdentityInput.permissions` / `.consent` banned as speculative surface | Shipped with production consumers. Finalization applies provider headers, the resolve path returns them, and the organic create path populates the input fields. | The client-cycle resolve path landed in the same series and is their caller, satisfying the minimalism rule the ban enforced. | +| Identifier bounds enforced at create and parse | Enforced at create (`generate` and the resolve endpoint), cookie read-back, and cookie write. Violations rejected outright, never rewritten. `MAX_EC_ID_LEN` in `ec/cookies.rs`. | Every identifier entry point is covered, and the pre-epic sanitizing rewrite was removed as a silent-divergence hazard. | | `provider = "none"` is valid alongside `legacy_providers` blocks | `none` (or an omitted selector) with any configured provider block is a startup error. | No `legacy_providers` exists in these PRs, so a block alongside statelessness can only be a mistake. | | Every selection key is closed and unknown keys are startup errors | Device and geo keys are closed. EC vendor keys are open. Unknown blocks are captured as raw values in core, the adapter deserializes its own block, and a selected key with no injected provider fails loudly. | Core never names a vendor, so a vendor provider adds no core change. | | Capability mismatch is a startup error at adapter wiring time | Configuration coherence fails at startup. A host-capability mismatch (missing `HostSignals`, uninjected vendor) fails loudly when the provider is built, stopping the request. | The adapter capability declaration that would move the check to startup is deferred with the capability matrix. | -| A minting provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to mint without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | +| A creating provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to create without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | | `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw fingerprints. | Selection is an explicit operator opt-in and the neutral default makes no host fingerprint call. | | The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host fingerprint processing, so the review decides whether the provider ships in the series. | | Geo default flip sequenced into the later permission-model step, with an acknowledgment guard | Landed as specified in the same series, with the default of none, `default_country` required and validated against `permissions.yaml`, the `assume_single_jurisdiction` acknowledgment, and a failed lookup resolving to the requires-signal floor with error logging (`GeoStatus`, resolved in core so all adapters agree). | The permission model shipped in PR #1045, so the constraints exist where the draft required them. | diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index cc53032a01..7e35617bce 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -300,7 +300,7 @@ deployment that lists the same integrations gets the same responses. built-in one. 2. **Capabilities round trip.** The same test integration declares an identity, a geo and a device provider. With the three selectors naming - it, a request is served by all three (the minted identifier carries its + it, a request is served by all three (the created identifier carries its code, the resolved country and the device signals are its). With a selector naming a module that lacks the capability, startup fails with an error that names the module and the capability. diff --git a/docs/superpowers/specs/provider-code-registry.md b/docs/superpowers/specs/provider-code-registry.md index 0a2014c54b..e6771337ee 100644 --- a/docs/superpowers/specs/provider-code-registry.md +++ b/docs/superpowers/specs/provider-code-registry.md @@ -4,7 +4,7 @@ Four-character codes (`[a-z0-9]`, zero-padded) that namespace Edge Cookie identifiers. Every EC provider MUST allocate a code here before it can exist: the `EdgeCookieProvider::code()` trait method is mandatory, and core applies the code as the `{code}~` prefix of every identifier the provider -mints, checks it at read-back, and keys the identity graph with it. A +creates, checks it at read-back, and keys the identity graph with it. A provider only ever sees its own value part, so identifiers from different providers can never collide in the cookie, the graph, or a withdrawal, and every identifier records which provider created it. @@ -19,12 +19,12 @@ The class of provider expected to grow this table is one that consumes a web-browser-supplied unique identifier, arriving either as a new web platform feature or from a user-installed extension, delivered to the edge through the client-cycle resolve path and verified by the provider before -minting. +creating. -| Code | Provider | Allocated | Status | -| ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------------- | -| `hmac` | Built-in HMAC EC provider. Mints `hmac~<64 hex>.<6 alnum>`; dual-reads its pre-envelope bare form so deployed cookies keep working (retirement condition in `provider_owns_id`) | 2026-08-02 | active | -| `hs00` | Built-in host-signal EC provider (opt-in; TLS JA4 plus HTTP/2 signals plus client IP) | 2026-08-25 | active | -| `cfix` | Client-fixed demonstration provider (compiled only behind the `client-fixed-demo` cargo feature) | 2026-08-25 | active, test and demo only | -| `51dd` | 51Degrees Identifier (51Did) vendor provider | 2026-08-25 | reserved | -| `t0..` | Prefix family reserved for in-tree test providers (`t0cc`, `t0op`, and similar); never valid in configuration | 2026-08-25 | reserved | +| Code | Provider | Allocated | Status | +| ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------------- | +| `hmac` | Built-in HMAC EC provider. Creates `hmac~<64 hex>.<6 alnum>`; dual-reads its pre-envelope bare form so deployed cookies keep working (retirement condition in `provider_owns_id`) | 2026-08-02 | active | +| `hs00` | Built-in host-signal EC provider (opt-in; TLS JA4 plus HTTP/2 signals plus client IP) | 2026-08-25 | active | +| `cfix` | Client-fixed demonstration provider (compiled only behind the `client-fixed-demo` cargo feature) | 2026-08-25 | active, test and demo only | +| `51dd` | 51Degrees Identifier (51Did) vendor provider | 2026-08-25 | reserved | +| `t0..` | Prefix family reserved for in-tree test providers (`t0cc`, `t0op`, and similar); never valid in configuration | 2026-08-25 | reserved | From 26f695474f63a31b265e88b1781db26bc04dfa30 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 10:22:10 +0100 Subject: [PATCH 007/192] Remove the reserved 51Degrees provider code row and apply spec house-style fixes Delete the reserved 51dd provider-code registry row so the unpublished vendor provider is not named in the public spec. Correct section 8's count from four to seven, replace several with many, and restructure clause-joining colons into separate sentences across the provider specs. The 51Degrees contribution attribution lines are kept deliberately. --- ...26-07-30-client-cycle-ec-resolve-design.md | 52 +++++----- ...integration-response-header-hook-design.md | 2 +- .../2026-07-30-permission-model-design.md | 94 +++++++++---------- .../2026-07-30-pluggable-providers-design.md | 42 ++++----- ...07-30-provider-migration-rollout-design.md | 48 +++++----- ...-08-27-integration-provider-seam-design.md | 14 +-- .../specs/provider-code-registry.md | 7 +- 7 files changed, 129 insertions(+), 130 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 4abab609e9..63e9226d78 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -32,7 +32,7 @@ defers, and why the feature is normative rather than deferred. ## 1. Overview A **client-cycle** EC provider establishes the identifier via a browser -round trip: server-injected first-party JS obtains or derives a value in the +round trip, where server-injected first-party JS obtains or derives a value in the page (typically a signed envelope from a vendor identity system), posts it to a Trusted Server endpoint, and the endpoint, after provider-specific verification, sets the first-party `ts-ec` cookie. @@ -41,11 +41,11 @@ This differs from server-side providers in one security-critical way: **the identifier is attacker-influenceable input**, not server-derived evidence. Everything in this spec follows from that. -Why this feature is normative in the series rather than deferred: the first +This feature is normative in the series rather than deferred because the first vendor integration this project targets works client-side by design, since the page script talks to the vendor's identity system and hands the result to the edge, so the server-side path alone cannot carry it. The 2026-07-31 -draft deferred the feature for lack of a consumer; the consumer now exists +draft deferred the feature for lack of a consumer. The consumer now exists as a planned vendor provider, and v1 builds the endpoint that provider will verify against. @@ -55,7 +55,7 @@ verify against. | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Cross-site identity fixation** | `text/plain` POST is a CORS-simple request: any page on the web can `fetch(resolveUrl, {method: "POST", credentials: "include", body: payload})` with no preflight | An attacker pins a chosen identity onto a victim's first-party cookie jar, a login-CSRF for the ad-identity layer, and the victim's activity accretes to an attacker-controlled ID | | **Replay** | A captured valid payload (from the attacker's own session or a leak) replayed against another browser | Same as fixation, without needing to create payloads | -| **Phantom identity** | Endpoint sets the cookie without an identity-graph row | Later requests carry an EC that the KV graph has never seen; downstream sync and withdrawal logic operate on an identity that half-exists (the organic generation path explicitly refuses to write a cookie when the graph write fails, for exactly this reason) | +| **Phantom identity** | Endpoint sets the cookie without an identity-graph row | Later requests carry an EC that the KV graph has never seen. Downstream sync and withdrawal logic operate on an identity that half-exists (the organic generation path explicitly refuses to write a cookie when the graph write fails, for exactly this reason) | | **Un-tombstoneable identity** | Core does not recognize the provider's identifier shape | Withdrawal cannot expire or tombstone the identity, which is a compliance failure and not only a bug | | **Amplification** | The page script cannot observe an HttpOnly cookie, so it cannot know the cookie is already set | A POST on every page view of every session (PR #838's JS gated on reading a cookie its own server marked HttpOnly, making the guard permanently false) | @@ -65,12 +65,12 @@ verify against. 1. **Reject cross-site requests with an origin check.** v1 authorizes a request only when its `Origin` header names the publisher's configured - domain or a subdomain of it; a missing or foreign `Origin` is rejected + domain or a subdomain of it. A missing or foreign `Origin` is rejected with `403`. Browsers always send `Origin` on POST `fetch`, so its absence means a non-browser caller, which has no business on a page-script endpoint. The 2026-07-31 draft asked for an exact origin allowlist as new configuration plus an optional session-bound CSRF - token; v1 derives the allowed set from `publisher.domain` (existing, + token. v1 derives the allowed set from `publisher.domain` (existing, operator-trusted configuration) and admits the publisher's own subdomains, because the publisher controls their subdomain namespace, and the draft's sibling-subdomain concern applies to `Sec-Fetch-Site: @@ -80,41 +80,41 @@ same-site` (which v1 does not consult at all), not to a suffix match on configured apex. 2. **Verify the payload per provider.** The endpoint hands the posted payload to the selected provider's `resolve_from_client` and creates only - what the provider returns; whether the payload is trustworthy is the + what the provider returns. Whether the payload is trustworthy is the provider's responsibility, stated on the trait. A real vendor provider - verifies a signed, audience-bound, expiring envelope; the draft's + verifies a signed, audience-bound, expiring envelope. The draft's session-binding and replay analysis (see §3.9) is the bar that verification must clear when the vendor scheme lands. The demo provider verifies a fixed constant and is compiled out of production builds (§5). `resolve_from_client` is normative in v1, with a no-op default so server-side providers are untouched. -3. **Preserve the identity-graph invariant.** Implemented: the graph row is +3. **Preserve the identity-graph invariant.** Implemented. The graph row is written before the cookie is set, keyed by the provider's `normalize_id_for_kv` canonical form, exactly like the organic create path. No graph available → no create (`204`), same as organic generation; a graph write failure → `503`, no cookie. -4. **Round-trip through the lifecycle contract.** Implemented: read-back +4. **Round-trip through the lifecycle contract.** Implemented. Read-back goes through the selected provider's `accepts_id`, the KV key through `normalize_id_for_kv`, and withdrawal reaches the row like any other identity. A round-trip test drives an opaque client identifier through organic deferral, resolve, cookie set, and verbatim read-back. 5. **Exist on every adapter, where parity means identical behavior, - including identical refusal.** Partially implemented, documented: the + including identical refusal.** Partially implemented, documented. The Fastly adapter routes the endpoint (passing the same bot-gated identity graph as organic generation, so unrecognized clients cannot create through resolve either). The Axum, Cloudflare, and Spin adapters deliberately do not route it, matching `identify` and `batch-sync`, - which need the same platform KV wiring those adapters do not have; the + which need the same platform KV wiring those adapters do not have. The route list in the Spin adapter documents all three together. The draft's stronger ask, identical startup rejection of the client-cycle selection on adapters that cannot serve it, is the agreed follow-up when the portability adapters gain KV (§7.4). 6. **Be uncacheable and permission-gated on the provider's full - declaration.** Implemented: every response carries `Cache-Control: + declaration.** Implemented. Every response carries `Cache-Control: no-store`, and the gate is the selected provider's complete `required_permissions()` through the same resolved permission state as organic creating, not a hard-coded storage check. -7. **Bound every input.** Implemented: request body at most 65,536 bytes, where +7. **Bound every input.** Implemented. Request body at most 65,536 bytes, where an advertised `Content-Length` over the limit answers `413` before the read, and the read body is re-checked so a missing or false length does not bypass the bound. `Content-Type` allowlist: `text/plain` and @@ -132,8 +132,8 @@ no-store`, and the gate is the selected provider's complete `415` content type, `503` graph-write failure, `204` closed gate / no provider / no graph / unverified payload. Tests exercise each rejection. 8. **Define behavior against an existing identity, with no silent - replacement.** Implemented: resolving to the same identity refreshes - idempotently; resolving to a different identity while the request + replacement.** Implemented. Resolving to the same identity refreshes + idempotently. Resolving to a different identity while the request carries a recognized EC is rejected with `409`. Any legitimate re-identification flow (account link, vendor migration) is an explicit linking design this spec does not authorize (§7). @@ -146,7 +146,7 @@ no-store`, and the gate is the selected provider's complete envelope format that does not exist yet, and a CAS-class storage primitive no production adapter exposes today. Designing the reservation against a hypothetical envelope would repeat the mistake - this series exists to fix. v1's stance: the endpoint is safe without it + this series exists to fix. v1's stance is that the endpoint is safe without it for the providers v1 ships (the demo creates a constant, feature-gated out of production), and the reservation lands with the first vendor scheme, designed against its real envelope, with the draft's §3.9 as @@ -176,7 +176,7 @@ the bar for the vendor-scheme implementation: ## 4. Requirements on the page script - **The re-post guard must not depend on reading an HttpOnly cookie.** - Implemented as the draft's first option: the resolve response sets a + Implemented as the draft's first option. The resolve response sets a non-HttpOnly companion marker cookie (`ts-ecr=1`) carrying no identity, which is the only signal the page has that a resolve succeeded. The marker shares the Edge Cookie's scope and lifetime and is expired @@ -196,19 +196,19 @@ the bar for the vendor-scheme implementation: The interaction between provider-keyed bundle content and content-hash / SRI pinning remains open (§7.6). - **Any constant shared between Rust and TS is asserted equal by a test.** - Implemented: a Rust test reads the page-script source and asserts the + Implemented. A Rust test reads the page-script source and asserts the fixed word and the marker cookie name match their Rust constants, so a rename on either side fails the build instead of silently breaking the round trip. ## 5. Demo providers -Implemented as required: the `client-fixed` demonstration provider (fixed +Implemented as required. The `client-fixed` demonstration provider (fixed identifier, constant-equality verification) is compiled only behind the `client-fixed-demo` cargo feature. In a production build the settings validator rejects the selection at startup with a direct message, and the provider builder rejects it again as defense in depth. A fixed shared word -is not an identity; the demo exists to exercise verify-before-create end to +is not an identity. The demo exists to exercise verify-before-create end to end in tests and demonstrations. ## 6. Testing @@ -251,10 +251,10 @@ end in tests and demonstrations. | Draft position | v1 (PR #1046) | Why | | ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| Feature deferred, `resolve_from_client` de-normalized | Feature normative; trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | -| Origin check via new allowlist config + CSRF token | Publisher apex + subdomains from existing `publisher.domain`; missing/foreign `Origin` → `403` | Uses existing operator-trusted configuration; explicit allowlist stays open for multi-domain publishers (§7.5) | -| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme; draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | -| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph); portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes; startup rejection follow-up recorded (§7.4) | -| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option; testable and observable | +| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | +| Origin check via new allowlist config + CSRF token | Publisher apex + subdomains from existing `publisher.domain`. Missing/foreign `Origin` → `403` | Uses existing operator-trusted configuration. Explicit allowlist stays open for multi-domain publishers (§7.5) | +| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | +| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | +| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | | Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | | Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 0182495fbd..ce1631ce15 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -1128,7 +1128,7 @@ and active-scope-change rules. `security_cookie_same_site` accepts exactly emitted `Secure` attribute, and the cookie never carries `HttpOnly`. Both exposure booleans default to `false`. ClientID-to-origin additionally -requires the owner-scoped overlay capability. Host fingerprints additionally +requires the owner-scoped overlay capability. Host signals additionally require qualified JA4 availability and sign-offs 23/28. A selected adapter that cannot preserve admitted request-header field-line order or enforce the request/body limits fails startup for protection rather than synthesizing diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 0cedf6f630..d0252acf29 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -45,12 +45,12 @@ These are the initial sources. Issues #777 and #779 also envision publisher interaction and external services as permission sources. That source interface remains **explicitly deferred**, not silently dropped. §10 records the divergence, and the documentation (`docs/guide/permission-model.md`) already -frames consent as one source among several so a later source plugs into the +frames consent as one source among many so a later source plugs into the same mechanism. Core code resolves permissions through a per-permission `ConsentSignal` closure (`Grant`, `Revoke`, `Neutral`), so a new source is a new producer of that signal, not a new resolution algorithm. -Scope: the model governs decisions Trusted Server makes. A downstream protocol +Scope. The model governs decisions Trusted Server makes. A downstream protocol receives the full regulatory context only where that protocol defines fields for it (OpenRTB consent fields, and proxy-mode forwarding of raw strings). The draft's stronger rule, that identity rows carry normalized per-permission @@ -129,7 +129,7 @@ or replaces the file and rebuilds to change policy. The file is not read at runtime. This keeps the mechanism the draft rejected, for a reason the draft's own -premise no longer supports: the draft required policy to flow through the +premise no longer supports. The draft required policy to flow through the runtime config pipeline (`ts config push`, staged activation, §5.5), and that activation apparatus does not exist. Publishing a `[permissions]` TOML section with no activation protocol would reintroduce exactly the mixed-revision and @@ -148,7 +148,7 @@ Within the embedded model, the draft's specific complaints are answered: a misspelled override key fails loudly instead of being swallowed. - Two rule keys naming the same location in different case are rejected at parse, so one spelling cannot silently overwrite another. -- Auditability lives where the draft placed it, in version control: the file +- Auditability lives where the draft placed it, in version control. The file ships in the repository, and its history is the change log. The `include_str!` path still reaches above the crate root, so the crate @@ -229,7 +229,7 @@ Format rules, as implemented: - A rule key is a bare country or a `country/region` pair. Keys are matched case-insensitively, and a region entry takes precedence over its country entry. -- The **signals** section is new relative to the draft: the TCF purpose to +- The **signals** section is new relative to the draft. The TCF purpose to Data Use mapping, the opt-out source list, and the opt-out revoke set are data in the file, so no signal-to-permission policy lives in the code. The `signals.tcf.authoritative` flag governs only whether a present TCF @@ -275,7 +275,7 @@ Not implemented. `detect_jurisdiction`, driven by the runtime lists jurisdiction source for the auction consent gate and for `ConsentContext.jurisdiction`, while the permission model resolves against `permissions.yaml` independently. The drift risk the draft named is real and -stands recorded: adding a country to one source has no effect on the other, +stands recorded. Adding a country to one source has no effect on the other, and no CI test asserts consistency between the legacy lists and the policy table. Unifying the two, with the auction gate reading a policy regime class, travels with the dispatch migration (§7.4) as deferred follow-up. @@ -318,7 +318,7 @@ sources count and what they revoke or grant. The code decides only the order. closed rather than degrading to the no-signal baseline, which under a `granted` baseline would turn garbage into a grant. It never withdraws (§4.2). An **expired** TCF record is deliberately a distinct state, not - malformed: the decoded record is cleared, the raw string is kept for + malformed. The decoded record is cleared, the raw string is kept for proxy forwarding, and acquisition proceeds as if the record were absent, so the baseline applies. 4. **Only then does a present TCF record decide the mapped Data Uses**, when @@ -363,8 +363,8 @@ outcome is also a **withdrawal** is a separate, narrower question (§4.2). ### 4.2 Withdrawal vs. absence -Withdrawal (destructive: expire the `ts-ec` cookie, write the identity-graph -tombstone) and non-grant (the permission is simply unset, EC response +Withdrawal (destructive, expiring the `ts-ec` cookie and writing the +identity-graph tombstone) and non-grant (the permission is simply unset, EC response headers stripped, nothing egressed) are distinct outcomes, never conflated. "Baseline" below means the resolved acquisition rule for `necessary.operations.storage` in the request's jurisdiction, resolved once @@ -385,7 +385,7 @@ The implemented trigger, exhaustively (nothing else withdraws): signals only, satisfying the draft's live-request constraint by construction. 2. **US-style opt-outs never withdraw.** GPC and sale opt-outs are use - restrictions: they suppress the permissions the policy revokes (EC + restrictions, which suppress the permissions the policy revokes (EC headers stripped, nothing egressed) but never trigger destruction, so lifting the opt-out restores the identity. 3. **A malformed record never withdraws.** It suppresses only (§4, step 3). @@ -394,12 +394,12 @@ The implemented trigger, exhaustively (nothing else withdraws): made a choice is never stripped of an existing identity. 5. **A policy change is not a user signal.** There are no runtime policy edits (§3.1), and a rebuild that tightens a baseline does not itself - tombstone: withdrawal still requires the affirmative refusal above on a + tombstone, because withdrawal still requires the affirmative refusal above on a live request. The draft's additional trigger, an explicit storage-withdrawal or authenticated deletion request honored in every jurisdiction, has no -implemented carrier: no such endpoint exists. It is recorded as deferred +implemented carrier, as no such endpoint exists. It is recorded as deferred (§11), and when it arrives it joins this list as a global trigger. `ec_storage_withdrawn` (in `ec/consent.rs`, surfaced as @@ -428,7 +428,7 @@ associated consistency and retention contracts. That machinery depends on storage primitives (linearizable per-key CAS, independent durability domains) the current adapters do not qualify. The 2026-07-31 draft remains the reference design for that work. Until it lands, the known gaps the -draft called out stand: cookie expiry is not fenced on the tombstone +draft called out stand. Cookie expiry is not fenced on the tombstone commit, and revocation durability is bounded by the KV store's behavior. ### 4.4 Signal normalization @@ -444,7 +444,7 @@ per-permission `ConsentSignal` closure. The implemented pipeline: 2. Resolve standalone-TCF vs GPP-embedded-TCF conflicts per the configured mode (`restrictive`, `permissive`, `newest`), preserving the pre-epic selection algorithm. -3. Apply the expiry check: a TCF record older than the configured maximum +3. Apply the expiry check, where a TCF record older than the configured maximum age has its decoded form cleared, the `expired` flag set, and its raw string preserved. Expiry is its own state, excluded from malformed-present, and resolves as absent for acquisition. @@ -459,18 +459,18 @@ so the pre-epic order stands. Recorded in §11. store (not on the `EcContext` construction path), a request carrying no consent signals falls back to the consent persisted for that EC ID, with the jurisdiction re-derived from the current request's geo. Staleness is -enforced by the store: entries are written with a TTL equal to +enforced by the store, where entries are written with a TTL equal to `max_consent_age_days`, so an entry older than a live record's allowed age has expired out of the store. A live signal always wins because the fallback is consulted only when the request carries none. The draft's declared change, running the loaded record through the full normalization -pipeline, is not implemented: the loaded record substitutes directly. The +pipeline, is not implemented. The loaded record substitutes directly. The narrow read is permission-exempt by construction, since determining storage cannot itself require storage. **Proxy mode.** Proxy mode still skips semantic decoding entirely. The draft's minimal opt-out extraction was not implemented, but the fail-open -consequence the draft feared does not arise under the permission model: a +consequence the draft feared does not arise under the permission model. A present record in proxy mode is present-but-undecoded, which blocks every baseline grant (§4, step 3), and the GPC header needs no decoding, so the GPC opt-out is honored directly. No grants are ever derived in proxy mode. @@ -545,9 +545,9 @@ than constructing the status by hand. provider that performs its own fallible lookup, and none of the providers shipped in this workspace is one. Fastly's `geo_lookup` returns an `Option` and the SDK collapses every hostcall, buffer and parse failure -into `None` before it reaches the caller; the Cloudflare provider reads -request headers, which cannot error; the Axum and Spin providers resolve -nothing at all; and `DisabledGeo`, the default whenever +into `None` before it reaches the caller. The Cloudflare provider reads +request headers, which cannot error. The Axum and Spin providers resolve +nothing at all. `DisabledGeo`, the default whenever `[geo] provider` is not `"platform"`, returns nothing by construction. So **a host geo outage today does not reach this floor.** It surfaces as `Ok(None)`, which is `NoLocation`, and falls back to the deployer's @@ -557,7 +557,7 @@ outage does, and should read §5.3 and the default-country guidance with that in mind. The floor becomes reachable when a vendor geo crate under `crates/geo/` does a real lookup that can fail. -At the floor, an explicit valid grant still counts: a TCF record consenting +At the floor, an explicit valid grant still counts. A TCF record consenting to a mapped purpose sets that permission under `requires_signal`, exactly the divergence-from-deny-all the draft declared for this row. Absent, malformed, or refusing evidence sets nothing. @@ -583,7 +583,7 @@ makes the dangerous migration config (a permissive `default_country` with geo unset) an explicit operator decision rather than an accident, closing the highest-severity finding of the PR #838 review. -The guard's consumer list is narrower than the draft's: the draft enumerated +The guard's consumer list is narrower than the draft's. The draft enumerated every jurisdiction consumer (EC provider, regime-gated auction dispatch, raw-EC and EID egress). In the implementation the EC provider is the only consumer whose behavior the policy gates, because auction dispatch was not @@ -593,7 +593,7 @@ dispatch joins the model, the guard's trigger list grows with it. ### 5.4 Defaults: one deployer fallback plus a protective floor `[geo] default_country` is **required in every mode** and is validated at -startup: it must be set, and it must resolve to a rule in +startup, meaning it must be set, and it must resolve to a rule in `permissions.yaml`. It accepts a country (`FR`) or a country/region key (`US/CA`), matched case-insensitively, so a no-geo single-state deployment can select its state rule. It covers two states the draft kept separate: @@ -604,7 +604,7 @@ can select its state rule. It covers two states the draft kept separate: The draft's `rules.default` policy entry does not exist, so "resolved-but-unmatched" falls to the same deployer default as "unresolved". The separation the draft treated as safety-critical is the -one the implementation does keep: a **failed** lookup never reaches the +one the implementation does keep, where a **failed** lookup never reaches the deployer default and resolves at the requires-signal floor instead (§5.2). With no default configured startup fails, and in the unreachable belt-and-braces case where resolution still finds no rule, the floor @@ -612,7 +612,7 @@ applies. ### 5.5 Policy revision activation (deferred) -Not implemented, and currently moot: policy is compiled into the binary +Not implemented, and currently moot. Policy is compiled into the binary (§3.1), so the deployed artifact is the policy identity and there is no runtime activation to coordinate. The draft's activation design, covering the JCS-canonical policy digest and ordinal pair, the activation register @@ -628,17 +628,17 @@ requires a live user signal (§4.2), never a policy change. | Condition | Resolution behavior | | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Geo lookup reports a failure | Requires-signal floor for every permission, never the deployer default; error logged. No provider shipped today can report one, so a host geo outage lands on the row above instead (§5.2) | +| Geo lookup reports a failure | Requires-signal floor for every permission, never the deployer default, with the error logged. No provider shipped today can report one, so a host geo outage lands on the row above instead (§5.2) | | No geo provider configured | `default_country` baseline, guarded by `assume_single_jurisdiction` (§5.3) | | Country resolved, no matching rule | `default_country` baseline (§5.4) | | Region resolved, no region rule | Country rule | | `default_country` unset or names no rule | Startup failure (§3.3) | | EC provider configured, no geo, no acknowledgment | Startup failure (§5.3) | -| Malformed `permissions.yaml` | Parse error at settings load, once per instance; the embedded file is a build-time constant, never per-request | -| Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition); never withdraws; opt-outs still honored | -| Expired TCF record | Distinct state, not malformed; treated as absent, so the baseline applies | +| Malformed `permissions.yaml` | Parse error at settings load, once per instance, because the embedded file is a build-time constant, never per-request | +| Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition), never withdraws, and still honors opt-outs | +| Expired TCF record | Distinct state, not malformed, and treated as absent, so the baseline applies | | Signals contradict (opt-out plus consent) | Opt-out wins (§4) | -| No EC provider selected | Identity fails closed: nothing created, an incoming cookie value never used or egressed (§7) | +| No EC provider selected | Identity fails closed, with nothing created and an incoming cookie value never used or egressed (§7) | The posture is fail-closed. Every ambiguous state resolves to the configured baseline or more restrictive. @@ -653,7 +653,7 @@ Consumers of the resolved set in the implementation: every declared permission is set (`ec_allowed`). A provider that requires nothing always runs. **Geo** is ungated because gating it is circular, jurisdiction being an input to permission resolution. - **Device** is ungated by a separate, deliberate decision: its + **Device** is ungated by a separate, deliberate decision, because its security-classification role must run for traffic that has granted nothing, and operator selection is the recorded authorization (providers spec §5). The built-in Edge Cookie providers declare @@ -661,7 +661,7 @@ Consumers of the resolved set in the implementation: 2. **EC lifecycle.** Creation requires the provider's declared permissions through the gate above. Withdrawal follows §4.2. Recognition and - revocation of an existing identifier are never permission-gated: the + revocation of an existing identifier are never permission-gated, and the withdrawal path runs precisely when `ec_allowed` is false, reading the raw cookie value kept for that purpose. @@ -678,11 +678,11 @@ Consumers of the resolved set in the implementation: | OpenRTB `user.id` on the `/auction` endpoint | `ec_sharing_allowed` | | Identify endpoint (partner-facing) | `ec_sharing_allowed` | | Pull sync (browser-request-scoped) | `ec_sharing_allowed`, from the live request resolution | - | Batch sync (context-free S2S) | Authenticated; withdrawn or missing rows are ineligible | + | Batch sync (context-free S2S) | Authenticated, and withdrawn or missing rows are ineligible | | KV EID resolution for auctions | `ec_allowed`, then the EID pair gate on the result | | Publisher navigation and page-bids `user.id` | `ec_sharing_allowed` (the storage plus personalised-ads pair) | - With **no EC provider configured**, identity fails closed: the gate is + With **no EC provider configured**, identity fails closed, meaning the gate is closed rather than open by default (`ec_allowed` is false), so a cookie value present on the request is treated as absent and never used or egressed. This replaces PR #838's vacuously-true `is_none_or` check. @@ -702,7 +702,7 @@ Consumers of the resolved set in the implementation: 4. **Server-side auction dispatch (not migrated).** Dispatch is still gated by the consent subsystem (`consent_allows_server_side_auction`), not by - the permission model: when the jurisdiction is GDPR or unknown, or an EU + the permission model. When the jurisdiction is GDPR or unknown, or an EU TCF signal is present, dispatch requires an effective TCF record consenting to Purpose 1, and otherwise no bid request leaves (a no-bid response, with no PBS/APS call and no UA/IP/geo forwarding). Known @@ -715,7 +715,7 @@ Consumers of the resolved set in the implementation: (§11) together with §3.4. The client-cycle resolve endpoint (`/_ts/api/v1/ec/resolve`) is a further -consumer: a provider-derived identifier posted by the page is accepted only +consumer, where a provider-derived identifier posted by the page is accepted only through the same provider and permission gates. ### 7.1 Contextual OpenRTB v1 allowlist (deferred) @@ -741,8 +741,8 @@ Implemented, in `permissions.rs`, `ec/consent.rs`, `ec/mod.rs`, destroying, consent is not a withdrawal, GPC alone never withdraws, sale opt-outs never withdraw, no signal never withdraws, malformed never withdraws. -- **Fail-closed acquisition.** Malformed records block baseline grants; - each undecodable record family is detected; an expired TCF record is not +- **Fail-closed acquisition.** Malformed records block baseline grants, + each undecodable record family is detected, and an expired TCF record is not treated as malformed and resolves at the baseline. - **Geo status.** A failed lookup resolves at the requires-signal floor (permissions and the storage baseline both), driven through a @@ -790,10 +790,10 @@ acceptance contract, not two: | #779 says | This spec says | Why | | -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | -| Unmatched countries fall to `default_country` | Adopted: unmatched and unresolved requests both fall to the required `[geo] default_country`; a **failed** lookup floors instead (§5) | The failure state is the one that must never reach a permissive default; the draft's `rules.default` split was not kept | -| The full TCF purpose vocabulary is modeled | Adopted and extended: all eleven purposes are signal-resolved, and the full Privacy Taxonomy is carried as declared baseline (§2) | The joint taxonomy work made whole-taxonomy declaration the goal; `denied` defaults keep undeclared uses inert | -| Policy is an embedded file | Adopted: `permissions.yaml` is compiled into the build (§3.1); runtime configuration is deferred follow-up | The runtime push and activation pipeline does not exist; version control is the audit trail meanwhile | -| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signals; further sources are deferred, and the `ConsentSignal` closure is their seam (§1) | Shipping an interface with no second source repeats the inert-surface mistake; the extension seam is defined | +| Unmatched countries fall to `default_country` | Adopted. Unmatched and unresolved requests both fall to the required `[geo] default_country`, and a **failed** lookup floors instead (§5) | The failure state is the one that must never reach a permissive default, and the draft's `rules.default` split was not kept | +| The full TCF purpose vocabulary is modeled | Adopted and extended. All eleven purposes are signal-resolved, and the full Privacy Taxonomy is carried as declared baseline (§2) | The joint taxonomy work made whole-taxonomy declaration the goal, and `denied` defaults keep undeclared uses inert | +| Policy is an embedded file | Adopted. `permissions.yaml` is compiled into the build (§3.1), and runtime configuration is deferred follow-up | The runtime push and activation pipeline does not exist, and version control is the audit trail meanwhile | +| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signals. Further sources are deferred, and the `ConsentSignal` closure is their seam (§1) | Shipping an interface with no second source repeats the inert-surface mistake, and the extension seam is defined | ## 11. Revision record vs the 2026-07-31 draft @@ -803,11 +803,11 @@ PR #1045). | Draft position | Implemented position | Why | | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Vocabulary is two TCF-purpose identifiers, enforced permissions only (§2) | IAB Privacy Taxonomy Data Uses: eleven named purposes all signal-resolved, plus 53 taxonomy Data Uses carried as declared but unenforced baseline flags | The joint taxonomy adoption postdates the draft; whole-taxonomy declaration serves completeness and demonstration, with `denied` defaults keeping unenforced flags inert | -| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Policy is `permissions.yaml`, compiled into the build with `include_str!`, parsed once and covered by tests | The runtime config push and activation apparatus does not exist; publishing runtime policy without it would recreate the hazards the draft cataloged; runtime policy is deferred follow-up | -| Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated; the field returns with that work | -| Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted: a detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema; `requires_signal` is now expressible per rule | -| Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated; a failed lookup floors separately | One deployer knob is simpler; the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | +| Vocabulary is two TCF-purpose identifiers, enforced permissions only (§2) | IAB Privacy Taxonomy Data Uses, with eleven named purposes all signal-resolved, plus 53 taxonomy Data Uses carried as declared but unenforced baseline flags | The joint taxonomy adoption postdates the draft, and whole-taxonomy declaration serves completeness and demonstration, with `denied` defaults keeping unenforced flags inert | +| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Policy is `permissions.yaml`, compiled into the build with `include_str!`, parsed once and covered by tests | The runtime config push and activation apparatus does not exist, so publishing runtime policy without it would recreate the hazards the draft cataloged. Runtime policy is deferred follow-up | +| Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated, and the field returns with that work | +| Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted. A detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema. `requires_signal` is now expressible per rule | +| Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated, and a failed lookup floors separately | One deployer knob is simpler, and the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | | Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first; the EU/EEA coverage test guards the shipped table against the typo class; ISO-assignment checks are future hardening | | Three-class signal taxonomy with regime-scoped grant acceptance; the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source; the US posture is a `granted` baseline that opt-outs revoke; explicit non-opt-out values grant nothing | A simpler two-signal model without regimes; the cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | | Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification; per-family scoping needs the full §4.5 decoder work | diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index b8000564e2..3932938736 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -96,16 +96,16 @@ default_country = "FR" # required (section 6) ``` **The `host-signals` EC provider** (identity from HMAC over the host TLS JA4 -and HTTP/2 fingerprints plus the client IP) was deliberately dropped from the +and HTTP/2 signals plus the client IP) was deliberately dropped from the 2026-07-31 draft. It has since shipped in PR #1044 as an opt-in built-in (`[ec.providers.host-signals]`), implemented against the host-agnostic `HostSignals` capability rather than a Fastly API, so any host that supplies -the fingerprints can run it and a host that supplies none cannot build it. -When the host supplies no fingerprint at all the provider defers with a +the signals can run it and a host that supplies none cannot build it. +When the host supplies no signal at all the provider defers with a warning instead of degrading to an IP-only identifier under the host-signals name. **An open review question stands on whether this provider should ship in the series at all**, because its identifier shape shares the built-in -HMAC grammar and a sign-off row defers host fingerprint processing. The +HMAC grammar and a sign-off row defers host signal processing. The question is flagged for the series review and this spec does not present the provider as settled either way. @@ -124,7 +124,7 @@ an EC value through the selected provider: | **Create** | EC generation on first eligible request, and the client-cycle resolve endpoint | The provider returns the identifier (`generate` server-side, `resolve_from_client` for the client cycle) and only core writes the cookie, after enforcing the global bounds below. | | **Recognize** | Reading `ts-ec` back from the request, deciding `ec_was_present`, withdrawal checks, and every path that hands the value onward: the origin URL in `append_ec_id`, the click-target URL in `handle_first_party_click`, and the proxied body an integration builds | `accepts_id` answers whether a value is a well-formed identifier the provider issues. A value the selected provider does not recognize is treated as absent, so it is never used or egressed, while the raw cookie value stays visible to withdrawal handling. The egress paths reach the same answer through `edge_cookie::recognized_ec_id`, and a deployment with no provider selected recognizes nothing and so egresses nothing. | | **KV key** | Identity-graph row reads and writes | `normalize_id_for_kv` returns the key form. The default lowercases the built-in HMAC hash segment and preserves the suffix, keeping today's keys. An opaque or case-sensitive provider overrides to the identity function so distinct identifiers never collapse into one row. | -| **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader: it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | +| **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader, because it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | **Invariant:** for every provider `P` and every identifier `id` created by `P`, `id` round-trips read-back byte for byte. A test in `ec/mod.rs` proves @@ -152,7 +152,7 @@ straddle the envelope never count each other, and `cluster_size` under-reports for as long as both populations coexist. That undercount is accepted rather than bridged, for three reasons. -`cluster_size` is reported in the identify response and gates nothing: the +`cluster_size` is reported in the identify response and gates nothing, and the only place its value is read at all is a cache short circuit in `evaluate_cluster` that tests whether a value is stored, not what it is, and the `cluster_trust_threshold` and `cluster_recheck_secs` settings that a @@ -192,7 +192,7 @@ wrong place, because what a provider may see is not the control. What a provider may do with what it sees is the control, and that is the permission model. So `RequestInfo` carries everything the request carries, whether or not code in -this repository reads it yet. The rule stands for behavioural traits, where a +this repository reads it yet. The rule stands for behavioral traits, where a method with no caller really is dead weight. How the surface observed in PR #838 resolved in the implementation: @@ -269,9 +269,9 @@ providers cannot silently mix identity populations, and a withdrawal always acts on a key that can only belong to one provider. The built-in HMAC provider creates `hmac~<64 hex>.<6 alphanumeric>` and dual-reads its pre-envelope bare form for one release cycle so deployed cookies keep -working; the bare form belongs to hmac alone. Codes are allocated +working, and the bare form belongs to hmac alone. Codes are allocated append-only in `provider-code-registry.md`, and a leading digit is valid -(`51dd`). +(`1a2b`). The draft's alternative shape (`parse` returning a typed `EcId`, `graph_key_suffix`, `cluster_prefix`, `verify`, and a version-carrying @@ -328,14 +328,14 @@ structural: selection defaults.** Device classification is not an input to permission resolution. The neutral `builtin` classifier reads only the User-Agent and makes no host call. The draft went further and made selecting a - fingerprint-reading device provider a startup error pending a separate + host-signal-reading device provider a startup error pending a separate security design. The implementation instead ships `[device] provider = "fastly"` as a selectable opt-in. The Fastly adapter injects a - `HostSignals` service carrying the TLS JA4 and HTTP/2 fingerprints, and + `HostSignals` service carrying the TLS JA4 and HTTP/2 signals, and the provider uses them to strengthen the browser/bot gate that guards EC writes. Identity rows persist the derived classification fields (the JA4 class segment and a 12-hex-character hash prefix of the HTTP/2 SETTINGS - fingerprint), not raw fingerprints, and the neutral default persists + signal), not raw signals, and the neutral default persists neither because the builtin provider produces no such fields. ## 6. Selection, validation, and failure modes @@ -344,7 +344,7 @@ All configuration validation happens at **settings construction** (`Settings::finalize_deserialized` runs every check below), so a misconfiguration expressible in configuration alone is a startup error, never a silent behavior change. A selection that only the running host can -satisfy (an injected vendor provider, or host fingerprints) fails loudly +satisfy (an injected vendor provider, or host signals) fails loudly when the provider is built, stopping the request rather than degrading. | Configuration state | Behavior | @@ -354,7 +354,7 @@ when the provider is built, stopping the request rather than degrading. | `provider = "none"` (explicit stateless) | Valid, and means exactly what omitting the selector means. Any configured provider block alongside it is a startup error, the same stray-block rule as below. | | A configured `[ec.providers.]` block that is not the selected one | **Startup error** (checked for the `hmac` block and every vendor block). An unreferenced block is almost always a mistyped selector or a stale block, and accepting it silently invites configuration drift. | | A selected vendor key whose provider the adapter did not inject | Loud failure when the provider is built, naming the key, so the deployment never silently runs stateless. | -| `provider = "host-signals"` on a host that supplies no fingerprints | Loud failure when the provider is built. A host that cannot produce `HostSignals` cannot run the provider. | +| `provider = "host-signals"` on a host that supplies no signals | Loud failure when the provider is built. A host that cannot produce `HostSignals` cannot run the provider. | | `provider = "client-fixed"` in a production build | Startup error. The demonstration provider is compiled only behind the `client-fixed-demo` cargo feature. | | No `provider`, no providers block | Valid, the neutral default for that concern. | | Deprecated `[ec] passphrase` | Migrated to `provider = "hmac"` with the passphrase in `[ec.providers.hmac]`, with a deprecation warning naming the new location. Both forms together are rejected so a half-edited file fails loudly instead of one form silently winning. | @@ -406,7 +406,7 @@ What a switch does, precisely: treated as absent. It never becomes the request's active identity, never egresses to a partner, and is rejected on the pull-sync, batch-sync and admin paths too. This is the §5 guarantee and it is the half of the - behavior that matters most: two providers' identity populations can never + behavior that matters most, which is that two providers' identity populations can never mix. - **The browser cookie.** A later withdrawal still expires the `ts-ec` cookie, because that path keys off the raw cookie being present rather @@ -443,7 +443,7 @@ logged, none silent: | `generate` returns an error | No identity this request. The organic caller logs at error level and the request proceeds stateless. No cookie is written. | | A provider creates an identifier outside the global bounds | Rejected at create, never rewritten. The organic path yields no identity. The resolve endpoint returns 400. | | Identity-graph write fails at create | The create is undone (no identifier, no cookie), with the error logged. The resolve endpoint returns 503. The next eligible request retries. | -| The host-signals provider finds no TLS/HTTP-2 fingerprints | Defers with a warning. No identity this request, and no degraded IP-only identifier is created under the host-signals name. | +| The host-signals provider finds no TLS/HTTP-2 signals | Defers with a warning. No identity this request, and no degraded IP-only identifier is created under the host-signals name. | | Geo lookup **fails** (the provider errors) | Every permission resolves to the requires-signal floor, and the failure is logged at error level. The failure is **not** papered over with the `default_country` baseline. | | Geo resolves **no location**, or a country/region with no rule | The `[geo] default_country` baseline applies. This is the configured-default case, deliberately distinct from the failure row above (`GeoStatus` in `ec/consent.rs`). | | An incoming cookie value fails the bounds at read-back | Treated as absent, with a warning naming the source. | @@ -462,12 +462,12 @@ The draft specified a delimiter-free physical key grammar with fixed-width segments, a provider-code registry, record classes for family revocation, authority state, negative-intent outbox, rowless withdrawal, and deployment metadata, wire schemas with known-answer vectors, and a per-field graph-row -contract. The provider-code registry is now implemented: codes are +contract. The provider-code registry is now implemented, with codes allocated in `provider-code-registry.md`, carried as the `{code}~` prefix of every created identifier, and therefore present in every graph key. The key grammar differs from the draft in one deliberate way, a tilde separator instead of delimiter-free fixed width, because pre-envelope bare -identifiers remain deployed and a code such as `51dd` is valid hex, so +identifiers remain deployed and a code such as `1a2b` is valid hex, so delimiter-free parsing could misread a legacy identifier during the migration window. The remainder (record classes, family revocation, authority state, outbox, rowless withdrawal, wire schemas, per-field @@ -588,7 +588,7 @@ Implemented, in the crates named: acknowledgment. - Geo builder tests showing the default selects no geo, `none` selects no geo explicitly, and `platform` selects the host implementation. -- Host-signals provider tests covering creating from fingerprints, +- Host-signals provider tests covering creating from host signals, deferring without them, and the loud failure of a selected but uninjected vendor provider. @@ -644,8 +644,8 @@ this revision describes. | Every selection key is closed and unknown keys are startup errors | Device and geo keys are closed. EC vendor keys are open. Unknown blocks are captured as raw values in core, the adapter deserializes its own block, and a selected key with no injected provider fails loudly. | Core never names a vendor, so a vendor provider adds no core change. | | Capability mismatch is a startup error at adapter wiring time | Configuration coherence fails at startup. A host-capability mismatch (missing `HostSignals`, uninjected vendor) fails loudly when the provider is built, stopping the request. | The adapter capability declaration that would move the check to startup is deferred with the capability matrix. | | A creating provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to create without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | -| `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw fingerprints. | Selection is an explicit operator opt-in and the neutral default makes no host fingerprint call. | -| The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host fingerprint processing, so the review decides whether the provider ships in the series. | +| `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw signals. | Selection is an explicit operator opt-in and the neutral default makes no host signal call. | +| The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host signal processing, so the review decides whether the provider ships in the series. | | Geo default flip sequenced into the later permission-model step, with an acknowledgment guard | Landed as specified in the same series, with the default of none, `default_country` required and validated against `permissions.yaml`, the `assume_single_jurisdiction` acknowledgment, and a failed lookup resolving to the requires-signal floor with error logging (`GeoStatus`, resolved in core so all adapters agree). | The permission model shipped in PR #1045, so the constraints exist where the draft required them. | | All adapters serve the full EC feature set identically | Selector behavior is identical through the shared builders and core constructors. The EC API routes (identify, batch-sync, ec/resolve) are Fastly-only, documented in the Spin route list. | The portability adapters do not yet wire platform KV, and the gap is documented rather than silent. | | Conformance suite, adapter capability matrix, delimiter-free key grammar, `verify`, `legacy_providers`, `versions` / `mint_version` | None of these are in PR #1043 or #1044. All are tracked follow-up work, deferred, not silently dropped. | The shipped seam did not need them, and each returns with the feature that gives it a production caller, per the spec's own minimalism rule. | diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index efe904a96d..5886f7f71b 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -1,7 +1,7 @@ # Design Spec: Provider and Permission Model, Migration and Rollout **Status:** Revised against the implemented series (PRs #1043-#1047), -2026-08-25. The §8 sign-off rows remain the series' decision ledger; rows the +2026-08-25. The §8 sign-off rows remain the series' decision ledger, and rows the implementation now satisfies are marked with their PR so the task force can ratify rather than re-litigate. **Author:** Engineering @@ -124,16 +124,16 @@ for the follow-up work that will implement them. | 3a | US-state request, explicit not-opted-out GPP/USP value → EC allowed | No US signal grants anything. N/A, absent, reserved, and unknown values grant nothing | Implemented (#1045) in a stricter form than drafted, where signals only revoke, so the drafted "explicit not-opted-out may grant" class does not exist (sign-offs 3, 17) | | 3b | US-state request, TCF record refusing Purpose 1, no US opt-out → no EC | Refusal beats coexisting non-TCF grant signals | Implemented (#1045), where an authoritative TCF refusal revokes its mapped uses. Under the shipped `granted` US baseline the refusal suppresses without tombstoning | | 3c | Consent-record conflict modes, expiry, KV fallback, proxy mode | Per the permission spec §4.4 matrix, whose changed row is that malformed-present blocks acquisition | Malformed-present fail-closed is implemented (#1045). The rest of the consent normalization pipeline is carried forward unchanged by the series | -| 3d | Valid plus expired consent records: conflict resolution can select the expired record | Expired sources drop before conflict resolution | Open. Not addressed by the series | +| 3d | Valid plus expired consent records, where conflict resolution can select the expired record | Expired sources drop before conflict resolution | Open. Not addressed by the series | | 3e | Only the GPP sale field (and USP) is consulted, with sharing/targeted opt-outs ignored | Sale, sharing, and targeted-advertising opt-outs deny the personalised-ads uses, and none affects storage or destroys identity | Partially implemented (#1045), where sale, USP, and Sec-GPC are honored and never destructive. The sharing and targeted-advertising GPP fields are not yet decoded (sign-off 3 open) | | 3f | Non-privacy-state US traffic (for example Wyoming) is non-regulated → EC allowed | Country-level `US` is a protective floor, region rules may be stricter, and regionless traffic never degrades to non-regulated | Implemented (#1045), where `permissions.yaml` maps country `US` to `us-opt-out`, with state rows able to override | -| 3g | Graph rows persist JA4 class, H2 fingerprint hash, and buyer-facing quality metadata | The draft discontinued these for new rows | **Contradicted by the series, flagged.** Rows still persist device signals, including fingerprint hash prefixes, when the opt-in providers run (#1044/#1046). Tied to the open host-signal question (sign-off 22) | +| 3g | Graph rows persist JA4 class, H2 signal hash, and buyer-facing quality metadata | The draft discontinued these for new rows | **Contradicted by the series, flagged.** Rows still persist device signals, including signal hash prefixes, when the opt-in providers run (#1044/#1046). Tied to the open host-signal question (sign-off 22) | | 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB with citation and sign-off | **The shipped default adopts `granted` storage for GB (#1045), flagged.** The citation and sign-off the draft required do not exist yet. The task force owns this row | | 5 | No country resolvable (geo failure) → no EC (fail-closed) | Protective failure profile, where permissions resolve at the requires-signal floor and `default_country` is reserved for unmatched requests in acknowledged static-jurisdiction mode | Implemented (#1045), where a failed lookup resolves at the requires-signal floor, logged at error level, and never falls back to `default_country` (sign-off 18) | | 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, identity never tombstoned | Refusal blocks new grants everywhere, and existing identity is never tombstoned where the baseline is `granted` | Implemented (#1045), where an authoritative refusal revokes its mapped uses everywhere and withdrawal is scoped to non-granted baselines | | 7 | Country resolved but in no regulation list → EC created, EIDs pass through | Governed by the deployment's default rule. The implementation expresses this as the required `[geo] default_country`, naming the `permissions.yaml` rule for unmatched requests | Implemented (#1045) with a changed mechanism, since no `rules.default` entry exists and `default_country` is required and validated at startup | | 8 | Opt-out signal outside US states → ignored today | Mapped use restrictions are honored globally, and opt-outs never tombstone identity | Implemented (#1045), where the signal mapping is jurisdiction-free and suppresses even TCF-consented uses, without destruction (sign-off 1) | -| 9 | Fastly bot gate requires JA4 plus platform class before KV-backed EC writes | The draft deferred host fingerprinting and startup-failed `[device] provider = "fastly"` | **Contradicted by the series, flagged.** The `fastly` device provider ships opt-in with `builtin` (UA-only) as the default (#1044). Whether the host-signal surface stays is sign-off 22, open | +| 9 | Fastly bot gate requires JA4 plus platform class before KV-backed EC writes | The draft deferred host signal processing and startup-failed `[device] provider = "fastly"` | **Contradicted by the series, flagged.** The `fastly` device provider ships opt-in with `builtin` (UA-only) as the default (#1044). Whether the host-signal surface stays is sign-off 22, open | | 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral default flips only together with the permission model's jurisdiction guard, never in an intermediate step | Implemented as sequenced (#1044 kept the platform default, and #1045 flipped geo off by default together with `default_country` and the acknowledgment guard) | | 11a | Raw EC egress on jurisdiction-gated paths today (`user.id`, EIDs, identify, pull sync) | Gated by the sharing pair (storage plus personalised-ads), at least as strict as today for every path | Implemented (#1045), where `ec_sharing_allowed` gates `user.id`, the identify response, and pull sync, and `gate_eids_by_permissions` gates EIDs, all on the same pair | | 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header without today's jurisdiction gate | Gated by the egress inventory (both purposes) | Open. Not implemented by the series (sign-off 8) | @@ -201,7 +201,7 @@ passphrase = "replace-with-32-plus-byte-random-secret" Requirements, each marked with its implementation state: -1. **The transition has a dual-read release; loud rejection comes one +1. **The transition has a dual-read release, and loud rejection comes one release later.** Implemented (#1043) in the simple form, where the current release accepts the old shape, mapping `[ec] passphrase` to the `hmac` provider internally and logging a deprecation warning per @@ -461,38 +461,38 @@ the code with the row open. | # | Recommended resolution | Where | Decision record | Status | | --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | Honor mapped use opt-outs globally. Destructive identity effects are limited to explicit storage withdrawal, authenticated deletion, or a qualifying live TCF Purpose 1 refusal. | permission §4, §4.2 | (none) | open. Implemented by #1045 (jurisdiction-free suppression, withdrawal only on a live TCF storage refusal under a non-granted baseline). Ratify | -| 2 | GPP/USP sale opt-outs suppress the personalised-ads uses only; they neither revoke storage nor delete the identity. | permission §4.5 | (none) | open. Partially implemented by #1045 (no opt-out deletes or destructively revokes). The shipped default maps opt-outs to `revokes: all` rather than personalised-ads only, a `permissions.yaml` choice to ratify | -| 3 | Sharing/targeted-advertising opt-outs suppress the personalised-ads uses; an explicit applicable not-opted-out value may grant them. Neither affects storage. | permission §4.5 | (none) | open. Not implemented (the sharing and targeted-advertising GPP fields are not decoded, and no US signal grants anything in the shipped model) | +| 2 | GPP/USP sale opt-outs suppress the personalised-ads uses only. They neither revoke storage nor delete the identity. | permission §4.5 | (none) | open. Partially implemented by #1045 (no opt-out deletes or destructively revokes). The shipped default maps opt-outs to `revokes: all` rather than personalised-ads only, a `permissions.yaml` choice to ratify | +| 3 | Sharing/targeted-advertising opt-outs suppress the personalised-ads uses. An explicit applicable not-opted-out value may grant them. Neither affects storage. | permission §4.5 | (none) | open. Not implemented (the sharing and targeted-advertising GPP fields are not decoded, and no US signal grants anything in the shipped model) | | 4 | US auction dispatch may continue while personalised-ads is unset only through permission §7's positive `ContextualAuctionView` and its sole normative inline manifest. | permission §7.1 | (none) | open. The strip half is implemented by #1045 (EIDs and `user.id` withheld when the pair is unset). The positive contextual projection is not implemented | -| 5 | Country-only and regionless US traffic use a protective country-wide `us-opt-out` floor; state rules may be stricter. | permission §3.4 | (none) | open. Implemented by #1045 (`permissions.yaml` maps country `US` to `us-opt-out` with state overrides available). Ratify | -| 6 | Raw regulatory strings reach only the positively registered OpenRTB field that requires each source; all other destinations default deny. Identity rows retain normalized provenance/digests, not raw consent snapshots. | permission §7; providers §6.3 | (none) | open. Not addressed by the series | +| 5 | Country-only and regionless US traffic use a protective country-wide `us-opt-out` floor. State rules may be stricter. | permission §3.4 | (none) | open. Implemented by #1045 (`permissions.yaml` maps country `US` to `us-opt-out` with state overrides available). Ratify | +| 6 | Raw regulatory strings reach only the positively registered OpenRTB field that requires each source. All other destinations default deny. Identity rows retain normalized provenance/digests, not raw consent snapshots. | permission §7; providers §6.3 | (none) | open. Not addressed by the series | | 7 | Reject legacy batch-sync traffic until live-browser provenance backfill makes the row re-evaluable. | rollout §6 item 6; permission §7 | (none) | open. Not implemented (no provenance exists to recompute) | | 8 | Gate proxy, click, and Testlight identity forwarding on the sharing pair (storage plus personalised-ads). | §2 row 11b | (none) | open. Not implemented by the series | -| 9 | Defer integration-owned cookie operations from the v1 response hook; require a complete read/use/withdraw lifecycle before admission. | hook §3 | (none) | open. Overtaken (#1047 removed the whole hook from the series, so no cookie surface shipped). The deferral returns with the hook's first consumer | -| 10 | Do not create a blanket session-cookie exemption; every cookie must be covered by an approved permission or narrowly defined security-use authority. | hook §3 | (none) | open. Hook not shipped, unaffected | +| 9 | Defer integration-owned cookie operations from the v1 response hook, and require a complete read/use/withdraw lifecycle before admission. | hook §3 | (none) | open. Overtaken (#1047 removed the whole hook from the series, so no cookie surface shipped). The deferral returns with the hook's first consumer | +| 10 | Do not create a blanket session-cookie exemption. Every cookie must be covered by an approved permission or narrowly defined security-use authority. | hook §3 | (none) | open. Hook not shipped, unaffected | | 11 | Require a durable per-family negative-intent outbox in a failure domain independent of its strong target and checked freshly by every identity consumer, with a globally visible breaker over positive identity operations when neither can commit. | permission §4.3 | (none) | open. Not implemented (part of the durable-suppression follow-up) | | 12 | Adapters that cannot meet the revocation-storage contract migrate stateless rather than weakening the contract. | rollout §6 item 2; recipe §5 | (none) | open. Direction implemented (`provider = "none"` spells stateless in #1043; the identity endpoints were already Fastly-only before the series, and #1046 keeps `resolve` on the same footing). The formal capability gate is deferred. Ratify the direction | -| 13 | Keep batch sync fail-closed at cutover; stage partner communication and cleanup using explicit coverage thresholds, windows, and pause actions. | rollout §6 item 6 | (none) | open. Not implemented (no provenance cutover exists yet) | -| 14 | Policy tightening does not reinterpret historical refusal as a destructive event; destructive withdrawal requires fresh, live qualifying evidence. | permission §4.2 trigger 2 | (none) | open. Implemented by #1045 (withdrawal evaluates the live request's TCF record only, and historical records are never reinterpreted). Ratify | -| 15 | Descope the client cycle and `rewrite_legacy`; ship the v1 integration hook as headers-only. | client spec status; providers §6.1; hook §3 | (none) | open. Overtaken (the client cycle shipped hardened as #1046 instead of descoped, `rewrite_legacy` does not exist, and the hook shipped not at all per #1047). Re-decide against the shipped shape | +| 13 | Keep batch sync fail-closed at cutover, and stage partner communication and cleanup using explicit coverage thresholds, windows, and pause actions. | rollout §6 item 6 | (none) | open. Not implemented (no provenance cutover exists yet) | +| 14 | Policy tightening does not reinterpret historical refusal as a destructive event. Destructive withdrawal requires fresh, live qualifying evidence. | permission §4.2 trigger 2 | (none) | open. Implemented by #1045 (withdrawal evaluates the live request's TCF record only, and historical records are never reinterpreted). Ratify | +| 15 | Descope the client cycle and `rewrite_legacy`, and ship the v1 integration hook as headers-only. | client spec status; providers §6.1; hook §3 | (none) | open. Overtaken (the client cycle shipped hardened as #1046 instead of descoped, `rewrite_legacy` does not exist, and the hook shipped not at all per #1047). Re-decide against the shipped shape | | 16 | Persist use-opt-out suppression until ordered explicit authorization for that use or identity deletion, with TCF `LastUpdated` or an authenticated monotonic revision proving order. | permission §4.3 | (none) | open. Not implemented (suppression in the series is request-scoped with no durable record) | | 17 | N/A, absent, reserved, unknown, and unsupported values never grant processing. | permission §4.5 | (none) | open. Implemented by #1045 in a stricter form (signals only revoke, so no US-signal value grants anything). Ratify | | 18 | A selected geo provider's lookup failure uses the compiled-in protective profile; `default_country` is only for acknowledged static-jurisdiction mode. | permission §5.2 | (none) | open. Implemented by #1045 (failed lookups resolve at the requires-signal floor, logged at error level, never `default_country`). Ratify | -| 19 | Use immutable version-addressed whole-config publication plus prepare/commit activation of the complete tuple, with authenticated fleet membership, bounded admission lease, quiescence, and an activation journal; a second unanimous model transition advances model epoch, minimum binary generation, and row schema floor atomically. | permission §5.5; rollout §6.1 | (none) | open. Not implemented (`ts config push` publishes and validates without the activation protocol) | -| 20 | N+1 keeps v1 creation and pre-epic live gating, reads/enforces N+2 negative state for rollback safety, and never originates durable use suppression; new-shape settings alone do not activate the new writer/model. | migration §4.4 | (none) | open. Overtaken in part (the shipped migration is a one-release dual-read of `[ec] passphrase` in #1043 with mixed forms rejected and nothing durable added, so the full interim waits for the durable design) | -| 21 | Expire and re-create rowless legacy cookies without continuity; a prefix match cannot authenticate the cookie suffix. | providers §5 | (none) | open. Not implemented (no rowless classification exists, and a cookie with no row is never shared) | -| 22 | Defer host JA4/H2 fingerprint processing to a separate approved design; reject `[device] provider = "fastly"` at startup and do not persist fingerprint-derived classifications. | providers §5 | (none) | open. **Contradicted by the series and flagged for review** (#1044 ships the `fastly` device provider and the host-signal EC provider opt-in, and device signals including fingerprint hash prefixes persist in rows) | +| 19 | Use immutable version-addressed whole-config publication plus prepare/commit activation of the complete tuple, with authenticated fleet membership, bounded admission lease, quiescence, and an activation journal. A second unanimous model transition advances model epoch, minimum binary generation, and row schema floor atomically. | permission §5.5; rollout §6.1 | (none) | open. Not implemented (`ts config push` publishes and validates without the activation protocol) | +| 20 | N+1 keeps v1 creation and pre-epic live gating, reads/enforces N+2 negative state for rollback safety, and never originates durable use suppression. New-shape settings alone do not activate the new writer/model. | migration §4.4 | (none) | open. Overtaken in part (the shipped migration is a one-release dual-read of `[ec] passphrase` in #1043 with mixed forms rejected and nothing durable added, so the full interim waits for the durable design) | +| 21 | Expire and re-create rowless legacy cookies without continuity. A prefix match cannot authenticate the cookie suffix. | providers §5 | (none) | open. Not implemented (no rowless classification exists, and a cookie with no row is never shared) | +| 22 | Defer host JA4/H2 signal processing to a separate approved design. Reject `[device] provider = "fastly"` at startup and do not persist signal-derived classifications. | providers §5 | (none) | open. **Contradicted by the series and flagged for review** (#1044 ships the `fastly` device provider and the host-signal EC provider opt-in, and device signals including signal hash prefixes persist in rows) | | 23 | Permit a narrow `SecurityUse` authority for DataDome only, with the exact bounded surface the hook spec defines. | hook §4a; permission §7 | (none) | open. Hook not shipped, unaffected | -| 24 | Malformed/absence suppression overrides a permissive baseline but clears on newer valid evidence; it is not sticky like an explicit use opt-out. | permission §4.3, §4.1 | (none) | open. Implemented by #1045 by construction (the fail-closed block is request-scoped, so newer valid evidence re-resolves). Ratify together with row 16's durable design | -| 25 | Enforce a stored-jurisdiction/provenance horizon for batch sync: moves into stricter regimes fail closed by the horizon, and moves out require a live visit. | permission §7 | (none) | open. Not implemented | +| 24 | Malformed/absence suppression overrides a permissive baseline but clears on newer valid evidence. It is not sticky like an explicit use opt-out. | permission §4.3, §4.1 | (none) | open. Implemented by #1045 by construction (the fail-closed block is request-scoped, so newer valid evidence re-resolves). Ratify together with row 16's durable design | +| 25 | Enforce a stored-jurisdiction/provenance horizon for batch sync, where moves into stricter regimes fail closed by the horizon, and moves out require a live visit. | permission §7 | (none) | open. Not implemented | | 26 | Aggregate embedded GPP GPC with `Sec-GPC` by OR as a global, non-destructive use opt-out. | permission §4.5 | (none) | open. Partially implemented by #1045 (`Sec-GPC` is an honored non-destructive source). The embedded GPP GPC subfield is not read | | 27 | In proxy mode, decode only mapped opt-out fields and derive no grants. | permission §4.4 | (none) | open. Partially (the shipped model derives no grants from any US signal anywhere). The proxy-mode decode restriction is not separately implemented | | 28 | Require product and written vendor conformance approval for the reduced DataDome surface the hook spec pins. | hook §4a.2 | (none) | open. Hook not shipped, unaffected | | 29 | Accept rowless roaming-cookie expiry as a bounded residual only with telemetry, an explicit maximum lifetime, operator documentation, and a removal/sunset criterion. | providers §5 | (none) | open. Not implemented | -| 30 | Saturation blocks rowless admission for that prefix but never revokes an authenticated real row without its exact suffix; monitor NAT-cohort pressure. | providers §5 | (none) | open. Not implemented | -| 31 | Keep replay history bounded by evicting expired/grant entries first and retaining restrictive state for its full horizon; saturation never shortens a later opt-out. | permission §4.3; providers wire schema | (none) | open. Not implemented | +| 30 | Saturation blocks rowless admission for that prefix but never revokes an authenticated real row without its exact suffix. Monitor NAT-cohort pressure. | providers §5 | (none) | open. Not implemented | +| 31 | Keep replay history bounded by evicting expired/grant entries first and retaining restrictive state for its full horizon. Saturation never shortens a later opt-out. | permission §4.3; providers wire schema | (none) | open. Not implemented | | 32 | Accept official GPP sections 24-27 version 1, pin their layouts to the vendored IAB commit, and treat complete decoder/fixture support as a release prerequisite, with full vendoring evidence. | permission §4.5.1 | (none) | open. Not implemented (the shipped decoder consults the GPP sale field, with no vendored IAB corpus) | -| 33 | Treat any malformed or unsupported-version **mapped** GPP section as a global blocker for grants to the permissions its schema maps, while still honoring decodable opt-outs elsewhere and never deriving withdrawal from malformed bytes; unknown unmapped section IDs remain non-contributing. | permission §4.5 | (none) | open. Partially implemented by #1045 (a present undecodable record blocks baseline grants). The per-section mapped-blocker rule is not implemented | +| 33 | Treat any malformed or unsupported-version **mapped** GPP section as a global blocker for grants to the permissions its schema maps, while still honoring decodable opt-outs elsewhere and never deriving withdrawal from malformed bytes. Unknown unmapped section IDs remain non-contributing. | permission §4.5 | (none) | open. Partially implemented by #1045 (a present undecodable record blocks baseline grants). The per-section mapped-blocker rule is not implemented | | 34 | Permit providers whose canonical identifiers cannot fit an injective graph suffix to use the `sha256-detect` mode: domain-separated collision resistance plus stored canonical-identifier comparison, fail-closed collision handling, no overwrite/join. | providers §2, §6.3 | (none) | open. Premise revised by #1043 (identifiers are globally bounded at 256 bytes and the graph is keyed by `normalize_id_for_kv`). No `sha256-detect` mode exists | ## Revision record vs the 2026-07-31 draft @@ -500,7 +500,7 @@ the code with the row open. | Draft position | Revised position | Why | | -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Epic #777-#781 pending sign-off, no implementation | Implemented as five stacked PRs #1043-#1047, with §8 staying the ledger and rows marked for ratification | The series shipped the seam, selection, permission model, resolve endpoint, and docs, so the task force ratifies rather than re-litigates | -| `[device] provider = "fastly"` startup-fails pending a separate fingerprinting design (§2 row 9, row 22) | The `fastly` device provider and host-signal EC provider ship opt-in, and device signals persist in graph rows. The identifier-collision defect the review found (host-signal identifiers shared the HMAC grammar and keyspace) is fixed by the mandatory provider-code envelope, so host-signal identifiers are `hs00~` namespaced; the policy question of row 22 is unchanged by the collision fix | Implementation choice, deliberately flagged as the open review question rather than presented as settled | +| `[device] provider = "fastly"` startup-fails pending a separate host-signal design (§2 row 9, row 22) | The `fastly` device provider and host-signal EC provider ship opt-in, and device signals persist in graph rows. The identifier-collision defect the review found (host-signal identifiers shared the HMAC grammar and keyspace) is fixed by the mandatory provider-code envelope, so host-signal identifiers are `hs00~` namespaced. The policy question of row 22 is unchanged by the collision fix | Implementation choice, deliberately flagged as the open review question rather than presented as settled | | US recipe: `requires_signal` with an extended grant-signal class (§2 rows 3/3a) | Shipped default: `granted` US baseline, `revokes: all` on any opt-out, and no US signal grants anything | The shipped signal model is revoke-only, which is simpler and stricter on grants, and the baseline choice is deployer-editable yaml, flagged in rows 3 and 5 | | `[permissions]` TOML policy published at runtime | `permissions.yaml` compiled into the build, no `[permissions]` block | Policy stays reviewable in version control and the partial-policy trap cannot be written | | `rules.default` worldwide default entry (§2 row 7) | Required `[geo] default_country` naming the fallback rule, validated at startup | Same role, one mechanism, loud when missing | diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 7e35617bce..3fbb27b702 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -44,7 +44,7 @@ the code lands. ## 1. The problem, with the code that causes it Every claim here was read from `main` at b7fcb5d4c (28 August), which the -seventh PR targets; the five series PRs do not touch these files. +seventh PR targets, and the five series PRs do not touch these files. 1. **The registry is closed.** `IntegrationRegistry::new` takes only `&Settings` and iterates a fixed table @@ -137,7 +137,7 @@ modules so a vendor crate rebuild invalidates the server-side template cache. The registry verifies a carried module's declared hash against its source when it is built, so a stale literal is a startup error rather than a stale script served under a valid-looking URL. Covering carried modules in -the template cache fingerprint means the publisher entry point needs the +the template cache hash means the publisher entry point needs the registry, so it takes the configuration and the registry as one argument rather than two. The served script keeps its cache rule, being the `?v=` query matched at serve time (there is no integrity attribute on the tag today, and @@ -271,7 +271,7 @@ Two more places every move must touch, found by mapping `main`: files included, so a vendor move that leaves its entry behind breaks the build rather than a test. The guard cannot derive its list from the registrations, because `include_str!` paths are fixed at compile time, so - this change drops the nine vendors' files from the guard instead: a module + this change drops the nine vendors' files from the guard instead, because a module crate is outside the core neutrality guarantee, and a move then deletes nothing there. - The `ts audit` command carries its own vendor table (detection patterns @@ -326,7 +326,7 @@ that the project pays for today, most recently in PR #1054. ## 8. What implementing this found A probe integration built outside `trusted-server-core` and registered -through an adapter exercised every seam end to end. Four things surfaced +through an adapter exercised every seam end to end. Seven things surfaced that reading the code did not, and they are recorded here rather than left for each vendor to rediscover. @@ -343,7 +343,7 @@ for each vendor to rediscover. crate keeps a literal beside its `include_str!`, and on a checkout that rewrites line endings the embedded file changes and the literal stops matching, which fails startup on that machine only. A generated helper - or a documented build-script recipe removes the trap; the probe pins the + or a documented build-script recipe removes the trap, and the probe pins the file's line endings and tests the literal, which every vendor would otherwise have to reinvent. 3. **A provider is resolved more than once per request.** A proxy that @@ -405,7 +405,7 @@ defines, and both should land before the first vendor is asked to use it. | 3 | A registration may carry its own browser JavaScript | Proposed | | 4 | Deploy validation moves onto the registration | Proposed | | 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | -| 6 | This change is complete in itself: after it, no vendor move needs a core change | Proposed | +| 6 | This change is complete in itself, so after it no vendor move needs a core change | Proposed | | 7 | Identity, geo and device providers are capabilities of a module registration (§3.6), the #1043 review's rule applied to all three | Proposed | | 8 | No provider is built into core: HMAC and the User-Agent-only device provider are Tech Lab-owned modules configured under `[integrations.]`, and core keeps only `none` | Proposed | | 9 | This spec and its core implementation precede #1043; 51Degrees implements the core seam, the nine vendor moves in §4 stay one PR each | Proposed | @@ -419,4 +419,4 @@ defines, and both should land before the first vendor is asked to use it. | 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | | 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | -| 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged; only this status line and this row are new. | +| 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged, and only this status line and this row are new. | diff --git a/docs/superpowers/specs/provider-code-registry.md b/docs/superpowers/specs/provider-code-registry.md index e6771337ee..475237b208 100644 --- a/docs/superpowers/specs/provider-code-registry.md +++ b/docs/superpowers/specs/provider-code-registry.md @@ -2,17 +2,17 @@ Four-character codes (`[a-z0-9]`, zero-padded) that namespace Edge Cookie identifiers. Every EC provider MUST allocate a code here before it can -exist: the `EdgeCookieProvider::code()` trait method is mandatory, and core +exist, because the `EdgeCookieProvider::code()` trait method is mandatory, and core applies the code as the `{code}~` prefix of every identifier the provider creates, checks it at read-back, and keys the identity graph with it. A provider only ever sees its own value part, so identifiers from different providers can never collide in the cookie, the graph, or a withdrawal, and every identifier records which provider created it. -Allocation is a reviewed commit to this file; codes are immutable and never +Allocation is a reviewed commit to this file. Codes are immutable and never recycled, including for retired providers. A leading digit is valid. The tilde separator keeps parsing exact while pre-envelope identifiers remain -deployed: a legacy bare identifier contains no tilde and dual-reads under +deployed, where a legacy bare identifier contains no tilde and dual-reads under the built-in HMAC provider only. The class of provider expected to grow this table is one that consumes a @@ -26,5 +26,4 @@ creating. | `hmac` | Built-in HMAC EC provider. Creates `hmac~<64 hex>.<6 alnum>`; dual-reads its pre-envelope bare form so deployed cookies keep working (retirement condition in `provider_owns_id`) | 2026-08-02 | active | | `hs00` | Built-in host-signal EC provider (opt-in; TLS JA4 plus HTTP/2 signals plus client IP) | 2026-08-25 | active | | `cfix` | Client-fixed demonstration provider (compiled only behind the `client-fixed-demo` cargo feature) | 2026-08-25 | active, test and demo only | -| `51dd` | 51Degrees Identifier (51Did) vendor provider | 2026-08-25 | reserved | | `t0..` | Prefix family reserved for in-tree test providers (`t0cc`, `t0op`, and similar); never valid in configuration | 2026-08-25 | reserved | From 115a88ce5b2e982ed72ffff4cc972cb2b374ca4a Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 10:27:13 +0100 Subject: [PATCH 008/192] Finish the clause-punctuation sweep on the permission model spec --- .../2026-07-30-permission-model-design.md | 30 +++++++++---------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index d0252acf29..531a382438 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -808,19 +808,19 @@ PR #1045). | Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated, and the field returns with that work | | Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted. A detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema. `requires_signal` is now expressible per rule | | Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated, and a failed lookup floors separately | One deployer knob is simpler, and the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | -| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first; the EU/EEA coverage test guards the shipped table against the typo class; ISO-assignment checks are future hardening | -| Three-class signal taxonomy with regime-scoped grant acceptance; the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source; the US posture is a `granted` baseline that opt-outs revoke; explicit non-opt-out values grant nothing | A simpler two-signal model without regimes; the cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | -| Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification; per-family scoping needs the full §4.5 decoder work | -| Normalization runs expiry before conflict resolution, a declared change (§4.4) | Conflict resolution still runs before the expiry check | The reordering was not implemented; the expired state itself (distinct from malformed, absent for acquisition) was adopted | -| Persisted-KV consent flows through the full normalization pipeline with an explicit TTL comparison (§4.4) | The loaded record substitutes directly when the request carries no signals, jurisdiction re-derived; staleness is enforced by the store TTL (`max_consent_age_days`) | The store-level TTL delivers the staleness bound without a second normalization pass | -| Proxy mode gains minimal opt-out extraction (§4.4) | Proxy mode still skips decoding; a present record blocks all grants via the malformed-present rule and the GPC header opt-out is honored without decoding | The permission-layer outcome is equally or more restrictive with no new decode paths; revisit with the §4.5 decoder work | -| Withdrawal has four triggers including an explicit storage-withdrawal or authenticated deletion request (§4.2) | The TCF Purpose 1 refusal under a non-granted baseline is the only trigger; opt-outs, malformed records, absence, and policy changes never withdraw (adopted) | No deletion endpoint exists to carry the extra trigger; the narrowest destructive surface shipped first | -| §4.3 durability protocol: family records first, suppression and authority-state records, outbox, breaker, strong reads | Cookie expiry plus best-effort identity-graph tombstones per presented identifier, with failures logged | The protocol requires storage primitives (linearizable CAS, independent durability domains) the adapters do not yet qualify; deferred with the providers-spec storage work | -| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out; the revoke set is policy-declared, shipped as `all` (which also drops storage) | The full decoder and registry vendoring are their own project; the policy-declared revoke set gives deployers the scoping lever meanwhile | -| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists; the built binary is the policy identity | With no runtime policy there is nothing to activate; the draft remains the reference design for the runtime-config follow-up | -| §3.4 single jurisdiction truth, and §7 dispatch gated on the policy regime with a contextual projection | Auction dispatch keeps the consent-subsystem gate (effective TCF Purpose 1 for GDPR or unknown jurisdictions); `detect_jurisdiction` and its lists remain; no contextual view | Dispatch migration is follow-up; the legacy-list drift risk the draft named still stands and is recorded rather than resolved | -| Every raw-EC egress path is pair-gated, with per-row tests and a denylist check (§7) | Pair gating is centralized in `ec_sharing_allowed` (auction endpoint `user.id`, publisher navigation and page-bids `user.id`, identify, pull sync) and `gate_eids_by_permissions` (EIDs everywhere); batch sync checks row state only | Partial adoption; aligning the remaining paths, the S2S stored-provenance authority, and the inventory tests is recorded follow-up | -| Identity rows never store raw consent strings, only normalized provenance and a digest (§1) | The identity-graph entry stores the raw TCF and GPP strings with the row | The normalized provenance schema belongs to the providers-spec storage work; until then rows carry the raw strings | -| No signals block in policy; the signal mapping is fixed in the spec | New: a `signals` section in `permissions.yaml` declares the TCF purpose map, opt-out sources, and revoke set, with `tcf.authoritative` governing only TCF's own effect | Moves signal policy from code into deployer-editable data; the flag can never let a TCF record override an opt-out, preserving §4 precedence | -| The §5.3 no-geo guard covers every jurisdiction consumer | The guard fires when an Edge Cookie provider is configured with no geo provider | The EC provider is the only policy-gated consumer today; the trigger list grows when dispatch and further egress paths join the model | +| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first, and the EU/EEA coverage test guards the shipped table against the typo class. ISO-assignment checks are future hardening | +| Three-class signal taxonomy with regime-scoped grant acceptance, where the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source, the US posture is a `granted` baseline that opt-outs revoke, and explicit non-opt-out values grant nothing | A simpler two-signal model without regimes. The cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | +| Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification, and per-family scoping needs the full §4.5 decoder work | +| Normalization runs expiry before conflict resolution, a declared change (§4.4) | Conflict resolution still runs before the expiry check | The reordering was not implemented, though the expired state itself (distinct from malformed, absent for acquisition) was adopted | +| Persisted-KV consent flows through the full normalization pipeline with an explicit TTL comparison (§4.4) | The loaded record substitutes directly when the request carries no signals, jurisdiction re-derived, and staleness is enforced by the store TTL (`max_consent_age_days`) | The store-level TTL delivers the staleness bound without a second normalization pass | +| Proxy mode gains minimal opt-out extraction (§4.4) | Proxy mode still skips decoding, so a present record blocks all grants via the malformed-present rule and the GPC header opt-out is honored without decoding | The permission-layer outcome is equally or more restrictive with no new decode paths, and this is revisited with the §4.5 decoder work | +| Withdrawal has four triggers including an explicit storage-withdrawal or authenticated deletion request (§4.2) | The TCF Purpose 1 refusal under a non-granted baseline is the only trigger, and opt-outs, malformed records, absence, and policy changes never withdraw (adopted) | No deletion endpoint exists to carry the extra trigger, so the narrowest destructive surface shipped first | +| §4.3 durability protocol: family records first, suppression and authority-state records, outbox, breaker, strong reads | Cookie expiry plus best-effort identity-graph tombstones per presented identifier, with failures logged | The protocol requires storage primitives (linearizable CAS, independent durability domains) the adapters do not yet qualify, so it is deferred with the providers-spec storage work | +| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out. The revoke set is policy-declared, shipped as `all` (which also drops storage) | The full decoder and registry vendoring are their own project, and the policy-declared revoke set gives deployers the scoping lever meanwhile | +| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists, and the built binary is the policy identity | With no runtime policy there is nothing to activate, and the draft remains the reference design for the runtime-config follow-up | +| §3.4 single jurisdiction truth, and §7 dispatch gated on the policy regime with a contextual projection | Auction dispatch keeps the consent-subsystem gate (effective TCF Purpose 1 for GDPR or unknown jurisdictions). `detect_jurisdiction` and its lists remain, with no contextual view | Dispatch migration is follow-up. The legacy-list drift risk the draft named still stands and is recorded rather than resolved | +| Every raw-EC egress path is pair-gated, with per-row tests and a denylist check (§7) | Pair gating is centralized in `ec_sharing_allowed` (auction endpoint `user.id`, publisher navigation and page-bids `user.id`, identify, pull sync) and `gate_eids_by_permissions` (EIDs everywhere). Batch sync checks row state only | Partial adoption. Aligning the remaining paths, the S2S stored-provenance authority, and the inventory tests is recorded follow-up | +| Identity rows never store raw consent strings, only normalized provenance and a digest (§1) | The identity-graph entry stores the raw TCF and GPP strings with the row | The normalized provenance schema belongs to the providers-spec storage work, and until then rows carry the raw strings | +| No signals block in policy, and the signal mapping is fixed in the spec | New. A `signals` section in `permissions.yaml` declares the TCF purpose map, opt-out sources, and revoke set, with `tcf.authoritative` governing only TCF's own effect | Moves signal policy from code into deployer-editable data, and the flag can never let a TCF record override an opt-out, preserving §4 precedence | +| The §5.3 no-geo guard covers every jurisdiction consumer | The guard fires when an Edge Cookie provider is configured with no geo provider | The EC provider is the only policy-gated consumer today, and the trigger list grows when dispatch and further egress paths join the model | | `default_country` is required only in the acknowledged static no-geo mode (§5.4) | Required always and startup-validated against `permissions.yaml` | It is the baseline for unmatched requests in every mode, so it must always exist | From 580081a480a322d8c109fa294c2ad7b972da097f Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 11:40:40 +0100 Subject: [PATCH 009/192] Finish the clause-punctuation sweep on the response-header-hook spec --- ...integration-response-header-hook-design.md | 256 +++++++++--------- 1 file changed, 128 insertions(+), 128 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index ce1631ce15..53d8869234 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -1,7 +1,7 @@ # Design Spec: Integration Response-Header Hook **Status:** Not implemented in the series. PR #1047 carries the -documentation set only; the hook was removed from the earlier draft of that +documentation set only. The hook was removed from the earlier draft of that PR because it has no consumer, which is this spec's own §-rule for speculative surface. The spec is retained as the design bar for the hook when its first consumer arrives (an integration that must set response @@ -32,7 +32,7 @@ governed by this spec, §4a is the normative PR-specific delta and supersedes that baseline wherever its generic request/effect API, header ordering, session-by-header behavior, endpoint/request surface, cookie lifecycle, challenge transport, or response-pointer behavior conflicts. The baseline -continues to provide historical context; it is not a second normative source +continues to provide historical context. It is not a second normative source for those surfaces. Integrations can today rewrite request-path behavior (proxies, attribute @@ -61,18 +61,18 @@ processed. which **core validates and applies**, attributing each to its integration id. PR #838's shape handed the integration an unrestricted `&mut HeaderMap`, which makes §3's collision - policy unenforceable by construction: core cannot validate or attribute + policy unenforceable by construction, because core cannot validate or attribute writes it never sees. An API that cannot express a violation beats one that promises to catch it. - **Every adapter calls the apply point** on its outbound-response path for processed documents. The call site lives in shared response-finalization - code where one exists; where adapters finalize independently, each adapter + code where one exists. Where adapters finalize independently, each adapter gains the call and a test proving it. - **Ordering is three stages, and the last one is inviolable:** core response-header handling (EC Set-Cookie emission, EC header clearing, privacy headers) → integration operations → **final cache/privacy invariant enforcement**, which no integration operation can override. - Running the hook dead-last would be wrong: current `main` deliberately + Running the hook dead-last would be wrong, because current `main` deliberately runs cookie-cache protection _after_ arbitrary header changes, stripping surrogate caching and forcing private/no-store on any response that sets a cookie, a hook applied after that recheck could combine an appended @@ -82,7 +82,7 @@ processed. cookie rule. Core **snapshots the complete pre-hook cache restriction state**, whether the restriction came from core's own classification (processed auction HTML is marked private even when no cookie is - emitted; today's final helper returns early without `Set-Cookie`) **or + emitted, and today's final helper returns early without `Set-Cookie`) **or from the origin** (an origin-supplied `private, no-store` that core merely passed through), and the post-hook response may only be **equal or stronger** on the privacy axis: integrations can tighten @@ -92,7 +92,7 @@ processed. are orthogonal constraints (RFC 9111: `no-cache` permits shared storage subject to revalidation; `private` forbids shared storage), so "replace `private` with the stronger `no-cache`" would make a - personalized response shared-storable. The merge: each of the **six sticky directives**, `no-store`, `no-cache`, + personalized response shared-storable. Under the merge, each of the **six sticky directives**, `no-store`, `no-cache`, `private`, `must-revalidate`, `proxy-revalidate`, `no-transform`, is present-in-snapshot-or-mutation ⇒ present-in-final (this "snapshot or mutation ⇒ final" rule scopes to exactly these six), with two refinements. First, `must-understand` is the deliberate @@ -127,14 +127,14 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may a matching rule four adapters would implement identically). Merging them per-directive on unrestricted responses was a hole (an unrestricted `CDN-Cache-Control: max-age=60` could become a year), and - they are additionally stripped from any restricted response; and the + they are additionally stripped from any restricted response, and the final `Vary` is the **union of the complete snapshot `Vary` set**, origin-supplied members included, not only core-required ones, and the mutation. Ordering is normative so the final `Vary` reaches TS's own cache key, not just the wire: **mutation/invariant → final `Vary` computation → cache-key construction → body/metadata commit**, with three identity rules. Cache matching uses the **exact final publisher - request** (post-overlay view; keying from the redacted view would + request** (post-overlay view, because keying from the redacted view would collapse personalized variants). The final `Vary` name list has one cross-adapter grammar. Core collects every @@ -145,7 +145,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may empty member or invalid token introduced by a mutation invalidates that batch. If the reverted snapshot itself is malformed, the invariant replaces the final value with `Vary: *`, forces `no-store`, and writes no cache - artifact. `*` in either source likewise dominates every other member: the + artifact. `*` in either source likewise dominates every other member, so the normalized result is the single `*` and is uncacheable. For an ordinary list, the wire response emits one lowercase `, `-joined value, while the variant descriptor stores `vary_names` as the exact sorted JSON array of @@ -154,11 +154,11 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may A normalized name nominates one request header. Digest construction obtains **all** values for that header from the exact final publisher request after - overlay, preserving received field-line order and value octets; it does not + overlay, preserving received field-line order and value octets. It does not comma-fold, trim, or split those request values. The `` bytes in the HMAC input below are always the normalized lowercase ASCII name, and a name - is digested once even if it appeared repeatedly in `Vary`. Known-answer - normalization fixture: snapshot/mutation lines `Vary: X-Tenant , + is digested once even if it appeared repeatedly in `Vary`. In the known-answer + normalization fixture, snapshot/mutation lines `Vary: X-Tenant , Accept-Encoding` and `Vary: accept-encoding` produce wire value `accept-encoding, x-tenant` and descriptor value `["accept-encoding", "x-tenant"]`, then digest each nominated request @@ -187,10 +187,10 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may [{ "id": "", "key_base64url": "" }] }`. `keys` is sorted by `id`, contains 1..4 unique entries, rejects unknown fields, and every value decodes to exactly 32 CSPRNG bytes. An id is derived, - not operator-invented: the first 16 lowercase hex characters of - SHA-256 over the raw key bytes; a supplied mismatch or one id bound to + not operator-invented, and is the first 16 lowercase hex characters of + SHA-256 over the raw key bytes. A supplied mismatch or one id bound to different bytes is fatal. The current id must exist in the array. - Every cache entry stores its id; raw keys never enter config, cache + Every cache entry stores its id. Raw keys never enter config, cache artifacts, logs, or metrics. Startup **fails** when response caching is enabled and the keyring/current key does not resolve (digests are never computed unkeyed). @@ -207,22 +207,22 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may still-unknown ID, malformed descriptor, missing artifact, expiry, or revision mismatch is a miss for that descriptor, never a comparison under another key. Multiple matching descriptors are corruption and make the entire base - lookup a miss with a metric; index order never chooses a winner. + lookup a miss with a metric. Index order never chooses a winner. Publication writes and verifies the immutable artifact first, then CAS-adds or replaces its complete descriptor in the index. Rekeying or a changed `Vary` set inserts the new artifact/descriptor before removing the old - descriptor; a crash may leave a safely unreachable artifact or two + descriptor. A crash may leave a safely unreachable artifact or two nonmatching descriptors but cannot point to a partial artifact. Index capacity eviction removes expired descriptors first and otherwise the - least-recently-used complete descriptor; it never rewrites a digest under a + least-recently-used complete descriptor. It never rewrites a digest under a new key ID. This is the one meaning of “insert-new-then-index-update rekey” in the capability matrix and 304 rules. Rotation atomically replaces the secret entry with a new valid keyring containing the new current key **and all still-live previous keys**. Fleet - propagation may be mixed only in the safe direction: a process with the old - keyring can write/read the old id; the variant index exposes that id before + propagation may be mixed only in the safe direction, where a process with the old + keyring can write/read the old id. The variant index exposes that id before lookup, so a process seeing an unknown descriptor id refreshes the keyring once, then treats a still-unknown descriptor as a cache miss. It never guesses, probes with the current key, or computes unkeyed. A previous key may @@ -250,13 +250,13 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may are independent axes, so no single most-restrictive point exists): the response is treated as **uncacheable for the storage decision** (`no-store`-equivalent in the invariant) and any mutation batch - merging against the malformed value is rejected whole; among - well-formed values, duplicate directives keep the strongest; quoted and unquoted - forms are equivalent; conflicting `max-age` values keep the smallest; + merging against the malformed value is rejected whole. Among + well-formed values, duplicate directives keep the strongest, quoted and unquoted + forms are equivalent, conflicting `max-age` values keep the smallest, and unknown extension directives are dropped **from mutations only, while unknown directives already in the snapshot are preserved verbatim** (a downstream cache may honor a restrictive extension TS does not - recognize; dropping it would weaken origin policy, RFC 9111 §5.2.3); + recognize, so dropping it would weaken origin policy, RFC 9111 §5.2.3); `Expires` participates in the freshness bound via **RFC 9111 §4.2.1's freshness-lifetime algorithm, referenced directly**: the `Expires`-derived lifetime is `Expires − Date` (absent `Date` → @@ -270,7 +270,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may origin's shorter or already-expired `Expires`; and `Vary: *` is treated as uncacheable-by-shared-caches (no-store-equivalent for the invariant). Conformance fixtures cover each rule. Middle-stage placement also keeps - the earlier property: an integration mutation is not silently stripped + the earlier property. An integration mutation is not silently stripped by ordinary core handling, only by the invariant pass, which logs the downgrade it applies. @@ -294,14 +294,14 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may consent/privacy headers core emits; (b) reserved cookie _names_ within `Set-Cookie`, `ts-ec`, `ts-eids`, and the other `ts-*` cookies core owns. **Cookie operations are deferred out of the v1 hook, headers only.** The write-side gate alone ("persistent cookies require P1") was shown - insufficient: it never modeled reading, using, forwarding, or + insufficient, because it never modeled reading, using, forwarding, or withdrawing the cookie, so a P1-granted-then-withdrawn integration cookie would keep arriving on every request with nothing required to expire, hide, or stop egressing it, and an advertising-identifier cookie needs P4 the contract never expressed. Rather than ship "inside the permission model" as a claim the model does not back, `append_set_cookie` and the typed cookie builder are **deferred** to a - follow-up spec whose entry bar is: declared per-cookie required + follow-up spec whose entry bar comprises declared per-cookie required permissions, a typed authorized request-side view, stripping from unauthorized integration/proxy inputs, mandatory expiry on destructive P1 withdrawal, and startup-unique (name, domain, path) ownership. @@ -309,14 +309,14 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may construction rejects a duplicate ID (current code silently coalesces, which corrupts attribution and budgets), with a duplicate-ID test in the done-when. The registration's `behavior_revision` follows §2's bump - contract; configuration-dependent behavior is captured separately by the + contract. Configuration-dependent behavior is captured separately by the effective-config digest. Model-only activation is separate from both, so the **one cache revision tuple** contains exactly `integration_registry_revision`, `effective_config_revision`, `active_policy_digest`, `active_policy_ordinal`, `model_epoch`, `activation_generation`, and `hook_invariant_revision` from the strong active tuple at publication. Every processed artifact, mutation IR/read-set bundle, - variant descriptor, and variant-index update stores that complete tuple; a + variant descriptor, and variant-index update stores that complete tuple. A lookup, local conditional, HEAD update, or 304 replay requires byte-for-byte equality with current active. In particular, the `permissions_v2` model CAS misses every `pre_epic_v1` artifact even though config/policy bytes did not @@ -331,7 +331,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may constants next to the definitions they protect, not duplicated in the hook. - For non-reserved headers, the mutator API distinguishes **append** from - **replace** explicitly; append/replace legality comes from a **core-owned field registry**, + **replace** explicitly. Append/replace legality comes from a **core-owned field registry**, not adapter judgment, and the v1 registry admits **inert fields only**: "headers-only" is not automatically permission-neutral, since `Link` (preload/prefetch), `Reporting-Endpoints`/NEL, CSP report @@ -349,7 +349,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may merge), `Content-Language` (append), `X-Robots-Tag` (append), `Retry-After` (replace-only), `Content-Location` (replace-only); everything else known is classified reserved or rejected by the rules - above, and growing the admitted set is a spec change to this list (cookies: see the §3 deferral above). Replacing a + above, and growing the admitted set is a spec change to this list (for cookies, see the §3 deferral above). Replacing a header the origin set is a deliberate act, visible in the mutator's code. - Later registrations see earlier mutations (order = registration order, which is deterministic). @@ -362,37 +362,37 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may a **cumulative final-response budget** (≤ 128 headers / ≤ 32 KiB total, counting `name: value` plus separators, within any lower adapter ceiling, and those ceilings are the enumerated capability cells **below**, - with the counting rule fixed exactly: counted bytes = Σ over emitted + with the counting rule fixed exactly, where counted bytes = Σ over emitted fields of `len(name) + 2 + len(value) + 2` (the `": "` and CRLF separators), validated against core's budget at startup so a batch - that passes core can never fail only on one adapter; a snapshot + that passes core can never fail only on one adapter. A snapshot already **over** the core budget before any mutation rejects every ordinary batch, the budget bounds additions and never bricks an over-budget origin response, which passes through and is counted; security follows the replacement/reserve rule below) bounds the sum across integrations. The budget has normative priority - partitions: ordinary mutators may consume at most **112 headers / 24 KiB**, + partitions, where ordinary mutators may consume at most **112 headers / 24 KiB**, reserving 16 headers / 8 KiB for the core-owned security channel. Ordinary batches remain registration-ordered within their partition. A security `Continue` batch uses the reserve and, if necessary, evicts whole accepted - ordinary batches in reverse registration order until it fits; it never + ordinary batches in reverse registration order until it fits. It never removes half a batch and never drops origin fields. Ordinary output can therefore never crowd out security effects. If the immutable origin head plus the security batch alone exceeds the full budget, the security batch is rejected atomically and the request follows the documented security fail-open path with a dedicated metric, origin fields are not silently sacrificed. A security `Respond` owns a replacement - response: all ordinary mutation batches are discarded and the challenge + response. All ordinary mutation batches are discarded and the challenge is validated against the full 128-header / 32-KiB budget. A base publisher response already over the full budget still passes unchanged, but no - ordinary mutation applies; security `Continue` applies only if the final + ordinary mutation applies. Security `Continue` applies only if the final response fits after all ordinary batches are removed. These are separate outcomes and metrics. | Adapter | Header-count / total-bytes ceiling (capability cell) | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | | Axum | no platform ceiling below the core budget (native HTTP stack), cell fixed at ≥ 128 headers / ≥ 32 KiB | - | Fastly | **qualification-pending**: the measured platform ceiling is recorded in this cell by the adapter-qualification commit; unrecorded ⇒ hook startup fails | + | Fastly | **qualification-pending**: the measured platform ceiling is recorded in this cell by the adapter-qualification commit, and unrecorded ⇒ hook startup fails | | Cloudflare | **qualification-pending**: same rule | | Spin | **qualification-pending**: same rule | @@ -400,31 +400,31 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may error (shrink the core budget or raise the ceiling, never a silent per-adapter divergence). - Hook/cache eligibility has the following concrete adapter cells; any + Hook/cache eligibility has the following concrete adapter cells, and any `qualification-pending` cell fails startup when the depending feature is selected: | Capability | Fastly | Axum (dev) | Cloudflare | Spin | | -------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------- | ------------------------------ | -------------------------------- | - | Runtime secret lookup for Vary-HMAC/DataDome | wired secret store; qualify key-rotation behavior | dev secret binding required | qualify Workers secret binding | qualify component secret binding | - | Persisted processed artifact + mutation IR/read sets | qualification-pending | in-process dev implementation required; non-durable | qualification-pending | qualification-pending | + | Runtime secret lookup for Vary-HMAC/DataDome | wired secret store, and qualify key-rotation behavior | dev secret binding required | qualify Workers secret binding | qualify component secret binding | + | Persisted processed artifact + mutation IR/read sets | qualification-pending | in-process dev implementation required, non-durable | qualification-pending | qualification-pending | | Atomic artifact/metadata entry commit | qualification-pending | implementation required | qualification-pending | qualification-pending | | `Vary` variant index + insert-new-then-index-update rekey | qualification-pending | implementation required | qualification-pending | qualification-pending | | DataDome field-line order, trusted IP/port, fixed HTTPS backend/no-redirect, and exact form limits | qualification-pending | qualification-pending | qualification-pending | qualification-pending | - | SecurityUse JA4 request evidence | platform value available; exact-field/payload qualification and sign-offs 23/28 pending | unavailable | unavailable | unavailable | + | SecurityUse JA4 request evidence | platform value available, with exact-field/payload qualification and sign-offs 23/28 pending | unavailable | unavailable | unavailable | The qualification commit records storage lifetime, maximum object size, concurrency semantics, torn-write behavior, and fault-injection evidence; “platform has KV” is not a qualifying cell. `expose_host_fingerprints_to_vendor = true` also requires a qualified - SecurityUse JA4 cell; unsupported or pending is a startup error, while the + SecurityUse JA4 cell. Unsupported or pending is a startup error, while the default `false` remains portable. Each mutator receives an **immutable, redacted snapshot of the response head** (status and headers as of its turn, prior integrations' - accepted operations applied) as its read context; it never holds a + accepted operations applied) as its read context. It never holds a mutable reference (§2). Redaction is a security boundary, not - tidiness: the hook runs after core queues the EC `Set-Cookie`, so an + tidiness, because the hook runs after core queues the EC `Set-Cookie`, so an unredacted view would hand a mutator the raw EC to copy into `X-Vendor-Identity` or its own cookie, walking around `AuthorizedIdentity` entirely. The snapshot therefore @@ -434,7 +434,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may Each registration also declares the complete set of response fields its decision may read, including status as a distinguished input. Core records the union with the accepted operation batch. Undeclared reads are a hard - conformance failure in tests; an integration unable to declare a complete + conformance failure in tests. An integration unable to declare a complete read set marks itself `revalidation = "refetch"`, which forbids IR replay after any origin metadata change. Operations arrive as **attributed batches bound to a registration @@ -443,7 +443,7 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may ordinary response mutators, one global order, core finalization → ordinary mutators → security effects → invariant pass, so the security layer's precedence over publisher-facing mutations holds through - both position and its reserved/response-owning budget rule; the current flat effects vector satisfies neither + both position and its reserved/response-owning budget rule. The current flat effects vector satisfies neither attribution nor budgets and will be restructured accordingly. Validation and budgeting are **atomic per batch**: a batch that exceeds its budget is rejected whole (logged, attributed), never partially @@ -464,15 +464,15 @@ Which responses the hook runs on, enumerated so two implementations cannot diverge silently: In this table, **persisted post-hook finals** means the cache-safe ordinary -artifact only: origin metadata plus accepted ordinary mutation IR and the +artifact only, comprising origin metadata plus accepted ordinary mutation IR and the cache/privacy invariant result, with `Set-Cookie`, core request-specific identity fields, security-channel effects, and origin validators excluded. -The security request filter evaluates every request before cache selection; a +The security request filter evaluates every request before cache selection. A fresh `Respond` bypasses the artifact, while a fresh `Continue` batch is applied to the persisted ordinary artifact and the invariant pass reruns before emission. This applies equally to ordinary hits, local conditionals, origin-revalidation 304s, and HEAD. Therefore "`Set-Cookie` is never replayed" -means never replayed from storage; a freshly validated per-request typed cookie +means never replayed from storage. A freshly validated per-request typed cookie operation may still emit on that response. Security `Respond` outputs are always `private, no-store` and never become artifacts. @@ -497,11 +497,11 @@ fixture, never a runtime wildcard. | Processed HTML document (rewritten by TS) | Yes | | Streamed processed document | Yes, operations apply to the header block before first byte | | Pass-through proxy response (not processed) | No, TS is a transparent proxy for it | -| Served-from-cache processed document | Serves the **persisted post-hook finals** captured at cache fill (revision-versioned; mismatch = miss), mutators run at fill/processing time only, so a normal hit and a conditional hit return identical policy metadata **by construction**, with no purity or determinism assumption about mutators | +| Served-from-cache processed document | Serves the **persisted post-hook finals** captured at cache fill (revision-versioned, mismatch = miss), mutators run at fill/processing time only, so a normal hit and a conditional hit return identical policy metadata **by construction**, with no purity or determinism assumption about mutators | | Redirect (3xx) | No | | Error responses TS itself generates (4xx/5xx) | No | -| `304 Not Modified` for a processed representation | **Persisted-metadata pass**: a cached processed 200 stores its final post-hook headers, accepted mutation-operation batches, and the union of every mutator's declared response-field read set (the persisted mutation IR), versioned by §3's complete cache revision tuple, including model epoch and logical activation generation. A local conditional hit re-emits persisted finals only when the complete tuple matches current active. An origin-revalidation 304 is staged and diffed against separately stored origin-side metadata. (a) Any byte-coupled field changed (`Content-Encoding`, `Content-Type`, validators, digests) → invalidate and fetch/process a full 200. (b) A changed metadata field that intersects any persisted mutator read set, or an artifact/mutator lacking a complete read-set declaration, is also unsafe → full 200 refetch and ordinary hook execution; deterministic replay of old operations cannot stand in for re-evaluating a decision made from changed inputs. (c) If every changed field is outside every declared read set and belongs to the enumerated safe-update set (`Cache-Control`, reserved CDN cache fields, `Expires`, `Date`, `Age`, `Vary`, registry-admitted mutable fields), replay the persisted deterministic operations over updated origin metadata and rerun invariants. Updated origin metadata, finals, IR, read sets, and complete revision tuple publish in one atomic entry commit; changed `Vary` uses insert-new-entry-then-index-update ordering. (d) No change → re-emit persisted finals. Artifact absence or any tuple mismatch triggers an unconditional recovery fetch so TS obtains bytes. For processed-document GET/HEAD routes, TS is explicitly the authoritative server for the transformed representation: after processing the full 200 it evaluates RFC 9110 §13 preconditions against **processed** validators; this is not evaluation of origin validators by an intermediary cache. Other methods are never eligible for this recovery path. `Set-Cookie` and origin validators are never replayed. Absent metadata → cache miss | -| `HEAD` of a processed document | **Serves the persisted GET artifact when one exists**, parity with GET/304 by construction, since RFC 9111 §4.3.5 lets HEAD metadata update a stored GET response and mutators are not required to be deterministic; with no persisted artifact, HEAD is processed like a GET (headers only) and its finals stored as a **distinct head-only artifact type that never satisfies a later GET**, when a stored GET artifact exists a HEAD may **update** it only when the comparison, made against the separately stored **origin-side** metadata, never the processed artifact's (origin and rewritten lengths differ by construction, so the processed length is never the comparand and HEAD metadata never touches processed-side headers), finds validators and origin `Content-Length` matching and no byte-coupled representation field changed (RFC 9111 §4.3.5); qualifying updates land origin-side under the same atomic-commit discipline as the 304 rules, and any mismatch **invalidates** the stored GET artifact rather than updating it | +| `304 Not Modified` for a processed representation | **Persisted-metadata pass**: a cached processed 200 stores its final post-hook headers, accepted mutation-operation batches, and the union of every mutator's declared response-field read set (the persisted mutation IR), versioned by §3's complete cache revision tuple, including model epoch and logical activation generation. A local conditional hit re-emits persisted finals only when the complete tuple matches current active. An origin-revalidation 304 is staged and diffed against separately stored origin-side metadata. (a) Any byte-coupled field changed (`Content-Encoding`, `Content-Type`, validators, digests) → invalidate and fetch/process a full 200. (b) A changed metadata field that intersects any persisted mutator read set, or an artifact/mutator lacking a complete read-set declaration, is also unsafe → full 200 refetch and ordinary hook execution, because deterministic replay of old operations cannot stand in for re-evaluating a decision made from changed inputs. (c) If every changed field is outside every declared read set and belongs to the enumerated safe-update set (`Cache-Control`, reserved CDN cache fields, `Expires`, `Date`, `Age`, `Vary`, registry-admitted mutable fields), replay the persisted deterministic operations over updated origin metadata and rerun invariants. Updated origin metadata, finals, IR, read sets, and complete revision tuple publish in one atomic entry commit, and changed `Vary` uses insert-new-entry-then-index-update ordering. (d) No change → re-emit persisted finals. Artifact absence or any tuple mismatch triggers an unconditional recovery fetch so TS obtains bytes. For processed-document GET/HEAD routes, TS is explicitly the authoritative server for the transformed representation, and after processing the full 200 it evaluates RFC 9110 §13 preconditions against **processed** validators. This is not evaluation of origin validators by an intermediary cache. Other methods are never eligible for this recovery path. `Set-Cookie` and origin validators are never replayed. Absent metadata → cache miss | +| `HEAD` of a processed document | **Serves the persisted GET artifact when one exists**, parity with GET/304 by construction, since RFC 9111 §4.3.5 lets HEAD metadata update a stored GET response and mutators are not required to be deterministic. With no persisted artifact, HEAD is processed like a GET (headers only) and its finals stored as a **distinct head-only artifact type that never satisfies a later GET**, when a stored GET artifact exists a HEAD may **update** it only when the comparison, made against the separately stored **origin-side** metadata, never the processed artifact's (origin and rewritten lengths differ by construction, so the processed length is never the comparand and HEAD metadata never touches processed-side headers), finds validators and origin `Content-Length` matching and no byte-coupled representation field changed (RFC 9111 §4.3.5); qualifying updates land origin-side under the same atomic-commit discipline as the 304 rules, and any mismatch **invalidates** the stored GET artifact rather than updating it | | Informational `1xx`, `204`, `205`, `206` | No, enumerated so adapters do not infer independently | This deliberately narrows #782's general "outbound response" phrasing to @@ -515,19 +515,19 @@ The security channel (today: DataDome) runs under a distinct, typed it never authorizes TS-controlled advertising identity, graph linkage, partner egress, other integrations, or general raw-value observability. Request-scoped raw security evidence may be disclosed only to the fixed DataDome Protection API -endpoint and only from that integration's explicit field allowlist; it is not +endpoint and only from that integration's explicit field allowlist. It is not persisted in the identity graph, exposed to publisher origin or other integrations, or emitted in logs. It carries its own configured retention and deletion path. An advertising opt-out does not erase a strictly security-scoped identifier, while an authenticated deletion request or expiry under the security retention policy does. This is not a general exception. The path-only `Request` remains publisher-originated data and is -explicitly covered by vendor retention/DSR sign-off; query strings and full +explicitly covered by vendor retention/DSR sign-off. Query strings and full referrers are never in the security view. Every degree of freedom is closed: - **Host evidence is not a back door to the device provider.** `[device] provider = "fastly"` is an explicit opt-in selection (#1044) and - the hook does not widen it: no JA4-derived classification is stored by a + the hook does not widen it, and no JA4-derived classification is stored by a mutator. If DataDome's Protection API is allowed to receive request-scoped `TlsProtocol`/`JA4` evidence, its registration enumerates each field, proves vendor necessity and payload bounds, and keeps it ephemeral under @@ -539,10 +539,10 @@ referrers are never in the security view. Every degree of freedom is closed: server-side DataDome identifier mapping. On an authenticated TS deletion request it excludes the route from vendor validation, emits a typed `datadome` cookie deletion, and sends no ClientID to DataDome for that - request; re-presentation retries deletion. A lost browser response can leave + request, and re-presentation retries deletion. A lost browser response can leave the cookie until its configured `security_cookie_max_age`, which is the bounded residual sign-off 23 accepts. This operation does **not** claim to - erase data already held by DataDome: vendor-side retention and data-subject + erase data already held by DataDome, because vendor-side retention and data-subject deletion require a named contractual/API procedure in the decision record. If no such vendor procedure exists, operator documentation says so and may not describe TS cookie deletion as vendor-data deletion. @@ -552,10 +552,10 @@ referrers are never in the security view. Every degree of freedom is closed: operation, and the registration is not a placeholder, for DataDome it pins, **aligned to documented vendor behavior where hardening was not intended**: cookie name exactly `datadome`; one configured ownership - tuple for every set and deletion: path exactly `/` and + tuple for every set and deletion, with path exactly `/` and `security_cookie_domain = "host-only"` (default) or one explicit normalized ASCII domain. Host-only mode requires the vendor `Domain` attribute to be - absent; explicit-domain mode requires it to equal the configured domain + absent. Explicit-domain mode requires it to equal the configured domain exactly, TS never accepts a different domain and never rewrites one scope into another. The explicit value cannot exceed the registrable domain, computed against the **vendored @@ -569,11 +569,11 @@ referrers are never in the security view. Every degree of freedom is closed: are separate requirements) and a vendor cookie using `Expires` is normalized to its Max-Age equivalent (both present → `Max-Age` wins, per RFC 10025); a normalized lifetime exceeding the ceiling rejects - the whole operation batch; and the parser is total: repeated `Cookie` + the whole operation batch. The parser is total, where repeated `Cookie` request fields are joined with `"; "` (semicolon-space, the order-preserving join of RFC 10025) before parsing, and **duplicate `datadome` pairs after the join make the request-side - identity ambiguous: treated as cookie-absent for the vendor call and + identity ambiguous, so they are treated as cookie-absent for the vendor call and counted**, while cookies under other names pass through untouched; `Set-Cookie` fields are **never combined**, and a vendor response carrying more than one `datadome` `Set-Cookie` field, a `Set-Cookie` @@ -584,7 +584,7 @@ referrers are never in the security view. Every degree of freedom is closed: each reject the operation batch (the vendor cookie is well-formed; strictness is safe). `Expires` normalizes as `Max-Age = max(0, floor(expires − now))` whole seconds on the server - wall clock at parse time (the shared skew-bounded basis; a result of + wall clock at parse time (the shared skew-bounded basis, where a result of 0 is a deletion), and the 512-byte limit measures the **normalized** serialized `name=value` plus attributes in bytes; `Max-Age` is capped by required operator configuration @@ -595,18 +595,18 @@ referrers are never in the security view. Every degree of freedom is closed: Where the contract **is** deliberately narrower than the vendor, the spec-pinned pointer allowlist starting at ClientID-only against DataDome's mandatory response-directed mapping set, that reduction - needs explicit product **and vendor** acceptance: sign-off item 28; - a violating operation is rejected whole (the batch rule). **Both + needs explicit product **and vendor** acceptance under sign-off item 28. A + violating operation is rejected whole (the batch rule). **Both sessionByHeader is startup-rejected in v1, one state, not three**: TS never sends `X-DataDome-X-Set-Cookie: true`; a vendor `X-Set-Cookie` **invalidates the batch → Continue** (its matrix cell, since a session mode the fleet never requested must not half-apply); and an incoming browser `X-DataDome-ClientID` is **not forwarded to the vendor** (cookie-only session identity, an earlier revision translated - `X-Set-Cookie` into an ordinary cookie, which is not equivalent: + `X-Set-Cookie` into an ordinary cookie, which is not equivalent, because header-session clients expect JavaScript to receive `X-Set-Cookie` and `X-DD-B`, and the higher-priority header session would never see - a cookie update; the older DataDome spec's "always send + a cookie update. The older DataDome spec's "always send X-DataDome-X-Set-Cookie when the header ID is used" is superseded for v1 by this section). Supporting header mode later means the full vendor protocol, typed owner-scoped `X-Set-Cookie`/`X-DD-B` forwarding, @@ -617,7 +617,7 @@ referrers are never in the security view. Every degree of freedom is closed: cookie to be readable by its JavaScript and warns against `HttpOnly`, so **every same-origin page script can observe it**, and a Respond serves vendor-owned HTML under the publisher origin (vendor scripts - with same-origin access to cookies, storage, and APIs; publisher CSP + with same-origin access to cookies, storage, and APIs, while publisher CSP can conversely break the challenge). Those browser-side observers, same-origin vendor code, CSP interaction, and challenge redirects enter **sign-offs 23/28** for ratification (both decision records @@ -627,28 +627,28 @@ referrers are never in the security view. Every degree of freedom is closed: other integrations' request views, publisher-origin proxy forwarding, proxy/click/Testlight upstreams, auction/page-bids request serialization, and logs (redaction list), each surface a tested row - of the inventory; only the security channel itself observes it; vendor - egress goes only to the fixed DataDome Protection API authority and path; - redirects are not followed; deletion is always possible - through the `SecurityUse` lifecycle; and advertising withdrawal never + of the inventory. Only the security channel itself observes it. Vendor + egress goes only to the fixed DataDome Protection API authority and path. + Redirects are not followed. Deletion is always possible + through the `SecurityUse` lifecycle, and advertising withdrawal never grants access to or reuses the identifier. No other request filter inherits the cookie capability. - **Cookie ownership makes deletion total for the scope TS creates.** While DataDome is enabled, `datadome` is a security-owned name across the final - response: before the security batch applies, core removes and meters every - origin, core, cached, or ordinary-mutator `Set-Cookie` for that name; - unrelated cookie names remain separate field lines. Only the typed security + response, and before the security batch applies, core removes and meters every + origin, core, cached, or ordinary-mutator `Set-Cookie` for that name. + Unrelated cookie names remain separate field lines. Only the typed security operation may emit it. Authenticated deletion emits the same configured `(name, domain mode/domain, path)` tuple with `Max-Age=0`; it does not guess a scope from the request cookie, whose wire form carries no Domain or Path. Candidate validation rejects a change of domain mode/domain while the previous active DataDome configuration can still have a live cookie. The supported migration is disable + wait at least the previous - `security_cookie_max_age` + activate the new scope; a faster scope change + `security_cookie_max_age` + activate the new scope. A faster scope change requires a separate bounded deletion-fan-out design. The permission spec §5.5 whole-settings serve fence applies before cookie processing. A bounded old-generation admission validation may survive only during the pre-promotion - drain; the register's promotion-not-before plus member quiescence proves it + drain. The register's promotion-not-before plus member quiescence proves it and every admitted effect ended before the activation CAS. After that CAS, no instance may emit, refresh, or delete a `datadome` cookie until it has loaded and leased the exact new active tuple. A stale instance stops at serve @@ -659,11 +659,11 @@ referrers are never in the security view. Every degree of freedom is closed: - **The incoming `X-DataDome-ClientID` request header is owner-only, like the cookie.** DataDome prioritizes the header over the cookie, so leaving it in the shared request would hand other integrations and - upstream routing the same identifier the cookie boundary strips: core + upstream routing the same identifier the cookie boundary strips, so core **removes it from the shared request** before integrations and upstream routing run, it joins `RedactedRequestView`'s enumerated strip set (providers spec), **and in v1 it is stripped for the - vendor too: the Protection API request's ClientID derives only from + vendor too, because the Protection API request's ClientID derives only from the `datadome` cookie, never from the incoming header** (DataDome's contract requires `X-DataDome-X-Set-Cookie: true` whenever a header-supplied ClientID is forwarded, so forwarding the header under @@ -674,27 +674,27 @@ referrers are never in the security view. Every degree of freedom is closed: sent. Only DataDome-returned overlay data reaches the publisher, never the raw browser-supplied header. - **The pointer protocol has a total parser contract**, adapters - cannot differ where malformed batches fail open: the pointer list is + cannot differ where malformed batches fail open, where the pointer list is tokenized by the vendor's documented space separation, repeated pointer header fields are concatenated with a single SP before tokenizing, tokens split on runs of SP/HTAB, empty tokens ignored, - then names are ASCII-lowercased before duplicate detection; duplicate + then names are ASCII-lowercased before duplicate detection. Duplicate names after normalization, invalid names, more than 16 pointers, or more than 4 KiB of pointer payload render the batch invalid (→ Continue, the vendor's fail-open). **Pointed-field multiplicity is closed**: for singleton fields (`Location`, `Content-Type`, `X-DataDome`, `X-DD-B`, `X-Set-Cookie`) more than one instance in the vendor response invalidates the batch atomically, never a - first/last/join choice an adapter makes; list-valued fields + first/last/join choice an adapter makes. List-valued fields (`Cache-Control`, `Pragma`) are joined per RFC 9110 §5.3 before their - matrix outcome applies; `Set-Cookie` multiplicity follows the typed + matrix outcome applies. `Set-Cookie` multiplicity follows the typed cookie rule (exactly one `datadome` field, above). No both-source priority rule exists in v1: the header session form (`X-Set-Cookie`) is matrix-governed as batch-invalid, so "header form wins" is unreachable and deleted. - **Request-header pointers are a positive, enumerated allowlist with no default publisher-origin identifier exposure.** - "Documented enrichment headers" is not enforceable; the registration + "Documented enrichment headers" is not enforceable. The registration enumerates the exact names from the **inline DataDome field contract in §4a.2**, spec-pinned today to exactly **`X-DataDome-ClientID`**, admitted only when the @@ -707,12 +707,12 @@ referrers are never in the security view. Every degree of freedom is closed: resolving what was a contradiction. When the opt-in is false, the vendor-returned ClientID is discarded and the publisher origin is not an identifier observer. When true, it applies only to an owner-scoped - publisher-upstream overlay, never the shared request; startup logs the + publisher-upstream overlay, never the shared request. Startup logs the additional consumer, operator documentation must disclose its purpose and retention, and a fixture proves no other surface can read it. Everything else, authentication, `Cookie`, `Forwarded`/`X-Forwarded-*`, other identity, consent, and - routing-authority fields, is rejected by name and by class: a + routing-authority fields, is rejected by name and by class, because a compromised endpoint must not replace origin credentials, inject `ts-ec`, or spoof client location. - **Browser-response headers are decision-scoped through the single @@ -729,7 +729,7 @@ referrers are never in the security view. Every degree of freedom is closed: decision (challenge/deny) owns its body but may describe it with **`Content-Type` only**, encoding and validator fields (`Content-Encoding`, `ETag`, `Last-Modified`, digests) stay reserved - even for Respond: challenge bodies are simple and uncacheable, the + even for Respond, because challenge bodies are simple and uncacheable, the allowlist does not admit those fields, and ambiguity here decides whether a challenge enforces or silently fails open (batch rejection → Continue). If the vendor ever requires more, it arrives as a reviewed @@ -740,10 +740,10 @@ referrers are never in the security view. Every degree of freedom is closed: deadline of 3000 ms on the instance's monotonic clock, measured from immediately before vendor-backend acquisition/dispatch to the final body byte**, meaning connection setup and request send are inside the - window; the 1500 ms first-byte bound (the older spec's figure, now + window, and the 1500 ms first-byte bound (the older spec's figure, now first-byte only) runs from the same origin on the same clock. At expiry the decision is final (batch fails → Continue) and the vendor - request is canceled; cancellation and resource cleanup complete + request is canceled. Cancellation and resource cleanup complete asynchronously and never delay the response. TS sends `Accept-Encoding: identity`, and because that does not _guarantee_ identity coding, a response arriving with any `Content-Encoding` is @@ -756,7 +756,7 @@ referrers are never in the security view. Every degree of freedom is closed: vendor does not guarantee method-invariant challenge bodies, so the validated HEAD bytes cannot establish the GET length (a vendor-guaranteed equivalence, if ever ratified, may restore the - field as a reviewed change; the older DataDome spec's HEAD handling + field as a reviewed change, and the older DataDome spec's HEAD handling remains superseded). Exceeding size, first-byte, or total deadline fails the batch → Continue. - **One pointer contract, one place.** The single normative @@ -768,7 +768,7 @@ referrers are never in the security view. Every degree of freedom is closed: the vendor's documented `Set-Cookie X-DD-B` allow-example while another invalidated the whole batch). No `X-DD-*` wildcard exists: every name is enumerated, `X-DD-B` included and forwarded exactly once - as the vendor's documented cookie-mode browser-response signal; it is + as the vendor's documented cookie-mode browser-response signal. It is never copied into publisher-upstream or another integration. The documented vendor responses (both the challenge example and the `Set-Cookie X-DD-B` allow example) are **verbatim fixtures asserting @@ -783,29 +783,29 @@ referrers are never in the security view. Every degree of freedom is closed: - **One global order:** core finalization → ordinary mutators → security effects → **final cache/privacy invariant pass, unconditionally last**. Security precedence over publisher-facing - mutations comes from its position, not a "wins" rule; nothing outranks + mutations comes from its position, not a "wins" rule. Nothing outranks the invariant pass, or a challenge could combine `Set-Cookie` with public caching. The older DataDome spec's "applies last, after finalization" wording is **superseded by this order**. That baseline remains - unchanged; this PR-specific section prevents an implementation from placing + unchanged. This PR-specific section prevents an implementation from placing DataDome after the invariant pass and reopening the public-cache-plus-cookie bug. - The channel adopts the shared layers: structured attributed batches (§3, atomic per batch, a 302 must never lose `Location` to a budget while keeping its cookie), reserved header names, budgets, and the - invariant pass, with one sequencing rule fail-open depends on: the + invariant pass, with one sequencing rule fail-open depends on, where the complete challenge batch is **validated and budgeted before the Respond decision commits**, so a rejection converts to Continue while - the publisher route is still available; discovering the rejection + the publisher route is still available, because discovering the rejection after Respond has short-circuited routing would leave nothing to fail open _to_. ### 4a.1 Public Suffix List snapshot (normative) The vendored Mozilla PSL revision used for registrable-domain -computation in §4a: the implementation PR vendors the list file +computation in §4a, and the implementation PR vendors the list file and records its upstream commit hash here. Rules: ICANN **and** private -sections apply; hostnames are IDNA-mapped before matching; IP literals +sections apply, hostnames are IDNA-mapped before matching, and IP literals and single-label hosts have no registrable domain (cookie falls back to host-only). Updating the snapshot is a reviewed spec change. @@ -821,7 +821,7 @@ host-only). Updating the snapshot is a reviewed spec change. The implementation copies the source bytes at that commit without editing and writes the lowercase 64-hex SHA-256 plus one trailing LF (no filename or other fields) to the required hash path. CI verifies the bytes, hash, and -commit reference together; updating any one without the others fails. The +commit reference together. Updating any one without the others fails. The provenance file is canonical JSON with exactly `{upstream_repository, upstream_commit_oid, upstream_commit_tree_oid, source_path, source_blob_oid, source_sha256_hex}`. The vendoring PR description @@ -849,7 +849,7 @@ this subsection means that DataDome protection is enabled, not that an operator supply an authority. Redirect following is disabled. No TS-controlled advertising identifier, consent-store key, graph value, request query, or full referrer is admitted. The normalized publisher URL path remains disclosed and -may itself contain publisher-chosen data; sign-offs 23/28 must classify that +may itself contain publisher-chosen data. Sign-offs 23/28 must classify that surface, its retention, and DSR handling rather than calling the entire URL identity-free. @@ -859,7 +859,7 @@ Core-derived fields: - `RequestModuleName`, `ModuleVersion`, `TimeRequest`, `Port` - `ServerName`, `ServerRegion` - `ClientID` from the single unambiguous `datadome` cookie only. The form key - is always present because the Protection API declares it mandatory; its + is always present because the Protection API declares it mandatory. Its value is the empty string when no unambiguous cookie exists - `CookiesLen`, `AuthorizationLen`, and `PostParamLen` as lengths only - `HeadersList`, containing only the source header names admitted by the @@ -875,7 +875,7 @@ Core derives those fields identically on every qualified adapter: - `Key` is the resolved DataDome server secret and is never obtained from request/config text; `IP` and `Port` are the remote address and TCP source port from trusted connection metadata. Missing `Key`, `IP`, or `Port` skips - the call through the metered fail-open path; no sentinel is synthesized + the call through the metered fail-open path. No sentinel is synthesized - `Method` is the validated HTTP method token; `Protocol` is exactly `http` or `https` from the adapter request URI; `Host` is the normalized ASCII request authority with a non-default port retained; `ServerHostname` is trusted TLS @@ -889,11 +889,11 @@ Core derives those fields identically on every qualified adapter: Unix timestamp in decimal microseconds, captured once before integration processing and constrained to `0..=2^53-1` - `ServerName` is the adapter-qualified deployment/service name and - `ServerRegion` is its adapter-qualified region code; either is omitted when + `ServerRegion` is its adapter-qualified region code. Either is omitted when the platform cannot supply it without request input - `CookiesLen`, `AuthorizationLen`, and `PostParamLen` are decimal byte counts of the received field/body surfaces before redaction. Overflow beyond an - unsigned 64-bit count skips the call; it never wraps or truncates + unsigned 64-bit count skips the call. It never wraps or truncates Exact request-header value mappings: @@ -918,7 +918,7 @@ Exact request-header value mappings: Request-field multiplicity is normalized **before** parsing, truncation, and form encoding, and adapters expose every received field line rather than a preselected first/last value. Every admitted value line must contain valid HTTP -field-value octets **and** valid UTF-8 after OWS removal; otherwise the vendor +field-value octets **and** valid UTF-8 after OWS removal, otherwise the vendor call is skipped through the metered fail-open path, because adapter-specific byte-to-string replacement is forbidden: @@ -942,14 +942,14 @@ byte-to-string replacement is forbidden: Repetition skips the vendor call before either length or `HeadersList` is constructed. `AuthorizationLen` is the byte length of the one OWS-normalized value. `PostParamLen` is always the byte length of the body - actually presented to core, not the numeric `content-length` value; a + actually presented to core, not the numeric `content-length` value. A malformed or body-inconsistent `content-length` is rejected by the shared HTTP request boundary before integrations run. - Multiple `cookie` field lines are permitted. Core OWS-normalizes them and joins them in received order with the literal bytes `; ` for the shared RFC cookie parser. `CookiesLen` is the byte length of that canonical joined value. `ClientID` is populated only when the parsed result contains exactly - one syntactically valid `datadome` pair; malformed cookie syntax or duplicate + one syntactically valid `datadome` pair. Malformed cookie syntax or duplicate `datadome` pairs produces the required empty `ClientID` value without exposing another cookie. The original cookie values never enter the vendor payload. @@ -957,10 +957,10 @@ byte-to-string replacement is forbidden: every admitted received field line in original line order, so repeated list fields and cookie lines remain repeated. A rejected request produces no `HeadersList` and no vendor call. Per-field caps apply to the single - normalized value; the 24,576-byte cap applies after complete form encoding. + normalized value. The 24,576-byte cap applies after complete form encoding. For an optional mapped value, zero received lines omits both source and mapped -field; one or more lines whose OWS-normalized values are all empty omits the +field. One or more lines whose OWS-normalized values are all empty omits the mapped form field but retains each received source name in `HeadersList`. If at least one list-valued line is nonempty, empty siblings remain represented in the exact received-order `, ` join. Mandatory `ClientID` and the three length @@ -971,10 +971,10 @@ every host and assert byte-identical form fields, lengths, `HeadersList`, and reject/omit outcomes. The corpus includes repeated list fields, identical and different singleton duplicates, multiple cookies, duplicate `datadome` cookies, empty values, invalid octets, and headers whose individual values -contain commas; invalid UTF-8 is a skip, never replacement decoding. +contain commas. Invalid UTF-8 is a skip, never replacement decoding. `true-client-ip`, `x-forwarded-for`, and `x-real-ip` are not admitted in v1. -The trusted `IP` field already supplies connection provenance; copying raw +The trusted `IP` field already supplies connection provenance, and copying raw forwarding headers would let a client or unqualified proxy manufacture vendor evidence. A future adapter-normalized forwarding chain requires a separately named typed field and vendor sign-off, never reuse of the raw header mapping. @@ -999,7 +999,7 @@ list. The following limits are bytes of the decoded field value before form encoding. Truncation is UTF-8-boundary-safe. `XForwardedForIP` alone truncates -from the end; every other bounded field retains its prefix: +from the end, while every other bounded field retains its prefix: - 8 bytes: `SecCHDeviceMemory`, `SecCHUAMobile`, `SecFetchStorageAccess`, `SecFetchUser` @@ -1022,12 +1022,12 @@ unbounded per-field by the vendor table but remain subject to the total bound. The complete `application/x-www-form-urlencoded` body, including field names, `=`/`&` separators, and percent-encoding expansion, must be at most **24,576 bytes**. Core constructs and measures the whole payload before issuing the -request. It does not silently drop optional fields to fit: overflow skips the +request. It does not silently drop optional fields to fit. Overflow skips the vendor call and takes the same metered fail-open `Continue` path as a transport failure. This is deliberately narrower than DataDome's currently documented required -surface: notably, it withholds `CookiesList` and omits empty source-header +surface. Notably, it withholds `CookiesList` and omits empty source-header fields. Product/vendor sign-off 28 therefore requires written confirmation that this exact reduced profile is supported. Until that confirmation and adapter conformance fixtures exist, the DataDome integration is not @@ -1042,38 +1042,38 @@ is rejected. | Header | Direction | Scope | | --------------------- | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `X-DataDome-ClientID` | response → upstream overlay | Disabled by default; admitted only with `expose_client_id_to_origin = true`. Owner-scoped publisher overlay only, never the shared request view or another integration | +| `X-DataDome-ClientID` | response → upstream overlay | Disabled by default. Admitted only with `expose_client_id_to_origin = true`. Owner-scoped publisher overlay only, never the shared request view or another integration | #### 4a.2.3 The single pointer matrix (normative, decision × session mode × pointer) This is the one authoritative browser-response contract. Session mode -is **cookie** in v1 (sessionByHeader is startup-rejected; a header-mode +is **cookie** in v1 (sessionByHeader is startup-rejected, and a header-mode column is added by the sign-off-23 opt-in, never implicitly). No wildcard rows exist, every accepted name is enumerated, and **every cell terminates in exactly one outcome**. | Pointer | Respond (cookie mode) | Continue (cookie mode) | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------- | -| `Set-Cookie` | typed `datadome` cookie operation (hook spec §4a parser; exactly one `datadome` field) | typed `datadome` cookie operation | +| `Set-Cookie` | typed `datadome` cookie operation (hook spec §4a parser, exactly one `datadome` field) | typed `datadome` cookie operation | | `X-Set-Cookie` | **invalidate the batch → Continue** (mode mismatch: TS never requests header sessions in v1; a vendor response asserting one must not half-apply) | **invalidate the batch** → effects dropped | | `Location` | forward on a 3xx Respond, replace; **on a non-3xx Respond → invalidate the batch → Continue** (a redirect field without a redirect status is a malformed decision) | **invalidate the batch** (effects dropped) | | `Content-Type` | forward (owns its body), replace | **invalidate the batch** (effects dropped) | -| `Cache-Control` | restricted merge; invariant pass last | **invalidate the batch** (effects dropped) | +| `Cache-Control` | restricted merge, invariant pass last | **invalidate the batch** (effects dropped) | | `Pragma` | drop-individually, logged (response `Pragma` has no standardized meaning, RFC 9111 §5.4) | drop-individually, logged | | `X-DataDome` | forward, owner-scoped typed telemetry | forward, owner-scoped typed telemetry | -| `X-DD-B` | forward as a browser-response security signal; never copy to publisher-upstream or another integration | forward as a browser-response security signal | +| `X-DD-B` | forward as a browser-response security signal, never copy to publisher-upstream or another integration | forward as a browser-response security signal | | anything not listed | invalidate the batch → Continue | invalidate → effects dropped | Singleton-field multiplicity (`Location`, `Content-Type`, `X-DataDome`, `X-DD-B`, `X-Set-Cookie` appearing more than once) invalidates the -batch atomically; list-valued fields (`Cache-Control`, `Pragma`) join +batch atomically. List-valued fields (`Cache-Control`, `Pragma`) join per RFC 9110 §5.3 before their cell applies (hook spec §4a). `X-DD-B` is security-owned when DataDome is enabled. Before applying the fresh security batch, core removes every pre-existing instance from the origin, cached ordinary artifact, 304 metadata update, core response, or ordinary mutator. A valid pointed vendor value then uses **replace-all** and the final -response cardinality must be exactly one; if the fresh vendor batch does not +response cardinality must be exactly one. If the fresh vendor batch does not point to it, final cardinality is zero. Append is never allowed. Fixtures cover origin collision, cache-hit collision, 304 collision, repeated vendor fields, and one valid fresh value, proving “exactly once” at final emission rather than @@ -1081,7 +1081,7 @@ merely inside the vendor batch. **Fixtures**: DataDome's documented challenge response (`Set-Cookie`, `Pragma`, `X-DataDome`, `Cache-Control`) asserts the decision stays -**Respond** with exactly the mapped fields; the documented allow example +**Respond** with exactly the mapped fields. The documented allow example (`Set-Cookie X-DD-B`) asserts Continue proceeds with the cookie applied and `X-DD-B` forwarded exactly once, neither fixture may fail open. @@ -1109,7 +1109,7 @@ expose_client_id_to_origin = false expose_host_fingerprints_to_vendor = false ``` -The Protection API authority is not configurable: it is the core-owned +The Protection API authority is not configurable. It is the core-owned `https://api-fastly.datadome.co/validate-request` endpoint defined in §4a.2.1, with redirects disabled. The baseline `protection_api_origin` field is a startup error under this delta, as are `sessionByHeader`, `session_by_header`, @@ -1149,7 +1149,7 @@ different evidence. on, with cookie emission v1 reserves. 3. **At least one real consumer ships in the same PR**, an existing integration registering a mutator for a real need (or, failing a real - need, the feature waits; scaffolding with only self-referential tests is + need, the feature waits, and scaffolding with only self-referential tests is dead code and will be removed). 4. Every adapter applies mutations on its outbound path, with a per-adapter route test asserting an integration-set header appears in the response. @@ -1190,14 +1190,14 @@ ordinary mutator API. ## 6. Divergences from issue #782 -This spec supersedes #782 on the following points; the issue is updated to +This spec supersedes #782 on the following points. The issue is updated to reference this spec when the implementing PR (the hook's return with its first consumer, §7) merges: | #782 says | This spec says | Why | | -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -| Mutations apply to the outbound response generally | Eligibility is the explicit §3a matrix, centered on processed documents | Pass-through/error/304 mutation has different semantics per adapter; enumerating beats implying | -| Ship the trait + registry + adapter application | Additionally: structured operations API (§2), reserved surface (§3), and a real consumer in the same PR (§4, item 3) | PR #838 shipped the trait with zero call sites; an unrestricted `&mut HeaderMap` cannot enforce any collision policy | +| Mutations apply to the outbound response generally | Eligibility is the explicit §3a matrix, centered on processed documents | Pass-through/error/304 mutation has different semantics per adapter, and enumerating beats implying | +| Ship the trait + registry + adapter application | Additionally: structured operations API (§2), reserved surface (§3), and a real consumer in the same PR (§4, item 3) | PR #838 shipped the trait with zero call sites, and an unrestricted `&mut HeaderMap` cannot enforce any collision policy | ## 7. Disposition (2026-08-25) @@ -1211,7 +1211,7 @@ first consumer, §7) merges: that must set response headers, for example `Accept-CH` client-hint requests or detection results such as §4a's security channel. The returning PR implements this spec, not PR #838's shape. -- Recorded for that future design, from review of the earlier draft: the +- Recorded for that future design, from review of the earlier draft. The `&mut HeaderMap` shape concern stands. Handing a mutator a mutable header map makes §3's collision policy unenforceable by construction, because core cannot validate or attribute writes it never sees, so the From 47ef74163964ac2077ca44225db41a570f2c153f Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 23:22:51 +0100 Subject: [PATCH 010/192] Restatus the unmerged provider specs and correct the seam spec counts The four series specs (client-cycle EC resolve, permission model, pluggable providers, migration and rollout) each carried a Status line saying they were implemented. The code they describe is only in PRs #1043 to #1047 and none of those is merged, so the line read as shipped behavior. Each now says Proposed, names the PR that carries the implementation and states that it is not yet on main, keeping the existing revision dates and notes. The integration provider seam spec carried counts and line references that do not hold on main at d516a9e94. Corrected against that commit: - Section 4 said migration_guards.rs embeds "the thirteen vendor files". The directory holds 23 .rs files (2 infrastructure, 6 in nextjs/, 2 in datadome/, 13 top-level integration modules), the guard embeds 20 of them and 9 of those 20 belong to the nine vendors, with osano.rs and the two datadome/ files absent. builders() registers 13 integrations, which is a different 13 from the file count. - Section 3.5 gave no counts for the prepare and finalize calls. There are nine production prepare_request call sites across the four adapters and a tenth in core, and the single production finalize_response call site is in core rather than in any adapter. - Section 8 item 3 described a proxy resolving geo twice, which does not happen on main. The real double resolution is the adapter EC context build against handle_auction on POST /auction. - Section 8 item 5 understated the Spin gap and misdescribed Cloudflare. Cloudflare covers every route it registers and has no health route, while Spin skips its first-party bindings as well as its inline admin stubs. - Line references: settings.rs:166 to :215, auction/mod.rs:49 to the list at :51 to :53, publisher.rs:4361 to :4369. Section 6 now requires the round trip to be proven on the Fastly adapter, the primary deployment target, rather than on any adapter, because Fastly has no library target and the round trip otherwise only runs on the Axum dev server. --- ...26-07-30-client-cycle-ec-resolve-design.md | 3 +- .../2026-07-30-permission-model-design.md | 3 +- .../2026-07-30-pluggable-providers-design.md | 5 +- ...07-30-provider-migration-rollout-design.md | 9 +- ...-08-27-integration-provider-seam-design.md | 123 ++++++++++++------ 5 files changed, 95 insertions(+), 48 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 63e9226d78..d8b161ae81 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -1,6 +1,7 @@ # Design Spec: Client-Cycle Edge Cookie Providers and the Resolve Endpoint -**Status:** Implemented (hardened v1) in PR #1046, on the threat model below. +**Status:** Proposed. PR #1046 carries the implementation (hardened v1, on +the threat model below) and is not yet merged to main. The full anti-replay reservation machinery (§3.9) and the vendor envelope verification it serves are deliberately not in v1: they land with the first real vendor scheme, which brings the concrete envelope format the diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 531a382438..6e9438a4bd 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -1,6 +1,7 @@ # Design Spec: Jurisdiction Permission Model -**Status:** Implemented in PR #1045; revised against the implementation, 2026-08-25. +**Status:** Proposed. PR #1045 carries the implementation and is not yet +merged to main. Revised against that implementation, 2026-08-25. **Author:** Engineering **Issue references:** #779 **Related specs:** `2026-07-30-pluggable-providers-design.md`, diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 3932938736..e7fd775c9d 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -1,7 +1,8 @@ # Design Spec: Pluggable Edge Cookie, Device, and Geo Providers -**Status:** Implemented in PR #1043 (Edge Cookie provider seam) and PR #1044 -(device and geo selection); revised against the implementation, 2026-08-25. +**Status:** Proposed. The implementation is carried by PR #1043 (Edge Cookie +provider seam) and PR #1044 (device and geo selection), neither of which is +yet merged to main. Revised against that implementation, 2026-08-25. **Author:** Engineering **Issue references:** #777, #778, #780, #781 **Related specs:** `2026-07-30-permission-model-design.md`, diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 5886f7f71b..bea38238c4 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -1,9 +1,10 @@ # Design Spec: Provider and Permission Model, Migration and Rollout -**Status:** Revised against the implemented series (PRs #1043-#1047), -2026-08-25. The §8 sign-off rows remain the series' decision ledger, and rows the -implementation now satisfies are marked with their PR so the task force can -ratify rather than re-litigate. +**Status:** Proposed. The implementation is carried by the series PRs #1043 to +#1047, none of which is yet merged to main, and this document was revised +against that series on 2026-08-25. The §8 sign-off rows remain the series' +decision ledger, and rows the implementation now satisfies are marked with +their PR so the task force can ratify rather than re-litigate. **Author:** Engineering **Issue references:** #777-#781 (epic) **Related specs:** `2026-07-30-pluggable-providers-design.md`, diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 3fbb27b702..e61e3525c8 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -67,11 +67,12 @@ seventh PR targets, and the five series PRs do not touch these files. imports and calls each vendor's config type by name (`crates/trusted-server-core/src/config.rs:136` to `:166`). 4. **Auction providers are a second closed table.** - `crates/trusted-server-core/src/auction/mod.rs:49` lists Prebid, APS and - the ad server mock. + The list of Prebid, APS and the ad server mock is at + `crates/trusted-server-core/src/auction/mod.rs:51` to `:53`, inside + `provider_builders()` at `:49`. 5. **Two vendors reach further into core.** DataDome drives cache privacy and the origin fetch decision through a marker type - (`html_processor.rs:303`, `publisher.rs:4361` to `:4381`, + (`html_processor.rs:303`, `publisher.rs:4369` to `:4381`, `publisher.rs:2653`), and GPT diagnostics is called by name from all four adapters (for example `crates/trusted-server-adapter-fastly/src/app.rs:564`). @@ -182,9 +183,16 @@ the neutral form. extension meaning "this response is personalized to the request, do not share it", set by any integration. Core keeps the behavior, being the full body buffer and private cache, and stops naming a vendor. -- **Request prepare and finalize.** The direct GPT diagnostics calls in the - four adapters move behind hooks on the registration, so an adapter runs - whatever its registrations declare. +- **Request prepare and finalize.** The direct GPT diagnostics calls move + behind hooks on the registration, so an adapter runs whatever its + registrations declare. Those calls reach beyond the adapter edge. On + `main` nine production `prepare_request` call sites sit in the four + adapters (two in Fastly, two in Axum, two in Cloudflare and three in + Spin) and a tenth sits in core's `handle_publisher_request` + (`publisher.rs:4050`). `finalize_response` has one production call site + and it is in core rather than in any adapter (`publisher.rs:4783`), so + the finalize hook has to run on the core response path and not only at + the adapter edge. ### 3.6 Identity, geo and device as registration capabilities @@ -262,18 +270,27 @@ TypeScript, its config type and its tests into `[integrations.]` configuration tables need no change, because `IntegrationSettings` is a flattened map that already accepts unknown vendor -keys (`crates/trusted-server-core/src/settings.rs:166`). +keys (`crates/trusted-server-core/src/settings.rs:215`). Two more places every move must touch, found by mapping `main`: -- `crates/trusted-server-core/src/migration_guards.rs` embeds every core - source file by relative path with `include_str!`, the thirteen vendor - files included, so a vendor move that leaves its entry behind breaks the - build rather than a test. The guard cannot derive its list from the - registrations, because `include_str!` paths are fixed at compile time, so - this change drops the nine vendors' files from the guard instead, because a module - crate is outside the core neutrality guarantee, and a move then deletes - nothing there. +- `crates/trusted-server-core/src/migration_guards.rs` embeds core source + files by relative path with `include_str!`, so a vendor move that leaves + an embedded entry behind breaks the build rather than a test. On `main` + the directory `crates/trusted-server-core/src/integrations/` holds 23 + `.rs` files, being 2 infrastructure files (`mod.rs` and `registry.rs`), + 6 files in the `nextjs/` subdirectory, 2 in the `datadome/` submodule and + 13 top-level integration modules. The guard embeds 20 of those 23, and 9 + of the 20 are files of the nine vendors in the table above. The three it + does not embed are `osano.rs` and the two `datadome/` files, so the guard + is already incomplete and the Osano move has no guard entry to delete. + Separately, `builders()` registers 13 integrations, which is not the same + 13 as the file count, because `adserver_mock` is a file with no + registration while `nextjs` is a registration held in a subdirectory. The + guard cannot derive its list from the registrations, because + `include_str!` paths are fixed at compile time, so this change drops the + vendors' files from the guard instead, because a module crate is outside + the core neutrality guarantee, and a move then deletes nothing there. - The `ts audit` command carries its own vendor table (detection patterns and configuration section names in `crates/trusted-server-cli/src/commands/audit/analyzer.rs` and @@ -292,18 +309,24 @@ deployment that lists the same integrations gets the same responses. ## 6. Acceptance -1. **A round trip with a non-default implementation.** A test integration - defined outside `trusted-server-core`, carrying its own JavaScript, - registers through an adapter, appears in the served bundle with the - right hash, runs its hooks in the right order, and is rejected on a - duplicate id. A seam is only proven by an implementation that is not the - built-in one. -2. **Capabilities round trip.** The same test integration declares an - identity, a geo and a device provider. With the three selectors naming - it, a request is served by all three (the created identifier carries its - code, the resolved country and the device signals are its). With a - selector naming a module that lacks the capability, startup fails with - an error that names the module and the capability. +1. **A round trip with a non-default implementation, on Fastly.** A test + integration defined outside `trusted-server-core`, carrying its own + JavaScript, registers through an adapter, appears in the served bundle + with the right hash, runs its hooks in the right order, and is rejected + on a duplicate id. A seam is only proven by an implementation that is + not the built-in one. The round trip has to run on the Fastly adapter, + which is the primary deployment target, and not only on the Axum dev + server, because a seam proven on the dev server alone is not a seam any + production deployment can use. The Fastly adapter carries only + `src/main.rs` and so has no library target for an external crate to + register through (§8 item 7), so meeting this criterion means giving + that adapter a composition entry point a vendor crate can reach. +2. **Capabilities round trip, on the same adapter.** The same test + integration declares an identity, a geo and a device provider. With the + three selectors naming it, a request is served by all three (the created + identifier carries its code, the resolved country and the device signals + are its). With a selector naming a module that lacks the capability, + startup fails with an error that names the module and the capability. 3. **Parity.** The existing integration and parity suites pass unchanged, because the built-in set still registers through the same path. 4. **No vendor left behind.** The rewritten deploy-validation test shows @@ -346,12 +369,22 @@ for each vendor to rediscover. or a documented build-script recipe removes the trap, and the probe pins the file's line endings and tests the literal, which every vendor would otherwise have to reinvent. -3. **A provider is resolved more than once per request.** A proxy that - wants the resolved location calls the geo provider itself while the - request path has already called it. `CLAUDE.md`'s principle that a - vendor sharing one backend makes a single call per request needs a - per-request provider context to hang that on, which this change does not - introduce. +3. **A provider is resolved more than once per request.** On `main` this is + core against core rather than a proxy against the request path. Every + adapter resolves geo once to build the EC context (`build_ec_context` at + `crates/trusted-server-adapter-axum/src/app.rs:162`, + `crates/trusted-server-adapter-cloudflare/src/app.rs:142` and + `crates/trusted-server-adapter-spin/src/app.rs:346`, and + `build_ec_request_state` at + `crates/trusted-server-adapter-fastly/src/app.rs:394`). On + `POST /auction` the same request then reaches `handle_auction`, which + looks the same client IP up a second time to fill in the auction's + device info (`crates/trusted-server-core/src/auction/endpoints.rs:262`). + Those two are the only production geo call sites in the tree, so the + auction route is where the duplication shows. `CLAUDE.md`'s principle + that a vendor sharing one backend makes a single call per request needs + a per-request provider context to hang that on, which this change does + not introduce. 4. **One core reader still reaches into a vendor's payload.** The `hb_adid` fallback in the publisher reads the APS renderer's fields, so the APS migration needs a neutral answer for it rather than only the @@ -359,13 +392,22 @@ for each vendor to rediscover. 5. **Request preparation covers different routes on each host.** Every adapter runs preparers before routing, but not on the same set of - routes: one runs them on every route but the health check, one skips a - batch endpoint and its admin diagnostics deliberately, one covers three - of its paths, and one skips its inline admin stubs. A module that strips - its own reserved query or cookie is therefore protected on a different - set of routes depending on the host it is deployed to. Making that - uniform means routing each adapter's hand-written handlers through one - wrapper, which is worth doing before a vendor depends on it. + routes. Cloudflare covers everything it registers, because every route + binding goes through one wrapper (`make_handler`) and the adapter has no + health route at all. Axum covers every route but `GET /health`, which is + bound to an inline closure. Fastly covers every route but `GET /health`, + which short-circuits before the app is built, plus the S2S batch sync + and the two admin lookup routes, which return deliberately before the + preparer runs. Spin is the widest gap, covering only `POST /auction`, + the two page-bids GET routes and the publisher fallback, so its + `/health`, its discovery and signature-verification routes, its inline + admin stubs and all six of its first-party bindings (`proxy`, `click`, + `sign` on GET and POST, and `proxy-rebuild` on GET and POST) run with no + preparer at all. A module that strips its own reserved query or cookie + is therefore protected on a different set of routes depending on the + host it is deployed to. Making that uniform means routing each adapter's + hand-written handlers through one wrapper, which is worth doing before a + vendor depends on it. 6. **A module's validate function runs nowhere in a real deployment.** Building the registry calls only a builder's build function, and the @@ -420,3 +462,4 @@ defines, and both should land before the first vendor is asked to use it. | 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | | 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged, and only this status line and this row are new. | +| 2026-08-31 | Corrected the counts and line references the review found, against `main` at d516a9e94: the source-file guard counts (§4), the prepare and finalize call-site counts (§3.5), the real double geo resolution on `POST /auction` (§8 item 3), the per-adapter preparer coverage (§8 item 5), and the `settings.rs`, `auction/mod.rs` and `publisher.rs` line references. §6 now requires the round trip on Fastly rather than on any adapter. | From 369dd23cc0ed0195f72faf04107ec1a1547dec87 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 1 Sep 2026 04:26:59 +0100 Subject: [PATCH 011/192] Answer the PR #1084 review findings in the series specs Rewrite the resolve endpoint's origin check as an exact serialized-origin comparison, defaulting to the single origin https://{publisher.domain} with an optional operator-configured list, and delete the sibling subdomain justification that did not hold. Record that the origin check is defense in depth under the provider's envelope verification, require the sibling subdomain, wrong scheme and non-default port rejection cases, and close open question 7.5. Require core to expire the ts-ecr marker on any request carrying a ts-ec the selected provider does not own, in both the client-cycle and pluggable provider specs, so a provider switch restarts the client cycle instead of leaving a visitor with a marker and no identity. Record the constraints on carrying resolved permissions to the browser without designing the mechanism, which belongs with the first vendor module. Require a nonempty DeviceProvider::required_permissions from a module-supplied provider to be refused at registry resolution rather than described as a documented no-op. Replace the DataDome singleton-header rule that skipped the vendor call on a repeated field, which would have created an attacker-controlled bot detection bypass and described no shipped behavior. Core now passes repeated field lines through as received and leaves interpretation to the vendor integration, keeping only the protocol-level content-length rule. Correct the APS migration row and add a seam spec finding for the pre-existing browser-side APS coupling in core TypeScript. --- ...26-07-30-client-cycle-ec-resolve-design.md | 116 ++++++++++++------ ...integration-response-header-hook-design.md | 77 +++++++++--- .../2026-07-30-pluggable-providers-design.md | 77 ++++++++---- ...-08-27-integration-provider-seam-design.md | 48 +++++--- 4 files changed, 227 insertions(+), 91 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index d8b161ae81..8b8b0a9916 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -65,20 +65,23 @@ verify against. `POST /_ts/api/v1/ec/resolve` MUST: 1. **Reject cross-site requests with an origin check.** v1 authorizes a - request only when its `Origin` header names the publisher's configured - domain or a subdomain of it. A missing or foreign `Origin` is rejected - with `403`. Browsers always send `Origin` on POST `fetch`, so its - absence means a non-browser caller, which has no business on a - page-script endpoint. The 2026-07-31 draft asked for an exact origin - allowlist as new configuration plus an optional session-bound CSRF - token. v1 derives the allowed set from `publisher.domain` (existing, - operator-trusted configuration) and admits the publisher's own - subdomains, because the publisher controls their subdomain namespace, and the - draft's sibling-subdomain concern applies to `Sec-Fetch-Site: -same-site` (which v1 does not consult at all), not to a suffix match on - the publisher's own apex. An explicit multi-origin allowlist remains - open (§7.5) for publishers whose pages run on domains other than the - configured apex. + request only when its `Origin` header, serialized as scheme, lowercased + host and effective port, is byte-equal to an accepted origin. The + default accepted set is the single origin `https://{publisher.domain}` + and nothing else, so a sibling subdomain, the `http://` scheme and a + non-default port are all refused. A missing or foreign `Origin` is + rejected with `403`. Browsers always send `Origin` on POST `fetch`, so + its absence means a non-browser caller, which has no business on a + page-script endpoint. An operator may configure further accepted + origins, each of them compared the same exact way, for a publisher + whose pages are served from `www` or from another domain. This is the + exact allowlist the 2026-07-31 draft asked for, and it replaces the + suffix match on `publisher.domain` an earlier revision of this spec + described, which admitted every subdomain of the apex including ones + the publisher may not control. The origin check is defense in depth + and not the primary control. The primary control is the provider's + envelope verification with audience and session binding, which §3.9 + parks and which still has to land with the vendor scheme. 2. **Verify the payload per provider.** The endpoint hands the posted payload to the selected provider's `resolve_from_client` and creates only what the provider returns. Whether the payload is trustworthy is the @@ -183,15 +186,45 @@ the bar for the vendor-scheme implementation: marker shares the Edge Cookie's scope and lifetime and is expired together with it on withdrawal, so a visitor who later re-establishes the permission can resolve again. +- **The marker must not outlive the provider that set it.** The marker + carries no identity, so unlike the `ts-ec` cookie it is not namespaced + by the provider code envelope, and a long `Max-Age` that only + withdrawal expires would leave it standing across a provider switch. + The visitor would then hold a marker saying a resolve has already + succeeded with no identity behind it, and the page script would + suppress the re-post that would start a new one, so the visitor sits + with no identity rather than a restarted one. Core therefore expires + the marker on any request that carries a `ts-ec` the selected provider + does not own, which is the same `{code}~` ownership test core already + applies to the identifier itself. A switch then restarts the client + cycle instead of stalling it. - **The page leg is permission-gated before vendor contact.** The demo module contacts no vendor (it posts a constant), so the draft's injection-time and live-CMP gating requirements bind the first vendor - module, not v1: a vendor module must not derive identity or contact the - vendor for a visitor whose resolved permissions do not satisfy the - provider's declaration, must re-check immediately before vendor contact - (consent can change between document delivery and asynchronous vendor - contact, including BFCache restoration), and its injection is keyed off - the provider selection exactly as the demo's is today. + module rather than v1. A vendor module must not derive identity or + contact the vendor for a visitor whose resolved permissions do not + satisfy the provider's declaration, must re-check immediately before + vendor contact (consent can change between document delivery and + asynchronous vendor contact, including BFCache restoration), and its + injection is keyed off the provider selection exactly as the demo's is + today. +- **How the resolved permissions reach the browser is not designed here.** + The requirement above assumes the page can read the server's decision, + and nothing in v1 carries that decision to the page. The binding + constraint is that the JavaScript bundle is composed at startup and + served under a content hash, so one body is shared across every visitor + and cannot carry per-visitor permission state, which means the signal + has to be per request. The candidate carriers are a response header on + the document, a value injected into the document during HTML + processing, a first-party endpoint the page fetches, and a + non-HttpOnly cookie set alongside the marker. This spec names them and + deliberately does not choose between them. Whichever is chosen, the + server's resolved permission decision is the authority, and an in-page + CMP read is only a withdrawal re-check layered under it and never a + substitute for it, because a page-side read can narrow what the server + resolved and must never widen it. Designing the mechanism is out of + scope for this set of PRs and belongs with the first vendor module, + which is the first consumer that needs it. - The JS module ships through the standard integration bundle mechanism, loaded only when a client-cycle provider is the selected EC provider. The interaction between provider-keyed bundle content and content-hash / @@ -214,13 +247,17 @@ end in tests and demonstrations. ## 6. Testing -- Endpoint unit tests cover: origin rejection (missing and foreign), - subdomain acceptance, content-type rejection, the body bound, the - identifier bound, the different-identity conflict, the no-graph refusal, - the closed permission gate, the unverified payload, and the marker, - cache-control, and graph-row effects of a success. +- Endpoint unit tests cover origin rejection (missing, foreign, a sibling + subdomain of the publisher apex, the `http://` scheme, and a non-default + port), acceptance of an operator-configured extra origin, content-type + rejection, the body bound, the identifier bound, the different-identity + conflict, the no-graph refusal, the closed permission gate, the + unverified payload, and the marker, cache-control, and graph-row effects + of a success. - A round-trip test drives a client identifier through organic deferral, resolve, cookie set, and verbatim read-back recognition. +- A test asserts that a request carrying a `ts-ec` the selected provider + does not own expires the `ts-ecr` marker. - The cross-language constant test pins the endpoint's shared constants to the page-script source. - Remaining for the vendor scheme: a real-browser integration round trip @@ -241,21 +278,26 @@ end in tests and demonstrations. 3. Rate limiting / abuse posture at the edge for an unauthenticated POST. 4. Startup rejection of the client-cycle selection on adapters that cannot route the endpoint, once the portability adapters gain platform KV. -5. An explicit multi-origin allowlist for publishers whose pages run on - domains other than the configured apex (v1 authorizes the apex and its - subdomains). +5. **Answered, and kept here for the record.** An explicit multi-origin + allowlist for publishers whose pages run on domains other than the + configured apex. §3.1 now requires exact serialized-origin comparison, + with a default accepted set of the single origin + `https://{publisher.domain}` and an optional operator-configured list + of further exact origins for the `www` or other-domain case, so the + question is settled rather than carried. 6. How JS module selection keyed off EC provider configuration coexists with content-hashed/SRI-pinned bundles, meaning per-config hashes, cache keying, and the config-push story for them. ## 8. Revision record vs the 2026-07-31 draft -| Draft position | v1 (PR #1046) | Why | -| ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | -| Origin check via new allowlist config + CSRF token | Publisher apex + subdomains from existing `publisher.domain`. Missing/foreign `Origin` → `403` | Uses existing operator-trusted configuration. Explicit allowlist stays open for multi-domain publishers (§7.5) | -| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | -| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | -| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | -| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | -| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | +| Draft position | v1 (PR #1046) | Why | +| ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | +| Origin check via new allowlist config + CSRF token | Exact serialized-origin comparison (scheme, lowercased host, effective port). Default accepted set is the single origin `https://{publisher.domain}`, plus an optional operator-configured list of further exact origins. Missing/foreign `Origin` → `403` | The draft's exact allowlist is adopted. An earlier revision of this spec allowed any suffix match on the apex, which admitted every subdomain including ones the publisher may not control, and its justification did not hold. This closes §7.5 | +| Marker cookie survives whatever happens to the identity it marks | Core expires the `ts-ecr` marker on any request carrying a `ts-ec` the selected provider does not own | The marker is not namespaced by the provider code envelope, so without this a provider switch leaves a visitor with a marker, no identity, and a page script that will not re-post | +| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | +| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | +| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | +| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | +| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 53d8869234..c297d95c9d 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -932,19 +932,52 @@ byte-to-string replacement is forbidden: parsing and is then bounded. Commas inside an individual value are not split and reserialized. - Every other admitted value-bearing source header in the exact mapping above - is singleton. Zero lines means omit the DataDome field. Exactly one valid - line is OWS-normalized and processed. Two or more lines, even identical, - are ambiguous and skip the vendor call through the metered fail-open path; - core never chooses first, last, or comma-joined. In particular this applies - to `origin`, `referer`, `user-agent`, `content-type`, `from`, every remaining + is presented as received. Zero lines means omit the DataDome field. Each + received line has its leading and trailing optional whitespace removed and + is then checked for valid field-value octets and valid UTF-8, exactly as + above. Two or more lines, identical or not, are preserved as separate lines + in received order and reach the integration that way. Core does not drop a + line, comma-join the lines, choose the first or the last, or skip the + vendor call because a field arrived more than once. This covers `origin`, + `referer`, `user-agent`, `content-type`, `from`, every remaining `sec-ch-*`/`sec-fetch-*` field, and `x-requested-with`. -- `authorization` and `content-length` are security singletons for this view. - Repetition skips the vendor call before either length or `HeadersList` is - constructed. `AuthorizationLen` is the byte length of the one - OWS-normalized value. `PostParamLen` is always the byte length of the body - actually presented to core, not the numeric `content-length` value. A - malformed or body-inconsistent `content-length` is rejected by the shared - HTTP request boundary before integrations run. +- **Why core does not decide this.** Header multiplicity is attacker + controlled, so a core rule that skipped the vendor call whenever an + admitted field arrived twice would let any client turn bot detection off + by sending one header twice. That is a bypass this document would have + created rather than described, because nothing in the shipped code behaves + that way. The protection path reads each mapped field through a single + `headers().get()` in `header_value` + (`crates/trusted-server-core/src/integrations/datadome/protection.rs`) and + calls the Protection API regardless of how many lines arrived. The + principle behind the rewrite is that core must not invent vendor-specific + header normalization and must not decide on a vendor's behalf to skip a + vendor call. How a repeated or otherwise ambiguous field is interpreted, + and how it is encoded into the vendor's payload, is the vendor's detection + logic, so it belongs to the vendor integration, which sees every received + line in received order through `HeadersList` and the presented field + lines. +- **Where that logic belongs.** The DataDome integration is core code today, + in `crates/trusted-server-core/src/integrations/datadome.rs` and the + `crates/trusted-server-core/src/integrations/datadome/` directory. Vendor + detection logic of this kind belongs in the vendor's own module rather + than in core, and the integration provider seam + (`2026-08-27-integration-provider-seam-design.md`) is where that move is + designed. This is a direction for that migration rather than a complaint + about the code as it stands. +- `content-length` stays a protocol-level singleton. A repeated, malformed, + or body-inconsistent `content-length` is rejected by the shared HTTP + request boundary before any integration runs, which is an HTTP + correctness rule and not a vendor decision, so it is unchanged. + `PostParamLen` is always the byte length of the body actually presented to + core, not the numeric `content-length` value. +- `authorization` never reaches the vendor payload as a value. + `AuthorizationLen` is the byte length of the OWS-normalized value. When + more than one `authorization` line arrives, every line is presented in + received order and how they are counted is the integration's decision, + as it is for any other repeated field. Repetition no longer skips the + vendor call, for the same reason as above, because that let the client + decide whether the vendor is consulted at all. - Multiple `cookie` field lines are permitted. Core OWS-normalizes them and joins them in received order with the literal bytes `; ` for the shared RFC cookie parser. `CookiesLen` is the byte length of that canonical joined @@ -969,9 +1002,12 @@ fields follow their explicit rules instead of this optional-field omission. Adapter qualification fixtures feed the same ordered repeated-field corpus to every host and assert byte-identical form fields, lengths, `HeadersList`, and reject/omit outcomes. The corpus includes repeated list fields, identical and -different singleton duplicates, multiple cookies, duplicate `datadome` -cookies, empty values, invalid octets, and headers whose individual values -contain commas. Invalid UTF-8 is a skip, never replacement decoding. +different duplicates of the pass-through fields, multiple cookies, duplicate +`datadome` cookies, empty values, invalid octets, and headers whose individual +values contain commas. For a duplicated pass-through field the asserted +outcome is that every received line survives in received order and that the +vendor call still happens, never a skip. Invalid UTF-8 is a skip, never +replacement decoding. `true-client-ip`, `x-forwarded-for`, and `x-real-ip` are not admitted in v1. The trusted `IP` field already supplies connection provenance, and copying raw @@ -1211,6 +1247,17 @@ first consumer, §7) merges: that must set response headers, for example `Accept-CH` client-hint requests or detection results such as §4a's security channel. The returning PR implements this spec, not PR #838's shape. +- Corrected on 2026-09-01, from review of PR #1084. §4a.2 previously + required core to treat two or more lines of an admitted singleton + request header as ambiguous and to skip the vendor call. Header + multiplicity is attacker controlled, so that rule would have given any + client a way to turn bot detection off by repeating a header, and it + described no shipped behavior, because `header_value` in + `crates/trusted-server-core/src/integrations/datadome/protection.rs` + reads each mapped field with a single `headers().get()` and the + Protection API is called regardless. Core now passes repeated lines + through as received and leaves the interpretation to the vendor + integration. - Recorded for that future design, from review of the earlier draft. The `&mut HeaderMap` shape concern stands. Handing a mutator a mutable header map makes §3's collision policy unenforceable by construction, diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index e7fd775c9d..8bfa1840a7 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -213,13 +213,18 @@ resolved in the implementation: behavior beyond gating. The gate itself has already run before `generate` is called, so a provider cannot use the fields to authorize itself. - `required_permissions` on `DeviceProvider` and `PlatformGeo`: **present, - with an empty default and no enforcement point.** The draft removed the - method from both traits because PR #838's copies were decorative. The - implementation keeps one uniform declaration seam across all three traits - instead. The built-in device and geo providers declare empty sets, and - core enforces the declaration only for the EC provider (section 5), so - nothing reads as a gate that is not one. The geo circularity argument - stands unchanged and is restated in section 5. + with an empty default, and refused rather than ignored when a module + declares one.** The draft removed the method from both traits because PR + #838's copies were decorative. The implementation keeps one uniform + declaration seam across all three traits instead. The built-in device + and geo providers declare empty sets, and core enforces the declaration + as a request gate only for the EC provider (section 5). Because no + per-request device gate exists to honor a device declaration, a nonempty + `required_permissions` from a module-supplied device provider is + rejected when the registry resolves the selection and the deployment + fails to start, so nothing reads as a gate that is not one. Section 5 + gives the reasoning. The geo circularity argument stands unchanged and + is restated in section 5. The implemented `EdgeCookieProvider` surface (`ec/provider.rs`): @@ -317,7 +322,7 @@ work with the permission model spec, which owns identity-state persistence and egress typing. The revision record lists them as deferred. The gate applies to EC providers **only**. Geo and device are ungated for -two different reasons, stated separately because only one of them is +different reasons, stated separately because only one of them is structural: - **Geo: circularity.** The permission set is resolved from jurisdiction, @@ -338,6 +343,21 @@ structural: class segment and a 12-hex-character hash prefix of the HTTP/2 SETTINGS signal), not raw signals, and the neutral default persists neither because the builtin provider produces no such fields. +- **Device: a declared permission is refused at startup, not ignored at + request time.** `DeviceProvider::required_permissions` has no + enforcement point on the device path, so a declaration cannot be + honored. Before the module seam that was inert, because core and the + host supplied the only two device providers and both declare the empty + set. With a vendor seam the method reads as a promise a vendor could + build on, and a vendor device provider declaring a permission would + still run on every request while appearing to be gated. Core therefore + rejects a nonempty `required_permissions` from a module-supplied device + provider when the registry resolves the selection, failing the + deployment at startup with a message naming the provider and the + permissions it declared. That refusal is lifted only when a real + per-request device gate exists, which is separate design work. The + built-in and host device providers are unaffected, because both declare + the empty set. ## 6. Selection, validation, and failure modes @@ -417,6 +437,15 @@ What a switch does, precisely: the canonical form is the owning provider's own normalization and the owning provider is no longer configured. Those rows stay as they are until their one-year entry TTL expires. +- **The `ts-ecr` client-cycle marker.** The marker carries no identity, so + it is not namespaced by the `{code}~` envelope and a switch would + otherwise leave it standing with its long `Max-Age` intact. Core expires + it on any request carrying a `ts-ec` the selected provider does not own, + using the same ownership test as read-back above, so a visitor whose + identifier has just become unrecognized is not left behind a marker that + tells the page script a resolve has already succeeded. Without that the + visitor would sit with no identity instead of a restarted one. The + client-cycle spec states the rule in full. **What a deployer must do about revocation.** Treat a provider switch as a one-way retirement of the identity population, and deal with the previous @@ -636,19 +665,19 @@ one acceptance contract: One row per divergence between the 2026-07-31 draft and the implementation this revision describes. -| Draft position | Implemented position | Why | -| ----------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Trait surface is a canonicalizing `parse` returning a typed id, plus `graph_key_suffix`, `cluster_prefix`, and `verify` | `accepts_id` (recognition) plus `normalize_id_for_kv` (KV key form), defaults matching the built-in shape. No typed id, key suffix, cluster capability, or `verify`. `keys_equal` stays out, as the draft required. | Recognition and KV keying are the two operations core performs today. A byte-for-byte round-trip test with a non-default provider pins the contract. | -| `GeneratedEdgeCookie::response_headers` and `IdentityInput.permissions` / `.consent` banned as speculative surface | Shipped with production consumers. Finalization applies provider headers, the resolve path returns them, and the organic create path populates the input fields. | The client-cycle resolve path landed in the same series and is their caller, satisfying the minimalism rule the ban enforced. | -| Identifier bounds enforced at create and parse | Enforced at create (`generate` and the resolve endpoint), cookie read-back, and cookie write. Violations rejected outright, never rewritten. `MAX_EC_ID_LEN` in `ec/cookies.rs`. | Every identifier entry point is covered, and the pre-epic sanitizing rewrite was removed as a silent-divergence hazard. | -| `provider = "none"` is valid alongside `legacy_providers` blocks | `none` (or an omitted selector) with any configured provider block is a startup error. | No `legacy_providers` exists in these PRs, so a block alongside statelessness can only be a mistake. | -| Every selection key is closed and unknown keys are startup errors | Device and geo keys are closed. EC vendor keys are open. Unknown blocks are captured as raw values in core, the adapter deserializes its own block, and a selected key with no injected provider fails loudly. | Core never names a vendor, so a vendor provider adds no core change. | -| Capability mismatch is a startup error at adapter wiring time | Configuration coherence fails at startup. A host-capability mismatch (missing `HostSignals`, uninjected vendor) fails loudly when the provider is built, stopping the request. | The adapter capability declaration that would move the check to startup is deferred with the capability matrix. | -| A creating provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to create without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | -| `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw signals. | Selection is an explicit operator opt-in and the neutral default makes no host signal call. | -| The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host signal processing, so the review decides whether the provider ships in the series. | -| Geo default flip sequenced into the later permission-model step, with an acknowledgment guard | Landed as specified in the same series, with the default of none, `default_country` required and validated against `permissions.yaml`, the `assume_single_jurisdiction` acknowledgment, and a failed lookup resolving to the requires-signal floor with error logging (`GeoStatus`, resolved in core so all adapters agree). | The permission model shipped in PR #1045, so the constraints exist where the draft required them. | -| All adapters serve the full EC feature set identically | Selector behavior is identical through the shared builders and core constructors. The EC API routes (identify, batch-sync, ec/resolve) are Fastly-only, documented in the Spin route list. | The portability adapters do not yet wire platform KV, and the gap is documented rather than silent. | -| Conformance suite, adapter capability matrix, delimiter-free key grammar, `verify`, `legacy_providers`, `versions` / `mint_version` | None of these are in PR #1043 or #1044. All are tracked follow-up work, deferred, not silently dropped. | The shipped seam did not need them, and each returns with the feature that gives it a production caller, per the spec's own minimalism rule. | -| `required_permissions` removed from the device and geo traits, added to the EC trait only at the permission-model step | Present on all three traits from the start, with empty defaults. Core enforces the EC declaration (gate in `EcContext`, landing in PR #1045). No device or geo enforcement point exists. | One uniform declaration seam, with an empty default that gates nothing, avoids the decorative-gate hazard while keeping the interface stable. The geo circularity stands. | -| Flat crate directories (`crates/trusted-server-geo-fastly`), no placeholder directories | Nested directories per capability (`crates/device/fastly`, `crates/geo/fastly`, `crates/edgecookie/`), flat package names. `crates/edgecookie` ships a README before its first crate. | Nested directories scale per vendor, package names already carry the naming convention, and the README stakes out the vendor location. | +| Draft position | Implemented position | Why | +| ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Trait surface is a canonicalizing `parse` returning a typed id, plus `graph_key_suffix`, `cluster_prefix`, and `verify` | `accepts_id` (recognition) plus `normalize_id_for_kv` (KV key form), defaults matching the built-in shape. No typed id, key suffix, cluster capability, or `verify`. `keys_equal` stays out, as the draft required. | Recognition and KV keying are the two operations core performs today. A byte-for-byte round-trip test with a non-default provider pins the contract. | +| `GeneratedEdgeCookie::response_headers` and `IdentityInput.permissions` / `.consent` banned as speculative surface | Shipped with production consumers. Finalization applies provider headers, the resolve path returns them, and the organic create path populates the input fields. | The client-cycle resolve path landed in the same series and is their caller, satisfying the minimalism rule the ban enforced. | +| Identifier bounds enforced at create and parse | Enforced at create (`generate` and the resolve endpoint), cookie read-back, and cookie write. Violations rejected outright, never rewritten. `MAX_EC_ID_LEN` in `ec/cookies.rs`. | Every identifier entry point is covered, and the pre-epic sanitizing rewrite was removed as a silent-divergence hazard. | +| `provider = "none"` is valid alongside `legacy_providers` blocks | `none` (or an omitted selector) with any configured provider block is a startup error. | No `legacy_providers` exists in these PRs, so a block alongside statelessness can only be a mistake. | +| Every selection key is closed and unknown keys are startup errors | Device and geo keys are closed. EC vendor keys are open. Unknown blocks are captured as raw values in core, the adapter deserializes its own block, and a selected key with no injected provider fails loudly. | Core never names a vendor, so a vendor provider adds no core change. | +| Capability mismatch is a startup error at adapter wiring time | Configuration coherence fails at startup. A host-capability mismatch (missing `HostSignals`, uninjected vendor) fails loudly when the provider is built, stopping the request. | The adapter capability declaration that would move the check to startup is deferred with the capability matrix. | +| A creating provider with no identity-graph store is a startup error | Not implemented. `ec_store` stays optional. The resolve endpoint refuses to create without a graph. The organic path persists rows whenever the graph is configured. | Portability adapters run without platform KV. Whether configuration should force the pairing is follow-up work. | +| `[device] provider = "fastly"` is startup-rejected pending a separate security design | Shipped as a selectable opt-in. The Fastly adapter injects `HostSignals`, the provider strengthens the browser/bot gate, and rows persist derived classes, not raw signals. | Selection is an explicit operator opt-in and the neutral default makes no host signal call. | +| The `host-signals` EC provider is deliberately dropped and its selection rejected | Shipped in PR #1044 as an opt-in built-in that defers with a warning when the host supplies no signals. **Open, flagged for the series review**, not settled either way. | Its identifier shape shares the HMAC grammar, and a sign-off row defers host signal processing, so the review decides whether the provider ships in the series. | +| Geo default flip sequenced into the later permission-model step, with an acknowledgment guard | Landed as specified in the same series, with the default of none, `default_country` required and validated against `permissions.yaml`, the `assume_single_jurisdiction` acknowledgment, and a failed lookup resolving to the requires-signal floor with error logging (`GeoStatus`, resolved in core so all adapters agree). | The permission model shipped in PR #1045, so the constraints exist where the draft required them. | +| All adapters serve the full EC feature set identically | Selector behavior is identical through the shared builders and core constructors. The EC API routes (identify, batch-sync, ec/resolve) are Fastly-only, documented in the Spin route list. | The portability adapters do not yet wire platform KV, and the gap is documented rather than silent. | +| Conformance suite, adapter capability matrix, delimiter-free key grammar, `verify`, `legacy_providers`, `versions` / `mint_version` | None of these are in PR #1043 or #1044. All are tracked follow-up work, deferred, not silently dropped. | The shipped seam did not need them, and each returns with the feature that gives it a production caller, per the spec's own minimalism rule. | +| `required_permissions` removed from the device and geo traits, added to the EC trait only at the permission-model step | Present on all three traits from the start, with empty defaults. Core enforces the EC declaration (gate in `EcContext`, landing in PR #1045). No device or geo per-request enforcement point exists, so a nonempty device declaration from a module-supplied provider is refused when the registry resolves the selection and the deployment fails to start. | One uniform declaration seam keeps the interface stable, and refusing at startup what core cannot honor per request avoids the decorative-gate hazard the draft aimed at. The geo circularity stands. | +| Flat crate directories (`crates/trusted-server-geo-fastly`), no placeholder directories | Nested directories per capability (`crates/device/fastly`, `crates/geo/fastly`, `crates/edgecookie/`), flat package names. `crates/edgecookie` ships a README before its first crate. | Nested directories scale per vendor, package names already carry the naming convention, and the README stakes out the vendor location. | diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index e61e3525c8..53e9e9d9a3 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -261,12 +261,12 @@ One vendor per PR, after this change lands. Each moves its Rust, its TypeScript, its config type and its tests into `crates/integrations/`, and the adapter that wants it depends on it. -| Vendor | What it needs | -| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | -| Didomi, Google Tag Manager, Lockr, Osano, Permutive, Sourcepoint | Move as they are. Coupled only through the builder table, deploy validation and the JS map. | -| APS | Also needs the auction provider seam and the generalized renderer contract in §3.4, both of which this change delivers. | -| GPT (the `gpt` proxy and `gpt_diagnostics`) | The proxy moves as it is. The diagnostics half needs the prepare and finalize hooks in §3.5. | -| DataDome | Needs the neutral response-shaping hook in §3.5, and about forty test literals move with it. | +| Vendor | What it needs | +| ---------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Didomi, Google Tag Manager, Lockr, Osano, Permutive, Sourcepoint | Move as they are. Coupled only through the builder table, deploy validation and the JS map. | +| APS | Needs the auction provider seam and the generalized renderer contract in §3.4, both of which this change delivers, and browser-side work it does not deliver, because core TypeScript imports APS directly (§8 item 8). | +| GPT (the `gpt` proxy and `gpt_diagnostics`) | The proxy moves as it is. The diagnostics half needs the prepare and finalize hooks in §3.5. | +| DataDome | Needs the neutral response-shaping hook in §3.5, and about forty test literals move with it. | `[integrations.]` configuration tables need no change, because `IntegrationSettings` is a flattened map that already accepts unknown vendor @@ -349,7 +349,7 @@ that the project pays for today, most recently in PR #1054. ## 8. What implementing this found A probe integration built outside `trusted-server-core` and registered -through an adapter exercised every seam end to end. Seven things surfaced +through an adapter exercised every seam end to end. Eight things surfaced that reading the code did not, and they are recorded here rather than left for each vendor to rediscover. @@ -425,6 +425,23 @@ for each vendor to rediscover. adapter. The other three adapters expose both entry points. Fastly is the primary deployment target, so this one decides whether the seam is usable in production or only in the dev server. +8. **Core TypeScript imports APS directly, so generalizing the Rust + renderer alone does not move APS out.** On `main` at d516a9e94, + `crates/trusted-server-js/lib/src/core/auction.ts:5` imports + `parseApsRendererDescriptor` from `../integrations/aps/render` and calls + it at line 139, `crates/trusted-server-js/lib/src/core/request.ts:2` + imports `dispatchApsRendering` and `renderApsCreative` from the same + module and calls both at lines 56 to 59, and + `crates/trusted-server-js/lib/src/core/types.ts:69` fixes the shared + renderer type with `export type AuctionBidRenderer = ApsRendererV1`. + That coupling is pre-existing on `main` and is introduced by no PR in + this stack. §3.4 does not reach it either, because §3.4 generalizes the + Rust `BidRenderer` enum and the serialized descriptor, not the browser + code that consumes them. Moving APS therefore needs the browser side + generalized too, so that core TypeScript names no vendor. Designing + that, whether as a browser-side renderer registry or in some other + shape, is out of scope for this stack and belongs with the APS + migration in §4. Items 1, 6 and 7 are the ones a vendor meets on its first day, and item 7 decides whether any of this is reachable on the platform most deployments @@ -454,12 +471,13 @@ defines, and both should land before the first vendor is asked to use it. ## Revision record -| Date | Change | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | -| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | -| 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | -| 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | -| 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | -| 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged, and only this status line and this row are new. | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | +| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | +| 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | +| 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | +| 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | +| 2026-08-30 | Moved the five series design specs and the provider-code registry into this PR from PRs #1043 to #1047, so every normative document is reviewed before the code that implements it. Document content is unchanged, and only this status line and this row are new. | | 2026-08-31 | Corrected the counts and line references the review found, against `main` at d516a9e94: the source-file guard counts (§4), the prepare and finalize call-site counts (§3.5), the real double geo resolution on `POST /auction` (§8 item 3), the per-adapter preparer coverage (§8 item 5), and the `settings.rs`, `auction/mod.rs` and `publisher.rs` line references. §6 now requires the round trip on Fastly rather than on any adapter. | +| 2026-09-01 | Answered the review finding that generalizing the Rust `BidRenderer` does not move APS out. Recorded the pre-existing browser-side coupling in core TypeScript as §8 item 8, and corrected the APS migration row in §4 to name the browser-side work. Designing a browser-side renderer contract stays out of scope for this stack. | From 6718cbf95d76cc01234aa9a85f0e4e8c892f78f1 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 1 Sep 2026 05:01:02 +0100 Subject: [PATCH 012/192] Qualify the completeness claim against the browser-side APS coupling Sign-off row 6 said no vendor move needs a core change after this design. Section 8 item 8 now records that TSJS core still imports the APS renderer directly, so an APS move needs a browser renderer contract that this design does not provide. Row 6 now claims only the Rust side, which is what the design actually delivers. --- .../specs/2026-08-27-integration-provider-seam-design.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 53e9e9d9a3..d9eb5692a0 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -464,7 +464,7 @@ defines, and both should land before the first vendor is asked to use it. | 3 | A registration may carry its own browser JavaScript | Proposed | | 4 | Deploy validation moves onto the registration | Proposed | | 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | -| 6 | This change is complete in itself, so after it no vendor move needs a core change | Proposed | +| 6 | This change completes the Rust side, so after it no vendor move needs a Rust core change. The browser side is not complete, because TSJS core still imports the APS renderer directly (§8 item 8), so an APS move still needs a browser renderer contract | Proposed | | 7 | Identity, geo and device providers are capabilities of a module registration (§3.6), the #1043 review's rule applied to all three | Proposed | | 8 | No provider is built into core: HMAC and the User-Agent-only device provider are Tech Lab-owned modules configured under `[integrations.]`, and core keeps only `none` | Proposed | | 9 | This spec and its core implementation precede #1043; 51Degrees implements the core seam, the nine vendor moves in §4 stay one PR each | Proposed | @@ -474,7 +474,7 @@ defines, and both should land before the first vendor is asked to use it. | Date | Change | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | -| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a core change (§8 row 6). | +| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a Rust core change (§8 row 6). The browser side is recorded as outstanding in §8 item 8. | | 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | | 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | From 538642414c42765961f67e76bc1ec814802532bc Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 1 Sep 2026 14:05:42 +0100 Subject: [PATCH 013/192] Format the provider specs for the docs Prettier gate and correct two wording claims Prettier realigns table columns in six specs that the 1 September review edits widened, which the format-docs workflow checks. The client-cycle spec no longer says the origin check normalizes to an effective port, because origins_match compares the port as the browser wrote it and adds or removes nothing. The response-header-hook spec no longer claims Trusted Server rejects a malformed or body-inconsistent content-length, because no core code does that. It is HTTP framing handled by the host HTTP layer, and the point that matters is that no integration reads the value. --- ...26-07-30-client-cycle-ec-resolve-design.md | 23 +++--- ...integration-response-header-hook-design.md | 59 +++++++------ .../2026-07-30-permission-model-design.md | 82 +++++++++---------- .../2026-07-30-pluggable-providers-design.md | 6 +- ...07-30-provider-migration-rollout-design.md | 48 +++++------ ...-08-27-integration-provider-seam-design.md | 24 +++--- 6 files changed, 121 insertions(+), 121 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 8b8b0a9916..1ffde33baf 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -66,7 +66,8 @@ verify against. 1. **Reject cross-site requests with an origin check.** v1 authorizes a request only when its `Origin` header, serialized as scheme, lowercased - host and effective port, is byte-equal to an accepted origin. The + host and the port as the browser wrote it, is byte-equal to an accepted + origin. No port is added or removed on either side. The default accepted set is the single origin `https://{publisher.domain}` and nothing else, so a sibling subdomain, the `http://` scheme and a non-default port are all refused. A missing or foreign `Origin` is @@ -291,13 +292,13 @@ end in tests and demonstrations. ## 8. Revision record vs the 2026-07-31 draft -| Draft position | v1 (PR #1046) | Why | -| ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | -| Origin check via new allowlist config + CSRF token | Exact serialized-origin comparison (scheme, lowercased host, effective port). Default accepted set is the single origin `https://{publisher.domain}`, plus an optional operator-configured list of further exact origins. Missing/foreign `Origin` → `403` | The draft's exact allowlist is adopted. An earlier revision of this spec allowed any suffix match on the apex, which admitted every subdomain including ones the publisher may not control, and its justification did not hold. This closes §7.5 | -| Marker cookie survives whatever happens to the identity it marks | Core expires the `ts-ecr` marker on any request carrying a `ts-ec` the selected provider does not own | The marker is not namespaced by the provider code envelope, so without this a provider switch leaves a visitor with a marker, no identity, and a page script that will not re-post | -| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | -| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | -| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | -| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | -| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | +| Draft position | v1 (PR #1046) | Why | +| ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | +| Origin check via new allowlist config + CSRF token | Exact serialized-origin comparison (scheme, lowercased host, port as written). Default accepted set is the single origin `https://{publisher.domain}`, plus an optional operator-configured list of further exact origins. Missing/foreign `Origin` → `403` | The draft's exact allowlist is adopted. An earlier revision of this spec allowed any suffix match on the apex, which admitted every subdomain including ones the publisher may not control, and its justification did not hold. This closes §7.5 | +| Marker cookie survives whatever happens to the identity it marks | Core expires the `ts-ecr` marker on any request carrying a `ts-ec` the selected provider does not own | The marker is not namespaced by the provider code envelope, so without this a provider switch leaves a visitor with a marker, no identity, and a page script that will not re-post | +| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | +| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | +| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | +| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | +| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index c297d95c9d..dabfb2aa2e 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -389,12 +389,12 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may response fits after all ordinary batches are removed. These are separate outcomes and metrics. - | Adapter | Header-count / total-bytes ceiling (capability cell) | - | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | - | Axum | no platform ceiling below the core budget (native HTTP stack), cell fixed at ≥ 128 headers / ≥ 32 KiB | + | Adapter | Header-count / total-bytes ceiling (capability cell) | + | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | + | Axum | no platform ceiling below the core budget (native HTTP stack), cell fixed at ≥ 128 headers / ≥ 32 KiB | | Fastly | **qualification-pending**: the measured platform ceiling is recorded in this cell by the adapter-qualification commit, and unrecorded ⇒ hook startup fails | - | Cloudflare | **qualification-pending**: same rule | - | Spin | **qualification-pending**: same rule | + | Cloudflare | **qualification-pending**: same rule | + | Spin | **qualification-pending**: same rule | A recorded cell below core's 128-header / 32 KiB budget is a startup error (shrink the core budget or raise the ceiling, never a silent @@ -404,13 +404,13 @@ no-store` in that case regardless of the mutation; `max-age`/`s-maxage` may `qualification-pending` cell fails startup when the depending feature is selected: - | Capability | Fastly | Axum (dev) | Cloudflare | Spin | - | -------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------- | ------------------------------ | -------------------------------- | - | Runtime secret lookup for Vary-HMAC/DataDome | wired secret store, and qualify key-rotation behavior | dev secret binding required | qualify Workers secret binding | qualify component secret binding | - | Persisted processed artifact + mutation IR/read sets | qualification-pending | in-process dev implementation required, non-durable | qualification-pending | qualification-pending | - | Atomic artifact/metadata entry commit | qualification-pending | implementation required | qualification-pending | qualification-pending | - | `Vary` variant index + insert-new-then-index-update rekey | qualification-pending | implementation required | qualification-pending | qualification-pending | - | DataDome field-line order, trusted IP/port, fixed HTTPS backend/no-redirect, and exact form limits | qualification-pending | qualification-pending | qualification-pending | qualification-pending | + | Capability | Fastly | Axum (dev) | Cloudflare | Spin | + | -------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | --------------------------------------------------- | ------------------------------ | -------------------------------- | + | Runtime secret lookup for Vary-HMAC/DataDome | wired secret store, and qualify key-rotation behavior | dev secret binding required | qualify Workers secret binding | qualify component secret binding | + | Persisted processed artifact + mutation IR/read sets | qualification-pending | in-process dev implementation required, non-durable | qualification-pending | qualification-pending | + | Atomic artifact/metadata entry commit | qualification-pending | implementation required | qualification-pending | qualification-pending | + | `Vary` variant index + insert-new-then-index-update rekey | qualification-pending | implementation required | qualification-pending | qualification-pending | + | DataDome field-line order, trusted IP/port, fixed HTTPS backend/no-redirect, and exact form limits | qualification-pending | qualification-pending | qualification-pending | qualification-pending | | SecurityUse JA4 request evidence | platform value available, with exact-field/payload qualification and sign-offs 23/28 pending | unavailable | unavailable | unavailable | The qualification commit records storage lifetime, maximum object size, @@ -492,17 +492,17 @@ The 304 **safe-update set** is exactly `Cache-Control`, `Expires`, `Date`, adding any name requires a reviewed spec/registry revision and conformance fixture, never a runtime wildcard. -| Response | Hook runs? | -| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Processed HTML document (rewritten by TS) | Yes | -| Streamed processed document | Yes, operations apply to the header block before first byte | -| Pass-through proxy response (not processed) | No, TS is a transparent proxy for it | -| Served-from-cache processed document | Serves the **persisted post-hook finals** captured at cache fill (revision-versioned, mismatch = miss), mutators run at fill/processing time only, so a normal hit and a conditional hit return identical policy metadata **by construction**, with no purity or determinism assumption about mutators | -| Redirect (3xx) | No | -| Error responses TS itself generates (4xx/5xx) | No | +| Response | Hook runs? | +| ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Processed HTML document (rewritten by TS) | Yes | +| Streamed processed document | Yes, operations apply to the header block before first byte | +| Pass-through proxy response (not processed) | No, TS is a transparent proxy for it | +| Served-from-cache processed document | Serves the **persisted post-hook finals** captured at cache fill (revision-versioned, mismatch = miss), mutators run at fill/processing time only, so a normal hit and a conditional hit return identical policy metadata **by construction**, with no purity or determinism assumption about mutators | +| Redirect (3xx) | No | +| Error responses TS itself generates (4xx/5xx) | No | | `304 Not Modified` for a processed representation | **Persisted-metadata pass**: a cached processed 200 stores its final post-hook headers, accepted mutation-operation batches, and the union of every mutator's declared response-field read set (the persisted mutation IR), versioned by §3's complete cache revision tuple, including model epoch and logical activation generation. A local conditional hit re-emits persisted finals only when the complete tuple matches current active. An origin-revalidation 304 is staged and diffed against separately stored origin-side metadata. (a) Any byte-coupled field changed (`Content-Encoding`, `Content-Type`, validators, digests) → invalidate and fetch/process a full 200. (b) A changed metadata field that intersects any persisted mutator read set, or an artifact/mutator lacking a complete read-set declaration, is also unsafe → full 200 refetch and ordinary hook execution, because deterministic replay of old operations cannot stand in for re-evaluating a decision made from changed inputs. (c) If every changed field is outside every declared read set and belongs to the enumerated safe-update set (`Cache-Control`, reserved CDN cache fields, `Expires`, `Date`, `Age`, `Vary`, registry-admitted mutable fields), replay the persisted deterministic operations over updated origin metadata and rerun invariants. Updated origin metadata, finals, IR, read sets, and complete revision tuple publish in one atomic entry commit, and changed `Vary` uses insert-new-entry-then-index-update ordering. (d) No change → re-emit persisted finals. Artifact absence or any tuple mismatch triggers an unconditional recovery fetch so TS obtains bytes. For processed-document GET/HEAD routes, TS is explicitly the authoritative server for the transformed representation, and after processing the full 200 it evaluates RFC 9110 §13 preconditions against **processed** validators. This is not evaluation of origin validators by an intermediary cache. Other methods are never eligible for this recovery path. `Set-Cookie` and origin validators are never replayed. Absent metadata → cache miss | -| `HEAD` of a processed document | **Serves the persisted GET artifact when one exists**, parity with GET/304 by construction, since RFC 9111 §4.3.5 lets HEAD metadata update a stored GET response and mutators are not required to be deterministic. With no persisted artifact, HEAD is processed like a GET (headers only) and its finals stored as a **distinct head-only artifact type that never satisfies a later GET**, when a stored GET artifact exists a HEAD may **update** it only when the comparison, made against the separately stored **origin-side** metadata, never the processed artifact's (origin and rewritten lengths differ by construction, so the processed length is never the comparand and HEAD metadata never touches processed-side headers), finds validators and origin `Content-Length` matching and no byte-coupled representation field changed (RFC 9111 §4.3.5); qualifying updates land origin-side under the same atomic-commit discipline as the 304 rules, and any mismatch **invalidates** the stored GET artifact rather than updating it | -| Informational `1xx`, `204`, `205`, `206` | No, enumerated so adapters do not infer independently | +| `HEAD` of a processed document | **Serves the persisted GET artifact when one exists**, parity with GET/304 by construction, since RFC 9111 §4.3.5 lets HEAD metadata update a stored GET response and mutators are not required to be deterministic. With no persisted artifact, HEAD is processed like a GET (headers only) and its finals stored as a **distinct head-only artifact type that never satisfies a later GET**, when a stored GET artifact exists a HEAD may **update** it only when the comparison, made against the separately stored **origin-side** metadata, never the processed artifact's (origin and rewritten lengths differ by construction, so the processed length is never the comparand and HEAD metadata never touches processed-side headers), finds validators and origin `Content-Length` matching and no byte-coupled representation field changed (RFC 9111 §4.3.5); qualifying updates land origin-side under the same atomic-commit discipline as the 304 rules, and any mismatch **invalidates** the stored GET artifact rather than updating it | +| Informational `1xx`, `204`, `205`, `206` | No, enumerated so adapters do not infer independently | This deliberately narrows #782's general "outbound response" phrasing to processed documents (§6). @@ -965,12 +965,11 @@ byte-to-string replacement is forbidden: (`2026-08-27-integration-provider-seam-design.md`) is where that move is designed. This is a direction for that migration rather than a complaint about the code as it stands. -- `content-length` stays a protocol-level singleton. A repeated, malformed, - or body-inconsistent `content-length` is rejected by the shared HTTP - request boundary before any integration runs, which is an HTTP - correctness rule and not a vendor decision, so it is unchanged. - `PostParamLen` is always the byte length of the body actually presented to - core, not the numeric `content-length` value. +- `content-length` is not a vendor decision. It is HTTP framing, parsed by + the host HTTP layer beneath Trusted Server before any integration runs, + and no integration reads it as a value. `PostParamLen` is always the byte + length of the body actually presented to core, not the numeric + `content-length` value, so the header never reaches the vendor payload. - `authorization` never reaches the vendor payload as a value. `AuthorizationLen` is the byte length of the OWS-normalized value. When more than one `authorization` line arrives, every line is presented in @@ -1230,8 +1229,8 @@ This spec supersedes #782 on the following points. The issue is updated to reference this spec when the implementing PR (the hook's return with its first consumer, §7) merges: -| #782 says | This spec says | Why | -| -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | +| #782 says | This spec says | Why | +| -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | Mutations apply to the outbound response generally | Eligibility is the explicit §3a matrix, centered on processed documents | Pass-through/error/304 mutation has different semantics per adapter, and enumerating beats implying | | Ship the trait + registry + adapter application | Additionally: structured operations API (§2), reserved surface (§3), and a real consumer in the same PR (§4, item 3) | PR #838 shipped the trait with zero call sites, and an unrestricted `&mut HeaderMap` cannot enforce any collision policy | diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 6e9438a4bd..fe806cbe22 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -627,19 +627,19 @@ requires a live user signal (§4.2), never a policy change. ## 6. Failure-mode matrix (normative, implemented) -| Condition | Resolution behavior | -| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Condition | Resolution behavior | +| ------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Geo lookup reports a failure | Requires-signal floor for every permission, never the deployer default, with the error logged. No provider shipped today can report one, so a host geo outage lands on the row above instead (§5.2) | -| No geo provider configured | `default_country` baseline, guarded by `assume_single_jurisdiction` (§5.3) | -| Country resolved, no matching rule | `default_country` baseline (§5.4) | -| Region resolved, no region rule | Country rule | -| `default_country` unset or names no rule | Startup failure (§3.3) | -| EC provider configured, no geo, no acknowledgment | Startup failure (§5.3) | -| Malformed `permissions.yaml` | Parse error at settings load, once per instance, because the embedded file is a build-time constant, never per-request | -| Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition), never withdraws, and still honors opt-outs | -| Expired TCF record | Distinct state, not malformed, and treated as absent, so the baseline applies | -| Signals contradict (opt-out plus consent) | Opt-out wins (§4) | -| No EC provider selected | Identity fails closed, with nothing created and an incoming cookie value never used or egressed (§7) | +| No geo provider configured | `default_country` baseline, guarded by `assume_single_jurisdiction` (§5.3) | +| Country resolved, no matching rule | `default_country` baseline (§5.4) | +| Region resolved, no region rule | Country rule | +| `default_country` unset or names no rule | Startup failure (§3.3) | +| EC provider configured, no geo, no acknowledgment | Startup failure (§5.3) | +| Malformed `permissions.yaml` | Parse error at settings load, once per instance, because the embedded file is a build-time constant, never per-request | +| Undecodable record present (TCF, GPP, or USP) | Revokes every Data Use (fail-closed acquisition), never withdraws, and still honors opt-outs | +| Expired TCF record | Distinct state, not malformed, and treated as absent, so the baseline applies | +| Signals contradict (opt-out plus consent) | Opt-out wins (§4) | +| No EC provider selected | Identity fails closed, with nothing created and an incoming cookie value never used or egressed (§7) | The posture is fail-closed. Every ambiguous state resolves to the configured baseline or more restrictive. @@ -789,12 +789,12 @@ their deferred features. This spec supersedes #779 on the following points, so there is one acceptance contract, not two: -| #779 says | This spec says | Why | -| -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | -| Unmatched countries fall to `default_country` | Adopted. Unmatched and unresolved requests both fall to the required `[geo] default_country`, and a **failed** lookup floors instead (§5) | The failure state is the one that must never reach a permissive default, and the draft's `rules.default` split was not kept | -| The full TCF purpose vocabulary is modeled | Adopted and extended. All eleven purposes are signal-resolved, and the full Privacy Taxonomy is carried as declared baseline (§2) | The joint taxonomy work made whole-taxonomy declaration the goal, and `denied` defaults keep undeclared uses inert | -| Policy is an embedded file | Adopted. `permissions.yaml` is compiled into the build (§3.1), and runtime configuration is deferred follow-up | The runtime push and activation pipeline does not exist, and version control is the audit trail meanwhile | -| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signals. Further sources are deferred, and the `ConsentSignal` closure is their seam (§1) | Shipping an interface with no second source repeats the inert-surface mistake, and the extension seam is defined | +| #779 says | This spec says | Why | +| -------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | +| Unmatched countries fall to `default_country` | Adopted. Unmatched and unresolved requests both fall to the required `[geo] default_country`, and a **failed** lookup floors instead (§5) | The failure state is the one that must never reach a permissive default, and the draft's `rules.default` split was not kept | +| The full TCF purpose vocabulary is modeled | Adopted and extended. All eleven purposes are signal-resolved, and the full Privacy Taxonomy is carried as declared baseline (§2) | The joint taxonomy work made whole-taxonomy declaration the goal, and `denied` defaults keep undeclared uses inert | +| Policy is an embedded file | Adopted. `permissions.yaml` is compiled into the build (§3.1), and runtime configuration is deferred follow-up | The runtime push and activation pipeline does not exist, and version control is the audit trail meanwhile | +| Permission sources are open-ended (#777: publisher interaction, external services may grant) | Sources are jurisdiction, policy, and the §4 signals. Further sources are deferred, and the `ConsentSignal` closure is their seam (§1) | Shipping an interface with no second source repeats the inert-surface mistake, and the extension seam is defined | ## 11. Revision record vs the 2026-07-31 draft @@ -802,26 +802,26 @@ One row per divergence between the draft and the implementation this revision was verified against (branch `split/5-response-hook-docs`, PR #1045). -| Draft position | Implemented position | Why | -| --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Vocabulary is two TCF-purpose identifiers, enforced permissions only (§2) | IAB Privacy Taxonomy Data Uses, with eleven named purposes all signal-resolved, plus 53 taxonomy Data Uses carried as declared but unenforced baseline flags | The joint taxonomy adoption postdates the draft, and whole-taxonomy declaration serves completeness and demonstration, with `denied` defaults keeping unenforced flags inert | -| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Policy is `permissions.yaml`, compiled into the build with `include_str!`, parsed once and covered by tests | The runtime config push and activation apparatus does not exist, so publishing runtime policy without it would recreate the hazards the draft cataloged. Runtime policy is deferred follow-up | -| Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated, and the field returns with that work | -| Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted. A detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema. `requires_signal` is now expressible per rule | -| Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated, and a failed lookup floors separately | One deployer knob is simpler, and the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | -| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first, and the EU/EEA coverage test guards the shipped table against the typo class. ISO-assignment checks are future hardening | -| Three-class signal taxonomy with regime-scoped grant acceptance, where the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source, the US posture is a `granted` baseline that opt-outs revoke, and explicit non-opt-out values grant nothing | A simpler two-signal model without regimes. The cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | -| Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification, and per-family scoping needs the full §4.5 decoder work | -| Normalization runs expiry before conflict resolution, a declared change (§4.4) | Conflict resolution still runs before the expiry check | The reordering was not implemented, though the expired state itself (distinct from malformed, absent for acquisition) was adopted | -| Persisted-KV consent flows through the full normalization pipeline with an explicit TTL comparison (§4.4) | The loaded record substitutes directly when the request carries no signals, jurisdiction re-derived, and staleness is enforced by the store TTL (`max_consent_age_days`) | The store-level TTL delivers the staleness bound without a second normalization pass | -| Proxy mode gains minimal opt-out extraction (§4.4) | Proxy mode still skips decoding, so a present record blocks all grants via the malformed-present rule and the GPC header opt-out is honored without decoding | The permission-layer outcome is equally or more restrictive with no new decode paths, and this is revisited with the §4.5 decoder work | -| Withdrawal has four triggers including an explicit storage-withdrawal or authenticated deletion request (§4.2) | The TCF Purpose 1 refusal under a non-granted baseline is the only trigger, and opt-outs, malformed records, absence, and policy changes never withdraw (adopted) | No deletion endpoint exists to carry the extra trigger, so the narrowest destructive surface shipped first | -| §4.3 durability protocol: family records first, suppression and authority-state records, outbox, breaker, strong reads | Cookie expiry plus best-effort identity-graph tombstones per presented identifier, with failures logged | The protocol requires storage primitives (linearizable CAS, independent durability domains) the adapters do not yet qualify, so it is deferred with the providers-spec storage work | -| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out. The revoke set is policy-declared, shipped as `all` (which also drops storage) | The full decoder and registry vendoring are their own project, and the policy-declared revoke set gives deployers the scoping lever meanwhile | -| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists, and the built binary is the policy identity | With no runtime policy there is nothing to activate, and the draft remains the reference design for the runtime-config follow-up | -| §3.4 single jurisdiction truth, and §7 dispatch gated on the policy regime with a contextual projection | Auction dispatch keeps the consent-subsystem gate (effective TCF Purpose 1 for GDPR or unknown jurisdictions). `detect_jurisdiction` and its lists remain, with no contextual view | Dispatch migration is follow-up. The legacy-list drift risk the draft named still stands and is recorded rather than resolved | -| Every raw-EC egress path is pair-gated, with per-row tests and a denylist check (§7) | Pair gating is centralized in `ec_sharing_allowed` (auction endpoint `user.id`, publisher navigation and page-bids `user.id`, identify, pull sync) and `gate_eids_by_permissions` (EIDs everywhere). Batch sync checks row state only | Partial adoption. Aligning the remaining paths, the S2S stored-provenance authority, and the inventory tests is recorded follow-up | -| Identity rows never store raw consent strings, only normalized provenance and a digest (§1) | The identity-graph entry stores the raw TCF and GPP strings with the row | The normalized provenance schema belongs to the providers-spec storage work, and until then rows carry the raw strings | -| No signals block in policy, and the signal mapping is fixed in the spec | New. A `signals` section in `permissions.yaml` declares the TCF purpose map, opt-out sources, and revoke set, with `tcf.authoritative` governing only TCF's own effect | Moves signal policy from code into deployer-editable data, and the flag can never let a TCF record override an opt-out, preserving §4 precedence | -| The §5.3 no-geo guard covers every jurisdiction consumer | The guard fires when an Edge Cookie provider is configured with no geo provider | The EC provider is the only policy-gated consumer today, and the trigger list grows when dispatch and further egress paths join the model | -| `default_country` is required only in the acknowledged static no-geo mode (§5.4) | Required always and startup-validated against `permissions.yaml` | It is the baseline for unmatched requests in every mode, so it must always exist | +| Draft position | Implemented position | Why | +| --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Vocabulary is two TCF-purpose identifiers, enforced permissions only (§2) | IAB Privacy Taxonomy Data Uses, with eleven named purposes all signal-resolved, plus 53 taxonomy Data Uses carried as declared but unenforced baseline flags | The joint taxonomy adoption postdates the draft, and whole-taxonomy declaration serves completeness and demonstration, with `denied` defaults keeping unenforced flags inert | +| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Policy is `permissions.yaml`, compiled into the build with `include_str!`, parsed once and covered by tests | The runtime config push and activation apparatus does not exist, so publishing runtime policy without it would recreate the hazards the draft cataloged. Runtime policy is deferred follow-up | +| Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated, and the field returns with that work | +| Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted. A detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema. `requires_signal` is now expressible per rule | +| Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required `[geo] default_country` covers unmatched and unresolved requests in every mode, startup-validated, and a failed lookup floors separately | One deployer knob is simpler, and the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | +| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and unknown fields on detailed rules | Smaller surface shipped first, and the EU/EEA coverage test guards the shipped table against the typo class. ISO-assignment checks are future hardening | +| Three-class signal taxonomy with regime-scoped grant acceptance, where the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | TCF is the only grant source, the US posture is a `granted` baseline that opt-outs revoke, and explicit non-opt-out values grant nothing | A simpler two-signal model without regimes. The cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | +| Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Any present-but-undecodable record revokes every Data Use for the request | Strictly more restrictive simplification, and per-family scoping needs the full §4.5 decoder work | +| Normalization runs expiry before conflict resolution, a declared change (§4.4) | Conflict resolution still runs before the expiry check | The reordering was not implemented, though the expired state itself (distinct from malformed, absent for acquisition) was adopted | +| Persisted-KV consent flows through the full normalization pipeline with an explicit TTL comparison (§4.4) | The loaded record substitutes directly when the request carries no signals, jurisdiction re-derived, and staleness is enforced by the store TTL (`max_consent_age_days`) | The store-level TTL delivers the staleness bound without a second normalization pass | +| Proxy mode gains minimal opt-out extraction (§4.4) | Proxy mode still skips decoding, so a present record blocks all grants via the malformed-present rule and the GPC header opt-out is honored without decoding | The permission-layer outcome is equally or more restrictive with no new decode paths, and this is revisited with the §4.5 decoder work | +| Withdrawal has four triggers including an explicit storage-withdrawal or authenticated deletion request (§4.2) | The TCF Purpose 1 refusal under a non-granted baseline is the only trigger, and opt-outs, malformed records, absence, and policy changes never withdraw (adopted) | No deletion endpoint exists to carry the extra trigger, so the narrowest destructive surface shipped first | +| §4.3 durability protocol: family records first, suppression and authority-state records, outbox, breaker, strong reads | Cookie expiry plus best-effort identity-graph tombstones per presented identifier, with failures logged | The protocol requires storage primitives (linearizable CAS, independent durability domains) the adapters do not yet qualify, so it is deferred with the providers-spec storage work | +| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out. The revoke set is policy-declared, shipped as `all` (which also drops storage) | The full decoder and registry vendoring are their own project, and the policy-declared revoke set gives deployers the scoping lever meanwhile | +| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists, and the built binary is the policy identity | With no runtime policy there is nothing to activate, and the draft remains the reference design for the runtime-config follow-up | +| §3.4 single jurisdiction truth, and §7 dispatch gated on the policy regime with a contextual projection | Auction dispatch keeps the consent-subsystem gate (effective TCF Purpose 1 for GDPR or unknown jurisdictions). `detect_jurisdiction` and its lists remain, with no contextual view | Dispatch migration is follow-up. The legacy-list drift risk the draft named still stands and is recorded rather than resolved | +| Every raw-EC egress path is pair-gated, with per-row tests and a denylist check (§7) | Pair gating is centralized in `ec_sharing_allowed` (auction endpoint `user.id`, publisher navigation and page-bids `user.id`, identify, pull sync) and `gate_eids_by_permissions` (EIDs everywhere). Batch sync checks row state only | Partial adoption. Aligning the remaining paths, the S2S stored-provenance authority, and the inventory tests is recorded follow-up | +| Identity rows never store raw consent strings, only normalized provenance and a digest (§1) | The identity-graph entry stores the raw TCF and GPP strings with the row | The normalized provenance schema belongs to the providers-spec storage work, and until then rows carry the raw strings | +| No signals block in policy, and the signal mapping is fixed in the spec | New. A `signals` section in `permissions.yaml` declares the TCF purpose map, opt-out sources, and revoke set, with `tcf.authoritative` governing only TCF's own effect | Moves signal policy from code into deployer-editable data, and the flag can never let a TCF record override an opt-out, preserving §4 precedence | +| The §5.3 no-geo guard covers every jurisdiction consumer | The guard fires when an Edge Cookie provider is configured with no geo provider | The EC provider is the only policy-gated consumer today, and the trigger list grows when dispatch and further egress paths join the model | +| `default_country` is required only in the acknowledged static no-geo mode (§5.4) | Required always and startup-validated against `permissions.yaml` | It is the baseline for unmatched requests in every mode, so it must always exist | diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index 8bfa1840a7..3585f58336 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -125,7 +125,7 @@ an EC value through the selected provider: | **Create** | EC generation on first eligible request, and the client-cycle resolve endpoint | The provider returns the identifier (`generate` server-side, `resolve_from_client` for the client cycle) and only core writes the cookie, after enforcing the global bounds below. | | **Recognize** | Reading `ts-ec` back from the request, deciding `ec_was_present`, withdrawal checks, and every path that hands the value onward: the origin URL in `append_ec_id`, the click-target URL in `handle_first_party_click`, and the proxied body an integration builds | `accepts_id` answers whether a value is a well-formed identifier the provider issues. A value the selected provider does not recognize is treated as absent, so it is never used or egressed, while the raw cookie value stays visible to withdrawal handling. The egress paths reach the same answer through `edge_cookie::recognized_ec_id`, and a deployment with no provider selected recognizes nothing and so egresses nothing. | | **KV key** | Identity-graph row reads and writes | `normalize_id_for_kv` returns the key form. The default lowercases the built-in HMAC hash segment and preserves the suffix, keeping today's keys. An opaque or case-sensitive provider overrides to the identity function so distinct identifiers never collapse into one row. | -| **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader, because it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | +| **Withdraw** | Expiring the cookie and writing revocation markers | The identifiers eligible for a **graph tombstone** are exactly those the selected provider owns, dispatched on the `{code}~` prefix first and then `accepts_id`, never a shape check the provider cannot influence. Expiring the **cookie** is broader, because it keys off the raw cookie being present, so it still fires for an identifier the selected provider does not own (see the switching case, §6.1). | **Invariant:** for every provider `P` and every identifier `id` created by `P`, `id` round-trips read-back byte for byte. A test in `ec/mod.rs` proves @@ -375,7 +375,7 @@ when the provider is built, stopping the request rather than degrading. | `provider = "none"` (explicit stateless) | Valid, and means exactly what omitting the selector means. Any configured provider block alongside it is a startup error, the same stray-block rule as below. | | A configured `[ec.providers.]` block that is not the selected one | **Startup error** (checked for the `hmac` block and every vendor block). An unreferenced block is almost always a mistyped selector or a stale block, and accepting it silently invites configuration drift. | | A selected vendor key whose provider the adapter did not inject | Loud failure when the provider is built, naming the key, so the deployment never silently runs stateless. | -| `provider = "host-signals"` on a host that supplies no signals | Loud failure when the provider is built. A host that cannot produce `HostSignals` cannot run the provider. | +| `provider = "host-signals"` on a host that supplies no signals | Loud failure when the provider is built. A host that cannot produce `HostSignals` cannot run the provider. | | `provider = "client-fixed"` in a production build | Startup error. The demonstration provider is compiled only behind the `client-fixed-demo` cargo feature. | | No `provider`, no providers block | Valid, the neutral default for that concern. | | Deprecated `[ec] passphrase` | Migrated to `provider = "hmac"` with the passphrase in `[ec.providers.hmac]`, with a deprecation warning naming the new location. Both forms together are rejected so a half-edited file fails loudly instead of one form silently winning. | @@ -473,7 +473,7 @@ logged, none silent: | `generate` returns an error | No identity this request. The organic caller logs at error level and the request proceeds stateless. No cookie is written. | | A provider creates an identifier outside the global bounds | Rejected at create, never rewritten. The organic path yields no identity. The resolve endpoint returns 400. | | Identity-graph write fails at create | The create is undone (no identifier, no cookie), with the error logged. The resolve endpoint returns 503. The next eligible request retries. | -| The host-signals provider finds no TLS/HTTP-2 signals | Defers with a warning. No identity this request, and no degraded IP-only identifier is created under the host-signals name. | +| The host-signals provider finds no TLS/HTTP-2 signals | Defers with a warning. No identity this request, and no degraded IP-only identifier is created under the host-signals name. | | Geo lookup **fails** (the provider errors) | Every permission resolves to the requires-signal floor, and the failure is logged at error level. The failure is **not** papered over with the `default_country` baseline. | | Geo resolves **no location**, or a country/region with no rule | The `[geo] default_country` baseline applies. This is the configured-default case, deliberately distinct from the failure row above (`GeoStatus` in `ec/consent.rs`). | | An incoming cookie value fails the bounds at read-back | Treated as absent, with a warning naming the source. | diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index bea38238c4..8d8d4c3ce7 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -125,16 +125,16 @@ for the follow-up work that will implement them. | 3a | US-state request, explicit not-opted-out GPP/USP value → EC allowed | No US signal grants anything. N/A, absent, reserved, and unknown values grant nothing | Implemented (#1045) in a stricter form than drafted, where signals only revoke, so the drafted "explicit not-opted-out may grant" class does not exist (sign-offs 3, 17) | | 3b | US-state request, TCF record refusing Purpose 1, no US opt-out → no EC | Refusal beats coexisting non-TCF grant signals | Implemented (#1045), where an authoritative TCF refusal revokes its mapped uses. Under the shipped `granted` US baseline the refusal suppresses without tombstoning | | 3c | Consent-record conflict modes, expiry, KV fallback, proxy mode | Per the permission spec §4.4 matrix, whose changed row is that malformed-present blocks acquisition | Malformed-present fail-closed is implemented (#1045). The rest of the consent normalization pipeline is carried forward unchanged by the series | -| 3d | Valid plus expired consent records, where conflict resolution can select the expired record | Expired sources drop before conflict resolution | Open. Not addressed by the series | +| 3d | Valid plus expired consent records, where conflict resolution can select the expired record | Expired sources drop before conflict resolution | Open. Not addressed by the series | | 3e | Only the GPP sale field (and USP) is consulted, with sharing/targeted opt-outs ignored | Sale, sharing, and targeted-advertising opt-outs deny the personalised-ads uses, and none affects storage or destroys identity | Partially implemented (#1045), where sale, USP, and Sec-GPC are honored and never destructive. The sharing and targeted-advertising GPP fields are not yet decoded (sign-off 3 open) | | 3f | Non-privacy-state US traffic (for example Wyoming) is non-regulated → EC allowed | Country-level `US` is a protective floor, region rules may be stricter, and regionless traffic never degrades to non-regulated | Implemented (#1045), where `permissions.yaml` maps country `US` to `us-opt-out`, with state rows able to override | -| 3g | Graph rows persist JA4 class, H2 signal hash, and buyer-facing quality metadata | The draft discontinued these for new rows | **Contradicted by the series, flagged.** Rows still persist device signals, including signal hash prefixes, when the opt-in providers run (#1044/#1046). Tied to the open host-signal question (sign-off 22) | +| 3g | Graph rows persist JA4 class, H2 signal hash, and buyer-facing quality metadata | The draft discontinued these for new rows | **Contradicted by the series, flagged.** Rows still persist device signals, including signal hash prefixes, when the opt-in providers run (#1044/#1046). Tied to the open host-signal question (sign-off 22) | | 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB with citation and sign-off | **The shipped default adopts `granted` storage for GB (#1045), flagged.** The citation and sign-off the draft required do not exist yet. The task force owns this row | | 5 | No country resolvable (geo failure) → no EC (fail-closed) | Protective failure profile, where permissions resolve at the requires-signal floor and `default_country` is reserved for unmatched requests in acknowledged static-jurisdiction mode | Implemented (#1045), where a failed lookup resolves at the requires-signal floor, logged at error level, and never falls back to `default_country` (sign-off 18) | | 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, identity never tombstoned | Refusal blocks new grants everywhere, and existing identity is never tombstoned where the baseline is `granted` | Implemented (#1045), where an authoritative refusal revokes its mapped uses everywhere and withdrawal is scoped to non-granted baselines | | 7 | Country resolved but in no regulation list → EC created, EIDs pass through | Governed by the deployment's default rule. The implementation expresses this as the required `[geo] default_country`, naming the `permissions.yaml` rule for unmatched requests | Implemented (#1045) with a changed mechanism, since no `rules.default` entry exists and `default_country` is required and validated at startup | | 8 | Opt-out signal outside US states → ignored today | Mapped use restrictions are honored globally, and opt-outs never tombstone identity | Implemented (#1045), where the signal mapping is jurisdiction-free and suppresses even TCF-consented uses, without destruction (sign-off 1) | -| 9 | Fastly bot gate requires JA4 plus platform class before KV-backed EC writes | The draft deferred host signal processing and startup-failed `[device] provider = "fastly"` | **Contradicted by the series, flagged.** The `fastly` device provider ships opt-in with `builtin` (UA-only) as the default (#1044). Whether the host-signal surface stays is sign-off 22, open | +| 9 | Fastly bot gate requires JA4 plus platform class before KV-backed EC writes | The draft deferred host signal processing and startup-failed `[device] provider = "fastly"` | **Contradicted by the series, flagged.** The `fastly` device provider ships opt-in with `builtin` (UA-only) as the default (#1044). Whether the host-signal surface stays is sign-off 22, open | | 10 | Fastly always resolves geo per request | Only with `[geo] provider = "platform"`. The neutral default flips only together with the permission model's jurisdiction guard, never in an intermediate step | Implemented as sequenced (#1044 kept the platform default, and #1045 flipped geo off by default together with `default_country` and the acknowledgment guard) | | 11a | Raw EC egress on jurisdiction-gated paths today (`user.id`, EIDs, identify, pull sync) | Gated by the sharing pair (storage plus personalised-ads), at least as strict as today for every path | Implemented (#1045), where `ec_sharing_allowed` gates `user.id`, the identify response, and pull sync, and `gate_eids_by_permissions` gates EIDs, all on the same pair | | 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header without today's jurisdiction gate | Gated by the egress inventory (both purposes) | Open. Not implemented by the series (sign-off 8) | @@ -469,23 +469,23 @@ the code with the row open. | 6 | Raw regulatory strings reach only the positively registered OpenRTB field that requires each source. All other destinations default deny. Identity rows retain normalized provenance/digests, not raw consent snapshots. | permission §7; providers §6.3 | (none) | open. Not addressed by the series | | 7 | Reject legacy batch-sync traffic until live-browser provenance backfill makes the row re-evaluable. | rollout §6 item 6; permission §7 | (none) | open. Not implemented (no provenance exists to recompute) | | 8 | Gate proxy, click, and Testlight identity forwarding on the sharing pair (storage plus personalised-ads). | §2 row 11b | (none) | open. Not implemented by the series | -| 9 | Defer integration-owned cookie operations from the v1 response hook, and require a complete read/use/withdraw lifecycle before admission. | hook §3 | (none) | open. Overtaken (#1047 removed the whole hook from the series, so no cookie surface shipped). The deferral returns with the hook's first consumer | +| 9 | Defer integration-owned cookie operations from the v1 response hook, and require a complete read/use/withdraw lifecycle before admission. | hook §3 | (none) | open. Overtaken (#1047 removed the whole hook from the series, so no cookie surface shipped). The deferral returns with the hook's first consumer | | 10 | Do not create a blanket session-cookie exemption. Every cookie must be covered by an approved permission or narrowly defined security-use authority. | hook §3 | (none) | open. Hook not shipped, unaffected | | 11 | Require a durable per-family negative-intent outbox in a failure domain independent of its strong target and checked freshly by every identity consumer, with a globally visible breaker over positive identity operations when neither can commit. | permission §4.3 | (none) | open. Not implemented (part of the durable-suppression follow-up) | | 12 | Adapters that cannot meet the revocation-storage contract migrate stateless rather than weakening the contract. | rollout §6 item 2; recipe §5 | (none) | open. Direction implemented (`provider = "none"` spells stateless in #1043; the identity endpoints were already Fastly-only before the series, and #1046 keeps `resolve` on the same footing). The formal capability gate is deferred. Ratify the direction | -| 13 | Keep batch sync fail-closed at cutover, and stage partner communication and cleanup using explicit coverage thresholds, windows, and pause actions. | rollout §6 item 6 | (none) | open. Not implemented (no provenance cutover exists yet) | +| 13 | Keep batch sync fail-closed at cutover, and stage partner communication and cleanup using explicit coverage thresholds, windows, and pause actions. | rollout §6 item 6 | (none) | open. Not implemented (no provenance cutover exists yet) | | 14 | Policy tightening does not reinterpret historical refusal as a destructive event. Destructive withdrawal requires fresh, live qualifying evidence. | permission §4.2 trigger 2 | (none) | open. Implemented by #1045 (withdrawal evaluates the live request's TCF record only, and historical records are never reinterpreted). Ratify | -| 15 | Descope the client cycle and `rewrite_legacy`, and ship the v1 integration hook as headers-only. | client spec status; providers §6.1; hook §3 | (none) | open. Overtaken (the client cycle shipped hardened as #1046 instead of descoped, `rewrite_legacy` does not exist, and the hook shipped not at all per #1047). Re-decide against the shipped shape | +| 15 | Descope the client cycle and `rewrite_legacy`, and ship the v1 integration hook as headers-only. | client spec status; providers §6.1; hook §3 | (none) | open. Overtaken (the client cycle shipped hardened as #1046 instead of descoped, `rewrite_legacy` does not exist, and the hook shipped not at all per #1047). Re-decide against the shipped shape | | 16 | Persist use-opt-out suppression until ordered explicit authorization for that use or identity deletion, with TCF `LastUpdated` or an authenticated monotonic revision proving order. | permission §4.3 | (none) | open. Not implemented (suppression in the series is request-scoped with no durable record) | | 17 | N/A, absent, reserved, unknown, and unsupported values never grant processing. | permission §4.5 | (none) | open. Implemented by #1045 in a stricter form (signals only revoke, so no US-signal value grants anything). Ratify | | 18 | A selected geo provider's lookup failure uses the compiled-in protective profile; `default_country` is only for acknowledged static-jurisdiction mode. | permission §5.2 | (none) | open. Implemented by #1045 (failed lookups resolve at the requires-signal floor, logged at error level, never `default_country`). Ratify | | 19 | Use immutable version-addressed whole-config publication plus prepare/commit activation of the complete tuple, with authenticated fleet membership, bounded admission lease, quiescence, and an activation journal. A second unanimous model transition advances model epoch, minimum binary generation, and row schema floor atomically. | permission §5.5; rollout §6.1 | (none) | open. Not implemented (`ts config push` publishes and validates without the activation protocol) | | 20 | N+1 keeps v1 creation and pre-epic live gating, reads/enforces N+2 negative state for rollback safety, and never originates durable use suppression. New-shape settings alone do not activate the new writer/model. | migration §4.4 | (none) | open. Overtaken in part (the shipped migration is a one-release dual-read of `[ec] passphrase` in #1043 with mixed forms rejected and nothing durable added, so the full interim waits for the durable design) | | 21 | Expire and re-create rowless legacy cookies without continuity. A prefix match cannot authenticate the cookie suffix. | providers §5 | (none) | open. Not implemented (no rowless classification exists, and a cookie with no row is never shared) | -| 22 | Defer host JA4/H2 signal processing to a separate approved design. Reject `[device] provider = "fastly"` at startup and do not persist signal-derived classifications. | providers §5 | (none) | open. **Contradicted by the series and flagged for review** (#1044 ships the `fastly` device provider and the host-signal EC provider opt-in, and device signals including signal hash prefixes persist in rows) | +| 22 | Defer host JA4/H2 signal processing to a separate approved design. Reject `[device] provider = "fastly"` at startup and do not persist signal-derived classifications. | providers §5 | (none) | open. **Contradicted by the series and flagged for review** (#1044 ships the `fastly` device provider and the host-signal EC provider opt-in, and device signals including signal hash prefixes persist in rows) | | 23 | Permit a narrow `SecurityUse` authority for DataDome only, with the exact bounded surface the hook spec defines. | hook §4a; permission §7 | (none) | open. Hook not shipped, unaffected | | 24 | Malformed/absence suppression overrides a permissive baseline but clears on newer valid evidence. It is not sticky like an explicit use opt-out. | permission §4.3, §4.1 | (none) | open. Implemented by #1045 by construction (the fail-closed block is request-scoped, so newer valid evidence re-resolves). Ratify together with row 16's durable design | -| 25 | Enforce a stored-jurisdiction/provenance horizon for batch sync, where moves into stricter regimes fail closed by the horizon, and moves out require a live visit. | permission §7 | (none) | open. Not implemented | +| 25 | Enforce a stored-jurisdiction/provenance horizon for batch sync, where moves into stricter regimes fail closed by the horizon, and moves out require a live visit. | permission §7 | (none) | open. Not implemented | | 26 | Aggregate embedded GPP GPC with `Sec-GPC` by OR as a global, non-destructive use opt-out. | permission §4.5 | (none) | open. Partially implemented by #1045 (`Sec-GPC` is an honored non-destructive source). The embedded GPP GPC subfield is not read | | 27 | In proxy mode, decode only mapped opt-out fields and derive no grants. | permission §4.4 | (none) | open. Partially (the shipped model derives no grants from any US signal anywhere). The proxy-mode decode restriction is not separately implemented | | 28 | Require product and written vendor conformance approval for the reduced DataDome surface the hook spec pins. | hook §4a.2 | (none) | open. Hook not shipped, unaffected | @@ -498,20 +498,20 @@ the code with the row open. ## Revision record vs the 2026-07-31 draft -| Draft position | Revised position | Why | -| -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Epic #777-#781 pending sign-off, no implementation | Implemented as five stacked PRs #1043-#1047, with §8 staying the ledger and rows marked for ratification | The series shipped the seam, selection, permission model, resolve endpoint, and docs, so the task force ratifies rather than re-litigates | +| Draft position | Revised position | Why | +| ----------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Epic #777-#781 pending sign-off, no implementation | Implemented as five stacked PRs #1043-#1047, with §8 staying the ledger and rows marked for ratification | The series shipped the seam, selection, permission model, resolve endpoint, and docs, so the task force ratifies rather than re-litigates | | `[device] provider = "fastly"` startup-fails pending a separate host-signal design (§2 row 9, row 22) | The `fastly` device provider and host-signal EC provider ship opt-in, and device signals persist in graph rows. The identifier-collision defect the review found (host-signal identifiers shared the HMAC grammar and keyspace) is fixed by the mandatory provider-code envelope, so host-signal identifiers are `hs00~` namespaced. The policy question of row 22 is unchanged by the collision fix | Implementation choice, deliberately flagged as the open review question rather than presented as settled | -| US recipe: `requires_signal` with an extended grant-signal class (§2 rows 3/3a) | Shipped default: `granted` US baseline, `revokes: all` on any opt-out, and no US signal grants anything | The shipped signal model is revoke-only, which is simpler and stricter on grants, and the baseline choice is deployer-editable yaml, flagged in rows 3 and 5 | -| `[permissions]` TOML policy published at runtime | `permissions.yaml` compiled into the build, no `[permissions]` block | Policy stays reviewable in version control and the partial-policy trap cannot be written | -| `rules.default` worldwide default entry (§2 row 7) | Required `[geo] default_country` naming the fallback rule, validated at startup | Same role, one mechanism, loud when missing | -| Graph store mandatory at startup for creating providers (§2 row 12, §4.4b) | No startup requirement, and with no graph no identifier is created, per request | The phantom-cookie rule holds without a breaking startup change, and graphless deployments run identity-less | -| `legacy_providers` reader chain for provider switches (§4.8) | Not implemented, so a switch restarts identity | Deferred until a second server-side provider makes switching real | -| N+1/N+2 negative-record machinery, model epochs, `m00` mirror (§4.1, row 20) | Deferred with rows 11, 16, 19, 20. The shipped dual-read is one release of `[ec] passphrase` mapping with mixed forms rejected | Nothing durable beyond pre-existing withdrawal tombstones ships, so binary rollback strands no new state | -| Staged activation, `push_sequence`, quiescence, `ts config gc` (§6.3, §6.1) | Basic `ts config push` / `ts config validate` only | The activation protocol belongs to the deferred durable-suppression rollout (row 19) | -| Migration guide with committed per-adapter fixtures and a full gated metric set (§5, §6.4) | `configuration.md` / `edge-cookies.md` document the migrated shape, while fixtures, the guide page, and metrics remain outstanding | Documentation shipped for configuration, and the operational guide is the remaining deliverable before a release | -| Client cycle descoped and hook shipped headers-only (row 15) | The client cycle shipped hardened (#1046) and the hook shipped not at all (#1047) | Hardening replaced descoping, and the hook had no consumer, which is the hook spec's own admission rule | -| Example config ships the migrated happy path uncommented (§4.9) | Identity off by default with selector and block commented together, and validation closes the silent-stateless trap | Loud startup validation, not an uncommented default, is what prevents PR #838's silent-stateless state | -| Environment override documented but `#[cfg(test)]`-only in PR #838 (§7) | Override applied as typed EdgeZero app-config overlays at `ts config push`; the existing CLI overlay test covers the mechanism, and a provider-specific override test is still to write | The same compiled binary switches providers at deployment through the published configuration | -| Pinned known-answer HMAC vectors committed (§3) | Stability tested per inputs, and the pinned vector is still to commit | The cross-version CI pin remains open work under §3 | -| GB storage baseline change only with citation and sign-off (§2 row 4) | The shipped `permissions.yaml` adopts `granted` storage for GB without a recorded citation | Flagged in row 4, since the task force owns the decision and its record | +| US recipe: `requires_signal` with an extended grant-signal class (§2 rows 3/3a) | Shipped default: `granted` US baseline, `revokes: all` on any opt-out, and no US signal grants anything | The shipped signal model is revoke-only, which is simpler and stricter on grants, and the baseline choice is deployer-editable yaml, flagged in rows 3 and 5 | +| `[permissions]` TOML policy published at runtime | `permissions.yaml` compiled into the build, no `[permissions]` block | Policy stays reviewable in version control and the partial-policy trap cannot be written | +| `rules.default` worldwide default entry (§2 row 7) | Required `[geo] default_country` naming the fallback rule, validated at startup | Same role, one mechanism, loud when missing | +| Graph store mandatory at startup for creating providers (§2 row 12, §4.4b) | No startup requirement, and with no graph no identifier is created, per request | The phantom-cookie rule holds without a breaking startup change, and graphless deployments run identity-less | +| `legacy_providers` reader chain for provider switches (§4.8) | Not implemented, so a switch restarts identity | Deferred until a second server-side provider makes switching real | +| N+1/N+2 negative-record machinery, model epochs, `m00` mirror (§4.1, row 20) | Deferred with rows 11, 16, 19, 20. The shipped dual-read is one release of `[ec] passphrase` mapping with mixed forms rejected | Nothing durable beyond pre-existing withdrawal tombstones ships, so binary rollback strands no new state | +| Staged activation, `push_sequence`, quiescence, `ts config gc` (§6.3, §6.1) | Basic `ts config push` / `ts config validate` only | The activation protocol belongs to the deferred durable-suppression rollout (row 19) | +| Migration guide with committed per-adapter fixtures and a full gated metric set (§5, §6.4) | `configuration.md` / `edge-cookies.md` document the migrated shape, while fixtures, the guide page, and metrics remain outstanding | Documentation shipped for configuration, and the operational guide is the remaining deliverable before a release | +| Client cycle descoped and hook shipped headers-only (row 15) | The client cycle shipped hardened (#1046) and the hook shipped not at all (#1047) | Hardening replaced descoping, and the hook had no consumer, which is the hook spec's own admission rule | +| Example config ships the migrated happy path uncommented (§4.9) | Identity off by default with selector and block commented together, and validation closes the silent-stateless trap | Loud startup validation, not an uncommented default, is what prevents PR #838's silent-stateless state | +| Environment override documented but `#[cfg(test)]`-only in PR #838 (§7) | Override applied as typed EdgeZero app-config overlays at `ts config push`; the existing CLI overlay test covers the mechanism, and a provider-specific override test is still to write | The same compiled binary switches providers at deployment through the published configuration | +| Pinned known-answer HMAC vectors committed (§3) | Stability tested per inputs, and the pinned vector is still to commit | The cross-version CI pin remains open work under §3 | +| GB storage baseline change only with citation and sign-off (§2 row 4) | The shipped `permissions.yaml` adopts `granted` storage for GB without a recorded citation | Flagged in row 4, since the task force owns the decision and its record | diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index d9eb5692a0..cec7ca0ffb 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -457,24 +457,24 @@ defines, and both should land before the first vendor is asked to use it. ## 9. Sign-off -| # | Decision | Status | -| --- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | -| 1 | Vendor integrations belong outside core, behind the registration contract | Proposed | -| 2 | Tech Lab engineering reviews vendor crates, and does not maintain them | Proposed, governance | -| 3 | A registration may carry its own browser JavaScript | Proposed | -| 4 | Deploy validation moves onto the registration | Proposed | -| 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | -| 6 | This change completes the Rust side, so after it no vendor move needs a Rust core change. The browser side is not complete, because TSJS core still imports the APS renderer directly (§8 item 8), so an APS move still needs a browser renderer contract | Proposed | -| 7 | Identity, geo and device providers are capabilities of a module registration (§3.6), the #1043 review's rule applied to all three | Proposed | -| 8 | No provider is built into core: HMAC and the User-Agent-only device provider are Tech Lab-owned modules configured under `[integrations.]`, and core keeps only `none` | Proposed | -| 9 | This spec and its core implementation precede #1043; 51Degrees implements the core seam, the nine vendor moves in §4 stay one PR each | Proposed | +| # | Decision | Status | +| --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | +| 1 | Vendor integrations belong outside core, behind the registration contract | Proposed | +| 2 | Tech Lab engineering reviews vendor crates, and does not maintain them | Proposed, governance | +| 3 | A registration may carry its own browser JavaScript | Proposed | +| 4 | Deploy validation moves onto the registration | Proposed | +| 5 | The nine existing integrations migrate one PR each, on the schedule in §4 | Proposed | +| 6 | This change completes the Rust side, so after it no vendor move needs a Rust core change. The browser side is not complete, because TSJS core still imports the APS renderer directly (§8 item 8), so an APS move still needs a browser renderer contract | Proposed | +| 7 | Identity, geo and device providers are capabilities of a module registration (§3.6), the #1043 review's rule applied to all three | Proposed | +| 8 | No provider is built into core: HMAC and the User-Agent-only device provider are Tech Lab-owned modules configured under `[integrations.]`, and core keeps only `none` | Proposed | +| 9 | This spec and its core implementation precede #1043; 51Degrees implements the core seam, the nine vendor moves in §4 stay one PR each | Proposed | ## Revision record | Date | Change | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-08-27 | First draft, written against `split/5-response-hook-docs`. | -| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a Rust core change (§8 row 6). The browser side is recorded as outstanding in §8 item 8. | +| 2026-08-27 | Brought the bid renderer contract into scope (§3.4), so that after this change no vendor move needs a Rust core change (§8 row 6). The browser side is recorded as outstanding in §8 item 8. | | 2026-08-28 | Corrected line references to `main` at b7fcb5d4c and added what mapping `main` found: composition of the served script moves into core (§3.2), registration enumeration and the auction-only `adserver_mock` case (§3.3), the duplicate-id gap (§3.1), the source-file guard and the `ts audit` vendor table (§4), the renderer risk (§7). | | 2026-08-28 | Recorded what implementing the seam found (§8): the operator CLI skips a vendor's deploy rules, a carried module's hash literal is fragile, providers resolve more than once per request, and one core reader still reads an APS payload. Recorded the construction-time hash check in §3.2. | | 2026-08-28 | Adopted the #1043 review's registration shape for identity and applied its rule to geo and device, with no provider built into core (§3.6, §6 item 2, §8 rows 7 to 9). Recorded the relationship to #986 and reordered the series so this spec and its implementation come first. | From fa1df47503ff5d47b6db75cd93ef8c4f9f08c77d Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 1 Sep 2026 15:02:18 +0100 Subject: [PATCH 014/192] Cite RFC 6454 for the resolve endpoint's origin comparison The client-cycle spec described the origin check as a string comparison of the serialized origin. The implementation on the client-resolve branch now applies the RFC 6454 same-origin test, comparing the scheme, host and port triple with a missing port meaning the scheme's default, so the spec names that rule and the RFC rather than describing a comparison of its own. --- ...26-07-30-client-cycle-ec-resolve-design.md | 30 ++++++++++--------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 1ffde33baf..c793df81ff 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -65,16 +65,18 @@ verify against. `POST /_ts/api/v1/ec/resolve` MUST: 1. **Reject cross-site requests with an origin check.** v1 authorizes a - request only when its `Origin` header, serialized as scheme, lowercased - host and the port as the browser wrote it, is byte-equal to an accepted - origin. No port is added or removed on either side. The + request only when its `Origin` header is the same origin as an accepted + origin under RFC 6454, meaning the scheme, host and port triple of §4 + compared by the §5 rule, with a missing port standing for the scheme's + default, and with a value that is not a serialized origin under §6.1 + never matching. The default accepted set is the single origin `https://{publisher.domain}` and nothing else, so a sibling subdomain, the `http://` scheme and a non-default port are all refused. A missing or foreign `Origin` is rejected with `403`. Browsers always send `Origin` on POST `fetch`, so its absence means a non-browser caller, which has no business on a page-script endpoint. An operator may configure further accepted - origins, each of them compared the same exact way, for a publisher + origins, each of them compared by the same RFC 6454 rule, for a publisher whose pages are served from `www` or from another domain. This is the exact allowlist the 2026-07-31 draft asked for, and it replaces the suffix match on `publisher.domain` an earlier revision of this spec @@ -292,13 +294,13 @@ end in tests and demonstrations. ## 8. Revision record vs the 2026-07-31 draft -| Draft position | v1 (PR #1046) | Why | -| ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | -| Origin check via new allowlist config + CSRF token | Exact serialized-origin comparison (scheme, lowercased host, port as written). Default accepted set is the single origin `https://{publisher.domain}`, plus an optional operator-configured list of further exact origins. Missing/foreign `Origin` → `403` | The draft's exact allowlist is adopted. An earlier revision of this spec allowed any suffix match on the apex, which admitted every subdomain including ones the publisher may not control, and its justification did not hold. This closes §7.5 | -| Marker cookie survives whatever happens to the identity it marks | Core expires the `ts-ecr` marker on any request carrying a `ts-ec` the selected provider does not own | The marker is not namespaced by the provider code envelope, so without this a provider switch leaves a visitor with a marker, no identity, and a page script that will not re-post | -| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | -| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | -| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | -| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | -| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | +| Draft position | v1 (PR #1046) | Why | +| ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Feature deferred, `resolve_from_client` de-normalized | Feature normative. Trait method kept with a no-op default | The first vendor integration works client-side by design, so the endpoint is on the critical path for the series' first real provider | +| Origin check via new allowlist config + CSRF token | RFC 6454 same-origin comparison of the scheme, host and port triple. Default accepted set is the single origin `https://{publisher.domain}`, plus an optional operator-configured list of further exact origins. Missing/foreign `Origin` → `403` | The draft's exact allowlist is adopted. An earlier revision of this spec allowed any suffix match on the apex, which admitted every subdomain including ones the publisher may not control, and its justification did not hold. This closes §7.5 | +| Marker cookie survives whatever happens to the identity it marks | Core expires the `ts-ecr` marker on any request carrying a `ts-ec` the selected provider does not own | The marker is not namespaced by the provider code envelope, so without this a provider switch leaves a visitor with a marker, no identity, and a page script that will not re-post | +| §3.9 reservation/replay machinery required before code | Deferred to the vendor scheme. Draft text retained verbatim as the bar | The design needs the real envelope's unique id and session binding, and a CAS-class primitive no production adapter exposes today | +| Identical behavior or identical startup refusal, 4 ways | Fastly routes it (bot-gated graph). Portability adapters documented as deliberately not routing, like identify | Same platform-KV constraint as the existing EC API routes. Startup rejection follow-up recorded (§7.4) | +| Marker cookie or injected variable (design must pick) | Marker cookie (`ts-ecr`), expired with the EC cookie | The draft's own first option. Testable and observable | +| Demo gated by cargo feature or `#[cfg(test)]` | Both: `client-fixed-demo` feature + startup rejection in the settings validator | Defense in depth | +| Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified | (none) | From 48e6adefda9751ffe27e69318a32c7d5e30cd447 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 1 Sep 2026 15:32:46 +0100 Subject: [PATCH 015/192] Record the chosen page carrier for resolved permissions --- ...26-07-30-client-cycle-ec-resolve-design.md | 46 +++++++++++-------- 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index c793df81ff..a202fc3ffd 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -10,7 +10,7 @@ defers, and why the feature is normative rather than deferred. **Author:** Engineering (revised against the implementation, 2026-08-25) **Issue references:** #778 (series), successor spec of the 2026-07-31 draft **Related specs:** `2026-07-30-pluggable-providers-design.md` -**Last updated:** 2026-08-25 +**Last updated:** 2026-09-01 > **Context.** PR #838 shipped, undeclared and unspec'd, a second provider > _type_: a "client-cycle" EC provider whose identifier is established by a @@ -211,23 +211,33 @@ the bar for the vendor-scheme implementation: asynchronous vendor contact, including BFCache restoration), and its injection is keyed off the provider selection exactly as the demo's is today. -- **How the resolved permissions reach the browser is not designed here.** - The requirement above assumes the page can read the server's decision, - and nothing in v1 carries that decision to the page. The binding - constraint is that the JavaScript bundle is composed at startup and - served under a content hash, so one body is shared across every visitor - and cannot carry per-visitor permission state, which means the signal - has to be per request. The candidate carriers are a response header on - the document, a value injected into the document during HTML - processing, a first-party endpoint the page fetches, and a - non-HttpOnly cookie set alongside the marker. This spec names them and - deliberately does not choose between them. Whichever is chosen, the - server's resolved permission decision is the authority, and an in-page - CMP read is only a withdrawal re-check layered under it and never a - substitute for it, because a page-side read can narrow what the server - resolved and must never widen it. Designing the mechanism is out of - scope for this set of PRs and belongs with the first vendor module, - which is the first consumer that needs it. +- **How the resolved permissions reach the browser is now chosen.** The + requirement above assumes the page can read the server's decision, and + v1 had no carrier for it. The binding constraint is that the JavaScript + bundle is composed at startup and served under a content hash, so one + body is shared across every visitor and cannot carry per-visitor + permission state, which means the signal has to be per request. The + permission-model PR (#1045) picks the injected-value carrier and + delivers it. The page receives `window.tsjs.permissions`, an object + `{"set": ["necessary.operations.storage", "..."]}` naming the Data Uses + set for the request, using the same keys as `permissions.yaml` and + `Permission::as_str()`. Under inline assembly it is injected as a + `` into the head, last among its own head markup and after the shim and the bootstrap that the loader relies on, so the loader is fetched from the module's own first-party route. The setting is `false` by default, because a page that already carries the loader would load it twice. + +**Why.** A publisher whose page templates forbid third-party scripts cannot put a Google tag on the page, so it has to come from the proxy. Nothing did that. The Tag Manager module re-serves a tag the page already carries, which needs a page that already loads Google. And a page that loads Google's tags itself sends measurement data before any consent default is set. ## 3. What is not in this change - An AdSense unit. The per-unit `` markup is a property of the publisher's own page, not of this configuration. - Removing an inline `gtag('config', ...)` from the publisher's page, which would mean buffering every response. +- A guide page of its own under `docs/guide/integrations/`. The configuration guide's `[tag.google-tag]` section and its table of middleware state the contract. + +**Open questions.** Each of these is a question about the code. This spec records what the code does and leaves the questions open. + +1. Should a setting leave out the noscript iframe? The module writes the iframe whenever `container_id` is set (`body_inserts` in `crates/tag/google-tag/src/lib.rs`), so a visitor whose browser has scripts turned off asks Google for `ns.html`, and no consent default can run there. +2. Should the module read the reader's permission state? The module declares no permission (`Middleware::required_permissions` returns none unless a middleware says otherwise), has no `prepare`, and ignores the context its `create` is handed, which holds no permission state. Google's loaders therefore reach every reader, including one whose request carries a Global Privacy Control signal. The question is whether to write them only for readers whose permission state allows it, or to record that they are written for every reader. +3. With `region` set, should the module also write a default with no region? Google's consent guide says a default command without a region sets the default for all visitors that no regional command covers, and the module writes one command that carries the regions, so a visitor outside the listed regions gets no default from this module. +4. Should one test run a page through both modules? All nine page tests select this module alone, so a loader that the Tag Manager module's fetch rewrite made first party is simulated by a page that already carries the first-party path (`a_loader_the_fetch_rewrite_made_first_party_is_removed_as_well`). +5. Should the crate have the `module_constant_is_the_crate_folder` test? `AGENTS.md` says each page integration holds its name in a `MODULE` constant that a test in the crate holds to the crate's folder, and 18 other crates have that test, but this crate has none. +6. Should the paths no test runs get tests? They are a document with no ``, where every head snippet is dropped and the noscript iframe is still written, a document with no ``, a `wait_for_update` of 0 and of 60001, an empty `body_open_inserts`, which registers no handler, and the AdSense loader after the tag snippet. +7. Should the placeholder in the example settings fail validation, and should `AGENTS.md` list `tag.google-tag`? `G-REPLACE-ME` in `trusted-server.example.toml` passes the tag id pattern, unlike the Tag Manager example's `GTM-REPLACE-ME`, which is invalid on purpose, and lines 404 and 445 of `AGENTS.md` list the other page integrations without `tag.google-tag`. +8. Should the check the Tag Manager route makes call the function this module calls? The rule for which tag ids the Tag Manager module serves is written in `tag_id_is_allowed` and again in `first_party_gtag_js_path`, so the two agree only while their text stays the same. ## 4. Acceptance -The seam probe's middleware writes a marker immediately after the opening body tag, and its page test reads it back once, directly after ``, through the fetch processor. +CI passed on all 25 checks of the draft pull request at `1940d63f1`, and on all 19 checks of the fork's closed pull request at the same commit. The description of the pull request lists the checks that were run, says that Google's own handling of the snippets was not exercised, and leaves its manual test with `fastly compute serve` unticked. The module's 32 tests, one test in core, one in the seam probe and three in the ad tag module hold the claims in section 2 as follows. + +- **Each snippet is Google's text with only the id substituted.** For the tag snippet, `the_tag_snippet_is_googles_own_with_the_id_substituted`. For the AdSense loader, `the_adsense_loader_is_googles_own_with_the_account_substituted`. For the container script, `the_container_script_is_googles_own_with_the_id_substituted`. For the noscript iframe, `the_container_noscript_is_googles_own_with_the_id_substituted`. The consent snippet has no such test. +- **The consent defaults.** `every_consent_type_defaults_to_denied`, `the_tag_waits_for_the_platform_s_update_before_it_fires` (500 and no region), `a_publisher_may_scope_the_defaults_to_regions_and_set_the_wait`, `a_publisher_may_grant_a_consent_type_up_front` and `a_region_that_is_not_a_code_is_refused_by_name`. +- **The order within the module's markup.** `consent_defaults_come_before_the_config_command`, `consent_defaults_come_before_the_container` and `a_container_and_a_tag_are_both_written_container_first`. +- **Each id on its own.** `each_id_on_its_own_writes_the_consent_defaults_and_its_own_snippet`. +- **Where each loader points.** `the_loader_names_googles_host_when_tag_manager_is_not_enabled`, `the_loader_names_the_first_party_path_when_tag_manager_serves_the_tag`, `the_loader_names_googles_host_when_tag_manager_would_not_serve_the_tag`, `the_container_loader_names_googles_host_when_tag_manager_is_not_enabled`, `the_container_loader_names_the_first_party_path_for_the_same_container` and `the_container_loader_names_googles_host_for_a_different_container`. +- **The page's own loader.** `a_gtag_loader_is_recognised_however_it_is_written`, `a_page_already_loading_the_same_tag_ends_with_one_loader`, `a_loader_the_fetch_rewrite_made_first_party_is_removed_as_well` and `a_page_loading_another_tag_keeps_it`. +- **Placement on a page.** `the_snippets_land_in_the_head_once` and `the_snippet_reaches_a_real_shaped_head_before_the_publishers_own_scripts` hold the head snippets once, before the page's own head content, with no `[[fetch]]` middleware present. `the_noscript_lands_immediately_after_the_opening_body_tag_once`, `a_second_body_does_not_get_a_second_noscript` and `a_page_with_no_container_configured_is_left_alone_at_the_body` hold the noscript iframe. +- **The settings.** `an_unnamed_integration_does_not_register`, `a_config_naming_no_id_is_refused`, `an_id_of_the_wrong_shape_is_refused_by_name` and `an_unknown_setting_is_refused`. +- **The new place in core.** `a_body_open_insert_is_written_once_after_the_opening_body_tag` in `html_processor.rs` drives the serve processor with two inserts and a second ``. The seam probe's middleware writes a marker immediately after the opening body tag, and its page test, `the_middleware_changes_a_page_where_an_entry_names_it`, reads the marker back once, directly after ``, through the fetch processor. +- **The ad tag module's loader.** `head_inserts_write_the_loader_tag_when_asked`, `head_inserts_write_the_loader_tag_last` and `head_inserts_write_no_loader_tag_by_default` in `crates/ad-tag/google/src/lib.rs`. + +## Review log -The pull request's description records how it was checked, with every commit linted and tested on its own, in order, the full local check on its head and CI green on all 19 checks, and 5 deliberate faults made one at a time and each caught by a test. +1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/30-google-tag` at `1940d63f1`. Issues found, 5 of them blocking. The five were that the spec said the snippets land at the start of the head when they land before the script bundle's tag and after what a `[[fetch]]` middleware wrote, that it gave a reason for the module's place in the stock list that does not hold, that it never mentioned the `write_loader` setting the pull request adds to the Google ad tag module, that its consent design left out `wait_for_update`, `region` and the states a consent type can take, and that its header called a closed fork pull request ready for review and named a closed pull request as the place of the specs. This revision states what the code does at that head for each of the five, and adds the settings, the failure cases, the adapters, the tests that hold each claim and the code questions the review raised, which wait for code changes under Open questions. ## Revision record -| Date | Change | -| ---------- | --------------------------------------------------------------- | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| Date | Change | +| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-16 | Fixes from review round 1. The header and the pull request pointers, where the head snippets land, the reason for the stock list order, the settings, the consent defaults, the Tag Manager module, the ad tag module's loader setting, failure, adapters, compatibility, the content security policy, the tests, the open questions and the review log. | From e75b3d3456642280303f6474b90dcb69090f4c75 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 08:34:43 +0100 Subject: [PATCH 094/192] Install worker-build from its own lockfile The Cloudflare build script installed worker-build without --locked, so cargo resolved its dependencies afresh, and every zip 2.x release that worker-build 0.8.5 needs is now yanked on crates.io. With --locked, cargo uses the published Cargo.lock of worker-build, which pins zip 2.4.2. Fixes the setup failure reported in #1402. --- crates/trusted-server-adapter-cloudflare/build.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crates/trusted-server-adapter-cloudflare/build.sh b/crates/trusted-server-adapter-cloudflare/build.sh index dcabdee8eb..49650b948c 100644 --- a/crates/trusted-server-adapter-cloudflare/build.sh +++ b/crates/trusted-server-adapter-cloudflare/build.sh @@ -33,4 +33,4 @@ if [ -z "$WORKER_VERSION" ]; then echo "error: could not determine the worker crate version from Cargo.lock" >&2 exit 1 fi -cargo install -q --force --version "=$WORKER_VERSION" worker-build && worker-build --release +cargo install -q --force --locked --version "=$WORKER_VERSION" worker-build && worker-build --release From ef4358612bdd9fbae0c0a19cd715671aa04992e2 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 08:38:13 +0100 Subject: [PATCH 095/192] The viewport sizes design takes its round 1 review The design now says which requests the viewport bands reach and why the server-side auctions are not narrowed, and it drops the claim of deliberate faults that no description records. Six code questions, including a size applied after first paint, are recorded as open questions. --- .../specs/2026-10-12-viewport-sizes-design.md | 143 ++++++++++++++++-- 1 file changed, 128 insertions(+), 15 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md b/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md index 03a5505332..33d18b961e 100644 --- a/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md +++ b/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md @@ -1,19 +1,24 @@ # Design Spec: A format can name the viewport widths at which the page asks for it -**Status:** Proposed, 2026-10-12. Implemented on the fork as jwrosewell/trusted-server#24, ready for review, answering IABTechLab/trusted-server#1287. This pull request (#1084) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1370](https://github.com/IABTechLab/trusted-server/pull/1370), row 29 of the stack, answering [IABTechLab/trusted-server#1287](https://github.com/IABTechLab/trusted-server/issues/1287). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-08-27-integration-provider-seam-design.md` -**Related PRs:** #1094, jwrosewell/trusted-server#24 -**Last updated:** 2026-10-12 +**Related specs:** `2026-07-24-prebid-refresh-gam-path-opt-out-design.md` (the wrapper on `refresh()` that runs the refresh auction, whose sizes this change takes from the band) and `2026-07-24-prevent-duplicate-gpt-slot-requests-design.md` (how the head bootstrap and the bundle define a slot on the publisher's div), both on `main` of IABTechLab/trusted-server +**Related PRs:** [IABTechLab/trusted-server#1370](https://github.com/IABTechLab/trusted-server/pull/1370) builds on the device signals in [IABTechLab/trusted-server#1369](https://github.com/IABTechLab/trusted-server/pull/1369) (row 28), replaces [IABTechLab/trusted-server#1317](https://github.com/IABTechLab/trusted-server/pull/1317) (closed, an earlier pull request for this change against `main`) and was reviewed on [jwrosewell/trusted-server#24](https://github.com/jwrosewell/trusted-server/pull/24) (closed) +**Issues:** [IABTechLab/trusted-server#1287](https://github.com/IABTechLab/trusted-server/issues/1287). [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack +**Last updated:** 2026-10-16 -> **Why this spec exists.** A slot's formats are the sizes a page asks for, and the page asks for all of them at every width. +> **Why this spec exists.** A slot lists the sizes a page asks for, and does not record the widths at which the page shows each size. This spec says what the two viewport bounds on a format add, which requests for sizes the bounds reach, and which requests are still offered every format. ## 1. Problem A slot's formats are the sizes a page asks for, and the page asks for all of them at every width. A page that shows a billboard on a wide screen and a small banner on a phone has one slot and several formats, so a bidder is offered a billboard the page cannot show, and the ad tag defines the slot with every size at every width. +Two scripts define the slot in the browser. The head bootstrap is the script that the Google ad tag writes into the head of the page, and the bundle is the script bundle that the server adds to the page. Three requests then ask for the slot's sizes. The first is the ad server request that the Google Publisher Tag (GPT) makes for the slot. The second is the refresh auction, which the bundle runs in the browser when a publisher refreshes a slot. The third is a pair of server-side auctions, being the initial-page auction that the server runs when it serves the document and the page-bids auction that the server runs when a single-page application moves to a new path and the bundle asks for bids. + ## 2. Design +Line numbers are those of `054f3a2a0`, the head of the pull request. Paths are under `crates/`, where `core/` is `trusted-server-core/src/`, `js/` is `trusted-server-js/lib/src/` and `tag/` is `ad-tag/google/src/`. + ```toml [[creative_opportunities.slot]] id = "billboard" @@ -26,22 +31,130 @@ formats = [ ] ``` -A format may name the viewport widths, in CSS pixels, at which the page asks for it, so a slot asks for a large size only where the page has room for one. Either bound may be left out, and a minimum above a maximum is refused when the settings load. When any format of a slot names a bound, the slot's page data carries `viewport_sizes`, one entry for each band of widths, widest first, and the Google ad tag's head bootstrap and the bundle's slot definition apply it as a size mapping when they define the slot. A slot whose formats name no bound asks for every format at every width, as before. The bounds apply in the browser only, and a server-side auction is still offered every format. +**The bounds.** A format may name `min_viewport_width` and `max_viewport_width`, the narrowest and the widest viewport at which the page asks for the format, so a slot asks for a large size only where the page has room for one. Each bound is a whole number of CSS pixels and is inclusive, and either may be left out to leave that side open. Neither bound has a default and neither is written out when unset, so a format that names neither is read and written as it was (`core/creative_opportunities.rs`, lines 1037 to 1042). Equal bounds name one width, and formats whose ranges overlap are all asked for where the ranges overlap. A minimum above a maximum is refused (lines 1055 to 1063). The refusal comes when the settings load from a file or from the stored configuration that every adapter reads (`Settings::prepare_runtime`, `core/settings.rs` line 4518, reached from `core/config_payload.rs` line 99), and again in the validation that runs before settings are pushed (`core/config.rs` line 360). + +**The bands.** `CreativeOpportunitySlot::viewport_sizes` (`core/creative_opportunities.rs`, lines 954 to 982) turns the formats of a slot into bands. A band is a range of viewport widths over which the slot asks for the same sizes. Each band is recorded as the narrowest width at which it applies and the `[width, height]` sizes asked for from that width up to the width of the next band, in the order the formats are written. The bands start at zero and at every width where the formats on offer change, being each minimum and the width one above each maximum, and neighbouring bands that offer the same sizes become one band. A band can be empty, which means the slot asks for nothing at those widths, as when the only format of a slot has a minimum, which leaves every narrower width empty. A slot whose formats name no bound has no bands and asks for every format at every width, as before. For the slot above the bands are, widest first: + +| From viewport width | Sizes asked for | +| ------------------- | --------------- | +| 1040 | 970x250 | +| 800 | 728x90 | +| 600 | 300x250 | +| 0 | 320x50 | + +**The page data.** When a slot has bands, its page data carries them as `viewport_sizes`, a list with one `[[width, 0], sizes]` entry for each band, widest first. The pair `[width, 0]` is the viewport pair that GPT's size mapping takes, written with a height of zero because the bands are by width alone, so the browser passes each entry to the mapping as it stands. For the slot above the list is: + +```json +[ + [[1040, 0], [[970, 250]]], + [[800, 0], [[728, 90]]], + [[600, 0], [[300, 250]]], + [[0, 0], [[320, 50]]] +] +``` + +The slot's `formats` still lists every size. `build_slot_json` (`core/publisher.rs`, lines 6821 to 6854) builds the page data for each of the three ways the page data reaches the browser, being the head script that sets `window.tsjs.adSlots` (`build_ad_slots_script`), the list of slots that a shared page receives with each request (`seam_ad_slots_json`) and the answer to the page-bids request (`handle_page_bids`). + +## 3. Which requests the bands reach + +The bands reach the ad server request for a slot that Trusted Server defines and the refresh auction, and they do not reach the server-side auctions. + +| Request | Sizes offered | +| ----------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | +| The ad server request for a slot that Trusted Server defined | The sizes of the band that GPT finds for the viewport, through a size mapping | +| The ad server request for a slot that the publisher defined first | The sizes the publisher defined, with no mapping added | +| The refresh auction | The sizes of the band the viewport is in when the refresh runs, and no auction at all for a slot whose band is empty | +| The initial-page auction and the page-bids auction | Every format | + +**The ad server request.** The head bootstrap and the bundle each define the slot with every size and then, when the slot has bands, apply the bands as a size mapping, which is GPT's way of giving a slot different sizes at different viewport widths. Each does so in a function named `defineWithViewportSizes` (`tag/gpt_bootstrap.js`, lines 361 to 383, and `js/integrations/gpt/index.ts`, lines 858 to 877). Each calls the function in two places, being the `adInit` pass (`tag/gpt_bootstrap.js` line 855, `js/integrations/gpt/index.ts` line 1382) and the fallback that defines the slot later when a publisher claimed the placement but the claim ran out before the publisher made an ad request (`tag/gpt_bootstrap.js` line 707, `js/integrations/gpt/index.ts` line 1186). The mapping is set once, when Trusted Server defines the slot, and no code sets it again when the window is resized. When GPT has no `googletag.sizeMapping` or the slot has no `defineSizeMapping`, nothing is mapped and nothing is logged, so the slot is requested with every size (`tag/gpt_bootstrap.js` lines 369 to 375, `js/integrations/gpt/index.ts` line 869). + +**The refresh auction.** The refresh auction compares `window.innerWidth` with the start of each band every time a publisher's refresh runs, and offers the slot the sizes of the widest band that the width reaches (`injectedSlotBandSizes`, `js/integrations/prebid/index.ts`, lines 1137 to 1146). Those sizes become the banner sizes of the auction's ad unit (same file, lines 2762 to 2768). The bundle's Prebid adapter posts the ad unit, with its banner sizes, to the auction endpoint, which is `/auction` unless the Prebid configuration names another path (same file, lines 2357 and 2375 to 2395, with `buildAdRequest` in `js/core/auction.ts`, lines 76 to 114), and the server reads the sizes as the formats of the slot (`convert_tsjs_to_auction_request`, `core/auction/formats.rs`, lines 168 to 217). The ad unit also lists the client-side bidders that the publisher configured for the slot (same file, line 2784), so Prebid asks those bidders about the same sizes. A slot whose band is empty at the current width is left out of the auction (`js/integrations/prebid/index.ts`, lines 1154 to 1157 and 2720 to 2724), its stale targeting is cleared as for every slot left out, and GPT still refreshes the slot. A slot with no bands, or a slot that the bundle cannot match to a slot of the page data, is offered the sizes it was offered before. + +**The server-side auctions.** Both server-side auctions build their slots with `to_ad_slot`, which lists every format whatever the bounds (`build_auction_request`, `core/publisher.rs`, lines 6308 to 6312, called for the initial page at line 5521 in `handle_publisher_request` and for page-bids at line 7906 in `handle_page_bids`). The server holds the bands but reads no viewport width from either request. The page-bids request carries the path alone (`js/integrations/gpt/index.ts`, line 1564), and a search of the crates and the guides for the `Viewport-Width` and `Sec-CH-Viewport-Width` headers (section 6) finds none. So the server cannot choose a band for a reader, and a bidder in either auction is still offered a billboard on a phone, which is the problem section 1 describes. A bid for a size outside the band can therefore win. With the slot above, a 970x250 bid can win on a phone 375 pixels wide whose mapping allows only 320x50. Nothing compares the size of a winning bid with the band, and a winning bid that carries no size of its own is sized as the slot's first configured format, not as the band's (`js/integrations/gpt/index.ts`, lines 2182 to 2184). Open question 2 asks about both. What the ad server does with a winner whose size the mapping leaves out was not exercised. + +**Slots the publisher defines.** Whoever defines the GPT slot first decides whether the slot has a mapping. Where the publisher defined the slot before `adInit` looked for the slot, Trusted Server reuses the slot and adds no mapping (`tag/gpt_bootstrap.js` lines 700 to 705 and 845 to 851, `js/integrations/gpt/index.ts` lines 1180 to 1184 and 1370 to 1376). The refresh auction does not tell the two cases apart, because `findInjectedSlotForRefresh` (`js/integrations/prebid/index.ts`, line 1173) finds the page data of a slot by element id whoever defined the slot. So a slot that the publisher defined is offered only the band's sizes in the refresh auction, while the ad server request of that slot carries the sizes the publisher defined. Open question 3 asks about this. + +**When the band takes effect, and the space for the slot.** The sizes a slot asks for are fixed when its first ad request is made. The first `adInit` of a document runs after the window's `load` event and then two animation frames (`ts.scheduleInitialAdInit`, `tag/gpt_bootstrap.js` lines 329 to 350, and `installScheduleInitialAdInit`, `js/integrations/gpt/index.ts` lines 809 to 833), so the band takes effect after the first paint. No code in this row sets, reserves or removes the space of a slot's container before the advert arrives, so the page's own markup and style decide that space. Where the one container matching a prefix of the slot's id is hidden at the viewport, `adInit` skips the slot and logs a debug message (`tag/gpt_bootstrap.js` lines 816 to 830). A page that has not reserved the space for the band's size therefore moves when the advert arrives, and a container whose band is empty keeps whatever space the page gave it. Open question 1 asks about this. + +## 4. Failure, rollout, adapters and privacy + +**Failure.** A minimum above a maximum stops the settings from loading, with a message that names the slot, the format and both bounds. A GPT without the size mapping interface maps nothing and says nothing (section 3). The mapping calls have no `try` and `catch` (open question 5). + +**Rollout.** A binary released before these fields refuses a format carrying either of them, because `CreativeOpportunityFormat` denies unknown fields (`core/creative_opportunities.rs`, line 1028), and the binary then cannot load its settings. So the binary that reads the fields goes out before any configuration uses them, and the configuration guide says so (`docs/guide/configuration.md`, lines 2792 to 2795). Rolling back reverses the order. A configuration that still names a bound has to be pushed again without the bound before the binary is rolled back, and the guide does not say so. A bundle from before this row ignores `viewport_sizes`, and a bundle from this row that is given page data without `viewport_sizes` asks for every format, so the page data, the head bootstrap and the bundle can be mixed in either order. + +A binary that refuses the settings answers every route with the startup error, which is status 500 on the Axum, Cloudflare and Fastly adapters and status 503 on Spin (`startup_error_router` in each adapter's `src/app.rs`). The route `/health` is the exception on two of the four. The Fastly adapter answers `/health` before it reads any settings (`crates/trusted-server-adapter-fastly/src/lib.rs`, line 267) and the Spin adapter registers `/health` on its startup error router (`crates/trusted-server-adapter-spin/src/app.rs`, line 647), so a health check on those two adapters stays green while every other route fails. + +**Adapters.** No adapter file changes in this row. All four adapters load their settings through `settings_from_config_blob_with` (`core/config_payload.rs`). In each adapter's own `src/app.rs`, the Cloudflare adapter calls the function at line 191 and the Spin adapter at line 140, and the Fastly adapter at line 238 and the Axum adapter at line 111 call `get_settings_from_config_store_with`, which calls the function at `core/settings_data.rs` line 120. The refusal, the page data and the browser code are therefore the same on each adapter. + +**Privacy.** The bands in a slot's page data come from the configuration and the request path alone, so the bands are the same for every reader. The browser reads the viewport width with `window.innerWidth` (`js/integrations/prebid/index.ts`, line 1144) and does not send the width. What goes out is the sizes of the band in the refresh auction, which tells the server and the bidders roughly how wide the viewport is. This row changes no consent check and adds no request, cookie or storage. + +## 5. What is not in this change + +- **Sizes by anything but viewport width.** The bands are the only dimension a format names, and the viewport of each entry is `[width, 0]`, so the viewport's height, the device's pixel ratio, its orientation and the width of the container take no part. +- **Narrowing the server-side auctions.** Section 3 gives the reason the server-side auctions are offered every format, and open question 2 asks about the difference from the refresh auction. +- **Space for a slot's container.** Section 3 says that the page's own markup and style decide the space, and open question 1 asks whether that is intended. +- **A mapping on a slot the publisher defined.** Open question 3 asks whether the refresh auction should narrow the sizes of such a slot. +- **Choosing a band again when the window is resized.** The refresh auction reads the width each time it runs, the mapping is set once, when Trusted Server defines the slot, and no code of the ad stack refreshes a slot or sets a mapping again when the window is resized. +- **A change to any adapter or to the command line tool.** Open question 6 lists what the command line tool prints. + +## 6. Acceptance + +**Checks that can be repeated.** In a clone of IABTechLab/trusted-server, this search prints six lines. Three of the six read the bands, being `tag/gpt_bootstrap.js` line 368, `js/integrations/gpt/index.ts` line 868 and `js/integrations/prebid/index.ts` line 1142, and the other three are the type declaration (`js/core/types.ts` line 34) and two comments. So the bands are read in three places in the browser, to define a mapping in the head bootstrap, to define a mapping in the bundle and to choose the refresh auction's sizes, and nothing compares a winning bid's size with a band. + +```text +git grep -n viewport_sizes 054f3a2a0 -- crates/trusted-server-js/lib/src crates/ad-tag/google/src/gpt_bootstrap.js +``` + +In core, this search prints five lines in the tests of `core/creative_opportunities.rs` and one line in `core/publisher.rs` (line 6847), so `build_slot_json` is the only caller of `viewport_sizes()` outside the tests. + +```text +git grep -n "viewport_sizes()" 054f3a2a0 -- crates/trusted-server-core/src +``` + +This search prints nothing, which is why section 3 says that the server reads no viewport width. + +```text +git grep -n -i "viewport-width" 054f3a2a0 -- crates docs +``` + +**CI.** Every check passed at `054f3a2a0`, the head of the pull request, being 25 of 25 on [IABTechLab/trusted-server#1370](https://github.com/IABTechLab/trusted-server/pull/1370) and 19 of 19 on the fork's closed pull request [jwrosewell/trusted-server#24](https://github.com/jwrosewell/trusted-server/pull/24). + +**Local checks.** The description of IABTechLab/trusted-server#1370 records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run. The description marks a manual run with `fastly compute serve` as not done. The description also says that the ad server's own handling of a size mapping was not exercised, the tests proving only that the mapping is defined with the bands that the page data carries. + +**Tests that exist.** + +- Core (`core/creative_opportunities.rs`, lines 1550 to 1673). Five tests of `viewport_sizes` check that a slot with no bound has no bands, that each band holds only the formats that fit it (with the four formats of section 2), that an unbounded format stays in every band, that a band is empty where no format fits, and that neighbouring bands offering the same formats become one. One test checks that a minimum above a maximum is refused, and the test calls the slot's `validate_runtime`. Two tests of the format check that an unbounded format is written without the fields and that both bounds are read as written. +- The page data (`core/publisher.rs`, line 25595). One test checks that a slot without bounds carries no `viewport_sizes`, that a slot with bounds carries them widest first, and that `formats` still lists every size. +- The head bootstrap and the bundle. The test `fallback adInit defines, targets, and displays a TS slot through the command queue` in `crates/trusted-server-js/lib/test/integrations/gpt/gpt_bootstrap.test.ts` and the test `displays TS-defined slots and does not include them in refresh` in `crates/trusted-server-js/lib/test/integrations/gpt/ad_init.test.ts` each run with and without bands. With bands, each checks that the mapping is built from the bands in order and applied to the slot, and without bands that no mapping is built. A test in the ad tag crate (`tag/lib.rs`, line 1355) checks that the bootstrap carries the call at the `adInit` site. +- The refresh auction (`crates/trusted-server-js/lib/test/integrations/prebid/index.test.ts`, the group `a slot with viewport bands`). Three tests check that a width of 375 pixels offers `[[320, 50]]`, that a width of 1280 pixels offers `[[970, 250]]`, and that a width of 700 pixels, where the band is empty, leaves the slot out of the auction and refreshes the slot through GPT. + +**What no test covers.** -A binary released before these fields refuses a format carrying either of them, which fails every request, so the binary that reads them is promoted before any configuration uses them. The configuration guide says so. +1. The two fallback sites that define a slot (`tag/gpt_bootstrap.js` line 707, `js/integrations/gpt/index.ts` line 1186), because the tests of the head bootstrap and the bundle drive `adInit` alone. +2. GPT without `sizeMapping` or `defineSizeMapping`, where a slot with bands is defined with no mapping. Every test that gives bands gives GPT both. +3. The refusal of an inverted range through `Settings::from_toml` or `Settings::from_json_value`. The one test calls the slot's `validate_runtime`, and the loaders reach that function through `prepare_runtime` (section 2). +4. The server-side auctions for a slot with bounds. No test builds an auction request for a slot with a bounded format, so no test pins that every format is offered. +5. Equal bounds, and two bounded formats whose ranges partly overlap. +6. GPT and the ad server acting on a size mapping, an empty size list included. The tests use a stand-in for GPT, and the description records that the ad server's own handling of a size mapping was not exercised. -**Why.** A page that shows a billboard on a wide screen and a small banner on a phone has one slot and several formats, and without bounds the slot asks for every format at every width, so a bidder is offered a billboard the page cannot show. The bounds are the publisher's statement of what fits where, and the ad tag passes them to the ad server as its size mapping. +## 7. Open questions -## 3. What is not in this change +The questions below are for the reviewer. This revision records what the code does and decides none of the questions. Line numbers and paths follow section 2. -- Sizes by anything but viewport width. The bands are the only dimension a format names. +1. **A size applied after the first paint.** Should the size of a slot be settled before the first paint, so that the page does not shift when an advert arrives? The band takes effect when the first ad request is made, after the window's `load` event and two animation frames, and nothing in this row reserves the space of the slot's container for the band's size or removes the container where the band asks for nothing (section 3). No test of this row runs GPT itself, so GPT's response to an empty size list, which is what an empty band becomes in the mapping, was not exercised. +2. **A winning bid outside the band.** Is it intended that the initial-page auction and the page-bids auction are offered every format while the refresh auction is offered only the band's sizes? On a phone a 970x250 bid can win while the mapping allows 320x50, nothing compares the size of a winning bid with the band, and a winner that carries no size of its own is sized as the slot's first configured format (`js/integrations/gpt/index.ts`, lines 2182 to 2184). The server reads no viewport width from either request (section 3). +3. **A slot the publisher defined.** Should the refresh auction narrow the sizes of a slot that carries no mapping because the publisher defined the slot first? Today the bidders are offered only the band's sizes (`js/integrations/prebid/index.ts`, lines 2720 to 2724 and 2762 to 2768) while the ad server request carries the publisher's sizes. +4. **Paths no test covers.** Should tests cover the two fallback sites that define a slot, GPT without the size mapping interface, and the refusal of an inverted range through the settings loaders (section 6)? +5. **Mapping calls with no `try` and `catch`.** The calls that follow `defineSlot` in `defineWithViewportSizes` have no `try` and `catch` (`tag/gpt_bootstrap.js` lines 361 to 383, `js/integrations/gpt/index.ts` lines 858 to 877). An exception from them would skip `addService` and the release of the claim on the placement (`tag/gpt_bootstrap.js` lines 706 to 713, `js/integrations/gpt/index.ts` lines 1185 to 1192), so the slot would be defined and not added to the ad service. Should a failure in the mapping calls leave the slot defined with every size instead? Nothing in this row shows GPT throwing from these calls. +6. **Four places that do not describe the row.** First, the comment on `build_ad_slots_script` (`core/publisher.rs`, lines 7526 to 7529) lists the property names of the page data without `viewport_sizes`. Second, `ts config ad-templates --details` prints each format as its size and media type, without its bounds (`crates/trusted-server-cli/src/commands/config/ad_templates.rs`, lines 490 to 497). Third, in the configuration guide the new section sits between the example of `[creative_opportunities]` and the paragraph that begins "The same switch can be overridden", so that paragraph no longer follows the switch it names (`docs/guide/configuration.md`, lines 2772 to 2797). Fourth, the new section of the guide does not say that the server-side auctions are still offered every format. Should each be corrected? -## 4. Acceptance +## Review log -The pull request's description records how it was checked, with CI green on all 19 checks, and 4 deliberate faults made one at a time and each caught by a test. +1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/29-viewport-sizes` at `054f3a2a0`. Issues found, two of them blocking. The blocking issues were that the spec never named the refresh auction, which offers a bidder only the sizes of the band, called the bounds browser only and gave no reason why the server-side auctions are still offered every format, and that its last paragraph claimed four deliberate faults that no description or commit message records. This revision states which requests the bands reach and why the server-side auctions are not among them, what a band is and what an empty band means, which slots get a mapping, when the band takes effect, what a wrong order of rollout and rollback does, what each adapter runs, what the refresh auction tells a bidder about the viewport, the checks recorded and the tests that exist, while six open questions wait for code changes, being the size applied after the first paint, the winning bid outside the band, the slot the publisher defined, the paths no test covers, the mapping calls with no `try` and `catch`, and four places that do not describe the row. ## Revision record -| Date | Change | -| ---------- | --------------------------------------------------------------- | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-16 | Fixes from review round 1. The header, the pull requests it names and the related specs, the three requests that ask for a slot's sizes and which of them the bands reach, the refresh auction that the second commit of the row (`9c2e6e802`) added after the first draft, what a band is and what an empty band means, the bounds and the page data, slots the publisher defines, when the band takes effect and who holds the space, failure, rollout, rollback, the adapters and privacy, the checks recorded with the claim of deliberate faults removed, the tests that exist and those that do not, the open questions and the review log. | From 4053440b3054e7358f88f0e4808b9164f816a1c2 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 08:52:41 +0100 Subject: [PATCH 096/192] The inspection endpoints design takes its round 2 review Fastly and Spin never run the request preparers or filters for these addresses, and a request with another method reaches Cloudflare's catch-all route and goes on to the publisher. --- .../specs/2026-10-12-inspection-endpoints-design.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md b/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md index b5310f0bcb..8380744564 100644 --- a/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md +++ b/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** IABTechLab/trusted-server#1350, IABTechLab/trusted-server#1348, jwrosewell/trusted-server#4 (closed), IABTechLab/trusted-server#1294 (closed) **Issues:** IABTechLab/trusted-server#1268 -**Last updated:** 2026-10-15 +**Last updated:** 2026-10-16 > **Why this spec exists.** A reader, a regulator or an auditor has only the operator's word for what a deployment runs, decides and keeps. Three endpoints in this change show those three things, and this spec says where each endpoint answers, what each shows and to whom. @@ -29,11 +29,13 @@ A deployment decides what a request is permitted, holds the settings that decide **Pages.** A page is the JSON answer, indented and escaped, inside a short HTML document marked `noindex`, and the page says that `.json` added to its address gives the same information as data. The data page says nothing of a data form because it has none. -**Methods.** `GET` is the only method any adapter answers at these addresses. Fastly, Axum and Spin send `POST`, `HEAD`, `OPTIONS`, `PUT`, `PATCH` and `DELETE` on these addresses to the publisher fallback, the catch-all route that forwards a request to the publisher's origin, as they do for their other named addresses. Cloudflare registers no other method for them. +**Methods.** `GET` is the only method any adapter answers at these addresses. Fastly, Axum and Spin send `POST`, `HEAD`, `OPTIONS`, `PUT`, `PATCH` and `DELETE` on these addresses to the publisher fallback, the catch-all route that forwards a request to the publisher's origin, as they do for their other named addresses. Cloudflare registers no other method for them, so those methods reach its shared catch-all route, described next. + +**Other methods on Cloudflare.** Cloudflare registers its shared catch-all route at `/` and `/{*rest}` for every method (lines 984 to 987 of `crates/trusted-server-adapter-cloudflare/src/app.rs` at `497d553c6`), so a `POST`, `HEAD`, `OPTIONS`, `PUT`, `PATCH` or `DELETE` to one of these addresses matches that route. The route's function, `dispatch` (lines 688 to 780), refuses a request only for the administration paths, being those that start with `/_ts/admin` and the retired `/admin/keys` aliases (`deny_admin_diagnostic_fallback`), and none of these five addresses is one of them. The function then runs the modules' request preparers (explained under "What runs before the handler" below), and then serves a tsjs asset for a `GET`, hands the request to a module route that matches its method and path, or otherwise treats the request as one for the publisher. A `POST` to `/_ts/config`, `/_ts/permissions` or `/_ts/data` therefore ends in the publisher branch (lines 736 to 777), after the preparers have run, unless a module has declared a route for that method and address. **What runs around the handler.** Every adapter wraps these routes in its outer layers. The response finalizer adds its geo and version headers, then applies the operator's `response_headers`. An operator's header replaces a header of the same name on the permissions and configuration answers, except `Cache-Control` and the headers that control a shared cache, which an operator cannot set on a response marked `no-store`. A `[[handlers]]` rule (HTTP Basic authentication for the paths its pattern matches) whose pattern matches an address answers `401` to a request without the rule's credentials, before the handler runs. The settings require a rule only for the `/_ts/admin` addresses, so none is required for these five. -**What runs before the handler.** Fastly answers `/_ts/config` before it builds the request's services. It answers `/_ts/permissions` and `/_ts/data` after it builds the request's services and before the modules' request preparers (each can clean up a query parameter or cookie its module reserves) and the request filters (bot protection modules use them to challenge a request) run. Spin answers all three before its preparers run and runs no filters. Axum and Cloudflare run the preparers, then answer, and run no filters. On Axum and Cloudflare a preparer that refuses a request ends it with an error response before the handler runs. A preparer's change to the request therefore reaches the permissions answer on Axum and Cloudflare and not on Fastly or Spin. No adapter runs an Edge Cookie finalizer (the step that sets the cookie and writes the identity row) for these addresses, because Fastly answers before its finalizer runs and the other three adapters have none, so nothing is written for the reader. +**What runs before the handler.** On Fastly the modules' request preparers (each can clean up a query parameter or cookie its module reserves) and the request filters (bot protection modules use them to challenge a request) never run for these addresses, because Fastly returns the answer from the route's handler and does not go on to the step that runs them. Fastly builds no services for `/_ts/config`, and builds the request's services for `/_ts/permissions` and `/_ts/data` before it answers. On Spin the preparers never run for these addresses either, because the route's handler answers, and Spin has no filters. Axum and Cloudflare run the preparers, then answer, and run no filters. On Axum and Cloudflare a preparer that refuses a request ends it with an error response before the handler runs. A preparer's change to the request therefore reaches the permissions answer on Axum and Cloudflare and not on Fastly or Spin. No adapter runs an Edge Cookie finalizer (the step that sets the cookie and writes the identity row) for these addresses, because Fastly answers before its finalizer runs and the other three adapters have none, so nothing is written for the reader. **The permissions endpoint.** The answer is the evaluation a page receives as `window.tsjs.permissions`, which holds `set` (the permissions that are set), `awaiting` (the ones still waiting for a signal), `signals` (each signal a module read and found valid, as received) and `tdls` (the terms documents the data is available under), with three things added that a person asking why a permission is not set needs and a page does not. @@ -145,7 +147,7 @@ Nothing selects this change. All five addresses are registered on all four adapt **Open questions.** Each of these is a question about the code. This spec records what the code does and leaves the questions open. 1. Should the permissions and data addresses have a switch like `[inspect] config = false`? The change gives them none and records no reason for the difference. -2. Should the permissions answer be resolved after the modules' request preparers have run, as on Axum and Cloudflare, or before, as on Fastly and Spin? +2. Should the permissions answer be resolved after the modules' request preparers have run, as on Axum and Cloudflare, or without them, as on Fastly and Spin? 3. Is every setting that shows by default meant to be public? `trusted_client_ip.auth_header`, the name of the header that carries the shared secret, shows today. 4. Should the data page also send `Cross-Origin-Opener-Policy: same-origin`? It sends no such header today. 5. Should the configuration view be built once, where it is built again on every request today? @@ -177,6 +179,7 @@ On `33282f0c9`, the last of the five original commits, the whole check also pass ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/9-visibility-endpoints` at `497d553c6`. Issues found, three of them blocking. The design said a deployment whose settings do not select the change behaves as before when all five addresses answer on all four adapters with no setting, named no address, method, `.json` form, status or header for the permissions and configuration endpoints, and described the data page's protection as `Content-Security-Policy: sandbox` when the code sends a longer policy and five more headers and keeps them under an operator's headers. This revision states what the code does at that head for the addresses, the defaults and the one switch, the failure paths, the rollout, the tests and their gaps, and lists five questions that wait for code changes. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/9-visibility-endpoints` at `497d553c6`. Approved, none blocking. This revision says that the modules' request preparers and the request filters never run for these addresses on Fastly and Spin, in place of saying that the answer comes before them, and says where a request with a method other than `GET` goes on Cloudflare, which is the shared catch-all route and then the publisher. ## Revision record @@ -184,3 +187,4 @@ On `33282f0c9`, the last of the five original commits, the whole check also pass | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The addresses with their methods, forms, statuses and headers, the defaults and the one switch, what runs around each handler, the data page's full header set and its protection from an operator's headers, the failure paths, the rollout, the tests and the untested paths, the open questions, the header and the acceptance lines brought to the series head, and the review log. | +| 2026-10-16 | Fixes from review round 2. The statement that the preparers and filters never run for these addresses on Fastly and Spin, where a request with a method other than `GET` goes on Cloudflare, the matching wording of the second open question, and the review log. | From e94beb649a0b72932001cc483897af382146c0c5 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 08:52:49 +0100 Subject: [PATCH 097/192] The cookie constant design takes its round 2 review Row 27 renames the EIDs cookie constant and row 41 removes it from core, and #1304 is named as closed unmerged in favour of #1358. --- .../specs/2026-10-12-cookie-constant-design.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-cookie-constant-design.md b/docs/superpowers/specs/2026-10-12-cookie-constant-design.md index 195397969f..554fed7064 100644 --- a/docs/superpowers/specs/2026-10-12-cookie-constant-design.md +++ b/docs/superpowers/specs/2026-10-12-cookie-constant-design.md @@ -3,9 +3,9 @@ **Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1358](https://github.com/IABTechLab/trusted-server/pull/1358), answering [IABTechLab/trusted-server#1275](https://github.com/IABTechLab/trusted-server/issues/1275). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-07-30-permission-model-design.md` -**Related PRs:** IABTechLab/trusted-server#1358, which is row 17 of the stack, builds on auction telemetry as a module in [IABTechLab/trusted-server#1357](https://github.com/IABTechLab/trusted-server/pull/1357) and was reviewed on [jwrosewell/trusted-server#12](https://github.com/jwrosewell/trusted-server/pull/12) (closed). The permission model in [IABTechLab/trusted-server#1345](https://github.com/IABTechLab/trusted-server/pull/1345) added the constant this change moves, and the integration seam is in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) +**Related PRs:** IABTechLab/trusted-server#1358, which is row 17 of the stack, builds on auction telemetry as a module in [IABTechLab/trusted-server#1357](https://github.com/IABTechLab/trusted-server/pull/1357) and was reviewed on [jwrosewell/trusted-server#12](https://github.com/jwrosewell/trusted-server/pull/12) (closed). [IABTechLab/trusted-server#1304](https://github.com/IABTechLab/trusted-server/pull/1304), the earlier pull request on this repository for the same commit, is closed without being merged, and the review continues on IABTechLab/trusted-server#1358. The permission model in [IABTechLab/trusted-server#1345](https://github.com/IABTechLab/trusted-server/pull/1345) added the constant this change moves, and the integration seam is in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) **Issues:** [IABTechLab/trusted-server#1275](https://github.com/IABTechLab/trusted-server/issues/1275). [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-15 +**Last updated:** 2026-10-16 > **Why this spec exists.** `COOKIE_MTM_PREF`, the name of the cookie that the Preference Management Platform (PMP) writes the Model Terms for Marketing preference into, is declared in core's constants on the rows below this one, although the Model Terms signal module is the only code that reads the cookie. This change declares the name in that module and changes no behavior. @@ -54,7 +54,7 @@ Only the Axum adapter's tests send the cookie through the module, in `permission ## 4. What is not in this change -- **Other cookie names in core's constants.** `COOKIE_TS_EIDS` (line 14), `COOKIE_SHAREDID` (line 16) and the four names of the IAB consent cookies, `euconsent-v2`, `__gpp`, `__gpp_sid` and `us_privacy` (lines 151 to 154), each name the cookie of one scheme or one vendor and stay in core's constants on this row. The Fastly adapter reads `COOKIE_SHAREDID` at lines 636 and 955 of its `src/app.rs`. Row 41 ([IABTechLab/trusted-server#1382](https://github.com/IABTechLab/trusted-server/pull/1382)) moves the name of the EIDs cookie out of core, and row 42 ([IABTechLab/trusted-server#1383](https://github.com/IABTechLab/trusted-server/pull/1383)) moves the consent cookie names. `COOKIE_SHAREDID` is still in core on row 50. +- **Other cookie names in core's constants.** `COOKIE_TS_EIDS` (line 14), `COOKIE_SHAREDID` (line 16) and the four names of the IAB consent cookies, `euconsent-v2`, `__gpp`, `__gpp_sid` and `us_privacy` (lines 151 to 154), each name the cookie of one scheme or one vendor and stay in core's constants on this row. The Fastly adapter reads `COOKIE_SHAREDID` at lines 636 and 955 of its `src/app.rs`. The EIDs cookie's name changes at row 27 and leaves core at row 41. Row 27 ([IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), which lets a module name the browser's cookies) replaces `COOKIE_TS_EIDS` with `COOKIE_EIDS`, which holds `eids` and stays at line 17 of core's `constants.rs` through row 40, and core writes the cookie's whole name after the deployment's public prefix, so the name is `ts-eids` under core's own prefix. Row 41 ([IABTechLab/trusted-server#1382](https://github.com/IABTechLab/trusted-server/pull/1382)) deletes `COOKIE_EIDS` and the Prebid crate declares the name instead, as `EID_COOKIE` (`crates/auction/prebid/src/eids.rs`, line 24), so core names the EIDs cookie nowhere from row 41. Row 42 ([IABTechLab/trusted-server#1383](https://github.com/IABTechLab/trusted-server/pull/1383)) moves the consent cookie names. `COOKIE_SHAREDID` is still in core on row 50. - **The README and the guide.** The README of core's permission signal module (`crates/trusted-server-core/src/permission_signal/README.md`) names `__mtm_pref` in a sentence on lines 39 and 40, and the Edge Cookies guide (`docs/guide/edge-cookies.md`, line 96) lists the cookie among the signals that the permission signal modules read. Neither changes here. The README is the only file in `crates/trusted-server-core` that names the cookie, and it still does on row 50. Open question 2 asks about the README. - **The permission model design.** `2026-07-30-permission-model-design.md`, section 4.6, still says that the cookie's constant is `COOKIE_MTM_PREF` in core, and this change does not amend that document. - **What reaches Prebid Server.** The cookie reaches Prebid Server in every `consent_forwarding` mode, as it did before this change. Only the `openrtb_only` mode removes cookies from the forwarded `Cookie` header (line 1145 of `crates/auction/prebid-server/src/lib.rs`), and the mode removes the four in `CONSENT_COOKIE_NAMES` (`crates/trusted-server-core/src/cookies.rs`, lines 22 to 27), which do not include `__mtm_pref`. Open question 4 asks whether that is intended. @@ -95,6 +95,7 @@ These are for the reviewer. This revision records what the code does and decides ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/17-cookie-constant` at `af8af38f5`. Issues found, three of them blocking. The blocking issues were that the spec said the constant arrived in the module documentation set pull request when the permission model pull request added it, that the header sent reviewers to pull requests that are now closed and never named the open one, and that section 3 implied that a deployment's settings can select the change when no setting does. This revision names the pull requests that exist now, says where the constant arrived, lists the three files and what each changes, states that no setting is involved and that every deployment behaves as before, and adds the adapters, the names left in core, the acceptance checks and the untested paths, while four open questions wait for code changes, being the missing test of the cookie's name, the README's mention of the cookie, one stale comment and the cookie reaching Prebid Server. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/17-cookie-constant` at `af8af38f5`. Approved, none blocking. This revision names IABTechLab/trusted-server#1304, the earlier pull request on this repository, as closed without being merged, and says that the EIDs cookie's constant is renamed `COOKIE_EIDS` and stays in core's constants from row 27 until row 41 deletes it, where the earlier text said that row 41 alone moves the name out of core. ## Revision record @@ -102,3 +103,4 @@ These are for the reviewer. This revision records what the code does and decides | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The header and the pull requests it names, where the constant arrived, the three files and what each changes, the reworded comment, why a constant and not a setting, what changes for a deployment and for code, the adapters, the names left in core, the acceptance checks, the untested paths, the open questions and the review log. | +| 2026-10-16 | Fixes from review round 2. The earlier pull request on this repository, closed without being merged, named in the header, the rows at which the EIDs cookie's constant is renamed and then deleted from core (section 4), and the review log. | From cabf6e9324af871d9a21ffda2aada8c8fa273dcd Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 09:01:56 +0100 Subject: [PATCH 098/192] The attestation design takes its round 2 review The host row, the consent and privacy section and acceptance items 4 and 9 now match what the code at 59e09b6cd does and what its tests check, including the forwarded scheme headers Axum and Cloudflare pass on. --- .../specs/2026-10-12-attestation-design.md | 32 ++++++++++--------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-attestation-design.md b/docs/superpowers/specs/2026-10-12-attestation-design.md index 77764e2e35..3bbfff924f 100644 --- a/docs/superpowers/specs/2026-10-12-attestation-design.md +++ b/docs/superpowers/specs/2026-10-12-attestation-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-01-15-attestation-design.md` **Related PRs:** IABTechLab/trusted-server#1353, which is row 12 of the stack, builds on one app-config blob for each host in [IABTechLab/trusted-server#1352](https://github.com/IABTechLab/trusted-server/pull/1352) and was reviewed on [jwrosewell/trusted-server#7](https://github.com/jwrosewell/trusted-server/pull/7) (closed without merging on 15 October). [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) opens the integration so that a vendor module can live outside core, and [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354), row 13, follows this change **Issues:** [IABTechLab/trusted-server#161](https://github.com/IABTechLab/trusted-server/issues/161) asks for attestation, [IABTechLab/trusted-server#1235](https://github.com/IABTechLab/trusted-server/issues/1235) asks for attested release binaries, and [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-15 +**Last updated:** 2026-10-16 > **Why this spec exists.** The attestation issue (#161) asks for a reviewed design before any attestation lands. This spec is that design for the code at `59e09b6cd`. It states what the code does, what the code leaves out and which questions about the code are open. @@ -55,16 +55,16 @@ verify_url = "https://verifier.example/verify?host=publisher.example" The envelope carries the evidence's exact bytes as signed, in base64url (base64 written with `-` and `_` in place of `+` and `/`, and with no `=` padding), the signature with its key id and algorithm, a readable copy of the evidence that no check reads, and the address of the verifier. The evidence has these fields, in this order. -| Evidence field | Source | -| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `v` | The format version, `1` | -| `operator` | `[attestation] operator` | -| `publisher` | `publisher.domain` in the settings | -| `host` | The request's `Host` header, lower case, with any numeric port removed. The request address's host stands in only when there is no `Host` header. A forwarded host header is never read | -| `platform` | `name`, `serviceId`, `serviceVersion`, `staging` and `pop`, in that order. On Fastly, `name` is `fastly`. `serviceId`, `serviceVersion` and `pop` come from `FASTLY_SERVICE_ID`, `FASTLY_SERVICE_VERSION` and `FASTLY_POP` and read `unknown` when the variable is absent or empty. `staging` is true only when `FASTLY_IS_STAGING` is `1` | -| `build` | `version`, `builtAt`, `commit`, `run` and `tag`, in that order. `version` is the crate version. The other four are the facts the build was given, and `commit`, `run` and `tag` read `unknown` when the build was given none | -| `nonce` | The caller's `nonce` query parameter, only when one was sent | -| `issuedAt` | The platform's clock, RFC 3339 in UTC to the second | +| Evidence field | Source | +| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `v` | The format version, `1` | +| `operator` | `[attestation] operator` | +| `publisher` | `publisher.domain` in the settings | +| `host` | The request's `Host` header, lower case, with any numeric port removed. The request address's host stands in when there is no `Host` header and when the header is not readable text, meaning that it holds a byte that is neither printable ASCII nor a tab. The field is empty when the request address has no host either. A forwarded host header is never read | +| `platform` | `name`, `serviceId`, `serviceVersion`, `staging` and `pop`, in that order. On Fastly, `name` is `fastly`. `serviceId`, `serviceVersion` and `pop` come from `FASTLY_SERVICE_ID`, `FASTLY_SERVICE_VERSION` and `FASTLY_POP` and read `unknown` when the variable is absent or empty. `staging` is true only when `FASTLY_IS_STAGING` is `1` | +| `build` | `version`, `builtAt`, `commit`, `run` and `tag`, in that order. `version` is the crate version. The other four are the facts the build was given, and `commit`, `run` and `tag` read `unknown` when the build was given none | +| `nonce` | The caller's `nonce` query parameter, only when one was sent | +| `issuedAt` | The platform's clock, RFC 3339 in UTC to the second | The Axum, Cloudflare and Spin adapters report their own name in `platform.name`, `unknown` for `serviceId`, `serviceVersion` and `pop`, and `false` for `staging`. @@ -139,7 +139,7 @@ The same issue asks "Which build facts the runtime can report for #161, such as **Failure.** A failure stays at the attestation addresses, so no other address of the deployment changes. No other key signs in place of a key that cannot be read, because the key in force is the latest to have started. The build stops for the schedule faults listed under the build inputs. A key that is not a P-256 private key does not stop the build, and the endpoint then answers `503` from the second that key comes into force. The handler reads the time from `current_timestamp`, which gives `0` when the clock cannot be read as a time after the Unix epoch (`ec/mod.rs`, lines 1324 to 1326, and `attestation.rs`, line 337). With such a clock the endpoint answers `503` unless the schedule holds a key that starts at second `0` or earlier. The evidence is then signed with that key and says it was issued at `1970-01-01T00:00:00Z`. The code also has `checked_current_timestamp`, and the comment on that function says to use it instead of `current_timestamp` when a fallback could authorize a time-bounded decision. The handler does not use `checked_current_timestamp`. Open question 4 asks whether the handler should use it and answer `503` when the clock fails. -**Consent and privacy.** The handler reads the request's `Host` header, the request's `nonce` parameter and the scheme the request arrived over. It reads no cookie, no consent signal and no store, makes no outbound request and sets no cookie. The evidence holds the host and the nonce that the caller sent and facts about the deployment, and nothing else about the caller. The envelope tells whoever asks the operator's name, the publisher's domain, the service id and version, the location, the commit, the build run, the tag and the build time. Every answer allows any origin to read it. +**Consent and privacy.** The handler reads the request's `Host` header, the request's `nonce` parameter and the headers that name the scheme. The handler takes the scheme from the first of `Forwarded`, `X-Forwarded-Proto` and `Fastly-SSL` that gives one and uses `http` when none does, because the handler reads no TLS details of the connection. Only the JSON address that the page gives to copy uses the scheme, so the evidence never holds the scheme. Fastly and Spin remove those three headers from the request and then set the scheme themselves, Fastly from the TLS connection and Spin from the full URL that the Spin runtime reports. Axum and Cloudflare pass the headers on as they arrived, so on those two adapters the caller, or a proxy in front of the deployment, decides the scheme in the JSON address on the page. The handler reads no cookie, no consent signal and no store, makes no outbound request and sets no cookie. The evidence holds the host and the nonce that the caller sent and facts about the deployment, and nothing else about the caller. The envelope tells whoever asks the operator's name, the publisher's domain, the service id and version, the location, the commit, the build run, the tag and the build time. Every answer allows any origin to read it. ## 4. What is not in this change @@ -182,12 +182,12 @@ Test builds carry no key, so no router test sees signed evidence, and only the h 1. With no `[attestation]` section no attestation route is registered, and a read of `/_ts/attestation.json` reaches the publisher's origin (`attestation_is_not_served_without_its_section_parity`, `without_the_section_no_attestation_route_is_registered`). 2. With the section and a key in force, a GET of the endpoint's `.json` address answers `200` with an envelope whose signature verifies, with the public key alone, over the context, a line feed and the payload (the harness, `the_signature_covers_the_context_a_line_feed_and_the_payload`). 3. That signature does not verify with the other key of the schedule, under another context or once the host in the payload changes (the harness, `an_altered_payload_fails_verification`). -4. The evidence names the host the request arrived for, lower case and without a port, and a forwarded host header never replaces it. The publisher comes from the settings (`forwarded_host_headers_never_become_the_signed_host`, `the_signed_publisher_comes_from_the_settings`, the harness). +4. The evidence names the host the request arrived for, lower case and without a port, and a forwarded host header never replaces it. The publisher comes from the settings (`the_evidence_shown_is_the_payload_signed` and `the_host_falls_back_to_the_request_address` for the case and the port, `forwarded_host_headers_never_become_the_signed_host`, `the_signed_publisher_comes_from_the_settings`, the harness). 5. A nonce is signed back as sent, a malformed nonce answers `400`, and with no nonce the evidence carries none (`a_nonce_is_signed_back_as_sent`, `a_malformed_nonce_is_refused`, `no_nonce_is_signed_unless_asked_for`). 6. A build given no schedule, an empty schedule or blank lines answers `503` at the endpoint, as does a time before the first key and a key that cannot be read (`a_build_given_no_schedule_compiles_an_empty_one`, `no_usable_key_in_force_answers_service_unavailable`). 7. A schedule line of the wrong shape, or one that repeats a key id or a start, stops the build without showing the key (`a_schedule_line_of_any_other_shape_is_refused_without_showing_the_key`, `a_schedule_repeating_a_key_id_or_a_start_is_refused`). 8. The key in force is the latest to have started, and the warning for the last key is given once for each process (`the_key_in_force_is_the_latest_to_have_started`, `the_last_key_warning_is_given_once`). -9. Every answer carries `Cache-Control: no-store`, and the page and the envelope carry `Access-Control-Allow-Origin: *` (`answers_are_never_cached_and_readable_from_any_origin`, the harness). +9. Every answer carries `Cache-Control: no-store`, and the page and the envelope carry `Access-Control-Allow-Origin: *` (`answers_are_never_cached_and_readable_from_any_origin` for both headers on the page and the envelope, the harness for both on the envelope, and `attestation_prefix_parity`, `attestation_default_address_redirect_parity` and the Fastly adapter's tests for `Cache-Control` on the list at a prefix, the redirect, the `404` pages and the `503` answer of a build with no key). No test checks the headers of the `400` and `500` answers, of the `503` answer for a key that cannot be read, or of the `404` answer in JSON that the handler gives when the settings carry no section. 10. An endpoint of the wrong shape, at a reserved address or under a framework asset prefix is refused when the settings load, with the reason (`the_endpoint_is_an_absolute_path_under_an_underscore_prefix`, `an_endpoint_at_a_reserved_address_is_refused_naming_it`, `an_endpoint_under_a_framework_asset_prefix_is_refused_naming_it`), and every fixed route beneath an underscore prefix is in the reserved list on all four adapters. 11. Under a prefix of the deployment's own, every address and method the pages do not serve answers `404` from the deployment on all four adapters, and a read of the default address redirects to the endpoint with the query kept (`attestation_prefix_parity`, `attestation_default_address_redirect_parity` and the Fastly adapter's tests). @@ -202,11 +202,12 @@ Test builds carry no key, so no router test sees signed evidence, and only the h **Local checks.** The test plan in the description of IABTechLab/trusted-server#1353 ticks `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter, and leaves a manual run with `fastly compute serve` unticked. -**What no test covers.** The paths listed in section 5, which Open questions 2 to 5 ask about. +**What no test covers.** The paths listed in section 5, which Open questions 2 to 5 ask about, and the answers whose headers item 9 says no test checks. ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/12-attestation` at `59e09b6cd`. Issues found, four of them blocking. The four were that the `build` fields left out `build.tag`, that the spec said every adapter reports `staging` or `unknown` when Axum, Cloudflare and Spin sign `staging: false`, that the spec counted 37 tests where the module has 40, and that the header and the acceptance named pull requests that have closed and a count of 19 checks where the open pull request shows 25. This revision states what the code does at that head for the evidence, the build inputs, the configuration, the adapters, the answers, the failures, the compatibility of the section, the privacy of the answers, the tests and the acceptance, and leaves eight questions about the code under Open questions until the code changes, being what `staging` says off Fastly, the request preparers on Cloudflare, the build's check of a key, the clock, the missing tests, how a verifier gets the public keys, the rule for new evidence fields and the order of `run` and `tag` in the guide. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/12-attestation` at `59e09b6cd`. Issues found, none blocking. This revision says that the request address's host also stands in for the `Host` header when the header is not readable text, names the scheme headers under "Consent and privacy" and what they change, and names the tests behind Acceptance 4 and 9 and what no test checks. ## Revision record @@ -214,3 +215,4 @@ Test builds carry no key, so no router test sees signed evidence, and only the h | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The header and the pull requests it names, the `build` fields and the `tag`, the configuration fields and the endpoint rules, the methods, the adapters and what each reports for `staging`, the answers and the failures, the compatibility of the section, the privacy of the answers, the scope table and the quotations from the two issues, the key custody and what it costs, the test count and the tests, the acceptance, the open questions and the review log. | +| 2026-10-16 | Fixes from review round 2. The `host` fallback when the `Host` header is not readable text, the scheme headers and what they change under "Consent and privacy", the tests named in Acceptance 4 and 9 and what no test checks there, and the review log. | From ef591ca29ce08b1f133331db4ce616401c2ac110 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 09:16:48 +0100 Subject: [PATCH 099/192] The prefix module design takes its round 2 review Adds the two new public fields to the list of new public items and records that the code scanning alert on the module context has a second data flow starting in SelectedPrefixModule::default. --- docs/superpowers/specs/2026-10-12-prefix-module-design.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-prefix-module-design.md b/docs/superpowers/specs/2026-10-12-prefix-module-design.md index 67a3ded4ac..72704fd406 100644 --- a/docs/superpowers/specs/2026-10-12-prefix-module-design.md +++ b/docs/superpowers/specs/2026-10-12-prefix-module-design.md @@ -88,7 +88,7 @@ The registry then asks the selected module once, as a trial, for the prefixes of - The cookie constants hold the entry's name alone. `COOKIE_TS_EC`, `COOKIE_TS_EC_RESOLVED`, `COOKIE_TS_EC_PULL_COMPLETE`, `COOKIE_TS_EIDS` and `COOKIE_TS_TESTER` are now `COOKIE_EC`, `COOKIE_EC_RESOLVED`, `COOKIE_EC_PULL_COMPLETE`, `COOKIE_EIDS` and `COOKIE_TESTER`, with the values `ec`, `ecr`, `ec-pull-complete`, `eids` and `tester` (`constants.rs` lines 3 to 19). A crate outside core that read them writes `services.public_prefix().cookie_name(COOKIE_EC)` instead. Row 41 (the Prebid crate reads the cookie its script writes, [IABTechLab/trusted-server#1382](https://github.com/IABTechLab/trusted-server/pull/1382)) removes `COOKIE_EIDS` from core's constants. - `ServerKeyPrefix::for_settings` and `ServerKeyPrefix::for_domain` are removed. `ServerKeyPrefix::new(head, domain)` replaces both, and a caller takes `services.server_key_prefix()` where it had the settings (`platform/server_key.rs` lines 24 to 36). - Public functions that read, write or check a cookie by name take the public prefix as a new argument. They are `handle_data` (`inspect/data.rs` line 49), `handle_set_tester` and `handle_clear_tester` (`tester_cookie.rs` lines 50 and 91), `set_ec_cookie`, `expire_ec_cookie` and `expire_ec_resolved_marker` (`ec/cookies.rs` lines 89, 122 and 158), `clear_ec_on_response` (`ec/finalize.rs` line 511) and `reserved_response_effect` (`ec/module.rs` line 275). -- The new public items are the `prefix` module, `RuntimeServices::{prefixes, public_prefix, server_key_prefix, with_prefixes}`, `IntegrationRegistry::{prefix_module, prefixes}`, `IntegrationRegistrationBuilder::with_prefix_module`, `EcContext::public_prefix`, `HtmlProcessorConfig::with_public_prefix` and the `[prefix]` section, which is `Settings::prefix`. +- The new public items are the `prefix` module, `RuntimeServices::{prefixes, public_prefix, server_key_prefix, with_prefixes}`, `IntegrationRegistry::{prefix_module, prefixes}`, `IntegrationRegistrationBuilder::with_prefix_module`, `EcContext::public_prefix`, `HtmlProcessorConfig::with_public_prefix`, the two public fields those builders set, `IntegrationRegistration::prefix_module` (`registry.rs` line 677) and `HtmlProcessorConfig::public_prefix` (`html_processor.rs` line 120), and the `[prefix]` section, which is `Settings::prefix`. **For a deployment.** @@ -145,7 +145,7 @@ When the Fastly state failed to build, the entry point's finalize services carry These are for the reviewer. This revision records what the code does and decides none of them. -1. Should `Prefixes::trusted_server(settings: &Settings)` (`prefix.rs` line 238) take only the field it reads, being `publisher.domain`, in place of the whole settings with its secrets? CodeQL fails on IABTechLab/trusted-server#1368 with two high alerts of the rule "Cleartext logging of sensitive information", [216](https://github.com/IABTechLab/trusted-server/security/code-scanning/216) at `auction/orchestrator.rs` line 1613, which logs a count of planned auction providers, and [217](https://github.com/IABTechLab/trusted-server/security/code-scanning/217) at `module_context/inject.rs` line 158, which logs a module's name and why the module was skipped. Neither line logs anything sensitive. The data flow CodeQL reports for both starts at the call `SelectedPrefixModule::trusted_server()` in `resolve_prefix_module` (`registry.rs` line 1286), which takes no argument, and does not pass through `Prefixes::trusted_server`. Whether to narrow that input, rename the constructors or dismiss both alerts with a reason is not decided here. +1. Should `Prefixes::trusted_server(settings: &Settings)` (`prefix.rs` line 238) take only the field it reads, being `publisher.domain`, in place of the whole settings with its secrets? CodeQL fails on IABTechLab/trusted-server#1368 with two high alerts of the rule "Cleartext logging of sensitive information", [216](https://github.com/IABTechLab/trusted-server/security/code-scanning/216) at `auction/orchestrator.rs` line 1613, which logs a count of planned auction providers, and [217](https://github.com/IABTechLab/trusted-server/security/code-scanning/217) at `module_context/inject.rs` line 158, which logs a module's name and why the module was skipped. Neither line logs anything sensitive. The data flow CodeQL reports for both starts at the call `SelectedPrefixModule::trusted_server()` in `resolve_prefix_module` (`registry.rs` line 1286), which takes no argument. The alert at `module_context/inject.rs` line 158 has a second flow, which starts at the same call in `SelectedPrefixModule::default` (`prefix.rs` line 312). No flow passes through `Prefixes::trusted_server`. Whether to narrow that input, rename the constructors or dismiss both alerts with a reason is not decided here. 2. Should each of these comments and guide paragraphs say what the code does? `prefix.rs` lines 10, 103 and 155, `html_processor.rs` lines 118 and 1185, `docs/guide/configuration.md` line 949 and `docs/guide/integration-guide.md` line 788 name first-party storage keys, which nothing writes. `docs/guide/configuration.md` line 959 puts the identity-graph rows under the key prefix, where they take the Edge Cookie's identifier. The Fastly `middleware.rs` lines 94 to 96 say the geo module's answers are held under the prefix, and the Fastly `app.rs` line 487 says the publisher a request resolved to comes with "its tenant included", yet no geo code reads the prefix and nothing resolves a tenant. `settings.rs` lines 3997 and 4000, `docs/guide/configuration.md` lines 627 to 649 and `docs/guide/api-reference.md` lines 330 and 363 name `ts-tester` alone. 3. Should the startup trial, or the types, check more? The trial refuses a module only for naming a value its call lacks. `ServerKeyPrefix::new` takes any head and any domain (`platform/server_key.rs` lines 24 to 36). A module has no way to refuse when `PublicPrefix::new` returns `None`, because the trait's error type is `Withheld` (`prefix.rs` lines 161 and 168), so the probe calls `expect` (`crates/testing/seam-probe/src/lib.rs` line 400). An adapter that builds services and never calls `with_prefixes` gets `Prefixes::default()` without a word (`platform/types.rs` line 798). Nothing checks that a module's server key prefix carries the publisher's domain, or that it does not depend on the request's host. 4. Should tests cover the seven cases listed under "What the tests do not prove" in section 4? @@ -154,6 +154,7 @@ These are for the reviewer. This revision records what the code does and decides ## Review log 1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/27-prefix-module` at `9a9993423`. Issues found, 5 of them blocking. The five were that the spec named browser storage keys that the code never writes, that it said every shared store key takes the prefix when only the robots.txt answers do, that it said the public prefix carries no domain and the key domain never comes from the request, when the code allows a domain in the public prefix and only core's module follows the rule, that its acceptance section cited per-commit checks and 8 deliberate faults that neither pull request description records and 19 green checks that belong to the fork while a code scanning check fails on the pull request itself, and that its list of what is not in the change held the constant rename the change makes and left out four other changes. This revision states which cookies the public prefix names, the one use of the server key prefix and what keeps its own name, where the domain comes from, when a refused selection shows on each adapter, what changes for code and for a deployment, the adapters and the tests, and leaves five questions for code changes, being the two code scanning alerts, comments and guide text that describe what the code does not do, checks the startup trial does not make, tests that do not exist and the template key cookie check. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/27-prefix-module` at `9a9993423`. Issues found, none blocking. This revision adds the two new public fields to the list of new public items and names the second place the code scanning data flow starts. ## Revision record @@ -161,3 +162,4 @@ These are for the reviewer. This revision records what the code does and decides | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The status line and the pull requests it names, the cookies the public prefix names, the one use of the server key prefix and what keeps its own name, where the domain comes from, when a refused selection shows, what changes for code and for a deployment, the adapters, the acceptance rebuilt from the pull request description and the tests, the open questions and the review log. | +| 2026-10-16 | Fixes from review round 2. The list of new public items and where the code scanning data flow starts. | From 431b0aab0374e6530db26588288a9c94d90144d1 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 09:16:55 +0100 Subject: [PATCH 100/192] The Google tag module design takes its round 2 review Says a content security policy that allows inline scripts by hash blocks the snippets unless it lists their hashes, which the settings fix, and that the consent snippet follows Google's published consent guide. --- .../specs/2026-10-12-google-tag-module-design.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-google-tag-module-design.md b/docs/superpowers/specs/2026-10-12-google-tag-module-design.md index e371b0b658..bfd4aadba6 100644 --- a/docs/superpowers/specs/2026-10-12-google-tag-module-design.md +++ b/docs/superpowers/specs/2026-10-12-google-tag-module-design.md @@ -57,7 +57,7 @@ At least one of the three ids is required, because a table naming none asks the - The command always carries `wait_for_update`, 500 milliseconds unless the settings say otherwise, so the tags wait that long for a consent management platform's update before they fire with the defaults. - When `consent_defaults.region` lists codes, the codes are written as the `region` parameter of the same command, and the module then writes no command without a region (open question 3). - The module writes no `update` command and no other code in this repository writes one, so a type leaves its default only through code from outside the repository, such as a consent management platform's own. -- The snippet is the full script block of Google's consent guide without the guide's comments, with `wait_for_update` added and `region` added where it is set, and no test holds the snippet to the guide's text. +- The snippet follows the script block in [Google's published consent guide](https://developers.google.com/tag-platform/security/guides/consent), which `crates/tag/google-tag/src/lib.rs` line 314 cites, without the guide's comments, with `wait_for_update` added and `region` added where it is set, and no test holds the snippet to the guide's text. **The Tag Manager module.** `tag.google-tag-manager` is a `[[fetch]]` middleware that points the Google tag addresses a page already carries at the Tag Manager module's own first-party path, and serves those scripts from the publisher's origin. This module writes the tags on a page that carries none. Where each loader points is settled when the module registers, because markup written by a middleware never reaches the handlers that rewrite attributes. This module names a first-party `gtag/js` path for a tag id that is the Tag Manager module's `container_id` or is in its `allowed_tag_ids`, and a first-party `gtm.js` path only for the container that is the Tag Manager module's `container_id`, because that endpoint always answers with the container it is configured for. Otherwise, and whenever the Tag Manager module is not selected, the loader names Google's own address. The AdSense loader and the noscript iframe always name Google's address, because the Tag Manager module has no route for either. The Tag Manager crate offers `tag_id_is_well_formed`, `container_id_is_well_formed`, `first_party_gtag_js_path` and `first_party_gtm_js_path`, and this module calls all four, so the two modules accept the same id shapes. The rule for which tag ids the Tag Manager module serves is written twice in that crate, once in `first_party_gtag_js_path` and once in the check the route makes (`tag_id_is_allowed`), so the two agree because their text matches and not because they share code (open question 8). @@ -71,7 +71,7 @@ At least one of the three ids is required, because a table naming none asks the **Adapters.** The module needs nothing from an adapter. All four adapters, Fastly, Axum, Cloudflare and Spin, take the stock list of modules in `crates/trusted-server-modules/src/lib.rs`, where this module is listed just before the Tag Manager module. Nothing depends on that order, because the two modules run in different phases and the order of middleware within a phase is the order an entry names them, not the order of the list. The module's tests run in CI under `cargo test-fastly` and `cargo test-axum`, and not under `cargo test-cloudflare` or `cargo test-spin`, because those two aliases test only their own adapter's package. Row 43 ([IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384), `2026-10-14-no-modules-crate-design.md`) removes the crate that holds the stock list and gives each adapter its own list, with this module still listed before the Tag Manager module. -**Content-Security-Policy.** The inline scripts the module writes, being the consent defaults, the container script and the configuration script of the tag snippet, carry no `nonce` attribute, and the module reads and writes no policy header. A browser blocks them on a page whose policy allows inline scripts only by nonce or hash. +**Content-Security-Policy.** The inline scripts the module writes, being the consent defaults, the container script and the configuration script of the tag snippet, carry no `nonce` attribute, and the module reads and writes no policy header. A browser blocks them on a page whose policy allows inline scripts only by nonce. A policy that allows inline scripts by hash blocks them unless it lists their hashes. The text of each snippet is fixed by the settings (`crates/tag/google-tag/src/lib.rs` lines 319 to 349 and 359 to 370, with the loader addresses decided once at registration, lines 619 to 635), so the hashes stay the same from page to page. **The ad tag module's loader.** The same pull request adds one setting to the Google ad tag module, `[ad-tag.google] write_loader` (commit `59e966e95`). That module handles Google Publisher Tag (GPT), Google's script for ad slots, and writes the bootstrap, the shim and the slot handoff into the head in the fetch phase. Without the setting, a page that carries no GPT loader gets those and nothing that loads GPT, so its slots are never filled. With `write_loader = true` the module also writes `` into the head, last among its own head markup and after the shim and the bootstrap that the loader relies on, so the loader is fetched from the module's own first-party route. The setting is `false` by default, because a page that already carries the loader would load it twice. @@ -112,6 +112,7 @@ CI passed on all 25 checks of the draft pull request at `1940d63f1`, and on all ## Review log 1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/30-google-tag` at `1940d63f1`. Issues found, 5 of them blocking. The five were that the spec said the snippets land at the start of the head when they land before the script bundle's tag and after what a `[[fetch]]` middleware wrote, that it gave a reason for the module's place in the stock list that does not hold, that it never mentioned the `write_loader` setting the pull request adds to the Google ad tag module, that its consent design left out `wait_for_update`, `region` and the states a consent type can take, and that its header called a closed fork pull request ready for review and named a closed pull request as the place of the specs. This revision states what the code does at that head for each of the five, and adds the settings, the failure cases, the adapters, the tests that hold each claim and the code questions the review raised, which wait for code changes under Open questions. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/30-google-tag` at `1940d63f1`. Issues found, none blocking. This revision says what a policy that allows inline scripts by hash does with the snippets and that the consent snippet follows Google's published guide. ## Revision record @@ -119,3 +120,4 @@ CI passed on all 25 checks of the draft pull request at `1940d63f1`, and on all | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header and the pull request pointers, where the head snippets land, the reason for the stock list order, the settings, the consent defaults, the Tag Manager module, the ad tag module's loader setting, failure, adapters, compatibility, the content security policy, the tests, the open questions and the review log. | +| 2026-10-16 | Fixes from review round 2. The content security policy and the source of the consent snippet. | From b1eca1743dc838fdc1f561ea591904202386a2ae Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 09:16:59 +0100 Subject: [PATCH 101/192] The viewport sizes design takes its round 2 review Says the page-bids request carries no viewport width, sending only the path, the cookies and the X-TSJS-Page-Bids header, and cites the Spin health route at line 649. --- docs/superpowers/specs/2026-10-12-viewport-sizes-design.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md b/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md index 33d18b961e..dc46ea4d90 100644 --- a/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md +++ b/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md @@ -70,7 +70,7 @@ The bands reach the ad server request for a slot that Trusted Server defines and **The refresh auction.** The refresh auction compares `window.innerWidth` with the start of each band every time a publisher's refresh runs, and offers the slot the sizes of the widest band that the width reaches (`injectedSlotBandSizes`, `js/integrations/prebid/index.ts`, lines 1137 to 1146). Those sizes become the banner sizes of the auction's ad unit (same file, lines 2762 to 2768). The bundle's Prebid adapter posts the ad unit, with its banner sizes, to the auction endpoint, which is `/auction` unless the Prebid configuration names another path (same file, lines 2357 and 2375 to 2395, with `buildAdRequest` in `js/core/auction.ts`, lines 76 to 114), and the server reads the sizes as the formats of the slot (`convert_tsjs_to_auction_request`, `core/auction/formats.rs`, lines 168 to 217). The ad unit also lists the client-side bidders that the publisher configured for the slot (same file, line 2784), so Prebid asks those bidders about the same sizes. A slot whose band is empty at the current width is left out of the auction (`js/integrations/prebid/index.ts`, lines 1154 to 1157 and 2720 to 2724), its stale targeting is cleared as for every slot left out, and GPT still refreshes the slot. A slot with no bands, or a slot that the bundle cannot match to a slot of the page data, is offered the sizes it was offered before. -**The server-side auctions.** Both server-side auctions build their slots with `to_ad_slot`, which lists every format whatever the bounds (`build_auction_request`, `core/publisher.rs`, lines 6308 to 6312, called for the initial page at line 5521 in `handle_publisher_request` and for page-bids at line 7906 in `handle_page_bids`). The server holds the bands but reads no viewport width from either request. The page-bids request carries the path alone (`js/integrations/gpt/index.ts`, line 1564), and a search of the crates and the guides for the `Viewport-Width` and `Sec-CH-Viewport-Width` headers (section 6) finds none. So the server cannot choose a band for a reader, and a bidder in either auction is still offered a billboard on a phone, which is the problem section 1 describes. A bid for a size outside the band can therefore win. With the slot above, a 970x250 bid can win on a phone 375 pixels wide whose mapping allows only 320x50. Nothing compares the size of a winning bid with the band, and a winning bid that carries no size of its own is sized as the slot's first configured format, not as the band's (`js/integrations/gpt/index.ts`, lines 2182 to 2184). Open question 2 asks about both. What the ad server does with a winner whose size the mapping leaves out was not exercised. +**The server-side auctions.** Both server-side auctions build their slots with `to_ad_slot`, which lists every format whatever the bounds (`build_auction_request`, `core/publisher.rs`, lines 6308 to 6312, called for the initial page at line 5521 in `handle_publisher_request` and for page-bids at line 7906 in `handle_page_bids`). The server holds the bands but reads no viewport width from either request. The page-bids request carries no viewport width, as it sends the path, the cookies and the `X-TSJS-Page-Bids` header and nothing more (`js/integrations/gpt/index.ts`, lines 1564 to 1573), and a search of the crates and the guides for the `Viewport-Width` and `Sec-CH-Viewport-Width` headers (section 6) finds none. So the server cannot choose a band for a reader, and a bidder in either auction is still offered a billboard on a phone, which is the problem section 1 describes. A bid for a size outside the band can therefore win. With the slot above, a 970x250 bid can win on a phone 375 pixels wide whose mapping allows only 320x50. Nothing compares the size of a winning bid with the band, and a winning bid that carries no size of its own is sized as the slot's first configured format, not as the band's (`js/integrations/gpt/index.ts`, lines 2182 to 2184). Open question 2 asks about both. What the ad server does with a winner whose size the mapping leaves out was not exercised. **Slots the publisher defines.** Whoever defines the GPT slot first decides whether the slot has a mapping. Where the publisher defined the slot before `adInit` looked for the slot, Trusted Server reuses the slot and adds no mapping (`tag/gpt_bootstrap.js` lines 700 to 705 and 845 to 851, `js/integrations/gpt/index.ts` lines 1180 to 1184 and 1370 to 1376). The refresh auction does not tell the two cases apart, because `findInjectedSlotForRefresh` (`js/integrations/prebid/index.ts`, line 1173) finds the page data of a slot by element id whoever defined the slot. So a slot that the publisher defined is offered only the band's sizes in the refresh auction, while the ad server request of that slot carries the sizes the publisher defined. Open question 3 asks about this. @@ -82,7 +82,7 @@ The bands reach the ad server request for a slot that Trusted Server defines and **Rollout.** A binary released before these fields refuses a format carrying either of them, because `CreativeOpportunityFormat` denies unknown fields (`core/creative_opportunities.rs`, line 1028), and the binary then cannot load its settings. So the binary that reads the fields goes out before any configuration uses them, and the configuration guide says so (`docs/guide/configuration.md`, lines 2792 to 2795). Rolling back reverses the order. A configuration that still names a bound has to be pushed again without the bound before the binary is rolled back, and the guide does not say so. A bundle from before this row ignores `viewport_sizes`, and a bundle from this row that is given page data without `viewport_sizes` asks for every format, so the page data, the head bootstrap and the bundle can be mixed in either order. -A binary that refuses the settings answers every route with the startup error, which is status 500 on the Axum, Cloudflare and Fastly adapters and status 503 on Spin (`startup_error_router` in each adapter's `src/app.rs`). The route `/health` is the exception on two of the four. The Fastly adapter answers `/health` before it reads any settings (`crates/trusted-server-adapter-fastly/src/lib.rs`, line 267) and the Spin adapter registers `/health` on its startup error router (`crates/trusted-server-adapter-spin/src/app.rs`, line 647), so a health check on those two adapters stays green while every other route fails. +A binary that refuses the settings answers every route with the startup error, which is status 500 on the Axum, Cloudflare and Fastly adapters and status 503 on Spin (`startup_error_router` in each adapter's `src/app.rs`). The route `/health` is the exception on two of the four. The Fastly adapter answers `/health` before it reads any settings (`crates/trusted-server-adapter-fastly/src/lib.rs`, line 267) and the Spin adapter registers `/health` on its startup error router (`crates/trusted-server-adapter-spin/src/app.rs`, line 649), so a health check on those two adapters stays green while every other route fails. **Adapters.** No adapter file changes in this row. All four adapters load their settings through `settings_from_config_blob_with` (`core/config_payload.rs`). In each adapter's own `src/app.rs`, the Cloudflare adapter calls the function at line 191 and the Spin adapter at line 140, and the Fastly adapter at line 238 and the Axum adapter at line 111 call `get_settings_from_config_store_with`, which calls the function at `core/settings_data.rs` line 120. The refusal, the page data and the browser code are therefore the same on each adapter. @@ -151,6 +151,7 @@ The questions below are for the reviewer. This revision records what the code do ## Review log 1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/29-viewport-sizes` at `054f3a2a0`. Issues found, two of them blocking. The blocking issues were that the spec never named the refresh auction, which offers a bidder only the sizes of the band, called the bounds browser only and gave no reason why the server-side auctions are still offered every format, and that its last paragraph claimed four deliberate faults that no description or commit message records. This revision states which requests the bands reach and why the server-side auctions are not among them, what a band is and what an empty band means, which slots get a mapping, when the band takes effect, what a wrong order of rollout and rollback does, what each adapter runs, what the refresh auction tells a bidder about the viewport, the checks recorded and the tests that exist, while six open questions wait for code changes, being the size applied after the first paint, the winning bid outside the band, the slot the publisher defined, the paths no test covers, the mapping calls with no `try` and `catch`, and four places that do not describe the row. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/29-viewport-sizes` at `054f3a2a0`. Issues found, none blocking. This revision says what the page-bids request sends and corrects the line where the Spin adapter registers its health route. ## Revision record @@ -158,3 +159,4 @@ The questions below are for the reviewer. This revision records what the code do | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header, the pull requests it names and the related specs, the three requests that ask for a slot's sizes and which of them the bands reach, the refresh auction that the second commit of the row (`9c2e6e802`) added after the first draft, what a band is and what an empty band means, the bounds and the page data, slots the publisher defines, when the band takes effect and who holds the space, failure, rollout, rollback, the adapters and privacy, the checks recorded with the claim of deliberate faults removed, the tests that exist and those that do not, the open questions and the review log. | +| 2026-10-16 | Fixes from review round 2. What the page-bids request sends and the line of the Spin health route. | From 4482347f9aa12b09ec934352dcee65b2314d4ed6 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 09:30:06 +0100 Subject: [PATCH 102/192] The shared vendor tables design describes the two secret fixes and takes its round 2 review A refused shared or module table names the setting and never its value, and push checks that a shared table's secret settings name a key. The design's reading, secrets and error text sections, its refusal table and its tests now describe that code, and the round 2 review's three points are checked against the code and recorded. --- .../2026-10-12-shared-vendor-tables-design.md | 40 ++++++++++--------- 1 file changed, 21 insertions(+), 19 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md b/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md index e7f947fa3e..4d04690935 100644 --- a/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md +++ b/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md @@ -29,7 +29,7 @@ Four words are used as the code uses them. A builder (`IntegrationBuilder`) is w **The name.** A table is named in lower case letters, digits, `_` or `-`, with at least one and at most 255 bytes and no `.` (`SharedTables::validate_names`, `settings.rs` lines 389 to 409, which uses `module_name::is_valid`). Nothing checks the name against a folder, and no `crates/shared` folder exists at row 26. The 51Degrees client hints change (row 44, [IABTechLab/trusted-server#1386](https://github.com/IABTechLab/trusted-server/pull/1386)) adds `crates/shared/51degrees`, whose `vendor()` takes the table's name from the crate's own folder, and the AWS signing change (row 48, [IABTechLab/trusted-server#1391](https://github.com/IABTechLab/trusted-server/pull/1391)) adds `crates/shared/README.md`, which says a vendor's table is named by its folder. -**Reading a table.** A vendor's crates read their own table with `Settings::shared_config::(vendor)` (`settings.rs` lines 4776 to 4788), where `T` can be read from the table and has validation rules. The call answers `None` when the table is absent. Otherwise the call reads the table as `T` and then validates `T`. The read is strict when the vendor's type refuses unknown keys. A failure of either step is a configuration error that begins `[shared.] could not be read:` or `[shared.] failed validation:` and ends with the underlying error text (`SharedTables::get_typed`, lines 361 to 387). +**Reading a table.** A vendor's crates read their own table with `Settings::shared_config::(vendor)` (`settings.rs` lines 4776 to 4788), where `T` can be read from the table and has validation rules. The call answers `None` when the table is absent. Otherwise the call reads the table as `T` and then validates `T`. The read is strict when the vendor's type refuses unknown keys. A failure of either step is a configuration error that begins `[shared.] could not be read:` or `[shared.] failed validation:` and names the setting at fault, and for a validation the rule it broke, and never the setting's value (`SharedTables::get_typed` and `read_table`, see "Error text"). **Starting a builder from a table.** A builder marked with `IntegrationBuilder::with_shared_settings(vendor)` (`mod.rs` lines 645 to 660) is started by its vendor's table and by no section. The registry is built by `with_plan_and_registrations` (`registry.rs` lines 1617 to 1946), which does the following with the tables and the marked builders. @@ -65,9 +65,9 @@ A builder a table runs has no module name (item 2 above). The function `check_se - When the settings load from the config store (`settings_from_config_blob_with`, `config_payload.rs` lines 67 to 105), the loader first clears each declared secret setting whose own rule says the table does not use the setting. In that step the table counts as selecting its builder (`module_secrets.rs` lines 98 to 128). The loader then replaces the key name in each remaining secret with the value from the secret store. The declaration of the fields (`secret_fields`, lines 136 to 161) makes every level of each path optional, so an absent table or setting is passed over. The vendor's typed read and validate function see the resolved value. - The `/_ts/config` view masks every leaf that the loader recorded as a resolved secret (`inspect/config.rs` lines 324 to 346), and the same pass records the leaves of a shared table (`secret_resolution.rs` lines 124 to 142 and 294 to 299). -- At push the settings hold key names and no values. The check that each secret in use names a key (`validate_secret_key_references`, `config.rs` lines 505 to 524) finds a builder's secrets through the builder's module name and the section that selects it. A builder a table runs has neither, so the check never reaches the secrets of a shared table. `docs/guide/configuration-rules.md` line 134 says every secret setting is checked at push. +- At push the settings hold key names and no values. The check that each secret in use names a key (`validate_secret_key_references` in `config.rs`) finds a module's secrets through the builder's module name and the section that selects it, and the secrets of a builder that carries the mark through its vendor's table when the table is present, with the same rule for whether the table uses each setting. A secret setting a shared table uses that names no key is refused at push as `shared..`, as `docs/guide/configuration-rules.md` line 134 says of every secret setting. -**Error text.** A failed typed read or validation puts the underlying error text into the configuration error (`settings.rs` lines 378 and 383, and the same in `module_config`, lines 4765 and 4770). That text can quote the value that failed. The `serde` crate quotes a string it cannot read as the type asked for (`Unexpected::Str`, `serde_core` 1.0.228, `de/mod.rs` line 410), and the `validator` crate at the locked version 0.20.0 prints the parameters of each failed rule, which include the checked value (`display_impl.rs` lines 5 to 12, and for the `length` rule line 38 of `tokens/length.rs` in `validator_derive`). The table holds a resolved secret when the vendor's validate function runs at startup (`config_payload.rs` lines 93 to 102). Each adapter writes the whole failure to the service's log when the state fails to build (for example line 1503 of the Fastly adapter's `app.rs`) and answers requests with a generic error. Core's own settings are reported through `validation_error_summary` (`settings.rs` line 4975), which names the field, the rule and the rule's own message where it has one, and leaves out the checked value. +**Error text.** A failed typed read or validation of a shared table, and of a module's own table in `module_config`, names the table and the setting at fault and never quotes the setting's value (`read_table` in `settings.rs`). The value is kept out because the table holds a resolved secret when the vendor's validate function runs at startup (`config_payload.rs` lines 93 to 102), and each adapter writes the whole failure to the service's log when the state fails to build (for example line 1503 of the Fastly adapter's `app.rs`) and answers requests with a generic error. The `serde` crate quotes a string it cannot read as the type asked for (`Unexpected::Str`, `serde_core` 1.0.228, `de/mod.rs` line 410), so a read error names the path of the setting at fault, found with `serde_path_to_error`, and keeps `serde`'s own message only when it names keys alone, being a key missing, unknown or written twice. The `validator` crate at the locked version 0.20.0 prints a failed rule's parameters, which include the checked value, when the rule has no message of its own (`display_impl.rs` lines 7 to 10, and for the `length` rule line 38 of `tokens/length.rs` in `validator_derive`), so a validation error is reported through `validation_error_summary`, as core's own settings are, which names the field, the rule and the rule's own message where it has one, and never the rule's parameters. **What refuses, and when.** Push is `ts config validate`, `ts config diff` and `ts config push`, which run the same checks (`docs/guide/configuration-rules.md` lines 119 to 123) through `validate_settings_for_deploy_with` (`config.rs` lines 348 to 371). Push builds no registry. @@ -81,11 +81,11 @@ A builder a table runs has no module name (item 2 above). The function `check_se | `[geo] module` or `[device] module` naming a module that no running registration supplies, followed by the note that names the absent tables | `named_module`, `registry.rs` lines 1096 to 1123 | Startup only | | A `[[fetch]]` or `[[serve]]` entry naming a middleware that no running module supplies, with no note on absent tables | `check_middleware`, `registry.rs` lines 1405 to 1418 | Startup only | | `[robots-txt] modules` naming a module that nothing running supplies, with no note on absent tables | `check_robots_txt_selection`, `registry.rs` lines 1250 to 1283 | Startup only | -| A secret setting in use that names no key | `validate_secret_key_references`, `config.rs` lines 505 to 524 | Push, and never for a secret in a shared table | +| A secret setting in use that names no key | `validate_secret_key_references` in `config.rs` | Push, a shared table's secrets included | A refusal at startup stops the service from building its state, and each adapter then answers requests with a generic error and writes the cause to its log. A push that carries a table no builder carries the mark for is accepted, and the service fails when it next starts. The note on absent tables (`absent_shared_note`, lines 1036 to 1052) names every absent table among the builders that carry the mark, because the registry cannot tell which table would have supplied the name that was written. -**The adapters.** All four adapters pass builders to the settings load and to the registry build, so each resolves the secrets those builders declare and starts the builders a table runs (Axum `app.rs` lines 111 and 184, Fastly lines 237 and 316, Cloudflare lines 191 and 289, Spin lines 140 and 218). The registry build takes the stock list, `trusted_server_modules::builders()`, followed by the deployment's own builders on all four. The settings load takes the stock list alone on Axum, Cloudflare and Spin, and the stock list followed by the builders a deployment registered on Fastly. No builder in the stock list carries the mark at row 26. Only the Axum adapter has a test (section 5). +**The adapters.** All four adapters pass builders to the settings load and to the registry build, so each resolves the secrets those builders declare and starts the builders a table runs (Axum `app.rs` lines 111 and 184, Fastly lines 237 and 316, Cloudflare lines 191 and 289, Spin lines 140 and 218). The registry build takes the stock list, `trusted_server_modules::builders()`, followed by the deployment's own builders on all four. The settings load takes the stock list alone on Axum, Cloudflare and Spin, and the stock list followed by the builders a deployment registered on Fastly. The load declares and resolves only the secrets of the builders it is given, and runs only their validate functions (`settings_from_config_blob_with`, `config_payload.rs`), so on Axum, Cloudflare and Spin a builder a deployment adds that carries the mark gets key names in place of its shared secrets, and its validate function does not run at load. No builder in the stock list carries the mark at row 26. Only the Axum adapter has a test (section 5). **A key in both places.** Core does not refuse a key written both in a module's own table and in the shared table, and which one wins is the vendor crate's choice. No test covers the case. @@ -97,23 +97,25 @@ A deployment with no `[shared]` table behaves as before. No builder in the stock A configuration that carries a table can only be loaded by builds that have this change. At row 25, `Settings` has no `shared` field, so `[shared]` is read as the section of a module type, and `TypeSections::validate` refuses it with `[shared] selects no module` (`settings.rs` lines 305 to 318), at push and at startup. -The change edits `docs/guide/configuration.md` (a row in the table of sections and a "Shared Settings" section), `docs/guide/integration-guide.md` (a row in the table of builder methods and "A vendor's shared table"), the example settings file (a four-line note) and `CHANGELOG.md` (one line added and the line for `robots_txt` rewritten, see open question 5). +The change edits `docs/guide/configuration.md` (a row in the table of sections and a "Shared Settings" section), `docs/guide/integration-guide.md` (a row in the table of builder methods and "A vendor's shared table"), the example settings file (a four-line note) and `CHANGELOG.md` (one line added and the line for `robots_txt` rewritten, see open question 3). ## 4. What is not in this change - **A vendor's shared crate.** The stock build has no vendor with several modules sharing a service at row 26, so the probe is the one reader. The 51Degrees client hints change (row 44) adds the first, `crates/shared/51degrees`, which the builder in `crates/middleware/51degrees` uses, and that builder carries the mark. - **A check of a table's name against a folder.** See "The name". -- **A test of the secret path.** The `/_ts/config` view masks a secret in a table through the same mechanism as a module's own, and no test shows it (open question 6). +- **A test of the secret path.** The `/_ts/config` view masks a secret in a table through the same mechanism as a module's own, and no test shows it (open question 4). ## 5. Acceptance The seam probe gains a second builder, `shared_builder`, which `[shared.seam-probe]` runs and whose route, `/integrations/seam_probe_shared/marker`, answers the table's marker (`crates/testing/seam-probe/src/lib.rs` lines 754 to 866). The Axum adapter's test, `a_vendors_shared_table_runs_its_builder` (`crates/trusted-server-adapter-axum/tests/seam_probe.rs` lines 259 to 299), requests the route with the table and checks the marker in the body, requests the route without the table and checks only that the status is not `200` (line 283), and checks that a table with no builder carrying its mark makes `routes_with_registrations` fail with `[shared.seam-probe] is read by no module`. -**Tests.** The change adds 15 tests. +**Tests.** The change adds 18 tests. -- Six are in the `shared_tables` module of `settings.rs` (lines 11086 to 11213). A vendor reads its own table and no other. A table its vendor cannot read and a table that fails validation are each refused with the table named. A table not named as a vendor (`[shared.Example]` and `[shared."an.example"]`) is refused. The debug print shows the vendors alone while serialization carries the tables whole. Settings with no table are written without one. +- Seven are in the `shared_tables` module of `settings.rs`. A vendor reads its own table and no other. A table its vendor cannot read and a table that fails validation are each refused with the table named. A table that fails a rule and a table that cannot be read, each holding a resolved secret, are refused naming the setting and never the secret's value. A table not named as a vendor (`[shared.Example]` and `[shared."an.example"]`) is refused. The debug print shows the vendors alone while serialization carries the tables whole. Settings with no table are written without one. +- One more in `settings.rs` checks the same of a module's own table read through `module_config`. - Six are in `registry.rs` (lines 4803 to 4929). A builder a table runs is built only when the table is present. A table no builder carries the mark for is refused. A section cannot select a builder a table runs. A geo selection and a selection in a section that selects modules by name are each refused with the absent table named. A builder that carries the mark and a module name is refused. - Two are in `module_secrets.rs` (lines 201 to 251). One checks that a secret in a shared table is kept when the table is present and the secret is in use. The other checks that the secrets are declared at `shared.` followed by their path inside the table. +- One is in `config.rs`. Push refuses a secret setting a shared table uses when it names no key, and accepts one that names a key, one the table does not use, and a builder whose table is absent. - One is the Axum test above. **CI.** Every check passed at `5cbd69d20`, being 19 of 19 on [jwrosewell/trusted-server#21](https://github.com/jwrosewell/trusted-server/pull/21) and 25 of 25 on IABTechLab/trusted-server#1367. The description of IABTechLab/trusted-server#1367 also cites CI on `7926a0ecf`. @@ -124,27 +126,25 @@ The seam probe gains a second builder, `shared_builder`, which `[shared.seam-pro 1. A builder a table runs that supplies a geo, device or Edge Cookie module, a robots.txt contributor or middleware. The probe's shared builder registers one route and nothing else. 2. A shared secret resolved by the loader, cleared when not in use, or masked in `/_ts/config`. The two secret tests call `clear_unused` and `secret_fields` directly. -3. Anything at push for a shared table, including the check that a secret names a key. +3. Anything at push for a shared table other than the check that a secret names a key. 4. The Fastly, Cloudflare and Spin adapters. 5. A key written in both a module's own table and the shared table. -6. The error text of a failed typed read of a table that holds a secret. -7. A table that only modules with a module name read. +6. A table that only modules with a module name read. ## 6. Open questions These are for the reviewer. This revision records what the code does and decides none of the questions. -1. Should the push check that a secret in use names a key reach the secrets that a builder a table runs declares? Today the check never reaches them (see "Secrets"), and `docs/guide/configuration-rules.md` line 134 says every secret setting is checked. -2. Should a failed typed read or validation, of a shared table and of a module's own table (`module_config`), leave the failing value out of the message, as core's own settings do through `validation_error_summary`? No test makes a secret fail a rule to show what the message holds. -3. Should a refusal of a name in a section that selects modules by name still end with the note about absent tables? Adding a table cannot make such a name resolve, because a builder a table runs has no module name, and the test at `registry.rs` line 4896 asserts the note. -4. Should `with_shared_settings` refuse a vendor name that `validate_names` refuses? Today the builder is accepted, no table can ever start the builder, and nothing says so. -5. Should the line for `robots_txt` in `CHANGELOG.md` go back to the wording row 25 gives it (line 88 on row 25)? Row 26 rewrites the line (line 89 on row 26) to say a `[[fetch]]` or `[[serve]]` entry names the middleware, while `require_robots_txt_placed` (`settings.rs` lines 4521 to 4566) refuses settings that have no `[[fetch]]` entry on `/robots.txt` naming the middleware. -6. Should a test resolve a shared secret through `settings_from_config_blob_with`, check that an unused one is cleared, and check that `/_ts/config` masks the secret? Only the pieces are tested today. -7. Should push validation refuse a table that no builder carries the mark for? The registry check cannot run at push, because push builds no registry and the CLI links only the builders registered for deploy validation, so a push accepts the table and the service then fails to start. +1. Should a refusal of a name in a section that selects modules by name still end with the note about absent tables? Adding a table cannot make such a name resolve, because a builder a table runs has no module name, and the test at `registry.rs` line 4896 asserts the note. +2. Should `with_shared_settings` refuse a vendor name that `validate_names` refuses? Today the builder is accepted, no table can ever start the builder, and nothing says so. +3. Should the line for `robots_txt` in `CHANGELOG.md` go back to the wording row 25 gives it (line 88 on row 25)? Row 26 rewrites the line (line 89 on row 26) to say a `[[fetch]]` or `[[serve]]` entry names the middleware, while `require_robots_txt_placed` (`settings.rs` lines 4521 to 4566) refuses settings that have no `[[fetch]]` entry on `/robots.txt` naming the middleware. +4. Should a test resolve a shared secret through `settings_from_config_blob_with`, check that an unused one is cleared, and check that `/_ts/config` masks the secret? Only the pieces are tested today. +5. Should push validation refuse a table that no builder carries the mark for? The registry check cannot run at push, because push builds no registry and the CLI links only the builders registered for deploy validation, so a push accepts the table and the service then fails to start. ## Review log 1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/26-shared-tables` at `5cbd69d20`. Issues found, 2 of them blocking. The two were that the spec called a closed fork pull request ready for review, named a closed pull request as the one holding the design documents, said a table is named by a folder the code never checks and cited an acceptance record the pull request description does not hold, and that the spec never said what a builder run by a table can supply. This revision states what the code does at that head for the table's name, the builder a table runs and what the builder can supply, what keeps running without the table, the secrets, the error text, each refusal and when it runs, the adapters, compatibility and the tests, and lists seven questions that wait for code changes, being key names at push, values in error text, the note on refusals, a mark with a name no table can have, the changelog line for `robots_txt`, a test of the secret path and a table nothing reads at push. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/26-shared-tables` at `5cbd69d20`. Approved, none blocking. This revision confirms the 25 passing checks on IABTechLab/trusted-server#1367, says that the validator quotes a checked value only for a rule without a message of its own, and says that on Axum, Cloudflare and Spin a builder a deployment adds that carries the mark gets key names in place of its shared secrets. ## Revision record @@ -152,3 +152,5 @@ These are for the reviewer. This revision records what the code does and decides | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The status line and the pull requests it names, the rule for a table's name, the builder a table starts and what keeps running without the table, what a builder a table runs can supply, the secrets at load and at push, the error text, each refusal and when it runs, the adapters, permissions, compatibility, the acceptance rebuilt from the pull request description, the tests, the open questions and the review log. | +| 2026-10-16 | Fixes from the code review. A shared table or a module's own table that is refused names the setting at fault and never its value, and push checks that each secret setting a shared table uses names a key, each with a test, which closes the first two open questions. | +| 2026-10-16 | Points from review round 2. When the validator quotes a checked value, and what a builder a deployment adds that carries the mark gets on Axum, Cloudflare and Spin. | From fd092db86d2ca586b9c07c136ec4ad40e5ab5b83 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 09:37:28 +0100 Subject: [PATCH 103/192] Correct the acceptance and three clauses in the deployment headers spec The acceptance now says what the pull request description records, three partly true clauses are qualified, and a new open question covers headers core or an adapter writes directly. --- .../2026-10-12-diagnostic-headers-design.md | 21 +++++++++++-------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md b/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md index aff6f4bebc..f156867386 100644 --- a/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md +++ b/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md`, `2026-10-12-inspection-endpoints-design.md`, `2026-10-12-attestation-design.md` and `2026-10-13-middleware-settings-design.md` (row 38, built after this row), in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342), which holds every design document of the stack, and `2026-09-25-git-version-header-design.md`, which main already holds **Related PRs:** [IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356), [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355) (row 14, robots.txt from modules, which this row stacks on), [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10, page changes as middleware), [IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350) (row 9, inspection endpoints), [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353) (row 12, attestation), [IABTechLab/trusted-server#1379](https://github.com/IABTechLab/trusted-server/pull/1379) (row 38, middleware settings), [IABTechLab/trusted-server#1213](https://github.com/IABTechLab/trusted-server/pull/1213) (the git version header, merged to main), [jwrosewell/trusted-server#10](https://github.com/jwrosewell/trusted-server/pull/10) (closed), [IABTechLab/trusted-server#1302](https://github.com/IABTechLab/trusted-server/pull/1302) (closed) **Issues:** [IABTechLab/trusted-server#1273](https://github.com/IABTechLab/trusted-server/issues/1273) -**Last updated:** 2026-10-15 +**Last updated:** 2026-10-16 > **Why this spec exists.** The server passed a reader of any page three headers from the publisher's origin and wrote version headers of its own, so anyone could learn which node answered, whether a cache did and which release ran, without anyone having decided to tell them. This spec says which headers now stay off a response, where the removal runs, what a deployment writes to put the versions back on its pages, and what the change leaves as it was. @@ -46,8 +46,8 @@ The change was first called diagnostic headers, and the name of this file and of **Two places run the removal.** Both call into core's `http_util`, where the three names are the one list `DEPLOYMENT_HEADERS`. -1. The publisher fallback, which answers every request no other route takes, calls `remove_deployment_headers` on whatever the origin answers, on every path. A request for an unrouted `/_ts/anything` goes to the origin, and the origin's answer does not bring the three headers back. A page served from the store is built without the origin's headers, so the fallback has nothing to remove from that page. -2. The finalize middleware of each adapter calls `strip_deployment_headers` with the request's path and `ROUTE_NAMESPACE`, on the response of every route the router serves. A response that this server fetched from a third party for a proxied script, image or creative loses the three headers on any path outside `/_ts` as well, because the proxy passes an upstream's headers on, apart from the CORS headers that let other sites read the response. Responses that skip the finalize middleware are described under "Not in this change". +1. The publisher fallback, which answers every request no other route takes, calls `remove_deployment_headers` on whatever the origin answers, on every path. A request for an unrouted `/_ts/anything` goes to the origin, and the origin's answer does not bring the three headers back. A page served from the store is built from the stored copy and replays a fixed list of the origin's policy headers, such as `Content-Security-Policy` and `X-Frame-Options`. The list is `REPLAYABLE_POLICY_HEADERS` in `crates/trusted-server-core/src/platform/template_cache.rs` and names none of the three, so the fallback has nothing to remove from that page. +2. The finalize middleware of each adapter calls `strip_deployment_headers` with the request's path and `ROUTE_NAMESPACE`, on the response of every route the router serves. A response that this server fetched from a third party for a proxied script, image or creative loses the three headers on any path outside `/_ts` as well, because the proxy passes an upstream's headers on apart from a short list. The proxy that fetches scripts, images and creatives takes off the CORS headers that let other sites read the response, in `strip_cors_policy`, and the asset proxy, which serves a configured path from an asset origin, takes off `Set-Cookie`, `Strict-Transport-Security` and `Clear-Site-Data`, in `strip_asset_proxy_response_headers`. Both functions are in `crates/trusted-server-core/src/proxy.rs` and neither list holds any of the three. Responses that skip the finalize middleware are described under "Not in this change". ```rust pub const ROUTE_NAMESPACE: &str = "/_ts"; @@ -159,10 +159,11 @@ pub struct MiddlewareContext<'a> { 8. Should one test run a real router with the finalize middleware and the publisher fallback together, with a proxied third-party response among the responses? Today the adapter tests call the finalize middleware with a canned response, and the one fallback test calls the fallback directly. 9. Should `2026-09-25-git-version-header-design.md`, which still reads `Implemented`, and its plan carry a note that this change replaces what they describe? 10. Should the changelog's Security entry for the three headers carry **Breaking:**, as the entry for the version headers under Changed does, given that the pull request description lists the removal under what breaks? +11. Under the stack's rule that only middleware changes a response, a separate audit of the stack is listing the headers that core or an adapter still writes directly, such as `X-TS-ENV`, `X-Geo-Info-Available` and the six `x-ts-sandbox-*` counters, so should each of those headers move into a serve middleware? ## 4. Tests -- Core's `http_util` has five tests of the rule. A publisher's page loses all three headers. `/_ts`, `/_ts/permissions` and `/_ts/debug/ja4` keep them. Headers outside the list (`x-ts-version`, `x-geo-info-available` and `cache-control`) are left alone. `/_tsx`, `/_ts-archive` and `/_tsfoo/bar` lose the headers as pages. The boundary is the namespace passed in. +- Core's `http_util` has five tests of the rule. A publisher's page loses all three headers. `/_ts`, `/_ts/permissions` and `/_ts/debug/ja4` keep them. Headers outside the list (`x-ts-version`, `x-geo-info-available` and `cache-control`) are left alone. `/_tsx`, `/_ts-archive` and `/_tsfoo/bar` lose `x-served-by` as pages, the only one of the three headers that the test of these paths sends. The boundary is the namespace passed in. - One publisher test sends a `404` that carries the three headers for `/_ts/anything` and checks that none reaches the reader. - Each adapter has one test of its finalize middleware. A page path loses `x-served-by` and `x-cache-hits`, and an operator's `X-Cache` from `[response_headers]` is sent in place of the origin's. - `service_version` has seven tests of its own. The git version is written alone, the platform's version is written alone, both are written in that order, neither writes nothing, a version a header cannot carry is left out, the default carries the compiled-in git version, and the middleware runs in the serve phase under its name. @@ -174,15 +175,17 @@ The paths these tests leave out are listed as questions under Open questions. ## 5. Acceptance -The fork's CI passed on the series head, `a43a05a33`, all nineteen checks, and the description of IABTechLab/trusted-server#1356 records that. The descriptions of that pull request, of IABTechLab/trusted-server#1302 and of jwrosewell/trusted-server#10 hold no record of a lint and a test of each commit on its own, of a run of the whole check on the head, or of deliberate faults made one at a time, so this spec claims none of them. The description of #1356 also says that the responses the Fastly entry point sends outside the middleware are not exercised, and those are listed under "Not in this change". +The fork's CI passed on the series head, `a43a05a33`, all nineteen checks, and the description of IABTechLab/trusted-server#1356 records that, with a link to jwrosewell/trusted-server#10. The same description has every check command of its test plan ticked, being the Fastly and Axum tests and lints, the JavaScript tests, the WebAssembly build of the Fastly adapter and the format commands for the Rust code, the JavaScript and the documentation. A last line for the full workflow set of the repository, run on jwrosewell/trusted-server#10, is ticked as well, and manual testing with `fastly compute serve` is not. The description of IABTechLab/trusted-server#1356 does not say whether anyone also ran the commands by hand on the head. The descriptions of IABTechLab/trusted-server#1356, of IABTechLab/trusted-server#1302 and of jwrosewell/trusted-server#10 hold no record of a lint and a test of each commit on its own or of deliberate faults made one at a time, so this spec claims neither. The description of IABTechLab/trusted-server#1356 also says that the responses the Fastly entry point sends outside the middleware are not exercised, and those are listed under "Not in this change". ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/15-diagnostic-headers` at `a43a05a33`. Issues found, four of them blocking. The four were that the spec covered the three commits of 9 October and none of the five since, that the spec left `X-TS-Version` as it was and left out the breaking change the pull request lists for it, that the rule for paths beneath `/_ts` was wider than the code for an origin's answer while the inspection pages read as if they showed headers nothing writes, and that the acceptance cited a record the pull request descriptions do not hold. This revision states what the code does at that head for all eight commits, the paths beneath `/_ts`, the adapters, the version headers and the `service_version` middleware, the defaults, the rollout, what breaks, the failure and privacy answers, the tests and the acceptance, and lists under open questions what waits for code changes, being the exemption, the list of names, the headers still sent, an origin's version headers, the untested paths and the documents. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/15-diagnostic-headers` at `a43a05a33`. Issues found, none blocking. This revision corrects the acceptance, which said the pull request descriptions hold no record of a run of the whole check on the head although the description of the pull request for this row (IABTechLab/trusted-server#1356) has every check command ticked, and it qualifies three clauses that were only partly true, being what a stored page replays from the origin, what the proxies take off a response and which header the prefix test sends. ## Revision record -| Date | Change | -| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | -| 2026-10-15 | Fixes from review round 1. The five commits since 9 October, being the fallback removal, the response headers a serve middleware may write, the platform in the middleware context, the `service_version` middleware and the end of the version headers on every adapter, the function names, the paths beneath `/_ts` and what the exemption keeps, the adapters and the responses that skip the finalize step, the defaults and the rollout, what breaks, the failure and privacy answers, the tests, the open questions, the header and the acceptance lines brought to the series head, and the review log. | +| Date | Change | +| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-15 | Fixes from review round 1. The five commits since 9 October, being the fallback removal, the response headers a serve middleware may write, the platform in the middleware context, the `service_version` middleware and the end of the version headers on every adapter, the function names, the paths beneath `/_ts` and what the exemption keeps, the adapters and the responses that skip the finalize step, the defaults and the rollout, what breaks, the failure and privacy answers, the tests, the open questions, the header and the acceptance lines brought to the series head, and the review log. | +| 2026-10-16 | Fixes from review round 2. The acceptance says what the description of the pull request for this row records, being the fork's run of all nineteen checks on the head, the ticked check commands and the manual test left unticked, in place of saying that no description records a run of the whole check. Three clauses that were only partly true are qualified, being the headers a stored page replays from the origin, the headers each proxy takes off and the one header the prefix test sends. A new open question asks whether the headers that core or an adapter writes directly should move into a serve middleware. | From 3b9cf065c7edebddc0e9cfc1a6bfdc2e2271ee0b Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 09:42:02 +0100 Subject: [PATCH 104/192] The config by host design takes its round 2 review The pointer now names sections 2.4, 2.6 and 2.8. --- .../specs/2026-10-12-config-by-host-design.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-config-by-host-design.md b/docs/superpowers/specs/2026-10-12-config-by-host-design.md index 1e954b3e96..de09b4f018 100644 --- a/docs/superpowers/specs/2026-10-12-config-by-host-design.md +++ b/docs/superpowers/specs/2026-10-12-config-by-host-design.md @@ -34,7 +34,11 @@ fastly config-store-entry update --upsert \ --value '{host}' ``` -The selector entry sits in the `edgezero_runtime_env` store, which this row reads through EdgeZero's `runtime_env_config`. IAB Tech Lab's main has moved EdgeZero to a version without that store, so `2026-10-15-fastly-per-host-selector-design.md` replaces where this row reads the selector from with an entry in the application's own config store. The placeholder and the answers below stay as they are, apart from the changes in sections 2.2 to 2.4 of that design, which section 5 (Open questions) describes. +The selector entry sits in the `edgezero_runtime_env` store, which this row reads through EdgeZero's `runtime_env_config`. IAB Tech Lab's main has moved EdgeZero to `v0.0.9`, a version without that store, so `2026-10-15-fastly-per-host-selector-design.md` (the selector design) replaces where this row reads the selector from with an entry in the application's own config store (section 2.1 of the selector design). The placeholder and the answers below stay as they are, apart from three groups of changes in the selector design. + +1. Sections 2.2 to 2.4 change how a `Host` that is not UTF-8 text, a key that is too long and a selector that misspells the placeholder are answered, and which key `Hooks::routes()` reads. Open questions 1, 2, 6 and 8 in section 5 describe the four changes. +2. Section 2.4 also answers `500` for a blank selector and for a store that cannot be opened, which replaces the fallback to the logical store ID that this spec describes under "Why the placeholder goes in the selector that already exists" and in the risk table. A selector entry that is absent still reads the logical store ID. +3. Sections 2.6 and 2.8 replace the push and selector-writing steps, being the two `ts config push` commands and the `fastly config-store-entry update` command above and the steps for moving a service below. EdgeZero `v0.0.9` refuses any Fastly config key other than the logical store ID, so section 2.6 changes how a host's blob is pushed, and section 2.8 sets the order of the steps, with the selector written to its new entry (section 2.1). The entry point puts the request's host in the placeholder's place before anything reads settings, so the debug probes, the application build and the finalizers all read the blob of the publisher the request is for (`lib.rs:432`, before the reads at `:446`, `:467`, `:507` and `:646`). @@ -75,7 +79,7 @@ A switch of its own beside the sandbox bounds is the obvious design. EdgeZero's At the EdgeZero version this stack pins, a staged deploy copies every runtime entry of the service into a store of its own and replaces only the config key selectors, pointing each at `_staging` ([`staging_entries_from_production`](https://github.com/stackpop/edgezero/blob/683202c66948146ec360f490126f1d60f920288b/crates/edgezero-adapter-fastly/src/cli.rs#L3579-L3603)). A separate switch would be copied with the rest, and a staged version would then choose by host and read the production blobs. The selector is replaced, so a staged version reads the one staging blob for every host and never a production one. `2026-10-15-fastly-per-host-selector-design.md` (section 2.5) says how staging works under the EdgeZero version Tech Lab main uses. -The selector is also read by [`runtime_env_config`](https://github.com/stackpop/edgezero/blob/683202c66948146ec360f490126f1d60f920288b/crates/edgezero-adapter-fastly/src/lib.rs#L259-L282), and that function does not always fail when the selector cannot be read. A lookup that fails panics, so the request fails. Where the `edgezero_runtime_env` store cannot be opened, the function logs a warning and returns no entries, and an entry that is blank or carries a control character is read as absent ([`EnvConfig::store_key`](https://github.com/stackpop/edgezero/blob/683202c66948146ec360f490126f1d60f920288b/crates/edgezero-core/src/env_config.rs#L113-L122)). In both cases the key becomes the logical store ID (`app.rs:189`), so a blob left at that key answers every host. A service that chooses by host therefore holds no blob at the logical store ID, and the steps for moving a service (below) remove any blob there. +The selector is also read by [`runtime_env_config`](https://github.com/stackpop/edgezero/blob/683202c66948146ec360f490126f1d60f920288b/crates/edgezero-adapter-fastly/src/lib.rs#L259-L282), and that function does not always fail when the selector cannot be read. A lookup that fails panics, so the request fails. Where the `edgezero_runtime_env` store cannot be opened, the function logs a warning and returns no entries, and an entry that is blank or carries a control character is read as absent ([`EnvConfig::store_key`](https://github.com/stackpop/edgezero/blob/683202c66948146ec360f490126f1d60f920288b/crates/edgezero-core/src/env_config.rs#L113-L122)). In both cases the key becomes the logical store ID (`app.rs:189`), so a blob left at that key answers every host. Section 2.4 of the selector design replaces both cases with a `500` answer, for a blank selector and for a store that cannot be opened, and an entry that is absent still reads the logical store ID. A service that chooses by host therefore holds no blob at the logical store ID, and the steps for moving a service (below) remove any blob there. ### What keeps one publisher's settings from answering for another @@ -89,6 +93,8 @@ The selector is also read by [`runtime_env_config`](https://github.com/stackpop/ | A staged version reads production blobs | At the EdgeZero version this stack pins, a staged deploy replaces the selector, as above | | The store cannot say whether a host has a blob | The request is answered `500` and not served | +The first table row describes the code at `b8314cb52`. Under the selector design (section 2.4), a blank selector, a store that cannot be opened and a misspelled placeholder are each answered `500`, so a service that chooses by host reads the blob at the logical store ID only where the selector entry is absent. + ### The reusable sandbox `Sandbox` was `lifecycle::Sandbox`, one application for the life of the sandbox. It is `lifecycle::Sandbox`, which keeps each application under its blob's key. @@ -111,6 +117,8 @@ The service has one of each of the following, and nothing divides any of them be ### Moving a service onto a blob for each host +Sections 2.6 and 2.8 of the selector design replace steps 1 and 2 below. Section 2.6 changes how a blob is pushed, because EdgeZero `v0.0.9` refuses any Fastly config key other than the logical store ID, and section 2.8 sets the order of the steps and the entry that holds the selector. Step 3 still applies, and step 2 of section 2.8 gives the same advice. + 1. Push a blob for each host the service answers for, in lower case. 2. Write the selector. The entry point reads the selector on every request, so the change takes effect on the next request without a new version, which is why the blobs come first. 3. Remove the blob at the logical store ID, and push nothing there afterwards, because a `ts config push` without `--key` writes there. A blob left there answers every host if the selector ever reads as absent. @@ -168,6 +176,7 @@ The behaviours below are stated in the sections above. A change to the code woul ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/11-config-by-host` at `b8314cb52`. Issues found, 3 of them blocking. The header pointed at closed pull requests and called the fork pull request ready for review, the risk table put three header cells over two-cell rows, and the text said there is no fallback and that a selector that cannot be read fails the request, which is false when the runtime-env store cannot be opened. This revision points the header at the open draft pull request, repairs the table, states the fallback and the removal it needs, and adds the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests, whilst the code behaviours the review found wait for code changes and sit under Open questions. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/11-config-by-host` at `b8314cb52`. Approved, none blocking. This revision names, at the pointer and where this spec describes them, the sections of `2026-10-15-fastly-per-host-selector-design.md` that replace the fallback text (section 2.4) and the push and selector-writing steps (sections 2.6 and 2.8). ## Revision record @@ -175,3 +184,4 @@ The behaviours below are stated in the sections above. A change to the code woul | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Round 1 fixes. The header points at the open draft pull request, the risk table is repaired, the fallback to the logical store ID is stated, the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests are added, and the code findings sit under Open questions. The selector store points at the Fastly per-host selector design. | +| 2026-10-16 | Round 2 fixes. The pointer to the Fastly per-host selector design names section 2.4, whose `500` answers replace the fallback to the logical store ID, and sections 2.6 and 2.8, which replace the push and selector-writing steps. The same sections are named where this spec describes the fallback and the steps for moving a service. | From d506ff0682bdea0c924375687285fe42ca3d9887 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 09:42:08 +0100 Subject: [PATCH 105/192] The ads.txt sellers design takes its round 2 review It states the earlier pull requests closed unmerged and tightens two claims. --- .../2026-10-12-ads-txt-sellers-design.md | 39 ++++++++++--------- 1 file changed, 21 insertions(+), 18 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md b/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md index e6c874c354..60880eb81f 100644 --- a/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md +++ b/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md @@ -1,11 +1,11 @@ # Design Spec: The sellers a deployment adds to the publisher's ads.txt -**Status:** Proposed, 2026-10-12. Raised as draft pull request IABTechLab/trusted-server#1361, row 20 of the stack in IABTechLab/trusted-server#777, answering IABTechLab/trusted-server#1278, for Tech Lab review. The two earlier pull requests for this change, IABTechLab/trusted-server#1307 and jwrosewell/trusted-server#15, are closed. +**Status:** Proposed, 2026-10-12. Raised as draft pull request IABTechLab/trusted-server#1361, row 20 of the stack in IABTechLab/trusted-server#777, answering IABTechLab/trusted-server#1278, for Tech Lab review. The two earlier pull requests for this change, IABTechLab/trusted-server#1307 and jwrosewell/trusted-server#15, were closed on 2026-10-15 without being merged. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md`, `2026-10-12-middleware-media-types-design.md`, `2026-10-12-middleware-creates-a-missing-file-design.md` and `2026-10-14-no-modules-crate-design.md`, all in IABTechLab/trusted-server#1342, which holds every design document of the stack -**Related PRs:** IABTechLab/trusted-server#1361, IABTechLab/trusted-server#1307 (closed), jwrosewell/trusted-server#15 (closed), IABTechLab/trusted-server#1360 (row 19, middleware on a media type other than HTML), IABTechLab/trusted-server#1363 (row 22, a file the origin does not have), IABTechLab/trusted-server#1384 (row 43, which renames the project's own modules `iabtl`) and IABTechLab/trusted-server#1348 (row 7, the integration seam, which replaces the closed IABTechLab/trusted-server#1094) +**Related PRs:** IABTechLab/trusted-server#1361, IABTechLab/trusted-server#1307 (closed without being merged), jwrosewell/trusted-server#15 (closed without being merged), IABTechLab/trusted-server#1360 (row 19, middleware on a media type other than HTML), IABTechLab/trusted-server#1363 (row 22, a file the origin does not have), IABTechLab/trusted-server#1384 (row 43, which renames the project's own modules `iabtl`) and IABTechLab/trusted-server#1348 (row 7, the integration seam, which replaces the closed IABTechLab/trusted-server#1094) **Issues:** IABTechLab/trusted-server#1278 -**Last updated:** 2026-10-15 +**Last updated:** 2026-10-16 > **Why this spec exists.** A publisher who sells through this server has sellers to declare that the publisher's own `ads.txt` does not list. This spec says what the module that adds them writes into the file, which settings the module accepts and what the module does with each answer the origin gives. @@ -81,7 +81,7 @@ The module is a middleware for `text/plain`, the first to use the entries for a 4. Should the grammar follow the IAB ads.txt 1.1 specification more closely? Section 3.4.2 takes field 1 to be a domain name and says no field holds white space, yet `https://ssp.example` is accepted and the account id and the certification authority id may hold spaces. A variable's value is only checked for being present, while section 3.5.1 gives `OWNERDOMAIN` as a bare domain of which only the first counts, and `MANAGERDOMAIN` as one per country. 5. Should the module log when it writes nothing because the response declares a charset other than UTF-8? `middleware_stream_processors` in `publisher.rs` does not run the module for such a response and writes no log line saying so. 6. Should the origin's `ETag`, `Last-Modified` and range handling stay on a file the module changed? Core removes a reader's conditional and range headers only for an HTML document that it expects to change, so a request for `/ads.txt` normally reaches the origin with them. By reading the code, a crawler that holds the old file and asks again gets the origin's `304` after `lines` change and keeps the old file, and open questions 2 and 3 of the media types spec ask the same of every entry. -7. Should the two paths the tests miss get tests? A line with a line break inside it is refused by `check_line`, and no test sends one. Core ends every body with a call that carries no bytes and is marked as the last. The host-name replacement in front of the module holds back the last few bytes of each piece and hands them on in that final call, so the module's final call carries bytes or carries none, while the crate's tests end every non-empty file on a call that carries bytes. A fault that learns whether the file ended in a line break from the last call alone therefore passes every test. +7. Should the two paths the tests miss get tests? A line with a line break inside it is refused by `check_line`, and no test sends one. Core ends every body with a call that carries no bytes and is marked as the last. By reading `StreamingReplacer::process_chunk` in `streaming_replacer.rs`, the host-name replacement in front of the module holds back the end of what it has seen, in case a host name is split between two pieces, and hands those bytes on in that final call. The module's final call therefore carries bytes for every non-empty file that is valid UTF-8, and carries none after calls that carried bytes only where the replacement emits nothing at the end, which is where the bytes it holds back begin with bytes that are not valid UTF-8 (open question 3). The crate's tests end every non-empty file on a call that carries bytes, and give a call with no bytes only to an empty file. No test ends a body on a call with no bytes after calls that carried some, so a fault that learns whether the file ended in a line break from the last call alone, and counts an empty last call as ending in one, passes every test. ## 4. Acceptance @@ -101,19 +101,20 @@ The description of IABTechLab/trusted-server#1361 records CI green on all 19 che The tests are in `crates/ads-txt/sellers/src/lib.rs` and `crates/trusted-server-modules/src/lib.rs`. -| Behavior | Test | -| -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| The module's name is its crate folder's name, and a document that selects nothing registers nothing | `module_constant_is_the_crate_folder` and `an_unselected_module_does_not_register` | -| The registry gives `/ads.txt` the module's middleware and `/robots.txt` none, and the registration carries no script for the browser | `the_lines_are_offered_as_the_modules_middleware_on_text` | -| A document that selects the module and names it in no entry is refused by `validate` and by `register` | `a_document_naming_the_module_in_no_entry_is_refused` | -| Each refusal of a line in section 4 except a line break inside a line, with the line's number and the reason, from `validate` and from `register`, and an empty list | `a_line_that_is_not_a_record_or_a_variable_is_refused_for_its_reason` and `no_line_at_all_is_refused` | -| Records and variables the grammar allows are accepted | `records_and_variables_are_accepted_as_the_grammar_has_them` | -| The lines follow the file on a line of their own, with and without a final line break, and an empty file becomes the lines | `the_lines_follow_the_publishers_file_on_a_line_of_their_own` | -| A file ending in a carriage return and a line feed takes no second break | `a_file_whose_lines_end_with_a_carriage_return_takes_the_lines_after_its_last_one` | -| A file read 1, 3, 7 or 20 bytes at a time gives the same result, and the lines are written trimmed and in the case given | `a_file_read_in_pieces_is_the_same_file` and `the_lines_are_written_as_given_and_trimmed` | -| The middleware handles `text/plain` and not `text/html`, and runs in the fetch phase alone | `the_middleware_works_on_text_alone_in_the_fetch_phase` | -| The stock list names the middleware with its phase and media type, and the example settings and the guide's table agree with the list | `the_listed_middleware_are_the_ones_the_modules_register`, `the_template_s_entries_name_the_listed_middleware` and `the_guide_s_table_lists_the_same_middleware` in the modules crate | -| An unknown setting in `[ads-txt.sellers]` is refused, naming the table and the key | `every_stock_module_rejects_a_setting_it_does_not_know` in the modules crate | +| Behavior | Test | +| ------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| The module's name is its crate folder's name, and a document that selects nothing registers nothing | `module_constant_is_the_crate_folder` and `an_unselected_module_does_not_register` | +| The registry gives `/ads.txt` the module's middleware and `/robots.txt` none, and the registration carries no script for the browser | `the_lines_are_offered_as_the_modules_middleware_on_text` | +| A document that selects the module and names it in no entry is refused by `validate` and by `register` | `a_document_naming_the_module_in_no_entry_is_refused` | +| Each refusal of a line in section 4 except a line break inside a line, with the line's number and the reason, from `validate` and from `register` | `a_line_that_is_not_a_record_or_a_variable_is_refused_for_its_reason` | +| An empty list is refused with the reason that the module would add nothing, from `validate` alone, because the test does not call `register` | `no_line_at_all_is_refused` | +| Records and variables the grammar allows are accepted | `records_and_variables_are_accepted_as_the_grammar_has_them` | +| The lines follow the file on a line of their own, with and without a final line break, and an empty file becomes the lines | `the_lines_follow_the_publishers_file_on_a_line_of_their_own` | +| A file ending in a carriage return and a line feed takes no second break | `a_file_whose_lines_end_with_a_carriage_return_takes_the_lines_after_its_last_one` | +| A file read 1, 3, 7 or 20 bytes at a time gives the same result, and the lines are written trimmed and in the case given | `a_file_read_in_pieces_is_the_same_file` and `the_lines_are_written_as_given_and_trimmed` | +| The middleware handles `text/plain` and not `text/html`, and runs in the fetch phase alone | `the_middleware_works_on_text_alone_in_the_fetch_phase` | +| The stock list names the middleware with its phase and media type, and the example settings and the guide's table agree with the list | `the_listed_middleware_are_the_ones_the_modules_register`, `the_template_s_entries_name_the_listed_middleware` and `the_guide_s_table_lists_the_same_middleware` in the modules crate | +| An unknown setting in `[ads-txt.sellers]` is refused, naming the table and the key | `every_stock_module_rejects_a_setting_it_does_not_know` in the modules crate | Six things have no test. @@ -121,12 +122,13 @@ Six things have no test. 2. An entry with no `path` and an entry on another path (open question 1). 3. A file with a byte that is not valid UTF-8 (open question 3). 4. A line with a line break inside it, and a body whose last call carries no bytes after calls that carried some (open question 7). -5. A selected module with no `lines`. An empty list has a test and a table with no `lines` has none. +5. A selected module with no `lines`. An empty list has a test, which calls `validate` and never `register`, and a table with no `lines` has none. 6. `middleware_for` leaving the module out of the page entry `ts audit` writes. The test of `middleware_for` never hands the function this module's id. ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/20-ads-txt` at `de85e19c0`. Issues found, three of them blocking. The three were that the spec said the origin's file is read as it is and that an origin with no file gets the origin's answer, when core replaces the origin's host name in the file and the module writes its lines into an answer with a `text/plain` body of any status, that the spec named closed pull requests and a bare number that belongs to an unrelated pull request, and that the acceptance table left out variable names in either case and a domain with a space while the closing paragraph cited records neither pull request description holds. This revision states what the code does at that head for the origin's answers, the entry's path, the configuration, a reader's data, the adapters, failure and rollout, replaces the closing claims with what the pull request description records, names the rows that add the made file and the `iabtl` names, and lists seven questions that wait for code changes. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/20-ads-txt` at `de85e19c0`. Approved, none blocking. This revision says that the two earlier pull requests were closed without being merged, says definitely when the module's final call carries no bytes, and says the empty list test calls `validate` alone. ## Revision record @@ -134,3 +136,4 @@ Six things have no test. | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The header and the related specs, the problem statement, what the module writes, what core does to the file first, each kind of origin answer, a missing file, the entry's path, the configuration, consent and privacy, the adapters, failure, rollout and how soon a crawler sees a change, the rows that hold what this change leaves out, the open questions, the acceptance table and claims, the tests and the cases with none, and the review log. | +| 2026-10-16 | Fixes from review round 2. The two earlier pull requests are named as closed without being merged, open question 7 says when the module's final call carries no bytes and why no test reaches that call, and the tests table says the empty list test calls `validate` alone. | From 95e53c94a841707c21d07aa2ded302a8981dfd2c Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 09:42:13 +0100 Subject: [PATCH 106/192] The missing file design takes its round 2 review The acceptance gives each check count with its source. --- ...2026-10-12-middleware-creates-a-missing-file-design.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md b/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md index f991a06c99..53dd9b2848 100644 --- a/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md +++ b/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md`, `2026-10-12-middleware-media-types-design.md`, `2026-10-12-ads-txt-sellers-design.md` and `2026-10-12-robots-txt-middleware-design.md` (row 25, built after this row), all in IABTechLab/trusted-server#1342 **Related PRs:** IABTechLab/trusted-server#1363 builds on the consent prompt phases in [IABTechLab/trusted-server#1362](https://github.com/IABTechLab/trusted-server/pull/1362) (row 21). The `[[fetch]]` and `[[serve]]` entries come from [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10, page changes as middleware), the entries for a media type other than HTML from [IABTechLab/trusted-server#1360](https://github.com/IABTechLab/trusted-server/pull/1360) (row 19) and the sellers module from [IABTechLab/trusted-server#1361](https://github.com/IABTechLab/trusted-server/pull/1361) (row 20). [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366) (row 25) makes robots.txt a middleware and comes after [IABTechLab/trusted-server#1364](https://github.com/IABTechLab/trusted-server/pull/1364) (row 23) and [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24) **Issues:** [IABTechLab/trusted-server#1280](https://github.com/IABTechLab/trusted-server/issues/1280). [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-15 +**Last updated:** 2026-10-16 > **Why this spec exists.** A publisher whose origin has no `ads.txt` gets the origin's `404`, however many sellers the settings give, because a middleware named for the path runs on the origin's file and the origin has none. This change makes the file from the middleware instead. The sections below give when core makes the file, which entry makes the file, what the reader gets and what the change leaves as it was. @@ -154,9 +154,9 @@ Ten things have no test. ## 5. Acceptance -**CI.** Every check passed at `7723c5e56`, being 19 of 19 on jwrosewell/trusted-server#17 and 25 of 25 on IABTechLab/trusted-server#1363. The description of IABTechLab/trusted-server#1363 gives `7723c5e56` for the result in the test plan and `82c1a196e`, the last commit of the series, which four merges of row 21 follow, under "How it was verified". +**CI.** Every check passed at `7723c5e56`. IABTechLab/trusted-server#1363 has 25 check runs at that commit, all successful, and jwrosewell/trusted-server#17, now closed, has 19, all successful. The test plan in the description of IABTechLab/trusted-server#1363 gives the 19 of 19 as the fork's CI on `7723c5e56` and does not give the 25. Under "How it was verified" the description says CI passed on `82c1a196e` on the fork, which is the last commit of the series, and four merges of row 21 follow that commit. -**Local checks.** The description of IABTechLab/trusted-server#1363 ticks `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests, the JavaScript format check, the docs format check, the WebAssembly build of the Fastly adapter and the full workflow set run on the fork. The description leaves the manual test through `fastly compute serve` unticked. The description says that no file has been made through an adapter against a real origin, and that the rule is checked through the publisher path with a stub origin answering `404`. +**Local checks.** The description of IABTechLab/trusted-server#1363 ticks `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests, the JavaScript format check, the docs format check, the WebAssembly build of the Fastly adapter and "the full workflow set of this repository, run on the pull request named above". The line above the ticks names the fork's pull request, jwrosewell/trusted-server#17, as the place CI ran. The description leaves the manual test through `fastly compute serve` unticked. The description says that no file has been made through an adapter against a real origin, and that the rule is checked through the publisher path with a stub origin answering `404`. **Checks that can be repeated.** In a clone of IABTechLab/trusted-server, this diff prints `8 files changed, 448 insertions(+), 5 deletions(-)`, which is the footprint in section 2. @@ -173,6 +173,7 @@ git grep -n "\.making(" 7723c5e56 -- crates ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/22-creating` at `7723c5e56`. Issues found, four of them blocking. The four were that the spec said the made file is stored when core makes the file again on every request, that the rule left out four behaviors (any path under the prefix makes a file, the origin's `404` body and headers are dropped, an HTML entry makes nothing and the longest path wins), that the header named pull requests that are closed and never the series pull request, and that the acceptance said the pull request description records checks, 6 deliberate faults among them, which neither pull request description records. This revision states what the code does at that head for the trigger, the entry that makes the file, the response, the phases, caching, configuration, the adapters, consent and privacy, failure and the tests, replaces the acceptance claims with what the pull request description records, and leaves four questions about the code under Open questions until the code changes, being an entry whose `path` covers every path, a request with no host, the sellers check that accepts an entry that cannot make `/ads.txt` and the example settings file. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/22-creating` at `7723c5e56`. Approved, none blocking. This revision gives the 25 passing check runs on the pull request and the 19 on the fork's closed pull request separately, says that the description records only the 19, and quotes the description for where the full workflow set ran. ## Revision record @@ -180,3 +181,4 @@ git grep -n "\.making(" 7723c5e56 -- crates | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The header and the pull requests the header names, what the change touches, the rule as tables for the trigger, the entry and the response, the empty document, caching, configuration and rollout, the adapters, consent and privacy, failure, the rows that hold what the change leaves out, the open questions, the tests and the cases with none, the acceptance and the review log. | +| 2026-10-16 | Fixes from review round 2. The acceptance gives the 25 check runs on the pull request and the 19 on the fork's closed pull request separately, says that the description records only the 19, and quotes the description for where the full workflow set ran. | From 213ec229647a2876c3a8bb4aac04474c99d21f9d Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 09:56:36 +0100 Subject: [PATCH 107/192] The robots.txt from modules design takes its round 2 review It cites line 174, names the changelog's malformed-file claim and records the earlier CI commit. --- .../specs/2026-10-12-robots-txt-from-modules-design.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md b/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md index 6522676ac4..62cd1335cf 100644 --- a/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md +++ b/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md @@ -54,7 +54,7 @@ A contributor returns a `Contribution`, which is records before the first group 1. With `refuse_all` selected the file is the refusal and nothing else, whatever else is selected. 2. A path in `always_allow` stays open to every crawler a contribution refuses the whole site to, `*` included. Core adds a group of its own below the contributions and never edits them. A crawler that obeys only the first group it finds then still meets `Disallow: /` first, so the worst it can do is fail to read the one page. A narrower rule is kept as given, so `Disallow: /ads` still closes off `/ads.txt`. 3. The rules appear in the order `modules` names them, between the publisher's `top_text` above and the `Sitemap` line and `bottom_text` below. -4. A value cannot start a record of its own, because core takes every CR and LF out of each record's value, each comment, each record name and each `User-Agent` name as it writes them (`document.rs` lines 41 to 48 and 58 to 66). +4. A value cannot start a record of its own, because core takes every CR and LF out of each record's value, each comment, each record name and each `User-Agent` name as it writes them (`one_line` in `document.rs` lines 58 to 66, called for a record at lines 41 to 48 and for a `User-Agent` name at line 174). **Where those guarantees stop.** Core does not check the shape of a contribution beyond the line breaks. A group with no `User-Agent` line, and any record before the first group, are written as given (`document.rs` lines 159 to 180). RFC 9309 ends a group only at the next `User-Agent` line or at the end of the file (section 2.1) and allows empty lines inside a group (section 2.2), so a crawler reads a record that has no `User-Agent` line of its own as part of the group above the record, which can be the last group of the previous contribution or the publisher's `top_text`. Whether a contribution refuses the whole site is read from the text of its rules. `Disallow: /` counts, and so does `/` followed by `*` characters alone, such as `Disallow: /*` (`document.rs` lines 50 to 55). `Disallow: /*$` does not count, although it matches every path, so `always_allow` is not kept open for a crawler that is refused only that way. An empty `User-Agent` name is passed over, because there is nothing to address the kept-open group to (lines 257 to 274). Open question 7 asks about the shape of a contribution. @@ -127,7 +127,7 @@ On Fastly the configuration, permissions and data pages answer before the Edge C **How soon a crawler sees a change.** The `200` is cacheable for an hour, as the cache headers above say. A crawler or a cache can keep the previous file for up to an hour after the settings change, after a contributor's rules change or after a build starts serving the file. Where answers are held, a contributor's new rules reach the file on the first request after its `refresh` is due. An answer held under a fingerprint that has since changed is neither used nor replaced, because the new fingerprint gives a new key. That entry stays in the store for 30 days after it was last written where the store gives a lifetime, and until someone deletes that entry where the store does not. -**The guide and the changelog.** The configuration guide and `CHANGELOG.md` say a module's answer is held in the key-value store. Neither says that the Fastly and Axum adapters hold nothing (Open question 1). +**The guide and the changelog.** The configuration guide (`docs/guide/configuration.md` line 1057) and `CHANGELOG.md` (line 82) say a module's answer is held in the key-value store. Neither says that the Fastly and Axum adapters hold nothing (Open question 1). `CHANGELOG.md` line 82 also says "a contributor cannot produce a malformed one". The paragraph "Where those guarantees stop" in section 2 says otherwise, because core writes a group with no `User-Agent` line and a record before the first group as given (Open question 7). ## 4. What is not in this change @@ -185,13 +185,14 @@ Open question 8 asks whether adapter tests should cover the first four. ## 6. Acceptance -**What the description records.** The description of [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355) ticks `cargo test-fastly && cargo test-axum`, `cargo clippy-fastly && cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check, the WebAssembly build of the Fastly adapter and the full workflow set run on the fork's pull request, and leaves the manual run with `fastly compute serve` unticked. The description records 19 of 19 checks green on [jwrosewell/trusted-server#9](https://github.com/jwrosewell/trusted-server/pull/9) at `35eb5ae83`. The description says that no vendor module has contributed rules through the extension point, so the seam probe's contributor is the one proof of the extension point, and that the key-value store's lifetime on Fastly and Cloudflare is set but not read back. On Fastly the store is the unavailable store, so no lifetime is set there (Open question 1). The description does not record a lint and test run of each commit on its own, or any deliberate faults. +**What the description records.** The description of [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355) ticks `cargo test-fastly && cargo test-axum`, `cargo clippy-fastly && cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check, the WebAssembly build of the Fastly adapter and the full workflow set run on the fork's pull request, and leaves the manual run with `fastly compute serve` unticked. The description records 19 of 19 checks green on [jwrosewell/trusted-server#9](https://github.com/jwrosewell/trusted-server/pull/9) at `35eb5ae83`. The closing paragraph of the description also says "CI passed on `c66316b4a` on the fork". That commit is the merge of the robots.txt comment fix into the branch, and `35eb5ae83` contains that commit, so the description names two commits for the fork's checks. The description says that no vendor module has contributed rules through the extension point, so the seam probe's contributor is the one proof of the extension point, and that the key-value store's lifetime on Fastly and Cloudflare is set but not read back. On Fastly the store is the unavailable store, so no lifetime is set there (Open question 1). The description does not record a lint and test run of each commit on its own, or any deliberate faults. **CI.** Every check passed, 25 of 25 on IABTechLab/trusted-server#1355 and 19 of 19 on jwrosewell/trusted-server#9, as read on 2026-10-16. ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/14-robots-txt` at `35eb5ae83`. Issues found, four of them blocking. The four were that the spec said every answer is held in the key-value store when Fastly and Axum pass a store that holds nothing, that the spec said a contributor is given nothing about who asked when the services it receives carry the crawler's connection, that the spec said every response carries `X-Robots-Tag` when only the responses an adapter finalizes do, and that the header named pull requests that have closed instead of the series. This revision states what the code does at that head for the store each adapter passes, what a contributor is given and what the held key leaves out, the refusal and an empty answer, the steps each adapter runs before the handler, the responses that carry the header, the cache headers, the configuration, rollout and the tests, replaces the acceptance claims with what the pull request description records, names the rows that replace the route and the holding, and leaves eight questions under Open questions that wait for code changes, being what Fastly and Axum hold, what a contributor can read, host, scheme and method, an empty answer, the route on Fastly, the header after the filters, the shape of a contribution and the missing tests. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/14-robots-txt` at `35eb5ae83`. Approved, none blocking. This revision cites `document.rs` line 174 for the line breaks removed from `User-Agent` names, names the changelog's claim that a contributor cannot produce a malformed file among the statements that this spec contradicts, and records the earlier commit that the description says CI passed on. ## Revision record @@ -199,3 +200,4 @@ Open question 8 asks whether adapter tests should cover the first four. | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the settings, the refusal and the empty answer, what a contributor is given and what the key holds, the store each adapter passes, the steps each adapter runs before the handler, the responses that carry `X-Robots-Tag`, the cache headers, consent and privacy, what is refused and when, what changes for a deployment, the rows that replace the route and the holding, the open questions, the tests, the acceptance and the review log. | +| 2026-10-16 | Points from review round 2. The citation for the line breaks removed from `User-Agent` names, the changelog's claim that a contributor cannot produce a malformed file, and the earlier commit that the description says CI passed on. | From 77323802b21f769dc82c67f8db65b94919fb0ba6 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 09:56:41 +0100 Subject: [PATCH 108/192] The loading cache design takes its round 2 review It states how the two published crates differ and confirms the checks. --- docs/superpowers/specs/2026-10-12-loading-cache-design.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index ca9b923e2c..7ddabf1b6a 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -45,7 +45,7 @@ store_idle_seconds = 2592000 **Why.** The same answer is wanted by every instance and every request, and a cache that lets every miss load at once asks a module's service as many times as there are requests. Loading each key once at a time in a process, with the value shared by every waiter, is the pattern of `LruLoadingCache` in the 51Degrees [.NET](https://github.com/51Degrees/caching-dotnet) and [Java](https://github.com/51Degrees/pipeline-java) pipelines, which the published crate's documentation points to. Core keeps only the interface. Core's cache code depends on no caching package. The only place core writes the module name `51degrees` is `settings.rs`, in the documentation of `LoadingCacheConfig::module` (line 3055) and in the test `a_loading_cache_module_is_read_and_written_back` (lines 7036 to 7055), and the open questions ask about it. -**The pin.** Row 24 pins `fiftyone-caching` at the pre-release `4.5.8-49` and row 31 at the release `4.5.8`. Row 31 changes `Cargo.toml` and `Cargo.lock` and nothing else, and has no design of its own. The two crate versions contain the same source files and differ only in their version strings and in the commit they were published from, so everything this document says about the published crate applies to both rows. +**The pin.** Row 24 pins `fiftyone-caching` at the pre-release `4.5.8-49` and row 31 at the release `4.5.8`. Row 31 changes `Cargo.toml` and `Cargo.lock` and nothing else, and has no design of its own. The two published crates, [`4.5.8-49`](https://crates.io/crates/fiftyone-caching/4.5.8-49) and [`4.5.8`](https://crates.io/crates/fiftyone-caching/4.5.8), were downloaded from crates.io on 2026-10-16 and compared file by file. The 21 files under `src/`, the 16 files under `tests/` and `README.md` are identical in the two. The other four files differ. `Cargo.toml`, `Cargo.toml.orig` and `Cargo.lock` differ in the crate's own version and in the version of `fiftyone-pipeline-core` that the crate names as an optional dependency, and `Cargo.lock` also holds the checksum of that version of `fiftyone-pipeline-core`. `.cargo_vcs_info.json` names the commit that each crate was published from, [`762578bf6`](https://github.com/51Degrees/rust/commit/762578bf65767856f6f3a5f5945dfb6f8679633a) for `4.5.8-49` and [`e2fff6f66`](https://github.com/51Degrees/rust/commit/e2fff6f66097b81d831b7e93c9c35372831e4cc4) for `4.5.8`, and marks both as published from a working tree with uncommitted changes. Everything this document says about the published crate therefore applies to both rows. **Where row 24 and row 31 differ.** The two rows differ in three places. @@ -145,7 +145,7 @@ git grep -n -e '\.detachable' -e 'cloudflare::with_context' -e WaitUntil -e loca - `crates/trusted-server-modules/src/lib.rs`. The stock list order, which names `loading-cache.51degrees`, and `a_feature_selects_the_stock_loading_cache_by_name`. - `crates/trusted-server-adapter-fastly/src/value_cache.rs`, which `cargo test-fastly` runs under Viceroy, Fastly's local runtime. Seven tests, including `a_missing_key_is_reserved_and_its_fill_is_then_a_hit`, `a_dropped_reservation_passes_the_obligation_on` and `a_stale_entry_is_reserved_rather_than_served`. -**CI.** Every check passed on the head of each row. That is 19 of 19 on the closed pull requests on the fork, jwrosewell/trusted-server#19 at `844b01fe2` and jwrosewell/trusted-server#26 at `4b6ef9c05`, and 25 of 25 on IABTechLab/trusted-server#1365 and IABTechLab/trusted-server#1372. +**CI.** Every check passed on the head of each row. That is 19 of 19 on the closed pull requests on the fork, jwrosewell/trusted-server#19 at `844b01fe2` and jwrosewell/trusted-server#26 at `4b6ef9c05`, and 25 of 25 on IABTechLab/trusted-server#1365 and IABTechLab/trusted-server#1372, as read on 2026-10-16. **Local checks.** The description of IABTechLab/trusted-server#1365 records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run. The description marks a manual run with `fastly compute serve` as not done. @@ -174,6 +174,7 @@ These are for the reviewer. This revision records what the code does and decides ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/31-caching-release` at `4b6ef9c05`, with the design's code read on `split/24-loading-cache` at `844b01fe2`. Issues found, three of them blocking. The three were that the spec said a feature that names no module gets `NoLoadingCache`, when a stock build gives it the one loading cache module linked, that the spec said loads run on a pool of threads on a native host, when every load runs in its caller and Axum has no key-value store, and that the spec said a module is asked once per refresh however many instances want its rules, when only a store that reserves a key makes other instances wait. This revision states what the code does on both rows, adds the adapters, the configuration, consent and privacy, failure, compatibility, the tests and the pull requests that exist now, and leaves seven code questions under Open questions, which wait for code changes. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/31-caching-release` at `4b6ef9c05`, with the design's code read on `split/24-loading-cache` at `844b01fe2`. Issues found, none blocking. This revision states how the two published crates differ, from a file by file comparison of the crates themselves, and confirms the check counts, the pull request states and the description of IABTechLab/trusted-server#1365 that this spec cites. ## Revision record @@ -181,3 +182,4 @@ These are for the reviewer. This revision records what the code does and decides | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the default when a table names no module, how loads run, who waits for a load, where rows 24 and 31 differ, the adapters, the configuration, consent and privacy, failure, what changes for a deployment, the tests and what none covers, the open questions and the review log. | +| 2026-10-16 | Points from review round 2. How the two published crates differ, from a comparison of the crates themselves, and the date on which the check counts were read. | From efc0cf894f38dfc3f1885f48cee9107b1aae964a Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 10:05:19 +0100 Subject: [PATCH 109/192] The cookie constant design describes the three fixes Core names no Model Terms cookie, Prebid Server receives the preference only where it receives the consent cookies, and a test checks the cookie name against the platform's own Set-Cookie strings. The open questions those fixes answer are moved out and the tests are listed. --- .../2026-10-12-cookie-constant-design.md | 35 ++++++++++--------- 1 file changed, 19 insertions(+), 16 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-cookie-constant-design.md b/docs/superpowers/specs/2026-10-12-cookie-constant-design.md index 554fed7064..a3fa250a13 100644 --- a/docs/superpowers/specs/2026-10-12-cookie-constant-design.md +++ b/docs/superpowers/specs/2026-10-12-cookie-constant-design.md @@ -7,7 +7,7 @@ **Issues:** [IABTechLab/trusted-server#1275](https://github.com/IABTechLab/trusted-server/issues/1275). [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack **Last updated:** 2026-10-16 -> **Why this spec exists.** `COOKIE_MTM_PREF`, the name of the cookie that the Preference Management Platform (PMP) writes the Model Terms for Marketing preference into, is declared in core's constants on the rows below this one, although the Model Terms signal module is the only code that reads the cookie. This change declares the name in that module and changes no behavior. +> **Why this spec exists.** `COOKIE_MTM_PREF`, the name of the cookie that the Preference Management Platform (PMP) writes the Model Terms for Marketing preference into, is declared in core's constants on the rows below this one, although the Model Terms signal module is the only code that reads the cookie. This change declares the name in that module, and the fixes after review withhold the cookie from Prebid Server wherever the consent cookies are withheld. ## 1. Problem @@ -33,11 +33,15 @@ Line numbers in this document are those of commit `af8af38f5`, the head of IABTe **What the module does with the cookie does not change.** The diff changes no other line of the module. `preference()` reads the first pair named `__mtm_pref` in the request's first `Cookie` header, through `RequestInfo::cookie`. A cookie that is absent, or whose value is not one of the three words once the spaces around it are removed, is read as no answer, and the module then answers `Neutral` for every permission, so the country and region rules stand (lines 117 to 127 and 189 to 194). What each word grants or revokes, and the terms the module declares, are as on row 16. +**The fixes after review.** Three commits on top of the move fix what the reviews found. `f8d0ccfe3` takes the cookie's name out of the README of core's permission signal module, which now says the module declares the name, and corrects "two words" to three in the comment on `valid_signal` and in the `mtm` row of the Permission Signals guide. `389903405` withholds the cookie from Prebid Server, as below. `71e039d4c` adds the test of the cookie's name in section 5. + +**What reaches Prebid Server.** With `consent_forwarding = "openrtb_only"`, the Prebid Server crate removes the four consent cookies and `__mtm_pref` from the `Cookie` header it sends, because the preference records the visitor's choice as the consent cookies do. The crate reads the name from the Model Terms crate, so core still names no scheme's cookie. In `cookies_only` and `both` the header is forwarded whole, the preference included. + **Why a constant and not a setting.** The name is what the module and whatever platform writes the cookie have to agree on, and the constant's comment says that the name is "not a Trusted Server name", so that anyone can build a platform that sets the cookie. A deployment that renamed the cookie would stop reading the answer of every platform that keeps to the agreed name. A setting would also be the first setting of any permission signal module, and the `[permission-signal]` section accepts only `modules` today and refuses any other key by name (`crates/trusted-server-core/src/settings.rs`, lines 1334 to 1340). ## 3. What changes for a deployment -**Nothing a deployment can observe.** This change adds no setting, so no setting selects the change and every deployment gets it. The cookie's name and the way the module reads the cookie are the same, so no deployment behaves differently and the PMP needs no change. The `[permission-signal]` section accepts the same key, `modules`, with the same default and the same checks. A deployment that lists `mtm`, or leaves `modules` out so that every module an adapter offers runs, reads the same cookie as before. +**One change a deployment can observe.** A deployment with `consent_forwarding = "openrtb_only"` no longer sends `__mtm_pref` to Prebid Server. This change adds no setting, so every deployment gets it. The cookie's name and the way the module reads the cookie are the same, so no other deployment behaves differently and the PMP needs no change. The `[permission-signal]` section accepts the same key, `modules`, with the same default and the same checks. A deployment that lists `mtm`, or leaves `modules` out so that every module an adapter offers runs, reads the same cookie as before. **What breaks for code.** The constant moves from `trusted_server_core::constants::COOKIE_MTM_PREF` to `trusted_server_permission_signal_mtm::COOKIE_MTM_PREF`, and stays public. Only two files in this repository imported the old path, being the module itself and the Axum adapter's test, and the change updates both. `main` does not declare the constant, so no crate built against `main` imports it. A crate built against rows 4 to 16 that imports the old path stops compiling until the crate changes the path. No row after 17 imports the constant from core (checked through row 50). @@ -55,20 +59,19 @@ Only the Axum adapter's tests send the cookie through the module, in `permission ## 4. What is not in this change - **Other cookie names in core's constants.** `COOKIE_TS_EIDS` (line 14), `COOKIE_SHAREDID` (line 16) and the four names of the IAB consent cookies, `euconsent-v2`, `__gpp`, `__gpp_sid` and `us_privacy` (lines 151 to 154), each name the cookie of one scheme or one vendor and stay in core's constants on this row. The Fastly adapter reads `COOKIE_SHAREDID` at lines 636 and 955 of its `src/app.rs`. The EIDs cookie's name changes at row 27 and leaves core at row 41. Row 27 ([IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), which lets a module name the browser's cookies) replaces `COOKIE_TS_EIDS` with `COOKIE_EIDS`, which holds `eids` and stays at line 17 of core's `constants.rs` through row 40, and core writes the cookie's whole name after the deployment's public prefix, so the name is `ts-eids` under core's own prefix. Row 41 ([IABTechLab/trusted-server#1382](https://github.com/IABTechLab/trusted-server/pull/1382)) deletes `COOKIE_EIDS` and the Prebid crate declares the name instead, as `EID_COOKIE` (`crates/auction/prebid/src/eids.rs`, line 24), so core names the EIDs cookie nowhere from row 41. Row 42 ([IABTechLab/trusted-server#1383](https://github.com/IABTechLab/trusted-server/pull/1383)) moves the consent cookie names. `COOKIE_SHAREDID` is still in core on row 50. -- **The README and the guide.** The README of core's permission signal module (`crates/trusted-server-core/src/permission_signal/README.md`) names `__mtm_pref` in a sentence on lines 39 and 40, and the Edge Cookies guide (`docs/guide/edge-cookies.md`, line 96) lists the cookie among the signals that the permission signal modules read. Neither changes here. The README is the only file in `crates/trusted-server-core` that names the cookie, and it still does on row 50. Open question 2 asks about the README. +- **The Edge Cookies guide.** `docs/guide/edge-cookies.md` (line 96) lists the cookie among the signals that the permission signal modules read. That is a guide and not core, and it does not change. - **The permission model design.** `2026-07-30-permission-model-design.md`, section 4.6, still says that the cookie's constant is `COOKIE_MTM_PREF` in core, and this change does not amend that document. -- **What reaches Prebid Server.** The cookie reaches Prebid Server in every `consent_forwarding` mode, as it did before this change. Only the `openrtb_only` mode removes cookies from the forwarded `Cookie` header (line 1145 of `crates/auction/prebid-server/src/lib.rs`), and the mode removes the four in `CONSENT_COOKIE_NAMES` (`crates/trusted-server-core/src/cookies.rs`, lines 22 to 27), which do not include `__mtm_pref`. Open question 4 asks whether that is intended. -- **A setting or a new test.** The change adds neither. Section 5 says what no test covers. +- **A setting.** The change adds none. ## 5. Acceptance -**Checks that can be repeated.** In a clone of IABTechLab/trusted-server, this search prints one line, which is in the README of core's permission signal module, so no Rust file in `crates/trusted-server-core` names the constant or the cookie. +**Checks that can be repeated.** In a clone of IABTechLab/trusted-server, this search prints nothing, so no file in `crates/trusted-server-core` names the constant or the cookie. ```text -git grep -n -e COOKIE_MTM_PREF -e __mtm_pref af8af38f5 -- crates/trusted-server-core +git grep -n -e COOKIE_MTM_PREF -e __mtm_pref 71e039d4c -- crates/trusted-server-core ``` -Because no test fails if the literal changes (see below), this diff must also print two lines that are the same apart from their `-` and `+`, being the removed and the added declaration, `pub const COOKIE_MTM_PREF: &str = "__mtm_pref";`. +This diff must also print two lines that are the same apart from their `-` and `+`, being the removed and the added declaration, `pub const COOKIE_MTM_PREF: &str = "__mtm_pref";`. ```text git diff 7b19336ec...af8af38f5 | grep __mtm_pref @@ -78,19 +81,18 @@ git diff 7b19336ec...af8af38f5 | grep __mtm_pref **Local checks.** The description of IABTechLab/trusted-server#1358 records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run. The description marks a manual run with `fastly compute serve` as not done. +**Tests added.** + +1. **The cookie's name.** `the_platforms_own_set_cookie_strings_name_the_cookie_the_module_reads`, in the Model Terms crate, asserts that the name in each of the PMP's `Set-Cookie` strings (`PMP_SET_COOKIE`) equals `COOKIE_MTM_PREF`. With the constant changed to another name, this test fails and the crate's other 14 tests pass. +2. **What reaches Prebid Server.** `openrtb_only_withholds_the_model_terms_preference_from_prebid_server`, `cookies_only_forwards_the_model_terms_preference_to_prebid_server` and `both_forwards_the_model_terms_preference_to_prebid_server`, in the Prebid Server crate, send a page request carrying a session cookie, `euconsent-v2` and `__mtm_pref` through the dispatch path and read the `Cookie` header Prebid Server receives. Without the fix the first fails. + **What no test covers.** -1. **The cookie's name.** No test compares the name the PMP writes with `COOKIE_MTM_PREF`. The module's tests build the cookie header from the constant (`with_cookie`, line 261) and so does the Axum test (line 630). The fixtures of the PMP's `Set-Cookie` strings (`PMP_SET_COOKIE`, lines 292 to 313, which the comment above them calls the PMP's own output) carry the literal `__mtm_pref`, but the tests that use them read only the value and the absence of `HttpOnly` (lines 316 to 319 and 323 to 357), never the name. From reading the tests, a change to the literal at line 74 would move the code and the tests together and fail none of them. -2. **The adapters' own lists.** No test refers to `shipped_signal_modules`, the private function in each adapter's `src/app.rs` that lists the modules, so no test checks that an adapter registers the Model Terms module. The Fastly, Cloudflare and Spin adapters also have no test that reads the cookie. +1. **The adapters' own lists.** No test refers to `shipped_signal_modules`, the private function in each adapter's `src/app.rs` that lists the modules, so no test checks that an adapter registers the Model Terms module. The Fastly, Cloudflare and Spin adapters also have no test that reads the cookie. ## 6. Open questions -These are for the reviewer. This revision records what the code does and decides none of them. - -1. Should a test assert that the cookie name in each `PMP_SET_COOKIE` fixture equals `COOKIE_MTM_PREF`, so that a changed literal fails a test? The name matters because a module that reads the wrong name sees no answer from any visitor, and a visitor's `standard` choice then no longer revokes the four uses that depend on browsing history (lines 94 to 103 and 217 to 224) where the country baseline grants them. -2. Should the README of core's permission signal module stop naming `__mtm_pref`? The README says on its lines 21 and 22 that core "holds no scheme's wire format and no scheme's meaning", and the identifier modules design (`2026-10-14-identifier-modules-design.md`, row 41) says that core names the Model Terms cookie nowhere. -3. Should the doc comment on `valid_signal` (lines 239 and 240 of the module), which says that a value that is not "one of the two words" is not an answer, say three words, as the module reads three? The row for `mtm` in the Permission Signals guide (`docs/guide/permission-signals.md`, line 123) also says "one of two words". -4. Is it intended that `__mtm_pref` reaches Prebid Server when the four consent cookies are removed in `openrtb_only` mode? With the name out of core, removing the cookie would need a list that core can read without naming the cookie. On row 42 the list moves into the Prebid Server crate, built from the constants of the TCF, GPP and US Privacy crates, and still does not hold the Model Terms cookie. Neither the issue (IABTechLab/trusted-server#1275) nor the pull request description (IABTechLab/trusted-server#1358) says whether that is intended. +None remain. The four questions of the round 1 revision, being the test of the cookie's name, the README naming the cookie, the "two words" comment and the cookie reaching Prebid Server in `openrtb_only` mode, are fixed as section 2 says. ## Review log @@ -104,3 +106,4 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The header and the pull requests it names, where the constant arrived, the three files and what each changes, the reworded comment, why a constant and not a setting, what changes for a deployment and for code, the adapters, the names left in core, the acceptance checks, the untested paths, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. The earlier pull request on this repository, closed without being merged, named in the header, the rows at which the EIDs cookie's constant is renamed and then deleted from core (section 4), and the review log. | +| 2026-10-16 | Fixes from code review. `__mtm_pref` is withheld from Prebid Server in `openrtb_only` mode (`389903405`), a test checks the cookie's name against the PMP's `Set-Cookie` strings (`71e039d4c`), core's README no longer names the cookie, and the comment on `valid_signal` and the Permission Signals guide say three words (`f8d0ccfe3`). | From 6c5bf2603c438c3943a2baa2c4827a18d9754502 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 10:08:08 +0100 Subject: [PATCH 110/192] Add the settings and consumers design One declaration model in core and one check, at startup and at push, that reports every settings section with no consumer and every consumer with no settings, in place of the separate checks each feature wrote. For IABTechLab/trusted-server#1403. --- ...026-10-16-settings-and-consumers-design.md | 167 ++++++++++++++++++ 1 file changed, 167 insertions(+) create mode 100644 docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md diff --git a/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md b/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md new file mode 100644 index 0000000000..67757c6ba5 --- /dev/null +++ b/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md @@ -0,0 +1,167 @@ +# Design Spec: Every settings section has a consumer, and every consumer has its settings + +**Status:** Proposed, 2026-10-16 +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-10-12-page-middleware-design.md` (row 10, entries that name middleware), `2026-10-12-shared-vendor-tables-design.md` (row 26), `2026-10-13-middleware-settings-design.md` (row 38), `2026-10-14-no-modules-crate-design.md` (row 43, each adapter lists the modules it links) and `2026-10-15-module-cache-declarations-design.md` (row 32), all in IABTechLab/trusted-server#1342, which holds every design document of the stack +**Related PRs:** none yet +**Issues:** IABTechLab/trusted-server#1403 (settings without a consumer and consumers without settings), with the related IABTechLab/trusted-server#1125, #1132, #1201, #1097, #1142, #1148, #1134, #1102, #1140, #1141 and #1139 +**Last updated:** 2026-10-16 + +> **Why this spec exists.** A settings section and the consumer that acts on it are checked against each other by a dozen separate checks that run at different times and stop at the first gap, and several pairs are not checked at all, so this spec proposes one declaration model in core and one check that reports every gap at startup and at push. + +Line references are at `3aaf720c0`, the head of `split/50-cookies-by-path`, which is the head of IABTechLab/trusted-server#1400 and the top of the stack. A file named without a folder is in `crates/trusted-server-core/src`, and `registry.rs` and `mod.rs` are `integrations/registry.rs` and `integrations/mod.rs`. + +## 1. Problem + +Four words are used as the code uses them. A builder (`IntegrationBuilder`, `mod.rs` line 452) is what a crate hands to core to say what the crate offers. The registry is what core builds at startup from every builder (`IntegrationRegistry::with_plan_and_registrations`, `registry.rs` lines 2033 to 2400). Startup is an adapter loading the settings and building its state, which each adapter does through that function (Fastly `app.rs` line 360, Axum `app.rs` line 240, Cloudflare `app.rs` line 295, Spin `app.rs` line 219). Push is `ts config validate`, `ts config diff` and `ts config push`, which validate through `validate_settings_for_deploy_with` (`config.rs` lines 311 to 341). + +A consumer is whatever acts on a settings section, being a middleware an entry names, a module a section selects, a request filter, a page request preparer or a part of the adapter. Many features need both halves, and today the two halves are matched in four different ways. + +**Checks in the registry, at startup only.** Push builds no registry. `validate_phase_entries` checks the shape of each entry and says that whether a name is supplied "is only knowable where the registry is built" (`settings.rs` lines 4776 to 4791), and the section selection check is left out of push deliberately, "because a vendor crate the CLI never links may supply the name" (`registry.rs` lines 2163 to 2168). The registry's checks are these. + +- An entry naming a middleware no running module supplies is refused (`registry.rs` lines 1669 to 1685). The note that follows names a module no section selects (lines 1118 to 1134) and never names an absent shared table. +- A `[middleware.]` table at a name no running module supplies is refused (lines 1702 to 1723). +- A middleware no entry names is a warning, that it "changes no page" (lines 1724 to 1731). +- A `[shared.]` table no builder reads is refused (lines 2044 to 2069). +- A section selecting a module no builder supplies is refused, with a note naming absent shared tables (lines 1774 to 1825 and 1162 to 1178). +- A geo, Edge Cookie, device or analytics module that runs and is not selected is a warning (lines 1199 to 1212). + +**Checks a feature wrote for itself.** The InMobi prompt refuses `[cmp]` selecting it with no `text/html` entry naming it, whatever the entry's `path` (`crates/cmp/inmobi/src/lib.rs` lines 141 to 158). The ads.txt sellers do the same for `text/plain` (`crates/ads-txt/iabtl/src/lib.rs` lines 152 to 167). Core's robots.txt check requires `path = "/robots.txt"` exactly (`settings.rs` lines 4414 to 4462). Each of the three runs at push and at startup, because a builder's validate function runs at both (`config.rs` lines 394 to 406) and `prepare_runtime` runs at both (`config.rs` line 323 and `settings.rs` lines 4357 to 4406). The cache table check is the one core check that runs at push as well (`mod.rs` lines 1008 to 1025, called from `config.rs` line 330). + +**No check.** These pairs are not matched at all. + +- The 51Degrees client hints crate registers a request filter and a middleware whenever `[shared.51degrees]` is present (`crates/middleware/51degrees/src/lib.rs` lines 110 to 132). Core runs every registered filter (`registry.rs` lines 2640 to 2674), so with no entry naming the middleware the filter still asks the service on every page request (`lib.rs` lines 352 to 381), and a request carrying the filter's state skips the shared page template and is sent `no-store, private` (`publisher.rs` lines 5536 to 5540 and 6396 to 6402). +- The DataDome page tag writes nothing when `client_side_key` is blank (`crates/bot-protection/datadome/src/lib.rs` lines 1016 to 1026), and the startup checks look only at the tag's URL (lines 509 to 511), so an entry naming `bot-protection.datadome.tag` with no key is accepted silently. +- Only the Fastly adapter runs request filters (Fastly `app.rs` lines 836 to 867), dispatches asset routes (lines 1191 to 1194 and 1310 to 1313), reads the analytics sink (lines 389 to 392) and configures origin certificates from `certificate_check` (Fastly `backend.rs` lines 492 to 496). The Axum adapter's origin client takes no certificate setting (Axum `platform.rs` lines 321 to 333), and the Cloudflare and Spin adapters use the setting only in a backend's name (Cloudflare `platform.rs` lines 70 to 78, Spin `platform.rs` lines 92 to 100). With `[geo] module = "platform"` the registry leaves the adapter's own lookup in place (`registry.rs` lines 1480 to 1482), and the Axum and Spin lookups answer no location for every request (Axum `platform.rs` lines 210 to 218, Spin `platform.rs` lines 809 to 817). Each of these is accepted on the adapters without the part, with no log. +- `cluster_trust_threshold` and `cluster_recheck_secs` in `[ec]` are read into the settings (`settings.rs` lines 783 to 795) and read by no crate, which IABTechLab/trusted-server#1097 records. + +**One gap at a time.** Every check returns its first error (for example `registry.rs` lines 1679, 1716, 1815 and 2063), so a document with two gaps fails twice before it starts. + +**One-off notices.** Where a setting is known to do nothing, the code that knows says so in its own way, being the Fastly warning when `certificate_check` is off (Fastly `app.rs` lines 484 to 490) and the DataDome warning for its reserved GraphQL flag (`lib.rs` lines 513 to 515). + +## 2. Design + +One model in core, in four points. + +1. Every consumer declares the settings it needs. +2. Every settings section has a consumer. +3. Each adapter declares the parts it has. +4. Startup, `ts config validate` and `ts config push` give one report listing every gap. + +### 2.1 The declaration types in core + +Core already holds one declaration a builder makes about a consumer, `SuppliedMiddleware`, which names a middleware, its phase and its media type (`mod.rs` lines 376 to 385), and the registry already checks that declaration against the middleware the builder really supplies (`check_described_middleware`, `registry.rs` lines 1591 to 1630). The new types extend that. They name no vendor, standard or platform, and every name in them is a string the declaring crate supplies. + +```rust +/// What one consumer a builder supplies needs before it acts. +pub struct ConsumerNeeds { + pub consumer: Consumer, + /// The settings the consumer reads, each required or optional. + pub settings: &'static [SettingsNeed], + /// What places the consumer in a deployment. + pub placed_by: Placement, + /// The adapter parts the consumer cannot act without. + pub parts: &'static [AdapterPart], +} + +pub enum Consumer { + /// A middleware, by the name an entry writes. + Middleware(&'static str), + /// A module a section selects, by its full name. + Module(&'static str), + /// A request filter or a page request preparer that only feeds the + /// named middleware, so it runs only where that middleware is placed. + Feeds(&'static str), +} + +pub struct SettingsNeed { + /// The table, written as in the document, such as `shared.`. + pub table: &'static str, + /// Keys inside the table the consumer cannot act without. + pub keys: &'static [&'static str], + pub required: bool, +} + +pub enum Placement { + /// An entry in the phase, for the media type, and on the path when one + /// is given. + Entry { + phase: MiddlewarePhase, + media_type: &'static str, + path: Option<&'static str>, + }, + /// The section's `module` or `modules` naming the consumer. + Section(&'static str), + /// Present whenever its settings are, such as an adapter part. + WithSettings, +} + +/// A capability an adapter may have, named by core in neutral terms. +pub struct AdapterPart(pub &'static str); +``` + +Core names the parts it knows of as constants, for example running request filters, dispatching asset routes, reading the telemetry sink, setting origin certificate checks and a host geo lookup. A part is a capability and never a platform. + +### 2.2 How each crate and each adapter declares + +**A vendor crate** declares on its builder, next to the declarations a builder already makes (`with_shared_settings`, `with_secret_settings`, `with_middleware` and `with_cache_features`, `mod.rs` lines 655 to 741). A new `with_needs(&'static [ConsumerNeeds])` takes one entry for each consumer the builder supplies. The 51Degrees client hints builder (`crates/middleware/51degrees/src/lib.rs` lines 94 to 100) would declare its middleware as needing `[shared.51degrees]` with `endpoint` and `license_key`, placed by a `[[serve]]` entry for `text/html`, and its request filter as `Feeds("51degrees.client_hints")` needing the part that runs request filters. Each vendor declares in its own folder, so core gains no vendor name. + +**A middleware or module core supplies** declares in core in the same types. The robots.txt middleware would declare `[robots-txt]` as required and a `[[fetch]]` entry for `text/plain` on `/robots.txt` as its placement, which is what `require_robots_txt_placed` checks by hand today (`settings.rs` lines 4414 to 4462). + +**A section core reads directly** declares its consumer as a part. `[[proxy.asset_routes]]` would need the part that dispatches asset routes, `certificate_check = false` the part that sets origin certificate checks, and `[geo] module = "platform"` the host geo lookup. A field kept only so that old documents still load, as IABTechLab/trusted-server#1097 may decide for `cluster_recheck_secs`, is declared as having no consumer, and the report names it as accepted and unused. + +**An adapter** declares the parts it has as one constant list in its own crate, which it hands to the registry build beside its builders. The registry build takes builders alone today (`registry.rs` lines 2033 to 2037), and each adapter already passes its own list of builders (for example Axum `app.rs` lines 109 to 145, from row 43), so the parts travel the same way. + +### 2.3 The one check and the one report + +After every builder has registered, and before any check above returns, core goes through the declarations once. A gap is one of four kinds. + +1. A settings section that no linked and running consumer declares, or that only consumers nothing places declare. +2. A placed consumer whose required settings are absent, with the table and keys named. +3. A consumer whose settings are present and which nothing places. Its feeding request filters and preparers do not run. +4. A section or a consumer that needs a part the adapter lacks, with the adapter and the part named. + +Every gap is collected, and startup refuses with one report in which each line names the section, the consumer, the missing half and what to add. A document with two gaps gets both lines. The existing refusals in section 1 become lines of this report, with their wording kept where it already names what to add. + +A request filter or preparer declared with `Feeds` is run only when its middleware is placed. Preparers that sanitize a reserved query or cookie in a deployment that does not run their module are collected for every builder on purpose (`registry.rs` lines 2103 to 2108), so those declare no `Feeds` and keep running. + +### 2.4 The same check at validate and at push + +`TrustedServerAppConfig` validates through a trait that takes no arguments, so the tool registers the builders it validates against once, before validating (`config.rs` lines 87 to 106). The `ts` tool registers the builders it links (`crates/trusted-server-cli/src/app_config.rs` lines 35 to 81) at the start of every command (`run.rs` line 126). Push would run the check in 2.3 over those builders' declarations, so every gap a stock builder can cause is caught before the document is written to the config store. A selection that names a module the tool does not link is reported as a line saying that the name is not one this tool can check, rather than passed in silence. Adapter parts reach push the same way as the builders, registered once for the adapter the document is pushed for (open question 2). + +### 2.5 A consumer that runs on defaults + +A consumer whose settings are optional declares `required: false`, and leaving its table out is then not a gap. The 51Degrees client hints middleware reads `[middleware.51degrees.client_hints]` and writes `Critical-CH` unless the table turns it off (`crates/middleware/51degrees/src/lib.rs` lines 191 to 202), so it declares that table optional. A loading cache with no `[cache.]` table loads through no cache at all (`platform/loading_cache.rs` lines 240 to 242 and 269 to 272), so a module that declares a cache says in its declaration whether running with no cache is intended, and the report names a missing cache table only where it is not. + +## 3. What is not in this change + +- **Unknown keys and integration IDs.** IABTechLab/trusted-server#1201 covers keys a table does not know, which serde refuses or drops before any consumer sees them. This model starts from tables that read correctly. +- **The health route and the startup failure response.** IABTechLab/trusted-server#1142 decides what an adapter answers once startup refuses, and this model only decides what startup refuses. +- **Adding the missing adapter parts.** Dispatching asset routes on every adapter (IABTechLab/trusted-server#1125) or honoring `certificate_check` on Axum (IABTechLab/trusted-server#1132) remain separate changes. This model makes their absence a named gap until then. +- **Deciding the Edge Cookie cluster settings.** IABTechLab/trusted-server#1097 decides whether they gain a reader or leave. + +## 4. Acceptance + +1. For each row of the cases table in IABTechLab/trusted-server#1403, one test with the settings and no consumer and, where the consumer cannot act without its settings, one test with the consumer and no settings. Each runs through the real settings load and state build of each adapter, as the Axum adapter's seam probe tests already build the adapter's routes from settings (`crates/trusted-server-adapter-axum/tests/seam_probe.rs` line 94), and through the Fastly, Cloudflare and Spin builds, not through the check called directly. +2. Each of those tests is seen to fail with its declaration removed before the test is relied on. +3. A document with two gaps of different kinds gets both lines in one report. +4. `ts config validate`, given the builders the tool links, reports the same lines as startup for every gap a linked builder can cause. +5. A test over every builder each adapter links (for example Axum `app.rs` lines 109 to 145) finds a declaration for each middleware, module, request filter and page request preparer the builder registers, so a new consumer cannot leave one out. `check_described_middleware` (`registry.rs` lines 1591 to 1630) is the model for matching a declaration to what is registered. + +## 5. Open questions + +1. Should every gap refuse at once, or should the two kinds that only warn today, a middleware no entry names (`registry.rs` lines 1724 to 1731) and a module that runs and is not selected (lines 1199 to 1212), warn for one release first? A deployment that carries `[shared.51degrees]` with no entry starts today and would stop. +2. How does push learn which adapter a document is for, and where does push read that adapter's parts from, when the `ts` tool does not link the adapter crates? +3. Should a selection naming a module the tool does not link fail push, or be reported and allowed, as section 2.4 proposes? +4. Should the InMobi and ads.txt sellers placements carry a path, as robots.txt does, or should all three count any entry for the media type? +5. Should the Edge Cookie cluster settings be declared as having no consumer, or wait for the decision in IABTechLab/trusted-server#1097? + +## Review log + +No review rounds yet. + +## Revision record + +| Date | Change | +| ---------- | ----------------------------------------------------------------------------------------------------------------- | +| 2026-10-16 | First draft, from the code at `3aaf720c0` and the issue that states the problem (IABTechLab/trusted-server#1403). | From a788825b9db0bb2c83239606313efc1715eb896b Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 10:10:32 +0100 Subject: [PATCH 111/192] The administration design takes its round 2 review Question 5 gains a changelog line, and the sandbox answer is tied to its feature, which is off by default. --- ...0-12-administration-off-the-publisher-domain-design.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md index c385efb732..357a0a3df4 100644 --- a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md +++ b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md @@ -77,7 +77,7 @@ The check sits in the publisher fallback, so a route that the router answers fir - Five debugging aids that also live on the publisher's domain, all off unless a publisher switches them on. They are of the same kind as what is removed here and are left for a decision of their own. - `GET /_ts/set-tester` and `GET /_ts/clear-tester`, which set and clear a marker cookie when `[tester_cookie]` is on. - `GET /_ts/debug/ja4`, which shows a caller the TLS fingerprint of its own connection when `[debug] ja4_endpoint_enabled` is on. - - `GET /_ts/debug/sandbox`, which shows the counters of the reusable sandbox (the instance identifier, the request ordinal, the application build count and the request correlation id) when `[debug] sandbox_metrics_enabled` is on. + - `GET /_ts/debug/sandbox`, which shows the counters of the reusable sandbox (the instance identifier, the request ordinal, the application build count and the request correlation id) when `[debug] sandbox_metrics_enabled` is on, in a build of the Fastly adapter that has its `reusable-sandbox` feature. - `[debug] auction_html_comment`, which writes auction internals into a page as an HTML comment. - `[debug] inject_adm_for_testing`, which switches on a test path that carries raw creative markup. - `ts keys` and `ts cache purge` drive the Fastly CLI alone, as the routes they replace were served by the Fastly adapter alone. On Axum, Cloudflare and Spin the key and purge routes answered `501` (not supported) before this change, and every path beneath `/_ts/admin` answers `404` on all four adapters now. @@ -85,7 +85,7 @@ The check sits in the publisher fallback, so a route that the router answers fir ## 4. Upgrade - A configuration that carries `[[handlers]]` is refused when it is validated and when the service starts, with directions (`crates/trusted-server-core/src/settings.rs`, lines 3521 to 3538, with tests at lines 5649 to 5707). The key itself is refused, so `handlers = []` is refused too. Loading the list and ignoring it would leave an operator who gated a staging site believing the site was still gated. The settings this version writes carry no `handlers` and no store ids. -- A `[[handlers]]` entry could also gate a route of this server's own that the application answers, `GET /_ts/config` for instance. Nothing here replaces that gate, so a route of the server's that should not be public stays switched off, and `[inspect] config = false` answers `/_ts/config` with not found. `GET /_ts/debug/ja4` was never gated that way, because the Fastly entry point answers that path before the application is built (`crates/trusted-server-adapter-fastly/src/lib.rs`, lines 465 to 481), and `GET /_ts/debug/sandbox` is answered there too (line 444). +- A `[[handlers]]` entry could also gate a route of this server's own that the application answers, `GET /_ts/config` for instance. Nothing here replaces that gate, so a route of the server's that should not be public stays switched off, and `[inspect] config = false` answers `/_ts/config` with not found. `GET /_ts/debug/ja4` was never gated that way, because the Fastly entry point answers that path before the application is built (`crates/trusted-server-adapter-fastly/src/lib.rs`, lines 465 to 481), and `GET /_ts/debug/sandbox` is answered there too (line 444), but only in a build that has the `reusable-sandbox` feature (line 443). The feature is off by default, and the build command in `fastly.toml` does not pass it. - A configuration that carries `config_store_id` or `secret_store_id` under `[request_signing]` is refused the same way, and enabling request signing no longer needs them. - A build before this change refuses a configuration without a handler covering `/_ts/admin`, and one with `[request_signing]` and no store ids (row 12, `crates/trusted-server-core/src/settings.rs`, lines 4161 and 4322 to 4329). So no one configuration starts both builds, and the new configuration and the new build go live together. A staged deploy reads a configuration key of its own (`ts deploy --staging`, `ts config push --staging`), which lets the pair be checked together first. - **The switch.** `ts deploy` and `ts config push` are separate commands, so the build and the configuration do not change in one step. When the settings fail to load, the router that the Fastly adapter builds for a failed startup answers every route, the publisher's pages included, with the startup error (`crates/trusted-server-adapter-fastly/src/app.rs`, lines 75 to 82), which is status `500` for a configuration error. A request that loads the settings between the first of the two changes and the second therefore meets that error. @@ -122,13 +122,14 @@ Each of these is a question about the code. This spec records what the code does 2. Should `secret-store-entry create` be given an option to replace a secret that already has the name? A rotation repeated after a half-finished one, with a secret already stored under the new key id, is not tested. 3. Should `validate_endpoint` refuse an `[attestation] endpoint` beneath `/_ts/admin`, with a test? Nothing else refuses the prefix. 4. Which tests should be added? `/_ts/admin/cache/purge` for each adapter, and the double-encoded forms for Fastly (`crates/trusted-server-adapter-fastly/src/app.rs`, lines 2123 to 2145). -5. Should three comments and one changelog entry be corrected? The comment above `reader_url_surrogate_key` says that the endpoint and the CLI need it (`crates/trusted-server-core/src/platform/template_cache.rs`, line 211). The comment in the publisher path says the keys are those the purge endpoint uses (`crates/trusted-server-core/src/publisher.rs`, line 4564). The module comment of request signing says the store ids come from the settings (`crates/trusted-server-core/src/request_signing/mod.rs`, lines 20 and 21). The changelog entry for the three Edge Cookie routes names `GET /_ts/data` after all three without saying that the page covers neither lookup by id nor the cookie echo (`CHANGELOG.md`, line 63). +5. Should three comments and two changelog entries be corrected? The comment above `reader_url_surrogate_key` says that the endpoint and the CLI need it (`crates/trusted-server-core/src/platform/template_cache.rs`, line 211). The comment in the publisher path says the keys are those the purge endpoint uses (`crates/trusted-server-core/src/publisher.rs`, line 4564). The module comment of request signing says the store ids come from the settings (`crates/trusted-server-core/src/request_signing/mod.rs`, lines 20 and 21). The changelog entry for the two store id settings says that only the removed key rotation routes read them (`CHANGELOG.md`, line 61), where on row 12 the checks that both were filled in read them too (`crates/trusted-server-core/src/auction/plan.rs`, lines 725 to 731, and `crates/trusted-server-core/src/config.rs`, lines 458 to 464). The changelog entry for the three Edge Cookie routes names `GET /_ts/data` after all three without saying that the page covers neither lookup by id nor the cookie echo (`CHANGELOG.md`, line 63). 6. Should Axum run the closed path check before the modules' request preparers, as the other three adapters do? 7. Which order should the guides give for the switch and for a rollback? Section 4 says what each build does with the other's configuration, and nothing in the code or the guides gives a sequence. ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/13-removals` at `f735e0f85`. Issues found, four of them blocking. The four were that the table named `GET /_ts/data` as the replacement for `GET /_ts/admin/eids` when that page reads only the record of the caller's own `ts-ec` cookie, that the upgrade note said a `[[handlers]]` entry could gate `GET /_ts/debug/ja4` when the Fastly entry point answers that path before any middleware, that the spec said the commands tell a `404` from other failures without saying that only the test fake is known to print the wording they look for, and that the header named closed pull requests and a count of 19 checks where the open pull request shows 25. This revision states what the code does at that head for the table, the commands and their flags, the closed paths, the effects on a publisher's own site, the upgrade, the switch and the rollback, the tests and the acceptance, and leaves seven questions about the code under Open questions until the code changes, being the wording of a `404`, the replacing of a secret, the attestation endpoint beneath `/_ts/admin`, the missing tests, four stale texts, the order of the preparers on Axum and the order of the switch. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/13-removals` at `f735e0f85`. Approved, none blocking. This revision adds the changelog entry that still says only the removed key rotation routes read the two store ids to open question 5, says that the sandbox answer in the Fastly entry point exists only in a build with the `reusable-sandbox` feature, and confirms the check counts of the row 13 pull request and the fork's. ## Revision record @@ -136,3 +137,4 @@ Each of these is a question about the code. This spec records what the code does | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the unfinished callout removed, the table header and a row for each Edge Cookie route, what an operator loses, the commands and their flags and token scopes, the closed paths and what they leave open, the effects on a publisher's own site, five debugging aids, the upgrade with the switch and the rollback, the tests, the acceptance without the unlisted faults, the open questions and the review log. | +| 2026-10-16 | Fixes from review round 2. Open question 5 names the changelog entry on the two store id settings and the checks on row 12 that also read them, the sandbox answer is stated to exist only in a build with the `reusable-sandbox` feature, and the check counts of 25 on the row 13 pull request and 19 on the fork's were read again and stand. | From 90281c1e01d4e5da6af200cb8e6f5a0fd4f6e11f Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 10:10:38 +0100 Subject: [PATCH 112/192] The device signals design takes its round 2 review The guide, changelog and score claims now match the code. --- .../specs/2026-10-12-device-signals-design.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-device-signals-design.md b/docs/superpowers/specs/2026-10-12-device-signals-design.md index 7289794b4f..f0c9dddd0b 100644 --- a/docs/superpowers/specs/2026-10-12-device-signals-design.md +++ b/docs/superpowers/specs/2026-10-12-device-signals-design.md @@ -17,11 +17,11 @@ Paths and line numbers in this document are those of `b680d0780`, the head of IA ## 2. Design -Two answers a device module may give beyond the coarse browser-or-bot signals the Edge Cookie gate uses. No code in this repository reads the first, the score. When a module gives the second, the attributes, core sends them in the bid request to every demand source. +Two answers a device module may give beyond the coarse browser-or-bot signals the Edge Cookie gate uses. No production code in this repository reads the first, the score. When a module gives the second, the attributes, core sends them in the bid request to every demand source. **The score.** `DeviceSignals::human_confidence` says how confident the module is that a person is looking at the page (`crates/trusted-server-core/src/ec/device.rs`, lines 115 and 123). `High` means high confidence that a person is viewing the page, `Medium` means the evidence is mixed and the request cannot be placed either way, `Low` means almost certainly no person is directly involved in viewing the content, and `Unknown` means not established. A module that scores the request writes the field in the signals that `detect` returns, and every other module leaves it `Unknown`, which is the default. `Unknown` is kept apart from `Low` because a request alone cannot establish a person, so a first-time visitor is never called a machine. The built-in module, the Fastly module and `DeviceSignals::unknown` all leave the score `Unknown` (lines 169 and 203). -Only the Fastly adapter calls `detect` in production, once for each request at its entry point (`crates/trusted-server-adapter-fastly/src/lib.rs`, lines 590 and 1089), so the score can only be written on Fastly. The Axum, Cloudflare and Spin adapters create no device signals at all. No row of the stack through row 50 adds a reader of the score. The module context, which holds what core hands a module when core calls the module, passes `&DeviceSignals` to a module that names the type, so a module could read the score. No module in this repository reads the score. The score is not stored in the identity graph row, the record kept for an Edge Cookie, because `to_kv_device` (line 209) copies five signals into the row, being `is_mobile`, `ja4_class`, `platform_class`, `h2_fp_hash` and `known_browser`. `looks_like_browser` is not copied either. On Fastly, `looks_like_browser` gates the Edge Cookie writes as it did (`crates/trusted-server-adapter-fastly/src/app.rs`, lines 643 to 688). The redirect middlewares that would treat `Low`, `Medium` and `High` differently are kept out of this change so that the set of changes does not grow. +Only the Fastly adapter calls `detect` in production, once for each request at its entry point (`crates/trusted-server-adapter-fastly/src/lib.rs`, lines 590 and 1089), so the score can only be written on Fastly. The Axum, Cloudflare and Spin adapters create no device signals at all. No production code reads the score, and no row of the stack through row 50 adds a production reader. Two tests read it, being `a_request_alone_establishes_no_person`, which reads it at line 912 of `device.rs`, and the Axum test of the seam probe, which reads it at line 971 of `crates/trusted-server-adapter-axum/tests/seam_probe.rs`. The module context, which holds what core hands a module when core calls the module, passes `&DeviceSignals` to a module that names the type, so a module could read the score, and no module in this repository does. The score is not stored in the identity graph row, the record kept for an Edge Cookie, because `to_kv_device` (line 209) copies five signals into the row, being `is_mobile`, `ja4_class`, `platform_class`, `h2_fp_hash` and `known_browser`. `looks_like_browser` is not copied either. On Fastly, `looks_like_browser` gates the Edge Cookie writes as it did (`crates/trusted-server-adapter-fastly/src/app.rs`, lines 643 to 688). The redirect middlewares that would treat `Low`, `Medium` and `High` differently are kept out of this change so that the set of changes does not grow. **The attributes.** `DeviceAttributes` is what a bidder is told about the device, being the OpenRTB device type, make, model, operating system and version, and screen width and height (`device.rs`, lines 41 to 65). Every field is optional because every source is partial. The device type and the two screen sizes are `i32`, a 32-bit whole number, and the other four fields are strings. `DeviceModule::advertising_attributes` gives the module's answer, and the default answers nothing, so no existing module changes (line 273). What core does with the answer, in the order it happens: @@ -51,7 +51,7 @@ The call is the same kind of call as the one `detect` is handed, but the call ca ## 4. What changes for a deployment -**Settings.** No setting is new. `[device] module` already selects the device module. A deployment that leaves the setting unset, or writes `builtin` or `fastly`, sends the bid requests it sent before, because core makes no call. Row 49 renames the `fastly` value to `platform`. A deployment that writes the name of a module that a crate outside core supplies has that module asked at the three points, so the module's attributes reach the device object of every bid request to every demand source as soon as the module is selected. No setting limits the attributes to one demand source or turns the attributes off while the module stays selected. The configuration guide (`docs/guide/configuration.md`, line 860) and the integration guide (`docs/guide/integration-guide.md`, line 647) describe both answers. +**Settings.** No setting is new. `[device] module` already selects the device module. A deployment that leaves the setting unset, or writes `builtin` or `fastly`, sends the bid requests it sent before, because core makes no call. Row 49 renames the `fastly` value to `platform`. A deployment that writes the name of a module that a crate outside core supplies has that module asked at the three points, so the module's attributes reach the device object of every bid request to every demand source as soon as the module is selected. No setting limits the attributes to one demand source or turns the attributes off while the module stays selected. The configuration guide (`docs/guide/configuration.md`, line 860) describes both answers. The integration guide (`docs/guide/integration-guide.md`, line 647) describes only `DeviceModule::advertising_attributes` and does not mention the score. **Adapters.** All four adapters put the selected device module into each request's services and call all three handlers, so the attributes work on all four. Only Fastly calls `detect`, so only Fastly can write the score. Only the Axum adapter has a test of the new answers. @@ -62,12 +62,12 @@ The call is the same kind of call as the one `detect` is handed, but the call ca | Fastly | `crates/trusted-server-adapter-fastly/src/app.rs`, line 484 | Yes, `crates/trusted-server-adapter-fastly/src/lib.rs`, line 1089 | None | | Spin | `crates/trusted-server-adapter-spin/src/app.rs`, line 308 | No | None | -**What breaks for code.** `DeviceSignals` and `DeviceInfo` each gain a public field, `human_confidence` and `attributes`, and both are built with struct literals, so a literal of either that does not name the new field stops compiling. In this repository that meant editing nine files, six in core (`auction/formats.rs`, `auction/routing.rs`, `auction/telemetry.rs`, `auction/test_support.rs`, `integrations/registry.rs` and `publisher.rs`) and three outside core (the demonstration ad server, the Prebid Server crate and the seam probe). A `DeviceModule` written before this change still compiles, because `advertising_attributes` has a default. The changelog lists the change under Added and says nothing of the two new fields (`CHANGELOG.md`, line 90). +**What breaks for code.** `DeviceSignals` and `DeviceInfo` each gain a public field, `human_confidence` and `attributes`, and both are built with struct literals, so a literal of either that does not name the new field stops compiling. In this repository that meant editing nine files, six in core (`auction/formats.rs`, `auction/routing.rs`, `auction/telemetry.rs`, `auction/test_support.rs`, `integrations/registry.rs` and `publisher.rs`) and three outside core (the demonstration ad server, the Prebid Server crate and the seam probe). A `DeviceModule` written before this change still compiles, because `advertising_attributes` has a default. The changelog lists the change under Added and names `DeviceSignals::human_confidence` and `DeviceModule::advertising_attributes`, but it does not name `DeviceInfo::attributes` and has no note that a struct literal of either type stops compiling (`CHANGELOG.md`, line 90). ## 5. What is not in this change - A module that scores a person or resolves a device. The stock build's two device modules, the built-in one and the Fastly one, read the User-Agent and the host's signals, and answer `Unknown` for the score and nothing for the attributes. The seam probe's module is the one module that answers either. -- A reader of the score. No code reads the score (section 2). +- A reader of the score. No production code reads the score (section 2). - A check of a module's values, a time limit on the call, or a change to what the call carries (section 7). - A setting. @@ -105,11 +105,12 @@ These are for the reviewer. This revision records what the code does and decides 4. **What the call carries.** Should the call carry the consent signals and the resolved permissions, which the handlers hold, and should the attributes pass the permission gate that EIDs pass? Today the call carries every header and the client address and neither of the two, and no device module can ask for a permission because the registry refuses one that declares a permission. The consent gate in front of the call does not read the Global Privacy Control signal (section 3). 5. **Two comments.** The comment on `DeviceAttributes` says that merging the two answers would put advertising fields into "a type the identity path serialises" (`device.rs`, lines 34 and 35), but `DeviceSignals` derives no `Serialize` (line 84) and the identity path writes `KvDevice`. The comment on `DeviceInfo::attributes` says that a stored request from before the field existed "still reads" (`types.rs`, lines 119 to 121), but a search of the tree finds no code that deserializes an `AuctionRequest`. Should the comments change, or does a reader of stored requests exist that the search missed? 6. **Where the score belongs.** Section 2 keeps the signals apart from the attributes by what each answers and by whether core stores each. The score answers a question about a person and not about a browser, and core stores the score nowhere, so the score matches the attributes on both counts, yet the score sits in the signals that `detect` returns. Neither the code nor IABTechLab/trusted-server#1286 says why. What is the reason for placing the score with the signals? -7. **The guide and the changelog.** The integration guide says that `advertising_attributes` is handed "the same call as `detect`" (`docs/guide/integration-guide.md`, line 647), which holds for the kind of call and not for what the call carries (section 3). The changelog lists the change under Added and says nothing of the two new public fields that stop struct literals compiling (`CHANGELOG.md`, line 90). Should either change? +7. **The guide and the changelog.** The integration guide says that `advertising_attributes` is handed "the same call as `detect`" (`docs/guide/integration-guide.md`, line 647), which holds for the kind of call and not for what the call carries (section 3). The changelog lists the change under Added and names `DeviceSignals::human_confidence` and `DeviceModule::advertising_attributes`, but it does not name `DeviceInfo::attributes` and gives no breaking-change note for the two new public fields that stop struct literals compiling (`CHANGELOG.md`, line 90). Should either change? ## Review log 1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/28-device-signals` at `b680d0780`. Issues found, five of them blocking. The blocking issues were that the spec described a device type name mapping that the code no longer has, that the spec named two places where core asks the module when the code has three, that the spec's acceptance section claimed a proof by the Axum test and a per-commit record that neither pull request holds, that the spec said the module is handed the call that `detect` is handed and left out that the module sees every header and that the attributes go under no permission check, and that the spec's header pointed at closed pull requests and said Proposed. This revision states what the code does for each, adds the consent and privacy, deployment, adapter and untested-path sections, and leaves seven questions in section 7 for code changes, being the missing test of the three points, the unchecked values, the missing time limit, what the call carries, two comments, where the score belongs, and the guide and the changelog. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/28-device-signals` at `b680d0780`. Issues found, two of them blocking. This revision says that the integration guide describes only the attributes, that the changelog names the score and `DeviceModule::advertising_attributes` but not `DeviceInfo::attributes` and gives no breaking-change note, and that no production code reads the score although two tests do, and it confirms the pull request facts in the header and the acceptance. ## Revision record @@ -117,3 +118,4 @@ These are for the reviewer. This revision records what the code does and decides | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header and the pull requests the header names, the removed type name mapping, the three places core asks, what the call carries, the consent and privacy section, what changes for a deployment and for code, the adapters, the acceptance without the process claims, the untested paths, the open questions and the review log. | +| 2026-10-16 | Fixes from review round 2. What the integration guide describes, what the changelog entry names and leaves out, and the statements that no code reads the score, which now say no production code does. The pull request facts in the header and the acceptance were read against the pull requests and stand. | From 1b509096e0810c8001a468d9bd8ddaff76a1fd71 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 12:45:26 +0100 Subject: [PATCH 113/192] The settings and consumers design takes its round 1 review The declaration types can now say what a consumer is, including core code and an adapter part, and what a setting must be for the consumer to need it. Each adapter lists its parts in a parts.toml in its own crate, read at startup and by ts config validate and push with --adapter, so core and the tool name no platform. The fourteen smaller findings and the host device case are applied. --- ...026-10-16-settings-and-consumers-design.md | 224 +++++++++++++----- 1 file changed, 161 insertions(+), 63 deletions(-) diff --git a/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md b/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md index 67757c6ba5..4a4f9b22a4 100644 --- a/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md +++ b/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md @@ -2,20 +2,20 @@ **Status:** Proposed, 2026-10-16 **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-10-12-page-middleware-design.md` (row 10, entries that name middleware), `2026-10-12-shared-vendor-tables-design.md` (row 26), `2026-10-13-middleware-settings-design.md` (row 38), `2026-10-14-no-modules-crate-design.md` (row 43, each adapter lists the modules it links) and `2026-10-15-module-cache-declarations-design.md` (row 32), all in IABTechLab/trusted-server#1342, which holds every design document of the stack +**Related specs:** `2026-10-12-page-middleware-design.md` (row 10, entries that name middleware), `2026-10-12-analytics-module-design.md` (row 16, telemetry as a module `[analytics]` selects), `2026-10-12-inmobi-choice-module-design.md` (row 18, the InMobi consent prompt), `2026-10-12-ads-txt-sellers-design.md` (row 20, the ads.txt sellers), `2026-10-12-robots-txt-middleware-design.md` (row 25, robots.txt as a middleware), `2026-10-12-shared-vendor-tables-design.md` (row 26), `2026-10-15-module-cache-declarations-design.md` (row 32), `2026-10-13-middleware-settings-design.md` (row 38), `2026-10-14-no-modules-crate-design.md` (row 43, each adapter lists the modules it links) and `2026-10-14-51degrees-client-hints-design.md` (row 44, the 51Degrees client hints), all in the pull request that holds every design document of the stack (IABTechLab/trusted-server#1342) **Related PRs:** none yet -**Issues:** IABTechLab/trusted-server#1403 (settings without a consumer and consumers without settings), with the related IABTechLab/trusted-server#1125, #1132, #1201, #1097, #1142, #1148, #1134, #1102, #1140, #1141 and #1139 +**Issues:** Settings without a consumer and consumers without settings (IABTechLab/trusted-server#1403), whose body lists the related issues of the same shape **Last updated:** 2026-10-16 > **Why this spec exists.** A settings section and the consumer that acts on it are checked against each other by a dozen separate checks that run at different times and stop at the first gap, and several pairs are not checked at all, so this spec proposes one declaration model in core and one check that reports every gap at startup and at push. -Line references are at `3aaf720c0`, the head of `split/50-cookies-by-path`, which is the head of IABTechLab/trusted-server#1400 and the top of the stack. A file named without a folder is in `crates/trusted-server-core/src`, and `registry.rs` and `mod.rs` are `integrations/registry.rs` and `integrations/mod.rs`. +Line references are at `3aaf720c0`, the head of `split/50-cookies-by-path`, which is the head of the cookies by path change (IABTechLab/trusted-server#1400) and the top of the stack. A file named without a folder is in `crates/trusted-server-core/src`, and `registry.rs` and `mod.rs` are `integrations/registry.rs` and `integrations/mod.rs`. A file named with an adapter is in that adapter's `src` folder, so Fastly `app.rs` is `crates/trusted-server-adapter-fastly/src/app.rs`. Every other file is named in full. ## 1. Problem Four words are used as the code uses them. A builder (`IntegrationBuilder`, `mod.rs` line 452) is what a crate hands to core to say what the crate offers. The registry is what core builds at startup from every builder (`IntegrationRegistry::with_plan_and_registrations`, `registry.rs` lines 2033 to 2400). Startup is an adapter loading the settings and building its state, which each adapter does through that function (Fastly `app.rs` line 360, Axum `app.rs` line 240, Cloudflare `app.rs` line 295, Spin `app.rs` line 219). Push is `ts config validate`, `ts config diff` and `ts config push`, which validate through `validate_settings_for_deploy_with` (`config.rs` lines 311 to 341). -A consumer is whatever acts on a settings section, being a middleware an entry names, a module a section selects, a request filter, a page request preparer or a part of the adapter. Many features need both halves, and today the two halves are matched in four different ways. +A consumer is whatever acts on a settings section, being a middleware an entry names, a module a section selects, a request filter, a page request preparer, code in core or a part of the adapter. Many features need both halves, and today the two halves are matched in four different ways. **Checks in the registry, at startup only.** Push builds no registry. `validate_phase_entries` checks the shape of each entry and says that whether a name is supplied "is only knowable where the registry is built" (`settings.rs` lines 4776 to 4791), and the section selection check is left out of push deliberately, "because a vendor crate the CLI never links may supply the name" (`registry.rs` lines 2163 to 2168). The registry's checks are these. @@ -24,20 +24,20 @@ A consumer is whatever acts on a settings section, being a middleware an entry n - A middleware no entry names is a warning, that it "changes no page" (lines 1724 to 1731). - A `[shared.]` table no builder reads is refused (lines 2044 to 2069). - A section selecting a module no builder supplies is refused, with a note naming absent shared tables (lines 1774 to 1825 and 1162 to 1178). -- A geo, Edge Cookie, device or analytics module that runs and is not selected is a warning (lines 1199 to 1212). +- A geo, Edge Cookie, device, prefix or analytics module on offer that its section does not select is a warning (lines 1199 to 1212, which the prefix selection calls at line 1400 as the other four do). -**Checks a feature wrote for itself.** The InMobi prompt refuses `[cmp]` selecting it with no `text/html` entry naming it, whatever the entry's `path` (`crates/cmp/inmobi/src/lib.rs` lines 141 to 158). The ads.txt sellers do the same for `text/plain` (`crates/ads-txt/iabtl/src/lib.rs` lines 152 to 167). Core's robots.txt check requires `path = "/robots.txt"` exactly (`settings.rs` lines 4414 to 4462). Each of the three runs at push and at startup, because a builder's validate function runs at both (`config.rs` lines 394 to 406) and `prepare_runtime` runs at both (`config.rs` line 323 and `settings.rs` lines 4357 to 4406). The cache table check is the one core check that runs at push as well (`mod.rs` lines 1008 to 1025, called from `config.rs` line 330). +**Checks a feature wrote for itself.** The InMobi prompt refuses `[cmp]` selecting it with no `text/html` entry naming it, whatever the entry's `path` (`crates/cmp/inmobi/src/lib.rs` lines 141 to 158). The ads.txt sellers do the same for `text/plain` (`crates/ads-txt/iabtl/src/lib.rs` lines 152 to 167). Core's robots.txt check requires `path = "/robots.txt"` exactly (`settings.rs` lines 4414 to 4462). Each of the three runs at push and at startup, because a builder's validate function runs at both (`config.rs` lines 394 to 406) and `prepare_runtime` runs at both (`config.rs` line 323 at push, and `settings.rs` line 4290 as the settings load). Core's cache table check runs at both as well (`mod.rs` lines 1008 to 1025, called from `config.rs` line 330 at push and from `mod.rs` line 1041 as an adapter builds its state). **No check.** These pairs are not matched at all. -- The 51Degrees client hints crate registers a request filter and a middleware whenever `[shared.51degrees]` is present (`crates/middleware/51degrees/src/lib.rs` lines 110 to 132). Core runs every registered filter (`registry.rs` lines 2640 to 2674), so with no entry naming the middleware the filter still asks the service on every page request (`lib.rs` lines 352 to 381), and a request carrying the filter's state skips the shared page template and is sent `no-store, private` (`publisher.rs` lines 5536 to 5540 and 6396 to 6402). +- The 51Degrees client hints crate registers a request filter and a middleware whenever `[shared.51degrees]` is present (`crates/middleware/51degrees/src/lib.rs` lines 110 to 132). Core runs every registered filter (`registry.rs` lines 2640 to 2674), so on Fastly, with no entry naming the middleware, the filter still asks the service on every page request (`crates/middleware/51degrees/src/lib.rs` lines 352 to 381). - The DataDome page tag writes nothing when `client_side_key` is blank (`crates/bot-protection/datadome/src/lib.rs` lines 1016 to 1026), and the startup checks look only at the tag's URL (lines 509 to 511), so an entry naming `bot-protection.datadome.tag` with no key is accepted silently. -- Only the Fastly adapter runs request filters (Fastly `app.rs` lines 836 to 867), dispatches asset routes (lines 1191 to 1194 and 1310 to 1313), reads the analytics sink (lines 389 to 392) and configures origin certificates from `certificate_check` (Fastly `backend.rs` lines 492 to 496). The Axum adapter's origin client takes no certificate setting (Axum `platform.rs` lines 321 to 333), and the Cloudflare and Spin adapters use the setting only in a backend's name (Cloudflare `platform.rs` lines 70 to 78, Spin `platform.rs` lines 92 to 100). With `[geo] module = "platform"` the registry leaves the adapter's own lookup in place (`registry.rs` lines 1480 to 1482), and the Axum and Spin lookups answer no location for every request (Axum `platform.rs` lines 210 to 218, Spin `platform.rs` lines 809 to 817). Each of these is accepted on the adapters without the part, with no log. -- `cluster_trust_threshold` and `cluster_recheck_secs` in `[ec]` are read into the settings (`settings.rs` lines 783 to 795) and read by no crate, which IABTechLab/trusted-server#1097 records. +- Only the Fastly adapter runs request filters (Fastly `app.rs` lines 836 to 867), dispatches asset routes (lines 1191 to 1194 and 1310 to 1313), reads the analytics sink (lines 389 to 392) and configures origin certificates from `certificate_check` (Fastly `backend.rs` lines 492 to 496). The Axum adapter's origin client takes no certificate setting (Axum `platform.rs` lines 321 to 333), and the Cloudflare and Spin adapters use the setting only in a backend's name (Cloudflare `platform.rs` lines 70 to 78, Spin `platform.rs` lines 92 to 100). With `[geo] module = "platform"` the registry leaves the adapter's own lookup in place (`registry.rs` lines 1480 to 1482), and the Axum and Spin lookups answer no location for every request (Axum `platform.rs` lines 210 to 218, Spin `platform.rs` lines 809 to 817). With `[device] module = "platform"` the registry supplies no module either (`registry.rs` line 1323), and only the Fastly adapter builds a host device module (Fastly `lib.rs` lines 1256 to 1263), from the one host device crate, which only the Fastly adapter links (`crates/trusted-server-adapter-fastly/Cargo.toml` line 65, `crates/device/fastly/src/lib.rs` line 69). Each of these is accepted on the adapters without the part, with no log. +- `cluster_trust_threshold` and `cluster_recheck_secs` in `[ec]` are read into the settings (`settings.rs` lines 783 to 795) and read by no crate, which the issue on the two unread Edge Cookie settings records (IABTechLab/trusted-server#1097). -**One gap at a time.** Every check returns its first error (for example `registry.rs` lines 1679, 1716, 1815 and 2063), so a document with two gaps fails twice before it starts. +**One gap at a time.** The first check that fails stops the start. Most checks also stop at their first gap (for example `registry.rs` lines 1679, 1716 and 1815), and the shared table check, which names every unread table in one error (`registry.rs` lines 2046 to 2066), still stops the start before any later check runs. So a document with two gaps of different kinds fails twice before it starts. -**One-off notices.** Where a setting is known to do nothing, the code that knows says so in its own way, being the Fastly warning when `certificate_check` is off (Fastly `app.rs` lines 484 to 490) and the DataDome warning for its reserved GraphQL flag (`lib.rs` lines 513 to 515). +**One-off notices.** Where a setting is known to do nothing, the code that knows says so in its own way, as the DataDome warning for its reserved GraphQL flag does (`crates/bot-protection/datadome/src/lib.rs` lines 513 to 515). The Fastly warning when `certificate_check` is off (Fastly `app.rs` lines 484 to 490) is not one of these, because Fastly honors the setting (Fastly `backend.rs` lines 492 to 496) and the warning says that certificate checks are off. ## 2. Design @@ -50,13 +50,14 @@ One model in core, in four points. ### 2.1 The declaration types in core -Core already holds one declaration a builder makes about a consumer, `SuppliedMiddleware`, which names a middleware, its phase and its media type (`mod.rs` lines 376 to 385), and the registry already checks that declaration against the middleware the builder really supplies (`check_described_middleware`, `registry.rs` lines 1591 to 1630). The new types extend that. They name no vendor, standard or platform, and every name in them is a string the declaring crate supplies. +Core already holds one declaration a builder makes about a consumer, `SuppliedMiddleware`, which names a middleware, its phase and its media type (`mod.rs` lines 376 to 385), and the registry already checks that declaration against the middleware the builder really supplies (`check_described_middleware`, `registry.rs` lines 1591 to 1630). The new types extend that. They name no vendor, standard or platform. Every name in them is a string the declaring crate supplies, apart from the adapter parts, which core names by what each part does. ```rust -/// What one consumer a builder supplies needs before it acts. +/// One consumer a builder or core supplies, and what it needs before it acts. pub struct ConsumerNeeds { pub consumer: Consumer, - /// The settings the consumer reads, each required or optional. + /// The settings the consumer reads, besides the selection or the shared + /// table that runs its builder. pub settings: &'static [SettingsNeed], /// What places the consumer in a deployment. pub placed_by: Placement, @@ -67,101 +68,198 @@ pub struct ConsumerNeeds { pub enum Consumer { /// A middleware, by the name an entry writes. Middleware(&'static str), - /// A module a section selects, by its full name. + /// A module, by the name a section selects it by. Module(&'static str), - /// A request filter or a page request preparer that only feeds the - /// named middleware, so it runs only where that middleware is placed. - Feeds(&'static str), + /// A request filter, by the name it is registered with. + RequestFilter(&'static str), + /// A page request preparer, by the name it is registered with. + Preparer(&'static str), + /// Code in core that acts on a section of core's own, by the section. + Core(&'static str), + /// A part of the adapter that acts on a setting core reads. + AdapterPart(AdapterPart), + /// Nothing. The setting is read into the settings and acted on by no + /// code, and is kept so that documents carrying it still load. + Nothing, } pub struct SettingsNeed { - /// The table, written as in the document, such as `shared.`. + /// The table, written as in the document, such as `shared.51degrees`, + /// `proxy` or `bot-protection.datadome`. pub table: &'static str, - /// Keys inside the table the consumer cannot act without. - pub keys: &'static [&'static str], + /// What the table, as written, has to say. + pub condition: Condition, + /// Whether the consumer cannot act without it. A setting that is not + /// required only adjusts the consumer, so leaving it out is not a gap. pub required: bool, } +pub enum Condition { + /// The table is written, or for an array of tables, at least one is. + TableWritten, + /// The key is written, whatever its value. + KeyWritten(&'static str), + /// The key is written with this value. + KeyIs(&'static str, Literal), + /// The key is written and is not blank. + KeyNotBlank(&'static str), +} + +pub enum Literal { + Bool(bool), + Str(&'static str), +} + pub enum Placement { - /// An entry in the phase, for the media type, and on the path when one - /// is given. + /// An entry naming the middleware, for the media type its + /// `SuppliedMiddleware` gives, in `phase` when one is given or else in + /// any phase the middleware runs in, and with this `path` when one is + /// given. An entry naming it for another media type or path is refused. Entry { - phase: MiddlewarePhase, - media_type: &'static str, + phase: Option, path: Option<&'static str>, }, - /// The section's `module` or `modules` naming the consumer. - Section(&'static str), - /// Present whenever its settings are, such as an adapter part. - WithSettings, + /// The section that selects the module. + Section, + /// Wherever the named middleware is placed, for a request filter or a + /// preparer that only feeds it. + Feeds(&'static str), + /// Wherever the consumer runs, for one nothing else places, such as a + /// request filter that feeds no middleware, code in core or an adapter + /// part. + WhenRunning, + /// Wherever its crate is linked, whatever the settings select, for a + /// preparer that strips its module's reserved query or cookie. + WhenLinked, } -/// A capability an adapter may have, named by core in neutral terms. -pub struct AdapterPart(pub &'static str); +/// A part an adapter may have, named by what it does and never by a +/// platform. +pub enum AdapterPart { + /// Runs the registered request filters before routing. + RequestFilters, + /// Dispatches `[[proxy.asset_routes]]`. + AssetRoutes, + /// Hands an auction's rows to the sink `[analytics]` selects. + TelemetrySink, + /// Sets its origin client's certificate checks from `certificate_check`. + OriginCertificateSetting, + /// Answers `[geo] module = "platform"` with a host geo lookup. + HostGeoLookup, + /// Answers `[device] module = "platform"` with a host device module. + HostDeviceModule, +} ``` -Core names the parts it knows of as constants, for example running request filters, dispatching asset routes, reading the telemetry sink, setting origin certificate checks and a host geo lookup. A part is a capability and never a platform. +An `Entry` placement takes the media type from the middleware's `SuppliedMiddleware`, so the two declarations cannot disagree. Its own phase is needed because a middleware that runs in both phases may still have to be placed in one, as robots.txt has to be placed in `[[fetch]]` (`settings.rs` lines 4408 to 4413). + +A condition is data rather than a function such as `ModuleSecretSetting::in_use` (`mod.rs` lines 359 to 367), so a line of the report can say what the setting has to be. Like `in_use`, it is judged on the table as written (2.3). + +A consumer is matched to what a builder registers by its name. A request filter and a preparer have no name today, because a filter carries only its integration id (`registry.rs` lines 532 to 541), which every filter of one builder shares, and a preparer is a bare function (`mod.rs` lines 323 to 324). So `with_request_filter` (`registry.rs` lines 753 to 756) and `with_request_preparer` (`mod.rs` lines 597 to 600) each take the name the declaration uses, beside the filter or the function. ### 2.2 How each crate and each adapter declares -**A vendor crate** declares on its builder, next to the declarations a builder already makes (`with_shared_settings`, `with_secret_settings`, `with_middleware` and `with_cache_features`, `mod.rs` lines 655 to 741). A new `with_needs(&'static [ConsumerNeeds])` takes one entry for each consumer the builder supplies. The 51Degrees client hints builder (`crates/middleware/51degrees/src/lib.rs` lines 94 to 100) would declare its middleware as needing `[shared.51degrees]` with `endpoint` and `license_key`, placed by a `[[serve]]` entry for `text/html`, and its request filter as `Feeds("51degrees.client_hints")` needing the part that runs request filters. Each vendor declares in its own folder, so core gains no vendor name. +**A vendor crate** declares on its builder, next to the declarations a builder already makes (`with_shared_settings`, `with_secret_settings`, `with_middleware` and `with_cache_features`, `mod.rs` lines 655 to 741). A new `with_needs(&'static [ConsumerNeeds])` takes one entry for each middleware, request filter and preparer the builder supplies. A module gets an entry only when it needs a setting of its own or a part, because `with_module_name` already declares the module and the section that selects it (`mod.rs` lines 684 to 698 and 772 to 789). Each vendor declares in its own folder, so core gains no vendor name. + +**Core** declares its own consumers in the same types. Its robots.txt middleware is one. A section that core reads and the adapter acts on is declared too. Its consumer is code in core where core acts on the section as well. Core checks the asset routes as the settings load and matches a request's path to one (`settings.rs` line 2803), and the adapter then dispatches the request. Its consumer is an adapter part where core only carries the value, as for `certificate_check`. A setting that nothing reads, kept so that old documents still load, is declared with `Nothing`, and so is a vendor's reserved setting. + +The declarations for the cases in the issue this design answers (IABTechLab/trusted-server#1403), with the host device case added, would be these. + +| Case | Declared in | Consumer | Settings | Placement | Parts | +| ---------------------------- | -------------------------------- | --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | ---------------- | +| 51Degrees client hints | `crates/middleware/51degrees` | `Middleware("51degrees.client_hints")` | `endpoint` and `license_key` in `shared.51degrees` not blank, required, and `middleware.51degrees.client_hints` written, optional (2.5) | `Entry` | none | +| 51Degrees client hints | `crates/middleware/51degrees` | `RequestFilter("51degrees.client_hints")` | none of its own | `Feeds("51degrees.client_hints")` | `RequestFilters` | +| DataDome protection | `crates/bot-protection/datadome` | `RequestFilter("bot-protection.datadome")` | `enable_protection = true` in `bot-protection.datadome`, required | `WhenRunning` | `RequestFilters` | +| DataDome page tag | `crates/bot-protection/datadome` | `Middleware("bot-protection.datadome.tag")` | `client_side_key` in `bot-protection.datadome` not blank, required | `Entry` | none | +| DataDome GraphQL flag | `crates/bot-protection/datadome` | `Nothing` | `enable_graphql_support = true` in `bot-protection.datadome` | not used | none | +| InMobi consent prompt | `crates/cmp/inmobi` | `Middleware("cmp.inmobi")` | `script_url` in `cmp.inmobi` not blank, required | `Entry` | none | +| ads.txt sellers | `crates/ads-txt/iabtl` | `Middleware("ads-txt.iabtl")` | `lines` in `ads-txt.iabtl` written, required | `Entry` (open question 3) | none | +| robots.txt | core | `Middleware("robots_txt")` | `robots-txt` written, required | `Entry` in `[[fetch]]` on `/robots.txt` | none | +| Middleware settings | the middleware's crate | that middleware | `middleware.` written, optional | that middleware's | none | +| Shared vendor tables | the vendor's crate | each consumer the vendor's builder supplies | what each consumer needs in `shared.` | that consumer's | that consumer's | +| Section selections | the module's crate | the module, declared by `with_module_name` | none of its own | `Section` | none | +| Loading cache tables | the module's crate | the module, declared by `with_cache_features` | `cache.` written, optional (2.5) | the module's | none | +| Auction analytics | `crates/analytics/tinybird` | `Module("analytics.tinybird")` | none of its own | `Section` | `TelemetrySink` | +| Asset routes | core | `Core("proxy.asset_routes")` | `proxy.asset_routes` written, required | `WhenRunning` | `AssetRoutes` | +| Origin certificate check | core | `AdapterPart(OriginCertificateSetting)` | `certificate_check = false` in `proxy`, required | `WhenRunning` | its own | +| Host geo lookup | core | `AdapterPart(HostGeoLookup)` | `module = "platform"` in `geo`, required | `WhenRunning` | its own | +| Host device module | core | `AdapterPart(HostDeviceModule)` | `module = "platform"` in `device`, required | `WhenRunning` | its own | +| Edge Cookie cluster settings | core | `Nothing` | each of `cluster_trust_threshold` and `cluster_recheck_secs` in `ec` written | not used | none | + +A builder declares every middleware, request filter and preparer it supplies, not only those in the table. The Google ad tag diagnostics builder, for example, declares its preparer as `Preparer("ad-tag.google.diagnostics")` placed by `WhenLinked` (`crates/ad-tag/google/src/diagnostics.rs` lines 46 to 58), because the preparer removes the reserved `ts_console` query and cookie whether or not the module runs (lines 317 to 320). + +**An adapter** declares the parts it has in `parts.toml` in its own crate, by the names core gives them in lower case with hyphens, for example `parts = ["host-geo-lookup"]`. The adapter reads the file as it is built (`include_str!`) and hands the parts to the check with its builders, before it builds the registry. The `ts` tool reads the same file at push (2.4), so startup and push cannot disagree about an adapter. The list cannot go in the adapter's block in `edgezero.toml`, because EdgeZero refuses a table it does not know in that block and a key it does not know in the block's `adapter` table (`crates/edgezero-core/src/manifest.rs` lines 814 to 861 in EdgeZero at `683202c66`, the revision the workspace `Cargo.toml` pins at lines 94 to 99), and the block's other tables hold EdgeZero's own build, command and logging settings. A list of adapters and their parts kept in core or in the tool would name every platform and repeat what each adapter declares. At `3aaf720c0` the four files would hold these parts. + +| Part | Fastly | Axum | Cloudflare | Spin | +| -------------------------- | ------------------------------------------------- | ---------------------------------- | ----------------------------------- | ---------------------------------- | +| `RequestFilters` | yes, `app.rs` lines 836 to 867 | no | no | no | +| `AssetRoutes` | yes, `app.rs` lines 1191 to 1194 and 1310 to 1313 | no | no | no | +| `TelemetrySink` | yes, `app.rs` lines 389 to 392 | no | no | no | +| `OriginCertificateSetting` | yes, `backend.rs` lines 492 to 496 | no, `platform.rs` lines 321 to 333 | no, `platform.rs` lines 70 to 78 | no, `platform.rs` lines 92 to 100 | +| `HostGeoLookup` | yes, `app.rs` line 621 | no, `platform.rs` lines 210 to 218 | yes, `platform.rs` lines 724 to 733 | no, `platform.rs` lines 809 to 817 | +| `HostDeviceModule` | yes, `lib.rs` lines 1256 to 1263 | no | no | no | -**A middleware or module core supplies** declares in core in the same types. The robots.txt middleware would declare `[robots-txt]` as required and a `[[fetch]]` entry for `text/plain` on `/robots.txt` as its placement, which is what `require_robots_txt_placed` checks by hand today (`settings.rs` lines 4414 to 4462). +### 2.3 The one check and the one report -**A section core reads directly** declares its consumer as a part. `[[proxy.asset_routes]]` would need the part that dispatches asset routes, `certificate_check = false` the part that sets origin certificate checks, and `[geo] module = "platform"` the host geo lookup. A field kept only so that old documents still load, as IABTechLab/trusted-server#1097 may decide for `cluster_recheck_secs`, is declared as having no consumer, and the report names it as accepted and unused. +The check runs once, before any builder builds. Every declaration is static and the check reads the document as written, so it needs nothing a build makes. Run after the builds, it would come too late, because the shared table refusal runs before any builder (`registry.rs` lines 2044 to 2069, ahead of the loop at line 2077) and a build can refuse first (line 2152, as the InMobi build does at `crates/cmp/inmobi/src/lib.rs` line 190). -**An adapter** declares the parts it has as one constant list in its own crate, which it hands to the registry build beside its builders. The registry build takes builders alone today (`registry.rs` lines 2033 to 2037), and each adapter already passes its own list of builders (for example Axum `app.rs` lines 109 to 145, from row 43), so the parts travel the same way. +The check reads each table as the document writes it, before serde fills a default, so a default can neither hide a missing key nor make an unwritten one look written. Module, shared and middleware tables are kept as written already, which is how `Settings::module_config` reads a module's table (`settings.rs` lines 4653 to 4680). Core's typed sections and a vendor's typed table fill defaults, for example for the two Edge Cookie cluster settings (`settings.rs` lines 786 and 794) and for DataDome's `client_side_key` (`crates/bot-protection/datadome/src/lib.rs` lines 342 to 343 and 456), so the load keeps the document's own tables beside the typed settings for the check to read. -### 2.3 The one check and the one report +A consumer runs when its builder runs, which is when a section selects its module, when its shared table is written, or always for core's own, and when every setting it requires holds. It is placed as its placement says, in the deployment as a whole, so a middleware is placed when any entry of the document names it, whatever the path of a given request. A gap is one of five kinds, and each line names the setting, the consumer, the missing half and what to add. -After every builder has registered, and before any check above returns, core goes through the declarations once. A gap is one of four kinds. +1. A written table or key that no linked consumer declares, or that only consumers which do not run declare, and an entry or a selection that names nothing a linked crate declares. A module's table that its section does not select is one, which core refuses today (`provider_table.rs` lines 564 to 567). +2. A placed consumer that does not run, naming the selection, shared table or required setting it lacks. +3. A consumer that runs and that nothing places. +4. A placed consumer that runs and needs a part the adapter lacks, naming the adapter and the part. An `AdapterPart` consumer needs its own part. +5. A written setting whose consumer is `Nothing`, which the line names as accepted and unused. -1. A settings section that no linked and running consumer declares, or that only consumers nothing places declare. -2. A placed consumer whose required settings are absent, with the table and keys named. -3. A consumer whose settings are present and which nothing places. Its feeding request filters and preparers do not run. -4. A section or a consumer that needs a part the adapter lacks, with the adapter and the part named. +A request filter or preparer placed by `Feeds` gets no line of kind 2 or 3 of its own, because the line for its middleware covers it, so one state gives one line. -Every gap is collected, and startup refuses with one report in which each line names the section, the consumer, the missing half and what to add. A document with two gaps gets both lines. The existing refusals in section 1 become lines of this report, with their wording kept where it already names what to add. +Each line has a level. Kinds 1, 2 and 4 refuse. Kind 3 refuses when a setting the consumer declares holds, because that setting then does nothing, as the InMobi, ads.txt and robots.txt checks refuse today. It warns when only its builder's selection or shared table runs it, as the warnings for a middleware no entry names and for a module on offer that its section does not select do today. Kind 5 warns, and so do the lines in 2.4 for a name or for parts that the tool cannot check at push. Startup and push refuse when any line refuses, and give every line either way, so a document with two gaps gets both lines. The existing refusals and warnings in section 1 become lines of this report, with their wording kept where it already names what to add. The registry's checks after the build stay only to catch a registration that contradicts its builder's declarations, as `check_described_middleware` does today. -A request filter or preparer declared with `Feeds` is run only when its middleware is placed. Preparers that sanitize a reserved query or cookie in a deployment that does not run their module are collected for every builder on purpose (`registry.rs` lines 2103 to 2108), so those declare no `Feeds` and keep running. +A request filter or preparer placed by `Feeds` is kept only where its middleware is placed, which the registry decides once as it is built, so it runs on every request wherever its middleware is placed, as filters run today. A preparer placed by `WhenLinked` is kept for every linked builder, as every preparer is collected today (`registry.rs` lines 2103 to 2108). ### 2.4 The same check at validate and at push -`TrustedServerAppConfig` validates through a trait that takes no arguments, so the tool registers the builders it validates against once, before validating (`config.rs` lines 87 to 106). The `ts` tool registers the builders it links (`crates/trusted-server-cli/src/app_config.rs` lines 35 to 81) at the start of every command (`run.rs` line 126). Push would run the check in 2.3 over those builders' declarations, so every gap a stock builder can cause is caught before the document is written to the config store. A selection that names a module the tool does not link is reported as a line saying that the name is not one this tool can check, rather than passed in silence. Adapter parts reach push the same way as the builders, registered once for the adapter the document is pushed for (open question 2). +`TrustedServerAppConfig` validates through a trait that takes no arguments, so the tool registers the builders it validates against once, before validating (`config.rs` lines 87 to 106). The `ts` tool registers the builders it links (`crates/trusted-server-cli/src/app_config.rs` lines 35 to 81) at the start of every command (`crates/trusted-server-cli/src/run.rs` line 126). Push would run the check in 2.3 over those builders' declarations, so every gap a linked builder can cause is caught before the document is written to the config store. A kind 1 line for a name that no crate the tool links declares, being a selection, an entry, a shared table, a middleware table or a cache table, warns at push and says that the tool cannot check the name, because a vendor crate the tool never links may supply it. That is why push leaves a selection unchecked on purpose today (`registry.rs` lines 2163 to 2168). The cache table check refuses such a name at push today (`config.rs` line 330), so this would change it (open question 2). + +The parts reach push from the adapter's own `parts.toml` (2.2). `ts config push` and `ts config diff` already take `--adapter` (the tool's tests parse it at `crates/trusted-server-cli/src/run.rs` lines 626 to 647), and `ts config validate` takes none (lines 785 to 799), so the tool adds an optional `--adapter` to `ts config validate`, beside the arguments it takes from EdgeZero (lines 4 to 7 and 75 to 76). Given an adapter, the tool finds the adapter's block in `edgezero.toml` through the manifest loader it already uses (`crates/trusted-server-cli/src/app_config.rs` line 12), reads `parts.toml` in the folder the block's `crate` key names (for example `edgezero.toml` line 56 for Axum), and registers the parts as it registers the builders, before EdgeZero validates. Neither core nor the tool names a platform, because the adapter's name comes from the command line and its folder from the manifest. With no adapter given, `ts config validate` checks everything but the parts, and a line that warns says so. A block with no `crate` key, or a folder with no `parts.toml`, gets the same warning, so the parts are never passed over in silence. ### 2.5 A consumer that runs on defaults -A consumer whose settings are optional declares `required: false`, and leaving its table out is then not a gap. The 51Degrees client hints middleware reads `[middleware.51degrees.client_hints]` and writes `Critical-CH` unless the table turns it off (`crates/middleware/51degrees/src/lib.rs` lines 191 to 202), so it declares that table optional. A loading cache with no `[cache.]` table loads through no cache at all (`platform/loading_cache.rs` lines 240 to 242 and 269 to 272), so a module that declares a cache says in its declaration whether running with no cache is intended, and the report names a missing cache table only where it is not. +A consumer whose setting is optional declares it with `required: false`, and leaving it out is then not a gap. The 51Degrees client hints middleware reads `[middleware.51degrees.client_hints]` and writes `Critical-CH` unless the table turns it off (`crates/middleware/51degrees/src/lib.rs` lines 191 to 202), so it declares that table optional. A module's `[cache.]` table is optional in the same way, because a declared cache with no table takes the one loading cache module the build links (`mod.rs` lines 1083 to 1104), and every adapter's stock list links one (for example Axum `app.rs` line 140). Only a build that links no loading cache module, or several, loads a cache with no table through no cache at all (`platform/loading_cache.rs` lines 240 to 242 and 269 to 272). Neither is a stock build, and this model leaves the case as it is. ## 3. What is not in this change -- **Unknown keys and integration IDs.** IABTechLab/trusted-server#1201 covers keys a table does not know, which serde refuses or drops before any consumer sees them. This model starts from tables that read correctly. -- **The health route and the startup failure response.** IABTechLab/trusted-server#1142 decides what an adapter answers once startup refuses, and this model only decides what startup refuses. -- **Adding the missing adapter parts.** Dispatching asset routes on every adapter (IABTechLab/trusted-server#1125) or honoring `certificate_check` on Axum (IABTechLab/trusted-server#1132) remain separate changes. This model makes their absence a named gap until then. -- **Deciding the Edge Cookie cluster settings.** IABTechLab/trusted-server#1097 decides whether they gain a reader or leave. +- **Unknown keys and integration IDs.** The issue on unknown integration IDs and fields (IABTechLab/trusted-server#1201) covers keys a table does not know, which serde refuses or drops before any consumer sees them. This model starts from tables that read correctly. +- **The health route and the startup failure response.** The issue on one health and startup failure contract across the adapters (IABTechLab/trusted-server#1142) decides what an adapter answers once startup refuses, and this model only decides what startup refuses. +- **Adding the missing adapter parts.** Dispatching asset routes on every adapter (IABTechLab/trusted-server#1125) or honoring `certificate_check` on Axum (IABTechLab/trusted-server#1132) remain separate changes. This model makes their absence a named gap until then, and an adapter that gains a part adds it to its `parts.toml`. +- **Deciding the Edge Cookie cluster settings.** This model declares them read by nothing, which is true today, and the issue on the two unread settings (IABTechLab/trusted-server#1097) decides whether they gain a reader, which then replaces the declaration, or leave with it. +- **Running a request filter only on the paths its middleware covers.** A filter that feeds a middleware runs on every request wherever the middleware is placed, as filters run today. +- **The Fastly warning when `certificate_check` is off.** It warns that certificate checks are off on an adapter that honors the setting (section 1), so it is not a notice that the setting does nothing, and it stays. ## 4. Acceptance -1. For each row of the cases table in IABTechLab/trusted-server#1403, one test with the settings and no consumer and, where the consumer cannot act without its settings, one test with the consumer and no settings. Each runs through the real settings load and state build of each adapter, as the Axum adapter's seam probe tests already build the adapter's routes from settings (`crates/trusted-server-adapter-axum/tests/seam_probe.rs` line 94), and through the Fastly, Cloudflare and Spin builds, not through the check called directly. +1. For each row of the cases table in the issue this design answers (IABTechLab/trusted-server#1403), and for the host device case in section 1, one test with the settings and no consumer and, where the consumer cannot act without its settings, one test with the consumer and no settings. Each runs through the real settings load and state build of each adapter, as the Axum adapter's tests already build the adapter's routes from settings (`crates/trusted-server-adapter-axum/tests/seam_probe.rs` line 94), and through the Fastly, Cloudflare and Spin builds, not through the check called directly. 2. Each of those tests is seen to fail with its declaration removed before the test is relied on. -3. A document with two gaps of different kinds gets both lines in one report. -4. `ts config validate`, given the builders the tool links, reports the same lines as startup for every gap a linked builder can cause. -5. A test over every builder each adapter links (for example Axum `app.rs` lines 109 to 145) finds a declaration for each middleware, module, request filter and page request preparer the builder registers, so a new consumer cannot leave one out. `check_described_middleware` (`registry.rs` lines 1591 to 1630) is the model for matching a declaration to what is registered. +3. A document with two gaps of different kinds gets both lines in one report, and a document whose only lines warn starts and is pushed. +4. `ts config validate --adapter ` and `ts config push --adapter `, given the builders the tool links and the parts in that adapter's `parts.toml`, report the same lines as that adapter's startup for every gap a linked builder or the adapter's parts can cause. `ts config validate` with no adapter reports every other gap, with one line that warns that the parts were not checked. +5. A test over every builder each adapter links (for example Axum `app.rs` lines 109 to 145) checks both ways. Every middleware, module, request filter and page request preparer a builder registers has a declaration of the same name, and every declaration is registered by its builder when the declaration's settings hold, so a declaration cannot outlive its consumer. `check_described_middleware` (`registry.rs` lines 1591 to 1630) is the model only for matching by name, because it checks a registered middleware against its description and skips a described one that the registration leaves out (lines 1596 to 1603). +6. On each adapter, for each part its `parts.toml` names, a test sees the part act, and for each part the file does not name, a test sees a document that needs the part refused, so the file cannot claim a part the adapter lacks. ## 5. Open questions -1. Should every gap refuse at once, or should the two kinds that only warn today, a middleware no entry names (`registry.rs` lines 1724 to 1731) and a module that runs and is not selected (lines 1199 to 1212), warn for one release first? A deployment that carries `[shared.51degrees]` with no entry starts today and would stop. -2. How does push learn which adapter a document is for, and where does push read that adapter's parts from, when the `ts` tool does not link the adapter crates? -3. Should a selection naming a module the tool does not link fail push, or be reported and allowed, as section 2.4 proposes? -4. Should the InMobi and ads.txt sellers placements carry a path, as robots.txt does, or should all three count any entry for the media type? -5. Should the Edge Cookie cluster settings be declared as having no consumer, or wait for the decision in IABTechLab/trusted-server#1097? +1. Should the lines that refuse under this model, where today there is only a warning or nothing at all, warn for one release first? They are the kind 3 lines where a setting the consumer declares holds, which today get the warning for a middleware no entry names (`registry.rs` lines 1724 to 1731), and the kind 2 and kind 4 lines for gaps that are accepted today with no log. A deployment that carries `[shared.51degrees]` with no entry starts today and would stop, and so would one on Axum with `[geo] module = "platform"`. +2. Should a name that no crate the tool links declares refuse at push, as the cache table check does today (`config.rs` line 330), or warn, as section 2.4 proposes and as push leaves a selection unchecked on purpose today (`registry.rs` lines 2163 to 2168)? Refusing catches a misspelt name before the document is written, and warning lets a deployment push, with the stock tool, names that its own crates supply. +3. Should the ads.txt sellers' placement require `path = "/ads.txt"`, as the robots.txt placement requires its path? An entry with no path appends the lines to every `text/plain` file it covers (`crates/ads-txt/iabtl/src/lib.rs` lines 254 to 262), and the module's check counts any `text/plain` entry today (lines 152 to 167), so the declaration keeps that placement until this is decided. ## Review log -No review rounds yet. +1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/50-cookies-by-path` at `3aaf720c0`. Issues found, two of them blocking. The declaration types could not say that a consumer is code in core, an adapter part, a request filter that feeds no middleware or a preparer that runs without its module, nor that a setting must have a given value or must not be blank, and push and validate had no way to see the parts of the adapter a document is for. This revision gives the types those consumers and conditions, has each adapter declare its parts in a `parts.toml` in its own crate that the tool reads for the adapter `--adapter` names, and applies the other findings. ## Revision record -| Date | Change | -| ---------- | ----------------------------------------------------------------------------------------------------------------- | -| 2026-10-16 | First draft, from the code at `3aaf720c0` and the issue that states the problem (IABTechLab/trusted-server#1403). | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 2026-10-16 | First draft, from the code at `3aaf720c0` and the issue that states the problem (IABTechLab/trusted-server#1403). | +| 2026-10-16 | Round 1 fixes. The types gain consumers for code in core, adapter parts and settings nothing reads, conditions on a key's value, named filters and preparers, and placements for a consumer nothing else places and for a preparer that runs whenever its crate is linked. Each adapter declares its parts in `parts.toml`, which push, diff and a new `ts config validate --adapter` read. Lines have levels, the check runs before any build on the settings as written, the host device case is added, and the claims on the push-time checks, the loading cache default and the one-off notices are corrected. | From d95c5120ee38c1569f345403898b851a1661fe3f Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 13:27:21 +0100 Subject: [PATCH 114/192] The settings and consumers design takes its round 2 review A consumer declares the tables it reads apart from the conditions it runs under, so a key at its default or a switch turned off gives no line. Core's doc comments name no vendor, a condition can hold for an absent key, core's own middleware takes its media type from what core already describes, and the root files are named with their place. --- ...026-10-16-settings-and-consumers-design.md | 107 ++++++++++-------- 1 file changed, 58 insertions(+), 49 deletions(-) diff --git a/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md b/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md index 4a4f9b22a4..5bd063fddb 100644 --- a/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md +++ b/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md @@ -9,7 +9,7 @@ > **Why this spec exists.** A settings section and the consumer that acts on it are checked against each other by a dozen separate checks that run at different times and stop at the first gap, and several pairs are not checked at all, so this spec proposes one declaration model in core and one check that reports every gap at startup and at push. -Line references are at `3aaf720c0`, the head of `split/50-cookies-by-path`, which is the head of the cookies by path change (IABTechLab/trusted-server#1400) and the top of the stack. A file named without a folder is in `crates/trusted-server-core/src`, and `registry.rs` and `mod.rs` are `integrations/registry.rs` and `integrations/mod.rs`. A file named with an adapter is in that adapter's `src` folder, so Fastly `app.rs` is `crates/trusted-server-adapter-fastly/src/app.rs`. Every other file is named in full. +Line references are at `3aaf720c0`, the head of `split/50-cookies-by-path`, which is the head of the cookies by path change (IABTechLab/trusted-server#1400) and the top of the stack. A file named without a folder is in `crates/trusted-server-core/src`, and `registry.rs` and `mod.rs` are `integrations/registry.rs` and `integrations/mod.rs`. A file named with an adapter is in that adapter's `src` folder, so Fastly `app.rs` is `crates/trusted-server-adapter-fastly/src/app.rs`. `edgezero.toml`, `trusted-server.example.toml` and the workspace `Cargo.toml` are at the repository root, and every other file is named in full. ## 1. Problem @@ -56,9 +56,12 @@ Core already holds one declaration a builder makes about a consumer, `SuppliedMi /// One consumer a builder or core supplies, and what it needs before it acts. pub struct ConsumerNeeds { pub consumer: Consumer, - /// The settings the consumer reads, besides the selection or the shared - /// table that runs its builder. - pub settings: &'static [SettingsNeed], + /// Tables the consumer reads, whatever they hold, besides the tables its + /// builder's own declarations name. + pub reads: &'static [&'static str], + /// What the settings, as written, have to say for the consumer to run, + /// besides the selection or the shared table that runs its builder. + pub runs_when: &'static [SettingsNeed], /// What places the consumer in a deployment. pub placed_by: Placement, /// The adapter parts the consumer cannot act without. @@ -78,20 +81,18 @@ pub enum Consumer { Core(&'static str), /// A part of the adapter that acts on a setting core reads. AdapterPart(AdapterPart), - /// Nothing. The setting is read into the settings and acted on by no - /// code, and is kept so that documents carrying it still load. + /// Nothing. Its `runs_when` names a setting that is read into the + /// settings and acted on by no code, and is kept so that documents + /// carrying it still load. Nothing, } +/// What one table, as written, has to say. pub struct SettingsNeed { - /// The table, written as in the document, such as `shared.51degrees`, - /// `proxy` or `bot-protection.datadome`. + /// The table, written as in the document, such as `shared.`, + /// `middleware.` or `proxy`. pub table: &'static str, - /// What the table, as written, has to say. pub condition: Condition, - /// Whether the consumer cannot act without it. A setting that is not - /// required only adjusts the consumer, so leaving it out is not a gap. - pub required: bool, } pub enum Condition { @@ -101,6 +102,8 @@ pub enum Condition { KeyWritten(&'static str), /// The key is written with this value. KeyIs(&'static str, Literal), + /// The key is absent, or written with another value. + KeyIsNot(&'static str, Literal), /// The key is written and is not blank. KeyNotBlank(&'static str), } @@ -111,10 +114,10 @@ pub enum Literal { } pub enum Placement { - /// An entry naming the middleware, for the media type its - /// `SuppliedMiddleware` gives, in `phase` when one is given or else in - /// any phase the middleware runs in, and with this `path` when one is - /// given. An entry naming it for another media type or path is refused. + /// An entry naming the middleware, for the media type its description + /// gives, in `phase` when one is given or else in any phase the + /// middleware runs in, and with this `path` when one is given. An entry + /// naming it for another media type or path is refused. Entry { phase: Option, path: Option<&'static str>, @@ -151,7 +154,9 @@ pub enum AdapterPart { } ``` -An `Entry` placement takes the media type from the middleware's `SuppliedMiddleware`, so the two declarations cannot disagree. Its own phase is needed because a middleware that runs in both phases may still have to be placed in one, as robots.txt has to be placed in `[[fetch]]` (`settings.rs` lines 4408 to 4413). +An `Entry` placement takes the media type from the middleware's description, so the two cannot disagree. A builder's middleware is described by its `SuppliedMiddleware` (`mod.rs` lines 376 to 385). Core's own middleware, which no builder supplies and which the registry adds itself (`registry.rs` lines 2376 to 2392), is described in `supplied_middleware` (`mod.rs` lines 939 to 952). An entry has a phase of its own because a middleware that runs in both phases may still have to be placed in one, as a middleware that makes a file the origin lacks has to be, since only a `[[fetch]]` entry makes one (`settings.rs` lines 4408 to 4413). + +Reading and running are declared apart. A consumer reads a table whatever the table's keys hold, and a key's value only decides, through `runs_when`, whether the consumer runs. So `certificate_check = true`, the default (`trusted-server.example.toml` lines 436 to 438), is a key in a table core reads, and it runs no adapter part. A module reads its own table, a builder its shared table and a module the tables of the caches it declares, as `with_module_name`, `with_shared_settings` and `with_cache_features` already say, and core reads its own sections. So `reads` lists only other tables, such as a middleware's `[middleware.]`. A condition is data rather than a function such as `ModuleSecretSetting::in_use` (`mod.rs` lines 359 to 367), so a line of the report can say what the setting has to be. Like `in_use`, it is judged on the table as written (2.3). @@ -159,32 +164,34 @@ A consumer is matched to what a builder registers by its name. A request filter ### 2.2 How each crate and each adapter declares -**A vendor crate** declares on its builder, next to the declarations a builder already makes (`with_shared_settings`, `with_secret_settings`, `with_middleware` and `with_cache_features`, `mod.rs` lines 655 to 741). A new `with_needs(&'static [ConsumerNeeds])` takes one entry for each middleware, request filter and preparer the builder supplies. A module gets an entry only when it needs a setting of its own or a part, because `with_module_name` already declares the module and the section that selects it (`mod.rs` lines 684 to 698 and 772 to 789). Each vendor declares in its own folder, so core gains no vendor name. +**A vendor crate** declares on its builder, next to the declarations a builder already makes (`with_shared_settings`, `with_secret_settings`, `with_middleware` and `with_cache_features`, `mod.rs` lines 655 to 741). A new `with_needs(&'static [ConsumerNeeds])` takes one entry for each middleware, request filter and preparer the builder supplies. A module gets an entry only when it runs under a condition of its own, reads a table besides its own or needs a part, because `with_module_name` already declares the module, and with it the module's table and the section that selects it (`mod.rs` lines 684 to 698 and 772 to 789). Each vendor declares in its own folder, so core gains no vendor name. **Core** declares its own consumers in the same types. Its robots.txt middleware is one. A section that core reads and the adapter acts on is declared too. Its consumer is code in core where core acts on the section as well. Core checks the asset routes as the settings load and matches a request's path to one (`settings.rs` line 2803), and the adapter then dispatches the request. Its consumer is an adapter part where core only carries the value, as for `certificate_check`. A setting that nothing reads, kept so that old documents still load, is declared with `Nothing`, and so is a vendor's reserved setting. The declarations for the cases in the issue this design answers (IABTechLab/trusted-server#1403), with the host device case added, would be these. -| Case | Declared in | Consumer | Settings | Placement | Parts | -| ---------------------------- | -------------------------------- | --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | ---------------- | -| 51Degrees client hints | `crates/middleware/51degrees` | `Middleware("51degrees.client_hints")` | `endpoint` and `license_key` in `shared.51degrees` not blank, required, and `middleware.51degrees.client_hints` written, optional (2.5) | `Entry` | none | -| 51Degrees client hints | `crates/middleware/51degrees` | `RequestFilter("51degrees.client_hints")` | none of its own | `Feeds("51degrees.client_hints")` | `RequestFilters` | -| DataDome protection | `crates/bot-protection/datadome` | `RequestFilter("bot-protection.datadome")` | `enable_protection = true` in `bot-protection.datadome`, required | `WhenRunning` | `RequestFilters` | -| DataDome page tag | `crates/bot-protection/datadome` | `Middleware("bot-protection.datadome.tag")` | `client_side_key` in `bot-protection.datadome` not blank, required | `Entry` | none | -| DataDome GraphQL flag | `crates/bot-protection/datadome` | `Nothing` | `enable_graphql_support = true` in `bot-protection.datadome` | not used | none | -| InMobi consent prompt | `crates/cmp/inmobi` | `Middleware("cmp.inmobi")` | `script_url` in `cmp.inmobi` not blank, required | `Entry` | none | -| ads.txt sellers | `crates/ads-txt/iabtl` | `Middleware("ads-txt.iabtl")` | `lines` in `ads-txt.iabtl` written, required | `Entry` (open question 3) | none | -| robots.txt | core | `Middleware("robots_txt")` | `robots-txt` written, required | `Entry` in `[[fetch]]` on `/robots.txt` | none | -| Middleware settings | the middleware's crate | that middleware | `middleware.` written, optional | that middleware's | none | -| Shared vendor tables | the vendor's crate | each consumer the vendor's builder supplies | what each consumer needs in `shared.` | that consumer's | that consumer's | -| Section selections | the module's crate | the module, declared by `with_module_name` | none of its own | `Section` | none | -| Loading cache tables | the module's crate | the module, declared by `with_cache_features` | `cache.` written, optional (2.5) | the module's | none | -| Auction analytics | `crates/analytics/tinybird` | `Module("analytics.tinybird")` | none of its own | `Section` | `TelemetrySink` | -| Asset routes | core | `Core("proxy.asset_routes")` | `proxy.asset_routes` written, required | `WhenRunning` | `AssetRoutes` | -| Origin certificate check | core | `AdapterPart(OriginCertificateSetting)` | `certificate_check = false` in `proxy`, required | `WhenRunning` | its own | -| Host geo lookup | core | `AdapterPart(HostGeoLookup)` | `module = "platform"` in `geo`, required | `WhenRunning` | its own | -| Host device module | core | `AdapterPart(HostDeviceModule)` | `module = "platform"` in `device`, required | `WhenRunning` | its own | -| Edge Cookie cluster settings | core | `Nothing` | each of `cluster_trust_threshold` and `cluster_recheck_secs` in `ec` written | not used | none | +| Case | Declared in | Consumer | Reads | Runs when | Placement | Parts | +| ---------------------------- | -------------------------------- | --------------------------------------------- | --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | --------------------------------------- | ---------------- | +| 51Degrees client hints | `crates/middleware/51degrees` | `Middleware("51degrees.client_hints")` | `middleware.51degrees.client_hints` (2.5) | `endpoint` and `license_key` in `shared.51degrees` not blank | `Entry` | none | +| 51Degrees client hints | `crates/middleware/51degrees` | `RequestFilter("51degrees.client_hints")` | none | no condition of its own | `Feeds("51degrees.client_hints")` | `RequestFilters` | +| DataDome protection | `crates/bot-protection/datadome` | `RequestFilter("bot-protection.datadome")` | none | `enable_protection = true` in `bot-protection.datadome` | `WhenRunning` | `RequestFilters` | +| DataDome page tag | `crates/bot-protection/datadome` | `Middleware("bot-protection.datadome.tag")` | none | `client_side_key` in `bot-protection.datadome` not blank, and `inject_client_side_tag` there not `false` (2.5) | `Entry` | none | +| DataDome GraphQL flag | `crates/bot-protection/datadome` | `Nothing` | none | `enable_graphql_support = true` in `bot-protection.datadome` | not used | none | +| InMobi consent prompt | `crates/cmp/inmobi` | `Middleware("cmp.inmobi")` | none | `script_url` in `cmp.inmobi` not blank | `Entry` | none | +| ads.txt sellers | `crates/ads-txt/iabtl` | `Middleware("ads-txt.iabtl")` | none | `lines` in `ads-txt.iabtl` written | `Entry` (open question 3) | none | +| robots.txt | core | `Middleware("robots_txt")` | none | `robots-txt` written | `Entry` in `[[fetch]]` on `/robots.txt` | none | +| Middleware settings | the middleware's crate | that middleware | `middleware.` | that middleware's | that middleware's | none | +| Shared vendor tables | the vendor's crate | each consumer the vendor's builder supplies | `shared.`, through `with_shared_settings` | what each consumer needs in `shared.` | that consumer's | that consumer's | +| Section selections | the module's crate | the module, declared by `with_module_name` | its own table, through `with_module_name` | no condition of its own | `Section` | none | +| Loading cache tables | the module's crate | the module, declared by `with_cache_features` | `cache.`, through `with_cache_features` (2.5) | the module's | the module's | none | +| Auction analytics | `crates/analytics/tinybird` | `Module("analytics.tinybird")` | its own table, through `with_module_name` | no condition of its own | `Section` | `TelemetrySink` | +| Asset routes | core | `Core("proxy.asset_routes")` | none | `proxy.asset_routes` written | `WhenRunning` | `AssetRoutes` | +| Origin certificate check | core | `AdapterPart(OriginCertificateSetting)` | none | `certificate_check = false` in `proxy` | `WhenRunning` | its own | +| Host geo lookup | core | `AdapterPart(HostGeoLookup)` | none | `module = "platform"` in `geo` | `WhenRunning` | its own | +| Host device module | core | `AdapterPart(HostDeviceModule)` | none | `module = "platform"` in `device` | `WhenRunning` | its own | +| Edge Cookie cluster settings | core | `Nothing`, one for each setting | none | `cluster_trust_threshold` or `cluster_recheck_secs` written in `ec`, one in each declaration | not used | none | + +Core's sections need no `reads`, because core always reads them. Under these declarations a setting written at its default, or a switch written off, gives no line. `certificate_check = true` runs no adapter part, `enable_protection = false` (`trusted-server.example.toml` line 936) registers no filter, and `inject_client_side_tag = true` (line 945) lets the tag run. A DataDome tag entry with a blank `client_side_key`, or with `inject_client_side_tag = false`, gets one line, that the tag is placed and does not run (2.3), naming the key. A builder declares every middleware, request filter and preparer it supplies, not only those in the table. The Google ad tag diagnostics builder, for example, declares its preparer as `Preparer("ad-tag.google.diagnostics")` placed by `WhenLinked` (`crates/ad-tag/google/src/diagnostics.rs` lines 46 to 58), because the preparer removes the reserved `ts_console` query and cookie whether or not the module runs (lines 317 to 320). @@ -205,17 +212,17 @@ The check runs once, before any builder builds. Every declaration is static and The check reads each table as the document writes it, before serde fills a default, so a default can neither hide a missing key nor make an unwritten one look written. Module, shared and middleware tables are kept as written already, which is how `Settings::module_config` reads a module's table (`settings.rs` lines 4653 to 4680). Core's typed sections and a vendor's typed table fill defaults, for example for the two Edge Cookie cluster settings (`settings.rs` lines 786 and 794) and for DataDome's `client_side_key` (`crates/bot-protection/datadome/src/lib.rs` lines 342 to 343 and 456), so the load keeps the document's own tables beside the typed settings for the check to read. -A consumer runs when its builder runs, which is when a section selects its module, when its shared table is written, or always for core's own, and when every setting it requires holds. It is placed as its placement says, in the deployment as a whole, so a middleware is placed when any entry of the document names it, whatever the path of a given request. A gap is one of five kinds, and each line names the setting, the consumer, the missing half and what to add. +A consumer runs when its builder runs, which is when a section selects its module, when its shared table is written, or always for core's own, and when every condition in its `runs_when` holds. It is placed as its placement says, in the deployment as a whole, so a middleware is placed when any entry of the document names it, whatever the path of a given request. A gap is one of five kinds, and each line names the setting, the consumer, the missing half and what to add. -1. A written table or key that no linked consumer declares, or that only consumers which do not run declare, and an entry or a selection that names nothing a linked crate declares. A module's table that its section does not select is one, which core refuses today (`provider_table.rs` lines 564 to 567). -2. A placed consumer that does not run, naming the selection, shared table or required setting it lacks. -3. A consumer that runs and that nothing places. +1. A written table that no consumer reads, or that only consumers whose builders do not run read, and an entry or a selection that names nothing a linked crate declares, or that names a middleware for a media type or path its placement does not allow. A module's table that its section does not select is one, which core refuses today (`provider_table.rs` lines 564 to 567). This kind looks at tables and names and never at a single key, so a key whose value fails a condition only decides whether a consumer runs. +2. A placed consumer that does not run, naming the selection, shared table or condition it lacks. +3. A consumer that a written setting runs, being its builder's selection or shared table or a condition of its own, and that nothing places. A consumer that nothing written runs, such as core's service version middleware, is never one, as today (`registry.rs` lines 1735 to 1746). 4. A placed consumer that runs and needs a part the adapter lacks, naming the adapter and the part. An `AdapterPart` consumer needs its own part. -5. A written setting whose consumer is `Nothing`, which the line names as accepted and unused. +5. A setting that a `Nothing` declaration names and the document writes, which the line names as accepted and unused. A request filter or preparer placed by `Feeds` gets no line of kind 2 or 3 of its own, because the line for its middleware covers it, so one state gives one line. -Each line has a level. Kinds 1, 2 and 4 refuse. Kind 3 refuses when a setting the consumer declares holds, because that setting then does nothing, as the InMobi, ads.txt and robots.txt checks refuse today. It warns when only its builder's selection or shared table runs it, as the warnings for a middleware no entry names and for a module on offer that its section does not select do today. Kind 5 warns, and so do the lines in 2.4 for a name or for parts that the tool cannot check at push. Startup and push refuse when any line refuses, and give every line either way, so a document with two gaps gets both lines. The existing refusals and warnings in section 1 become lines of this report, with their wording kept where it already names what to add. The registry's checks after the build stay only to catch a registration that contradicts its builder's declarations, as `check_described_middleware` does today. +Each line has a level. Kinds 1, 2 and 4 refuse. Kind 3 refuses when the consumer runs under a condition of its own, or a table in its `reads` is written, because that setting then does nothing, as the InMobi, ads.txt and robots.txt checks refuse today. It warns when only its builder's selection or shared table runs it, as the warnings for a middleware no entry names and for a module on offer that its section does not select do today. Kind 5 warns, and so do the lines in 2.4 for a name or for parts that the tool cannot check at push. Startup and push refuse when any line refuses, and give every line either way, so a document with two gaps gets both lines. The existing refusals and warnings in section 1 become lines of this report, with their wording kept where it already names what to add. The registry's checks after the build stay only to catch a registration that contradicts its builder's declarations, as `check_described_middleware` does today. A request filter or preparer placed by `Feeds` is kept only where its middleware is placed, which the registry decides once as it is built, so it runs on every request wherever its middleware is placed, as filters run today. A preparer placed by `WhenLinked` is kept for every linked builder, as every preparer is collected today (`registry.rs` lines 2103 to 2108). @@ -223,11 +230,11 @@ A request filter or preparer placed by `Feeds` is kept only where its middleware `TrustedServerAppConfig` validates through a trait that takes no arguments, so the tool registers the builders it validates against once, before validating (`config.rs` lines 87 to 106). The `ts` tool registers the builders it links (`crates/trusted-server-cli/src/app_config.rs` lines 35 to 81) at the start of every command (`crates/trusted-server-cli/src/run.rs` line 126). Push would run the check in 2.3 over those builders' declarations, so every gap a linked builder can cause is caught before the document is written to the config store. A kind 1 line for a name that no crate the tool links declares, being a selection, an entry, a shared table, a middleware table or a cache table, warns at push and says that the tool cannot check the name, because a vendor crate the tool never links may supply it. That is why push leaves a selection unchecked on purpose today (`registry.rs` lines 2163 to 2168). The cache table check refuses such a name at push today (`config.rs` line 330), so this would change it (open question 2). -The parts reach push from the adapter's own `parts.toml` (2.2). `ts config push` and `ts config diff` already take `--adapter` (the tool's tests parse it at `crates/trusted-server-cli/src/run.rs` lines 626 to 647), and `ts config validate` takes none (lines 785 to 799), so the tool adds an optional `--adapter` to `ts config validate`, beside the arguments it takes from EdgeZero (lines 4 to 7 and 75 to 76). Given an adapter, the tool finds the adapter's block in `edgezero.toml` through the manifest loader it already uses (`crates/trusted-server-cli/src/app_config.rs` line 12), reads `parts.toml` in the folder the block's `crate` key names (for example `edgezero.toml` line 56 for Axum), and registers the parts as it registers the builders, before EdgeZero validates. Neither core nor the tool names a platform, because the adapter's name comes from the command line and its folder from the manifest. With no adapter given, `ts config validate` checks everything but the parts, and a line that warns says so. A block with no `crate` key, or a folder with no `parts.toml`, gets the same warning, so the parts are never passed over in silence. +The parts reach push from the adapter's own `parts.toml` (2.2). `ts config push` and `ts config diff` already take `--adapter` (the tool's tests parse it at `crates/trusted-server-cli/src/run.rs` lines 626 to 647), and `ts config validate` takes none (lines 785 to 799), so the tool adds an optional `--adapter` to `ts config validate`, beside the arguments it takes from EdgeZero (lines 4 to 7 and 75 to 76). Given an adapter, the tool finds the adapter's block in `edgezero.toml` at the repository root through the manifest loader it already uses (`crates/trusted-server-cli/src/app_config.rs` line 12), reads `parts.toml` in the folder the block's `crate` key names (for example `edgezero.toml` line 56 for Axum), and registers the parts as it registers the builders, before EdgeZero validates. Neither core nor the tool names a platform, because the adapter's name comes from the command line and its folder from the manifest. With no adapter given, `ts config validate` checks everything but the parts, and a line that warns says so. A block with no `crate` key, or a folder with no `parts.toml`, gets the same warning, so the parts are never passed over in silence. ### 2.5 A consumer that runs on defaults -A consumer whose setting is optional declares it with `required: false`, and leaving it out is then not a gap. The 51Degrees client hints middleware reads `[middleware.51degrees.client_hints]` and writes `Critical-CH` unless the table turns it off (`crates/middleware/51degrees/src/lib.rs` lines 191 to 202), so it declares that table optional. A module's `[cache.]` table is optional in the same way, because a declared cache with no table takes the one loading cache module the build links (`mod.rs` lines 1083 to 1104), and every adapter's stock list links one (for example Axum `app.rs` line 140). Only a build that links no loading cache module, or several, loads a cache with no table through no cache at all (`platform/loading_cache.rs` lines 240 to 242 and 269 to 272). Neither is a stock build, and this model leaves the case as it is. +A table a consumer only reads goes in its `reads` and not in its `runs_when`, so leaving the table out is not a gap. The 51Degrees client hints middleware reads `[middleware.51degrees.client_hints]` and writes `Critical-CH` unless the table turns it off (`crates/middleware/51degrees/src/lib.rs` lines 191 to 202), so that table is in its `reads`. A key whose default lets a consumer run is declared with `KeyIsNot`, as the DataDome tag declares `inject_client_side_tag`, which is on unless it is written `false` (`crates/bot-protection/datadome/src/lib.rs` lines 346 to 347, 405 to 407 and 1024). A module's `[cache.]` table is read in the same way, through `with_cache_features`, because a declared cache with no table takes the one loading cache module the build links (`mod.rs` lines 1083 to 1104), and every adapter's stock list links one (for example Axum `app.rs` line 140). Only a build that links no loading cache module, or several, loads a cache with no table through no cache at all (`platform/loading_cache.rs` lines 240 to 242 and 269 to 272). Neither is a stock build, and this model leaves the case as it is. ## 3. What is not in this change @@ -242,20 +249,21 @@ A consumer whose setting is optional declares it with `required: false`, and lea 1. For each row of the cases table in the issue this design answers (IABTechLab/trusted-server#1403), and for the host device case in section 1, one test with the settings and no consumer and, where the consumer cannot act without its settings, one test with the consumer and no settings. Each runs through the real settings load and state build of each adapter, as the Axum adapter's tests already build the adapter's routes from settings (`crates/trusted-server-adapter-axum/tests/seam_probe.rs` line 94), and through the Fastly, Cloudflare and Spin builds, not through the check called directly. 2. Each of those tests is seen to fail with its declaration removed before the test is relied on. -3. A document with two gaps of different kinds gets both lines in one report, and a document whose only lines warn starts and is pushed. +3. A document with two gaps of different kinds gets both lines in one report, and a document whose only lines warn starts and is pushed. A document that writes settings at their defaults, such as `certificate_check = true`, `enable_protection = false` and `inject_client_side_tag = true`, gets no line for them. 4. `ts config validate --adapter ` and `ts config push --adapter `, given the builders the tool links and the parts in that adapter's `parts.toml`, report the same lines as that adapter's startup for every gap a linked builder or the adapter's parts can cause. `ts config validate` with no adapter reports every other gap, with one line that warns that the parts were not checked. -5. A test over every builder each adapter links (for example Axum `app.rs` lines 109 to 145) checks both ways. Every middleware, module, request filter and page request preparer a builder registers has a declaration of the same name, and every declaration is registered by its builder when the declaration's settings hold, so a declaration cannot outlive its consumer. `check_described_middleware` (`registry.rs` lines 1591 to 1630) is the model only for matching by name, because it checks a registered middleware against its description and skips a described one that the registration leaves out (lines 1596 to 1603). +5. A test over every builder each adapter links (for example Axum `app.rs` lines 109 to 145) checks both ways. Every middleware, module, request filter and page request preparer a builder registers has a declaration of the same name, and every declaration is registered by its builder when the builder runs and the declaration's `runs_when` holds, so a declaration cannot outlive its consumer. `check_described_middleware` (`registry.rs` lines 1591 to 1630) is the model only for matching by name, because it checks a registered middleware against its description and skips a described one that the registration leaves out (lines 1596 to 1603). 6. On each adapter, for each part its `parts.toml` names, a test sees the part act, and for each part the file does not name, a test sees a document that needs the part refused, so the file cannot claim a part the adapter lacks. ## 5. Open questions -1. Should the lines that refuse under this model, where today there is only a warning or nothing at all, warn for one release first? They are the kind 3 lines where a setting the consumer declares holds, which today get the warning for a middleware no entry names (`registry.rs` lines 1724 to 1731), and the kind 2 and kind 4 lines for gaps that are accepted today with no log. A deployment that carries `[shared.51degrees]` with no entry starts today and would stop, and so would one on Axum with `[geo] module = "platform"`. +1. Should the lines that refuse under this model, where today there is only a warning or nothing at all, warn for one release first? They are the kind 3 lines for a consumer that runs under a condition of its own or has a table in its `reads` written, which today get the warning for a middleware no entry names (`registry.rs` lines 1724 to 1731), the kind 1 lines for a table that is accepted today although no running consumer reads it, such as a `[middleware.]` table at a middleware that reads its module's table instead (`settings.rs` lines 4696 to 4698) or a `[cache.]` table whose module does not run, and the kind 2 and kind 4 lines for gaps that are accepted today with no log. A deployment that carries `[shared.51degrees]` with no entry starts today and would stop, and so would one on Axum with `[geo] module = "platform"`. 2. Should a name that no crate the tool links declares refuse at push, as the cache table check does today (`config.rs` line 330), or warn, as section 2.4 proposes and as push leaves a selection unchecked on purpose today (`registry.rs` lines 2163 to 2168)? Refusing catches a misspelt name before the document is written, and warning lets a deployment push, with the stock tool, names that its own crates supply. 3. Should the ads.txt sellers' placement require `path = "/ads.txt"`, as the robots.txt placement requires its path? An entry with no path appends the lines to every `text/plain` file it covers (`crates/ads-txt/iabtl/src/lib.rs` lines 254 to 262), and the module's check counts any `text/plain` entry today (lines 152 to 167), so the declaration keeps that placement until this is decided. ## Review log 1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/50-cookies-by-path` at `3aaf720c0`. Issues found, two of them blocking. The declaration types could not say that a consumer is code in core, an adapter part, a request filter that feeds no middleware or a preparer that runs without its module, nor that a setting must have a given value or must not be blank, and push and validate had no way to see the parts of the adapter a document is for. This revision gives the types those consumers and conditions, has each adapter declare its parts in a `parts.toml` in its own crate that the tool reads for the adapter `--adapter` names, and applies the other findings. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/50-cookies-by-path` at `3aaf720c0`. Issues found, two of them blocking. A written key whose value failed a consumer's condition counted as a setting no running consumer declares, so defaults such as `certificate_check = true` and `enable_protection = false` were refused, and the doc comment on a core type named two vendors. This revision declares the tables a consumer reads apart from the conditions it runs under, so that the first kind of gap looks only at tables and names, adds a condition for a key that is absent or has another value, takes the media type of core's own middleware from `supplied_middleware`, places the files at the repository root, and gives the doc comment neutral examples. ## Revision record @@ -263,3 +271,4 @@ A consumer whose setting is optional declares it with `required: false`, and lea | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | 2026-10-16 | First draft, from the code at `3aaf720c0` and the issue that states the problem (IABTechLab/trusted-server#1403). | | 2026-10-16 | Round 1 fixes. The types gain consumers for code in core, adapter parts and settings nothing reads, conditions on a key's value, named filters and preparers, and placements for a consumer nothing else places and for a preparer that runs whenever its crate is linked. Each adapter declares its parts in `parts.toml`, which push, diff and a new `ts config validate --adapter` read. Lines have levels, the check runs before any build on the settings as written, the host device case is added, and the claims on the push-time checks, the loading cache default and the one-off notices are corrected. | +| 2026-10-16 | Round 2 fixes. A consumer declares the tables it reads apart from the conditions it runs under, and the first kind of gap looks only at tables and names, so a default such as `certificate_check = true` is no longer refused. `KeyIsNot` holds for a key that is absent or has another value, core's own middleware takes its media type from `supplied_middleware`, the doc comment on `SettingsNeed` names no vendor, and the files at the repository root are named as such. | From 71bdb98b27cb992c83ee4dc017e3a2f89ed7a6a6 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 13:31:40 +0100 Subject: [PATCH 115/192] The settings and consumers design takes its round 3 check The fifth kind of gap applies only when the declaration's run conditions hold, so a switch written off gives no line. --- .../specs/2026-10-16-settings-and-consumers-design.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md b/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md index 5bd063fddb..2b849f4dc7 100644 --- a/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md +++ b/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md @@ -218,7 +218,7 @@ A consumer runs when its builder runs, which is when a section selects its modul 2. A placed consumer that does not run, naming the selection, shared table or condition it lacks. 3. A consumer that a written setting runs, being its builder's selection or shared table or a condition of its own, and that nothing places. A consumer that nothing written runs, such as core's service version middleware, is never one, as today (`registry.rs` lines 1735 to 1746). 4. A placed consumer that runs and needs a part the adapter lacks, naming the adapter and the part. An `AdapterPart` consumer needs its own part. -5. A setting that a `Nothing` declaration names and the document writes, which the line names as accepted and unused. +5. A setting that a `Nothing` declaration names, written so that the declaration's `runs_when` holds, which the line names as accepted and unused. A switch written off gives no line, as for any other consumer. A request filter or preparer placed by `Feeds` gets no line of kind 2 or 3 of its own, because the line for its middleware covers it, so one state gives one line. @@ -265,6 +265,8 @@ A table a consumer only reads goes in its `reads` and not in its `runs_when`, so 1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/50-cookies-by-path` at `3aaf720c0`. Issues found, two of them blocking. The declaration types could not say that a consumer is code in core, an adapter part, a request filter that feeds no middleware or a preparer that runs without its module, nor that a setting must have a given value or must not be blank, and push and validate had no way to see the parts of the adapter a document is for. This revision gives the types those consumers and conditions, has each adapter declare its parts in a `parts.toml` in its own crate that the tool reads for the adapter `--adapter` names, and applies the other findings. 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/50-cookies-by-path` at `3aaf720c0`. Issues found, two of them blocking. A written key whose value failed a consumer's condition counted as a setting no running consumer declares, so defaults such as `certificate_check = true` and `enable_protection = false` were refused, and the doc comment on a core type named two vendors. This revision declares the tables a consumer reads apart from the conditions it runs under, so that the first kind of gap looks only at tables and names, adds a condition for a key that is absent or has another value, takes the media type of core's own middleware from `supplied_middleware`, places the files at the repository root, and gives the doc comment neutral examples. +3. **Round 3, 2026-10-16.** A short check of the round 2 fixes in a fresh session, against `split/50-cookies-by-path` at `3aaf720c0`. All six were fixed truthfully and nothing blocking remains. This revision makes the fifth kind of gap apply only when the declaration's run conditions hold, so a switch written off gives no line. + ## Revision record | Date | Change | @@ -272,3 +274,4 @@ A table a consumer only reads goes in its `reads` and not in its `runs_when`, so | 2026-10-16 | First draft, from the code at `3aaf720c0` and the issue that states the problem (IABTechLab/trusted-server#1403). | | 2026-10-16 | Round 1 fixes. The types gain consumers for code in core, adapter parts and settings nothing reads, conditions on a key's value, named filters and preparers, and placements for a consumer nothing else places and for a preparer that runs whenever its crate is linked. Each adapter declares its parts in `parts.toml`, which push, diff and a new `ts config validate --adapter` read. Lines have levels, the check runs before any build on the settings as written, the host device case is added, and the claims on the push-time checks, the loading cache default and the one-off notices are corrected. | | 2026-10-16 | Round 2 fixes. A consumer declares the tables it reads apart from the conditions it runs under, and the first kind of gap looks only at tables and names, so a default such as `certificate_check = true` is no longer refused. `KeyIsNot` holds for a key that is absent or has another value, core's own middleware takes its media type from `supplied_middleware`, the doc comment on `SettingsNeed` names no vendor, and the files at the repository root are named as such. | +| 2026-10-16 | Round 3 fix. The fifth kind of gap applies only when the declaration's run conditions hold. | From 5e56ce85fb3cd2e2e892d2bc20e9b9e745b09de7 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 13:47:54 +0100 Subject: [PATCH 116/192] The consent prompt phases design takes its round 2 review and describes the two refusals A serve entry may not place the consent prompt on a page where the fetch phase writes other middleware's markup first, and the prompt is refused when both phases name it for one page. The open questions those answer move into the text, the round 2 points are applied, and a new open question asks whether each middleware should say if it writes markup ahead of the bundle. --- ...2026-10-12-consent-prompt-phases-design.md | 42 ++++++++++--------- 1 file changed, 22 insertions(+), 20 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md b/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md index 12c0620e7b..50150f8e93 100644 --- a/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md +++ b/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md` (the two phases and where a serve middleware's head markup goes), `2026-10-12-inmobi-choice-module-design.md` (amended in section 3) **Related PRs:** [IABTechLab/trusted-server#1362](https://github.com/IABTechLab/trusted-server/pull/1362) builds on the sellers module in [IABTechLab/trusted-server#1361](https://github.com/IABTechLab/trusted-server/pull/1361) (row 20), replaces [IABTechLab/trusted-server#1308](https://github.com/IABTechLab/trusted-server/pull/1308) (closed, an earlier pull request for this change against `main`) and was reviewed on [jwrosewell/trusted-server#16](https://github.com/jwrosewell/trusted-server/pull/16) (closed). The InMobi module is in [IABTechLab/trusted-server#1359](https://github.com/IABTechLab/trusted-server/pull/1359) (row 18), page changes as middleware are in [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10), and the change that removes the stock list of modules is [IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384) (row 43) **Issues:** [IABTechLab/trusted-server#1279](https://github.com/IABTechLab/trusted-server/issues/1279). [IABTechLab/trusted-server#1289](https://github.com/IABTechLab/trusted-server/issues/1289) audits Didomi, Sourcepoint and Osano against the rules a consent prompt is held to, and [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-15 +**Last updated:** 2026-10-16 > **Why this spec exists.** The InMobi, Didomi and Sourcepoint modules each supply one middleware, and each declared the fetch phase as the only phase it runs in, so a `[[serve]]` entry could not name any of them and the module chose the phase for the publisher. This change lets each of the three run in either phase. The phase decides whether a stored page carries the output and where the output lands in the head, and it does not decide which readers get the output. @@ -37,26 +37,26 @@ middleware = ["cmp.inmobi"] ## 3. Where a prompt lands in the head -A phase writes each middleware's head markup, being the tags a middleware writes into the page's ``, in the order the entry names the middleware. A fetch middleware's head markup goes into the snippet that core writes at the start of the head, ahead of the script bundle (`build_html_processor` in `crates/trusted-server-core/src/html_processor.rs`). A serve middleware's head markup goes straight before the bundle's tag (`create_serve_processor` in the same file), so a serve middleware's head markup lands after the head markup of every fetch middleware and before everything the page's own head carries. +A phase writes each middleware's head markup, being the tags a middleware writes into the page's ``, in the order the entry names the middleware. A fetch middleware's head markup goes into the snippet that core writes at the start of the head, ahead of the script bundle (`build_html_processor` in `crates/trusted-server-core/src/html_processor.rs`). A serve middleware's head markup goes straight before the bundle's tag (`create_serve_processor` in the same file), so a serve middleware's head markup lands after what every fetch middleware writes ahead of the bundle and before everything the page's own head carries. What a fetch middleware writes after the bundle (`after_bundle_inserts`) lands after a serve middleware's head markup, and no stock fetch middleware writes any. -The InMobi guide (`docs/guide/integrations/inmobi.md`) says why the prompt is written first, which is that nothing advertising related may run before the reader answers the prompt, and its first rule is that the prompt is written before any vendor's tag. A prompt in `[[fetch]]`, named before the middleware that write a vendor's tag, keeps that rule. A prompt in `[[serve]]` does not keep the rule where a fetch middleware writes a vendor's tag, because that tag then comes ahead of the whole of the prompt's tag, the two stubs included. Two stock middleware that run only in the fetch phase write a vendor's tag into the head, being `auction.prebid` (its configuration script and the tag that loads the bundle) and `ad-tag.google` (the `tsjs.adInit` bootstrap). `tag.google-tag-manager` writes no head markup, because it only moves the addresses in the page's own tags to the first-party path, so a prompt in `[[serve]]` still lands ahead of Tag Manager's tags. +The InMobi guide (`docs/guide/integrations/inmobi.md`) says why the prompt is written first, which is that nothing advertising related may run before the reader answers the prompt, and its first rule is that the prompt is written before any vendor's tag. A prompt in `[[fetch]]`, named before the middleware that write a vendor's tag, keeps that rule. A prompt in `[[serve]]` would not keep the rule where a fetch middleware writes a vendor's tag ahead of the bundle, because that tag would then come ahead of the whole of the prompt's tag, the two stubs included, so the InMobi module refuses that placement (below). Two stock middleware that run only in the fetch phase write a vendor's tag into the head, being `auction.prebid` (its configuration script and the tag that loads the bundle) and `ad-tag.google` (the `tsjs.adInit` bootstrap). `tag.google-tag-manager` writes no head markup, because it only moves the addresses in the page's own tags to the first-party path, so a prompt in `[[serve]]` still lands ahead of Tag Manager's tags. -Nothing refuses or warns when a prompt in `[[serve]]` follows a fetch middleware's vendor tag, because the registry checks names, phases and media types and not what is written, and no test pins the order (section 7). The guide advises placing the prompt in `[[fetch]]` wherever a fetch middleware writes a vendor's tag, and its statement of the first rule does not carry that condition (section 7). The Didomi and Sourcepoint middleware write settings and not a prompt, and those settings land ahead of the bundle that reads them in either phase. The audit that brings the two to the rules is [IABTechLab/trusted-server#1289](https://github.com/IABTechLab/trusted-server/issues/1289). +**The refusals.** The InMobi module refuses, when the service starts and in `ts config validate`, a `[[serve]]` entry that names `cmp.inmobi` for a page whose `[[fetch]]` entry names any other middleware, and a page for which entries of both phases name `cmp.inmobi`, where the prompt would be written twice, loader included (`require_the_prompt_once_and_first` in `crates/cmp/inmobi/src/lib.rs`). The registry checks names, phases and media types and not what a middleware writes, and nothing says which middleware write a vendor's tag ahead of the bundle, so the module refuses any, `tag.google-tag-manager` included, although a prompt in `[[serve]]` lands ahead of Tag Manager's tags (section 7). A page is judged by the entry each phase gives it, which depends only on which entry paths begin the page's path, so the module checks `/` and the path of each `text/html` entry, and entries of the two phases under paths that never meet are accepted. The order within one entry is the publisher's, as before. The Didomi and Sourcepoint middleware write settings and not a prompt, and those settings land ahead of the bundle that reads them in either phase. The audit that brings the two to the rules is [IABTechLab/trusted-server#1289](https://github.com/IABTechLab/trusted-server/issues/1289). -**What this amends.** The InMobi design (`2026-10-12-inmobi-choice-module-design.md`) says on its line 32 that the middleware writes its three parts where a `[[fetch]]` entry names the prompt and that the stored page carries them. The same holds for a `[[serve]]` entry, except that each reader's copy carries them and the stored page does not. The InMobi design says on its line 44 that the prompt is written before any vendor's tag, which holds in the serve phase only where no fetch middleware writes a vendor's tag. +**What this amends.** The InMobi design (`2026-10-12-inmobi-choice-module-design.md`) says on its line 32 that the middleware writes its three parts where a `[[fetch]]` entry names the prompt and that the stored page carries them. The same holds for a `[[serve]]` entry, except that each reader's copy carries them and the stored page does not. The InMobi design says on its line 44 that the prompt is written before any vendor's tag, which the module holds in the serve phase by refusing a `[[serve]]` entry for a page whose `[[fetch]]` entry names another middleware. ## 4. Failure, rollout and adapters -**Failure.** An entry that names a middleware in a phase the middleware does not declare is refused when the service starts, and the service then answers page requests with an error, 500 on the Axum, Cloudflare and Fastly adapters and 503 on Spin (`startup_error_router` in each adapter's `src/app.rs`, and `crates/trusted-server-core/src/error.rs`). The InMobi module also refuses a selected prompt that no entry of either phase names, in `ts config validate` and in the running service. Didomi and Sourcepoint do not refuse that case, and the registry logs a warning at startup that no entry names the middleware (`unnamed_middleware` in the registry). +**Failure.** An entry that names a middleware in a phase the middleware does not declare is refused when the service starts, and the service then answers page requests with an error, 500 on the Axum, Cloudflare and Fastly adapters and 503 on Spin (`startup_error_router` in each adapter's `src/app.rs`, and `crates/trusted-server-core/src/error.rs`). The InMobi module also refuses, in `ts config validate` and when the service starts, a selected prompt that no entry of either phase names, a prompt in `[[serve]]` for a page whose `[[fetch]]` entry names another middleware, and a prompt named for one page in both phases (section 3). Didomi and Sourcepoint refuse none of these, and where no entry names their middleware the registry logs a warning at startup (`unnamed_middleware` in the registry). **Rollout.** A build from before this change declares the fetch phase alone for the three middleware, so that build refuses settings that name one in a `[[serve]]` entry, and `ts config validate` does not warn of that because the phase check runs only when the service starts. Deploy the build that carries this change before pushing settings that name one of the three in `[[serve]]`. On Fastly the settings sit in a config store apart from the deployed build (`load_settings_from_config_store` in `crates/trusted-server-adapter-fastly/src/app.rs`), so the two steps can be done in either order, and the wrong order stops the service. Every change to the settings also changes the key that stored pages are held under, because the key carries a fingerprint of all the settings, which is a short code that changes when any setting changes (`template_fingerprint` in `crates/trusted-server-core/src/publisher.rs`). Moving a prompt between phases therefore leaves the stored pages unread until each page is stored again. -**Adapters.** The Axum, Cloudflare and Spin adapters buffer the page through `buffer_publisher_response_async` and the Fastly adapter streams it through `publisher_response_into_streaming_response`, and both ways run the serve entry (`run_serve_pass` on the buffered path and `reader_serve_processor` on the streaming path, both in `crates/trusted-server-core/src/publisher.rs`). The change adds nothing for one adapter. Only the Axum and Fastly adapters have a test that starts the adapter with a `[[serve]]` entry, and both use the seam probe and not a prompt. +**Adapters.** The Axum, Cloudflare and Spin adapters buffer the page through `buffer_publisher_response_async` and the Fastly adapter streams it through `publisher_response_into_streaming_response`, and both ways run the serve entry. On a page that is not stored, the serve entry runs inside the page's transform, where `create_html_stream_processor` hands the serve entry's chain to `create_html_processor_with_middleware` so that both phases run in one pass. On a stored page it runs on each reader's copy, through `run_serve_pass` on the buffered path and `reader_serve_processor` on the streaming path (all in `crates/trusted-server-core/src/publisher.rs`). The change adds nothing for one adapter. Only the Axum and Fastly adapters have a test that starts the adapter with a `[[serve]]` entry, and both use the seam probe and not a prompt. ## 5. What is not in this change - A way to give the output to some readers and not others. Neither an entry nor the three middleware tell readers apart (section 2), so every reader of a page that an entry covers gets the output, and one stored page does not serve readers with and without the prompt as the issue asks. -- A refusal or a warning when a prompt in `[[serve]]` is written after a fetch middleware's vendor tag (section 3). +- A way to tell which middleware write a vendor's tag ahead of the bundle. The InMobi module therefore refuses a prompt in `[[serve]]` after any middleware that a `[[fetch]]` entry names for the page, Tag Manager included (section 3). - Osano. Its module registers no middleware and changes no page from Rust (`crates/cmp/osano/src/lib.rs`), so it has no phase to choose. - A change to what any of the three writes. Each writes what it wrote before. - A change to `ts audit generate`, which still places the three in `[[fetch]]`. @@ -65,32 +65,34 @@ Nothing refuses or warns when a prompt in `[[serve]]` follows a fetch middleware - Each of the three modules declares both phases, with a test in each crate. - The InMobi module selected with no entry of either phase naming it is refused, as before, and the message names both lists. An entry of the serve phase alone is accepted, and the module then registers one piece of head markup for the serve phase. +- The InMobi module refuses a `[[serve]]` entry naming the prompt for a page whose `[[fetch]]` entry names another middleware, and the prompt named for one page in both phases, when the service starts and in `ts config validate`. One test builds the registry and drives a page through both phases for each placement of the prompt and a stand-in vendor's tag, and finds the whole prompt ahead of the tag wherever the service starts (`no_placement_the_service_starts_with_puts_a_vendor_tag_ahead_of_the_prompt`). Two tests hold each refusal to the pages both phases reach, under paths that meet and paths that do not (`a_prompt_after_the_fetch_phase_is_refused_on_the_pages_both_phases_reach` and `a_prompt_named_for_one_page_in_both_phases_is_refused`). +- Each of the three crates runs its middleware from a `[[serve]]` entry through the registry's check and a page, and the reader receives the page recorded with a `[[fetch]]` entry (`placed_in_serve_the_prompt_gives_a_reader_the_recorded_page` in the InMobi crate and `placed_in_serve_the_page_change_gives_a_reader_the_recorded_page` in the Didomi and Sourcepoint crates). - The stock list keeps each prompt in the fetch phase, which is the phase `ts audit generate` writes. Its test allows a middleware that runs in several phases to be listed in one of them, and checks that what is listed for a phase is registered for it and that everything registered is listed somewhere. The stock list's test asserted before this change that the registry's list for each phase equals the stock list's for that phase. The test now checks only that each listed name is registered and that each registered name is listed in some phase, so it no longer compares the registry's order with the stock list's, and a prompt listed under the serve phase passes. A second test, which this change leaves as it was, still holds the order of the stock list to the order of the stock builders (`the_listed_middleware_follow_the_order_the_modules_register_in`). The change that removes the stock list of modules (row 43, [IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384)) removes both tests, and each of the three modules then describes its middleware itself, in the fetch phase, with a test that holds the description to what the module registers (`assert_described_middleware_is_registered`). -**Checks recorded.** Every check passed at `938be5d3b`, the head of the pull request, being 25 of 25 on the pull request ([IABTechLab/trusted-server#1362](https://github.com/IABTechLab/trusted-server/pull/1362)) and 19 of 19 on the fork's closed pull request ([jwrosewell/trusted-server#16](https://github.com/jwrosewell/trusted-server/pull/16)). The description of the pull request records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run. The description marks a manual run with `fastly compute serve` as not done and says that no page has been served through an adapter with a prompt placed in the serve phase. +**Checks recorded.** Every check passed at `938be5d3b`, the head of the pull request, being 25 of 25 on the pull request ([IABTechLab/trusted-server#1362](https://github.com/IABTechLab/trusted-server/pull/1362)) and 19 of 19 on the fork's closed pull request ([jwrosewell/trusted-server#16](https://github.com/jwrosewell/trusted-server/pull/16)). The description of the pull request records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run. The description marks a manual run with `fastly compute serve` as not done and says that no page has been served through an adapter with a prompt placed in the serve phase. The refusals and serve tests of 2026-10-16 were checked locally with the 116 tests of the three crates, clippy on the three with every target, `cargo clippy-fastly` and `cargo fmt --all -- --check`, and each new refusal's tests fail when that refusal is removed. -**What no test covers.** No test runs a prompt placed in `[[serve]]` through a page or through the registry's check. The InMobi test for a serve entry stops at one piece of head markup (`a_prompt_named_in_a_serve_entry_alone_is_accepted`), the Didomi and Sourcepoint tests assert the phases alone, and the recorded page of each of the three crates is made with a `[[fetch]]` entry. +**What no test covers.** The page tests above run both phases in one pass, as a page that is not stored does. No test runs a prompt from a `[[serve]]` entry on a stored page, where the serve entry runs on each reader's copy (section 4), and no adapter test serves a page with a prompt placed in the serve phase. ## 7. Open questions -These are for the reviewer. This revision records what the code does and decides none of the questions. Line numbers are those of `938be5d3b`, the head of the pull request. +These are for the reviewer. Line numbers are those of `938be5d3b`. 1. The issue asks for one stored page that serves readers with and without the prompt. Is that still wanted, and if so, what tells readers apart, given that an entry is chosen by media type and request path alone and the three middleware ignore the reader? -2. Should the registry refuse, or warn of, a prompt in `[[serve]]` that follows a fetch middleware's vendor tag? Nothing does either, and no test pins the order. The InMobi order test writes both middleware in the fetch phase (`a_vendor_tag_named_after_the_prompt_is_written_after_it`). -3. Should a prompt named in both `[[fetch]]` and `[[serve]]` be refused? The registry's check and the InMobi module's own check accept that case, and the prompt's tag is written once in each phase, the loader tag included. No test and no document covers it. -4. Should a prompt in the serve phase be run through a page and through the registry's check, with a recorded page made with a `[[serve]]` entry for each of the three crates? None of the three crates places a middleware with `place_on_every_page` (`crates/trusted-server-core/src/html_processor.rs`, line 873). -5. Should the stock list's test, which now checks that each name is listed and registered and no longer compares equality, be tightened until the change in row 43 removes the test? A prompt listed under the serve phase passes the test, and the order `ts audit` writes is no longer compared with the registry's. -6. Four places in the guides still describe the fetch phase alone or contradict one another. `docs/guide/configuration.md` (lines 1934 to 1937) says that a `[[fetch]]` entry places the InMobi prompt. `docs/guide/integrations/didomi.md` (lines 82 and 83) and `docs/guide/integrations/sourcepoint.md` (line 52) say that the middleware runs on the pages a `[[fetch]]` entry names. The first rule in `docs/guide/integrations/inmobi.md` (line 100) says that the prompt is written before any vendor's tag, where the placement paragraph of the same guide (lines 68 to 71) says that a prompt in `[[serve]]` comes after a vendor's tag that a `[[fetch]]` entry writes. +2. Should the stock list's test, which now checks that each name is listed and registered and no longer compares equality, be tightened until the change in row 43 removes the test? A prompt listed under the serve phase passes the test, and the order `ts audit` writes is no longer compared with the registry's. +3. Three places in the guides still describe the fetch phase alone or name a tag that is not written ahead of the prompt. `docs/guide/integrations/didomi.md` (lines 82 and 83) and `docs/guide/integrations/sourcepoint.md` (line 52) say that the middleware runs on the pages a `[[fetch]]` entry names. `docs/guide/integrations/inmobi.md` (lines 67 and 68) gives Google Tag Manager as the vendor's tag that the prompt is named ahead of, where Tag Manager writes no head markup (section 3). +4. Should each middleware say whether it writes markup ahead of the script bundle, so that the InMobi module's refusal of a prompt in `[[serve]]` spares a `[[fetch]]` entry whose middleware write none there, such as `tag.google-tag-manager`? Nothing says so today, so the module refuses any middleware that the `[[fetch]]` entry for the page names (section 3). ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/21-cmp-phases` at `938be5d3b`. Issues found, four of them blocking. The blocking issues were that the spec said nothing of a prompt in the serve phase being written after the vendor tags the fetch phase writes, that it said one stored page could serve readers with and without the prompt when neither an entry nor the three modules tell readers apart, that its last paragraph claimed a run of each commit and deliberate faults that no description or commit message records, and that it said a serve entry was refused when the settings load when the refusal comes when the service starts. This revision states where a serve prompt lands in the head, what the serve phase gives and costs, when each refusal comes, what a wrong order of rollout does, what each adapter runs and what the description records as checked, while the missing refusal and test for the order, a prompt named in both phases, the missing serve tests, the weaker stock list test and four stale places in the guides wait for code changes as open questions. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/21-cmp-phases` at `938be5d3b`. Approved, none blocking. This revision narrows where a serve prompt lands to after the fetch markup ahead of the bundle, names the route a page that is not stored takes through the serve entry, confirms the 25 checks on the pull request and adds the Tag Manager example to the open question on the guides. ## Revision record -| Date | Change | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | -| 2026-10-15 | Fixes from review round 1. The header and the pull requests it names, when each refusal comes, what the serve phase gives and costs, which readers get the output, where a prompt lands in the head and what that does to the consent order, what this amends in the InMobi design, failure, rollout and the adapters, Osano, the stock list's weaker test and the row that removes the stock list, what is recorded as checked and what no test covers, the open questions and the review log. | +| Date | Change | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-15 | Fixes from review round 1. The header and the pull requests it names, when each refusal comes, what the serve phase gives and costs, which readers get the output, where a prompt lands in the head and what that does to the consent order, what this amends in the InMobi design, failure, rollout and the adapters, Osano, the stock list's weaker test and the row that removes the stock list, what is recorded as checked and what no test covers, the open questions and the review log. | +| 2026-10-16 | The InMobi module refuses a prompt in `[[serve]]` for a page whose `[[fetch]]` entry names another middleware and a prompt named for one page in both phases, with tests that drive a page through both phases, and each of the three crates runs its middleware from a `[[serve]]` entry through the registry and a page, which settles the open questions on the order, on both phases and on the serve tests (sections 3, 4, 5, 6 and 7). The configuration and InMobi guides describe the refusals. Fixes from review round 2, being where a serve prompt lands, the route a page that is not stored takes, the checks confirmed and the Tag Manager example in the open questions. | From 5456bc0cc6b15e094670c31772171be923a840b0 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 14:08:34 +0100 Subject: [PATCH 117/192] The InMobi Choice design takes its round 2 review and describes the three fixes Entries that leave a page without the prompt are refused, a CMP id outside 1 to 4095 is refused naming cmp_id, and a middleware's own markup keys a stored page so a stub change stores it again. The round 2 points are applied, each checked against the code. --- .../2026-10-12-inmobi-choice-module-design.md | 124 +++++++++--------- 1 file changed, 65 insertions(+), 59 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md b/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md index 1152ea0430..dd8a7c2dd4 100644 --- a/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md +++ b/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md @@ -52,13 +52,13 @@ The module serves no browser module of its own (`without_js()`, line 183), becau **Where the markup lands.** Core writes the head markup of every middleware an entry names at the start of the first ``, in the order the entry names them. Core's own permission state script and ad slots script come first when core has them, and core's script bundle follows (`html_processor.rs` lines 591 to 631). A document with no `` gets none (`middleware.rs` lines 227 and 228). In the recorded page, `src/fixtures/page-change.recorded.html`, the three parts take 3,512 bytes ahead of the bundle's tag, and the page's own `` comes after them (section 6). -**The fetch phase alone.** The middleware declares the fetch phase and no other (`phases`, lines 230 to 232, held by the test `the_prompt_runs_in_the_fetch_phase_alone`). The fetch phase changes the page as the origin sent it and is told nothing about the reader (`middleware.rs` lines 51 to 54), so the tag is the same for every reader. Where a deployment stores pages, which needs `assembly_mode = "esi"` and the Fastly adapter (the crate that runs Trusted Server on Fastly), the stored page carries the tag and each reader's copy is made from the stored page. Everywhere else, the default `inline` mode included, the page is rewritten for each response and carries the tag in the same way (`docs/guide/configuration.md`, "Shared template assembly"). The serve phase changes each reader's copy instead. An entry of the serve phase that names `cmp.inmobi` is refused when the service starts (section 3), and row 21 ([IABTechLab/trusted-server#1362](https://github.com/IABTechLab/trusted-server/pull/1362)) lets the prompt run in either phase. +**The fetch phase alone.** The middleware declares the fetch phase and no other (`phases`, lines 230 to 232, held by the test `the_prompt_runs_in_the_fetch_phase_alone`). The fetch phase changes the page as the origin sent it and is told nothing about the reader (`middleware.rs` lines 51 to 54), so the tag is the same for every reader. Where a deployment stores pages, which needs `assembly_mode = "esi"` and the Fastly adapter (the crate that runs Trusted Server on Fastly), the stored page carries the tag and each reader's copy is made from the stored page. A stored page is keyed by, among other things, the text each middleware gives as its `template_key`, which for this module is the whole tag, so a change to a stub stores the pages again (`template_fingerprint_with_build_digest` in `publisher.rs`). Everywhere else, the default `inline` mode included, the page is rewritten for each response and carries the tag in the same way (`docs/guide/configuration.md`, "Shared template assembly"). The serve phase changes each reader's copy instead. An entry of the serve phase that names `cmp.inmobi` is refused when the service starts (section 3), and row 21 ([IABTechLab/trusted-server#1362](https://github.com/IABTechLab/trusted-server/pull/1362)) lets the prompt run in either phase. -**Which pages get the prompt.** An entry covers one media type and, optionally, the requests under a path prefix. A response takes the first entry of its phase that covers it and runs that entry's middleware alone (`PhaseEntries::for_response`, `middleware.rs` lines 396 to 404). The module therefore writes its tag on a page only where the first entry that covers the page names `cmp.inmobi`. At this row core accepts an entry for `text/html` alone (`PhaseEntries::validate`, lines 426 to 435), and row 19 ([IABTechLab/trusted-server#1360](https://github.com/IABTechLab/trusted-server/pull/1360)) lets an entry cover other media types, which is when the module's own check for `text/html` begins to matter (section 3). +**Which pages get the prompt.** An entry covers one media type and, optionally, the requests under a path prefix. A response takes the first entry of its phase that covers it and runs that entry's middleware alone (`PhaseEntries::for_response`, `middleware.rs` lines 396 to 404). The module therefore writes its tag on a page only where the first entry that covers the page names `cmp.inmobi`, and refuses entries that leave a page without it (section 3). At this row core accepts an entry for `text/html` alone (`PhaseEntries::validate`, lines 426 to 435), and row 19 ([IABTechLab/trusted-server#1360](https://github.com/IABTechLab/trusted-server/pull/1360)) lets an entry cover other media types, which is when the module's own check for `text/html` begins to matter (section 3). -**What a vendor's tag is, and the order.** Core writes the head markup of the middleware an entry names in the order the entry names them (`MiddlewareChain::plan`, `middleware.rs` lines 574 to 602), so the entry's order is the head's order. In this document a vendor's tag is head markup that another middleware writes for a vendor's code, being a script that loads the code or sets it up. Of the stock middleware, which are those of the modules this repository ships, two that run in the fetch phase write one, being `auction.prebid` (its configuration script and the tag that loads the Prebid bundle) and `ad-tag.google` (the `tsjs.adInit` bootstrap script). `audience.permutive`, `identity.lockr` and `tag.google-tag-manager` write no head markup, because they only rewrite addresses that the page's own markup already holds, so where they stand in an entry does not change where the prompt lands. Markup that a serve middleware writes lands straight before the script bundle, after everything a fetch middleware wrote (`html_processor.rs` lines 433 to 459), so a prompt in the fetch phase is ahead of that markup. +**What a vendor's tag is, and the order.** Core writes the head markup of the middleware an entry names in the order the entry names them (`MiddlewareChain::plan`, `middleware.rs` lines 574 to 602), so the entry's order is the head's order. In this document a vendor's tag is head markup that another middleware writes for a vendor's code, being a script that loads the code or sets it up. Of the stock middleware, which are those of the modules this repository ships, four that run in the fetch phase write one, being `auction.prebid` (its configuration script and the tag that loads the Prebid bundle), `ad-tag.google` (the `tsjs.adInit` bootstrap script), and the other consent prompts `cmp.didomi` and `cmp.sourcepoint`, which write the settings of their own browser modules, `window.__tsjs_didomi` and `window.__tsjs_sourcepoint`, and for Sourcepoint, when `rewrite_sdk` is set, a trap on `window._sp_` (`ClientConfig`, `crates/cmp/didomi/src/lib.rs` lines 598 to 615, and `PageChange` and `head_markup`, `crates/cmp/sourcepoint/src/lib.rs` lines 1039 to 1052 and from line 1078). A section of a module type may select several modules (`SectionModules`, `crates/trusted-server-core/src/provider_table.rs` lines 323 to 331), so a deployment can select Didomi or Sourcepoint beside InMobi, and the stock list names `cmp.didomi` ahead of `cmp.inmobi` (`crates/trusted-server-modules/src/lib.rs` lines 125 and 126). `audience.permutive`, `identity.lockr` and `tag.google-tag-manager` write no head markup, because they only rewrite addresses that the page's own markup already holds, so where they stand in an entry does not change where the prompt lands. Markup that a serve middleware writes lands straight before the script bundle, after everything a fetch middleware wrote (`html_processor.rs` lines 433 to 459), so a prompt in the fetch phase is ahead of that markup. -Nothing in the code checks that the prompt is ahead of a vendor's tag. The test `a_vendor_tag_named_after_the_prompt_is_written_after_it` shows that a vendor's tag named first is written first. The stock list names `auction.prebid` ahead of `cmp.inmobi` (`crates/trusted-server-modules/src/lib.rs` lines 118 to 126), as does the `[[fetch]]` entry in `trusted-server.example.toml` (lines 323 to 338), and `ts audit generate` writes entries in the order of the stock list (`crates/trusted-server-cli/src/commands/audit/generate/mod.rs` lines 665 to 697). The same example file says to name `cmp.inmobi` before any middleware that writes a vendor's tag (lines 774 to 777). Open question 4 asks about this. +Nothing in the code checks that the prompt is ahead of a vendor's tag. The test `a_vendor_tag_named_after_the_prompt_is_written_after_it` shows that a vendor's tag named first is written first. The stock list names `auction.prebid` ahead of `cmp.inmobi` (`crates/trusted-server-modules/src/lib.rs` lines 118 to 126), as does the `[[fetch]]` entry in `trusted-server.example.toml` (lines 323 to 338), and `ts audit generate` writes entries in the order of the stock list (`crates/trusted-server-cli/src/commands/audit/generate/mod.rs` lines 665 to 697). The same example file says to name `cmp.inmobi` before any middleware that writes a vendor's tag (lines 774 to 777). Open question 2 asks about this. **What a consent prompt module keeps to.** The guide page (`docs/guide/integrations/inmobi.md`, lines 89 to 107) states six things this module keeps to. @@ -67,7 +67,7 @@ Nothing in the code checks that the prompt is ahead of a vendor's tag. The test 3. The loader is deferred. 4. The reader's answer reaches the permission model through the permission signal modules, which are the modules that read a reader's consent signals for the permission model, and through no other path. 5. A script address written into every page is checked at load against what it is and not what it looks like. -6. A selected prompt that no entry places is refused. +6. A selected prompt that the entries leave off any page is refused. The guide says these are what every `[cmp]` module is held to, and the description of IABTechLab/trusted-server#1359 says they will be. Only this module keeps to them today, because Didomi, Sourcepoint and Osano each change a page in their own way ([IABTechLab/trusted-server#1289](https://github.com/IABTechLab/trusted-server/issues/1289)). Inside this module, code holds the fifth and the sixth (`validate_script_url` and `require_an_entry`), tests hold the second and the third, and the first holds only because the entry's order is the head's order. The fourth is a statement about where the prompt writes the answer, which no test covers (section 4). @@ -79,28 +79,29 @@ The guide says these are what every `[cmp]` module is held to, and the descripti The module reads the table `[cmp.inmobi]`, and the table refuses a field it does not name (`deny_unknown_fields`, line 58). -| Field | Type | Default | What the code does | -| ------------ | ------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `script_url` | String | Required | The account's Choice loader. Spaces around the value are removed. The value is refused unless it parses as a URL, has the scheme `https`, has a host that is `cmp.inmobi.com` or `cdn.inmobi.com` once lower-cased, and carries no user name or password (`validate_script_url`, lines 90 to 128). The check does not look at the port, the path or the query. | -| `cmp_id` | Number | `10` | The CMP id that the GPP stub reports. The code calls it the IAB registered identifier, which is 10 for InMobi (lines 66 to 76). Any value from 0 to 65535 is accepted (`u16`). | +| Field | Type | Default | What the code does | +| ------------ | ------ | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `script_url` | String | Required | The account's Choice loader. Spaces around the value are removed. The value is refused unless it parses as a URL, has the scheme `https`, has a host that is `cmp.inmobi.com` or `cdn.inmobi.com` once lower-cased, and carries no user name or password (`validate_script_url`, lines 90 to 128). The check does not look at the port, the path or the query. | +| `cmp_id` | Number | `10` | The CMP id that the GPP stub reports. The code calls it the IAB registered identifier, which is 10 for InMobi (lines 66 to 76). A whole number from 1 to 4095 is accepted, and any other value is refused when the settings load with a message naming `cmp_id` (`deserialize_cmp_id`), because a TC string holds the id in 12 bits and the GPP CMP API gives 0 as what a stub reports before it has an id. | A selected module with no `[cmp.inmobi]` table, or a table without `script_url`, is read as an empty table, which fails because `script_url` is missing (`Settings::module_config`, `settings.rs` lines 4496 to 4520). -The module refuses three things. +The module refuses four things. - A `script_url` that is empty, is not a URL, is not `https`, is on a host other than `cmp.inmobi.com` or `cdn.inmobi.com`, or carries a user name or password. The value becomes a `` into the head, last among its own head markup and after the shim and the bootstrap that the loader relies on, so the loader is fetched from the module's own first-party route. The setting is `false` by default, because a page that already carries the loader would load it twice. @@ -85,27 +87,22 @@ At least one of the three ids is required, because a table naming none asks the **Open questions.** Each of these is a question about the code. This spec records what the code does and leaves the questions open. -1. Should a setting leave out the noscript iframe? The module writes the iframe whenever `container_id` is set (`body_inserts` in `crates/tag/google-tag/src/lib.rs`), so a visitor whose browser has scripts turned off asks Google for `ns.html`, and no consent default can run there. -2. Should the module read the reader's permission state? The module declares no permission (`Middleware::required_permissions` returns none unless a middleware says otherwise), has no `prepare`, and ignores the context its `create` is handed, which holds no permission state. Google's loaders therefore reach every reader, including one whose request carries a Global Privacy Control signal. The question is whether to write them only for readers whose permission state allows it, or to record that they are written for every reader. -3. With `region` set, should the module also write a default with no region? Google's consent guide says a default command without a region sets the default for all visitors that no regional command covers, and the module writes one command that carries the regions, so a visitor outside the listed regions gets no default from this module. -4. Should one test run a page through both modules? All nine page tests select this module alone, so a loader that the Tag Manager module's fetch rewrite made first party is simulated by a page that already carries the first-party path (`a_loader_the_fetch_rewrite_made_first_party_is_removed_as_well`). -5. Should the crate have the `module_constant_is_the_crate_folder` test? `AGENTS.md` says each page integration holds its name in a `MODULE` constant that a test in the crate holds to the crate's folder, and 18 other crates have that test, but this crate has none. -6. Should the paths no test runs get tests? They are a document with no ``, where every head snippet is dropped and the noscript iframe is still written, a document with no ``, a `wait_for_update` of 0 and of 60001, an empty `body_open_inserts`, which registers no handler, and the AdSense loader after the tag snippet. -7. Should the placeholder in the example settings fail validation, and should `AGENTS.md` list `tag.google-tag`? `G-REPLACE-ME` in `trusted-server.example.toml` passes the tag id pattern, unlike the Tag Manager example's `GTM-REPLACE-ME`, which is invalid on purpose, and lines 404 and 445 of `AGENTS.md` list the other page integrations without `tag.google-tag`. -8. Should the check the Tag Manager route makes call the function this module calls? The rule for which tag ids the Tag Manager module serves is written in `tag_id_is_allowed` and again in `first_party_gtag_js_path`, so the two agree only while their text stays the same. +1. Should the paths no test runs get tests? They are a document with no ``, a `wait_for_update` of 0 and of 60001, an empty `body_open_inserts`, which registers no handler, and the AdSense loader after the tag snippet. +2. Should the check the Tag Manager route makes call the function this module calls? The rule for which tag ids the Tag Manager module serves is written in `tag_id_is_allowed` and again in `first_party_gtag_js_path`, so the two agree only while their text stays the same. ## 4. Acceptance -CI passed on all 25 checks of the draft pull request at `1940d63f1`, and on all 19 checks of the fork's closed pull request at the same commit. The description of the pull request lists the checks that were run, says that Google's own handling of the snippets was not exercised, and leaves its manual test with `fastly compute serve` unticked. The module's 32 tests, one test in core, one in the seam probe and three in the ad tag module hold the claims in section 2 as follows. +CI passed on all 25 checks of the draft pull request at `1940d63f1`, and on all 19 checks of the fork's closed pull request at the same commit. The description of the pull request lists the checks that were run, says that Google's own handling of the snippets was not exercised, and leaves its manual test with `fastly compute serve` unticked. The code fixes of 2026-10-16 sit on top of that commit and have not yet been through CI. Locally the crate's 41 tests pass, clippy on the crate with warnings denied passes and `cargo fmt --all -- --check` passes, and each fix's new tests fail with that fix reverted. The module's 41 tests, one test in core, one in the seam probe and three in the ad tag module hold the claims in section 2 as follows. - **Each snippet is Google's text with only the id substituted.** For the tag snippet, `the_tag_snippet_is_googles_own_with_the_id_substituted`. For the AdSense loader, `the_adsense_loader_is_googles_own_with_the_account_substituted`. For the container script, `the_container_script_is_googles_own_with_the_id_substituted`. For the noscript iframe, `the_container_noscript_is_googles_own_with_the_id_substituted`. The consent snippet has no such test. -- **The consent defaults.** `every_consent_type_defaults_to_denied`, `the_tag_waits_for_the_platform_s_update_before_it_fires` (500 and no region), `a_publisher_may_scope_the_defaults_to_regions_and_set_the_wait`, `a_publisher_may_grant_a_consent_type_up_front` and `a_region_that_is_not_a_code_is_refused_by_name`. +- **The consent defaults.** `every_consent_type_defaults_to_denied`, `the_tag_waits_for_the_platform_s_update_before_it_fires` (500 and no region), `a_publisher_may_scope_the_defaults_to_regions_and_set_the_wait`, `a_publisher_may_grant_a_consent_type_up_front`, `a_region_that_is_not_a_code_is_refused_by_name` and `a_region_setting_leaves_every_other_visitor_with_all_four_denied`, which holds the second command with no region. +- **Who gets the loaders.** `a_reader_who_has_not_allowed_storage_gets_the_consent_defaults_alone`, `a_reader_who_allows_storage_gets_every_configured_loader` and `no_loader_is_written_where_no_permission_state_reached_the_middleware` plan the middleware through core's serve preparation. `a_page_for_a_reader_who_has_not_agreed_carries_no_google_loader` runs a page for a reader in Germany who has given no signal and a reader in California who has opted out, under the policy core ships. The page tests prepare both phases as core does for a page it does not store. - **The order within the module's markup.** `consent_defaults_come_before_the_config_command`, `consent_defaults_come_before_the_container` and `a_container_and_a_tag_are_both_written_container_first`. - **Each id on its own.** `each_id_on_its_own_writes_the_consent_defaults_and_its_own_snippet`. - **Where each loader points.** `the_loader_names_googles_host_when_tag_manager_is_not_enabled`, `the_loader_names_the_first_party_path_when_tag_manager_serves_the_tag`, `the_loader_names_googles_host_when_tag_manager_would_not_serve_the_tag`, `the_container_loader_names_googles_host_when_tag_manager_is_not_enabled`, `the_container_loader_names_the_first_party_path_for_the_same_container` and `the_container_loader_names_googles_host_for_a_different_container`. -- **The page's own loader.** `a_gtag_loader_is_recognised_however_it_is_written`, `a_page_already_loading_the_same_tag_ends_with_one_loader`, `a_loader_the_fetch_rewrite_made_first_party_is_removed_as_well` and `a_page_loading_another_tag_keeps_it`. -- **Placement on a page.** `the_snippets_land_in_the_head_once` and `the_snippet_reaches_a_real_shaped_head_before_the_publishers_own_scripts` hold the head snippets once, before the page's own head content, with no `[[fetch]]` middleware present. `the_noscript_lands_immediately_after_the_opening_body_tag_once`, `a_second_body_does_not_get_a_second_noscript` and `a_page_with_no_container_configured_is_left_alone_at_the_body` hold the noscript iframe. -- **The settings.** `an_unnamed_integration_does_not_register`, `a_config_naming_no_id_is_refused`, `an_id_of_the_wrong_shape_is_refused_by_name` and `an_unknown_setting_is_refused`. +- **The page's own loader.** `a_gtag_loader_is_recognised_however_it_is_written`, `a_page_already_loading_the_same_tag_ends_with_one_loader`, `a_loader_the_fetch_rewrite_made_first_party_is_removed_as_well` and `a_page_loading_another_tag_keeps_it`. `a_page_through_both_modules_ends_with_one_first_party_loader` runs the Tag Manager module's `[[fetch]]` middleware and this module's `[[serve]]` middleware on one page, which ends with one loader, on the first-party path. +- **Placement on a page.** `the_snippets_land_in_the_head_once` and `the_snippet_reaches_a_real_shaped_head_before_the_publishers_own_scripts` hold the head snippets once, before the page's own head content, with no `[[fetch]]` middleware present. `the_noscript_lands_immediately_after_the_opening_body_tag_once`, `a_second_body_does_not_get_a_second_noscript` and `a_page_with_no_container_configured_is_left_alone_at_the_body` hold the noscript iframe. `a_page_with_no_head_gets_the_noscript_and_none_of_the_head_snippets` holds a document with no ``. +- **The settings.** `an_unnamed_integration_does_not_register`, `a_config_naming_no_id_is_refused`, `an_id_of_the_wrong_shape_is_refused_by_name`, `an_unknown_setting_is_refused` and `the_example_s_placeholder_ids_are_refused`, which reads the tag and container placeholders from `trusted-server.example.toml`. `module_constant_is_the_crate_folder` holds `MODULE` to the crate's folder. - **The new place in core.** `a_body_open_insert_is_written_once_after_the_opening_body_tag` in `html_processor.rs` drives the serve processor with two inserts and a second ``. The seam probe's middleware writes a marker immediately after the opening body tag, and its page test, `the_middleware_changes_a_page_where_an_entry_names_it`, reads the marker back once, directly after ``, through the fetch processor. - **The ad tag module's loader.** `head_inserts_write_the_loader_tag_when_asked`, `head_inserts_write_the_loader_tag_last` and `head_inserts_write_no_loader_tag_by_default` in `crates/ad-tag/google/src/lib.rs`. @@ -116,8 +113,9 @@ CI passed on all 25 checks of the draft pull request at `1940d63f1`, and on all ## Revision record -| Date | Change | -| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | -| 2026-10-16 | Fixes from review round 1. The header and the pull request pointers, where the head snippets land, the reason for the stock list order, the settings, the consent defaults, the Tag Manager module, the ad tag module's loader setting, failure, adapters, compatibility, the content security policy, the tests, the open questions and the review log. | -| 2026-10-16 | Fixes from review round 2. The content security policy and the source of the consent snippet. | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-16 | Fixes from review round 1. The header and the pull request pointers, where the head snippets land, the reason for the stock list order, the settings, the consent defaults, the Tag Manager module, the ad tag module's loader setting, failure, adapters, compatibility, the content security policy, the tests, the open questions and the review log. | +| 2026-10-16 | Fixes from review round 2. The content security policy and the source of the consent snippet. | +| 2026-10-16 | Code fixes from review round 1, code findings 1 to 5 and 7 and the case with no head in finding 6. The loaders and the noscript iframe are written only for a reader whose permissions allow storage on the device, a region setting gives every other visitor a default with all four types denied, the example's tag id placeholder fails validation and `AGENTS.md` lists the module. Tests run a page through both modules and a document with no head, and hold the crate's name to its folder. Open questions 1 to 5 and 7 and the case with no head in 6 leave the list. | From a5ea93f7e856ae4cd873d7596a2fd636a8203f3d Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Fri, 16 Oct 2026 19:51:35 +0100 Subject: [PATCH 126/192] The device signals design describes the five fixes The device module's attributes call carries the request's permissions, a backend it registers waits no longer than the auction, core drops values a bidder cannot read, a request the consent gate denies never reaches the module, and two comments say only what the code does. The open questions those answer move into the body. --- .../specs/2026-10-12-device-signals-design.md | 68 +++++++++---------- 1 file changed, 32 insertions(+), 36 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-device-signals-design.md b/docs/superpowers/specs/2026-10-12-device-signals-design.md index aa3677ca1d..3b0f6060d4 100644 --- a/docs/superpowers/specs/2026-10-12-device-signals-design.md +++ b/docs/superpowers/specs/2026-10-12-device-signals-design.md @@ -13,7 +13,7 @@ A device module answers one question, whether the request looks like a browser, and the Edge Cookie gate reads the answer. A module that reaches a detection service knows far more, including how confident it is that a person is looking at the page and what the device is, and a bidder prices inventory on the device it is told about. Core has no place for either answer, so a module's knowledge stops at the gate and a bid request leaves without the device. -Paths and line numbers in this document are those of `b680d0780`, the head of IABTechLab/trusted-server#1369, unless a row is named. +Paths and line numbers in this document are those of `b680d0780`, the head of IABTechLab/trusted-server#1369, unless a row or `4badbdaf8` is named. `4badbdaf8` ends the five commits after `b680d0780` that fix what the reviews found in the code (see the revision record). ## 2. Design @@ -26,32 +26,32 @@ Only the Fastly adapter calls `detect` in production, once for each request at i **The attributes.** `DeviceAttributes` is what a bidder is told about the device, being the OpenRTB device type, make, model, operating system and version, and screen width and height (`device.rs`, lines 41 to 65). Every field is optional because every source is partial. The device type and the two screen sizes are `i32`, a 32-bit whole number, and the other four fields are strings. `DeviceModule::advertising_attributes` gives the module's answer, and the default answers nothing, so no existing module changes (line 273). What core does with the answer, in the order it happens: 1. **Which module is asked.** Core asks only the module that `RuntimeServices::device_module` returns, being a module that a registration supplied and `[device] module` names. A deployment that leaves the setting unset, or writes `builtin` or `fastly`, resolves no such module (`crates/trusted-server-core/src/integrations/registry.rs`, line 1226), so core makes no call and the built-in module and the Fastly module are never asked. -2. **Where core asks.** `attach_device_attributes` (`crates/trusted-server-core/src/auction/formats.rs`, line 560) runs at three points, each after the auction request is built and before the auction starts. The points are the auction endpoint (`crates/trusted-server-core/src/auction/endpoints.rs`, line 359), the publisher's server-side path (`crates/trusted-server-core/src/publisher.rs`, line 5544) and `handle_page_bids` (`publisher.rs`, line 7919). Each point is reached only after `consent_allows_server_side_auction` has allowed the auction, so a request on the consent-denied path, which makes no outbound call, does not ask. The auction endpoint's two early returns, for a disabled auction (`endpoints.rs`, lines 185 to 226) and for denied consent (lines 235 to 279), build a request only to echo its id and ask nothing. -3. **What core does with the answer.** Core attaches the answer unless the answer is `None` or has no field set (`is_empty`, `device.rs`, line 74). Such an answer leaves the request as it was and creates no device object, because an absent object and an empty one say the same to a bidder and the absent one is smaller. Any other answer is stored in the `attributes` field of the request's device information (`crates/trusted-server-core/src/auction/types.rs`, line 123). Where the request has no device information, core creates the device information holding only the attributes. Core does not check what the values are. +2. **Where core asks.** `attach_device_attributes` (`crates/trusted-server-core/src/auction/formats.rs`, line 560) runs at three points, each after the auction request is built and before the auction starts. The points are the auction endpoint (`crates/trusted-server-core/src/auction/endpoints.rs`, line 359), the publisher's server-side path (`crates/trusted-server-core/src/publisher.rs`, line 5544) and `handle_page_bids` (`publisher.rs`, line 7919). Each point is reached only after `consent_allows_server_side_auction` has allowed the auction, so a request on the consent-denied path, which makes no outbound call, does not ask. The auction endpoint's two early returns, for a disabled auction (`endpoints.rs`, lines 185 to 226) and for denied consent (lines 235 to 279), build a request only to echo its id and ask nothing. At `4badbdaf8` each point also hands over the permissions resolved for the request and the auction's own timeout, being `[auction] timeout_ms` at the auction endpoint and, on the other two, `[creative_opportunities] auction_timeout_ms` where it is set (`endpoints.rs`, lines 359 to 367, and `publisher.rs`, lines 5544 to 5552 and 7920 to 7932). +3. **What core does with the answer.** Core attaches the answer unless the answer is `None` or has no field set (`is_empty`, `device.rs`, line 74). Such an answer leaves the request as it was and creates no device object, because an absent object and an empty one say the same to a bidder and the absent one is smaller. Any other answer is stored in the `attributes` field of the request's device information (`crates/trusted-server-core/src/auction/types.rs`, line 123). Where the request has no device information, core creates the device information holding only the attributes. At `4badbdaf8` core first drops each value a bidder cannot read, being a device type outside the device types list of AdCOM 1.0, which OpenRTB 2.6 names for `devicetype` and which runs from 1 to 8, a screen width or height that is not positive, and a make, model, operating system or version that is empty or only white space. Core logs a warning naming the module and the field for each value it drops, and an answer left with no field set adds no device object (`DeviceAttributes::checked`, `device.rs`, lines 81 to 130, called at `formats.rs`, line 603). 4. **How the bid request carries the answer.** The common OpenRTB driver writes the fields into the device object as `devicetype`, `make`, `model`, `os`, `osv`, `w` and `h`, each absent where nothing resolved it (`crates/trusted-server-core/src/auction/openrtb.rs`, lines 408 to 419). The driver builds the object for every demand source, so no demand source maps a field, and no setting drops the attributes for one source (`crates/trusted-server-core/src/auction/demand.rs`, lines 62 to 88). Core stores the attributes nowhere. Row 45 moves the driver to `crates/auction-protocol/openrtb`, where the driver writes the same fields. 5. **The device type is OpenRTB's own number.** A module gives the type as the number in OpenRTB's device type table, and core maps no name to a number, so core holds no detection vendor's names. A field the module could not resolve stays absent and core invents no default, because a bidder reads an absent field as unknown and a wrong value would misprice the inventory. -6. **When the module cannot answer.** `advertising_attributes` returns an `Option` and has no other way to report a failure, so a module that cannot reach its service returns `None`, and core cannot tell that answer from the answer of a module with nothing to say. Core logs neither case. Core sets no time limit on the call, and each of the three handlers waits for the call, so the time a module takes is added to the time before the auction starts. +6. **When the module cannot answer.** `advertising_attributes` returns an `Option` and has no other way to report a failure, so a module that cannot reach its service returns `None`, and core cannot tell that answer from the answer of a module with nothing to say. Core logs neither case. Each of the three handlers waits for the call, so the time a module takes is added to the time before the auction starts. At `4badbdaf8` the services in the call register backends through `CappedBackend`, which caps the first-byte and between-bytes timers of every backend the module registers during the call at the auction's own timeout (`formats.rs`, lines 588 to 591 and 621 to 659, and `RuntimeServices::with_backend`, `crates/trusted-server-core/src/platform/types.rs`, line 431). The two timers bound how long a service may take to start answering and to send each further part, not the whole call, so connecting, a slow trickle of bytes, a backend registered before the call and work done without a backend are not bounded, as they are not for an auction provider. Core has no clock that could stop a call, and no adapter enforces a total deadline on a request. The cap binds where the adapter applies a backend's timers, which today is Fastly only. The Axum adapter sends to the address and ignores backend names, and the Cloudflare and Spin adapters register no backend, so the cap changes nothing on those three. 7. **Two calls for two answers.** Core calls `detect` and `advertising_attributes` separately, each with a module call of its own, being the module context together with the permissions the module declares, and the second call carries no device signals. A module that reaches an outside service to give both answers makes a call for each request on Fastly and another for each auction request, unless the module keeps the first answer itself. -**Signals and attributes stay apart.** `DeviceAttributes` is a type of its own and not a field of `DeviceSignals`, on purpose (`device.rs`, lines 29 to 39). Five of the signals reach the identity graph row through `to_kv_device`. The attributes answer what a bidder prices, and core stores them nowhere. +**Signals and attributes stay apart.** `DeviceAttributes` is a type of its own and not a field of `DeviceSignals`, on purpose (`device.rs`, lines 29 to 39). Five of the signals reach the identity graph row through `to_kv_device`. The attributes answer what a bidder prices, and core stores them nowhere. At `4badbdaf8` the comment on `DeviceAttributes` no longer says that the identity path serialises the signals, which derive no `Serialize`, and the comment on `DeviceInfo::attributes` no longer says that a stored request from before the field still reads, since no code reads a stored `AuctionRequest` (`device.rs`, lines 29 to 38, and `types.rs`, lines 116 to 121). ## 3. Consent and privacy Core now hands request data to a module in a new call and sends the module's answer to bidders. This section says what the call carries and what stands in front of the call and of the answer. -**What the module is handed.** `attach_device_attributes` builds a module call that carries the request line (method, host, scheme, path and query), the evidence, the settings and the services (`formats.rs`, lines 566 to 581). The evidence is every header on the request that the handler holds, and the client address, which is an empty string where none is known. The services include the HTTP client, so a module that reaches a detection service can send that service any of the evidence. The call carries no consent signals, no resolved permissions, no location, no device signals and no Edge Cookie identifier, although the handlers hold the first two. +**What the module is handed.** `attach_device_attributes` builds a module call that carries the request line (method, host, scheme, path and query), the evidence, the settings and the services (`formats.rs`, lines 566 to 581). The evidence is every header on the request that the handler holds, and the client address, which is an empty string where none is known. The services include the HTTP client, so a module that reaches a detection service can send that service any of the evidence. The call carries no consent signals, no resolved permissions, no location, no device signals and no Edge Cookie identifier, although the handlers hold the first two. At `4badbdaf8` the call carries the permissions resolved for the request (`formats.rs`, line 594), so the module can see what the reader has agreed to, and it still carries no consent signals, no location, no device signals and no Edge Cookie identifier. It carries the permissions and not the signals because a module sees only the permissions the signals resolve to (`docs/guide/permission-model.md`, the section Permission sources). The call is the same kind of call as the one `detect` is handed, but the call carries more than the call that `detect` gets on Fastly, where `derive_device_signals` shows `detect` the User-Agent header, the `Cookie` header only when a registered module is selected, and the client address (`crates/trusted-server-adapter-fastly/src/lib.rs`, lines 1054 to 1071). The other three adapters never call `detect`. -**What stands in front of the attributes.** The only consent gate is the one that lets the auction run, `consent_allows_server_side_auction` (`crates/trusted-server-core/src/consent/mod.rs`, lines 328 to 338). For a request that carries an EU signal from the IAB Transparency and Consent Framework (TCF), or comes from a GDPR or unknown jurisdiction, the function needs TCF Purpose 1, which is consent to store or read information on the device. For a request from a known jurisdiction outside the GDPR that carries no EU TCF signal, the function returns true, and the function does not read the Global Privacy Control signal. +**What stands in front of the attributes.** The only consent gate is the one that lets the auction run, `consent_allows_server_side_auction` (`crates/trusted-server-core/src/consent/mod.rs`, lines 328 to 338). For a request that carries an EU signal from the IAB Transparency and Consent Framework (TCF), or comes from a GDPR or unknown jurisdiction, the function needs TCF Purpose 1, which is consent to store or read information on the device. For a request from a known jurisdiction outside the GDPR that carries no EU TCF signal, the function returns true, and the function does not read the Global Privacy Control signal. The permissions the call carries at `4badbdaf8` are where a deployment's policy turns that signal into an opt-out. In the sample policy an opt-out from Global Privacy Control revokes device storage, sale, sharing, targeted advertising and profiling and leaves contextual advertising (`config/permissions/sample.yaml`, lines 542 to 551), so a module whose use needs one of the revoked Data Uses sees the opt-out in its call. -**Permissions.** The attributes are attached under no permission check, as the User-Agent (`ua`) and the client address (`ip`) are and unlike EIDs, the extended user identifiers a request may carry. EIDs pass `gate_eids_by_permissions` at both places that attach them (`endpoints.rs`, line 365, and `publisher.rs`, lines 6274 and 6275), while the User-Agent and the client address are set without a check (`formats.rs`, lines 219 to 230, and `publisher.rs`, lines 6282 to 6296 and 6339 to 6344). A device module cannot ask for a permission, because the registry refuses to start with a device module that declares one (`registry.rs`, lines 1244 to 1260). +**Permissions.** The attributes are attached under no permission check, as the User-Agent (`ua`) and the client address (`ip`) are and unlike EIDs, the extended user identifiers a request may carry. EIDs pass `gate_eids_by_permissions` at both places that attach them (`endpoints.rs`, line 365, and `publisher.rs`, lines 6274 and 6275), while the User-Agent and the client address are set without a check (`formats.rs`, lines 219 to 230, and `publisher.rs`, lines 6282 to 6296 and 6339 to 6344). A device module cannot ask for a permission, because the registry refuses to start with a device module that declares one (`registry.rs`, lines 1244 to 1260). `4badbdaf8` adds no core rule about the attributes and leaves that refusal in place. Instead the module receives the permission state, and a module whose use of the evidence needs a permission the reader has not given answers `None` without using the evidence, as the documentation of `advertising_attributes` says (`device.rs`, lines 322 to 333). Core does not hold a module to that, because a device module declares no permission for core to check. **The score.** The score is in no bid request, no stored row and no gate, so at this row the score changes nothing that a bidder, the identity graph or the Edge Cookie gate sees. ## 4. What changes for a deployment -**Settings.** No setting is new. `[device] module` already selects the device module. A deployment that leaves the setting unset, or writes `builtin` or `fastly`, sends the bid requests it sent before, because core makes no call. Row 49 renames the `fastly` value to `platform`. A deployment that writes the name of a module that a crate outside core supplies has that module asked at the three points, so the module's attributes reach the device object of every bid request to every demand source as soon as the module is selected. No setting limits the attributes to one demand source or turns the attributes off while the module stays selected. The configuration guide (`docs/guide/configuration.md`, line 860) describes both answers. The integration guide (`docs/guide/integration-guide.md`, line 647) describes only `DeviceModule::advertising_attributes` and does not mention the score. +**Settings.** No setting is new. `[device] module` already selects the device module. A deployment that leaves the setting unset, or writes `builtin` or `fastly`, sends the bid requests it sent before, because core makes no call. Row 49 renames the `fastly` value to `platform`. A deployment that writes the name of a module that a crate outside core supplies has that module asked at the three points, so the module's attributes reach the device object of every bid request to every demand source as soon as the module is selected. No setting limits the attributes to one demand source or turns the attributes off while the module stays selected. The configuration guide (`docs/guide/configuration.md`, line 860) describes both answers. The integration guide (`docs/guide/integration-guide.md`, line 647) describes only `DeviceModule::advertising_attributes` and does not mention the score. At `4badbdaf8` that row says what the call carries, in place of saying it is the same call as `detect`. **Adapters.** All four adapters put the selected device module into each request's services and call all three handlers, so the attributes work on all four. Only Fastly calls `detect`, so only Fastly can write the score. Only the Axum adapter has a test of the new answers. @@ -62,25 +62,27 @@ The call is the same kind of call as the one `detect` is handed, but the call ca | Fastly | `crates/trusted-server-adapter-fastly/src/app.rs`, line 484 | Yes, `crates/trusted-server-adapter-fastly/src/lib.rs`, line 1089 | None | | Spin | `crates/trusted-server-adapter-spin/src/app.rs`, line 308 | No | None | -**What breaks for code.** `DeviceSignals` and `DeviceInfo` each gain a public field, `human_confidence` and `attributes`, and both are built with struct literals, so a literal of either that does not name the new field stops compiling. In this repository that meant editing nine files, six in core (`auction/formats.rs`, `auction/routing.rs`, `auction/telemetry.rs`, `auction/test_support.rs`, `integrations/registry.rs` and `publisher.rs`) and three outside core (the demonstration ad server, the Prebid Server crate and the seam probe). A `DeviceModule` written before this change still compiles, because `advertising_attributes` has a default. The changelog lists the change under Added and names `DeviceSignals::human_confidence` and `DeviceModule::advertising_attributes`, but it does not name `DeviceInfo::attributes` and has no note that a struct literal of either type stops compiling (`CHANGELOG.md`, line 90). +**What breaks for code.** `DeviceSignals` and `DeviceInfo` each gain a public field, `human_confidence` and `attributes`, and both are built with struct literals, so a literal of either that does not name the new field stops compiling. In this repository that meant editing nine files, six in core (`auction/formats.rs`, `auction/routing.rs`, `auction/telemetry.rs`, `auction/test_support.rs`, `integrations/registry.rs` and `publisher.rs`) and three outside core (the demonstration ad server, the Prebid Server crate and the seam probe). A `DeviceModule` written before this change still compiles, because `advertising_attributes` has a default. The changelog lists the change under Added and names `DeviceSignals::human_confidence` and `DeviceModule::advertising_attributes`, but it does not name `DeviceInfo::attributes` and has no note that a struct literal of either type stops compiling (`CHANGELOG.md`, line 90). At `4badbdaf8` the public function `attach_device_attributes` also takes the permissions and the timeout, so a caller outside core would have to pass both, and no caller outside core exists in this repository. The changelog entry at `4badbdaf8` also says what the call carries and what core caps and drops. ## 5. What is not in this change -- A module that scores a person or resolves a device. The stock build's two device modules, the built-in one and the Fastly one, read the User-Agent and the host's signals, and answer `Unknown` for the score and nothing for the attributes. The seam probe's module is the one module that answers either. +- A module that scores a person or resolves a device. The stock build's two device modules, the built-in one and the Fastly one, read the User-Agent and the host's signals, and answer `Unknown` for the score and nothing for the attributes. The seam probe's module is the one module that answers either, beside the test double `DetectionServiceDevice` that `4badbdaf8` adds for the handler tests (`crates/trusted-server-core/src/platform/test_support.rs`, line 1072), which answers attributes and no score. - A reader of the score. No production code reads the score (section 2). -- A check of a module's values, a time limit on the call, or a change to what the call carries (section 7). +- A core gate on the attributes like the one EIDs pass, and a limit on the whole call rather than on the timers of the module's backends (sections 2 and 3). - A setting. ## 6. Acceptance -**What the tests show.** Four groups of tests cover the change, and none of them runs a handler. +**What the tests show.** At `b680d0780` four groups of tests cover the change, and none of them runs a handler. `4badbdaf8` adds the last two groups in the table, and the first group runs at `4badbdaf8` with the permissions and the timeout passed as well. -| Tests | What the tests show | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Five tests of `attach_device_attributes` (`formats.rs`, lines 691 to 823), four with a stand-in module that answers fixed attributes, empty attributes or nothing and one with no module | A module's attributes fill the device object, an answer of `None` or with every field absent adds none, a request with no selected module is left as it was, and an answer creates the device object where the request had none | -| Two tests of the OpenRTB driver (`crates/trusted-server-core/src/auction/openrtb/tests.rs`, lines 1171 to 1227) | Resolved attributes reach the device object of every implementation, and fields that nothing resolved stay absent | -| `a_request_alone_establishes_no_person` (`device.rs`, line 904) | Every way that core makes device signals leaves the score `Unknown` | -| The Axum adapter's test `the_resolved_device_module_answers_a_confidence_and_the_devices_attributes` (`crates/trusted-server-adapter-axum/tests/seam_probe.rs`, line 944) | `[device] module = "testing.seam-probe"` resolves the seam probe's device module, and the module answers what the module was written to answer | +| Tests | What the tests show | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Five tests of `attach_device_attributes` (`formats.rs`, lines 691 to 823), four with a stand-in module that answers fixed attributes, empty attributes or nothing and one with no module | A module's attributes fill the device object, an answer of `None` or with every field absent adds none, a request with no selected module is left as it was, and an answer creates the device object where the request had none | +| Two tests of the OpenRTB driver (`crates/trusted-server-core/src/auction/openrtb/tests.rs`, lines 1171 to 1227) | Resolved attributes reach the device object of every implementation, and fields that nothing resolved stay absent | +| `a_request_alone_establishes_no_person` (`device.rs`, line 904) | Every way that core makes device signals leaves the score `Unknown` | +| The Axum adapter's test `the_resolved_device_module_answers_a_confidence_and_the_devices_attributes` (`crates/trusted-server-adapter-axum/tests/seam_probe.rs`, line 944) | `[device] module = "testing.seam-probe"` resolves the seam probe's device module, and the module answers what the module was written to answer | +| Nine tests at `4badbdaf8` that run the handlers with the test double `DetectionServiceDevice`, five on the auction endpoint (`endpoints.rs`, lines 2120 to 2250) and four on the publisher's path and page-bids (`publisher.rs`, lines 29048 to 29127) | Each of `handle_auction`, `handle_publisher_request` and `handle_page_bids` asks the selected module with the reader's permissions and sends its attributes to the bidder, and none of them asks the module when consent denies the auction. On the auction endpoint, a module whose permission the reader has not given adds nothing, a backend the module registers has both timers capped at `[auction] timeout_ms`, and a device type outside the list, a size that is not positive and a blank string do not reach the bidder | +| `checked_keeps_only_the_values_a_bidder_can_read` (`device.rs`, line 978) and `an_answer_with_no_value_a_bidder_can_read_adds_no_device_object` (`formats.rs`, line 754), at `4badbdaf8` | Device types 1 and 8 are kept and 0 and 9 are dropped, a size of 1 is kept and 0 and -1 are dropped, empty and white space strings are dropped, and an answer with nothing left adds no device object | The seam probe's device module scores the request `Medium` and answers fixed attributes naming a tablet (`crates/testing/seam-probe/src/lib.rs`, lines 381 and 394 to 405). The Axum test builds its own module call from a request line and the settings (`tests/seam_probe.rs`, lines 955 to 961) and calls `detect` and `advertising_attributes` on the module directly. The test drives no auction and not the call that core builds, and the tablet type that the test reads, device type 5, is the probe's own literal (`seam-probe/src/lib.rs`, line 397). The test shows that the selector resolves a module from a crate outside core, and shows nothing about how core calls the module. Row 43 renames the seam probe crate to `iabtl`. @@ -88,24 +90,17 @@ The seam probe's device module scores the request `Medium` and answers fixed att **Local checks.** The description of IABTechLab/trusted-server#1369 records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run. The description marks a manual run with `fastly compute serve` as not done. -**What no test covers.** +**What no test covers.** At `4badbdaf8` the handler tests cover the three points with consent allowed and denied, so deleting one of the three calls, or adding a call on a path that makes no outbound call, fails a test, and the tests give core out-of-range numbers and blank strings. What remains: -1. **The three points.** No test runs `handle_auction`, `handle_publisher_request` or `handle_page_bids` with a device module in the request's services, with consent allowed or denied. The tests of `attach_device_attributes` call the function directly, so deleting one of the three calls, or adding a call on a path that makes no outbound call, fails no test. -2. **What the call carries.** The core tests pass a request with no headers (`empty_request`, `formats.rs`, line 666) and the Axum test builds its own call, so no test reads the evidence, the settings or the services that `attach_device_attributes` hands a module. -3. **The adapters.** No test in any adapter reads the device module off a request's services, and the Cloudflare, Fastly and Spin adapters have no test of the new answers. The Fastly adapter's test of `derive_device_signals` with the seam probe reads `platform_class` and not the score (`crates/trusted-server-adapter-fastly/src/app/seam_probe_tests.rs`, lines 595 to 650). -4. **A module's values.** No test gives core an out-of-range number or a blank string in a module's answer, so no test says what core does with either (section 7, question 2). +1. **What the call carries.** The core tests pass a request with no headers (`empty_request`, `formats.rs`, line 666) and the Axum test builds its own call. At `4badbdaf8` the handler tests read the permissions in the call and register a backend through its services, but no test reads the evidence or the settings that `attach_device_attributes` hands a module. +2. **The adapters.** No test in any adapter reads the device module off a request's services, and the Cloudflare, Fastly and Spin adapters have no test of the new answers. The Fastly adapter's test of `derive_device_signals` with the seam probe reads `platform_class` and not the score (`crates/trusted-server-adapter-fastly/src/app/seam_probe_tests.rs`, lines 595 to 650). No test runs the timer cap on Fastly, the one adapter where it binds. ## 7. Open questions These are for the reviewer. This revision records what the code does and decides none of them. -1. **A test of the three points.** Should a test run each of `handle_auction`, `handle_publisher_request` and `handle_page_bids` with a device module in the services, and one with consent denied? Today no test does (section 6), so removing one call, or adding a call on a path that makes no outbound call, fails nothing. -2. **Limits on a module's values.** Should core refuse a device type outside OpenRTB's table, a screen size that is not positive and a blank string? Today `device_type`, `screen_width` and `screen_height` take any `i32` and core forwards the number (`openrtb.rs`, lines 410, 418 and 419), and `is_empty` counts `Some("")` as a value, so an answer that holds only an empty `make` is attached and sent. -3. **A time limit.** Should core limit how long the handlers wait for `advertising_attributes`? The three handlers wait for the call before the auction starts (`publisher.rs`, line 5544), so a slow detection service delays the response. -4. **What the call carries.** Should the call carry the consent signals and the resolved permissions, which the handlers hold, and should the attributes pass the permission gate that EIDs pass? Today the call carries every header and the client address and neither of the two, and no device module can ask for a permission because the registry refuses one that declares a permission. The consent gate in front of the call does not read the Global Privacy Control signal (section 3). -5. **Two comments.** The comment on `DeviceAttributes` says that merging the two answers would put advertising fields into "a type the identity path serialises" (`device.rs`, lines 34 and 35), but `DeviceSignals` derives no `Serialize` (line 84) and the identity path writes `KvDevice`. The comment on `DeviceInfo::attributes` says that a stored request from before the field existed "still reads" (`types.rs`, lines 119 to 121), but a search of the tree finds no code that deserializes an `AuctionRequest`. Should the comments change, or does a reader of stored requests exist that the search missed? -6. **Where the score belongs.** Section 2 keeps the signals apart from the attributes by what each answers and by whether core stores each. The score answers a question about a person and not about a browser, and core stores the score nowhere, so the score matches the attributes on both counts, yet the score sits in the signals that `detect` returns. Neither the code nor IABTechLab/trusted-server#1286 says why. What is the reason for placing the score with the signals? -7. **The guide and the changelog.** The integration guide says that `advertising_attributes` is handed "the same call as `detect`" (`docs/guide/integration-guide.md`, line 647), which holds for the kind of call and not for what the call carries (section 3). The changelog lists the change under Added and names `DeviceSignals::human_confidence` and `DeviceModule::advertising_attributes`, but it does not name `DeviceInfo::attributes` and gives no breaking-change note for the two new public fields that stop struct literals compiling (`CHANGELOG.md`, line 90). Should either change? +1. **Where the score belongs.** Section 2 keeps the signals apart from the attributes by what each answers and by whether core stores each. The score answers a question about a person and not about a browser, and core stores the score nowhere, so the score matches the attributes on both counts, yet the score sits in the signals that `detect` returns. Neither the code nor IABTechLab/trusted-server#1286 says why. What is the reason for placing the score with the signals? +2. **The changelog.** The changelog lists the change under Added and names `DeviceSignals::human_confidence` and `DeviceModule::advertising_attributes`, but it does not name `DeviceInfo::attributes` and gives no breaking-change note for the two new public fields that stop struct literals compiling (`CHANGELOG.md`, line 90). Should it? ## Review log @@ -114,8 +109,9 @@ These are for the reviewer. This revision records what the code does and decides ## Revision record -| Date | Change | -| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | -| 2026-10-16 | Fixes from review round 1. The header and the pull requests the header names, the removed type name mapping, the three places core asks, what the call carries, the consent and privacy section, what changes for a deployment and for code, the adapters, the acceptance without the process claims, the untested paths, the open questions and the review log. | -| 2026-10-16 | Fixes from review round 2. What the integration guide describes, what the changelog entry names and leaves out, and the statements that no code reads the score, which now say no production code does. The pull request facts in the header and the acceptance were read against the pull requests and stand. | +| Date | Change | +| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-16 | Fixes from review round 1. The header and the pull requests the header names, the removed type name mapping, the three places core asks, what the call carries, the consent and privacy section, what changes for a deployment and for code, the adapters, the acceptance without the process claims, the untested paths, the open questions and the review log. | +| 2026-10-16 | Fixes from review round 2. What the integration guide describes, what the changelog entry names and leaves out, and the statements that no code reads the score, which now say no production code does. The pull request facts in the header and the acceptance were read against the pull requests and stand. | +| 2026-10-16 | Code fixes in the five commits ending at `4badbdaf8`, and open questions 1 to 5 and the guide half of question 7 moved into the body. The call carries the permissions resolved for the request, a backend the module registers in the call has its timers capped at the auction's timeout, core drops device values a bidder cannot read, tests run the three handlers with consent allowed and denied, and the two comments say only what the code does. | From b9eeebc129f7273d800249d68d72ace5400db041 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sat, 17 Oct 2026 00:30:45 +0100 Subject: [PATCH 127/192] The loading cache design describes the six fixes The design records the fixes made on 16 October 2026 to the loading cache pull request: a shared load is handed nothing of the reader whose request started it, each application state has a loading cache of its own on every host, CacheSpec loses the unused detachable field, the comments and guides say what a table naming no module takes, the deploy check refuses a loading cache module nothing offers, and the missing tests. The fixed items leave Open questions, the new tests are listed, and the revision record has a row for the fixes. --- .../specs/2026-10-12-loading-cache-design.md | 89 +++++++++---------- 1 file changed, 41 insertions(+), 48 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index 7ddabf1b6a..0051c4a121 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -9,7 +9,7 @@ > **Why this spec exists.** Each robots.txt module's answer is read from the key-value store by every request and asked of the module's service by every request that finds no fresh copy, so a burst of requests for one file asks the service once for each request, on every instance. Core has one cache, for page templates, and no interface through which a module can supply a cache. -Line numbers in this document are those of row 31 (`4b6ef9c05`) unless a row is named. +Line numbers in this document are those of row 31 (`4b6ef9c05`) unless a row is named. The fixes of 2026-10-16 are on row 24 and reach row 31 when it is merged forward, so where this document describes them it names items rather than lines. ## 1. Problem @@ -19,9 +19,9 @@ Core had one cache, for page templates (`crates/trusted-server-core/src/platform ## 2. Design -**The interface.** Core declares `PlatformLoadingCache` in `trusted_server_core::platform` (`crates/trusted-server-core/src/platform/loading_cache.rs`). Its `get_or_load` takes the request's `RuntimeServices`, a `CacheSpec`, a `StoreKey` and a `Loader`, and answers the value or a `LoadError`. A missing value is loaded once per key, every caller that asks while that load runs gets its value or its error, and a failed load is stored nowhere. `remove` drops a key from the memory copy and from the shared store. `keeps_values` says whether the cache keeps what it loads, and no code outside tests calls it on rows 24 and 31. A feature loads through the cache with `RuntimeServices::load_cached`, which asks the cache that the spec's name selects. `NoLoadingCache` holds nothing, runs every load and answers `false` to `keeps_values`. +**The interface.** Core declares `PlatformLoadingCache` in `trusted_server_core::platform` (`crates/trusted-server-core/src/platform/loading_cache.rs`). Its `get_or_load` takes `RuntimeServices`, a `CacheSpec`, a `StoreKey` and a `Loader`, and answers the value or a `LoadError`. A missing value is loaded once per key, every caller that asks while that load runs gets its value or its error, and a failed load is stored nowhere. `remove` drops a key from the memory copy and from the shared store. `keeps_values` says whether the cache keeps what it loads, and no code outside tests calls it on rows 24 and 31. A feature loads through the cache with `RuntimeServices::load_cached`, which asks the cache that the spec's name selects and hands it the request's services without the reader, as section 5 describes. `NoLoadingCache` holds nothing, runs every load and answers `false` to `keeps_values`. -**The spec and the key.** A `CacheSpec` carries the cache's name, three lifetimes (the brief copy in process memory, the copy in the host's shared store, and how long that shared copy may sit unused), the most entries the memory copy holds, and a `detachable` flag that no code reads. A `StoreKey` is written `//`, where the prefix is the publisher's `ServerKeyPrefix`, so publishers sharing a store never share a copy. Both rows lowercase the publisher's domain and drop one leading `www.`, and neither builds the domain from the raw `Host` header, because a `Host` header can say anything. Row 24 takes the domain from the loaded settings' `publisher.domain` and row 31 from the publisher the request resolved to, as the table at the end of this section shows. +**The spec and the key.** A `CacheSpec` carries the cache's name, three lifetimes (the brief copy in process memory, the copy in the host's shared store, and how long that shared copy may sit unused) and the most entries the memory copy holds. A `StoreKey` is written `//`, where the prefix is the publisher's `ServerKeyPrefix`, so publishers sharing a store never share a copy. Both rows lowercase the publisher's domain and drop one leading `www.`, and neither builds the domain from the raw `Host` header, because a `Host` header can say anything. Row 24 takes the domain from the loaded settings' `publisher.domain` and row 31 from the publisher the request resolved to, as the table at the end of this section shows. **How a feature selects its cache.** Each feature that holds answers this way owns a `[cache.]` table. On rows 24 and 31 core reads one, `[cache.robots-txt]`, because `[cache]` refuses any other key. Its `module` field names a module that offers a loading cache, by its short name (`51degrees`) or in full (`loading-cache.51degrees`). An adapter resolves every table with `loading_caches_from` when it builds its application state (the settings, modules and caches the adapter keeps to serve requests), and makes a module's cache once however many tables select it. @@ -33,15 +33,15 @@ store_seconds = 604800 store_idle_seconds = 2592000 ``` -**When `module` is absent.** The table takes the one loading cache module that the build links (`crates/trusted-server-core/src/integrations/mod.rs`, lines 831 to 837). The stock list links exactly one, `loading-cache.51degrees`, so a stock build holds each robots.txt module's answer with no `[cache.robots-txt]` table at all, and every line of the example above can be left out because each value shown is the default. A build that links none, or two or more, and writes no `module`, gives the table `NoLoadingCache`, which holds nothing and runs every load. Nothing is logged when that happens, so a second loading cache module added to the stock list stops the holding without any message. No setting turns holding off in a stock build, because `module` can only name a linked module. +**When `module` is absent.** The table takes the one loading cache module that the build links (`loading_caches_from` in `crates/trusted-server-core/src/integrations/mod.rs`). The stock list links exactly one, `loading-cache.51degrees`, so a stock build holds each robots.txt module's answer with no `[cache.robots-txt]` table at all, and every line of the example above can be left out because each value shown is the default. A build that links none, or two or more, and writes no `module`, gives the table `NoLoadingCache`, which holds nothing and runs every load. Nothing is logged when that happens, so a second loading cache module added to the stock list would stop the holding without any message, and the test `a_table_naming_no_module_holds_with_the_stock_loading_cache` fails instead. No setting turns holding off in a stock build, because `module` can only name a linked module. -**A module nothing offers.** A `module` that no linked crate offers a cache from is refused when the adapter builds its state. The message names the table and lists the loading cache modules the build has (`integrations/mod.rs`, lines 839 to 849). +**A module nothing offers.** A `module` that no linked crate offers a cache from is refused by the deploy check, `ts config validate`, and again when the adapter builds its state. Both use one lookup, `named_cache` in `integrations/mod.rs`, whose message names the table and lists the loading cache modules the build has. The deploy check runs it through `check_loading_cache_modules` from `validate_settings_for_deploy_with`. **The implementation.** The implementation is the module crate `crates/loading-cache/51degrees` (package `trusted-server-loading-cache-51degrees`, module `loading-cache.51degrees`), which 51Degrees maintains. The module crate is written with the published [`fiftyone-caching`](https://crates.io/crates/fiftyone-caching) crate ([source](https://github.com/51Degrees/rust/tree/main/caching)). The workspace builds the published crate with its default features off, so the published crate's own Fastly, Cloudflare, Spin and tokio modules are not compiled. The loading logic comes from the published crate, and the stores are written in the module crate (`stores.rs`) over core's `PlatformValueCache` and the host's key-value store. The module crate's `builder()` offers the cache through `IntegrationBuilder::with_loading_cache`, the stock list in `crates/trusted-server-modules` is the only code that names the module crate, and each adapter calls `loading_caches_from` with the builders it is given, so no adapter names the module crate. -**The two layers.** A call goes through two layers. The memory layer is one least recently used store for each cache, shared by every request the process serves, holding up to `memory_entries` values (64 for robots.txt) for `memory_seconds` each. Below the memory layer, a shared store layer is made for each call over the host's shared store, which is the host's value cache where the host gives one, being Fastly's cache, and the host's key-value store otherwise. A read looks in memory, then in the shared store, and only then loads. A copy in the shared store lasts at most `store_seconds`, leaves after `store_idle_seconds` unused, and ends sooner when the loaded value carries a lifetime of its own, which robots.txt sets to the module's refresh. The shared store holds each entry in the `fiftyone-caching` entry format, which carries the time the entry must be dropped, so a store that deletes late, or keeps no lifetimes, never gives back an entry past that time. A write to a key-value store carries a lifetime of at least 60 seconds because Workers KV refuses less, and the entry's own drop time holds the exact lifetime. +**The two layers.** A call goes through two layers. The memory layer is one least recently used store for each cache, shared by every request an adapter serves with one application state, holding up to `memory_entries` values (64 for robots.txt) for `memory_seconds` each. Below the memory layer, a shared store layer is made for each call over the host's shared store, which is the host's value cache where the host gives one, being Fastly's cache, and the host's key-value store otherwise. A read looks in memory, then in the shared store, and only then loads. A copy in the shared store lasts at most `store_seconds`, leaves after `store_idle_seconds` unused, and ends sooner when the loaded value carries a lifetime of its own, which robots.txt sets to the module's refresh. The shared store holds each entry in the `fiftyone-caching` entry format, which carries the time the entry must be dropped, so a store that deletes late, or keeps no lifetimes, never gives back an entry past that time. A write to a key-value store carries a lifetime of at least 60 seconds because Workers KV refuses less, and the entry's own drop time holds the exact lifetime. -**Who waits for a load.** Callers that share one memory copy and ask for the same key at the same moment share one load. The first caller runs its own loader inside its own call, because no thread pool or background task exists (`lib.rs` line 95 and `layer.rs` line 97 of the module crate), and the other callers wait for its value or its error. If the first caller is dropped before the load finishes, a waiting caller takes over and loads. Across processes, other instances wait only where the shared store reserves a missing key for the first instance that asks and makes the rest wait until that instance fills the key. Only Fastly's cache does this, once for each Fastly location. Workers KV, Spin's key-value store and an unavailable store reserve nothing, so instances that miss at the same moment each load, and instances that ask afterwards read the stored copy. A module's service is therefore asked once per refresh by one process, and by all the instances of one Fastly location together, but not by all the instances of a deployment together. +**Who waits for a load.** Callers that share one memory copy and ask for the same key at the same moment share one load. The first caller runs its own loader inside its own call, because no thread pool or background task exists (`LoadingCaches::new` in `lib.rs` and `Layer` in `layer.rs` of the module crate, and the test `a_load_runs_on_the_thread_of_the_caller_that_leads_it`), and the other callers wait for its value or its error. If the first caller is dropped before the load finishes, a waiting caller takes over and loads. Across processes, other instances wait only where the shared store reserves a missing key for the first instance that asks and makes the rest wait until that instance fills the key. Only Fastly's cache does this, once for each Fastly location. Workers KV, Spin's key-value store and an unavailable store reserve nothing, so instances that miss at the same moment each load, and instances that ask afterwards read the stored copy. A module's service is therefore asked once per refresh by the requests that share one memory copy, and by all the instances of one Fastly location together, but not by all the instances of a deployment together. **Why.** The same answer is wanted by every instance and every request, and a cache that lets every miss load at once asks a module's service as many times as there are requests. Loading each key once at a time in a process, with the value shared by every waiter, is the pattern of `LruLoadingCache` in the 51Degrees [.NET](https://github.com/51Degrees/caching-dotnet) and [Java](https://github.com/51Degrees/pipeline-java) pipelines, which the published crate's documentation points to. Core keeps only the interface. Core's cache code depends on no caching package. The only place core writes the module name `51degrees` is `settings.rs`, in the documentation of `LoadingCacheConfig::module` (line 3055) and in the test `a_loading_cache_module_is_read_and_written_back` (lines 7036 to 7055), and the open questions ask about it. @@ -55,24 +55,24 @@ store_idle_seconds = 2592000 | Where the key prefix comes from | `ServerKeyPrefix::for_settings`, from the loaded settings' `publisher.domain`, always headed `ts` | The `[prefix]` module (row 27) through `RuntimeServices::server_key_prefix()`, headed `ts` by core's own module, with a trailing dot dropped from the domain | | What asks the cache for robots.txt | The robots.txt route handler in `robots_txt/mod.rs` | The `robots_txt` middleware's preparation in `robots_txt/middleware.rs` (row 25) | -Everything else in this document is the same on both rows, apart from line numbers, the tests named in section 9 and the repeated changelog entry in the open questions. +Everything else in this document is the same on both rows, apart from line numbers, the tests named in section 9, the fixes of 2026-10-16 until row 31 is merged forward, and the repeated changelog entry in the open questions. ## 3. The adapters Each adapter calls `loading_caches_from` when it builds its application state (Fastly `app.rs` line 347, Cloudflare line 315, Spin line 244, Axum line 214) and hands the result to every request's services. What the cache keeps, and who waits, depends on the stores the adapter gives each request. -| Adapter | Shared store | Do other instances wait for one load? | Memory copy | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Fastly | Fastly's cache alone, and the key-value store is not used (`stores.rs` lines 61 to 73 of the module crate, `app.rs` line 443 of the Fastly adapter) | Yes, once for each Fastly location, because a missing key is reserved for the first instance that looks it up | One for each application state. Fastly serves one request for each instance unless the opt-in reusable sandbox keeps the state, so by default the copy lasts one request | -| Cloudflare | Workers KV where the deployment binds a namespace, and otherwise a store whose every call fails | No | One for the whole isolate, shared by every request the isolate serves, because `factory()` keeps one cache in a static on `wasm32-unknown-unknown` (`lib.rs` lines 53 to 59 of the module crate). The application state is built again for every request | -| Spin | Spin's key-value store where the component has access to one, and otherwise a store whose every call fails. Spin gives an entry no lifetime, so the entry is written without one and its own drop time ends its use | No | One for each application state, which Spin builds again for every request, so the copy lasts one request (see the note below the table) | -| Axum | None. The key-value store is `UnavailableKvStore`, which fails every call (`axum/platform.rs` lines 572 to 574, `platform/kv.rs` lines 18 to 24) | Not applicable | One for the process, because Axum, the native development server, builds its application state once. That copy is the only one, and a restart loads again | +| Adapter | Shared store | Do other instances wait for one load? | Memory copy | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Fastly | Fastly's cache alone, and the key-value store is not used (`stores.rs` lines 61 to 73 of the module crate, `app.rs` line 443 of the Fastly adapter) | Yes, once for each Fastly location, because a missing key is reserved for the first instance that looks it up | One for each application state. Fastly serves one request for each instance unless the opt-in reusable sandbox keeps the state, so by default the copy lasts one request | +| Cloudflare | Workers KV where the deployment binds a namespace, and otherwise a store whose every call fails | No | One for each application state, which Cloudflare builds again for every request, so the copy lasts one request and Workers KV holds the copy between requests. `factory()` makes a cache for each state on every host | +| Spin | Spin's key-value store where the component has access to one, and otherwise a store whose every call fails. Spin gives an entry no lifetime, so the entry is written without one and its own drop time ends its use | No | One for each application state, which Spin builds again for every request, so the copy lasts one request (see the note below the table) | +| Axum | None. The key-value store is `UnavailableKvStore`, which fails every call (`axum/platform.rs` lines 572 to 574, `platform/kv.rs` lines 18 to 24) | Not applicable | One for the process, because Axum, the native development server, builds its application state once. That copy is the only one, and a restart loads again | Spin builds its application state for every request because the Spin adapter calls `run_app` in [EdgeZero](https://github.com/stackpop/edgezero) (`edgezero-adapter-spin/src/lib.rs`, at the revision the workspace pins), which builds the application on every call. A shared store that fails is read as holding nothing. A failed read is a miss, and a failed write is logged at debug level while the value is still served (`stores.rs` lines 207 to 231 for a key-value store, and `value_cache.rs` of the Fastly adapter for Fastly's cache). On Axum the cache therefore keeps the memory copy only. -Every load on every adapter runs in the caller that leads it. On Cloudflare the isolate-wide memory copy lets one request wait on the load of another request, and the published crate's documentation says that a request doing so fails unless the crate's request scope and spawner are used (`cloudflare.rs` lines 37 to 52 of the published crate). Nothing in `crates/` uses the request scope or the spawner, and the first open question asks what follows. +Every load on every adapter runs in the caller that leads it. The published crate's documentation says that a Cloudflare request that waits on another request's load fails unless every request runs in the crate's request scope with its spawner (`cloudflare.rs` lines 37 to 52 of the published crate). Nothing in `crates/` uses the request scope or the spawner, so no memory copy is shared between Cloudflare requests. Each application state has a cache of its own (the test `each_application_state_is_given_a_cache_of_its_own`), and no Worker request waits on another request's load. ## 4. Configuration @@ -85,21 +85,21 @@ Every load on every adapter runs in the caller that leads it. On Cloudflare the | `store_seconds` | 604800 (a week) | 1 to 31536000 | The longest the copy in the shared store is used. The module's own refresh ends that copy sooner | | `store_idle_seconds` | 2592000 (30 days) | 1 to 31536000 | How long the copy in the shared store lasts unused | -The ranges are checked when the settings load and again by the deploy check, which refuses a value out of range and names the field (`settings.rs` lines 3102 to 3105 and 3149 to 3150). The deploy check validates nothing else about this table. A `module` that no linked crate offers a cache from passes `ts config validate` and is refused only when the adapter builds its state. Every adapter then answers with its startup error, from its start on Axum and on every request on the other three, because those three build the state for every request. +The ranges are checked when the settings load and again by the deploy check, which refuses a value out of range and names the field (`settings.rs` lines 3102 to 3105 and 3149 to 3150). The deploy check also refuses a `module` that no linked crate offers a cache from, with the message the adapter gives when it builds its state, as section 2 describes. Without that check a misspelt `module` would pass `ts config validate` and every adapter would answer with its startup error, from its start on Axum and on every request on the other three, because those three build the state for every request. ## 5. Consent and privacy **What is stored.** Only what a module answered for the publisher, as the JSON of a robots.txt `Contribution`, and nothing about a reader. The key is `/robots-txt/`, where the hash is the SHA-256, in hexadecimal, of the module's name, a newline and the module's fingerprint (a value that changes whenever the module's answer would, such as a hash of its settings). A fingerprint's contents therefore never appear in a store (`held_key` in `holding.rs`, lines 190 to 202, and the test `the_key_carries_nothing_of_the_fingerprint`). The answer kept for a failure is under `/robots-txt-kept/`. -**What a load is given.** The load of the first caller serves every caller waiting on it and every later reader of the copy. For robots.txt the module is called with the request's method, path, host and scheme with the query removed, and with the request's services (`holding.rs` builds the call from `ModuleContext::new(request.view()).with_services(&services)`). The call carries no reader evidence, connection, location, consent or permissions, and the contract of `RobotsTxtContributor::contribute` (`robots_txt/contributor.rs` lines 61 to 73) says the answer must be the same for every request the publisher receives. The test `the_contributor_is_asked_for_the_request_and_not_who_asked` checks what the call carries. +**What a load is given.** The load of the first caller serves every caller waiting on it and every later reader of the copy, so it is given nothing of the reader whose request started it. `RuntimeServices::without_reader` gives a copy of a request's services with the `ClientInfo` (the client's IP address, TLS details and connection fingerprints) and the host's signals about the client cleared, and the stores, caches, backends, HTTP client and modules kept. `RuntimeServices::load_cached` hands the cache that copy, and for robots.txt the module is called with the request's method, path, host and scheme with the query removed, and with that copy (`holding.rs` builds the call from `ModuleContext::new(request.view()).with_services(&services)`, where the services are `services.without_reader()`). The call carries no reader evidence, connection, location, consent or permissions, and the contract of `RobotsTxtContributor::contribute` (`robots_txt/contributor.rs` lines 61 to 73) says the answer must be the same for every request the publisher receives. The test `the_contributor_is_asked_for_the_request_and_not_who_asked` checks what the call carries, `the_contributor_is_handed_nothing_of_the_reader_who_asks_first` checks that no client address, TLS detail or host signal reaches the module, and `a_cache_is_handed_nothing_of_the_reader_whose_request_loads` checks what the cache is handed. -**What the types leave open.** The services in the call are those of the first caller's request. `RuntimeServices::client_info()` returns that request's `ClientInfo`, which holds the client's IP address, TLS details and connection fingerprints, so a module that read `ClientInfo` would give an answer that depends on one reader, and the cache would serve that answer to every reader. Nothing in the types stops a module from reading `ClientInfo`, and the contract forbids that only in words. The documentation of `PlatformLoadingCache` itself (`platform/loading_cache.rs` lines 172 to 206) carries no such rule, although the load of the first caller serves every waiter. +**What the types leave open.** The documentation of `Loader` and of `PlatformLoadingCache::get_or_load` states the rule, and the one caller on these rows, the robots.txt holding, follows it. A `Loader` is a closure that owns whatever it captures, so nothing in the types stops a later caller from capturing a request's own services, and for such a caller the rule holds only in words. ## 6. Failure - **A failed load.** The error reaches every caller waiting on that load and nothing is stored, so the next caller loads again. The test `a_failed_load_reaches_every_waiter_and_is_not_kept` checks that behavior with 16 callers. Where the shared store reserves keys, a reservation dropped without being filled passes to another instance, so instances waiting on a failed load do not wait for a timeout (the module crate's test `a_reserving_value_cache_is_filled_by_the_load_and_released_by_a_failure` and the Fastly adapter's test `a_dropped_reservation_passes_the_obligation_on`). -- **A first caller that is dropped.** A waiting caller takes over the load. No test covers that takeover. -- **A shared store that fails.** A failed read is a miss and a failed write is logged while the value is still served, as section 3 says. No test covers a failing store, Axum's unavailable store included. +- **A first caller that is dropped.** A waiting caller takes over the load with its own loader (the test `a_waiting_caller_loads_when_the_caller_leading_the_load_is_dropped`). +- **A shared store that fails.** A failed read is a miss and a failed write is logged while the value is still served, as section 3 says, so the memory copy alone answers (the test `a_store_that_fails_every_call_leaves_the_memory_copy_answering`, with Axum's `UnavailableKvStore`). - **A held copy that is unreadable.** A copy that no longer parses as a robots.txt `Contribution` is removed from the cache and the module is asked again (`holding.rs` lines 95 to 109, and the test `an_unreadable_held_copy_is_removed_and_the_contributor_asked_again`). - **A module that fails.** The last answer the module gave is served past its age. Every load that succeeds writes its answer apart from the cache for 30 days, in the host's value cache where it has one and in the key-value store otherwise, and nothing reads that answer unless the module fails (`holding.rs` lines 35 to 40 and 110 to 125). With none kept, the whole file answers `503` with `Retry-After: 600` and is not stored. A value cache may drop an entry early, so on Fastly a kept answer is likely and not certain to be there. Spin's key-value store gives an entry no lifetime, so a kept answer there stays until it is replaced. Because a failure is not stored, every request asks the module again while the module is down, and the requests in one process that arrive during one attempt share that attempt. @@ -128,48 +128,40 @@ The ranges are checked when the settings load and again by the deploy check, whi git grep -n -e trusted_server_loading_cache_51degrees -e trusted-server-loading-cache-51degrees 4b6ef9c05 -- crates ':!crates/loading-cache' ``` -This search prints nothing, so no code reads `CacheSpec::detachable` and nothing uses the published crate's Cloudflare request scope or spawner. +On row 24 after its fixes (`35640194a`), this search prints nothing, so `CacheSpec` has no `detachable` field and nothing uses the published crate's Cloudflare request scope or spawner. ```text -git grep -n -e '\.detachable' -e 'cloudflare::with_context' -e WaitUntil -e local_spawner 4b6ef9c05 -- crates +git grep -n -e detachable -e 'cloudflare::with_context' -e WaitUntil -e local_spawner 35640194a -- crates ``` -**Tests.** These tests pin the cache, its selection and its use for robots.txt on row 31. Row 24 has the same tests, apart from `the_held_and_kept_keys_follow_the_publisher`, which arrives with the prefix module on row 27, and row 24 builds the key prefix with `ServerKeyPrefix::for_domain`. +**Tests.** These tests pin the cache, its selection and its use for robots.txt on row 31, and those marked row 24 came with the fixes of 2026-10-16 and reach row 31 when it is merged forward. Row 24 has every test, apart from `the_held_and_kept_keys_follow_the_publisher`, which arrives with the prefix module on row 27, and row 24 builds the key prefix with `ServerKeyPrefix::for_domain`. -- `crates/loading-cache/51degrees/src/tests.rs`, which `cargo test-axum` runs natively and `cargo test-fastly` does not list. `many_callers_on_many_threads_load_a_missing_key_once_in_the_caller` (16 threads, one load), `a_failed_load_reaches_every_waiter_and_is_not_kept`, `the_store_answers_another_process_without_a_load`, `a_store_that_refuses_lifetimes_still_holds_the_entry`, `two_publishers_never_share_an_entry`, `the_memory_copy_is_brief_and_the_store_is_read_after_it`, `an_entry_left_unused_for_its_idle_time_is_loaded_again`, `a_value_with_a_lifetime_of_its_own_ends_by_then`, `a_reserving_value_cache_is_filled_by_the_load_and_released_by_a_failure`, `remove_drops_the_memory_copy_and_the_stored_one`, `the_builder_offers_a_cache_that_keeps_values` and `module_constant_is_the_crate_folder`. +- `crates/loading-cache/51degrees/src/tests.rs`, which `cargo test-axum` runs natively and `cargo test-fastly` does not list. `many_callers_on_many_threads_load_a_missing_key_once_in_the_caller` (16 threads, one load), `a_failed_load_reaches_every_waiter_and_is_not_kept`, `the_store_answers_another_process_without_a_load`, `a_store_that_refuses_lifetimes_still_holds_the_entry`, `two_publishers_never_share_an_entry`, `the_memory_copy_is_brief_and_the_store_is_read_after_it`, `an_entry_left_unused_for_its_idle_time_is_loaded_again`, `a_value_with_a_lifetime_of_its_own_ends_by_then`, `a_reserving_value_cache_is_filled_by_the_load_and_released_by_a_failure`, `remove_drops_the_memory_copy_and_the_stored_one`, `the_builder_offers_a_cache_that_keeps_values` and `module_constant_is_the_crate_folder`. Row 24 adds `each_application_state_is_given_a_cache_of_its_own`, `a_load_runs_on_the_thread_of_the_caller_that_leads_it`, `a_memory_lifetime_of_zero_keeps_no_memory_copy_and_still_loads_once` (16 threads, one load, and the store read once the load is done), `a_waiting_caller_loads_when_the_caller_leading_the_load_is_dropped` and `a_store_that_fails_every_call_leaves_the_memory_copy_answering`. - `crates/trusted-server-core/src/integrations/mod.rs`, in `loading_caches_from_tests`. `a_feature_selecting_nothing_takes_the_one_module_offered`, `a_feature_selecting_nothing_keeps_nothing_when_two_are_offered`, `a_feature_selecting_nothing_keeps_nothing_when_none_is_offered`, `a_feature_selects_a_module_by_its_short_name_or_in_full`, `a_feature_naming_a_module_nothing_offers_is_refused_naming_what_is_offered` and `with_nothing_offered_the_refusal_says_so`. -- `crates/trusted-server-core/src/platform/loading_cache.rs`. `two_publishers_never_share_a_key`, `a_key_names_the_publisher_then_the_cache` and `the_default_cache_runs_every_load`. +- `crates/trusted-server-core/src/config.rs`, on row 24. `deploy_validation_refuses_a_loading_cache_module_nothing_offers`. +- `crates/trusted-server-core/src/platform/loading_cache.rs`. `two_publishers_never_share_a_key`, `a_key_names_the_publisher_then_the_cache` and `the_default_cache_runs_every_load`. Row 24 adds `a_cache_is_handed_nothing_of_the_reader_whose_request_loads`. - `crates/trusted-server-core/src/platform/server_key.rs`. `a_leading_www_is_folded_and_every_other_label_kept`, and `the_domain_is_the_folded_one` on row 31, where row 24 has `the_prefix_comes_from_the_loaded_settings`. -- `crates/trusted-server-core/src/robots_txt/holding.rs`, which runs the holding against a core test double that does not join loads. `a_contributor_is_asked_once_and_its_answer_is_held_after_that`, `the_held_answer_is_keyed_by_the_publisher`, `the_held_and_kept_keys_follow_the_publisher`, `changed_settings_are_not_answered_from_the_old_copy`, `the_kept_answer_is_served_when_the_contributor_fails`, `the_kept_answer_outlives_the_instance_that_kept_it`, `a_store_without_lifetimes_still_keeps_the_answer`, `with_nothing_kept_a_failure_is_an_error_and_is_not_held`, `an_unreadable_held_copy_is_removed_and_the_contributor_asked_again`, `the_kept_answer_is_in_the_value_cache_where_the_host_has_one`, `the_key_carries_nothing_of_the_fingerprint` and `the_contributor_is_asked_for_the_request_and_not_who_asked`. +- `crates/trusted-server-core/src/robots_txt/holding.rs`, which runs the holding against a core test double that does not join loads. `a_contributor_is_asked_once_and_its_answer_is_held_after_that`, `the_held_answer_is_keyed_by_the_publisher`, `the_held_and_kept_keys_follow_the_publisher`, `changed_settings_are_not_answered_from_the_old_copy`, `the_kept_answer_is_served_when_the_contributor_fails`, `the_kept_answer_outlives_the_instance_that_kept_it`, `a_store_without_lifetimes_still_keeps_the_answer`, `with_nothing_kept_a_failure_is_an_error_and_is_not_held`, `an_unreadable_held_copy_is_removed_and_the_contributor_asked_again`, `the_kept_answer_is_in_the_value_cache_where_the_host_has_one`, `the_key_carries_nothing_of_the_fingerprint` and `the_contributor_is_asked_for_the_request_and_not_who_asked`. Row 24 adds `the_contributor_is_handed_nothing_of_the_reader_who_asks_first`. - `crates/trusted-server-core/src/settings.rs`. `loading_cache_lifetimes_are_read_with_defaults_for_what_is_unset`, `a_loading_cache_lifetime_out_of_range_is_refused_naming_it`, `unset_loading_cache_lifetimes_are_not_written_back` and `a_loading_cache_module_is_read_and_written_back`. -- `crates/trusted-server-modules/src/lib.rs`. The stock list order, which names `loading-cache.51degrees`, and `a_feature_selects_the_stock_loading_cache_by_name`. +- `crates/trusted-server-modules/src/lib.rs`. The stock list order, which names `loading-cache.51degrees`, and `a_feature_selects_the_stock_loading_cache_by_name`. Row 24 adds `a_table_naming_no_module_holds_with_the_stock_loading_cache` and `deploy_validation_knows_the_stock_loading_cache`. - `crates/trusted-server-adapter-fastly/src/value_cache.rs`, which `cargo test-fastly` runs under Viceroy, Fastly's local runtime. Seven tests, including `a_missing_key_is_reserved_and_its_fill_is_then_a_hit`, `a_dropped_reservation_passes_the_obligation_on` and `a_stale_entry_is_reserved_rather_than_served`. **CI.** Every check passed on the head of each row. That is 19 of 19 on the closed pull requests on the fork, jwrosewell/trusted-server#19 at `844b01fe2` and jwrosewell/trusted-server#26 at `4b6ef9c05`, and 25 of 25 on IABTechLab/trusted-server#1365 and IABTechLab/trusted-server#1372, as read on 2026-10-16. -**Local checks.** The description of IABTechLab/trusted-server#1365 records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run. The description marks a manual run with `fastly compute serve` as not done. +**Local checks.** The description of IABTechLab/trusted-server#1365 records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run. The description marks a manual run with `fastly compute serve` as not done. For the fixes of 2026-10-16 on row 24, commits `a1931fd7e` to `0d49c4601`, the ten tests they added were run with the fixes reverted and with a second loading cache in the stock list and a one second floor on the memory copy, where the seven those changes break failed and the other three passed, and again with everything restored, where all ten passed. The module crate's 17 tests, 142 tests of core and 5 of the stock list chosen by the code changed, clippy on those three packages with warnings refused, `cargo fmt --all -- --check` and `cargo check-cloudflare` also passed. Neither the full core suite nor `cargo clippy-fastly` was run there, and CI runs both. **What no test covers.** -1. **The real stock list with no `module` line.** The only test that selects the cache from the stock list names the module (`a_feature_selects_the_stock_loading_cache_by_name`). The default is tested with builders made for the test, so a second loading cache module added to the stock list would stop the holding and no test would fail. -2. **`memory_seconds = 0`.** No test sets it. The field says that zero keeps no memory copy and a missing value is still loaded once. -3. **A first caller dropped before its load finishes.** -4. **A shared store that fails,** Axum's unavailable store included. -5. **robots.txt through the real cache.** The holding tests use a double that does not join loads, and the module crate's tests use no robots.txt module, so the two never run together. -6. **Any adapter.** No adapter test loads through the cache on rows 24 and 31. The module crate's tests run natively and never on `wasm32-wasip1` or `wasm32-unknown-unknown`, and nothing runs two Cloudflare requests against one isolate's cache. -7. **Other instances waiting for one load.** The tests use one instance. The behavior comes from Fastly's cache, as the comments in `value_cache.rs` of the Fastly adapter describe. +1. **robots.txt through the real cache.** The holding tests use a double that does not join loads, and the module crate's tests use no robots.txt module, so the two never run together. +2. **Any adapter.** No adapter test loads through the cache on rows 24 and 31. The module crate's tests run natively and never on `wasm32-wasip1` or `wasm32-unknown-unknown`. `factory()` has no code that differs by target, so `each_application_state_is_given_a_cache_of_its_own` covers what the factory gives Cloudflare. +3. **Other instances waiting for one load.** The tests use one instance. The behavior comes from Fastly's cache, as the comments in `value_cache.rs` of the Fastly adapter describe. ## 10. Open questions -These are for the reviewer. This revision records what the code does and decides none of them. +These are for the reviewer. This revision records what the code does and decides neither of them. -1. Cloudflare shares one memory copy across the isolate and runs every load in its caller, and nothing in `crates/` uses the published crate's request scope or spawner, which the published crate says a request needs to wait on another request's load. Does a Cloudflare request that waits on another request's load succeed? The tests run natively, so none covers it. -2. Nothing reads `CacheSpec::detachable` (`platform/loading_cache.rs` lines 126 to 129), yet the documentation of `Loader` (lines 97 to 102), a comment in `holding.rs` (lines 156 to 157) and the integration guide (`docs/guide/integration-guide.md` lines 863 to 865) still describe loads that run apart from the caller. Should the flag and the text go? -3. Comments and guide text say that a table naming no module keeps nothing, which a stock build contradicts. They are in `settings.rs` (line 3056), `integrations/mod.rs` (lines 636 to 638 and 808 to 810), `platform/types.rs` (lines 182 to 184) and the integration guide (lines 841 to 842), and on row 31 `CHANGELOG.md` lines 13 and 14 repeat one entry. Should they say what section 2 says? -4. The deploy check validates the three lifetimes only (`settings.rs` lines 3149 to 3150), so a misspelt `module` passes `ts config validate` and fails when the adapter builds its state. Row 32 adds a deploy check that refuses a table whose name no linked module declares (commit `1133273b2`), and a `module` value that no linked crate offers a cache from is still refused only when the adapter builds its state. Should the deploy check refuse that value? -5. Should the documentation of `PlatformLoadingCache` say that a loader's answer must not depend on who asked, and should the services handed to a load leave out `ClientInfo`? -6. Core writes `51degrees` in the documentation of `LoadingCacheConfig::module` (`settings.rs` line 3055) and in the test `a_loading_cache_module_is_read_and_written_back` (lines 7036 to 7055). Should those use a name that belongs to no vendor? -7. Which of the cases that section 9 lists as covered by no test should get one? The first is the real stock list with no `module` line, because a second loading cache module added to the stock list would stop the holding and no test would fail. +1. Core writes `51degrees` in the documentation of `LoadingCacheConfig::module` (`settings.rs` line 3055) and in the test `a_loading_cache_module_is_read_and_written_back` (lines 7036 to 7055). Should those use a name that belongs to no vendor? +2. On rows 27 to 31 `CHANGELOG.md` repeats the robots.txt holding entry (lines 13 and 14 on row 31). The first is an older wording that a merge brought in on row 27, and row 24 does not carry it. Should row 27 drop it? ## Review log @@ -178,8 +170,9 @@ These are for the reviewer. This revision records what the code does and decides ## Revision record -| Date | Change | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | -| 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the default when a table names no module, how loads run, who waits for a load, where rows 24 and 31 differ, the adapters, the configuration, consent and privacy, failure, what changes for a deployment, the tests and what none covers, the open questions and the review log. | -| 2026-10-16 | Points from review round 2. How the two published crates differ, from a comparison of the crates themselves, and the date on which the check counts were read. | +| Date | Change | +| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the default when a table names no module, how loads run, who waits for a load, where rows 24 and 31 differ, the adapters, the configuration, consent and privacy, failure, what changes for a deployment, the tests and what none covers, the open questions and the review log. | +| 2026-10-16 | Points from review round 2. How the two published crates differ, from a comparison of the crates themselves, and the date on which the check counts were read. | +| 2026-10-16 | Code fixes on row 24. A load is handed services with nothing of the reader in them, each application state has a cache of its own on Cloudflare as elsewhere, `CacheSpec::detachable` and the text about loads that run apart from their caller are gone, the comments and guides say what a table naming no module takes, the deploy check refuses a `module` nothing offers, and tests cover the stock default, a zero memory lifetime, a dropped first caller and a failing store. Open questions 1, 2, 4, 5 and 7, and 3 apart from the repeated changelog entry, move into the text. | From 8bc27f4a6f161e36236e6274a9f8ce12cc87c7bb Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sat, 17 Oct 2026 07:41:54 +0100 Subject: [PATCH 128/192] The robots.txt middleware design describes both of its fixes The design describes the robots.txt answer made in place of any origin answer the middleware cannot take whole, with the request headers a range or conditional request carries removed, and the fix that asks contributors for the host the request was addressed to, never one a forwarded header names, with robots.txt answered on Fastly before the Edge Cookie steps and the request filters. The answer table, the tests and the open questions are checked against the code, and a request that names no host keeps the origin's answer. --- ...2026-10-12-robots-txt-middleware-design.md | 177 +++++++++--------- 1 file changed, 91 insertions(+), 86 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md b/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md index 659bdd79d7..02f20e2316 100644 --- a/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md +++ b/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md` (row 10), `2026-10-12-robots-txt-from-modules-design.md` (row 14), `2026-10-12-middleware-media-types-design.md` (row 19), `2026-10-12-middleware-creates-a-missing-file-design.md` (row 22), `2026-10-12-middleware-prepare-design.md` (row 23), `2026-10-12-loading-cache-design.md` (row 24), `2026-10-12-shared-vendor-tables-design.md` (row 26, built after this row), `2026-10-12-prefix-module-design.md` (row 27, built after this row) and `2026-10-15-asset-cache-design.md` (row 34, built after this row), all in IABTechLab/trusted-server#1342 **Related PRs:** [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366) builds on the loading cache in [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24). The preparation step comes from [IABTechLab/trusted-server#1364](https://github.com/IABTechLab/trusted-server/pull/1364) (row 23), the rule that makes a file the origin does not have from [IABTechLab/trusted-server#1363](https://github.com/IABTechLab/trusted-server/pull/1363) (row 22), the entries for a media type other than HTML from [IABTechLab/trusted-server#1360](https://github.com/IABTechLab/trusted-server/pull/1360) (row 19), the `[[fetch]]` and `[[serve]]` entries from [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10) and the robots.txt contributors from [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355) (row 14). The publisher's assets held through the loading cache are in [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375) (row 34), which comes after this row. **Issues:** [IABTechLab/trusted-server#1283](https://github.com/IABTechLab/trusted-server/issues/1283). [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack. -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-17 > **Why this spec exists.** Before this change a request for `/robots.txt` was answered by a handler of core's own that each of the four adapters registered as a route, while every other change to what a deployment serves is a middleware that an entry names. This change makes the file a middleware, so the same entries place the file. The sections below give what the code does, which answer from the origin reaches a crawler, what a deployment has to change and what the change leaves open. @@ -15,11 +15,11 @@ In this document a contributor is a module that gives rules for the file through the `RobotsTxtContributor` interface (row 14, `2026-10-12-robots-txt-from-modules-design.md`). The holding is the loading cache that keeps each contributor's answer (row 24, `2026-10-12-loading-cache-design.md`). The creating rule is the rule that makes a file the origin does not have (row 22, `2026-10-12-middleware-creates-a-missing-file-design.md`). The preparation is the step in which a middleware makes a decision with the request's services before the document is processed, and in which the middleware may refuse the request (row 23, `2026-10-12-middleware-prepare-design.md`). A stored copy is a copy of a response that a cache keeps to answer later requests, and section 2 says which stored copies exist for this file. -Line numbers in this document are those of commit `16248774e`, the head of IABTechLab/trusted-server#1366, unless a row is named. A file named without a folder is in `crates/trusted-server-core/src`, so `publisher.rs`, `settings.rs`, `middleware.rs`, `config.rs`, `error.rs`, `module_context.rs` and `integrations/registry.rs` are there. The module's own files are `robots_txt/mod.rs`, `robots_txt/middleware.rs` and `robots_txt/holding.rs`, always written with their folder because `middleware.rs` without a folder is core's file for entries and actions. A file of an adapter is written with its crate, such as `adapter-axum/src/app.rs` for `crates/trusted-server-adapter-axum/src/app.rs`. +Line numbers in this document are those of commit `16248774e`, the head of IABTechLab/trusted-server#1366, unless a row is named. The fixes this row takes after that commit, which the revision record lists, are cited by the names of their functions and tests. A file named without a folder is in `crates/trusted-server-core/src`, so `publisher.rs`, `settings.rs`, `middleware.rs`, `config.rs`, `error.rs`, `module_context.rs` and `integrations/registry.rs` are there. The module's own files are `robots_txt/mod.rs`, `robots_txt/middleware.rs` and `robots_txt/holding.rs`, always written with their folder because `middleware.rs` without a folder is core's file for entries and actions. A file of an adapter is written with its crate, such as `adapter-axum/src/app.rs` for `crates/trusted-server-adapter-axum/src/app.rs`. ## 2. Design -**What happens.** `/robots.txt` is made by `robots_txt`, a middleware of core's own that a `[[fetch]]` entry names. A request for the file goes through the publisher path as a request for any file does, so core asks the origin first. The middleware then replaces the origin's `text/plain` file with the file the settings describe, and the creating rule makes the file on an empty document where the origin answers `404` or answers with a page. +**What happens.** `/robots.txt` is made by `robots_txt`, a middleware of core's own that a `[[fetch]]` entry names. A request for the file goes through the publisher path as a request for any file does, so core asks the origin first, for its whole file. The middleware then replaces the origin's `text/plain` file with the file the settings describe, and the creating rule makes the file on an empty document in place of any other answer the origin gives to a `GET` or `HEAD` that names its host, a `404` and a page among them. The middleware works on `/robots.txt` alone. ```toml [robots-txt] @@ -34,7 +34,7 @@ middleware = ["robots_txt"] The middleware's name has an underscore where the section and the module folder have a hyphen. Core's own names use underscores, as `js_asset_proxy` (`integrations/js_asset_proxy.rs` 39), `service_version` (`service_version.rs` 19), `refuse_all` and `allow_all` (`robots_txt/contributor.rs` 17 and 23) do, and a module's name is its type folder and vendor, such as `ads-txt.sellers`. -**What the change touches.** The change adds 879 lines and removes 708 across 19 files, in four commits of its own with merges of row 24 between them. Commit `49b280292` makes the middleware and the part of the registry that creates the middleware, adds the stock list and the example settings, and removes the route. Commit `71dbb1755` changes the guides and the changelog. Commit `7b34f7c45` requires a `[[fetch]]` entry on exactly `/robots.txt` and refuses an entry on any other path or media type, adds the page rule, and removes the API reference for the route. Commit `895e66c30` makes the middleware take what it prepared for its own phase. +**What the change touches.** The change adds 879 lines and removes 708 across 19 files, in four commits of its own with merges of row 24 between them. Commit `49b280292` makes the middleware and the part of the registry that creates the middleware, adds the stock list and the example settings, and removes the route. Commit `71dbb1755` changes the guides and the changelog. Commit `7b34f7c45` requires a `[[fetch]]` entry on exactly `/robots.txt` and refuses an entry on any other path or media type, adds the page rule, and removes the API reference for the route. Commit `895e66c30` makes the middleware take what it prepared for its own phase. The table counts these four commits and not the fixes after `16248774e`. | File | Added | Removed | What changes | | ------------------------------------------------------------------ | ----- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -59,35 +59,34 @@ The middleware's name has an underscore where the section and the module folder **What a request for the file goes through.** 1. No route of any adapter names the path, so the request reaches the publisher fallback, which takes seven methods (`publisher_fallback_methods` in each adapter's `src/app.rs`, at line 634 on Axum, 510 on Cloudflare, 516 on Fastly and 355 on Spin). -2. `handle_publisher_request` asks the origin (`publisher.rs` 5754 to 5799). If the request to the origin fails, the request ends with an error response. -3. For a `GET` or a `HEAD`, the creating rule drops the origin's answer and puts an empty `text/plain` document with status `200` in its place where the origin answered `404`, or answered `200` with an HTML page (5807 to 5820). +2. `handle_publisher_request` asks the origin (`publisher.rs` 5754 to 5799). Before it asks, it removes the request's `If-None-Match`, `If-Modified-Since`, `Range` and `If-Range`, because the middleware says it needs the origin's whole body (`Middleware::needs_whole_body`, which `IntegrationRegistry::middleware_needs_whole_body` reads for the request's path). If the request to the origin fails, the request ends with an error response. +3. For a `GET` or a `HEAD` that names its host, the creating rule drops the origin's answer and puts an empty `text/plain` document with status `200` in its place, unless the answer is a `200` of `text/plain` in a charset and an encoding core reads, which the middleware is handed whole (`entry_making_the_document` and `is_handed_over_whole` in `publisher.rs`). It does so because the middleware says it writes the whole document (`Middleware::writes_whole_document`), and only on `/robots.txt`, the one path the middleware says it works on (`Middleware::handles_path`). For a request that names no host nothing is replaced, because core runs no middleware for such a request (case 11). 4. `classify_response_route` decides whether core processes the body (2018 to 2046). A body is processed when its type is text or JavaScript or JSON, the request names a host, the encoding is one core reads, and the status is not `204` or `205`. 5. When the body is processed, `prepare_document` runs the preparation of the middleware that the entries select for the response's media type and path, in both phases (6014 to 6058). The preparation makes the file, or refuses the request with `503`. 6. The body is streamed through `ReplaceBody`, which drops every chunk of the origin's body and writes the file on the last call (`robots_txt/middleware.rs` 120 to 133). 7. The adapter finalizes the response and, with `refuse_all` selected, writes `X-Robots-Tag: noindex, nofollow`. -**What the crawler gets.** The answer of the origin decides. The table lists each answer for a `GET`, and case 12 gives what other methods change. +**What the crawler gets.** The table lists each answer to a `GET` of `/robots.txt`. Case 12 gives what other methods change, and case 13 a longer path the entry covers. -| Case | The origin's answer | The crawler gets | Code | -| ---- | ------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | -| 1 | The request to the origin fails | An error response with status `502` and no file, whatever `[robots-txt]` selects | `publisher.rs` 5778 to 5799, `error.rs` 119, `http_error` in each adapter's `src/app.rs` | -| 2 | `404`, with any `Content-Type` | The file, made on an empty document, with status `200` and `Content-Type: text/plain; charset=utf-8`. The origin's headers and body are dropped | `publisher.rs` 5807 to 5820 and 2068 to 2083 | -| 3 | `200` with an HTML page, as a single-page application's catch-all gives | The same made file. This row widens the creating rule here and nowhere else, and the widening applies to this middleware alone, because `answers_robots_txt_with_a_page` looks for `robots_txt` among the middleware that the entry names | `publisher.rs` 2048 to 2066 and 5807 to 5820 | -| 4 | `200` with `text/plain` that declares UTF-8 or no charset, in an encoding core reads | The origin's response with its body replaced by the file. The status and the origin's other headers stay | `publisher.rs` 6014 to 6058, 6130 to 6169 and 7512 to 7514, `robots_txt/middleware.rs` 120 to 133 | -| 5 | `200` with `text/plain` that declares a charset other than UTF-8 | The origin's file, unchanged, because no processor is built. The preparation still runs, because `prepare_document` checks the media type alone | `publisher.rs` 707 to 712 and 775 to 778 | -| 6 | `200` with `text/plain` in an encoding core cannot read, `zstd` for example | The origin's file, unchanged, and no preparation | `publisher.rs` 2041 to 2043 and 6017 | -| 7 | `200` with a type that is neither `text/plain` nor HTML | The origin's answer, handled as the publisher path handles any answer of that type. The route answered with the file and never asked the origin (at row 24, `robots_txt/mod.rs` 317 to 320) | `publisher.rs` 2030 to 2035 | -| 8 | `401`, `403`, `410`, `5xx` or a redirect, with a type other than `text/plain` | The origin's answer, handled as the publisher path handles any answer of that status and type. The route answered with the file and never asked the origin | `publisher.rs` 2018 to 2046 | -| 9 | The statuses of case 8 with `text/plain` as the type | The origin's status and headers with the file as the body, because an entry looks at no status. No test covers this case | `publisher.rs` 2018 to 2046, and "Status codes" in the media types design | -| 10 | `206` or `304`, to a request that carries `Range` or a conditional header | These request headers reach the origin, because core removes them only for a page the ad stack answers and for a personalized HTML document. A `304` reaches the crawler with no body. One that declares `text/plain` is still prepared, because only `204` and `205` are kept from the processing, so it becomes the `503` of a refused request when a contributor fails with nothing held. A `206` of type `text/plain` keeps its status and `Content-Range` and takes the whole file as its body | `publisher.rs` 5238 to 5243, 3279 to 3285, 2024 to 2026, 6017 to 6035 and 6055 to 6057 | -| 11 | Any answer, to a request that names no host | The origin's answer, unchanged. A file made on an empty document is an empty `200`. The creating-rule spec records this as its open question 2 | `publisher.rs` 2037 to 2039 | -| 12 | Any answer, to a method other than `GET` and `HEAD` | The creating rule does not apply, so a `404` stays `404`. A `200` of type `text/plain` takes the file as its body as for a `GET`, because an entry looks at no method | `publisher.rs` 5809, and "How a response is matched" in the media types design | +| Case | The origin's answer | The crawler gets | Code | +| ---- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | The request to the origin fails | An error response with status `502` and no file, whatever `[robots-txt]` selects. RFC 9309 section 2.3.1.4 has a crawler take a server error as refusing everything | `publisher.rs` 5778 to 5799, `error.rs` 119, `http_error` in each adapter's `src/app.rs` | +| 2 | `404`, with any `Content-Type` | The file, made on an empty document, with status `200` and `Content-Type: text/plain; charset=utf-8`. The origin's headers and body are dropped | `entry_making_the_document` in `publisher.rs`, and `publisher.rs` 2068 to 2083 | +| 3 | `200` with an HTML page, as a single-page application's catch-all gives | The same made file, because a page is no `text/plain` answer the middleware is handed | `entry_making_the_document` and `is_handed_over_whole` in `publisher.rs` | +| 4 | `200` with `text/plain` that declares UTF-8 or no charset, in an encoding core reads | The origin's response with its body replaced by the file. The status and the origin's other headers stay | `publisher.rs` 6014 to 6058, 6130 to 6169 and 7512 to 7514, `robots_txt/middleware.rs` 120 to 133 | +| 5 | `200` with `text/plain` that declares a charset other than UTF-8 | The made file, because core builds no processor for the origin's file | `is_handed_over_whole` in `publisher.rs`, and `publisher.rs` 707 to 712 | +| 6 | `200` with `text/plain` in an encoding core cannot read, `zstd` for example | The made file, because core does not stream the origin's file | `is_handed_over_whole` and `classify_response_route` in `publisher.rs` | +| 7 | `200` with a type that is neither `text/plain` nor HTML | The made file | `is_handed_over_whole` in `publisher.rs` | +| 8 | `401`, `403`, `410`, `5xx`, `204` or a redirect, with a type other than `text/plain` | The made file. A crawler takes an answer from the `4xx` range as permission to crawl everything, and follows a redirect to rules of the origin's choosing (RFC 9309 sections 2.3.1.2 and 2.3.1.3) | `is_handed_over_whole` in `publisher.rs` | +| 9 | The statuses of case 8 with `text/plain` as the type | The made file, with status `200` in place of the origin's | `is_handed_over_whole` in `publisher.rs` | +| 10 | `206` or `304` | Core asks the origin without a crawler's `Range`, `If-Range`, `If-None-Match` and `If-Modified-Since`, as it does for a page the ad stack answers, so the origin answers as in cases 2 to 9. A `206` or `304` that an origin sends anyway has the file made in its place with status `200` | `IntegrationRegistry::middleware_needs_whole_body`, `publisher.rs` 5238 to 5243 and 3279 to 3285, and `is_handed_over_whole` in `publisher.rs` | +| 11 | Any answer, to a request that names no host | The origin's answer, unchanged, a `404` among them, because core runs no middleware for such a request and so makes no file for it | `publisher.rs` 2037 to 2039, and where `handle_publisher_request` calls `entry_making_the_document` in `publisher.rs` | +| 12 | Any answer, to a method other than `GET` and `HEAD` | The creating rule does not apply, so a `404` stays `404`. A `200` of type `text/plain` takes the file as its body as for a `GET`, because an entry looks at no method | `publisher.rs` 5809, and "How a response is matched" in the media types design | +| 13 | Any answer, to a request for a longer path the entry's prefix covers, such as `/robots.txt.bak` | The origin's answer, unchanged, because the middleware works on `/robots.txt` alone and core makes no file where no middleware the entry names works | `handles_path` in `robots_txt/middleware.rs`, `IntegrationRegistry::middleware_chain` and `entry_making_the_document` in `publisher.rs` | -Case 10 is by reading the code only, and no test covers case 10. +**What follows from the table.** The file reaches the crawler whatever the origin answers to a `GET` or `HEAD` that names its host, or the `503` that "When it fails" gives where no file can be made. It is the body of the origin's own `200` in case 4 and a made file with status `200` in cases 2, 3 and 5 to 10. The route answered `refuse_all` without asking anything, so that a site that was not ready to be crawled could say so with nothing provisioned at all (at row 24, `robots_txt/mod.rs` 317 to 320). The middleware still asks the origin first, so an origin that cannot be reached gives `502` (case 1), which a crawler takes as refusing everything. The preparation of the middleware still needs nothing for `refuse_all` (`robots_txt/middleware.rs` 56 to 58), and runs only after the origin has answered and only on a document core streams. A request that names no host (case 11) and a method other than `GET` and `HEAD` (case 12) can still get the origin's answer. -**What follows from the table.** The origin's answer comes first. The route answered `refuse_all` without asking anything, so that a site that was not ready to be crawled could say so with nothing provisioned at all (at row 24, `robots_txt/mod.rs` 317 to 320). The preparation of the middleware still needs nothing for `refuse_all` (`robots_txt/middleware.rs` 56 to 58), but the preparation runs only after the origin has answered and only when the answer takes the `Stream` route. The file reaches the crawler in cases 2, 3 and 4, and as the body of the origin's own answer in cases 9, 12 and the `206` of case 10, which keep the status the origin gave. In every other case the crawler gets the origin's answer or an error, whatever `[robots-txt]` selects. - -The middleware changes the body and never the status line. A file that the creating rule makes is a new `200` response, and a file that replaces the origin's body keeps the status the origin gave. A `HEAD` is the publisher path's and is answered as a `GET` without a body (`response_carries_body`, `publisher.rs` 3295 to 3300), as the row 22 test `a_head_request_for_a_made_file_gets_the_same_answer` shows for a made file. Three tests cover cases 2, 3 and 4 through the publisher path with a stub origin, and section 4 names the tests. No test covers any other case. +The middleware changes the body and never the status line. A file that the creating rule makes is a new `200` response, and a file that replaces the origin's body keeps the status the origin gave, which for a `GET` is the `200` of case 4. A `HEAD` is the publisher path's and is answered as a `GET` without a body (`response_carries_body`, `publisher.rs` 3295 to 3300), as the row 22 test `a_head_request_for_a_made_file_gets_the_same_answer` shows for a made file and each adapter's test shows for `/robots.txt`. Section 4 names the tests of each case. Case 1, and case 12 with a `200` of `text/plain`, have none. **Where the middleware is placed.** The middleware names both phases (`robots_txt/middleware.rs` 76 to 78), so an entry in either list may name it. For a file that is not HTML core runs the entries of both phases on every response, because the template cache stores HTML alone (`publisher.rs` 687 to 692), so the phase decides only the order among the middleware that the entries name. Naming the middleware in either phase does not place it, though. The settings require a `[[fetch]]` entry on the file, because the creating rule reads `settings.fetch` alone (`publisher.rs` 5810) and only a `[[fetch]]` entry makes the file where the origin has none. `Settings::prepare_runtime` calls `require_robots_txt_placed` whenever `[robots-txt]` is set (`settings.rs` 4382 to 4385 and 4422 to 4473). So the check runs when the settings load and in `ts config validate`, which reaches `prepare_runtime` through `validate_settings_for_deploy_with` (`config.rs` 108 to 126 and 348 to 371). @@ -100,15 +99,15 @@ The middleware changes the body and never the status line. A file that the creat | An entry in either list that names `robots_txt` for another media type or another path, or with no `path` | Refused when the settings load, with a message that names the entry | The entry would replace each file it covers with the robots.txt rules | | An entry that names `robots_txt` and no `[robots-txt]` section | Refused when the registry is built at startup, with "which no module that runs supplies" | The registry makes the middleware only for a section (`integrations/registry.rs` 1843 and 1380 to 1393) | -An entry's `path` is a prefix of the request path, compared as written (`middleware.rs` 525 to 529 and 539 to 545), so `path = "/robots.txt"` also covers `/robots.txt.bak`. The route that this row removes matched `/robots.txt` alone. The check in `require_robots_txt_placed` compares the path with `==` (`settings.rs` 4436), so only exactly `/robots.txt` counts as the file. Open question 2 asks whether the entry should cover that one path alone. +An entry's `path` is a prefix of the request path, compared as written (`middleware.rs` 525 to 529 and 539 to 545), so `path = "/robots.txt"` also covers `/robots.txt.bak`. The route that this row removes matched `/robots.txt` alone, and so does the middleware, which says with `handles_path` that it works on `/robots.txt` and on no other path. `IntegrationRegistry::middleware_chain` leaves a middleware out of the chain for a path it does not work on, and `entry_making_the_document` in `publisher.rs` makes no file where the chain is empty, so a longer path keeps the origin's answer (case 13). The path is compared without the query, so `/robots.txt?crawler=one` is the file. The check in `require_robots_txt_placed` compares the entry's path with `==` (`settings.rs` 4436), so only exactly `/robots.txt` counts as the file. Row 26 rewrites the changelog line for `robots_txt` (line 89 on that row) to say that a `[[fetch]]` or `[[serve]]` entry names the middleware, which is wider than the rule at row 25. `2026-10-12-shared-vendor-tables-design.md` raises the line as its open question 5. -**The file.** The preparation makes the file before any document is processed (`robots_txt/middleware.rs` 80 to 103). With `refuse_all` selected, whatever else `modules` names, the file is the refusal file, which allows the `always_allow` paths and refuses everything else. The preparation then asks no contributor and reads no store (`robots_txt/middleware.rs` 56 to 58, tested by `refusing_every_crawler_asks_no_contributor_and_reads_no_store`). Otherwise the preparation asks each contributor that `modules` names through the holding, in the order `modules` gives, and `document::assemble` writes the file around what the contributors return, with `allow_all` giving a group that allows every crawler everywhere (`robots_txt/mod.rs` 323 to 340). The robots.txt from modules design (row 14) gives the rules that the document keeps whatever a contributor returns. The registry resolves the names once every registration is in, because the contributors are known only then and a middleware cannot reach the registry (`robots_txt/mod.rs` 292 to 319 and `integrations/registry.rs` 1842 to 1853). The action takes the file that was prepared for the phase it runs in (`robots_txt/middleware.rs` 105 to 117), and `ReplaceBody` drops every chunk of the origin's body and writes the file on the last call (120 to 133). +**The file.** The preparation makes the file before any document is processed (`robots_txt/middleware.rs` 80 to 103). With `refuse_all` selected, whatever else `modules` names, the file is the refusal file, which allows the `always_allow` paths and refuses everything else. The preparation then asks no contributor and reads no store (`robots_txt/middleware.rs` 56 to 58, tested by `refusing_every_crawler_asks_no_contributor_and_reads_no_store`). Otherwise the preparation asks each contributor that `modules` names through the holding, in the order `modules` gives, and `document::assemble` writes the file around what the contributors return, with `allow_all` giving a group that allows every crawler everywhere (`robots_txt/mod.rs` 323 to 340). The robots.txt from modules design (row 14) gives the rules that the document keeps whatever a contributor returns. The registry resolves the names once every registration is in, because the contributors are known only then and a middleware cannot reach the registry (`robots_txt/mod.rs` 292 to 319 and `integrations/registry.rs` 1842 to 1853). The action takes the file that was prepared for the phase it runs in (`robots_txt/middleware.rs` 105 to 117), or writes the refusal, with the `always_allow` paths open, when nothing was prepared, and `ReplaceBody` drops every chunk of the origin's body and writes the file on the last call (120 to 133). -**What a contributor is given.** The holding hands a contributor a context built from the request's method, host, scheme and path with an empty query, and the runtime services (`robots_txt/holding.rs` 178 and `robots_txt/middleware.rs` 59 to 62). The other fields of `ModuleContext` stay empty (`module_context.rs` 230 to 262), so the contributor gets none of the reader's evidence, permissions, consent, location, device or Edge Cookie value. The runtime services are those of the request that started the load, and they include the connection's client information (`platform/types.rs` 163 to 165 and 337 to 341). So what keeps one crawler's details out of an answer that is held and served to every crawler is the rule in the contributor interface that an answer is the same for every request (`robots_txt/contributor.rs` 63 to 69), and core does not check that rule. Core reads no consent or permission state when it makes the file, and the file is the same for every crawler. +**What a contributor is given.** The holding hands a contributor a context built from the request's method, host, scheme and path with an empty query, and the runtime services (`robots_txt/holding.rs` 178 and `robots_txt/middleware.rs` 59 to 62). The host is the one the request was addressed to, read from `Host` or the address by `http_util::addressed_host` and never from `Forwarded` or `X-Forwarded-Host`, and the holding keys each answer on it. The publisher path's own host follows those headers, which Axum and Cloudflare leave in place, so `handle_publisher_request` reads the addressed host as well and hands it to every preparation context with `ModuleContext::with_addressed_host`, and the middleware asks for it with `ModuleCall::addressed_host`. A context that carries no addressed host asks no contributor and refuses the request, while `refuse_all` still makes its file. The other fields of `ModuleContext` stay empty (`module_context.rs` 230 to 262), so the contributor gets none of the reader's evidence, permissions, consent, location, device or Edge Cookie value. The runtime services are those of the request that started the load with nothing of its connection, being default client information and no host signals (`RuntimeServices::without_connection` in `platform/types.rs`, which `contribution_for` in `robots_txt/holding.rs` calls), so no crawler's connection reaches an answer that is held and served to every crawler of the host. The contributor interface also says that an answer must be the same for every request for its host (`RobotsTxtContributor::contribute` in `robots_txt/contributor.rs`), a rule core does not check. Core reads no consent or permission state when it makes the file, and the file is the same for every crawler of a host. -**Caching and stored copies.** Core stores no copy of `/robots.txt` at this row. The template cache accepts HTML alone, and `template_cache_ttl` refuses any other type (`publisher.rs` 7346 to 7352), so every request for the file that reaches core sends a request to the origin, and the middleware runs on the answer. A stored copy can exist only outside core's own code. A cache in front of core, such as a CDN or the host's edge cache, can keep the response where the response's cache headers let that cache keep it. On Fastly the platform's readthrough cache answers the origin fetch that core makes, and a request that is not for a document keeps the platform's default there (`publisher.rs` 4897 to 4935), so an answer that the origin lets that cache hold is held there (the asset cache design, "What the caches add on Fastly"). Row 34 (`2026-10-15-asset-cache-design.md`) lets core hold the origin's answer for a file in its own loading cache. From that row, where the asset gate holds, core sends a request to the origin once for the life of each held copy when the origin answers `200` with a usable lifetime, and on every request otherwise, a `404` among those answers, because that row holds a `200` alone. The middleware still runs on every request, so the copy that row holds is the origin's answer and not the file. +**Caching and stored copies.** Core stores no copy of `/robots.txt` at this row. The template cache accepts HTML alone, and `template_cache_ttl` refuses any other type (`publisher.rs` 7346 to 7352), so every request for the file that reaches core sends a request to the origin, and the middleware runs on the answer. A stored copy can exist only outside core's own code. A cache in front of core, such as a CDN or the host's edge cache, can keep the response where the response's cache headers let that cache keep it. On Fastly the platform's readthrough cache answers the origin fetch that core makes, and a request that is not for a document keeps the platform's default there (`publisher.rs` 4897 to 4935), so an answer that the origin lets that cache hold is held there (the asset cache design, "What the caches add on Fastly"). Row 34 (`2026-10-15-asset-cache-design.md`) lets core hold the origin's answer for a file in its own loading cache. From that row, where the asset gate holds, being a `GET` that is not for a document, names its host, carries no `Authorization` and no `Cookie` and is not one a module must answer itself, with a loading cache module that keeps values (row 34 `publisher.rs` 5407 to 5415), core sends a request to the origin once for the life of each held copy when the origin answers `200` with a usable lifetime. It sends one on every request otherwise, a `404` among those answers, because that row holds a `200` alone (row 34 `publisher.rs` 7393 to 7414 and 8494 to 8505), and an answer it may not hold is marked as such for 60 seconds, so the requests that follow ask the origin themselves rather than wait on a load (8154 to 8156 and 8398 to 8417). The held copy is the origin's body and headers as the origin sent them (8453 to 8556), and it goes through the creating rule and the middleware as the origin's answer would (5847 to 5848), so the middleware still runs on every request and the copy that row holds is the origin's answer and not the file. A `[[cache.asset_rules]]` rule is the only thing that sets cache headers on the file. The first enabled rule that matches the request path sets them on a `GET` or `HEAD` answer with a success status or `304` that is not HTML and is not already `private` or `no-store` (`publisher.rs` 2089 to 2118 and 5988 to 5994, `settings.rs` 3063 to 3079). With no matching rule a made file carries no `Cache-Control`, and a file that replaces the origin's keeps the headers the origin sent. The route's `public, max-age=3600` is gone. The `503` carries `Cache-Control: no-store, private` (`publisher.rs` 851 to 868 and 6055 to 6056, `response_privacy.rs` 84 to 104). @@ -134,12 +133,12 @@ The answers of the contributors are a different copy. The holding keeps each con 1. A `[robots-txt]` section needs the `[[fetch]]` entry, or the settings are refused when they load and by `ts config validate` (`settings.rs` 4382 to 4385). The message names the entry to add (4466 to 4472). A configuration written for rows 14 to 24, which had the section and no entry, is refused by this row. 2. A build of rows 14 to 24 refuses the new entry, because that build supplies no middleware named `robots_txt` ("which no module that runs supplies", `integrations/registry.rs` 1380 to 1393, the same lines at row 24). So the new configuration and the new build go live together. A deployment with no `[robots-txt]` section is not affected, because both builds load its settings. -3. No released version has the section. `[robots-txt]` is in the unreleased block of the changelog (line 98) and the last release is 1.2.0 (line 115), so only deployments that run a build from the stack are affected. The changelog records the removal of the route at line 61 without the Breaking mark that the entries for removed configuration carry in the same block, as lines 62 and 63 show. Open question 5 records this. +3. No released version has the section. `[robots-txt]` is in the unreleased block of the changelog (line 98) and the last release is 1.2.0 (line 115), so only deployments that run a build from the stack are affected. The changelog marks the removal of the route at line 61 as breaking, as the entries for removed configuration in the same block are, and says that the new configuration and the new build go live together. 4. The route's `Cache-Control: public, max-age=3600` is gone. The file carries a cache header only where a `[[cache.asset_rules]]` rule matches the path, or from the origin's own file. -5. Core sends a request to the origin for every request for the file, where the route never asked the origin. +5. Core sends a request to the origin for every request for the file, without a crawler's conditional and range headers, where the route never asked the origin. An origin that cannot be reached gives `502` (case 1). 6. A method other than `GET` and `HEAD` reached the publisher fallback before this row and still does. The answer to such a request now takes the file as its body when it is `200` with `text/plain` (case 12). -**Adapters.** All four adapters send the request to one core function and none names the file. Axum, Cloudflare and Spin hold the processed body in memory, up to `[publisher] max_buffered_body_bytes` (16 MiB by default, `settings.rs` 85 to 92), and answer with its length. Fastly streams the body. The output of the middleware is the same on both, because `ReplaceBody` writes the whole file on the last call and cannot fail. A failure while a body is processed ends a Fastly response part way and gives an error response on the other three, as the media types design says. The `503` of a refused request and the `502` for a failed request to the origin are decided before any body is sent, so they are the same on all four. +**Adapters.** All four adapters send the request to one core function. Axum, Cloudflare and Spin name no file. Fastly's publisher fallback runs the request preparers, then the Edge Cookie steps, which read the crawler's cookies, consent signals and location, then the request filters, and finalizes the Edge Cookie on the way out. With `[robots-txt]` in the settings, Fastly takes a `GET` or `HEAD` of `/robots.txt` to the publisher path once the modules have prepared the request, with an empty Edge Cookie context, no auction and no Edge Cookie state on the response (`asks_for_robots_txt` and `answer_robots_txt` in `adapter-fastly/src/app.rs`), so a filter's answer, such as a bot challenge, never stands in for the file, which a crawler would take as permission to crawl everything. Axum, Cloudflare and Spin hold the processed body in memory, up to `[publisher] max_buffered_body_bytes` (16 MiB by default, `settings.rs` 85 to 92), and answer with its length. Fastly streams the body. The output of the middleware is the same on both, because `ReplaceBody` writes the whole file on the last call and cannot fail. A failure while a body is processed ends a Fastly response part way and gives an error response on the other three, as the media types design says. The `503` of a refused request and the `502` for a failed request to the origin are decided before any body is sent, so they are the same on all four. | Adapter | Calls `handle_publisher_request` | Finishes the answer by | Registered the route at row 24 | Writes `X-Robots-Tag` | | ---------- | ----------------------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------- | ------------------------------------------ | @@ -150,69 +149,71 @@ The answers of the contributors are a different copy. The holding keeps each con **When it fails.** -1. **The request to the origin fails.** The crawler gets an error response with status `502` and the body `An internal error occurred`, and no file is made (`publisher.rs` 5778 to 5799, `error.rs` 119 and `http_error` in each adapter's `src/app.rs`). The route never asked the origin, so the route could not fail this way. +1. **The request to the origin fails.** The crawler gets an error response with status `502` and the body `An internal error occurred`, and no file is made (`publisher.rs` 5778 to 5799, `error.rs` 119 and `http_error` in each adapter's `src/app.rs`). RFC 9309 section 2.3.1.4 has a crawler take the `502` as refusing everything. The route never asked the origin, so the route could not fail this way. 2. **A contributor fails and its last answer is kept.** The file is made from the kept answer past its age, and the holding logs a warning (`robots_txt/holding.rs` 111 to 126). The last answer is kept apart for 30 days (`KEPT_LIFETIME`, 39 to 41), in the host's value cache when the host supplies one and in the key-value store otherwise (205 to 249). A value cache may drop the entry early, and Spin's key-value store gives an entry no lifetime, as the configuration guide says (lines 1076 and 1078). -3. **A contributor fails and nothing is kept.** The middleware refuses the request. The crawler gets `503` with `Retry-After: 600` (`RETRY_AFTER_SECONDS`, `robots_txt/mod.rs` 64 to 65), `Content-Type: text/plain; charset=utf-8`, `Cache-Control: no-store, private` and the body `robots.txt is temporarily unavailable`, and nothing is stored (`robots_txt/middleware.rs` 29 to 30 and 94 to 99, `publisher.rs` 851 to 868 and 6035 to 6058). One failed contributor among several fails the whole file, because leaving out its rules would allow by omission what those rules refused (`robots_txt/mod.rs` 26 to 29). The error is logged with its whole report (`robots_txt/middleware.rs` 95). -4. **Nothing was prepared when the action runs.** The origin's body passes unchanged and an error is logged (`robots_txt/middleware.rs` 111 to 115). The comment there says that nothing prepared means the preparation did not run, which no request can cause. +3. **A contributor fails and nothing is kept.** The middleware refuses the request. The crawler gets `503` with `Retry-After: 600` (`RETRY_AFTER_SECONDS`, `robots_txt/mod.rs` 64 to 65), `Content-Type: text/plain; charset=utf-8`, `Cache-Control: no-store, private` and the body `robots.txt is temporarily unavailable`, and nothing is stored (`robots_txt/middleware.rs` 29 to 30 and 94 to 99, `publisher.rs` 851 to 868 and 6035 to 6058). One failed contributor among several fails the whole file, because leaving out its rules would allow by omission what those rules refused (`robots_txt/mod.rs` 26 to 29). The error is logged (`robots_txt/middleware.rs` 95) with every address in what the contributor reported cut to its scheme and host (`addresses_cut_to_their_host` in `robots_txt/holding.rs`), so a key in the path, query or user part of an address is not logged, and the kept answer's warning carries the same text. +4. **Nothing was prepared when the action runs.** The file refuses every crawler, with the `always_allow` paths open, whatever `modules` selects, and an error is logged (`create` in `robots_txt/middleware.rs`). Core prepares every response it streams before the action runs, so no request reaches the action without its preparation. ## 3. What is not in this change - **A contributor in this repository.** `RobotsTxtContributor` comes from row 14, and the type folder of a contributor's crate is `robots-txt` (`MODULE_TYPE`, `robots_txt/mod.rs` 81 to 83), so such a crate belongs under `crates/robots-txt/`. No crate is there at this row or at any row through row 50. The only contributors in the tree are the test probe `SeamProbeRobotsTxt` (`crates/testing/seam-probe/src/lib.rs` 381 to 415) and the test doubles in `robots_txt/holding.rs`. A deployment that wants a contributor supplies a crate of its own. - **A copy of the origin's answer held by core.** Row 34 holds such a copy, as "Caching and stored copies" says. At this row core sends a request to the origin for every request for the file. - **Later changes to the module.** No row through row 50 changes `robots_txt/mod.rs`, `robots_txt/middleware.rs`, `robots_txt/contributor.rs` or `robots_txt/document.rs`. Row 27 changes `robots_txt/holding.rs` so that the publisher's key prefix comes from the runtime services (`2026-10-12-prefix-module-design.md`), and row 33 changes one of its tests (line 895 on that row), which builds its geo service as a `PlatformGeoModule`. Row 26 rewrites the changelog line for the middleware, as "Where the middleware is placed" says. -- **A `HEAD` of its own.** A `HEAD` for `/robots.txt` is the publisher path's, which answers it as a `GET` without a body. The row 22 test `a_head_request_for_a_made_file_gets_the_same_answer` covers a made file, and no test sends a `HEAD` for this file. -- **A run through an adapter.** No file has been served through any of the four adapters against a real origin, as the description of IABTechLab/trusted-server#1366 says, and the manual test through `fastly compute serve` is not ticked. +- **A `HEAD` of its own.** A `HEAD` for `/robots.txt` is the publisher path's, which answers it as a `GET` without a body. The row 22 test `a_head_request_for_a_made_file_gets_the_same_answer` covers a made file, and each adapter's test `robots_txt_is_served_through_the_publisher_path_with_its_tag` sends a `HEAD` for this file. +- **A run through an adapter.** No file has been served through any of the four adapters against a real origin, as the description of IABTechLab/trusted-server#1366 says, and the manual test through `fastly compute serve` is not ticked. Each adapter's tests serve the file against a stub origin. **Open questions.** Each of these is a question about the code. This spec records what the code does and leaves the questions open. -1. Should `refuse_all` reach the crawler whatever the origin answers? Today the origin's answer comes first (the table in section 2). An origin that cannot be reached gives `502` (case 1), a `401`, `403`, `410`, `5xx` or redirect with a type other than `text/plain` reaches the crawler as the origin gave it (case 8), a request that names no host gets an empty `200` for a made file (case 11), and a `text/plain` answer with a charset other than UTF-8 or an encoding core cannot read keeps the origin's own file (cases 5 and 6). The route answered the refusal without asking the origin. The module comment still says a refusal cannot be loosened (`robots_txt/mod.rs` 16 and 17), the comment on `RobotsTxtConfig::refuse_all` still says the refusal needs no network (159 to 163), and the action passes the origin's body when nothing was prepared (`robots_txt/middleware.rs` 111 to 115). The configuration guide (line 1079) and the description of IABTechLab/trusted-server#1366 say that any other answer passes as the origin gave it. The statement holds for cases 5 to 8 and 11, and not for case 9, the `206` of case 10 or a `200` in case 12. No test covers a case other than 2, 3 and 4. -2. Should `path = "/robots.txt"` cover `/robots.txt` alone? The `path` of an entry is a prefix, so a `text/plain` answer for `/robots.txt.bak` or `/robots.txtfoo` takes the file as its body, and a `404` for such a path is made into a `200` file (`middleware.rs` 604 to 623, `publisher.rs` 5810). The route this row removes matched the one path, and the settings check compares the path with `==` (`settings.rs` 4436) where the entry does not. No test uses a longer path. -3. Should core remove the conditional and range request headers for `/robots.txt`, as it does for a page the ad stack answers and for a personalized HTML document? Today they reach the origin (`publisher.rs` 5238 to 5243 and 3279 to 3285), so an origin `304` goes to the crawler with no body, or as the `503` when it declares `text/plain` and a contributor fails with nothing held, and a `206` of type `text/plain` takes the whole file as its body under the `206` status (case 10). The finding for case 10 is by reading the code, and no test sends such a request for the file. -4. Should a test serve `/robots.txt` through an adapter, and should a test run `ts config validate` on a section? No adapter test requests the path, because each requests another route, so the file's own `X-Robots-Tag`, a `HEAD`, a `POST`, and Fastly's streaming against the buffering of the other three are untested end to end, as the description says. The four settings tests reach `require_robots_txt_placed` through `Settings::from_toml`, and none runs `validate_settings_for_deploy_with`. -5. Should the changelog entry for the removed route be marked Breaking? Line 61 is not, though this row refuses a configuration that has the section and no entry, and a build of rows 14 to 24 refuses the new entry (Rollout, items 1 and 2). The section is itself unreleased (line 98), and lines 62 and 63 of the same block carry the mark. -6. Should an error that a contributor returns be logged whole? `robots_txt/holding.rs` 182 builds the load error from the whole report, `robots_txt/holding.rs` 116 to 118 and `robots_txt/middleware.rs` 95 write the report to the log, and core removes nothing from the report, so an error that names an address with a key in it would put the key in the log. The body of the `503` is a constant (`robots_txt/middleware.rs` 29 to 30), so no error text reaches a crawler. +1. Should a test run `ts config validate` on a section? The four settings tests reach `require_robots_txt_placed` through `Settings::from_toml`, and none runs `validate_settings_for_deploy_with`. +2. Should a request that names no host get the file? Core runs no middleware for such a request and so makes no file for it, and the origin's answer reaches it as the origin gave it (case 11). An origin's `404` then reads to a crawler as permission to crawl everything (RFC 9309 section 2.3.1.3), and an origin's own file keeps the origin's rules. ## 4. Tests The tests that cover this behavior are in the table, and the tests of the route go with the route. Each test is in `crates/trusted-server-core/src` unless a crate is named. -| Behavior | Test | -| ---------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| A `GET` that the origin answers `404` is made as a `200` file with `Content-Type: text/plain; charset=utf-8` (case 2) | `robots_txt_is_made_where_the_origin_has_none` in `publisher.rs` | -| A `GET` that the origin answers `200` with an HTML page gets the file (case 3) | `robots_txt_replaces_a_page_the_origin_answers_at_its_path` in `publisher.rs` | -| A `GET` that the origin answers `200` with `text/plain` has its body replaced by the file (case 4) | `robots_txt_replaces_the_origins_file` in `publisher.rs` | -| The prepared file replaces every chunk of the origin's body and is written at the last call | `the_prepared_file_replaces_whatever_the_origin_sent` in `robots_txt/middleware.rs` | -| With nothing prepared the origin's file passes | `with_nothing_prepared_the_origins_file_is_left_as_it_is` in `robots_txt/middleware.rs` | -| A prepared value of another type is not a file | `a_value_of_another_type_under_the_name_is_not_a_file` in `robots_txt/middleware.rs` | -| The middleware handles `text/plain` and not HTML, and names both phases | `the_middleware_works_on_text_in_either_phase` in `robots_txt/middleware.rs` | -| A chain plans the middleware on text | `a_chain_plans_the_middleware_on_text` in `robots_txt/middleware.rs` | -| A contributor that fails with nothing held refuses the request with `503`, a `Retry-After` of 600 seconds and the constant message | `with_nothing_held_a_failure_refuses_the_request` in `robots_txt/mod.rs` | -| `refuse_all` gives the refusal file, asks no contributor and reads no store, whatever else `modules` names | `refusing_every_crawler_asks_no_contributor_and_reads_no_store` in `robots_txt/mod.rs` | -| The registry makes the middleware only for settings that carry the section | `the_middleware_exists_only_with_the_section` in `robots_txt/mod.rs` | -| A contributor is asked for the request's host and path, without the query and without the reader's evidence | `the_middleware_asks_the_contributors_for_the_request_it_answers` in `robots_txt/mod.rs` | -| A name that nothing supplies is refused when the registry is built | `a_selection_nothing_supplies_is_refused_when_the_registry_is_built` in `robots_txt/mod.rs` | -| A section with no entry that places the middleware is refused | `a_robots_txt_section_with_no_entry_placing_the_middleware_is_refused` in `settings.rs` | -| A `[[fetch]]` entry on the file places the middleware, with a `[[serve]]` entry as well | `a_fetch_entry_on_the_files_path_places_robots_txt` in `settings.rs` | -| An entry for `text/plain` with no `path` that names the middleware is refused | `an_entry_naming_robots_txt_on_another_path_is_refused` in `settings.rs` | -| A `[[serve]]` entry alone is refused | `robots_txt_placed_only_in_serve_is_refused` in `settings.rs` | -| The stock list carries the middleware, and the section registers the middleware for the listed phase and media type | `core_s_robots_txt_middleware_is_registered_with_the_section` in `crates/trusted-server-modules/src/lib.rs` | -| The path is no route of the server, and a refusal tags a response of another route | `refusing_every_crawler_tags_every_response_and_registers_no_route` in the Axum, Cloudflare and Spin tests, and `refusing_every_crawler_tags_every_response` in Fastly's `seam_probe_tests.rs` | -| Without the section no blanket rule is placed on responses | `no_blanket_rule_is_placed_on_responses_without_the_section` in Fastly's `seam_probe_tests.rs`, and `a_response_carries_no_blanket_rule_without_the_section` in the Cloudflare and Spin tests | -| Axum refuses at startup a selection that nothing supplies | `robots_txt_selection_naming_a_module_that_is_not_running_fails_at_startup` in the Axum `seam_probe.rs` | - -Eleven things have no test. - -1. A failed request to the origin (case 1), and a `401`, `403`, `410`, `5xx` or redirect answer with or without `text/plain` (cases 8 and 9), with `refuse_all` or a contributor selected. -2. A `200` answer with a charset other than UTF-8 or an encoding core cannot read (cases 5 and 6), and one with a type that is neither `text/plain` nor HTML (case 7). -3. A `206` or `304` answer, and a request that carries `Range` or a conditional header (case 10). -4. A path longer than `/robots.txt` that the entry's prefix covers (open question 2). -5. A request that names no host (case 11). -6. A method other than `GET` and `HEAD`, and a `HEAD`, for `/robots.txt`. The one `HEAD` test is the row 22 test for a made file. -7. An adapter that serves `/robots.txt`, with the file's own `X-Robots-Tag`, and Fastly's streaming against the buffering of the other three. -8. `ts config validate`, or `validate_settings_for_deploy_with`, on a section. -9. A `[[serve]]` entry beside the `[[fetch]]` entry at run time. The settings test accepts both lists, and no test serves a file through both. -10. A file under a `[[cache.asset_rules]]` rule (`publisher.rs` 5988 to 5994). -11. An error text that names an address (open question 6). +| Behavior | Test | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| A `GET` that the origin answers `404` is made as a `200` file with `Content-Type: text/plain; charset=utf-8` (case 2) | `robots_txt_is_made_where_the_origin_has_none` in `publisher.rs` | +| A `GET` that the origin answers `200` with an HTML page gets the file (case 3) | `robots_txt_replaces_a_page_the_origin_answers_at_its_path` in `publisher.rs` | +| A `GET` that the origin answers `200` with `text/plain` has its body replaced by the file (case 4) | `robots_txt_replaces_the_origins_file` in `publisher.rs` | +| A `GET` that the origin answers `401`, `403`, `410`, `500`, `301`, `302`, `204`, `206` or `304`, or `200` with another charset, an encoding core cannot read or another type, gets the refusal made with status `200` (cases 5 to 10) | `robots_txt_is_made_in_place_of_any_answer_the_middleware_cannot_replace` in `publisher.rs` | +| The origin is asked for the file without `If-None-Match`, `If-Modified-Since`, `Range` and `If-Range`, and for a path no such middleware works on with them (case 10) | `the_origin_is_asked_for_the_whole_robots_txt` in `publisher.rs` | +| A request that names no host keeps the origin's `403` (case 11) | `robots_txt_for_a_request_that_names_no_host_keeps_the_origins_answer` in `publisher.rs` | +| A longer path the entry covers keeps the origin's `200`, `404` and `403`, and the file's path with a query is the file (case 13) | `a_longer_path_the_robots_txt_entry_covers_keeps_the_origins_answer` in `publisher.rs` | +| The prepared file replaces every chunk of the origin's body and is written at the last call | `the_prepared_file_replaces_whatever_the_origin_sent` in `robots_txt/middleware.rs` | +| With nothing prepared the file refuses every crawler, whatever `modules` selects | `with_nothing_prepared_the_file_refuses_every_crawler` in `robots_txt/middleware.rs` | +| A prepared value of another type is not a file, so the file refuses every crawler | `a_value_of_another_type_under_the_name_is_not_a_file` in `robots_txt/middleware.rs` | +| The middleware handles `text/plain` and not HTML, and names both phases | `the_middleware_works_on_text_in_either_phase` in `robots_txt/middleware.rs` | +| A chain plans the middleware on text | `a_chain_plans_the_middleware_on_text` in `robots_txt/middleware.rs` | +| A contributor that fails with nothing held refuses the request with `503`, a `Retry-After` of 600 seconds and the constant message | `with_nothing_held_a_failure_refuses_the_request` in `robots_txt/mod.rs` | +| A contributor's failure is logged with each address cut to its scheme and host, with nothing of its user part, path, query or fragment | `a_failure_gives_only_the_host_of_an_address_it_names` in `robots_txt/holding.rs` | +| `refuse_all` gives the refusal file, asks no contributor and reads no store, whatever else `modules` names | `refusing_every_crawler_asks_no_contributor_and_reads_no_store` in `robots_txt/mod.rs` | +| The registry makes the middleware only for settings that carry the section | `the_middleware_exists_only_with_the_section` in `robots_txt/mod.rs` | +| A contributor is asked for the request's host and path, without the query and without the reader's evidence | `the_middleware_asks_the_contributors_for_the_request_it_answers` in `robots_txt/mod.rs` | +| Each of two hosts is answered for itself through the publisher path | `each_host_is_answered_for_itself` in `robots_txt/mod.rs` | +| A forwarded host never reaches a contributor, which is asked for the host the request was addressed to | `a_forwarded_host_is_never_the_host_a_contributor_is_asked_for` in `robots_txt/mod.rs` | +| With no addressed host no contributor is asked and the request is refused, and `refuse_all` still makes its file | `without_the_addressed_host_only_the_refusal_is_made` in `robots_txt/mod.rs` | +| On Axum, which leaves forwarded headers in place, a claimed host never reaches a contributor | `a_forwarded_host_never_reaches_a_robots_txt_module` in the Axum `seam_probe.rs` | +| On Fastly a `GET` of the file is answered before the Edge Cookie steps and the filters, which still answer a `POST` | `robots_txt_is_answered_before_the_edge_cookie_steps_and_the_filters` in Fastly's `seam_probe_tests.rs` | +| A name that nothing supplies is refused when the registry is built | `a_selection_nothing_supplies_is_refused_when_the_registry_is_built` in `robots_txt/mod.rs` | +| A section with no entry that places the middleware is refused | `a_robots_txt_section_with_no_entry_placing_the_middleware_is_refused` in `settings.rs` | +| A `[[fetch]]` entry on the file places the middleware, with a `[[serve]]` entry as well | `a_fetch_entry_on_the_files_path_places_robots_txt` in `settings.rs` | +| An entry for `text/plain` with no `path` that names the middleware is refused | `an_entry_naming_robots_txt_on_another_path_is_refused` in `settings.rs` | +| A `[[serve]]` entry alone is refused | `robots_txt_placed_only_in_serve_is_refused` in `settings.rs` | +| The stock list carries the middleware, and the section registers the middleware for the listed phase and media type | `core_s_robots_txt_middleware_is_registered_with_the_section` in `crates/trusted-server-modules/src/lib.rs` | +| The path is no route of the server, and a refusal tags a response of another route | `refusing_every_crawler_tags_every_response_and_registers_no_route` in the Axum, Cloudflare and Spin tests, and `refusing_every_crawler_tags_every_response` in Fastly's `seam_probe_tests.rs` | +| Without the section no blanket rule is placed on responses | `no_blanket_rule_is_placed_on_responses_without_the_section` in Fastly's `seam_probe_tests.rs`, and `a_response_carries_no_blanket_rule_without_the_section` in the Cloudflare and Spin tests | +| Axum refuses at startup a selection that nothing supplies | `robots_txt_selection_naming_a_module_that_is_not_running_fails_at_startup` in the Axum `seam_probe.rs` | +| Through each adapter, against a stub origin, a `GET` has the origin's file replaced, a `HEAD` gets the made file with no body, a `POST` keeps the origin's `404`, and each carries `X-Robots-Tag` | `robots_txt_is_served_through_the_publisher_path_with_its_tag` in the Axum, Cloudflare and Spin `tests/routes.rs` and in Fastly's `src/app.rs` | + +Six things have no test. + +1. A failed request to the origin (case 1). +2. A contributor selected in place of `refuse_all` for the answers of cases 5 to 10, which the tests send with `refuse_all` selected. +3. A method other than `GET` and `HEAD` answered `200` with `text/plain` (case 12). +4. `ts config validate`, or `validate_settings_for_deploy_with`, on a section (open question 1). +5. A `[[serve]]` entry beside the `[[fetch]]` entry at run time. The settings test accepts both lists, and no test serves a file through both. +6. A file under a `[[cache.asset_rules]]` rule (`publisher.rs` 5988 to 5994). ## 5. Acceptance @@ -250,12 +251,16 @@ git grep -n "answers_robots_txt_with_a_page" 16248774e -- crates 1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/25-robots-txt-middleware` at `16248774e`. Issues found, two of them blocking. The two were that the spec said the settings refuse a section only when no `[[fetch]]` or `[[serve]]` entry names the middleware, when the code requires a `[[fetch]]` entry for `text/plain` on exactly `/robots.txt` and refuses an entry anywhere else, and that the spec left out that the origin's answer now decides what a crawler gets, so an unreachable origin, a `401`, a `403`, a `410` or a redirect reaches the crawler even when `refuse_all` is selected. This revision states what the code does at that head for the placement, each answer the origin can give, the file, what a contributor is given, caching and stored copies, configuration, rollout, the adapters and failure, replaces the acceptance claims with what the pull request description records, points the header at the series pull request and the stack issue, and leaves six questions about the code under Open questions until the code changes. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/25-robots-txt-middleware` at `16248774e`. Issues found, none blocking. This revision states what the code does with a `304` in case 10 and with cases 7 and 8, cites the default of `[cache.robots-txt] module` where the code sets it, and names row 33's change to a test of the holding and the Fastly test file the footprint counts. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/25-robots-txt-middleware` at `16248774e`. Issues found, none blocking. This revision states what the code does with a `304` in case 10 and with cases 7 and 8, cites the default of `[cache.robots-txt] module` where the code sets it, names row 33's change to a test of the holding and the Fastly test file the footprint counts, and checks the row 34 lines against that row's code. ## Revision record -| Date | Change | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | -| 2026-10-16 | Fixes from review round 1. The header and the pull requests the header names, what the change touches, what goes and what stays, the steps of a request, the answers of the origin as a table, the placement rule, the file, what a contributor is given, caching and stored copies, the settings as a table, the rollout, the adapters, failure, the rows that hold what this change leaves out, the open questions, the tests and the cases with none, the acceptance and the review log. | -| 2026-10-16 | Points from review round 2. The `304` of case 10, cases 7 and 8, the lines of the `[cache.robots-txt] module` default, row 33's change to a test of the holding, and the footprint row for the adapter tests. | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-16 | Fixes from review round 1. The header and the pull requests the header names, what the change touches, what goes and what stays, the steps of a request, the answers of the origin as a table, the placement rule, the file, what a contributor is given, caching and stored copies, the settings as a table, the rollout, the adapters, failure, the rows that hold what this change leaves out, the open questions, the tests and the cases with none, the acceptance and the review log. | +| 2026-10-16 | Open questions 1 to 3, 5 and 6, and the adapter half of 4, fixed in the code, each with a test. The file takes the place of any answer the middleware is not handed whole, the action writes the refusal when nothing was prepared, the middleware works on `/robots.txt` alone, the origin is asked for the whole file, each adapter serves the file in a test, the changelog marks the removal of the route as breaking, and a contributor's failure is logged with each address cut to its host. The answer table gains case 13, and the open questions are the two that are left. | +| 2026-10-16 | Points from review round 2. Cases 7, 8 and 10, the lines of the `[cache.robots-txt] module` default, row 33's change to a test of the holding, the footprint row for the adapter tests, and the row 34 lines checked against that row's code. | +| 2026-10-16 | Points from review round 2. The `304` of case 10, cases 7 and 8, the lines of the `[cache.robots-txt] module` default, row 33's change to a test of the holding, and the footprint row for the adapter tests. | +| 2026-10-16 | Contributors are asked for the host the request was addressed to, never a forwarded one, which the publisher path hands every preparation context, and Fastly answers a `GET` or `HEAD` of the file before the Edge Cookie steps and the request filters. What a contributor is given, the adapters and five tests. | +| 2026-10-17 | One text for both sets of code fixes above. The code that holds both keeps the origin's answer, a `404` among them, for a request that names no host, and hands a contributor nothing of the crawler's connection, so what happens, the steps of a request, case 11, open question 2 and what a contributor is given now say so. | From 409e8440a96ad1c4107dc78c29b46e0db2ff93db Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sat, 17 Oct 2026 10:16:43 +0100 Subject: [PATCH 129/192] The specs carry the design of ts config migrate The design of the command that rewrites a settings document written for main into the shape the stack reads, carried by the top pull request of the stack, so it is reviewed here with the others. --- .../specs/2026-10-17-config-migrate-design.md | 129 ++++++++++++++++++ 1 file changed, 129 insertions(+) create mode 100644 docs/superpowers/specs/2026-10-17-config-migrate-design.md diff --git a/docs/superpowers/specs/2026-10-17-config-migrate-design.md b/docs/superpowers/specs/2026-10-17-config-migrate-design.md new file mode 100644 index 0000000000..587fd8e255 --- /dev/null +++ b/docs/superpowers/specs/2026-10-17-config-migrate-design.md @@ -0,0 +1,129 @@ +# Design Spec: `ts config migrate` rewrites a settings document written for main into the shape the stack reads + +**Status:** Proposed, 2026-10-17. The top pull request of the stack, not yet raised, for Tech Lab review. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-10-14-source-modules-design.md`, `2026-10-13-middleware-settings-design.md`, `2026-10-14-rsc-flight-middleware-design.md`, `2026-10-15-geo-module-request-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 +**Related PRs:** none yet +**Issues:** none +**Last updated:** 2026-10-17 + +> **Why this spec exists.** The changelog of the stack lists 38 breaking entries under Unreleased, where main lists 10, and a settings document written for Tech Lab main does not load under the stack. Every operator would rewrite their document by hand from the changelog. `ts config migrate` makes the rewrite automatic where the document holds everything the rewrite needs, and says what is left where it does not. + +## 1. Problem + +A document written for Tech Lab main (`upstream/main` at `5bc388625`) is refused at startup by the stack. `[integrations]` is gone, the origin settings moved to `[source]`, the auction providers moved to `[demand]`, `[[handlers]]` is refused, and a module changes a page only where a `[[fetch]]` or `[[serve]]` entry names its middleware. The changelog lists each change under Unreleased, with 38 entries marked breaking, and the settings code holds a few more that no changelog line names. An operator who upgrades has to read all of them, work out which apply to their document, and rewrite the document by hand without losing the comments they wrote. + +Most of the rewrite is mechanical, because the old shape says everything the new one needs. A few changes need a decision only the operator can make, or a value the document does not hold, and a few change behavior in a way no setting can express. + +## 2. Design + +### 2.1 What the command does + +`ts config migrate ` reads the document, rewrites every shape the stack refuses or reads differently, and prints the result on stdout. On stderr it prints one line for each change it made, one line for each item left for the operator, and the result of validating the migrated document. + +- `--write` rewrites the file in place, and leaves the file untouched when nothing changes. +- `--check` changes nothing and prints nothing on stdout. It exits 1 when the document needs migrating, so CI can hold a document to the current shape. + +The command works on the file alone. It reads no `edgezero.toml`, applies no `TRUSTED_SERVER__` overlay and asks no platform anything, because the rewrite is a property of the document. It validates the result as `ts audit generate` validates a document it is about to write, through `TrustedServerAppConfig` and the same push-time validation `ts config validate` runs, with the modules the tool links. + +Each line on stderr starts with one of three words. + +- `changed:` a rewrite the command made. +- `to do:` an item the document cannot load without. It needs a decision or a value the document does not hold, so the command leaves the setting where it is and says what to do. +- `check:` a behavior change no setting expresses, printed with the rewrite it follows from, or for a document written for main. Nothing in the document has to change. + +The exit status is 0 when nothing is left to do and the migrated document passes validation, 1 when an item is left to do or the result does not pass, and 2 when the file cannot be read or is not TOML. With `--check` it is 0 when the document needs no migrating and 1 otherwise. + +### 2.2 Every change and how it is handled + +The table lists every breaking entry in `CHANGELOG.md` under Unreleased that concerns the document, and the changes in the settings code that no changelog line names, marked "settings code". Entries for crates outside core, for the `ts` tool and for request handling with no setting are grouped in the last rows. + +| Change | Handling | +| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `[integrations]` is gone, and each integration is a module selected in the section of its type | Rewritten. An enabled `[integrations.]` becomes the module in its section, `module` or `modules`, with its settings in `[
.]` and `enabled` dropped. One that was not enabled is commented out under its new name. `enabled` follows main's default for each id. | +| `[auction.providers.]` moved to `[demand] modules` and `[demand.]` | Rewritten. The name takes `_` for `-`, `profile` becomes `implementation` by module path, `profile_config` moves into the table, and `protocol`, which main only took as `openrtb-2.6`, is dropped. | +| `[auction.bidders.] provider` is `module`, naming the demand source | Rewritten, with the demand source's new name. | +| `[auction] mediator = "adserver_mock"` and `[integrations.adserver_mock]` moved to `[ad-server]` | Rewritten to `[ad-server] module = "iabtl"` and `[ad-server.iabtl]`. A mock ad server no mediator names is commented out. | +| `[integrations.aps] rendering_mode` moved to the `auction.aps` demand table | Rewritten into each `auction.aps` demand table. Main read it only from an enabled table. | +| `include_mediator_response` is `include_adserver_response` | Rewritten. | +| The origin settings moved from `[publisher]` to `[source]` | Rewritten to `[source] module = "origin"` and `[source.origin]`, with `origin_url` as `url`, `origin_host_header_override` as `host_header`, `origin_auth` as `auth` and `directory_index` unchanged. | +| An asset route's `origin_url` and `auth` moved to its `source` table | Rewritten to `[proxy.asset_routes.source] module = "origin"` and `[proxy.asset_routes.source.origin]`. | +| The signer spelling `s3_sig_v4` is no longer read | Rewritten to `s3_sigv4` in every `auth` table. | +| `[[handlers]]` and HTTP Basic authentication are removed | Rewritten where an entry guards only `/_ts/admin` or `/admin/keys`, where nothing is served now, by removing it. To do for any other entry, below. | +| `[request_signing]` does not take `config_store_id` or `secret_store_id` | Rewritten by removing both. Check, below, for where the ids go. | +| `[ec] passphrase` is read as deprecated (settings code) | Rewritten to `[ec] module = "hmac"` and `[ec.hmac] passphrase`, where the stack documents it. The deprecated key still loads, with a warning at startup, for one release. | +| `[ec.providers.]` is `[ec.]`, and the module names `host-signals` and `client-fixed` are `host_signals` and `client_fixed` | Rewritten, in the tables, the selection and an `implementation` line. | +| `[permission_signal] sources` is `[permission-signal] modules`, with `gpp-sale-opt-out` and `gpp_sale_opt_out` as `gpp` and `us_privacy` as `us-privacy` | Rewritten. | +| `[device] module = "fastly"` is `"platform"` | Rewritten. | +| The host's location lookup runs only where `[geo]` selects it (settings code) | Rewritten for a document written for main by selecting `[geo] module = "platform"`, the lookup main ran on every request. Without it every request is the top of the `permissions.yaml` rules, and an Edge Cookie module with no geo module is refused at startup. | +| No response carries `x-geo-*` unless `headers.geo` is selected and placed | Rewritten for a document written for main to `[headers] modules = ["geo"]`, with `headers.geo` in the `[[serve]]` entry. Check, below, for the pages it covers. | +| `[tinybird]` is `[analytics.tinybird]`, selected by `[analytics] module` | Rewritten, with `enabled` dropped. A table that was not enabled is commented out. Check, below, for the consent gate. | +| A module's page change runs only where a `[[fetch]]` or `[[serve]]` entry names its middleware | Rewritten by adding one `[[fetch]]` and one `[[serve]]` entry for `text/html`, naming the middleware of the selected modules in the order the linked builders supply them, which is the order `trusted-server.example.toml` shows, when the document has no entry at all. | +| The Flight rewriter is the middleware `framework.nextjs.flight` | Rewritten by adding a `[[fetch]]` entry for `text/x-component` where the Next.js module is selected. Check, below, where it is not. | +| The `/robots.txt` route is the `robots_txt` middleware | Rewritten by adding a `[[fetch]]` entry for `text/plain` on `/robots.txt` to a document with `[robots-txt]` and no entry naming `robots_txt`. | +| `ad-server.mock` is `ad-server.iabtl`, `[adserver]` is `[ad-server]`, and `ads-txt.sellers` is `ads-txt.iabtl` | Rewritten in the selection, the table, an `implementation` line and a `[[fetch]]` middleware name. A selected `[ads-txt]` with no entry naming its middleware gets a `[[fetch]]` entry for `text/plain` on `/ads.txt`. | +| A `[[handlers]]` entry that guards other paths | To do. Nothing asks for a password now, so the operator puts one on those paths at the origin or in the hosting platform and then removes the entry. The command never removes it, because the stack refuses the list so that no deployment believes a path is protected. | +| `[consent.gdpr] applies_in` and `[consent.us_states] privacy_states` are not read (settings code) | To do. The places each covers are the `jurisdiction` values in the `permissions.yaml` compiled into the build, so the operator carries any change of theirs there and removes the table. The command keeps the table, because the list is the value the operator has to carry. | +| A `template_cache_key_cookies` entry under the public prefix is refused | To do, naming the setting. | +| A setting the command cannot place | To do, leaving it where it is. These are an `[integrations.]` naming no module, an `enabled` that is not a boolean, `[integration]`, `rendering_mode` with no `auction.aps` source, a mediator with no mock ad server table, an unknown `profile`, a `profile_config` setting named as one of its provider's own, and both forms of a setting. | +| A key in a module table that main ignored and the module now refuses, in the tables of `gpt`, `didomi`, `lockr`, `permutive`, `sourcepoint`, `testlight`, `nextjs`, `js_asset_proxy`, `google_tag_manager`, `adserver_mock` and `[tinybird]` (settings code) | Reported through the validation result, and `ts config validate` names the key. | +| No response carries `x-ts-version` or the platform's version unless `service_version` is named | Check. The command does not name the middleware, because the changelog lists the version's removal under Security. | +| A path in `origin_url` is kept, and the request path is added after it | Check, where `[source.origin] url` has a path, which main dropped. | +| Core no longer moves the origin's address in a Flight response | Check, where the Next.js module is not selected. Selecting it changes every HTML page, so the command does not. | +| The `x-geo-*` headers go only on the pages the `[[serve]]` entry covers | Check. | +| A telemetry row is sent only for a reader whose permissions include `analytics.ad_reporting.measure_ad_performance` | Check, where the table was enabled. | +| The signing store ids go to `ts keys rotate` and `ts keys deactivate` | Check, where the command removed them. | +| Breaking entries for crates outside core: platform headers, staging, config store chunks, backend naming, the auction protocol, consent decoders, identifier modules, geo modules, cookie constants, the page traits, the modules crate, the seam probe and origin signers | Out of scope. They change code, not the document. | +| Breaking entries for the `ts` tool: `ts cache purge` and `ts prebid client` | Out of scope. | +| Breaking entries for requests with no setting: the admin path check and the removed admin, purge and key rotation routes | Out of scope. | +| Environment variable names, such as `TRUSTED_SERVER__SOURCE__ORIGIN__URL` | Out of scope. They live outside the document. | +| Main's own breaking entries the stack keeps: the auction maps, APS OpenRTB, `site.page`, `bid_param_zone_overrides`, string scalars, Sourcepoint and creative sanitization | Nothing to do. A document written for main already meets them, and the maps move again under the second row. | +| Earlier shapes of the stack the changelog does not name, such as `[ec] provider` | Out of scope. They never reached a release. | + +A document is taken as written for main when it has `[publisher]` and every table at its top level is one main read, which are `publisher`, `tester_cookie`, `trusted_client_ip`, `ec`, `integrations`, `handlers`, `response_headers`, `request_signing`, `rewrite`, `auction`, `consent`, `cache`, `proxy`, `creative_opportunities`, `image_optimizer`, `tinybird` and `debug`. Main refused any other table, and every migrated document has `[source]`, so a second run never takes it as written for main. The geo rows, and the checks about `x-ts-version`, Flight responses and the `x-geo-*` headers, apply to such a document alone, so a document already in the stack's shape that leaves `[geo]` out keeps running with no location. + +### 2.3 How the rewrite keeps what the operator wrote + +The command edits the document with `toml_edit`, the editor the tool already uses, so values, comments and formatting it does not touch come out as they went in. + +- A key that moves keeps its line and the comment written directly above it, and a table that moves keeps its place in the file and its comments. A new section such as `[cmp]` takes the place of the table it replaces, with that table's comments above it. +- A new table that follows another, such as `[source]` after `[publisher]`, comes after the comment lines that directly follow that table's last setting, because `toml_edit` holds those lines with the next table. A comment paragraph is the run of comment lines up to a blank line. +- A table removed with nothing to replace it, such as an admin handler, takes the paragraph directly above it and leaves the earlier ones, such as the file's opening comments, to the next table. +- A module that was not enabled is written as a comment block in its place, under its new name, so the operator runs it by uncommenting the block and selecting the module in its section. A module that has no settings, such as Osano, keeps only the comments above its old table. +- The new sections that replace nothing, `[source]`, `[geo]` and `[headers]`, follow `[publisher]`, and `[ec.hmac]` follows `[ec]`. +- A setting the stack does not read goes with the comment written directly above it. +- The page entries go at the end of the file, after the comment lines that follow the last table. +- Comment text is kept as written, so a comment may still name an old setting. +- A document with Windows line endings keeps them. + +Migrating a migrated document changes nothing, because each rule matches only a shape the stack refuses or a document written for main. + +### 2.4 What the command never does + +- It never prints a value from the document. Every line names the setting. The validation result gives the byte offset where reading failed or the phase that refused, as `ts audit generate` does, because a refusal can quote a value, and `ts config validate` prints the refusal whole. +- It never removes a setting the operator still has to decide on, such as a handler that guards a path of the publisher's own. +- It never names `service_version` or selects the Next.js module. It changes a value the document holds only where the stack renamed what the value names, which is a module, a middleware, a signer, an implementation or a demand source. +- It never writes the file without `--write`, and never writes it when nothing changes. + +## 3. What is not in this change + +- The breaking changes for crates outside core, for the `ts` tool and for request handling, as the table says. +- Pushing the result. The operator runs `ts config validate` and `ts config push` as before. +- Rewriting comment text, which would mean guessing what a comment refers to. +- Earlier shapes of the stack that never reached a release and that the changelog does not name. + +## 4. Tests + +- One test for each rewrite rule in the table, each on a small document, checking the result and the line it reports. +- One test for each to do item, checking that the setting is left in place, that the line names it, and that the exit status says work is left. +- A golden test that migrates main's own `trusted-server.example.toml`, copied from `upstream/main` into `crates/trusted-server-cli/tests/fixtures/main-trusted-server.example.toml` with the commit recorded in its first comment, with its placeholder domain and origin replaced. The result loads and passes validation under the stack's settings code, and migrating it again changes nothing. +- A test that the comments of a document survive and its tables keep their order, and one that a document with Windows line endings keeps them. +- A test that no line the command prints carries a value from the document, run on a document whose every value is a marker string. +- Tests of the command itself, that `--check` writes nothing and exits 1 when the document needs migrating and 0 when it does not, that `--write` writes the file, and that a file that is not TOML exits with an error that quotes nothing from it. +- A test that the page entries name the middleware in the order `trusted-server.example.toml` shows them. + +## Revision record + +| Date | Change | +| ---------- | ------------ | +| 2026-10-17 | First draft. | From 6a24039b627c5234d8f661012eaa3662d8bf8bfa Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:15:44 +0100 Subject: [PATCH 130/192] The analytics module design names the telemetry support table and its test The design records the adapter support table the telemetry guide gains and the Fastly test that each request carries the sink [analytics] selects, for IABTechLab/trusted-server#1148. --- .../specs/2026-10-12-analytics-module-design.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-analytics-module-design.md b/docs/superpowers/specs/2026-10-12-analytics-module-design.md index 5872dd7b44..045ad4b0d0 100644 --- a/docs/superpowers/specs/2026-10-12-analytics-module-design.md +++ b/docs/superpowers/specs/2026-10-12-analytics-module-design.md @@ -4,12 +4,12 @@ **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-diagnostic-headers-design.md` (row 15, which this row stacks on) and `2026-10-12-administration-off-the-publisher-domain-design.md` (row 13), all in IABTechLab/trusted-server#1342, and `2026-06-22-auction-prebid-metrics-tinybird-grafana-design.md`, which IAB Tech Lab's main already holds and whose configuration table this spec supersedes **Related PRs:** [IABTechLab/trusted-server#1357](https://github.com/IABTechLab/trusted-server/pull/1357), [IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356) (row 15, headers that describe the deployment, which this row stacks on), [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354) (row 13, administration leaves the publisher's domain), [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7, the integration seam), [jwrosewell/trusted-server#11](https://github.com/jwrosewell/trusted-server/pull/11) (closed), [IABTechLab/trusted-server#1303](https://github.com/IABTechLab/trusted-server/pull/1303) (closed) -**Issues:** [IABTechLab/trusted-server#1274](https://github.com/IABTechLab/trusted-server/issues/1274) -**Last updated:** 2026-10-16 +**Issues:** [IABTechLab/trusted-server#1274](https://github.com/IABTechLab/trusted-server/issues/1274), and [IABTechLab/trusted-server#1148](https://github.com/IABTechLab/trusted-server/issues/1148) for the telemetry each adapter sends +**Last updated:** 2026-10-17 > **Why this spec exists.** Auction telemetry had a section named after its vendor and a flag inside that section that switched it on, where other jobs select their module by name. This change makes telemetry a module that `[analytics]` selects, and this spec states what the code at row 16 does. -Paths in this document are relative to `crates/trusted-server-core/src`, with these exceptions. `tinybird.rs` and `app.rs` are in `crates/trusted-server-adapter-fastly/src`. Paths that start with `docs/` or `tinybird/`, and `CHANGELOG.md` and `trusted-server.example.toml`, are relative to the repository root. The files of EdgeZero are named in full where they appear. Line numbers are those of commit `7b19336ec`, the head of row 16, which is the branch `split/16-analytics` on the fork and which IABTechLab/trusted-server#1357 carries as `feature/16-analytics`. The code the three fix commits after it add is named by function rather than by line. +Paths in this document are relative to `crates/trusted-server-core/src`, with these exceptions. `tinybird.rs` and `app.rs` are in `crates/trusted-server-adapter-fastly/src`. Paths that start with `docs/` or `tinybird/`, and `CHANGELOG.md` and `trusted-server.example.toml`, are relative to the repository root. The files of EdgeZero are named in full where they appear. Line numbers are those of commit `7b19336ec`, the head of row 16, which is the branch `split/16-analytics` on the fork and which IABTechLab/trusted-server#1357 carries as `feature/16-analytics`. The code the fix commits after it add is named by function rather than by line. ## 1. Problem @@ -17,7 +17,7 @@ After the integration seam (the change that lets a vendor's module live outside ## 2. Design -**Auction telemetry is a module `[analytics]` selects.** The change sits above the integration seam and on top of the diagnostic headers change ([IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356)). It is three commits, with seven merges of the row below. Commit `7505ff704` adds the section, the selector, the refusals, the secret reference, the Fastly sink and the example configuration. Commit `75fc6977e` updates the configuration guide, the telemetry guide, the configuration rules, the Tinybird project's README and the changelog. Commit `7de3882f1` names the constant, accessors, helpers and tests analytics where they said telemetry, and gives the two `[analytics]` refusals the wording of the generic module section and of the registry, which checks a section's selection against the modules a deployment supplies. Together they change 11 files, adding 600 lines and removing 180. Three fix commits follow, for what review round 1 found in the code. Commit `cdb0ab122` sends no row about a reader who has not permitted ad performance measurement, commit `71fce6dbf` has the Axum, Cloudflare and Spin adapters warn at startup that they send nothing, and commit `4bf0d8a0e` corrects the changelog, the telemetry guide and the Tinybird project's README to match. +**Auction telemetry is a module `[analytics]` selects.** The change sits above the integration seam and on top of the diagnostic headers change ([IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356)). It is three commits, with seven merges of the row below. Commit `7505ff704` adds the section, the selector, the refusals, the secret reference, the Fastly sink and the example configuration. Commit `75fc6977e` updates the configuration guide, the telemetry guide, the configuration rules, the Tinybird project's README and the changelog. Commit `7de3882f1` names the constant, accessors, helpers and tests analytics where they said telemetry, and gives the two `[analytics]` refusals the wording of the generic module section and of the registry, which checks a section's selection against the modules a deployment supplies. Together they change 11 files, adding 600 lines and removing 180. Three fix commits follow, for what review round 1 found in the code. Commit `cdb0ab122` sends no row about a reader who has not permitted ad performance measurement, commit `71fce6dbf` has the Axum, Cloudflare and Spin adapters warn at startup that they send nothing, and commit `4bf0d8a0e` corrects the changelog, the telemetry guide and the Tinybird project's README to match. Two commits follow for [IABTechLab/trusted-server#1148](https://github.com/IABTechLab/trusted-server/issues/1148). Commit `99abf9f4f` adds a table of the telemetry each adapter sends to the telemetry guide, and a line to the Axum, Cloudflare and Spin guides, and commit `b1a47b8fc` adds a Fastly test that each request carries the sink `[analytics]` selects. ```toml # Before @@ -120,7 +120,7 @@ Row 16 adds or changes these tests. - **The refusals in section 2.** `an_analytics_section_selecting_nothing_is_refused` (line 6335), `an_analytics_module_this_build_does_not_have_is_refused` (line 6350, which checks both sentences of the message), `a_selected_analytics_module_with_no_table_is_told_what_it_needs` (line 6305), `the_analytics_module_table_refuses_a_key_it_does_not_know` (line 6394, with `enabled`) and `the_tinybird_table_is_refused_with_directions` (line 6421, with `enabled = true` and with `enabled = false`). - **The module's own settings.** `the_analytics_module_needs_a_regional_host` (line 6277), `the_analytics_module_needs_its_token` (line 6290), `an_analytics_token_reference_that_is_empty_or_absent_is_refused_when_the_settings_load` (line 6374) and `access_telemetry_is_rejected_until_an_emitter_is_wired` (line 6320). - **The secret reference.** In `config.rs`, `secret_metadata_lists_all_secret_paths_and_optionality` (line 996) checks the declared path and its optional flag, `app_config_new_rejects_an_analytics_secret_key_reference_that_is_empty_or_absent` (line 1254) checks an empty, a whitespace-only and an absent reference at deploy, `app_config_new_accepts_an_analytics_secret_key_reference` (line 1283) checks that a key name is accepted, `legacy_static_secret_store_selectors_are_accepted_but_not_serialized` (line 1071) checks that `secret_store` is accepted and not written back, and `settings_debug_redacts_resolved_static_credentials` (line 1105) checks that a resolved token does not reach debug output. In `config_payload.rs`, `resolves_all_static_credentials_from_the_mapped_default_store` (line 510) checks that a selected module's key name is replaced by the stored secret, and `an_unselected_analytics_module_table_is_refused_without_resolving_its_secret` (line 896) checks that a table nothing selects is refused with `[analytics] selects no module`, and that the error names neither the table's key nor a secret reference (lines 915 to 922). In `inspect/config/tests.rs`, `every_declared_core_secret_is_masked_and_never_served` (line 296) loads a document that holds the token's path. In `settings.rs`, `settings_debug_output_redacts_every_secret_field` (line 5331) checks that neither Tinybird token reaches debug output. -- **The sink.** In `tinybird.rs`, `settings_selecting_no_analytics_module_build_a_sink_that_sends_nothing` (line 520) checks that settings selecting nothing build a sink that is off, sends nothing and registers no backend, and `settings_selecting_the_analytics_module_build_a_sink_that_sends` (line 551) checks that settings selecting the module build a sink that is on. +- **The sink.** In `tinybird.rs`, `settings_selecting_no_analytics_module_build_a_sink_that_sends_nothing` (line 520) checks that settings selecting nothing build a sink that is off, sends nothing and registers no backend, and `settings_selecting_the_analytics_module_build_a_sink_that_sends` (line 551) checks that settings selecting the module build a sink that is on. In `app.rs`, `per_request_services_carry_the_sink_analytics_selects` builds the state from the shared test settings without and with the documented `[analytics]` block, and checks that each request's services carry a sink that sends exactly when the section selects the module. - **The example.** `documented_analytics_block_validates_when_uncommented` (`config.rs`, line 731) uncomments the block in `trusted-server.example.toml` and loads it. - **The measurement permission.** In `auction/telemetry.rs`, `no_row_is_built_for_a_reader_who_has_not_permitted_ad_measurement` checks that a skipped and a completed auction give no row without the permission, and that the permission alone lets the summary row through. `a_consent_denied_auction_sends_no_row_without_the_ad_measurement_permission` (`auction/endpoints.rs`), `navigation_sends_no_row_when_consent_denies_without_ad_measurement` and `page_bids_sends_no_row_when_consent_denies_without_ad_measurement` (`publisher.rs`) drive the three paths that skip for consent, with a User-Agent and a reader who has set no permission, and check that nothing reaches the sink. `auction_endpoint_consent_gate_returns_no_bid_without_contacting_providers` (`auction/endpoints.rs`) gives its refused reader the permission alone and checks that the skip is still recorded. The test contexts with the gate open permit measurement (`permissions_for_test` in `ec/mod.rs`), so the other telemetry tests keep their rows. - **The startup warning.** In `auction/telemetry.rs`, `an_adapter_that_sends_nothing_warns_once_when_analytics_selects_a_module` captures the log on its own thread and checks for one warning naming the adapter after two calls, and `no_warning_when_analytics_selects_nothing_or_a_supplied_sink_sends` checks that nothing is logged when nothing is selected or a supplied sink sends. The Axum, Cloudflare and Spin adapters each have `startup_logs_that_this_adapter_sends_no_auction_telemetry`, which builds the adapter's state from settings that carry the documented `[analytics]` block (`documented_block` in `test_support.rs`) and checks that startup logged. @@ -140,6 +140,8 @@ Row 16 adds or changes these tests. **The fix commits.** The three fix commits sit on `d76489aa4`, the head of `split/16-analytics` when they were made, whose first parent is `7b19336ec`. They have not been pushed, so no CI check has run on them. Locally at `4bf0d8a0e`, the tests of core and of the Axum, Cloudflare and Spin adapters pass, 3,020 of them with none failing, and so do clippy on those four packages with every target and warnings refused, `cargo fmt --all -- --check` and `cargo clippy-fastly`. Run against `d76489aa4` without the fix, the tests of the three paths that skip for consent each fail on a `consent_denied` summary row that carries the reader's complete User-Agent and page path. The Axum adapter's startup test fails without its call to the warning, and the core test of the warning fails on two lines without the flag that keeps it to one. +**The commits for #1148.** `99abf9f4f` and `b1a47b8fc` sit on `f0b8b6e08`, the head of `split/16-analytics` when they were made, and have not been pushed. Locally `per_request_services_carry_the_sink_analytics_selects` passes under Viceroy, and fails with `left: false` and `right: true` when `build_per_request_services` no longer hands each request the startup sink. Clippy on the Fastly adapter for `wasm32-wasip1`, with every target and feature and warnings refused, passes, and so does `cargo fmt --all -- --check`. + **What the descriptions do not record.** The descriptions of IABTechLab/trusted-server#1357, IABTechLab/trusted-server#1303 and jwrosewell/trusted-server#11 hold no record of a lint and a test of each commit on its own, or of deliberate faults made one at a time, so this spec claims neither. **Not exercised.** No event has been sent to Tinybird. The description of #1357 says so, and the sink's tests send to a recording stand-in. @@ -166,3 +168,4 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the accessor's name, the two refusal messages and the table, the checks at load and at deploy and what the declared secret field does, the single destination, the settings and their defaults, the adapters, what a row holds and when rows are sent, the failure of a refused setting, the June design's table, the upgrade order, the later row that moves the module into a crate, the tests, the acceptance, the open questions and the review log. | | 2026-10-16 | Code fixes from review round 1, findings C1 and C2. Rows are built only for a reader whose permissions include `MeasureAdPerformance`, and the Axum, Cloudflare and Spin adapters warn at startup when `[analytics]` selects a module. Open questions 1 and 2 leave the list, apart from the paragraph on `browser_family`, and the tests for both fixes are added. | | 2026-10-16 | Corrections from review round 2. The line cited for `MeasureAdPerformance`, what the test of an unselected table checks, the claims about rows 46 and 50, that IAB Tech Lab's main holds the June design, and the review log. | +| 2026-10-17 | For IABTechLab/trusted-server#1148, the telemetry guide's table of the telemetry each adapter sends and a line in the Axum, Cloudflare and Spin guides (`99abf9f4f`), and the Fastly test that each request carries the sink `[analytics]` selects (`b1a47b8fc`). | From 96471e8eb96ad521153faf2f30272d8428f5a330 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:15:46 +0100 Subject: [PATCH 131/192] The administration design records its code review fixes and the opid_store removal The design records the fixes from the code review of 16 October and, for IABTechLab/trusted-server#1149, the development files and the Viceroy template naming no opid_store, with the test that every store a development file names is declared. --- ...tration-off-the-publisher-domain-design.md | 39 ++++++++++--------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md index 357a0a3df4..6fa9cf8680 100644 --- a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md +++ b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md @@ -4,8 +4,8 @@ **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-inspection-endpoints-design.md` (row 9, which builds `GET /_ts/data`) and `2026-10-12-attestation-design.md` (row 12, which this row stacks on), in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) **Related PRs:** [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354) (row 13), [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353) (row 12, attestation, which row 13 stacks on), [IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350) (row 9, the inspection endpoints), [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7, the integration seam, which replaced the closed [IABTechLab/trusted-server#1094](https://github.com/IABTechLab/trusted-server/pull/1094)), [IABTechLab/trusted-server#1377](https://github.com/IABTechLab/trusted-server/pull/1377) (row 36, which later changes the closed path matcher) -**Issues:** [IABTechLab/trusted-server#1271](https://github.com/IABTechLab/trusted-server/issues/1271). [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-16 +**Issues:** [IABTechLab/trusted-server#1271](https://github.com/IABTechLab/trusted-server/issues/1271), and [IABTechLab/trusted-server#1149](https://github.com/IABTechLab/trusted-server/issues/1149) for a store the development files name. [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack +**Last updated:** 2026-10-17 ## 1. Problem @@ -49,21 +49,21 @@ Both commands use code from core that is compiled into the CLI. A CLI built from **What the two commands take care of.** The routes ran inside the service, where a store is read locally. The commands reach the same stores through the platform's API, so three things are handled that the routes did not have to handle. -- **A read that fails is not a missing entry.** The rotation code in core takes any failed read for an entry that does not exist (`crates/trusted-server-core/src/request_signing/rotation.rs`, lines 68 to 72 and 305 to 309). Through an API a read can also fail for a bad token or a network fault, and read that way a rotation would write a key list holding the new key alone, and a deactivation could take the current key out of the active set. The command's stores tell a `404` from any other failure and write nothing once a read has failed. They tell the two apart by the wording `the Fastly API returned 404` in the report of the `fastly` CLI (`crates/trusted-server-cli/src/commands/keys.rs`, lines 28 to 33). Only the test fake is known to print that wording. No recorded run of the `fastly` CLI has printed that wording, and the one live check saw `401` alone. If the real wording differs, every rotation stops before it writes anything, because a rotation reads the config store for the new key id, which is not stored yet (`rotation.rs`, lines 145 to 161), and a delete tried again after a partial failure reports an error in place of finishing. Open question 1 asks for a recorded run. +- **A read that fails is not a missing entry.** The rotation code in core takes any failed read for an entry that does not exist (`crates/trusted-server-core/src/request_signing/rotation.rs`, lines 68 to 72 and 305 to 309). Through an API a read can also fail for a bad token or a network fault, and read that way a rotation would write a key list holding the new key alone, and a deactivation could take the current key out of the active set. The command's stores tell a `404` from any other failure and write nothing once a read has failed. They tell the two apart by the wording `the Fastly API returned 404` in the report of the `fastly` CLI (`crates/trusted-server-cli/src/commands/keys.rs`, lines 28 to 33). Fastly CLI 15.1.0 and 16.1.0 print that wording for a missing config store, config store entry, secret store and secret store entry, as `ERROR: the Fastly API returned 404 Not Found: Record not found.` on the config side and `ERROR: the Fastly API returned 404 Not Found: Not Found (not found).` on the secret side, read without writing on 2026-10-16. The test fake answers with those texts (`11bf7b76d`). - **A delete can be run again.** The service's management client took a `404` on a delete for success (row 12, `crates/trusted-server-adapter-fastly/src/management_api.rs`, lines 244 and 324), which is how a delete tried again after a partial failure finished. The command does the same, after asking for the store, because the API also answers `404` for a store that does not exist. - **A value is never read as a flag.** Every value is joined to its flag with `=`. A key being retired may begin with `-`, since the rule for retiring (`kid_is_well_formed`) is the looser one the deactivate route applied, kept so that a key created under an earlier rule can still be removed. A new key must begin with a lower case letter (`kid_is_creatable`). Neither command takes `current-kid` or `active-kids` for a key. They are the two entries the config store keeps its key list in, and named as a key either would be written over or deleted. -`secret-store-entry create` is given `--name` and `--stdin` and no option to replace a secret that already has the name (`keys.rs`, lines 239 to 256). The removed client created or recreated (row 12, `management_api.rs`, lines 66 to 72), and the test fake replaces without a word (`keys.rs`, lines 533 to 540). What the real CLI does with a name that exists is not recorded. Open question 2 asks whether to replace a secret. +`secret-store-entry create` is given `--name` and `--stdin` and no option to replace a secret that already has the name (`keys.rs`, lines 239 to 256). The API refuses to create a secret under a name the store holds, so a rotation onto a secret that a half finished rotation left behind fails before it writes the config store. The removed client created or recreated (row 12, `management_api.rs`, lines 66 to 72). The test fake used to replace without a word, and now refuses as the API does, with a test of that rotation (`11bf7b76d`). **Closed paths.** `closed_path_response`, in core's new `crates/trusted-server-core/src/closed_paths.rs`, answers `404` to a request whose path begins with `/_ts/admin`, and to `/admin/keys` and the paths beneath it, before the publisher fallback, on all four adapters. The prefix is matched as plain text, so `/_ts/adminec` is closed with `/_ts/admin/ec`, while `/admin/keystore` and `/admin/login` are left to the publisher. The answer is the same for every method, `404` with `{"error":"not found"}`, `Cache-Control: no-store` and `X-Content-Type-Options: nosniff`, where some removed routes answered a method they did not serve with `405`. The answer does not say where administration went, and neither `closed_path_response` nor the adapters' calls to it write a log line of their own. An operator's script that was not updated sends its password and its key payload to a path that answers `404`, and never to the origin. The matcher is the one that was there before. It tests the path as sent and after each of up to four rounds of percent-decoding (turning `%2F` back into `/`), so a separator encoded up to four times is recognised and one encoded five or more times is not. Row 36 (the publisher origin as a folder of signed object storage, [IABTechLab/trusted-server#1377](https://github.com/IABTechLab/trusted-server/pull/1377)) shares one decoding helper between the origin check and the closed path check, so that a path still changing after the fourth round is closed. -Fastly, Cloudflare and Spin make the check the first step of their fallback, ahead of the modules' request preparers (each can clean up a query parameter or cookie its module reserves). Axum runs the preparers first (`crates/trusted-server-adapter-axum/src/app.rs`, lines 288 and 347), so a preparer that refuses a request answers a closed path before the check does. Open question 6 asks whether Axum should check first. +Fastly, Cloudflare and Spin make the check the first step of their fallback, ahead of the modules' request preparers (each can clean up a query parameter or cookie its module reserves). Axum makes the check in `fallback_handler`, ahead of the preparers, in the same order (`23bde99bc`). -The check sits in the publisher fallback, so a route that the router answers first is not covered by the check. The settings choose one such route, the `[attestation] endpoint`, and the list of addresses an endpoint may not take lost its six administration addresses in this change (`crates/trusted-server-core/src/attestation.rs`, lines 49 to 66). A configuration with `endpoint = "/_ts/admin/x"` therefore passes validation (`validate_endpoint`, line 684) and is served on `GET` and `HEAD` on all four adapters, although the API reference says that no route is served beneath the prefix (`docs/guide/api-reference.md`, line 985). Open question 3 asks whether the endpoint should be refused beneath that prefix. +The check sits in the publisher fallback, so a route that the router answers first is not covered by the check. The settings choose one such route, the `[attestation] endpoint`, and the list of addresses an endpoint may not take lost its six administration addresses in this change (`crates/trusted-server-core/src/attestation.rs`, lines 49 to 66). `validate_endpoint` refuses an endpoint whose page or JSON address the closed path check closes, as `reserved_endpoint` naming the address (`df0fc18a0`), so no route is served beneath the prefix, as the API reference says (`docs/guide/api-reference.md`, line 985). **On the publisher's own site.** @@ -72,6 +72,8 @@ The check sits in the publisher fallback, so a route that the router answers fir - **`/admin/keys` stays closed.** The path and the paths beneath it answer `404` on the publisher's own site, as they did before this change, so a publisher whose site uses `/admin/keys` for its own pages cannot serve them through this server. - No consent or permission logic changes. +**A store nothing opens.** `.env.dev` and `.env.example` set `TRUSTED_SERVER__SYNTHETIC__OPID_STORE=opid_store`, and the Viceroy template of the integration tests declared `opid_store`, which no code opens and `fastly.toml` does not declare. None of the three names it now (`1666c38e3`, for [IABTechLab/trusted-server#1149](https://github.com/IABTechLab/trusted-server/issues/1149)). + ## 3. What is not in this change - Five debugging aids that also live on the publisher's domain, all off unless a publisher switches them on. They are of the same kind as what is removed here and are left for a decision of their own. @@ -97,34 +99,31 @@ If refusing is too abrupt for a live deployment, the alternative is small. The r ## 5. Tests - **Closed paths in core.** Six tests in `closed_paths.rs` check each closed path with seven methods (`GET`, `POST`, `HEAD`, `OPTIONS`, `PUT`, `PATCH` and `DELETE`) for the `404` and `Cache-Control: no-store`. They cover the administration prefix and what is beneath it, encoded separators, separators encoded more than once, the `/admin/keys` alias, paths left to the publisher and the shape of the answer. -- **Each adapter.** Fastly, Axum, Cloudflare and Spin each have a test that sends a Basic `Authorization` header and a body with `GET` and `POST` to the closed paths and expects the `404` and `no-store`, and a test that sends the four key administration paths with every method the publisher fallback takes and expects `404`. The cross-adapter parity test sends the four key paths to Axum, Cloudflare and Spin with `POST` and `GET` and expects no `WWW-Authenticate`. +- **Each adapter.** Fastly, Axum, Cloudflare and Spin each have a test that sends a Basic `Authorization` header and a body with `GET` and `POST` to the closed paths and expects the `404` and `no-store`, and a test that sends the four key administration paths and `/_ts/admin/cache/purge` with every method the publisher fallback takes and expects `404`. The Fastly test of closed paths sends the double-encoded forms the other three send (`18aba165c`). The cross-adapter parity test sends the four key paths to Axum, Cloudflare and Spin with `POST` and `GET` and expects no `WWW-Authenticate`. - **Refusal.** The settings tests refuse `[[handlers]]` in TOML and in a stored document, refuse an empty list, refuse each store id in TOML and one in a stored document, and check that the settings written carry no `handlers` and no store ids. - **The site's own pages.** The DataDome test `only_the_servers_own_paths_are_internal` checks that `/admin/`, `/admin/login` and `/wp-admin/` are no longer the server's own paths, and the publisher test `an_authorization_never_reaches_the_shared_template` checks that a request with an `Authorization` header bypasses the shared template. -- **The commands.** The tests of `ts keys` and `ts cache purge` run against a fake of the `fastly` CLI. They check the key on standard input, `=` between each flag and its value, the reads that fail and write nothing, the delete tried again, the surrogate keys the two scopes name, and the refused service ids and page addresses. No test runs the real `fastly` CLI. +- **The commands.** The tests of `ts keys` and `ts cache purge` run against a fake of the `fastly` CLI. They check the key on standard input, `=` between each flag and its value, the reads that fail and write nothing, the delete tried again, the surrogate keys the two scopes name, and the refused service ids and page addresses. The fake answers a missing store, entry or secret with the text the CLI prints, and refuses to create a secret under a name the store holds, as the API does. No test runs the real `fastly` CLI. - **The purge itself.** The Viceroy test that stored a template, purged the template through the removed route and read the template again is removed, with its step in the workflow. Two tests in the Fastly adapter's `template_cache.rs` call the platform's purge on the key every template carries and on one page's reader key, and check that the stored template goes. - **The smoke scripts.** The four adapter smoke scripts lose the handler password and its missing-secret case, and no workflow runs them. +- **Development store names.** `every_store_a_development_file_names_is_declared`, in the integration tests' `tests/common/config.rs`, checks that every store an uncommented `*_STORE` line of `.env.dev` or `.env.example` names is declared in `fastly.toml` and in the Viceroy template (`e110aa025`). + +**Tests added after review.** `rotate_never_replaces_a_secret_stored_under_the_new_kid`, `the_recorded_404_of_the_real_cli_reads_as_not_found` and `the_value_is_read_from_the_fields_the_real_cli_prints` in the CLI, `an_endpoint_beneath_the_closed_administration_prefix_is_refused` in core, and `a_closed_path_is_answered_before_any_preparer_runs` in the Axum adapter's seam probe tests. -**What the tests leave out.** `/_ts/admin/cache/purge` is in no adapter test and only in the core test, and the Fastly test of closed paths lacks the double-encoded forms that the Axum, Cloudflare and Spin tests have. No test covers the order of the preparers on Axum or an attestation endpoint beneath `/_ts/admin`. Open questions 3, 4 and 6 ask about these. +**What the tests leave out.** No test runs the real `fastly` CLI against a store that exists, so a rotation, a retirement, a repeated `--delete` and a purge are not recorded against a real store or service. ## 6. Acceptance **CI.** All 25 checks pass on the pull request for this row ([IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354)) at `f735e0f85`. The fork's pull request ([jwrosewell/trusted-server#8](https://github.com/jwrosewell/trusted-server/pull/8)) ran 19 checks on the same commit. -**Local checks.** The description of the pull request records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run, and marks a manual run with `fastly compute serve` as not done. +**Local checks.** The description of the pull request records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run, and marks a manual run with `fastly compute serve` as not done. The five commits after review, `11bf7b76d` to `e75e307de`, were checked with rustfmt and Prettier and were not built locally. The two commits for #1149, `1666c38e3` and `e110aa025` on `7ba7d7b55`, were built locally. `every_store_a_development_file_names_is_declared` passes, and with the `opid_store` line put back in `.env.dev` it fails with ``.env.dev sets TRUSTED_SERVER__SYNTHETIC__OPID_STORE to `opid_store`, which fastly.toml does not declare``. Clippy on the integration tests, with every target and warnings refused, passes, and so does `cargo fmt --all -- --check`. -**The command lines.** Each command line the two commands build was given to Fastly CLI 16.1.0, where `.tool-versions` pins 15.1.0, with a token that is not one and with store and service ids that do not exist. The CLI accepted every line and the API refused each with `401`. No run has reached a store or a purge, so the answers of the real CLI to a store that exists, an entry that is missing and a secret that already exists are not recorded. Open questions 1 and 2 ask for those runs. +**The command lines.** Every flag the two commands pass is in the `--help` of Fastly CLI 15.1.0, which `.tool-versions` pins, and of 16.1.0. On 2026-10-16 both versions answered read-only calls for a missing config store, a missing entry, a missing secret store and a missing secret with the wording in section 2, and printed `item_value` as a top level string in `config-store-entry describe --json`. No run has written to a store or purged. ## 7. Open questions -Each of these is a question about the code. This spec records what the code does at `f735e0f85` and decides none of them. +One question is open, about the guides, and this spec does not decide it. -1. What does the real `fastly` CLI print for a `404`, and do the command lines work on the pinned 15.1.0? The wording the commands look for is the test fake's (`crates/trusted-server-cli/src/commands/keys.rs`, lines 28 to 33 and 487 to 489), the fake is written to answer as the real CLI does (lines 424 to 426), and no run of `ts keys rotate`, `ts keys deactivate`, a repeated `--delete` or `ts cache purge` has reached a real store or service. Recording those runs against a staging service would settle the wording, the shape of the `describe --json` answer that `item_value` reads (lines 157 to 167) and the flags. -2. Should `secret-store-entry create` be given an option to replace a secret that already has the name? A rotation repeated after a half-finished one, with a secret already stored under the new key id, is not tested. -3. Should `validate_endpoint` refuse an `[attestation] endpoint` beneath `/_ts/admin`, with a test? Nothing else refuses the prefix. -4. Which tests should be added? `/_ts/admin/cache/purge` for each adapter, and the double-encoded forms for Fastly (`crates/trusted-server-adapter-fastly/src/app.rs`, lines 2123 to 2145). -5. Should three comments and two changelog entries be corrected? The comment above `reader_url_surrogate_key` says that the endpoint and the CLI need it (`crates/trusted-server-core/src/platform/template_cache.rs`, line 211). The comment in the publisher path says the keys are those the purge endpoint uses (`crates/trusted-server-core/src/publisher.rs`, line 4564). The module comment of request signing says the store ids come from the settings (`crates/trusted-server-core/src/request_signing/mod.rs`, lines 20 and 21). The changelog entry for the two store id settings says that only the removed key rotation routes read them (`CHANGELOG.md`, line 61), where on row 12 the checks that both were filled in read them too (`crates/trusted-server-core/src/auction/plan.rs`, lines 725 to 731, and `crates/trusted-server-core/src/config.rs`, lines 458 to 464). The changelog entry for the three Edge Cookie routes names `GET /_ts/data` after all three without saying that the page covers neither lookup by id nor the cookie echo (`CHANGELOG.md`, line 63). -6. Should Axum run the closed path check before the modules' request preparers, as the other three adapters do? -7. Which order should the guides give for the switch and for a rollback? Section 4 says what each build does with the other's configuration, and nothing in the code or the guides gives a sequence. +1. Which order should the guides give for the switch and for a rollback? Section 4 says what each build does with the other's configuration, and nothing in the code or the guides gives a sequence. ## Review log @@ -138,3 +137,5 @@ Each of these is a question about the code. This spec records what the code does | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the unfinished callout removed, the table header and a row for each Edge Cookie route, what an operator loses, the commands and their flags and token scopes, the closed paths and what they leave open, the effects on a publisher's own site, five debugging aids, the upgrade with the switch and the rollback, the tests, the acceptance without the unlisted faults, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. Open question 5 names the changelog entry on the two store id settings and the checks on row 12 that also read them, the sandbox answer is stated to exist only in a build with the `reusable-sandbox` feature, and the check counts of 25 on the row 13 pull request and 19 on the fork's were read again and stand. | +| 2026-10-16 | Fixes from code review. The test fake answers with the 404 text the CLI prints and refuses to replace a secret (`11bf7b76d`), an attestation endpoint beneath `/_ts/admin` is refused (`df0fc18a0`), Axum answers a closed path before the preparers (`23bde99bc`), every adapter test sends the purge path and Fastly's the double-encoded paths (`18aba165c`), and three comments and two changelog entries are corrected (`e75e307de`). Open questions 1 to 6 are answered. | +| 2026-10-17 | For IABTechLab/trusted-server#1149, the development files and the Viceroy template no longer name `opid_store` (`1666c38e3`), and a test checks that every store a development file names is declared (`e110aa025`). | From 9c12b6cdcfa75a413316134402a07d55ee8af69d Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:15:48 +0100 Subject: [PATCH 132/192] The attestation design records its code fixes and the keys on Cloudflare and Spin The design records the three code fixes of 16 October and how the key schedule reaches a Cloudflare build and a Spin build. --- .../specs/2026-10-12-attestation-design.md | 71 ++++++++++--------- 1 file changed, 36 insertions(+), 35 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-attestation-design.md b/docs/superpowers/specs/2026-10-12-attestation-design.md index 3bbfff924f..4a32d3ae34 100644 --- a/docs/superpowers/specs/2026-10-12-attestation-design.md +++ b/docs/superpowers/specs/2026-10-12-attestation-design.md @@ -5,15 +5,15 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-01-15-attestation-design.md` **Related PRs:** IABTechLab/trusted-server#1353, which is row 12 of the stack, builds on one app-config blob for each host in [IABTechLab/trusted-server#1352](https://github.com/IABTechLab/trusted-server/pull/1352) and was reviewed on [jwrosewell/trusted-server#7](https://github.com/jwrosewell/trusted-server/pull/7) (closed without merging on 15 October). [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) opens the integration so that a vendor module can live outside core, and [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354), row 13, follows this change **Issues:** [IABTechLab/trusted-server#161](https://github.com/IABTechLab/trusted-server/issues/161) asks for attestation, [IABTechLab/trusted-server#1235](https://github.com/IABTechLab/trusted-server/issues/1235) asks for attested release binaries, and [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-17 -> **Why this spec exists.** The attestation issue (#161) asks for a reviewed design before any attestation lands. This spec is that design for the code at `59e09b6cd`. It states what the code does, what the code leaves out and which questions about the code are open. +> **Why this spec exists.** The attestation issue (#161) asks for a reviewed design before any attestation lands. This spec is that design for the code at `59e09b6cd` and the three fixes of 16 October at `694926002`. It states what the code does, what the code leaves out and which questions about the code are open. ## 1. Problem The Tech Lab's attestation issue (#161) asks for a reviewed design before any attestation lands, and its January proposal (`2026-01-15-attestation-design.md`) names two required core decisions, configuration kept apart from the build and modular integration crates. Nothing in core serves evidence of who operates a deployment and which build it runs, so a reader, a publisher or a regulator has only the operator's word. -Line references in this document are to commit `59e09b6cd`, the head of `feature/12-attestation`, unless a row is named. +Line references in this document are to commit `59e09b6cd`, the head of `feature/12-attestation`, unless a row is named. The fixes of 16 October, at `694926002`, are named by function rather than by line. ## 2. Design @@ -78,9 +78,9 @@ A build is given its facts and its keys as build inputs. This change adds the fi | `TRUSTED_SERVER_COMMIT`, `TRUSTED_SERVER_BUILD_RUN` and `TRUSTED_SERVER_BUILD_TAG` | `build.commit`, `build.run` and `build.tag` | `unknown`, which is also what a value that is empty or holds a control character gives | | `SOURCE_DATE_EPOCH` | `build.builtAt` | The time of the build, which is also what a value that is not a whole number of seconds gives | -The build writes the schedule into `attestation_keys.rs` in its output directory, and the attestation module includes that file, so each private key is text inside the binary. The build stops, with the number of the line and the reason, when a schedule line does not hold a key id, a start and a key separated by tabs, when the start is not a number, when a key id or key holds a character other than letters, digits and `-_.:+/=`, or when a line repeats the key id or the start of an earlier line. The message never repeats the key. A schedule file that cannot be read also stops the build. The build checks the shape and the characters of a key and does not check that the key is a P-256 private key (`key_schedule.rs`, lines 24 to 62). Open question 3 asks whether the build should check each key. +The build writes the schedule into `attestation_keys.rs` in its output directory, and the attestation module includes that file, so each private key is text inside the binary. The build stops, with the number of the line and the reason, when a schedule line does not hold a key id, a start and a key separated by tabs, when the start is not a number, when a key id or key holds a character other than letters, digits and `-_.:+/=`, when a line repeats the key id or the start of an earlier line, or when a key cannot sign, read with the function that signs (`read` in `signing_key.rs`). The message never repeats the key. A schedule file that cannot be read also stops the build. The shape and the characters are checked in `key_schedule.rs` (lines 24 to 62 at `59e09b6cd`), which then reads each key, however late it starts. -How a verifier gets the public keys, and the period each is in force, is not part of this change. No tool in this repository exports a public key. The harness (`scripts/attestation-local-test.sh`, described in section 5) makes its keys with OpenSSL and derives the public keys from them (lines 92 to 101), and the unit tests derive them from test keys. Open question 6 asks how a verifier gets the keys. +How a verifier gets the public keys, and the period each is in force, is not part of this change. No tool in this repository exports a public key. The harness (`scripts/attestation-local-test.sh`, described in section 5) makes its keys with OpenSSL and derives the public keys from them (lines 92 to 101), and the unit tests derive them from test keys. Open question 3 asks how a verifier gets the keys. **How it meets the scope of #161.** The first eight rows are the items of the proposed scope in #161, as labels. The last three are its "Done when", quoted. @@ -93,9 +93,9 @@ How a verifier gets the public keys, and the period each is in force, is not par | Freshness and replay | `issuedAt` and the caller's nonce are signed, so a verifier refuses an old time or a nonce that is not its own. Every answer is sent `Cache-Control: no-store` | | Which configuration changes affect the claims | `publisher.domain` and `[attestation] operator` are claims. `[attestation] context` is signed ahead of the evidence, so a verifier has to use exactly the same text. Nothing else of the settings is claimed, and `/_ts/config`, which the inspection endpoints of row 9 ([IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350)) added, is where a deployment shows the settings it runs | | Claim creation and signing in the deployment lifecycle | Keys are a build input. The key in force is the latest to have started, so a build carries the next key before it is needed. While the key in force is the last one the build holds, the deployment logs one warning for each process and goes on signing with that key. The proof reaches a verifier in the envelope, which a relying party fetches from the publisher's own address | -| Safe failure and rotation, exposing no secret | No key in force, or a key that cannot be read, answers `503` at the endpoint alone. A settings document cannot name a key, and the section refuses any field it does not read. The private keys are readable by whoever holds the binary or the build's output directory, as "What it costs" below says | +| Safe failure and rotation, exposing no secret | No key in force, or a clock that cannot be read, answers `503` at the endpoint alone, and a key that cannot sign stops the build. A settings document cannot name a key, and the section refuses any field it does not read. The private keys are readable by whoever holds the binary or the build's output directory, as "What it costs" below says | | "A reviewed design identifies the exact claim schema, signer/verifier ownership, configuration inputs, rotation, and failure semantics." | This spec. The claim schema is the evidence table, the signer is the operator's build and the verifier is outside this repository, the configuration inputs are the section and the build inputs, rotation is the key schedule, and failure semantics are the answers in section 3 | -| "Tests prove that an authorized deployment verifies and that tampered, stale, or mismatched claims fail." | The unit tests and the harness verify a signature with the public key alone, and show that a changed host in the payload, the other key of the schedule and another context each fail. A mismatch between the evidence and the domain a relying party asked about is for the verifier to refuse, and no verifier is in this repository. No test makes stale evidence fail. Open question 5 asks which tests to add | +| "Tests prove that an authorized deployment verifies and that tampered, stale, or mismatched claims fail." | The unit tests and the harness verify a signature with the public key alone, and show that a changed host in the payload, the other key of the schedule and another context each fail. A mismatch between the evidence and the domain a relying party asked about is for the verifier to refuse, and no verifier is in this repository. No test makes stale evidence fail. Open question 2 asks which tests to add | | "The agreed transport is exercised end to end and documented for operators and verifiers, or the investigation records why attestation cannot be supported as proposed." | `scripts/attestation-local-test.sh` exercises the transport on Fastly under Viceroy (Fastly's local runtime) and runs in CI, and `docs/guide/attestation.md` documents it for operators and verifiers. No test exercises signed evidence through Axum, Cloudflare or Spin, because test builds carry no key | **Where this meets #1235.** The issue for attested release binaries ([IABTechLab/trusted-server#1235](https://github.com/IABTechLab/trusted-server/issues/1235)) gives its reason as "An attestation only means something if every publisher runs the same bytes", and its Constraints say "No configuration may be compiled into the release binary. Anything a deployment varies goes through `trusted-server.toml` and `ts config push`, or through platform stores. Other features have to fit this rule, not the other way around." This change compiles the operator's signing keys into the build, so the keys do not fit that rule. The two meet here and the choice is one for the task force. The same issue asks for every compile-time input of the release binary to be listed, and this change adds five, being `TRUSTED_SERVER_ATTESTATION_KEYS`, `TRUSTED_SERVER_COMMIT`, `TRUSTED_SERVER_BUILD_RUN`, `TRUSTED_SERVER_BUILD_TAG` and `SOURCE_DATE_EPOCH`. @@ -114,30 +114,31 @@ The same issue asks "Which build facts the runtime can report for #161, such as **Adding the section.** A binary built from row 11, the row below this one, reads an `[attestation]` table as the section of a module type such as `[cmp]` or `[tag]`, because the settings read every top-level table they do not know as one (`TypeSections` in `settings.rs`). That binary then finds that `context`, `operator` and `verify_url` are not settings a module type's section reads (`SectionModules::from_entries` in `provider_table.rs`) and refuses the whole document. A service can therefore take a document that carries the section only once its binary carries this change. A service rolled back past this change needs a document without the section first, as the settings tests say of any configured section added since the base revision (`settings.rs`, lines 5244 to 5247). -**The evidence format.** The format version is `1`. `tag` joined the `build` fields on 13 October, after the first draft of this spec, and the version stayed `1` (commit `a656298ec`). A verifier written against the four build fields of that draft still verifies the signature of evidence that carries `tag`, because the bytes are checked as they arrived, and whether such a verifier accepts a field it does not know is the verifier's own rule. The code holds no rule for when a field added to the evidence needs a new version. Open question 7 asks what that rule is. +**The evidence format.** The format version is `1`. `tag` joined the `build` fields on 13 October, after the first draft of this spec, and the version stayed `1` (commit `a656298ec`). A verifier written against the four build fields of that draft still verifies the signature of evidence that carries `tag`, because the bytes are checked as they arrived, and whether such a verifier accepts a field it does not know is the verifier's own rule. The code holds no rule for when a field added to the evidence needs a new version. Open question 4 asks what that rule is. **What each adapter reports.** Fastly reads the service's environment, as the evidence table says. Axum, Cloudflare and Spin report their own name, `unknown` as the service id, service version and location, and `false` as `staging` (`PlatformIdentity::named`, lines 147 to 159 of `attestation.rs`). **How the adapters answer.** All four adapters register the attestation routes the same way, after the fixed routes and before the catch-all routes that send the rest to the publisher's origin. `attestation_prefix_parity` checks that Axum, Cloudflare and Spin answer the same cases alike, and the Fastly adapter has its own tests of the same cases. On every adapter the handler builds no Edge Cookie context, so it reads, creates and refreshes no Edge Cookie. On Fastly the answer also comes ahead of the request filters, so a verifier fetching the answer from a server is not turned away as a bot (`attestation_route_handler`, lines 1581 to 1597 of the Fastly adapter's `app.rs`). The harness runs the Fastly adapter only. -**Request preparers.** A request preparer is code that a registered integration runs on a request before routing, for example to remove a query parameter or a cookie that the integration reserves. Axum, Spin and Fastly answer the attestation routes without running the preparers, and the comment on the Axum handler says this is because the pages are read only. Cloudflare registers the handler through `make_handler`, which runs the preparers first and returns the error of a preparer that refuses the request as the answer (`adapter-cloudflare/src/app.rs`, lines 426 to 445 and 988 to 994). No parity test sends a request through a preparer, so none of them sees the difference. Open question 2 asks whether the preparers should run on all four adapters or on none. +**Request preparers.** A request preparer is code that a registered integration runs on a request before routing, for example to remove a query parameter or a cookie that the integration reserves. All four adapters answer the attestation routes without running the preparers, so no preparer can change or refuse the request whose host and nonce are signed. Cloudflare does so through its own `attestation_route_handler` rather than `make_handler`, which runs the preparers first. The Cloudflare route test `attestation_routes_skip_the_request_preparers` registers a preparer that refuses every request, shows that it refuses a request for the publisher's origin, and checks that the list at the prefix, the JSON address and the redirect from the default address answer as they do without it. No test sends a request through a preparer on the attestation routes of Axum, Fastly or Spin. **Router middleware.** Each adapter's router wraps every route, the attestation routes included, in its own middleware. That middleware runs basic authentication, so a `[[handlers]]` entry whose `path` pattern matches the endpoint makes the endpoint ask for credentials, and its last step adds the standard response headers to the answer. On Fastly the standard headers include geo headers for the caller's address when a lookup answers, and the operator's `[response_headers]`. **Answers.** The 400, 500 and 503 answers are JSON at both addresses. Every answer, the redirect and the errors included, carries `Cache-Control: no-store` and `Access-Control-Allow-Origin: *`, so a relying party's page can read the JSON from a browser and no cache hands an old answer out again. -| Status | When | Body | -| ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | -| 200 | Evidence was signed | The page, or the envelope at the `.json` address | -| 301 | A read of the default address, or its `.json` form, where the endpoint is not the default | `{"location":""}`, with a `Location` header | -| 400 | The `nonce` is empty, longer than 64 characters or holds a character other than `A` to `Z`, `a` to `z`, `0` to `9`, `_` and `-` | `{"error":"A nonce is 1 to 64 characters from A to Z, a to z, 0 to 9, _ and -."}` | -| 404 | Any other address beneath a prefix the deployment owns, and any method other than GET and HEAD on the pages and on the prefix there | A page pointing to the list at the prefix | -| 404 | The settings carry no `[attestation]` section and the handler is called anyway. No adapter registers a route then, so a request never reaches the handler | `{"error":"This deployment serves no attestation."}` | -| 500 | The evidence could not be written | `{"error":"The evidence could not be written."}` | -| 503 | No key in the build has come into force, or the build carries no key at all | `{"error":"No signing key is built in for this time, so no evidence can be signed."}` | -| 503 | The key in force is not base64 or is not a P-256 private key in SEC1 or PKCS #8 form | `{"error":"The signing key could not be read, so no evidence can be signed."}` | +| Status | When | Body | +| ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | +| 200 | Evidence was signed | The page, or the envelope at the `.json` address | +| 301 | A read of the default address, or its `.json` form, where the endpoint is not the default | `{"location":""}`, with a `Location` header | +| 400 | The `nonce` is empty, longer than 64 characters or holds a character other than `A` to `Z`, `a` to `z`, `0` to `9`, `_` and `-` | `{"error":"A nonce is 1 to 64 characters from A to Z, a to z, 0 to 9, _ and -."}` | +| 404 | Any other address beneath a prefix the deployment owns, and any method other than GET and HEAD on the pages and on the prefix there | A page pointing to the list at the prefix | +| 404 | The settings carry no `[attestation]` section and the handler is called anyway. No adapter registers a route then, so a request never reaches the handler | `{"error":"This deployment serves no attestation."}` | +| 500 | The evidence could not be written | `{"error":"The evidence could not be written."}` | +| 503 | No key in the build has come into force, or the build carries no key at all | `{"error":"No signing key is built in for this time, so no evidence can be signed."}` | +| 503 | The clock cannot be read as a time after the Unix epoch | `{"error":"The clock could not be read, so no evidence can be signed."}` | +| 503 | The key in force is not base64 or is not a P-256 private key in SEC1 or PKCS #8 form. The build refuses such a key, and the handler keeps this answer for one that still cannot sign when it runs | `{"error":"The signing key could not be read, so no evidence can be signed."}` | -**Failure.** A failure stays at the attestation addresses, so no other address of the deployment changes. No other key signs in place of a key that cannot be read, because the key in force is the latest to have started. The build stops for the schedule faults listed under the build inputs. A key that is not a P-256 private key does not stop the build, and the endpoint then answers `503` from the second that key comes into force. The handler reads the time from `current_timestamp`, which gives `0` when the clock cannot be read as a time after the Unix epoch (`ec/mod.rs`, lines 1324 to 1326, and `attestation.rs`, line 337). With such a clock the endpoint answers `503` unless the schedule holds a key that starts at second `0` or earlier. The evidence is then signed with that key and says it was issued at `1970-01-01T00:00:00Z`. The code also has `checked_current_timestamp`, and the comment on that function says to use it instead of `current_timestamp` when a fallback could authorize a time-bounded decision. The handler does not use `checked_current_timestamp`. Open question 4 asks whether the handler should use it and answer `503` when the clock fails. +**Failure.** A failure stays at the attestation addresses, so no other address of the deployment changes. No other key signs in place of a key that cannot be read, because the key in force is the latest to have started. The build stops for the schedule faults listed under the build inputs, a key that cannot sign among them however late it starts, so no build carries a key that would answer `503` from its start time. The build reads each key with the function the handler signs with (`read` in `signing_key.rs`, which the build script and `attestation.rs` both include), and the handler keeps its `503` for a key that still cannot sign when it runs. The handler reads the time with `checked_current_timestamp` and answers `503` and signs nothing when the clock cannot be read as a time after the Unix epoch. **Consent and privacy.** The handler reads the request's `Host` header, the request's `nonce` parameter and the headers that name the scheme. The handler takes the scheme from the first of `Forwarded`, `X-Forwarded-Proto` and `Fastly-SSL` that gives one and uses `http` when none does, because the handler reads no TLS details of the connection. Only the JSON address that the page gives to copy uses the scheme, so the evidence never holds the scheme. Fastly and Spin remove those three headers from the request and then set the scheme themselves, Fastly from the TLS connection and Spin from the full URL that the Spin runtime reports. Axum and Cloudflare pass the headers on as they arrived, so on those two adapters the caller, or a proxy in front of the deployment, decides the scheme in the JSON address on the page. The handler reads no cookie, no consent signal and no store, makes no outbound request and sets no cookie. The evidence holds the host and the nonce that the caller sent and facts about the deployment, and nothing else about the caller. The envelope tells whoever asks the operator's name, the publisher's domain, the service id and version, the location, the commit, the build run, the tag and the build time. Every answer allows any origin to read it. @@ -146,34 +147,31 @@ The same issue asks "Which build facts the runtime can report for #161, such as - **A verifier.** None is in this repository. The guide says what one checks, and the harness is a small one. - **The key custody**, as above. - **The platform fields off Fastly.** The Axum, Cloudflare and Spin adapters report their own name, `unknown` for the service id, service version and location, and `false` for `staging`. Open question 1 asks whether `false` is right. -- **A way for a verifier to get the public keys.** No tool in this repository exports a public key, and no document here says where a verifier finds the keys and the period of each. Open question 6 asks where a verifier finds them. +- **A way for a verifier to get the public keys.** No tool in this repository exports a public key, and no document here says where a verifier finds the keys and the period of each. Open question 3 asks where a verifier finds them. - **The compiled schedule** is checked by the harness alone, because test builds carry no key. The reading of a schedule file is unit tested. **Open questions.** Each of these is a question about the code. This spec records what the code does and leaves the questions open. Row 50 ([IABTechLab/trusted-server#1400](https://github.com/IABTechLab/trusted-server/pull/1400), at `3aaf720c0`) has the same code for each question. 1. Should the Axum, Cloudflare and Spin adapters say that `staging` is unknown, in place of signing `false`? The guide tells a relying party to check `platform.staging` when only answers from production are acceptable (`docs/guide/attestation.md`, lines 472 and 473), and that check passes on those three adapters whatever the environment. -2. Should the request preparers run on the attestation routes of every adapter, or of none? Cloudflare runs them and the other three do not, and no parity test sends a request through a preparer. -3. Should the build check that each key in the schedule is a P-256 private key? A bad key with a later start passes the build and answers `503` when its start time arrives. -4. Should the handler use `checked_current_timestamp` and answer `503` when the clock fails, where `current_timestamp` now gives `0`? -5. Which tests should be added? #161 asks for tests that stale claims fail, and no test makes stale evidence fail, because the harness checks only that fresh evidence is within five minutes of the harness's clock (`scripts/attestation-local-test.sh`, lines 282 to 284). `fastly_platform_identity` has no test, and the harness checks `platform.name` only. The order test leaves out `tag` (`attestation.rs`, lines 1896 to 1914). -6. How does a verifier get the public keys and the period each is in force? No tool in this repository exports them. -7. What rule says when a field may be added to the evidence under `v: 1` and when the version has to change? `tag` was added to `build` after the first draft of this spec with the version left at `1`. -8. Should the guide list `build.run` before `build.tag`? The evidence writes `run` first (`attestation.rs`, lines 167 and 168), and the guide lists `build.tag` first (`docs/guide/attestation.md`, lines 308 and 309). +2. Which tests should be added? #161 asks for tests that stale claims fail, and no test makes stale evidence fail, because the harness checks only that fresh evidence is within five minutes of the harness's clock (`scripts/attestation-local-test.sh`, lines 282 to 284). `fastly_platform_identity` has no test, and the harness checks `platform.name` only. The order test leaves out `tag` (`attestation.rs`, lines 1896 to 1914). +3. How does a verifier get the public keys and the period each is in force? No tool in this repository exports them. +4. What rule says when a field may be added to the evidence under `v: 1` and when the version has to change? `tag` was added to `build` after the first draft of this spec with the version left at `1`. +5. Should the guide list `build.run` before `build.tag`? The evidence writes `run` first (`attestation.rs`, lines 167 and 168), and the guide lists `build.tag` first (`docs/guide/attestation.md`, lines 308 and 309). ## 5. Tests -At `59e09b6cd` this change adds 54 tests and one harness. +At `59e09b6cd` this change adds 54 tests and one harness, and the fixes at `694926002` add three more tests, making 57. -- **`crates/trusted-server-core/src/attestation.rs` has 40 tests.** They cover the reading of a key schedule, the signature and the key in force, the evidence and the envelope, the nonce, the page, the host, the settings and the endpoint rules, and the routes and methods. The count was 37 before commit `96d8caf89`, which added the tests of a repeated key id or start, of the framework asset prefixes and of the single warning for the last key. +- **`crates/trusted-server-core/src/attestation.rs` has 40 tests.** They cover the reading of a key schedule, the signature and the key in force, the evidence and the envelope, the nonce, the page, the host, the settings and the endpoint rules, and the routes and methods. The count was 37 before commit `96d8caf89`, which added the tests of a repeated key id or start, of the framework asset prefixes and of the single warning for the last key. The fixes add `a_clock_that_cannot_be_read_signs_nothing` and `a_schedule_key_that_cannot_sign_is_refused_without_showing_it`, making 42. - **`crates/trusted-server-core/src/build_info.rs` has 3 tests.** The build time is RFC 3339 in UTC, the commit, the run and the tag are never empty, and the build script writes known instants correctly. - **The Fastly adapter has 5 route tests.** A prefix of the deployment's own is answered there and never by the origin, the default address redirects to the endpoint the settings name, no route is registered without the section, every fixed route beneath an underscore prefix is reserved, and every endpoint the settings accept builds a router. -- **The Axum, Cloudflare and Spin adapters have one test each.** The test `every_fixed_route_beneath_an_underscore_prefix_is_reserved` fails when a fixed route beneath an underscore prefix is missing from `RESERVED_PATHS`. +- **The Axum, Cloudflare and Spin adapters have one test each.** The test `every_fixed_route_beneath_an_underscore_prefix_is_reserved` fails when a fixed route beneath an underscore prefix is missing from `RESERVED_PATHS`. The fixes add `attestation_routes_skip_the_request_preparers` to the Cloudflare route tests, which registers a preparer that refuses every request and checks that the attestation routes answer without it. - **`crates/trusted-server-integration-tests/tests/parity.rs` has 3 tests** that run the three adapters that can run on a host against each other. They are `attestation_prefix_parity`, `attestation_default_address_redirect_parity` and `attestation_is_not_served_without_its_section_parity`. - **`scripts/attestation-local-test.sh` is the last step of the Fastly job in CI** (`.github/workflows/test.yml`, lines 80 to 82). It builds the Fastly adapter with a schedule of two keys, runs the adapter under Viceroy against a stub origin and checks the evidence with OpenSSL and the public key alone. It covers Fastly only. Test builds carry no key, so no router test sees signed evidence, and only the harness does, on Fastly. -**What no test covers.** A stale claim failing, the Fastly environment variables read into `platform`, a request preparer on an attestation route, the position of `tag` in the order test and a failing clock. The build also makes no check that a key is a P-256 private key, so there is no check for a test to cover. Open questions 2 to 5 ask about these. +**What no test covers.** A stale claim failing, the Fastly environment variables read into `platform`, a request preparer on the attestation routes of Axum, Fastly and Spin, and the position of `tag` in the order test. Open question 2 asks about these. The test of the clock hands `answer` a missing time, because a test cannot make the platform's clock fail, so `handle_attestation`, which passes the real clock, has no test of its own. The test of a key that cannot sign calls `constant_source`, which the build script calls, and no test runs a build given such a key. ## 6. Acceptance @@ -184,12 +182,13 @@ Test builds carry no key, so no router test sees signed evidence, and only the h 3. That signature does not verify with the other key of the schedule, under another context or once the host in the payload changes (the harness, `an_altered_payload_fails_verification`). 4. The evidence names the host the request arrived for, lower case and without a port, and a forwarded host header never replaces it. The publisher comes from the settings (`the_evidence_shown_is_the_payload_signed` and `the_host_falls_back_to_the_request_address` for the case and the port, `forwarded_host_headers_never_become_the_signed_host`, `the_signed_publisher_comes_from_the_settings`, the harness). 5. A nonce is signed back as sent, a malformed nonce answers `400`, and with no nonce the evidence carries none (`a_nonce_is_signed_back_as_sent`, `a_malformed_nonce_is_refused`, `no_nonce_is_signed_unless_asked_for`). -6. A build given no schedule, an empty schedule or blank lines answers `503` at the endpoint, as does a time before the first key and a key that cannot be read (`a_build_given_no_schedule_compiles_an_empty_one`, `no_usable_key_in_force_answers_service_unavailable`). -7. A schedule line of the wrong shape, or one that repeats a key id or a start, stops the build without showing the key (`a_schedule_line_of_any_other_shape_is_refused_without_showing_the_key`, `a_schedule_repeating_a_key_id_or_a_start_is_refused`). +6. A build given no schedule, an empty schedule or blank lines answers `503` at the endpoint, as do a time before the first key, a key that cannot be read and a clock that cannot be read, and nothing is signed then (`a_build_given_no_schedule_compiles_an_empty_one`, `no_usable_key_in_force_answers_service_unavailable`, `a_clock_that_cannot_be_read_signs_nothing`). +7. A schedule line of the wrong shape, one that repeats a key id or a start, or one whose key cannot sign, however late it starts, stops the build without showing the key (`a_schedule_line_of_any_other_shape_is_refused_without_showing_the_key`, `a_schedule_repeating_a_key_id_or_a_start_is_refused`, `a_schedule_key_that_cannot_sign_is_refused_without_showing_it`). 8. The key in force is the latest to have started, and the warning for the last key is given once for each process (`the_key_in_force_is_the_latest_to_have_started`, `the_last_key_warning_is_given_once`). 9. Every answer carries `Cache-Control: no-store`, and the page and the envelope carry `Access-Control-Allow-Origin: *` (`answers_are_never_cached_and_readable_from_any_origin` for both headers on the page and the envelope, the harness for both on the envelope, and `attestation_prefix_parity`, `attestation_default_address_redirect_parity` and the Fastly adapter's tests for `Cache-Control` on the list at a prefix, the redirect, the `404` pages and the `503` answer of a build with no key). No test checks the headers of the `400` and `500` answers, of the `503` answer for a key that cannot be read, or of the `404` answer in JSON that the handler gives when the settings carry no section. 10. An endpoint of the wrong shape, at a reserved address or under a framework asset prefix is refused when the settings load, with the reason (`the_endpoint_is_an_absolute_path_under_an_underscore_prefix`, `an_endpoint_at_a_reserved_address_is_refused_naming_it`, `an_endpoint_under_a_framework_asset_prefix_is_refused_naming_it`), and every fixed route beneath an underscore prefix is in the reserved list on all four adapters. 11. Under a prefix of the deployment's own, every address and method the pages do not serve answers `404` from the deployment on all four adapters, and a read of the default address redirects to the endpoint with the query kept (`attestation_prefix_parity`, `attestation_default_address_redirect_parity` and the Fastly adapter's tests). +12. Cloudflare answers the attestation routes without the request preparers, so a preparer that refuses every request leaves the list at the prefix, the JSON address and the redirect from the default address as they are (`attestation_routes_skip_the_request_preparers`). **Checks that can be repeated.** In a clone of IABTechLab/trusted-server at `59e09b6cd`, with the Rust toolchain and the Viceroy version that `.tool-versions` names, these commands pass. @@ -202,7 +201,7 @@ Test builds carry no key, so no router test sees signed evidence, and only the h **Local checks.** The test plan in the description of IABTechLab/trusted-server#1353 ticks `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter, and leaves a manual run with `fastly compute serve` unticked. -**What no test covers.** The paths listed in section 5, which Open questions 2 to 5 ask about, and the answers whose headers item 9 says no test checks. +**What no test covers.** The paths listed in section 5, which Open question 2 asks about, and the answers whose headers item 9 says no test checks. ## Review log @@ -216,3 +215,5 @@ Test builds carry no key, so no router test sees signed evidence, and only the h | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The header and the pull requests it names, the `build` fields and the `tag`, the configuration fields and the endpoint rules, the methods, the adapters and what each reports for `staging`, the answers and the failures, the compatibility of the section, the privacy of the answers, the scope table and the quotations from the two issues, the key custody and what it costs, the test count and the tests, the acceptance, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. The `host` fallback when the `Host` header is not readable text, the scheme headers and what they change under "Consent and privacy", the tests named in Acceptance 4 and 9 and what no test checks there, and the review log. | +| 2026-10-16 | Three fixes in the code, at `694926002`. The handler answers `503` and signs nothing when the clock cannot be read, the build reads every key in the schedule with the function that signs and stops at one that cannot sign, and Cloudflare answers the attestation routes without the request preparers, each with a test. Open questions 2, 3 and 4 of the previous revision leave the list, which is renumbered. | +| 2026-10-17 | The guide says how the key schedule reaches a Cloudflare build and a Spin build, and that the file is named by its absolute path (`e5e7fbf88`). | From e4dc86776ef854b63f8602ecf2a7eeb5697e2366 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:15:49 +0100 Subject: [PATCH 133/192] The loading cache design says what Cloudflare and Spin keep The Cloudflare and Spin adapters now declare their key-value stores, so the design says what each host keeps between requests and that a memory copy lasts one request on Cloudflare and Spin. --- .../specs/2026-10-12-loading-cache-design.md | 26 ++++++++++++------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index 0051c4a121..593c175eee 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-robots-txt-from-modules-design.md`, `2026-10-12-robots-txt-middleware-design.md`, `2026-10-12-prefix-module-design.md`, `2026-10-15-module-cache-declarations-design.md`, `2026-10-15-asset-cache-design.md` **Related PRs:** [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24) was reviewed on [jwrosewell/trusted-server#19](https://github.com/jwrosewell/trusted-server/pull/19) (closed) and [IABTechLab/trusted-server#1372](https://github.com/IABTechLab/trusted-server/pull/1372) (row 31) on [jwrosewell/trusted-server#26](https://github.com/jwrosewell/trusted-server/pull/26) (closed). The integration seam is in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348), robots.txt rules from modules in [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355), robots.txt as a middleware in [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366), the prefix module in [IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), module-declared caches in [IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373) and the asset cache in [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375) **Issues:** [IABTechLab/trusted-server#1282](https://github.com/IABTechLab/trusted-server/issues/1282). [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-17 > **Why this spec exists.** Each robots.txt module's answer is read from the key-value store by every request and asked of the module's service by every request that finds no fresh copy, so a burst of requests for one file asks the service once for each request, on every instance. Core has one cache, for page templates, and no interface through which a module can supply a cache. @@ -41,7 +41,7 @@ store_idle_seconds = 2592000 **The two layers.** A call goes through two layers. The memory layer is one least recently used store for each cache, shared by every request an adapter serves with one application state, holding up to `memory_entries` values (64 for robots.txt) for `memory_seconds` each. Below the memory layer, a shared store layer is made for each call over the host's shared store, which is the host's value cache where the host gives one, being Fastly's cache, and the host's key-value store otherwise. A read looks in memory, then in the shared store, and only then loads. A copy in the shared store lasts at most `store_seconds`, leaves after `store_idle_seconds` unused, and ends sooner when the loaded value carries a lifetime of its own, which robots.txt sets to the module's refresh. The shared store holds each entry in the `fiftyone-caching` entry format, which carries the time the entry must be dropped, so a store that deletes late, or keeps no lifetimes, never gives back an entry past that time. A write to a key-value store carries a lifetime of at least 60 seconds because Workers KV refuses less, and the entry's own drop time holds the exact lifetime. -**Who waits for a load.** Callers that share one memory copy and ask for the same key at the same moment share one load. The first caller runs its own loader inside its own call, because no thread pool or background task exists (`LoadingCaches::new` in `lib.rs` and `Layer` in `layer.rs` of the module crate, and the test `a_load_runs_on_the_thread_of_the_caller_that_leads_it`), and the other callers wait for its value or its error. If the first caller is dropped before the load finishes, a waiting caller takes over and loads. Across processes, other instances wait only where the shared store reserves a missing key for the first instance that asks and makes the rest wait until that instance fills the key. Only Fastly's cache does this, once for each Fastly location. Workers KV, Spin's key-value store and an unavailable store reserve nothing, so instances that miss at the same moment each load, and instances that ask afterwards read the stored copy. A module's service is therefore asked once per refresh by the requests that share one memory copy, and by all the instances of one Fastly location together, but not by all the instances of a deployment together. +**Who waits for a load.** Callers that share one memory copy and ask for the same key at the same moment share one load. The first caller runs its own loader inside its own call, because no thread pool or background task exists (`LoadingCaches::new` in `lib.rs` and `Layer` in `layer.rs` of the module crate, and the test `a_load_runs_on_the_thread_of_the_caller_that_leads_it`), and the other callers wait for its value or its error. If the first caller is dropped before the load finishes, a waiting caller takes over and loads. Across processes, other instances wait only where the shared store reserves a missing key for the first instance that asks and makes the rest wait until that instance fills the key. Only Fastly's cache does this, once for each Fastly location. Workers KV and Spin's key-value store, which the Cloudflare and Spin adapters declare, reserve nothing, so instances that miss at the same moment each load, and instances that ask afterwards read the stored copy. Axum's unavailable store keeps nothing, so there the memory copy alone answers. A module's service is therefore asked once per refresh by the requests that share one memory copy, and by all the instances of one Fastly location together, but not by all the instances of a deployment together. **Why.** The same answer is wanted by every instance and every request, and a cache that lets every miss load at once asks a module's service as many times as there are requests. Loading each key once at a time in a process, with the value shared by every waiter, is the pattern of `LruLoadingCache` in the 51Degrees [.NET](https://github.com/51Degrees/caching-dotnet) and [Java](https://github.com/51Degrees/pipeline-java) pipelines, which the published crate's documentation points to. Core keeps only the interface. Core's cache code depends on no caching package. The only place core writes the module name `51degrees` is `settings.rs`, in the documentation of `LoadingCacheConfig::module` (line 3055) and in the test `a_loading_cache_module_is_read_and_written_back` (lines 7036 to 7055), and the open questions ask about it. @@ -61,12 +61,12 @@ Everything else in this document is the same on both rows, apart from line numbe Each adapter calls `loading_caches_from` when it builds its application state (Fastly `app.rs` line 347, Cloudflare line 315, Spin line 244, Axum line 214) and hands the result to every request's services. What the cache keeps, and who waits, depends on the stores the adapter gives each request. -| Adapter | Shared store | Do other instances wait for one load? | Memory copy | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Fastly | Fastly's cache alone, and the key-value store is not used (`stores.rs` lines 61 to 73 of the module crate, `app.rs` line 443 of the Fastly adapter) | Yes, once for each Fastly location, because a missing key is reserved for the first instance that looks it up | One for each application state. Fastly serves one request for each instance unless the opt-in reusable sandbox keeps the state, so by default the copy lasts one request | -| Cloudflare | Workers KV where the deployment binds a namespace, and otherwise a store whose every call fails | No | One for each application state, which Cloudflare builds again for every request, so the copy lasts one request and Workers KV holds the copy between requests. `factory()` makes a cache for each state on every host | -| Spin | Spin's key-value store where the component has access to one, and otherwise a store whose every call fails. Spin gives an entry no lifetime, so the entry is written without one and its own drop time ends its use | No | One for each application state, which Spin builds again for every request, so the copy lasts one request (see the note below the table) | -| Axum | None. The key-value store is `UnavailableKvStore`, which fails every call (`axum/platform.rs` lines 572 to 574, `platform/kv.rs` lines 18 to 24) | Not applicable | One for the process, because Axum, the native development server, builds its application state once. That copy is the only one, and a restart loads again | +| Adapter | Shared store | Do other instances wait for one load? | Memory copy | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Fastly | Fastly's cache alone, and the key-value store is not used (`stores.rs` lines 61 to 73 of the module crate, `app.rs` line 443 of the Fastly adapter) | Yes, once for each Fastly location, because a missing key is reserved for the first instance that looks it up | One for each application state. Fastly serves one request for each instance unless the opt-in reusable sandbox keeps the state, so by default the copy lasts one request | +| Cloudflare | Workers KV, the `TRUSTED_SERVER_KV` binding the adapter declares in `Hooks::stores` (`app.rs` of the Cloudflare adapter). EdgeZero treats a declared store as required, so a Worker without the binding fails every request before any route runs | No | One for each application state, which Cloudflare builds again for every request, so the copy lasts one request and Workers KV holds the copy between requests. `factory()` makes a cache for each state on every host | +| Spin | Spin's `default` key-value store, which the adapter declares in `Hooks::stores` and `spin.toml` grants. A component that cannot open it fails every request before any route runs. Spin gives an entry no lifetime, so the entry is written without one and its own drop time ends its use | No | One for each application state, which Spin builds again for every request, so the copy lasts one request and the `default` store holds the copy between requests (see the note below the table) | +| Axum | None. The key-value store is `UnavailableKvStore`, which fails every call (`axum/platform.rs` lines 572 to 574, `platform/kv.rs` lines 18 to 24) | Not applicable | One for the process, because Axum, the native development server, builds its application state once. That copy is the only one, and a restart loads again | Spin builds its application state for every request because the Spin adapter calls `run_app` in [EdgeZero](https://github.com/stackpop/edgezero) (`edgezero-adapter-spin/src/lib.rs`, at the revision the workspace pins), which builds the application on every call. @@ -101,13 +101,15 @@ The ranges are checked when the settings load and again by the deploy check, whi - **A first caller that is dropped.** A waiting caller takes over the load with its own loader (the test `a_waiting_caller_loads_when_the_caller_leading_the_load_is_dropped`). - **A shared store that fails.** A failed read is a miss and a failed write is logged while the value is still served, as section 3 says, so the memory copy alone answers (the test `a_store_that_fails_every_call_leaves_the_memory_copy_answering`, with Axum's `UnavailableKvStore`). - **A held copy that is unreadable.** A copy that no longer parses as a robots.txt `Contribution` is removed from the cache and the module is asked again (`holding.rs` lines 95 to 109, and the test `an_unreadable_held_copy_is_removed_and_the_contributor_asked_again`). -- **A module that fails.** The last answer the module gave is served past its age. Every load that succeeds writes its answer apart from the cache for 30 days, in the host's value cache where it has one and in the key-value store otherwise, and nothing reads that answer unless the module fails (`holding.rs` lines 35 to 40 and 110 to 125). With none kept, the whole file answers `503` with `Retry-After: 600` and is not stored. A value cache may drop an entry early, so on Fastly a kept answer is likely and not certain to be there. Spin's key-value store gives an entry no lifetime, so a kept answer there stays until it is replaced. Because a failure is not stored, every request asks the module again while the module is down, and the requests in one process that arrive during one attempt share that attempt. +- **A module that fails.** The last answer the module gave is served past its age. Every load that succeeds writes its answer apart from the cache for 30 days, in the host's value cache where it has one and in the key-value store otherwise, and nothing reads that answer unless the module fails (`holding.rs` lines 35 to 40 and 110 to 125). With none kept, the whole file answers `503` with `Retry-After: 600` and is not stored. A value cache may drop an entry early, so on Fastly a kept answer is likely and not certain to be there. Spin's `default` store gives an entry no lifetime, so a kept answer there stays until it is replaced. Axum's key-value store fails every call, so Axum keeps no answer. Because a failure is not stored, every request asks the module again while the module is down, and the requests in one process that arrive during one attempt share that attempt. ## 7. What changes for a deployment **A stock build holds answers with no new setting.** The default in section 2 means a stock build holds each robots.txt module's answer as soon as the build includes this change. The shortest holding the settings allow is a memory copy of 0 seconds and a shared copy of 1 second. -**Fastly and Axum hold answers where they held none.** Row 23 gave Fastly's requests and Axum `UnavailableKvStore`, so neither held anything and every request asked the module. From row 24 Fastly holds each answer in its cache, shared by the instances at a location, and Axum holds each answer in memory for the life of the process. +**Every adapter holds answers where none held any.** Row 23 gave every request `UnavailableKvStore`, Fastly's and Axum's directly and Cloudflare's and Spin's because neither adapter declared a store for EdgeZero to open, so nothing was held and every request asked the module. From row 24 Fastly holds each answer in its cache, shared by the instances at a location, Cloudflare and Spin hold it in the key-value store each declares, and Axum holds each answer in memory for the life of the process. + +**Cloudflare and Spin need their store.** EdgeZero treats a declared store as required, so a Worker without the `TRUSTED_SERVER_KV` binding, or a Spin component that cannot open `default`, fails every request before any route runs, the Spin health check included. **The keys change.** Row 23 held a module's answer under `robots-txt:` and a hash of the publisher, the module's name and its fingerprint. Rows 24 and 31 hold the answer under `/robots-txt/`, where the hash covers the module's name and fingerprint and the publisher is in the prefix. A copy written before the change is never read again. The first request for each module after an upgrade asks the module, and a module that fails before it has answered once has no kept answer under the new key, so the file answers `503`. Old entries end by their own 30 day lifetime where the store gives lifetimes, and stay in a store that does not, such as Spin's, until someone deletes them. @@ -145,6 +147,8 @@ git grep -n -e detachable -e 'cloudflare::with_context' -e WaitUntil -e local_sp - `crates/trusted-server-core/src/settings.rs`. `loading_cache_lifetimes_are_read_with_defaults_for_what_is_unset`, `a_loading_cache_lifetime_out_of_range_is_refused_naming_it`, `unset_loading_cache_lifetimes_are_not_written_back` and `a_loading_cache_module_is_read_and_written_back`. - `crates/trusted-server-modules/src/lib.rs`. The stock list order, which names `loading-cache.51degrees`, and `a_feature_selects_the_stock_loading_cache_by_name`. Row 24 adds `a_table_naming_no_module_holds_with_the_stock_loading_cache` and `deploy_validation_knows_the_stock_loading_cache`. - `crates/trusted-server-adapter-fastly/src/value_cache.rs`, which `cargo test-fastly` runs under Viceroy, Fastly's local runtime. Seven tests, including `a_missing_key_is_reserved_and_its_fill_is_then_a_hit`, `a_dropped_reservation_passes_the_obligation_on` and `a_stale_entry_is_reserved_rather_than_served`. +- The Cloudflare and Spin adapters, on row 24. `the_declared_store_is_the_key_value_store_a_request_is_given` in each adapter's `tests/key_value_store.rs`, `every_wrangler_manifest_binds_the_declared_key_value_store` in the Cloudflare adapter's `tests/config_defaults.rs`, and `the_declared_key_value_store_is_a_label_spin_grants` in the Spin adapter's `tests/key_value_store.rs`. +- `crates/trusted-server-integration-tests/tests/integration.rs`, on row 24. `test_cloudflare_without_its_kv_binding_fails_every_request`, which CI runs under `wrangler dev` with the namespace binding removed and which expects `500` on every path. **CI.** Every check passed on the head of each row. That is 19 of 19 on the closed pull requests on the fork, jwrosewell/trusted-server#19 at `844b01fe2` and jwrosewell/trusted-server#26 at `4b6ef9c05`, and 25 of 25 on IABTechLab/trusted-server#1365 and IABTechLab/trusted-server#1372, as read on 2026-10-16. @@ -155,6 +159,7 @@ git grep -n -e detachable -e 'cloudflare::with_context' -e WaitUntil -e local_sp 1. **robots.txt through the real cache.** The holding tests use a double that does not join loads, and the module crate's tests use no robots.txt module, so the two never run together. 2. **Any adapter.** No adapter test loads through the cache on rows 24 and 31. The module crate's tests run natively and never on `wasm32-wasip1` or `wasm32-unknown-unknown`. `factory()` has no code that differs by target, so `each_application_state_is_given_a_cache_of_its_own` covers what the factory gives Cloudflare. 3. **Other instances waiting for one load.** The tests use one instance. The behavior comes from Fastly's cache, as the comments in `value_cache.rs` of the Fastly adapter describe. +4. **A robots.txt answer kept in Workers KV.** No test runs a robots.txt module under `wrangler dev`, because the Worker bundle CI builds links the stock modules alone and none of them contributes robots.txt rules. A test would need a contributor in that bundle whose answer differs from one load to the next. ## 10. Open questions @@ -176,3 +181,4 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the default when a table names no module, how loads run, who waits for a load, where rows 24 and 31 differ, the adapters, the configuration, consent and privacy, failure, what changes for a deployment, the tests and what none covers, the open questions and the review log. | | 2026-10-16 | Points from review round 2. How the two published crates differ, from a comparison of the crates themselves, and the date on which the check counts were read. | | 2026-10-16 | Code fixes on row 24. A load is handed services with nothing of the reader in them, each application state has a cache of its own on Cloudflare as elsewhere, `CacheSpec::detachable` and the text about loads that run apart from their caller are gone, the comments and guides say what a table naming no module takes, the deploy check refuses a `module` nothing offers, and tests cover the stock default, a zero memory lifetime, a dropped first caller and a failing store. Open questions 1, 2, 4, 5 and 7, and 3 apart from the repeated changelog entry, move into the text. | +| 2026-10-17 | Code fix on row 24. The Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests and a Worker or component without its store fails every request. Who waits for a load, the adapters, a module that fails, what changes for a deployment and the tests say so. | From a01ecbbebc996f7a68bb0f74ce299d0acf966bf8 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:15:51 +0100 Subject: [PATCH 134/192] The asset cache design says what Cloudflare and Spin keep between requests The design says how far one asset fetch reaches on each host now that the Cloudflare and Spin adapters declare their key-value stores. --- .../specs/2026-10-15-asset-cache-design.md | 23 ++++++++++--------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/docs/superpowers/specs/2026-10-15-asset-cache-design.md b/docs/superpowers/specs/2026-10-15-asset-cache-design.md index 13db203a00..d7b24940b8 100644 --- a/docs/superpowers/specs/2026-10-15-asset-cache-design.md +++ b/docs/superpowers/specs/2026-10-15-asset-cache-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-12-loading-cache-design.md`, `2026-09-15-852-template-and-origin-caching-design.md`, `2026-04-28-multi-backend-asset-proxy-design.md`, `2026-05-19-asset-s3-auth-fastly-io-design.md`, `2026-09-08-1138-per-cookie-template-cache-policy-design.md` **Related PRs:** IABTechLab/trusted-server#1375, reviewed on jwrosewell/trusted-server#29 (closed) **Issues:** IABTechLab/trusted-server#1324 -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-17 > **Why this spec exists.** A file that is not a page, whether the publisher's origin sends it or an asset route fetches it, is asked of its origin on every request and by every instance, so a burst of readers for one file is a burst of fetches. @@ -78,7 +78,7 @@ A stored copy is the origin's response, headers and body, as the origin sent it, **One fetch for many requests.** `get_or_fetch` loads the address's record through `assets-vary`. The request that leads that load fetches the origin and writes its own copy under its own values before the record is published, so a waiting request with the same values finds the copy. A waiting request with other values loads its own copy, and the requests waiting on those values share that load. Both caches are built with `detachable: false`, so a load runs in the request that started it. -**How far one fetch reaches.** The memory layer shares a load among the requests that share the layer. The stock module's layer lives as long as the application state, or on Cloudflare as long as the isolate. Fastly serves one request per instance unless the opt-in reusable sandbox is on, and Spin runs a fresh instance per request (see `AppState` in the Fastly and Spin adapters' `app.rs`), so on those two the memory layer is not shared between requests. Across instances only Fastly makes requests share a load. Only the Fastly adapter registers a value cache, and a lookup in that cache makes every other instance at the same location wait for the instance that fills the key (`crates/trusted-server-adapter-fastly/src/value_cache.rs`), so a missing key is loaded once for each location and not once for the service. Cloudflare and Spin share copies only through a key-value store, which makes no instance wait, so each instance that misses at the same moment fetches the origin itself, and Axum has no shared store at all. +**How far one fetch reaches.** The memory layer shares a load among the requests that share the layer. The stock module's layer lives as long as the application state. Fastly serves one request per instance unless the opt-in reusable sandbox is on, and Cloudflare and Spin build the application state for every request (see `AppState` in the three adapters' `app.rs`), so on those three the memory layer is not shared between requests. Across instances only Fastly makes requests share a load. Only the Fastly adapter registers a value cache, and a lookup in that cache makes every other instance at the same location wait for the instance that fills the key (`crates/trusted-server-adapter-fastly/src/value_cache.rs`), so a missing key is loaded once for each location and not once for the service. Cloudflare and Spin share copies only through the key-value store each adapter declares, `TRUSTED_SERVER_KV` and `default`, which makes no request wait, so each request that misses at the same moment fetches the origin itself and later requests read the stored copy, and Axum has no shared store at all. **Serving the request whose load fetched.** A response that cannot be shared is not stored for others, yet the request that fetched it must not fetch again. `HandBack` carries the response from the load to that request. `asset_from_cache` makes one `HandBack` for each lookup, `load_asset` puts a response that cannot be shared in the slot by its id, and the lookup ends by taking the slot's response when it finds `AssetEntry::NotShareable` or `Found::NoCopies`. A request that did not lead the load finds the slot empty and asks the origin. Two other cases use the same path. A body that outgrows a copy while it is read is handed back as what was read so far followed by the rest of the stream. A response whose `Vary` no longer matches its record is handled by `copy_loader`, which removes the record, writes the marker with a lifetime of zero seconds, and serves the request what its own fetch returned, so the next request learns the new `Vary`. @@ -109,14 +109,14 @@ The range check, `LoadingCacheConfig::check`, runs in `CacheSettings::prepare_ru **How each adapter takes it.** No adapter code changes. Each adapter calls `loading_caches_from` when it builds its application state and hands the result to each request's services, so the selection reaches `asset_from_cache` through `RuntimeServices::loading_cache_for`. -| Adapter | Where the selection is made and handed on | Shared store the stock module uses | How the origin body arrives | -| ---------- | ------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ------------------------------------- | -| Fastly | `crates/trusted-server-adapter-fastly/src/app.rs`, handed on with `.loading_caches(...)` | Fastly's cache, through `FastlyValueCache` | Streamed, and read up to 1 MiB | -| Axum | `crates/trusted-server-adapter-axum/src/app.rs`, handed on with `.with_loading_caches(...)` | None. The adapter's key-value store is `UnavailableKvStore`, so only the memory layer holds a copy | Whole, as the adapter does not stream | -| Cloudflare | `crates/trusted-server-adapter-cloudflare/src/app.rs`, with `.with_loading_caches(...)` | The key-value store the host binds as `TRUSTED_SERVER_KV`, when it binds one | Whole, as the adapter does not stream | -| Spin | `crates/trusted-server-adapter-spin/src/app.rs`, handed on with `.with_loading_caches(...)` | The host's default key-value store, which gives an entry no lifetime | Whole, as the adapter does not stream | +| Adapter | Where the selection is made and handed on | Shared store the stock module uses | How the origin body arrives | +| ---------- | ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ------------------------------------- | +| Fastly | `crates/trusted-server-adapter-fastly/src/app.rs`, handed on with `.loading_caches(...)` | Fastly's cache, through `FastlyValueCache` | Streamed, and read up to 1 MiB | +| Axum | `crates/trusted-server-adapter-axum/src/app.rs`, handed on with `.with_loading_caches(...)` | None. The adapter's key-value store is `UnavailableKvStore`, so only the memory layer holds a copy | Whole, as the adapter does not stream | +| Cloudflare | `crates/trusted-server-adapter-cloudflare/src/app.rs`, with `.with_loading_caches(...)` | The `TRUSTED_SERVER_KV` namespace, which the adapter declares, so a Worker without the binding fails every request | Whole, as the adapter does not stream | +| Spin | `crates/trusted-server-adapter-spin/src/app.rs`, handed on with `.with_loading_caches(...)` | The `default` key-value store, which the adapter declares and which gives an entry no lifetime | Whole, as the adapter does not stream | -The stock module, `crates/loading-cache/51degrees`, uses a host's value cache when the services carry one and the key-value store otherwise (`through_store` in `stores.rs`). Only the Fastly adapter registers a value cache. The module's memory layer lives as long as the application state, except on Cloudflare, where one cache lives as long as the isolate. Only the Fastly adapter overrides `supports_streaming_responses`, so the others hand `load_asset` a body that is already whole. +The stock module, `crates/loading-cache/51degrees`, uses a host's value cache when the services carry one and the key-value store otherwise (`through_store` in `stores.rs`). Only the Fastly adapter registers a value cache. The module's memory layer lives as long as the application state, which is one request on Cloudflare and Spin. Only the Fastly adapter overrides `supports_streaming_responses`, so the others hand `load_asset` a body that is already whole. **What the caches add on Fastly.** The origin fetch asks for the platform's default cache behavior (`PlatformCacheIntent::Default`). The Fastly adapter leaves the request alone for that intent (`apply_fastly_cache_intent`), so Fastly's readthrough cache stays on, and `apply_origin_cache_intent` keeps subresources on that default. Fastly's own cache therefore already holds a subresource for the lifetime the origin gives it, and a load can be answered from that cache and not from the origin. The asset caches add a second copy, held in the value cache for at most `store_seconds`. This change makes no measurement of the gain over the platform's cache on Fastly. The Cloudflare adapter likewise leaves the Workers runtime's default cache behavior in place for that intent (`outbound_cache_mode`). @@ -144,10 +144,10 @@ The stock module, `crates/loading-cache/51degrees`, uses a host's value cache wh - **A slow origin.** For a response that passes the header checks, the request that leads a load waits for the whole body, up to 1 MiB, before its reader is served, and each request waiting on that load waits as long. An origin fetch has a first byte timeout and a between bytes timeout of 15 seconds, on the publisher path (`DEFAULT_PUBLISHER_FIRST_BYTE_TIMEOUT`) and on an asset route (`DEFAULT_FIRST_BYTE_TIMEOUT`). A failed load reaches every request waiting on it and each of them then fetches the origin itself, so a reader can wait for one failed load and then for a fetch of their own. - **Removals.** When a response no longer varies as its record says, the request that fetched it removes the record, and a lookup that finds an entry it cannot read removes that entry. On Fastly a removal is a purge by surrogate key (`purge_surrogate_key` in the Fastly adapter's `value_cache.rs`), and on a key-value store it is a delete. An origin whose `Vary` differs from one response to the next causes a removal for each response that differs from the record. -- **Keys and variants without a limit.** Each query string, each host the reader asked, each scheme, and each combination of values for the headers `Vary` names is a key of its own. A miss writes the copy or the marker to the shared store, and the record too when the address has none, or the record alone when the origin's `Vary` allows no copies. This change sets no limit on the number of keys. `store_seconds` limits how long a copy is used and `store_idle_seconds` how long it lasts unused. Spin's key-value store gives an entry no lifetime, so there an entry that nobody reads again stays in the store, because only a read past its drop time, or a removal, takes it out. +- **Keys and variants without a limit.** Each query string, each host the reader asked, each scheme, and each combination of values for the headers `Vary` names is a key of its own. A miss writes the copy or the marker to the shared store, and the record too when the address has none, or the record alone when the origin's `Vary` allows no copies. This change sets no limit on the number of keys. `store_seconds` limits how long a copy is used and `store_idle_seconds` how long it lasts unused. Spin's `default` store, which the Spin adapter declares, gives an entry no lifetime, so there an entry that nobody reads again stays in the store, because only a read past its drop time, or a removal, takes it out. - **The fingerprint.** The publisher path computes `template_fingerprint`, a digest of all the serialized settings, for each eligible request, whether or not a copy is found. - **Memory.** The memory layer holds at most 256 records and 32 copies, and a copy's body is at most 1 MiB, so the copies take up to about 32 MiB for as long as the layer lives. The request that leads a load also holds the whole body while it reads. -- **Reads and writes of the shared store.** A request that misses the memory layer reads the shared store up to twice, once for the record and once for the copy. The record's load writes the record with the copy or the marker, or the record alone when the origin's `Vary` allows no copies, and a copy's load writes only the copy or the marker. The memory layer keeps an entry for at most `memory_seconds`, 5 seconds by default, and it is not shared between requests on Spin or, by default, on Fastly, so there every request reads the store. On Cloudflare and Spin the store is the host's key-value store, when the host supplies one, and a write to Cloudflare's store is given a lifetime of at least a minute (`KV_SHORTEST`), because Workers KV refuses less, with the entry's own drop time holding the exact lifetime. On Axum the adapter's store is `UnavailableKvStore`, so every read and write fails and is logged at debug level, a failed read counts as a miss, and only the memory layer holds copies. +- **Reads and writes of the shared store.** A request that misses the memory layer reads the shared store up to twice, once for the record and once for the copy. The record's load writes the record with the copy or the marker, or the record alone when the origin's `Vary` allows no copies, and a copy's load writes only the copy or the marker. The memory layer keeps an entry for at most `memory_seconds`, 5 seconds by default, and it is not shared between requests on Cloudflare, on Spin or, by default, on Fastly, so there every request reads the store. On Cloudflare and Spin the store is the key-value store the adapter declares, `TRUSTED_SERVER_KV` or `default`, and a write to Cloudflare's store is given a lifetime of at least a minute (`KV_SHORTEST`), because Workers KV refuses less, with the entry's own drop time holding the exact lifetime. On Axum the adapter's store is `UnavailableKvStore`, so every read and write fails and is logged at debug level, a failed read counts as a miss, and only the memory layer holds copies. - **Purges.** `ts cache purge` does not reach a copy. It purges by the `ts-template` key or by a page's key, and a copy in Fastly's cache carries neither, only a key made of `vc-` and a hash of its own key (the Fastly adapter's `value_cache.rs`). A copy on another adapter sits in the key-value store or, on Axum, in memory, and `ts cache purge` touches neither. **Rollout for existing deployments.** @@ -211,3 +211,4 @@ Joins with no test: | 2026-10-15 | First draft, written after the code. | | 2026-10-15 | Fixes from review round 1. The cookie rules the asset gate leaves out and what that lets through, the reason the code gives, the default-on state and the open questions on cookie rules and an off switch, `Pragma` in the gate, the verdict a response that fails a test leaves for 60 seconds, the module default, where the sizes and the `[cache.assets-vary]` table stand, the key-value store each adapter uses, how far one fetch reaches, what the caches add on Fastly, `[[cache.asset_rules]]`, the headers a route forwards, the cost and limits, the test doubles and the missing tests. | | 2026-10-16 | Fixes to the code, and from review round 2. A request that sends the publisher's origin a cookie neither reads a copy nor leads a load, `[cache.assets] module = "none"` turns both asset caches off, and a `[cache.assets]` lifetime out of range is refused when the settings load, each with its tests. Which entries each load writes, the tests that send a cookie, and the module default in the guide and the code comments. | +| 2026-10-17 | The code fix on row 24, where the Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests, and where each application state has its own memory layer, Cloudflare included. How far one fetch reaches, the adapters' shared stores, how long the memory layer lives, and the reads and writes of the shared store say so. | From 0c226994a8a26cbfabc1d62ab10017277ceb59ee Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:15:53 +0100 Subject: [PATCH 135/192] The robots.txt modules design says no adapter holds an answer before the stores open At the robots.txt modules change no adapter gives a request a working key-value store, so the design says nothing is held there until the stores open. --- ...26-10-12-robots-txt-from-modules-design.md | 41 ++++++++++--------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md b/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md index 62cd1335cf..a8cbd6885d 100644 --- a/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md +++ b/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-module-context-design.md` (row 8), `2026-10-12-loading-cache-design.md` (row 24) and `2026-10-12-robots-txt-middleware-design.md` (row 25), all in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342), which holds every design document of the stack **Related PRs:** [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355), which is row 14 of the stack, stacks on the removal of administration from the publisher's domain in [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354) (row 13) and asks contributors through the module context of [IABTechLab/trusted-server#1349](https://github.com/IABTechLab/trusted-server/pull/1349) (row 8). The loading cache in [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24) changes how answers are held, and the middleware in [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366) (row 25) replaces the route. [IABTechLab/trusted-server#1301](https://github.com/IABTechLab/trusted-server/pull/1301) and [jwrosewell/trusted-server#9](https://github.com/jwrosewell/trusted-server/pull/9) are closed, and so are [IABTechLab/trusted-server#1094](https://github.com/IABTechLab/trusted-server/pull/1094) and [jwrosewell/trusted-server#3](https://github.com/jwrosewell/trusted-server/pull/3), which this spec named before and which the integration seam in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7) and the module context in IABTechLab/trusted-server#1349 replace **Issues:** [IABTechLab/trusted-server#1272](https://github.com/IABTechLab/trusted-server/issues/1272). [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-17 > **Why this spec exists.** `/robots.txt` reaches the publisher's origin like any other path, so a deployment has no say in it. This spec says how the code at `35eb5ae83` gives a deployment that say, what the code leaves out and which questions about the code are open. @@ -76,29 +76,29 @@ and with no `always_allow` the file is `User-agent: *` and `Disallow: /`. `sitem **When an answer is asked for.** -| Case | Answer | -| --------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| A held answer younger than the contributor's `refresh` | The held answer, and the contributor is not asked | -| The contributor answers | Its answer, which is written to the store with a lifetime of 30 days and served until `refresh` is due again | -| The contributor fails and an answer is held | The held answer, past its age, and a warning in the log | -| The contributor fails and nothing is held | `503` with `Retry-After: 600` and `Cache-Control: no-store`, and the body `robots.txt is temporarily unavailable` | -| The store cannot be read, or a held entry does not read | Nothing is held, so the contributor is asked | -| The store cannot be written | The answer is served, and the failure is logged at debug level | -| The store is the unavailable store, as on Fastly and Axum | Every read and every write fails, so nothing is held, every request asks each contributor, and one failure is a `503` | -| The store gives an entry no lifetime, as Spin's does not | The answer is written without one, and written again under the same key when the contributor next answers. An entry held under a fingerprint that has since changed is never replaced and stays until it is deleted from the store | +| Case | Answer | +| --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| A held answer younger than the contributor's `refresh` | The held answer, and the contributor is not asked | +| The contributor answers | Its answer, which is written to the store with a lifetime of 30 days and served until `refresh` is due again | +| The contributor fails and an answer is held | The held answer, past its age, and a warning in the log | +| The contributor fails and nothing is held | `503` with `Retry-After: 600` and `Cache-Control: no-store`, and the body `robots.txt is temporarily unavailable` | +| The store cannot be read, or a held entry does not read | Nothing is held, so the contributor is asked | +| The store cannot be written | The answer is served, and the failure is logged at debug level | +| The store is the unavailable store, as on every adapter on this row | Every read and every write fails, so nothing is held, every request asks each contributor, and one failure is a `503` | +| The store gives an entry no lifetime, as Spin's does, though no adapter hands the file Spin's store on this row | The answer is written without one, and written again under the same key when the contributor next answers. An entry held under a fingerprint that has since changed is never replaced and stays until it is deleted from the store | Leaving out the rules of a contributor that failed would allow by omission whatever those rules refused, which is why one failure with nothing held fails the whole file. The `503` is a status in the 500 to 599 range, so a crawler that follows RFC 9309 section 2.3.1.4 assumes a complete disallow while the file is unreachable because of a server or network error. -**Which adapters hold answers.** Cloudflare and Spin pass the key-value store that the runtime hands the application, and the unavailable store when it hands none (Cloudflare `platform.rs` lines 618 to 621 and Spin `platform.rs` lines 726 to 729). The Cloudflare store is the namespace bound as `TRUSTED_SERVER_KV` in `wrangler.toml`, and the Spin store is the `default` store that `spin.toml` declares. Fastly and Axum pass the unavailable store on every request (Fastly `app.rs` lines 340 and 433 and Axum `platform.rs` lines 572 to 574), so on those two adapters nothing is held. No adapter test runs a store that holds answers (Open question 1). +**Which adapters hold answers.** Cloudflare and Spin pass the key-value store that the runtime hands the application, and the unavailable store when it hands none (Cloudflare `platform.rs` lines 618 to 621 and Spin `platform.rs` lines 726 to 729). The runtime hands neither adapter a store on this row, because neither declares one in `Hooks::stores`, so EdgeZero's `run_app` opens neither the namespace bound as `TRUSTED_SERVER_KV` in `wrangler.toml` nor the `default` store `spin.toml` grants. Fastly and Axum pass the unavailable store on every request (Fastly `app.rs` lines 340 and 433 and Axum `platform.rs` lines 572 to 574), so on every adapter nothing is held. Row 24 declares the two stores, where the loading cache holds answers (Open question 1). **The route.** `GET` and `HEAD /robots.txt` are registered on all four adapters, only when the settings carry the section. `HEAD` is answered with the headers of the `GET` and no body (`mod.rs` lines 283 to 304), and the contributor is asked for a `HEAD` as for a `GET`, with the method `HEAD`. Every other method on the path goes to the publisher fallback as before. What a request passes before the handler differs by adapter. -| Adapter | How the route is registered | The store | What the request passes before the handler | `X-Robots-Tag` coverage | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | -| Fastly | A row of `NAMED_ROUTES` for `GET` and `HEAD` (`app.rs` lines 1417 to 1421), left out without the section by `named_route_is_served` (lines 1425 to 1430 and 1530) | The unavailable store | The request's services, the request preparers, the Edge Cookie request state and the request filters, in that order (`execute_named`, lines 766 to 828). A failure in the Edge Cookie step, or a response from a filter, is the answer in place of the file. After the handler the response passes the Edge Cookie finalize step | Not on the two debug endpoints or on a failure before the settings are read. A filter's response header changes are applied after the tag | -| Axum | A row of `named_routes()` for `GET` and `HEAD` (`app.rs` lines 541 to 545), left out without the section by `named_route_is_served` (lines 550 to 555 and 813) | The unavailable store | The request preparers (`execute_handler`, line 289) | Not on a `404` or `405` that the router gives a request it does not route, as the configuration guide lists | -| Cloudflare | Registered in `build_router` (`app.rs` lines 892 to 904), with `GET` and `HEAD` to the handler and every other method to the publisher fallback | The runtime's namespace, else the unavailable store | The request preparers (`make_handler`, line 437) | Not on a `404` or `405` that the router gives a request it does not route, as the configuration guide lists | -| Spin | Registered in `build_router` as on Cloudflare (`app.rs` lines 1107 to 1128) | The runtime's store, else the unavailable store. The Spin store gives no lifetime | No request preparer. The handler builds the services and answers (lines 1110 to 1120). The router's middleware runs as for every route, and removes the forwarded headers | Not on a `404` or `405` that the router gives a request it does not route, as the configuration guide lists | +| Adapter | How the route is registered | The store | What the request passes before the handler | `X-Robots-Tag` coverage | +| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | +| Fastly | A row of `NAMED_ROUTES` for `GET` and `HEAD` (`app.rs` lines 1417 to 1421), left out without the section by `named_route_is_served` (lines 1425 to 1430 and 1530) | The unavailable store | The request's services, the request preparers, the Edge Cookie request state and the request filters, in that order (`execute_named`, lines 766 to 828). A failure in the Edge Cookie step, or a response from a filter, is the answer in place of the file. After the handler the response passes the Edge Cookie finalize step | Not on the two debug endpoints or on a failure before the settings are read. A filter's response header changes are applied after the tag | +| Axum | A row of `named_routes()` for `GET` and `HEAD` (`app.rs` lines 541 to 545), left out without the section by `named_route_is_served` (lines 550 to 555 and 813) | The unavailable store | The request preparers (`execute_handler`, line 289) | Not on a `404` or `405` that the router gives a request it does not route, as the configuration guide lists | +| Cloudflare | Registered in `build_router` (`app.rs` lines 892 to 904), with `GET` and `HEAD` to the handler and every other method to the publisher fallback | The unavailable store, as the adapter declares no store for the runtime to open | The request preparers (`make_handler`, line 437) | Not on a `404` or `405` that the router gives a request it does not route, as the configuration guide lists | +| Spin | Registered in `build_router` as on Cloudflare (`app.rs` lines 1107 to 1128) | The unavailable store, as on Cloudflare | No request preparer. The handler builds the services and answers (lines 1110 to 1120). The router's middleware runs as for every route, and removes the forwarded headers | Not on a `404` or `405` that the router gives a request it does not route, as the configuration guide lists | On Fastly the configuration, permissions and data pages answer before the Edge Cookie steps and the request filters, and the file does not (`app.rs` lines 761 to 792). A filter that responds, such as a bot-protection challenge, ends the route with its own response (`app.rs` lines 698 to 704 and 819 to 821), and RFC 9309 section 2.3.1.3 lets a crawler access any resource on the site when the answer for the file is in the 400 to 499 range. No test sends a filter's answer through this route (Open question 5). @@ -127,7 +127,7 @@ On Fastly the configuration, permissions and data pages answer before the Edge C **How soon a crawler sees a change.** The `200` is cacheable for an hour, as the cache headers above say. A crawler or a cache can keep the previous file for up to an hour after the settings change, after a contributor's rules change or after a build starts serving the file. Where answers are held, a contributor's new rules reach the file on the first request after its `refresh` is due. An answer held under a fingerprint that has since changed is neither used nor replaced, because the new fingerprint gives a new key. That entry stays in the store for 30 days after it was last written where the store gives a lifetime, and until someone deletes that entry where the store does not. -**The guide and the changelog.** The configuration guide (`docs/guide/configuration.md` line 1057) and `CHANGELOG.md` (line 82) say a module's answer is held in the key-value store. Neither says that the Fastly and Axum adapters hold nothing (Open question 1). `CHANGELOG.md` line 82 also says "a contributor cannot produce a malformed one". The paragraph "Where those guarantees stop" in section 2 says otherwise, because core writes a group with no `User-Agent` line and a record before the first group as given (Open question 7). +**The guide and the changelog.** The configuration guide (`docs/guide/configuration.md` lines 1057 and 1058) and `CHANGELOG.md` (line 85) say a module's answer is held in the key-value store a request is given, and that no adapter gives a request a working one, so nothing is held (Open question 1). `CHANGELOG.md` line 85 also says "a contributor cannot produce a malformed one". The paragraph "Where those guarantees stop" in section 2 says otherwise, because core writes a group with no `User-Agent` line and a record before the first group as given (Open question 7). ## 4. What is not in this change @@ -138,7 +138,7 @@ On Fastly the configuration, permissions and data pages answer before the Edge C **Open questions.** Each of these is a question about the code. This spec records what the code does and leaves the questions open. -1. **What Fastly and Axum hold.** Should the Fastly and Axum adapters pass a store that holds answers? Both pass the unavailable store on every request, so each request asks every contributor and one contributor's failure is a `503`. No adapter test runs a store that holds answers, and the configuration guide and the changelog say an answer is held in the key-value store without naming the adapters that do it. +1. **What the adapters hold.** Should the adapters pass a store that holds answers? All four pass the unavailable store on every request, Fastly and Axum directly and Cloudflare and Spin because neither declares a store for EdgeZero to open, so each request asks every contributor and one contributor's failure is a `503`. The adapter tests of a held answer put a store kept in memory on the request, where no runtime puts one on this row. Row 24 holds answers on every adapter through the loading cache, and there the Cloudflare and Spin adapters declare their stores. 2. **What a contributor can read.** Should the services a contributor is handed leave out the connection of the request that started the call? The documentation of `contribute` says the call carries nothing about who asked, while a contributor that names `&RuntimeServices` can read the client IP address, the TLS and HTTP/2 details and the host's client signals, and its answer is then served to every crawler. No test reads `client_info` from a contributor. 3. **Host, scheme and method.** Should the key include the host, the scheme and the method, or should a contributor not be handed them? The key holds none of the three, so the request that asks the contributor decides the answer for the requests that follow, and that request can be a `HEAD` or carry a `Forwarded` or `X-Forwarded-Host` header. Where answers are held, Spin removes both headers and Cloudflare removes neither. 4. **An empty answer.** Should a contribution with no groups count as a failure? Today it is an answer and is held like any other. A file made only of such answers is a `200` with one newline, which a crawler reads as permission to crawl everything. @@ -201,3 +201,4 @@ Open question 8 asks whether adapter tests should cover the first four. | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the settings, the refusal and the empty answer, what a contributor is given and what the key holds, the store each adapter passes, the steps each adapter runs before the handler, the responses that carry `X-Robots-Tag`, the cache headers, consent and privacy, what is refused and when, what changes for a deployment, the rows that replace the route and the holding, the open questions, the tests, the acceptance and the review log. | | 2026-10-16 | Points from review round 2. The citation for the line breaks removed from `User-Agent` names, the changelog's claim that a contributor cannot produce a malformed file, and the earlier commit that the description says CI passed on. | +| 2026-10-17 | Which adapters hold answers on this row. None does, Cloudflare and Spin included, because neither declares a store for EdgeZero to open, so the store table, the store column of the route table, the note on the guide and the changelog, and open question 1 say so, and say that row 24 declares the two stores. | From db22233707bf57d48f35eab973dd2f14c90d5b16 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:15:55 +0100 Subject: [PATCH 136/192] The robots.txt middleware design names the store each adapter opens The design names where each adapter keeps a robots.txt answer once the Cloudflare and Spin adapters declare their key-value stores. --- .../specs/2026-10-12-robots-txt-middleware-design.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md b/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md index 02f20e2316..1f3212d6b4 100644 --- a/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md +++ b/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md @@ -150,7 +150,7 @@ The answers of the contributors are a different copy. The holding keeps each con **When it fails.** 1. **The request to the origin fails.** The crawler gets an error response with status `502` and the body `An internal error occurred`, and no file is made (`publisher.rs` 5778 to 5799, `error.rs` 119 and `http_error` in each adapter's `src/app.rs`). RFC 9309 section 2.3.1.4 has a crawler take the `502` as refusing everything. The route never asked the origin, so the route could not fail this way. -2. **A contributor fails and its last answer is kept.** The file is made from the kept answer past its age, and the holding logs a warning (`robots_txt/holding.rs` 111 to 126). The last answer is kept apart for 30 days (`KEPT_LIFETIME`, 39 to 41), in the host's value cache when the host supplies one and in the key-value store otherwise (205 to 249). A value cache may drop the entry early, and Spin's key-value store gives an entry no lifetime, as the configuration guide says (lines 1076 and 1078). +2. **A contributor fails and its last answer is kept.** The file is made from the kept answer past its age, and the holding logs a warning (`robots_txt/holding.rs` 111 to 126). The last answer is kept apart for 30 days (`KEPT_LIFETIME`, 39 to 41), in the host's value cache when the host supplies one and in the key-value store otherwise (205 to 249), being the store the Cloudflare and Spin adapters declare, `TRUSTED_SERVER_KV` and `default`. Axum's key-value store fails every call, so Axum keeps no answer. A value cache may drop the entry early, and Spin's key-value store gives an entry no lifetime, as the configuration guide says (lines 1076 and 1078). 3. **A contributor fails and nothing is kept.** The middleware refuses the request. The crawler gets `503` with `Retry-After: 600` (`RETRY_AFTER_SECONDS`, `robots_txt/mod.rs` 64 to 65), `Content-Type: text/plain; charset=utf-8`, `Cache-Control: no-store, private` and the body `robots.txt is temporarily unavailable`, and nothing is stored (`robots_txt/middleware.rs` 29 to 30 and 94 to 99, `publisher.rs` 851 to 868 and 6035 to 6058). One failed contributor among several fails the whole file, because leaving out its rules would allow by omission what those rules refused (`robots_txt/mod.rs` 26 to 29). The error is logged (`robots_txt/middleware.rs` 95) with every address in what the contributor reported cut to its scheme and host (`addresses_cut_to_their_host` in `robots_txt/holding.rs`), so a key in the path, query or user part of an address is not logged, and the kept answer's warning carries the same text. 4. **Nothing was prepared when the action runs.** The file refuses every crawler, with the `always_allow` paths open, whatever `modules` selects, and an error is logged (`create` in `robots_txt/middleware.rs`). Core prepares every response it streams before the action runs, so no request reaches the action without its preparation. @@ -264,3 +264,4 @@ git grep -n "answers_robots_txt_with_a_page" 16248774e -- crates | 2026-10-16 | Points from review round 2. The `304` of case 10, cases 7 and 8, the lines of the `[cache.robots-txt] module` default, row 33's change to a test of the holding, and the footprint row for the adapter tests. | | 2026-10-16 | Contributors are asked for the host the request was addressed to, never a forwarded one, which the publisher path hands every preparation context, and Fastly answers a `GET` or `HEAD` of the file before the Edge Cookie steps and the request filters. What a contributor is given, the adapters and five tests. | | 2026-10-17 | One text for both sets of code fixes above. The code that holds both keeps the origin's answer, a `404` among them, for a request that names no host, and hands a contributor nothing of the crawler's connection, so what happens, the steps of a request, case 11, open question 2 and what a contributor is given now say so. | +| 2026-10-17 | The kept answer for a failed contributor names the stores the Cloudflare and Spin adapters declare on row 24, `TRUSTED_SERVER_KV` and `default`, and says that Axum keeps none. | From 21ced62354b5fd15e281d8cbd2519da50742e781 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:15:57 +0100 Subject: [PATCH 137/192] The module cache design names the shared store on each adapter The design names the shared store each adapter opens for a declared cache. --- .../specs/2026-10-15-module-cache-declarations-design.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md index b0106e054b..ced5d85380 100644 --- a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md +++ b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-12-loading-cache-design.md`, `2026-10-12-robots-txt-from-modules-design.md`, `2026-08-27-integration-provider-seam-design.md` **Related PRs:** IABTechLab/trusted-server#1373, reviewed on jwrosewell/trusted-server#27 (closed) **Issues:** IABTechLab/trusted-server#1320 -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-17 > **Why this spec exists.** A module that holds its own answers asks core for the cache of the name it chose, finds no `[cache.]` table for that name, and gets the cache that holds nothing, so every request loads the answer again. @@ -141,7 +141,7 @@ The adapters differ in how a failed state build answers, see 2.7. Only the Axum This change stores nothing new and reads no request data. The change decides which loading cache module holds a module's copies. Five facts follow. 1. **What core knows about a copy.** Core sees a cache's name, its lifetimes and the key a module builds with `StoreKey::new(prefix, cache, key)`. Core does not read a key or a value, and the change adds no consent check on this path. A module that holds an answer which depends on the visitor has to put what the answer depends on into its key, or not hold the answer. The change gives the module a cache and does not settle that question for the module. -2. **Who can read a copy.** A key starts with the publisher's key prefix and the cache's name, so two publishers sharing a store never share a copy. Within one publisher, the copy in the shared store is the one every instance reads. Two modules of one publisher keep their copies apart by convention and not by a check. Two builders cannot declare one name, and a module that gives its `CacheSpec` and its `StoreKey` the name it declared holds its copies apart from every other module's. Core does not check that a module does so, because `CacheSpec.name` is any `&'static str`, `StoreKey::new` takes any string for the cache, and `load_cached` picks the cache by the name in the `CacheSpec` it is given. A module that gives its `CacheSpec` another module's declared name, or `robots-txt`, is held by the cache that name selected. Its keys start the same way as that module's only if it also builds its `StoreKey` with that name, because the key is a separate argument to `load_cached`. See 2.10. +2. **Who can read a copy.** A key starts with the publisher's key prefix and the cache's name, so two publishers sharing a store never share a copy. Within one publisher, the copy in the shared store is the one every instance reads, being Fastly's cache, or the key-value store the Cloudflare and Spin adapters declare, `TRUSTED_SERVER_KV` and `default`, and Axum has no shared store. Two modules of one publisher keep their copies apart by convention and not by a check. Two builders cannot declare one name, and a module that gives its `CacheSpec` and its `StoreKey` the name it declared holds its copies apart from every other module's. Core does not check that a module does so, because `CacheSpec.name` is any `&'static str`, `StoreKey::new` takes any string for the cache, and `load_cached` picks the cache by the name in the `CacheSpec` it is given. A module that gives its `CacheSpec` another module's declared name, or `robots-txt`, is held by the cache that name selected. Its keys start the same way as that module's only if it also builds its `StoreKey` with that name, because the key is a separate argument to `load_cached`. See 2.10. 3. **How long a copy lasts.** A module's `CacheSpec` sets how long its copies last, and core applies no declared table's lifetimes, see 2.3. Where a module applies its table's lifetimes itself, the shortest the range check allows is no copy in process memory (`memory_seconds = 0`) and one second in the shared store (`store_seconds = 1`). 4. **Whether the document can turn a cache off.** The document cannot turn a declared cache off in a build that links one loading cache module. Such a build holds a declared cache's answers whether or not the document has a table, and a table naming `none`, or any other name that no linked crate offers a cache from, is refused when the state is built. Core already does the same for `[cache.robots-txt]`. Reviewers are asked to confirm that a declared cache has no off switch, see 2.10. 5. **What changes today.** No module that ships declares a cache, so no deployment holds a new kind of answer because of this change. @@ -253,3 +253,4 @@ These joins have no test. | 2026-10-15 | First draft, written after the code. | | 2026-10-15 | Fixes from review round 1. What core does with a declared table's lifetimes, the names core does not check, the changelog entry and the three changes the entry names, the order of a push, the other callers of `loading_caches_from`, the reserved and unchecked name forms, why the deploy check does not read `module`, the untested paths, the open questions, the review log, and the service call figure in section 1, dropped because it has no source. | | 2026-10-15 | Corrections from review round 2. What a module that borrows another module's cache name shares, the section the guide's push-order warning covers, the builders the deploy check is given, the round 1 log sentence on the code changes, and the follow-ups for the configuration guide, the integration guide, the rustdoc and the visibility of two functions. | +| 2026-10-17 | Who can read a copy names the shared store on each adapter, being Fastly's cache, the key-value store the Cloudflare and Spin adapters declare on row 24, `TRUSTED_SERVER_KV` and `default`, and none on Axum. | From e5a55f7d4b61ea6ad274e37945d67d61a951826e Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:16:09 +0100 Subject: [PATCH 138/192] The request addressing design takes the HTTP/2 host fix The request host falls back to the URI's authority when no header names one, for IABTechLab/trusted-server#1118, and the design records it with its tests. --- .../specs/2026-10-13-request-addressing-design.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-13-request-addressing-design.md b/docs/superpowers/specs/2026-10-13-request-addressing-design.md index 8f084c7959..443b833296 100644 --- a/docs/superpowers/specs/2026-10-13-request-addressing-design.md +++ b/docs/superpowers/specs/2026-10-13-request-addressing-design.md @@ -4,8 +4,8 @@ **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-12-config-by-host-design.md`, `2026-10-12-middleware-prepare-design.md`, `2026-10-12-prefix-module-design.md`, `2026-08-27-integration-provider-seam-design.md` and `2026-10-15-cookies-by-path-design.md`, all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** IABTechLab/trusted-server#1376, jwrosewell/trusted-server#30 (closed) -**Issues:** IABTechLab/trusted-server#1328 -**Last updated:** 2026-10-16 +**Issues:** IABTechLab/trusted-server#1328, IABTechLab/trusted-server#1118 +**Last updated:** 2026-10-17 > **Why this spec exists.** An adapter chooses the app-config blob that serves a request by the request's host alone, before any settings exist, so one host cannot serve several publishers under paths of their own. @@ -92,6 +92,8 @@ On Spin the hook is usually given no host, because Spin rarely passes the `Host` **Putting the removed path prefix back.** Core never writes the removed path prefix into a response. A deployment whose pages must carry it, in their links for example, does that with a middleware of its own, whose module's request preparer reads `RemovedPathPrefix` and leaves it for the middleware in `IntegrationRequestState`. That middleware runs in the serve phase. This has a cost. A request that carries any value in `IntegrationRequestState` keeps to the origin path, so its page is never read from a stored template or stored as one, and its HTML is sent `private, no-store` (see `IntegrationRequestState` in `crates/trusted-server-core/src/integrations/registry.rs`). Every page whose removed path prefix is put back this way is therefore made for its reader alone. The request to the origin for that page bypasses the platform's cache only when origin readthrough is on or the ad stack runs (`apply_origin_cache_intent` in `crates/trusted-server-core/src/publisher.rs`). Origin readthrough is off by default (`origin_readthrough_enabled`, see "Origin readthrough caching" in `docs/guide/configuration.md`), so with no ad stack the platform's cache can still answer under the origin's own `Cache-Control`. The stored template is keyed by the request's host and continuing path and by its blob's settings, not by the removed path prefix, so a page changed in the fetch phase could be stored with one removed path prefix and served under another that names the same blob. +**The host core processes a request for.** Once a request is routed, core reads the host it rewrites a page for and tells each middleware from `Forwarded`, then `X-Forwarded-Host`, then `Host`, and last from the URI's authority, as `address_request` gives the hook the authority where no `Host` is sent (`RequestInfo::from_request` in `crates/trusted-server-core/src/http_util.rs`). An HTTP/2 client may send its `:authority` and no `Host` header, and before the authority was read such a request had no host, so its page was returned unprocessed (IABTechLab/trusted-server#1118). + ## 3. What is not in this change - **The deployment's policy.** Which part of a path names a publisher, and how a value is spelled, belong to the deployment that supplies the hook. @@ -108,6 +110,8 @@ Tests in `crates/trusted-server-core/src/addressing.rs` cover the default answer Tests in `crates/trusted-server-adapter-fastly/src/config_selection.rs` cover a value taking the placeholder's place, the first existing value winning with no later lookup, every value missing refused and not served from the logical store ID or another host's blob, a value that is not a host name passed over without a lookup, a selector without the placeholder ignoring the values, and a failed lookup refusing and trying no later value. Tests in `crates/trusted-server-adapter-fastly/src/lib.rs` drive the entry point's selection under Viceroy, with no hook and with a "by host" hook choosing as before, the registered hook being the one asked, a rewritten request converted with its path, query and removed path prefix, the first existing value read from a real local store, and every value missing answered `421` with the request left as it was. Each of the Axum, Cloudflare and Spin route tests serves `//_ts/config.json` as the configuration endpoint through `routes_with_addressing`, and routes a "by host" request as the router with no hook does. +Tests in `crates/trusted-server-core/src/http_util.rs` cover the host read from the URI's authority, with its port, where no header names one, no host where nothing names one, and each of `Host`, `X-Forwarded-Host` and `Forwarded` taking precedence over the authority. `a_request_naming_its_host_only_in_its_uri_reaches_the_middleware`, in `crates/trusted-server-core/src/publisher.rs`, sends a request with no `Host` header through the publisher path and finds that the entry's middleware ran for the host the URI names. + Two paths have no test. No test drives `edgezero_main` with a hook registered through `run_with_addressing`, which appears only in a doctest that is not run, and no test covers the `500` that `addressed_router` answers when the continuing path cannot be set on the request. ## Review log @@ -122,3 +126,4 @@ Two paths have no test. No test drives `edgezero_main` with a hook registered th | 2026-10-13 | First draft. | | 2026-10-15 | Fixes from review round 1. The cost of putting the removed path prefix back, the shared browser origin as a limit, the host each adapter gives the hook, failure handling, rollout, the untested paths and the review log. | | 2026-10-15 | Corrections from review round 2. The origin fetch of a page whose removed path prefix is put back, the rollout order on Fastly, where the copy on `split/6-integration-seam` is brought up to date, and the removed path prefix named in full. | +| 2026-10-17 | The host core processes a request for falls back to the URI's authority where no header names one (IABTechLab/trusted-server#1118), with its tests. | From eed9b31ede7fe9aef05cad9750f89a2edc98e7a9 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:16:10 +0100 Subject: [PATCH 139/192] The backend naming design says what certificate_check does on each adapter Only the Fastly adapter honors certificate_check = false, the Axum adapter warns at startup that it does not, and the design says what Cloudflare and Spin do, for IABTechLab/trusted-server#1132. --- .../superpowers/specs/2026-10-14-backend-naming-design.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-14-backend-naming-design.md b/docs/superpowers/specs/2026-10-14-backend-naming-design.md index 3d53fd0392..1dfd81b1fe 100644 --- a/docs/superpowers/specs/2026-10-14-backend-naming-design.md +++ b/docs/superpowers/specs/2026-10-14-backend-naming-design.md @@ -4,8 +4,8 @@ **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-10-config-first-auction-provider-architecture-design.md`, `2026-03-19-edgezero-migration-design.md`, `2026-10-14-last-names-in-core-design.md` **Related PRs:** jwrosewell/trusted-server#42, IABTechLab/trusted-server#1390 -**Issues:** IABTechLab/trusted-server#1339 -**Last updated:** 2026-10-16 +**Issues:** IABTechLab/trusted-server#1339, IABTechLab/trusted-server#1132 +**Last updated:** 2026-10-17 > **Why this spec exists.** Core held a backend naming and transport timer policy for each of the four adapters, so that startup validation and each adapter's registration predicted the same backend names before any was registered. The policies called no platform, and they still named every platform in core, where the rule is that core names none. @@ -35,6 +35,8 @@ This change makes the first two. The third has no call to move because the comma **EdgeZero owns no adapter's registration.** The issue asks that a policy live in EdgeZero where EdgeZero already owns that platform's registration, and that the series say which. EdgeZero owns none. The Fastly adapter registers its backends itself through the platform SDK in `backend.rs`. At the pinned EdgeZero revision (`683202c`), EdgeZero's own Fastly proxy (`edgezero-adapter-fastly/src/proxy.rs`) registers backends under its own `edgezero-dynamic-` prefix for its own proxy path, and this policy never predicted those names. The Cloudflare, Spin and Axum adapters of EdgeZero and of this repository register no backend at all. So every policy lives in its adapter and nothing moves into EdgeZero. +**The certificate check.** A backend's specification carries `certificate_check`, which `proxy.certificate_check` sets for the origins core proxies. The Fastly adapter registers a backend that skips the check when it is `false`. The Axum, Cloudflare and Spin adapters register no backend, so the flag changes only the name each makes. The Axum client verifies every certificate against the public authorities built into it, and the Cloudflare and Spin clients take no option to skip the check. Axum, the adapter for local development where the setting is meant to be used, warns once at startup when it is `false`, so an operator whose self-signed origin still fails is told why (IABTechLab/trusted-server#1132). + **What breaks outside core.** A deployment sees no change. A crate outside core that implements `PlatformBackend` or checks a plan against a target must change, which is why `CHANGELOG.md` marks the change as breaking. Gone from core are `BackendNamingPolicy`, `AuctionTargetId`, `PredictedBackend`, `BackendNamingError` and `PlatformBackend::naming_policy`. `AuctionPlan::validate_for_target` takes a `&AuctionTargetDescriptor` in place of an `AuctionTargetId`, and a crate builds the descriptor with `AuctionTargetDescriptor::new(name, backend, capabilities)` and `AuctionTargetCapabilities::new`. The descriptor now holds the adapter's backend, so it is no longer `Copy`, `PartialEq` or `Eq`, and it is still `Clone` and `Debug`. It has no `naming_policy()`, its methods take `&self`, and `id()` returns the adapter's name as a `&'static str` where it returned an `AuctionTargetId`. `PlatformBackend` gains `auction_backend_budget` and `transport_timeout_bucket_count`, both with defaults. A crate that implements `PlatformBackend` deletes its `naming_policy` method, because a method the trait no longer has is a compile error, and overrides `canonicalize_transport_timeout_ms`, `auction_backend_budget` and `transport_timeout_bucket_count` where its host rounds timers or limits backend names. The default `canonicalize_transport_timeout_ms` no longer follows a policy, so a crate that answered `BackendNamingPolicy::Fastly` and took the Fastly rounding from the default now gets the exact bounded timer unless it overrides the method. `docs/guide/integration-guide.md` shows an implementation without `naming_policy`. ## 3. What is not in this change @@ -54,6 +56,7 @@ This change makes the first two. The third has no call to move because the comma - Core's four policy tests went with the policy and their cases moved to the adapters. Axum, Cloudflare and Spin each pin the name and the transport timer their backend makes for one specification, with the names the core tests pinned, and each adapter's capabilities test reads the descriptor it builds. Cloudflare and Spin keep a startup test that refuses a two-provider plan and accepts the same plan with the auction disabled. - The Fastly adapter keeps three of the policy's tests in `backend.rs`, namely the pinned name with its digest, the bracketed and bare forms of an IPv6 host naming one backend, and the two timer points. Its `platform.rs` keeps the timer ladder tests, which now run against the adapter's own policy and not the default method that read core's. `backend.rs` adds a refused control character in the scheme, where the host and the Host override were already tested, and a bucket count equal to the number of distinct timers a 2,000 ms ceiling reaches. The capabilities test pins the budget of 160. - The orchestrator test `planned_standard_instances_have_distinct_backends_and_independent_results` expected a 750 ms transport timer, which was the Fastly rounding that core's naming double applied to the test's 777 ms budget. The double now keeps the exact bounded value, so the test expects 777 ms for each of the two providers' backends, and those two assertions are the only ones in the orchestrator tests that changed. Core's tests no longer exercise the Fastly rounding, and the Fastly adapter's tests cover it. +- `startup_warns_once_that_turning_the_certificate_check_off_is_not_honored`, in the Axum adapter's `app.rs`, builds the state with `proxy.certificate_check = false` and finds one warning saying certificates are still verified, and builds it with the check on and finds none. - Not tested. Nothing checks that an adapter's descriptor and its runtime services use the same backend type. - Not tested. No test drives a plan through the Fastly backend to the budget of 160. The refusal is tested in core with a double and the 160 is pinned in the Fastly capabilities test, so the two are tested apart. - Not tested. The bucket count is compared with the distinct timers only at a 2,000 ms ceiling, where the coarse ladder is never reached. At a 60,000 ms ceiling the ladder test bounds the distinct timers at 20 and one assertion checks that two providers fit the budget, but nothing asserts that the count equals the timers the ladder reaches. @@ -72,3 +75,4 @@ This change makes the first two. The third has no call to move because the comma | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The real callers of the policy, the break for crates outside core, the descriptor resting on convention, the timer bucket default, the untested paths, the platform names left in core, what the issue asks, the pull request references and the review log. | | 2026-10-15 | Fixes from review round 2. The descriptor changes the compatibility paragraph left out, the Tech Lab pull request reference and the count of changed orchestrator assertions. | +| 2026-10-17 | The certificate check (IABTechLab/trusted-server#1132). The Axum adapter warns once at startup that it does not honor `proxy.certificate_check = false`, with its test, and the guides say which adapters honor the setting. | From 8ecb78394b8b0ecbdec6691d1682c5526e2061af Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:16:12 +0100 Subject: [PATCH 140/192] The specs carry the design of the request filters on every adapter The design of the change that runs the modules' request filters and the device module on the Axum, Cloudflare and Spin adapters as on Fastly. --- ...request-filters-on-every-adapter-design.md | 87 +++++++++++++++++++ 1 file changed, 87 insertions(+) create mode 100644 docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md diff --git a/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md new file mode 100644 index 0000000000..e23e3d92cd --- /dev/null +++ b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md @@ -0,0 +1,87 @@ +# Design Spec: Request filters and device detection run on every adapter + +**Status:** Proposed, 2026-10-17. A pull request above `ts config migrate`, not yet raised, for Tech Lab review. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-10-14-51degrees-client-hints-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 +**Related PRs:** none yet +**Issues:** none +**Last updated:** 2026-10-17 + +> **Why this spec exists.** Only the Fastly adapter runs the integration request filters and the device module `[device]` selects. On Axum, Cloudflare and Spin a module's request filter never runs, so the 51Degrees client hints middleware writes nothing and DataDome's protection never asks its service, and no module call carries a device classification. This change runs both on those three adapters where the Fastly adapter runs them. + +## 1. Problem + +The Fastly adapter runs three steps that no other adapter runs. + +1. **Request filters.** `run_pre_route_filters` in `crates/trusted-server-adapter-fastly/src/app.rs` calls `IntegrationRegistry::filter_request` for every named route except the inspection endpoints and batch sync, and for the publisher fallback. It runs after the modules' preparers and once the request's Edge Cookie context is read, and it hands every filter the location and the permissions resolved for that context. A filter that answers stands in for the route. A filter that lets the request continue has its request headers applied before the route, and its response headers are applied by the entry point after the response is finalized (`apply_terminal_response_effects` in `src/lib.rs`). +2. **Device detection.** `derive_device_signals` in `src/lib.rs` calls `build_device_module` and `DeviceModule::detect` at the entry point, and `build_ec_request_state` sets the result on the Edge Cookie context. Every module call made from that context carries the classification (`ModuleContext::with_request_state` in core). +3. **robots.txt first.** `dispatch_fallback` answers robots.txt once the preparers have run and before the Edge Cookie steps and the filters, with an empty Edge Cookie context, so no filter's answer stands in for the file and none of the crawler's cookies are read. + +No code in `crates/trusted-server-adapter-axum`, `-cloudflare` or `-spin` calls `filter_request` or `DeviceModule::detect`. So on those adapters the 51Degrees client hints middleware asks its service nothing and writes no header or tag, DataDome's protection filter never runs, `[device] module` changes nothing, and robots.txt is answered through the publisher path with the crawler's own Edge Cookie context. On Spin the discovery, signature verification and first-party routes and the page-bids preflight also skip the modules' preparers, which every other adapter runs for them. + +## 2. Design + +**One request cycle per adapter, in the order the Fastly adapter runs it.** Each of the three adapters gains the steps the Fastly adapter has, under the Fastly adapter's names, in `src/app.rs`. + +| Step | Function | What it does | +| --------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Preparers | `execute_named`, `execute_fallback` | `IntegrationRegistry::prepare_request`, once for each request, as before | +| robots.txt | `asks_for_robots_txt`, `answer_robots_txt` | Answers a `GET` or `HEAD` of `/robots.txt`, when `[robots-txt]` is present, through the publisher path with `EcContext::default()` and no auction | +| Device classification | `derive_device_signals` | Runs the selected device module, or the built-in one, with the request's module context | +| Edge Cookie context | `build_ec_context` | `EcContext::read_from_request_resolving_geo`, then `set_device_signals`. An error answers the request | +| Request filters | `run_pre_route_filters` | `IntegrationRegistry::filter_request` with the context's location and permissions. An answer, or an error, stands in for the route | +| Route | `run_named_route`, `dispatch_fallback` | The handler, given the context the filters saw | +| Response effects | `attach_request_filter_effects` | Leaves the filters' response headers on the response for the finalize middleware | +| Finalize | `FinalizeResponseMiddleware` | Finalizes the response, then `apply_terminal_response_effects` writes the filters' response headers and restores the privacy guards, as the Fastly entry point does | + +**Which routes run the cycle.** The routes the Fastly adapter runs it for, which are every named route but the inspection endpoints, and the publisher fallback, being robots.txt, tsjs, the module routes and the publisher's origin. The inspection endpoints are read only and stay ahead of the Edge Cookie context and the filters, as on Fastly. + +| Adapter | Named routes in the cycle | Inspection endpoints | Registration | +| ---------- | ----------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- | +| Axum | Every row of `named_routes()` but `/_ts/permissions`, `/_ts/permissions.json`, `/_ts/config`, `/_ts/config.json`, `/_ts/data` | `execute_named` answers them after the preparers | Unchanged. `named_route_handler` calls `execute_named` and `fallback_handler` calls `execute_fallback`, where both called `execute_handler` | +| Cloudflare | Discovery, `/verify-signature`, `/auction`, both page-bids paths (`GET` and `OPTIONS`), the four first-party paths | `make_handler`, after the preparers, as before | The same paths and methods, each handled by `named_route_handler` | +| Spin | The same routes as Cloudflare | Their own handlers, with no preparers, as before | The same paths and methods, each handled by `named_route_handler` | + +On Spin the discovery, signature verification and first-party routes and the page-bids preflight now run the preparers, because the cycle starts with them. Before this change they ran none. + +**Device classification.** `derive_device_signals` follows the Fastly adapter's function of the same name. A device module a registration supplies is shown the `User-Agent` and the `Cookie` header, and the built-in module the `User-Agent` alone, each in a module context carrying the request, that evidence, the settings and the services. Three things differ from the Fastly adapter. + +1. It runs where the adapter builds the Edge Cookie context, after the preparers, where the Fastly adapter runs it at the entry point before routing. A preparer that removes a reserved cookie therefore removes it before a device module sees it. +2. The registered module is the one the request's services carry (`RuntimeServices::device_module`), which `services_for_request` sets from the registry, and the module is handed those same services, so it reads the platform's key-value store when the services carry one. The Fastly entry point builds services of its own for the call. +3. These adapters supply no TLS or HTTP/2 signals, so `[device] module = "platform"` builds the built-in module, which classifies from the `User-Agent`, because `platform` adds the host's signals only where the platform supplies them. + +**Response effects.** A filter that lets the request continue can name response headers. The route's response carries them in its extensions as `RequestFilterEffects`, and `FinalizeResponseMiddleware` applies them after it writes the finalize headers, so a filter's header is written after an operator's `[response_headers]`, as on Fastly. `apply_terminal_response_effects` then enforces `no-store, private` on a response core marked `TerminalPrivateResponse` and runs `enforce_set_cookie_cache_privacy` and `enforce_uncacheable_cache_privacy`, the steps the Fastly entry point runs before it sends every response. + +**Configuration.** None. The filters a deployment's modules register and the module `[device]` selects now run on every adapter. + +**Consent and privacy.** Every filter is handed the permissions resolved for the request, as on Fastly. The 51Degrees filter sends the reader's `User-Agent` and `Sec-CH-UA` hints to the 51Degrees service for each page navigation an entry naming its middleware covers, whatever the permissions say, which is the behavior and the open question that `2026-10-14-51degrees-client-hints-design.md` describes for Fastly. DataDome's protection filter removes `x-ts-datadome-bypass` from every request it runs for, before its other checks. A device module a registration supplies sees the reader's cookies, as on Fastly. robots.txt is answered with an empty Edge Cookie context, so a crawler's cookies and consent signals are no longer read on these adapters. + +**Failure handling.** A filter that fails answers the request with the error, logged as `Failed to run integration request filters`, as on Fastly. An Edge Cookie context that cannot be read answers the request with the error before the filters run, which now covers every route in the cycle, including tsjs and the discovery, signature verification and first-party routes. The errors are a selected Edge Cookie module the request cannot build and a `Cookie` header that is not UTF-8, and startup refuses a selection the adapter can never build. A filter's outbound call is bounded only by the adapter's HTTP client, which applies no module's own timeout on these adapters. Axum's client allows 5 seconds to connect and 30 in all, Cloudflare leaves the call to the Workers runtime's limit, and Spin to the Spin runtime's default (`src/platform.rs` of each adapter). + +**Rollout.** A deployment whose modules register no filter, and that selects no device module, answers as before. It now reads the Edge Cookie context, with its geo lookup where `[geo]` selects a module, and classifies the device with the built-in module, on every route in the cycle, where it read the context for the auction, page-bids, module routes and the publisher alone. A deployment with `[shared.51degrees]` and a `[[serve]]` entry naming `51degrees.client_hints` now has the headers and the tag written on the pages the entry covers, and such a page keeps to the origin path and is sent `private, no-store`, because the filter leaves the hint names in the request's state. A deployment with DataDome's protection on now has its pages checked, and a challenge answers in place of the page. + +**Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, through the adapter's router built by its composition root, with a stub origin: + +- `a_filter_that_answers_stands_in_for_the_page`: the answer and its effect header are the response, and the origin is not asked. +- `a_filter_s_request_header_reaches_the_route_and_its_effect_is_written_last`: a module route sees the request header a filter added, and the filter's response header replaces the operator's header of the same name. +- `the_device_module_s_classification_reaches_the_module_call`: the module `[device]` selects classifies the request, shown its cookies, and a module route's call carries that classification. +- `robots_txt_is_answered_before_the_request_filters`: with a filter that answers every request, robots.txt is still the file, and the filter's header is absent. + +**Open questions.** None. + +## 3. What is not in this change + +- Creating and writing the Edge Cookie, and the resolve route, on these adapters, which the next pull request of the stack covers. +- Asset routes on these adapters, which the pull request after that covers. +- A timeout for a filter's outbound call on Axum, Cloudflare and Spin, which their HTTP clients do not offer. +- Moving the inspection endpoints ahead of the preparers on Axum and Cloudflare, where the Fastly adapter answers two of them before the preparers. + +## 4. Acceptance + +The four tests pass on each of the three adapters, and the existing tests of each adapter and the cross-adapter parity tests pass unchanged. + +## Revision record + +| Date | Change | +| ---------- | ------------ | +| 2026-10-17 | First draft. | From c460e8c10b95d3e367d1ecd7db160579638a3db3 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:16:14 +0100 Subject: [PATCH 141/192] The specs carry the design of the Edge Cookie on every adapter The design of the change that creates and writes the Edge Cookie on every adapter, with or without an identity graph, and serves the resolve route on each. --- ...-17-edge-cookie-on-every-adapter-design.md | 98 +++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md diff --git a/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md new file mode 100644 index 0000000000..4a759e8a4a --- /dev/null +++ b/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md @@ -0,0 +1,98 @@ +# Design Spec: The Edge Cookie is created and written on every adapter + +**Status:** Proposed, 2026-10-17. A pull request above the one that runs the request filters on every adapter, not yet raised, for Tech Lab review. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-10-17-request-filters-on-every-adapter-design.md`, `2026-10-15-cookies-by-path-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 +**Related PRs:** none yet +**Issues:** none +**Last updated:** 2026-10-18 + +> **Why this spec exists.** Only the Fastly adapter creates and writes the Edge Cookie and serves the resolve route. On Axum, Cloudflare and Spin an Edge Cookie module reads the cookie a request carries, but no page view creates one and no response writes one, so the stack's cookies-by-path change is never reached there. This change runs the Fastly adapter's Edge Cookie steps on those three adapters, which open no identity graph, and core writes a created Edge Cookie with or without a graph, so the cookie is written on every host, the graph is optional, and without one the graph's syncs are skipped. + +## 1. Problem + +The Fastly adapter runs the Edge Cookie lifecycle in four places (`crates/trusted-server-adapter-fastly/src/app.rs` and `src/lib.rs`). + +1. `build_ec_request_state` reads the request's Edge Cookie context with the device classification, takes the bot gate's answer from the classification (`DeviceSignals::looks_like_browser`), and keeps the identity graph, when `[ec] ec_store` names one, for a request that passes the gate. +2. `dispatch_fallback` creates an Edge Cookie for a real browser's page navigation (`EcContext::generate_if_needed`) before the publisher path runs. The integration proxy creates one for a navigation to a module route (`IntegrationRegistry::handle_proxy` in core). +3. Every response a route in the request cycle makes carries an `EcFinalizeState`, and the entry point runs `ec_finalize_response` on it once the response is finalized. That writes a created Edge Cookie, expires one on a withdrawal and reconciles the markers. +4. `POST /_ts/api/v1/ec/resolve` is a row of `NAMED_ROUTES`, answered by `handle_ec_resolve`. + +None of the four exists on Axum, Cloudflare or Spin. Spin's `named_fallback_paths` says the resolve route is left out because the identity graph is not wired there. + +Core writes a created Edge Cookie only with its identity-graph row. `ec_finalize_response` in `crates/trusted-server-core/src/ec/finalize.rs` skips the write when it is given no graph, so that on a deployment with a graph no browser carries an identifier the graph does not know. The three adapters open no graph, because the graph needs strongly consistent reads and conditional writes (`EcKvStore::key_exists` and `EcKvWriteMode::IfGenerationMatch` in `ec/kv_backend.rs`), which the platform key-value store these adapters can open does not offer (`get_bytes`, `put_bytes`, `put_bytes_with_ttl`, `delete` and `list_keys_page` in `edgezero_core::key_value_store`). Porting the Fastly steps alone would therefore create an Edge Cookie on these adapters and never write it. A Fastly service whose `[ec] ec_store` names no store writes none either, and the resolve route answers `204` with no cookie wherever it has no graph. + +## 2. Design + +**The Edge Cookie request state.** Each adapter's `build_ec_request_state` reads the context as before, with the device classification, and keeps the bot gate's answer beside it in an `EcRequestState`. A request the gate stops is logged as on Fastly. No identity graph is kept, because none is opened. + +**Creating the Edge Cookie.** `dispatch_fallback` calls `EcContext::generate_if_needed` for a real browser's page navigation (`is_navigation_request`) before the publisher path runs, with no graph, as the Fastly adapter does when it has none. The integration proxy creates one for a navigation as before, in core. + +**Writing it.** Every response a route in the request cycle makes carries an `EcFinalizeState`, holding the context and the request's services. These are the responses the Fastly adapter attaches it to, being the named routes but the inspection endpoints, the fallback's tsjs, module route and publisher answers, and a filter's answer. robots.txt carries none, as on Fastly. `FinalizeResponseMiddleware` takes the state off the response once it has written the finalize headers and runs `ec_finalize_response` with no graph, which applies a module's response headers, writes an Edge Cookie the request created, expires the cookie and its markers on a withdrawal, and reconciles the pull-sync marker. The request filters' response effects are applied after it, as on Fastly. The `sharedId` cookie the Fastly adapter carries to finalizing is read only by the graph's identifier writes, so it is not carried. + +**Core writes the cookie on every host.** `ec_finalize_response` writes a created Edge Cookie for a request that passed the bot gate and whose permissions allow the Edge Cookie (`EcContext::ec_allowed`), with or without an identity graph. With one, it records the request's identifiers on the cookie's row and writes the cookie once that row is authoritative, as before. Without one, it skips the graph steps and writes the cookie through `set_ec_cookie_on_response`, which takes the request's public prefix and, on a request addressed by path, the removed prefix as its `Path`, as core writes every cookie it owns. Core reads the bot gate from the device signals the adapter sets on the context (`EcContext::passed_bot_gate`, which a request with no device signals does not pass), because a missing graph no longer stops the cookie and so can no longer stand in for the gate. The Fastly adapter still opens no graph for a request the gate stops, so no graph row is written for it. + +| Step | Fastly | Axum, Cloudflare and Spin | +| ------------------------------------ | ------------------------------------------------------------------------------------------------- | -------------------------------------------------------- | +| Bot gate | `build_ec_request_state` | `build_ec_request_state` | +| Identity graph for the request | The `[ec] ec_store` graph, for a real browser | None | +| Creating on a page navigation | `dispatch_fallback`, for a real browser | `dispatch_fallback`, for a real browser | +| Finalizing | The entry point, after the finalize headers | `FinalizeResponseMiddleware`, after the finalize headers | +| Writing a created Edge Cookie | `ec_finalize_response`, once its graph row exists, or with no row where `ec_store` names no store | `ec_finalize_response`, with no row | +| Pull sync after the response is sent | When the graph and the partners allow it | Skipped | +| `POST /_ts/api/v1/ec/resolve` | A row of `NAMED_ROUTES` | A named route on each adapter, in the request cycle | + +**The resolve route.** Each adapter registers `POST /_ts/api/v1/ec/resolve` as a named route the way it registers its other named routes, so the request passes the preparers, the Edge Cookie context and the request filters, as on Fastly. It is answered by `handle_ec_resolve` with no graph, which answers as it does with one, so its refusals are the Fastly route's, being `403` for a missing or foreign `Origin` and `415` for another content type among them, and a client-side module's created Edge Cookie is set with its marker and no graph row. A request that did not pass the bot gate gets `204` with no cookie, as on Fastly. + +**One startup warning.** Where `[ec] ec_store` names an identity graph store, each adapter logs once, as it builds its state, that it opens no identity graph, so the Edge Cookie is still written but the data the graph would keep with it, such as partner IDs, is not. Cloudflare and Spin build their state for every request, so a flag keeps the line to one for each instance. + +**Configuration.** None. `[ec] ec_store` is accepted on these adapters as before, and logged once. + +**Consent and privacy.** A created Edge Cookie is written only where core's permission gate allows the Edge Cookie module and the request passed the bot gate, on every host, with a graph or without. A withdrawal expires the cookie and its markers, as on Fastly, and there is no graph row to tombstone. Without a graph no partner identifier is written against the Edge Cookie, and no pull sync, batch sync or identify call reads one, so the Edge Cookie stays a first-party identifier in the browser. + +**Failure handling.** A failure to create the Edge Cookie is logged and the page is served without one, as on Fastly. A partner registry that cannot be built from `[ec] partners` skips the finalize step with a logged error, as on Fastly. + +**Rollout.** A deployment that selects no Edge Cookie module is unchanged. One that selects a server-side module, such as `hmac`, now sets the Edge Cookie on a real browser's first page navigation where the permissions allow it, on these adapters and on a Fastly service with no `ec_store`, and that response is kept out of shared caches because it sets a cookie (`enforce_set_cookie_cache_privacy`). A returning browser keeps its cookie. A client-side module, such as `client_fixed`, gets its cookie from the resolve route on those hosts too. A crate outside core that finalizes the Edge Cookie or answers the resolve route sets the device signals on the context, or no created cookie is written. + +**Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, through the adapter's router, with a stub origin and a reader whose services place them in the US, where the sample permission rules grant the Edge Cookie with no signal, and `[geo] module = "platform"`, which leaves that location standing: + +- `a_first_page_view_gets_the_edge_cookie_under_the_public_prefix_and_path`: a page view addressed by path gets `ts-ec` with `Path=/publisher.example`, and the origin is asked at the continuing path. +- `a_returning_page_view_keeps_its_edge_cookie`: a page view addressed by host gets `ts-ec` under `/`, and the same page view carrying that cookie gets none. +- `a_bot_s_page_view_gets_no_edge_cookie`: a page view whose `User-Agent` names a crawler gets none. +- `the_resolve_route_answers`: a foreign `Origin` gets `403` with `Cache-Control: no-store` and the publisher's gets `204` with no cookie, the answer of a server-side module, which creates nothing from a posted value, and neither reaches the origin. +- `startup_warns_that_this_adapter_opens_no_identity_graph`: building the state with `[ec] ec_store` set logs the warning. + +The tests' `edge_cookie` helper also fails a response that writes the Edge Cookie more than once, so the first page view shows core alone writes it. + +In core: + +- `finalize_generated_ec_without_kv_writes_the_cookie_and_skips_the_graph` in `ec/finalize.rs`: a created Edge Cookie finalized with no graph gets the headers the same cookie gets with one, and no graph step runs, while with a graph the request's partner ID is recorded on the row. +- `a_generated_ec_is_written_only_for_a_request_that_passed_the_bot_gate` in `ec/finalize.rs`: a browser's created Edge Cookie is written, and a crawler's or that of a request with no device signals is not, with a graph and without. +- `resolve_answers_as_it_does_with_a_graph_when_there_is_none` in `ec/resolve.rs`: with no graph the route answers `200` with the Edge Cookie, its marker and `Cache-Control: no-store`, the headers it gives with a graph. +- `resolve_creates_nothing_for_a_request_that_did_not_pass_the_bot_gate` in `ec/resolve.rs`: a crawler, or a request with no device signals, gets `204` with no cookie and no row, with a graph and without. +- `the_bot_gate_passes_a_browser_and_stops_a_crawler_or_a_request_with_no_signals` in `ec/mod.rs`. + +In the Fastly adapter, under Viceroy, `with_no_ec_store_the_entry_point_writes_a_created_edge_cookie` in `src/app.rs`: with no `[ec] ec_store`, a browser's page view builds no graph, creates an Edge Cookie, and the entry point's finalize step writes it once. + +**Decisions.** Two questions this design left open, whether the resolve route should write a client-side module's cookie on an adapter that opens no graph and whether a Fastly service with no `ec_store` should write a created Edge Cookie, are answered yes by James Rosewell's decision of 17 October 2026, quoted here. + +> We have an EdgeCookie and then some data that is associated with it. One of those data items is the source information that is used to work out the permissions model. That includes permission-signal providers. Whether the sources for that permission state come from data in the EC values or from client side, or some other source doesn't matter. That's the principles. + +So the Edge Cookie is the identifier, the identity graph is one optional store of the data associated with it, the partner IDs among it, and the permission state comes from the permission model whatever its sources. A missing graph does not stop the cookie. + +## 3. What is not in this change + +- An identity graph on Axum, Cloudflare or Spin, which needs a strongly consistent store with conditional writes. +- `/_ts/api/v1/identify`, `/_ts/api/v1/batch-sync` and pull sync on those adapters, which need the graph, and the tester routes, which no other adapter registers. +- Asset routes on these adapters, which the next pull request of the stack covers. + +## 4. Acceptance + +The five tests pass on each of the three adapters, the core and Fastly tests above pass, and the existing tests of each adapter, core's Edge Cookie tests and the cross-adapter parity tests pass. + +## Revision record + +| Date | Change | +| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-17 | First draft. | +| 2026-10-18 | Core writes a created Edge Cookie with or without an identity graph, behind the bot gate it reads from the device signals, and the resolve route answers without a graph as it does with one, which answers both open questions as James Rosewell decided on 17 October 2026. | From 8b17eee83ed88698fb6b4d90837b55889027f386 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:16:16 +0100 Subject: [PATCH 142/192] The specs carry the design of asset routes on every adapter The design of the change that serves asset routes on the Axum, Cloudflare and Spin adapters as on Fastly. --- ...17-asset-routes-on-every-adapter-design.md | 65 +++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md diff --git a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md new file mode 100644 index 0000000000..dbc81ea696 --- /dev/null +++ b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md @@ -0,0 +1,65 @@ +# Design Spec: Asset routes are served on every adapter + +**Status:** Proposed, 2026-10-17. A pull request above the one that creates and writes the Edge Cookie on every adapter, not yet raised, for Tech Lab review. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-10-17-request-filters-on-every-adapter-design.md`, `2026-10-17-edge-cookie-on-every-adapter-design.md`, `2026-10-14-source-modules-design.md`, `2026-10-14-origin-signing-design.md`, `2026-10-15-asset-cache-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 +**Related PRs:** none yet +**Issues:** none +**Last updated:** 2026-10-17 + +> **Why this spec exists.** Only the Fastly adapter serves `[[proxy.asset_routes]]`. On Axum, Cloudflare and Spin a request under a route's prefix goes to the publisher's origin as an ordinary page request, so a route's own source, its signing and its held answers are never used there. This change dispatches asset routes on those three adapters as the Fastly adapter does. + +## 1. Problem + +`dispatch_fallback` in `crates/trusted-server-adapter-fastly/src/app.rs` checks a `GET` or `HEAD` against `Settings::asset_route_index_for_path` once robots.txt, the Edge Cookie context, the request filters, tsjs and the module routes have had their turn, and before the publisher's origin. `dispatch_asset_fallback` then asks the registry for the route's source (`IntegrationRegistry::route_source`) and answers with `proxy::handle_asset_proxy_request`, which locates the asset through the source module, signs the request where the source's `auth` table names a signer, looks for the answer among the copies its caches hold before it asks the origin, and keeps the origin's answer the way the publisher path keeps an asset. The response carries its `AssetProxyCachePolicy`, which the entry point applies after the finalize headers, and the request filters' effects, and no Edge Cookie state, because an asset response skips the Edge Cookie steps. + +No code in the Axum, Cloudflare or Spin adapter calls `handle_asset_proxy_request`. `docs/guide/api-reference.md` lists the route as Fastly's alone, and the Cloudflare HTTP client's note says these requests are not routed to it. + +## 2. Design + +**Dispatch.** Each adapter's `dispatch_fallback` checks a `GET` or `HEAD` against the asset routes where the Fastly adapter does, and `dispatch_asset_fallback` follows the Fastly function of that name: the route's source, `handle_asset_proxy_request`, the response's cache policy in its extensions, and the request filters' effects. An error answers with the error and the `no-store, private` policy, as on Fastly. No Edge Cookie state is attached, so the finalize middleware writes no Edge Cookie on an asset response. + +**Finalizing.** `FinalizeResponseMiddleware` takes the cache policy off the response and applies it after the finalize headers with the adapter's own shared cache header (`AssetProxyCachePolicy::apply_after_route_finalization`), before the Edge Cookie and the filters' effects, the order the Fastly entry point uses. + +**Held answers and the body.** `handle_asset_proxy_request` keeps and reads its answers through the request's loading caches and value cache, as on Fastly. Cloudflare and Spin rebuild their state for every request and have no value cache, so an answer is held across requests only by a loading cache module that keeps its copies in the platform's key-value store, when the services carry one. None of the three HTTP clients streams a response (`supports_streaming_responses`), so core asks for none and the asset body is held whole, where the Fastly adapter streams it. + +**Image Optimizer.** A route that enables `[proxy.asset_routes.image_optimizer]` sets Image Optimizer metadata on the request, which the Fastly adapter translates and the other three cannot. The Cloudflare and Spin HTTP clients already refuse such a request, as the `PlatformHttpRequest` contract asks of a client that cannot honor it, and the Axum client now refuses it too where it would have sent it unchanged. The request then answers with the error. + +| Adapter | Source and signing | Held answers | Body | Image Optimizer | +| ---------- | ------------------ | ------------------------------------------------------ | ---------- | ------------------------------- | +| Fastly | Yes | Loading caches and the value cache | Streamed | Translated | +| Axum | Yes | The loading caches of the process | Held whole | Refused by the client, an error | +| Cloudflare | Yes | A loading cache module's copies in the key-value store | Held whole | Refused by the client, an error | +| Spin | Yes | A loading cache module's copies in the key-value store | Held whole | Refused by the client, an error | + +**Configuration.** None. `[[proxy.asset_routes]]` now takes effect on every adapter. + +**Consent and privacy.** An asset route forwards only `ASSET_PROXY_FORWARD_HEADERS` to the route's origin, as on Fastly, so the reader's cookies are not sent there. Before this change the request went to the publisher's origin with the headers the publisher path forwards. + +**Failure handling.** A route whose source cannot be built, an origin that cannot be reached and a request the client refuses each answer with the error and `no-store, private`, as on Fastly. + +**Rollout.** A deployment with no asset routes is unchanged. One with asset routes now has their requests answered by each route's origin on Axum, Cloudflare and Spin, where the publisher's origin answered them, and a route that enables the Image Optimizer answers with an error there. + +**Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, with a stub origin and an asset route whose source signs with the stock `s3_sigv4` signer: + +- `a_request_under_an_asset_route_reaches_the_route_s_origin_signed`: the route's origin is asked, at the route's address, with an `AWS4-HMAC-SHA256` `Authorization` header, and its answer is the response. +- `a_request_outside_the_asset_routes_reaches_the_publisher_s_origin`: the publisher's origin is asked, unsigned. + +The Axum adapter's platform tests gain `a_request_with_image_optimizer_metadata_is_refused`. + +**Open questions.** None. + +## 3. What is not in this change + +- An Image Optimizer on Axum, Cloudflare or Spin. +- Streaming an asset body on those adapters, whose HTTP clients hold every response whole. + +## 4. Acceptance + +The tests pass on each of the three adapters, and the existing tests of each adapter and the cross-adapter parity tests pass. + +## Revision record + +| Date | Change | +| ---------- | ------------ | +| 2026-10-17 | First draft. | From cb2d14640d660bf4b88f46fc1a73c08be636306d Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 01:17:13 +0100 Subject: [PATCH 143/192] The Edge Cookie design states its decision as the design's own reasoning The decisions paragraph says why a missing graph does not stop the cookie in the design's own words: the Edge Cookie is the identifier, the identity graph is one store of the data associated with it, and the permission state comes from the permission model whatever its sources. --- ...2026-10-17-edge-cookie-on-every-adapter-design.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md index 4a759e8a4a..8064eac500 100644 --- a/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md @@ -74,9 +74,7 @@ In core: In the Fastly adapter, under Viceroy, `with_no_ec_store_the_entry_point_writes_a_created_edge_cookie` in `src/app.rs`: with no `[ec] ec_store`, a browser's page view builds no graph, creates an Edge Cookie, and the entry point's finalize step writes it once. -**Decisions.** Two questions this design left open, whether the resolve route should write a client-side module's cookie on an adapter that opens no graph and whether a Fastly service with no `ec_store` should write a created Edge Cookie, are answered yes by James Rosewell's decision of 17 October 2026, quoted here. - -> We have an EdgeCookie and then some data that is associated with it. One of those data items is the source information that is used to work out the permissions model. That includes permission-signal providers. Whether the sources for that permission state come from data in the EC values or from client side, or some other source doesn't matter. That's the principles. +**Decisions.** Two questions this design left open, whether the resolve route should write a client-side module's cookie on an adapter that opens no graph and whether a Fastly service with no `ec_store` should write a created Edge Cookie, are both answered yes. The Edge Cookie is the identifier, and the identity graph is one store of the data associated with it. The permission state comes from the permission model, whose sources may be values the graph holds, signals the client sends or any other source, so a missing graph does not stop the cookie. So the Edge Cookie is the identifier, the identity graph is one optional store of the data associated with it, the partner IDs among it, and the permission state comes from the permission model whatever its sources. A missing graph does not stop the cookie. @@ -92,7 +90,7 @@ The five tests pass on each of the three adapters, the core and Fastly tests abo ## Revision record -| Date | Change | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-17 | First draft. | -| 2026-10-18 | Core writes a created Edge Cookie with or without an identity graph, behind the bot gate it reads from the device signals, and the resolve route answers without a graph as it does with one, which answers both open questions as James Rosewell decided on 17 October 2026. | +| Date | Change | +| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-17 | First draft. | +| 2026-10-18 | Core writes a created Edge Cookie with or without an identity graph, behind the bot gate it reads from the device signals, and the resolve route answers without a graph as it does with one, which answers both open questions. | From e7e73c49555c14ad353d79fd43e13c53bbeab6e8 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 02:06:43 +0100 Subject: [PATCH 144/192] The designs end with their review logs The contributing guide asks a spec to end with its review log, one entry per round. Each design that had its review log ahead of its revision record now has the log last. No text changes. --- ...26-07-30-client-cycle-ec-resolve-design.md | 10 ++++----- ...integration-response-header-hook-design.md | 20 ++++++++--------- .../2026-07-30-permission-model-design.md | 10 ++++----- .../2026-07-30-pluggable-providers-design.md | 22 +++++++++---------- ...-08-27-integration-provider-seam-design.md | 10 ++++----- ...tration-off-the-publisher-domain-design.md | 10 ++++----- .../2026-10-12-ads-txt-sellers-design.md | 10 ++++----- .../2026-10-12-analytics-module-design.md | 10 ++++----- .../specs/2026-10-12-attestation-design.md | 10 ++++----- .../specs/2026-10-12-config-by-host-design.md | 10 ++++----- ...2026-10-12-consent-prompt-phases-design.md | 10 ++++----- .../2026-10-12-cookie-constant-design.md | 10 ++++----- .../specs/2026-10-12-device-signals-design.md | 10 ++++----- .../2026-10-12-diagnostic-headers-design.md | 10 ++++----- .../2026-10-12-google-tag-module-design.md | 10 ++++----- .../2026-10-12-inmobi-choice-module-design.md | 10 ++++----- .../2026-10-12-inspection-endpoints-design.md | 10 ++++----- .../specs/2026-10-12-loading-cache-design.md | 10 ++++----- ...iddleware-creates-a-missing-file-design.md | 10 ++++----- ...026-10-12-middleware-media-types-design.md | 10 ++++----- .../2026-10-12-middleware-prepare-design.md | 10 ++++----- .../specs/2026-10-12-module-context-design.md | 10 ++++----- .../2026-10-12-page-middleware-design.md | 10 ++++----- .../specs/2026-10-12-prefix-module-design.md | 10 ++++----- ...26-10-12-robots-txt-from-modules-design.md | 10 ++++----- ...2026-10-12-robots-txt-middleware-design.md | 12 +++++----- .../2026-10-12-shared-vendor-tables-design.md | 10 ++++----- .../specs/2026-10-12-viewport-sizes-design.md | 10 ++++----- .../2026-10-13-middleware-settings-design.md | 10 ++++----- .../2026-10-13-origin-redirect-design.md | 10 ++++----- ...-publisher-object-storage-origin-design.md | 10 ++++----- .../2026-10-13-request-addressing-design.md | 10 ++++----- ...026-10-14-51degrees-client-hints-design.md | 10 ++++----- .../2026-10-14-auction-protocol-design.md | 10 ++++----- .../specs/2026-10-14-backend-naming-design.md | 12 +++++----- .../2026-10-14-consent-decoders-design.md | 10 ++++----- .../2026-10-14-identifier-modules-design.md | 10 ++++----- .../2026-10-14-last-names-in-core-design.md | 10 ++++----- .../2026-10-14-no-modules-crate-design.md | 10 ++++----- .../specs/2026-10-14-origin-signing-design.md | 10 ++++----- ...2026-10-14-rsc-flight-middleware-design.md | 10 ++++----- .../specs/2026-10-14-source-modules-design.md | 10 ++++----- .../specs/2026-10-15-asset-cache-design.md | 10 ++++----- .../2026-10-15-cookies-by-path-design.md | 10 ++++----- .../2026-10-15-fastly-names-in-core-design.md | 10 ++++----- ...6-10-15-fastly-per-host-selector-design.md | 10 ++++----- .../2026-10-15-geo-module-request-design.md | 10 ++++----- ...-10-15-module-cache-declarations-design.md | 10 ++++----- ...026-10-16-settings-and-consumers-design.md | 14 ++++++------ 49 files changed, 260 insertions(+), 260 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index cc8c83fb86..7fd9e37d46 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -526,11 +526,6 @@ whether that is intended. | Demo gated by cargo feature or `#[cfg(test)]` | Compiled under `cfg(any(test, feature = "client-fixed-demo"))`, with startup rejection in the settings validator in a build with neither | Defense in depth | | Everything else in §3 (graph row, bounds, 409, no-store, full-declaration gate) | Implemented as specified. The graph row differs from organic creation as §3.3 says | (none) | -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/4-client-resolve` at `a70e9d20c`. Issues found, two of them blocking. The two were the statement that no production adapter offers a conditional storage write, which the Fastly store and the revival of withdrawn rows disprove, and the statement that resolve writes the graph row exactly as organic creation does, when resolve leaves a live row alone and revives a withdrawal tombstone. This revision states what the code does in both cases, adds the order of the checks, the configuration key, the adapters, the effects on other deployments, the demo module's cookie and the untested paths, and leaves five code questions under Open questions until the code changes. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes and the demo module's fix against `split/4-client-resolve` at `7ab530225`. Approved, none blocking. This revision cites RFC 6454 section 6.2 for the opaque origin, as the code does, and says the `500` is tested as the error that `error.rs` maps to `500`. - ## Revision record | Date | Change | @@ -540,3 +535,8 @@ whether that is intended. | 2026-10-15 | Fixes from review round 1. The conditional storage write claim, the graph write and its effect on withdrawal, the order of the checks and their answers, the configuration key, the adapters, the effects on other deployments, the marker and page script behavior, the demo module's cookie, the tests, the pull request and RFC section references, the open questions and the review log. | | 2026-10-16 | The demo module `client_fixed` accepts its own fixed word in `accepts_id`, so core recognizes its cookie, keeps the marker and tombstones the row on withdrawal (§5). | | 2026-10-16 | Review round 2, approved, with the RFC 6454 section and the 500 test wording corrected. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/4-client-resolve` at `a70e9d20c`. Issues found, two of them blocking. The two were the statement that no production adapter offers a conditional storage write, which the Fastly store and the revival of withdrawn rows disprove, and the statement that resolve writes the graph row exactly as organic creation does, when resolve leaves a live row alone and revives a withdrawal tombstone. This revision states what the code does in both cases, adds the order of the checks, the configuration key, the adapters, the effects on other deployments, the demo module's cookie and the untested paths, and leaves five code questions under Open questions until the code changes. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes and the demo module's fix against `split/4-client-resolve` at `7ab530225`. Approved, none blocking. This revision cites RFC 6454 section 6.2 for the opaque origin, as the code does, and says the `500` is tested as the error that `error.rs` maps to `500`. diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index e9434f921e..791d27d4a0 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -1601,6 +1601,16 @@ changes no code and decides none of them. 12. Whether §4a and the cache, `304` and `HEAD` rules move into specs of their own, so that this spec covers the headers-only API alone. +## Revision record + +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-08-25 | Disposition added (§7). | +| 2026-09-01 | Repeated lines of an admitted request header pass through as received in §4a.2 (§7). | +| 2026-10-06 | Module wording. | +| 2026-10-15 | Fixes from review round 1. The status and the closed pull request, a new section 1a on what the code does today, the cookie rule, the shared finalization code and the DataDome section name stated as the code has them, the reserved CDN cache fields taken from `EDGE_CACHE_HEADER_NAMES`, the three ids the id rule refuses, the template cache and the cache revision tuple marked as unreconciled or not built, the changes to shipped DataDome behavior in section 4a.4, the order of sections 4 and 4a, the sign-off rows, `must-revalidate` precedence, the duplicate-ID test in the done-when, the existing tests, the open questions and the review log. | +| 2026-10-16 | Fixes from review round 2. What the providers design now says (section 1a), the two core functions that every adapter's `apply_finalize_headers` calls (sections 1a and 2), the closed pull request that carried the documentation set (#1047, section 7), the revision dates and the review log. | + ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against @@ -1622,13 +1632,3 @@ changes no code and decides none of them. that one function is the only response-finalization code the adapters share, and the claim in section 7 that a pull request now closed (#1047) ships the documentation set. - -## Revision record - -| Date | Change | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-08-25 | Disposition added (§7). | -| 2026-09-01 | Repeated lines of an admitted request header pass through as received in §4a.2 (§7). | -| 2026-10-06 | Module wording. | -| 2026-10-15 | Fixes from review round 1. The status and the closed pull request, a new section 1a on what the code does today, the cookie rule, the shared finalization code and the DataDome section name stated as the code has them, the reserved CDN cache fields taken from `EDGE_CACHE_HEADER_NAMES`, the three ids the id rule refuses, the template cache and the cache revision tuple marked as unreconciled or not built, the changes to shipped DataDome behavior in section 4a.4, the order of sections 4 and 4a, the sign-off rows, `must-revalidate` precedence, the duplicate-ID test in the done-when, the existing tests, the open questions and the review log. | -| 2026-10-16 | Fixes from review round 2. What the providers design now says (section 1a), the two core functions that every adapter's `apply_finalize_headers` calls (sections 1a and 2), the closed pull request that carried the documentation set (#1047, section 7), the revision dates and the review log. | diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 8aaeff30c8..0e37b86676 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -1315,11 +1315,6 @@ include the changes that followed the first revision of this section on | Opt-out-beats-TCF pinning tests | Order tests in both directions, per-source tests, and selection tests (§8) | The tests pin the configured behavior rather than the retired rule | | No scheme read the PMP's answer | `mtm` reads the PMP's `__mtm_pref` cookie, answers for the eleven named Data Uses, and declares the versioned Model Terms for Marketing as the terms the data is available under (§4.6) | The PMP keeps its own copy of the answer and mirrors it into the cookie, which is what lets the server see the answer at all | -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/3-permissions` at `a028c00da`. Issues found, nine of them blocking. The nine were the claim that an unreadable record revokes everything ahead of the modules, four shipped modules where five ship, registration through an integration builder, a page state described with two members where it has four, a follow-up note that gave `ec_allowed` for two paths that use `ec_sharing_allowed`, the persisted consent, the resolve endpoint and the device module check that this row does not build, unknown rule fields that the parser does not reject, the Data Uses said to have no signal, and raw strings said to be kept for forwarding. This revision states what the code does in each case, adds the fifth module `mtm`, a table of what each adapter runs, an upgrade list and the privacy effects the round found, names the row that builds each item this row lacks, and leaves the code changes the round asks for under Open questions. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/3-permissions` at `a028c00da`. Approved, none blocking. One minor point stays, that the `split/` branches the spec names are the fork's, while the IAB Tech Lab drafts carry the same commits on `feature/` branches. - ## Revision record | Date | Change | @@ -1331,3 +1326,8 @@ include the changes that followed the first revision of this section on | 2026-10-07 | A module is named by its crate folder, so the selector reads `[permission-signal] modules`. | | 2026-10-15 | Fixes from review round 1. An unreadable record answered by its own module with nothing ahead of the modules, the fifth module `mtm` with its own section, registration through each adapter, the four members of the page state, the sharing pair on navigation and page-bids, the persisted consent, resolve endpoint and device module check named as built by other rows or not at all, what the policy parser does not reject, the Data Uses a signal reaches, the raw strings removed after assembly, a table of what each adapter runs, the upgrade list, the open questions and the review log. | | 2026-10-16 | Review round 2, approved with one minor point recorded in the Review log. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/3-permissions` at `a028c00da`. Issues found, nine of them blocking. The nine were the claim that an unreadable record revokes everything ahead of the modules, four shipped modules where five ship, registration through an integration builder, a page state described with two members where it has four, a follow-up note that gave `ec_allowed` for two paths that use `ec_sharing_allowed`, the persisted consent, the resolve endpoint and the device module check that this row does not build, unknown rule fields that the parser does not reject, the Data Uses said to have no signal, and raw strings said to be kept for forwarding. This revision states what the code does in each case, adds the fifth module `mtm`, a table of what each adapter runs, an upgrade list and the privacy effects the round found, names the row that builds each item this row lacks, and leaves the code changes the round asks for under Open questions. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/3-permissions` at `a028c00da`. Approved, none blocking. One minor point stays, that the `split/` branches the spec names are the fork's, while the IAB Tech Lab drafts carry the same commits on `feature/` branches. diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index d99ae23e24..d7fb92fe7f 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -869,6 +869,17 @@ either. a retirement condition for the bare reader that `module_owns_id` does not state. When the bare form stops being read is open. +## Revision record + +| Date | Change | +| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-07-31 | First draft. | +| 2026-08-25 | Revised against the code of the first two rows, which had landed as #1043 and #1044, with permission enforcement in #1045. | +| 2026-09-01 | The fallback baseline for a request with no resolvable country moved to the top node of the `rules:` tree in `permissions.yaml`, where `jurisdiction` is a per-node inherited attribute, and `[geo] default_country` was retired. The two `[consent]` applicability lists retire into the same tree (permission-model spec §3.2, §3.4 and §12). | +| 2026-10-06 | What an operator selects is a module, and the selector key reads `module` (§2.1). | +| 2026-10-15 | Fixes from review round 1. The page now describes the code on `split/2-device-geo` and names the row that builds each later feature, so the permission checks, the browser round trip, the registration hooks and the flip of the geo default are no longer described as built here. The geo default, the passphrase key, the Edge Cookie trait, the identity graph key, the parity of device and creation between adapters and the host signals capture are corrected. The selector key reads `module` or `modules` throughout §2.1. An adapter table, the missing settings, failure cases and rollout order, and a list of open questions are added. The history of earlier revisions moved into this table, and the former sections 12 and 13, which compared the page with issue #778 and with the first draft, are replaced by the open questions and this table. | +| 2026-10-16 | The host signal headers `x-ts-tls-ja4` and `x-ts-h2-fingerprint` are internal headers, so the Lockr and Permutive proxies drop them and the publisher origin request removes them, with a test for each path, and the open question about them is closed (§5, §10, §12). From review round 2, what each adapter answers when its startup check fails and that the list of implementations appears only in the log (§6). | + ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against @@ -894,14 +905,3 @@ either. blocking. This revision corrects what each adapter answers when its startup check fails, and says that the list of implementations appears only in the log. - -## Revision record - -| Date | Change | -| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-07-31 | First draft. | -| 2026-08-25 | Revised against the code of the first two rows, which had landed as #1043 and #1044, with permission enforcement in #1045. | -| 2026-09-01 | The fallback baseline for a request with no resolvable country moved to the top node of the `rules:` tree in `permissions.yaml`, where `jurisdiction` is a per-node inherited attribute, and `[geo] default_country` was retired. The two `[consent]` applicability lists retire into the same tree (permission-model spec §3.2, §3.4 and §12). | -| 2026-10-06 | What an operator selects is a module, and the selector key reads `module` (§2.1). | -| 2026-10-15 | Fixes from review round 1. The page now describes the code on `split/2-device-geo` and names the row that builds each later feature, so the permission checks, the browser round trip, the registration hooks and the flip of the geo default are no longer described as built here. The geo default, the passphrase key, the Edge Cookie trait, the identity graph key, the parity of device and creation between adapters and the host signals capture are corrected. The selector key reads `module` or `modules` throughout §2.1. An adapter table, the missing settings, failure cases and rollout order, and a list of open questions are added. The history of earlier revisions moved into this table, and the former sections 12 and 13, which compared the page with issue #778 and with the first draft, are replaced by the open questions and this table. | -| 2026-10-16 | The host signal headers `x-ts-tls-ja4` and `x-ts-h2-fingerprint` are internal headers, so the Lockr and Permutive proxies drop them and the publisher origin request removes them, with a test for each path, and the open question about them is closed (§5, §10, §12). From review round 2, what each adapter answers when its startup check fails and that the list of implementations appears only in the log (§6). | diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 0dcb5c6c5e..58ad6736b7 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -979,11 +979,6 @@ code. with the status `seeking vendor owner` until a vendor adopts them, given that §2 says Tech Lab engineering does not maintain vendor crates. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/7-integration-seam-impl` at `d3bbc32a5`. Issues found, five of them blocking. The five were that §3.6 and sign-off row 8 said no module is built into core when the HMAC module, the host-signal module and the built-in device module are, that §3.1 and sign-off row 11 said the built-in set is discovered at build time when two hand-written lists name it, that the §3.4 settings example failed startup four ways, that §5 listed every settings table as unchanged with no migration, compatibility or rollback text when the branch refuses the old tables, and that §3.7, acceptance 5 and sign-off row 12 described page middleware the branch lacks. This revision states what the code does in each case, names row 10 as the row that builds page middleware, corrects the settings example and adds the migration and rollback text, and leaves the move of the built-in modules into crates and the build-time discovery of the stock list under Open questions, because both need code changes. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/7-integration-seam-impl`. Approved, none blocking. This revision says the adapter lines cited in section 8 are where the two context builders are defined, and the device crate README's wording is left as a code follow-up. - ## Revision record | Date | Change | @@ -1002,3 +997,8 @@ code. | 2026-10-12 | One design document for each of the stack's pull requests above the seam, the 23 files dated 2026-10-12 in this folder, so this pull request carries every design document of the stack in one place. Each names the fork pull request that implements it and the Tech Lab issue it answers, and is written from both as they stand. | | 2026-10-15 | Fixes from review round 1, checked against `split/7-integration-seam-impl` at `d3bbc32a5`. Added a table of what row 7 delivers and what is left, the owners of the stock crates, what each selector means when unset, and the open questions (§10). Said what core builds in and what the adapters still inject (§3.6, sign-off row 8), that the stock list is written by hand (§3.1, sign-off row 11), the migration, compatibility and rollback of the settings change (§5, §7), that page middleware is the work of row 10 (§3.7, acceptance 5, sign-off row 12), and what the tests show (acceptance 1 to 3). Corrected the settings example (§3.4), the validation and hook functions, which sit on the builder (§3.3, §3.5), and the locations and counts (§3.5, §8 item 3). Added the facts on consent, failure, Cloudflare and Spin (§3.5, §3.6, §8 item 16) and the review log. Wrote §1 in the past tense and put the revision record in date order as one table. | | 2026-10-16 | Review round 2, approved, with the section 8 line citations described as where the builders are defined. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/7-integration-seam-impl` at `d3bbc32a5`. Issues found, five of them blocking. The five were that §3.6 and sign-off row 8 said no module is built into core when the HMAC module, the host-signal module and the built-in device module are, that §3.1 and sign-off row 11 said the built-in set is discovered at build time when two hand-written lists name it, that the §3.4 settings example failed startup four ways, that §5 listed every settings table as unchanged with no migration, compatibility or rollback text when the branch refuses the old tables, and that §3.7, acceptance 5 and sign-off row 12 described page middleware the branch lacks. This revision states what the code does in each case, names row 10 as the row that builds page middleware, corrects the settings example and adds the migration and rollback text, and leaves the move of the built-in modules into crates and the build-time discovery of the stock list under Open questions, because both need code changes. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/7-integration-seam-impl`. Approved, none blocking. This revision says the adapter lines cited in section 8 are where the two context builders are defined, and the device crate README's wording is left as a code follow-up. diff --git a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md index 6fa9cf8680..19d96373e4 100644 --- a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md +++ b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md @@ -125,11 +125,6 @@ One question is open, about the guides, and this spec does not decide it. 1. Which order should the guides give for the switch and for a rollback? Section 4 says what each build does with the other's configuration, and nothing in the code or the guides gives a sequence. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/13-removals` at `f735e0f85`. Issues found, four of them blocking. The four were that the table named `GET /_ts/data` as the replacement for `GET /_ts/admin/eids` when that page reads only the record of the caller's own `ts-ec` cookie, that the upgrade note said a `[[handlers]]` entry could gate `GET /_ts/debug/ja4` when the Fastly entry point answers that path before any middleware, that the spec said the commands tell a `404` from other failures without saying that only the test fake is known to print the wording they look for, and that the header named closed pull requests and a count of 19 checks where the open pull request shows 25. This revision states what the code does at that head for the table, the commands and their flags, the closed paths, the effects on a publisher's own site, the upgrade, the switch and the rollback, the tests and the acceptance, and leaves seven questions about the code under Open questions until the code changes, being the wording of a `404`, the replacing of a secret, the attestation endpoint beneath `/_ts/admin`, the missing tests, four stale texts, the order of the preparers on Axum and the order of the switch. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/13-removals` at `f735e0f85`. Approved, none blocking. This revision adds the changelog entry that still says only the removed key rotation routes read the two store ids to open question 5, says that the sandbox answer in the Fastly entry point exists only in a build with the `reusable-sandbox` feature, and confirms the check counts of the row 13 pull request and the fork's. - ## Revision record | Date | Change | @@ -139,3 +134,8 @@ One question is open, about the guides, and this spec does not decide it. | 2026-10-16 | Fixes from review round 2. Open question 5 names the changelog entry on the two store id settings and the checks on row 12 that also read them, the sandbox answer is stated to exist only in a build with the `reusable-sandbox` feature, and the check counts of 25 on the row 13 pull request and 19 on the fork's were read again and stand. | | 2026-10-16 | Fixes from code review. The test fake answers with the 404 text the CLI prints and refuses to replace a secret (`11bf7b76d`), an attestation endpoint beneath `/_ts/admin` is refused (`df0fc18a0`), Axum answers a closed path before the preparers (`23bde99bc`), every adapter test sends the purge path and Fastly's the double-encoded paths (`18aba165c`), and three comments and two changelog entries are corrected (`e75e307de`). Open questions 1 to 6 are answered. | | 2026-10-17 | For IABTechLab/trusted-server#1149, the development files and the Viceroy template no longer name `opid_store` (`1666c38e3`), and a test checks that every store a development file names is declared (`e110aa025`). | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/13-removals` at `f735e0f85`. Issues found, four of them blocking. The four were that the table named `GET /_ts/data` as the replacement for `GET /_ts/admin/eids` when that page reads only the record of the caller's own `ts-ec` cookie, that the upgrade note said a `[[handlers]]` entry could gate `GET /_ts/debug/ja4` when the Fastly entry point answers that path before any middleware, that the spec said the commands tell a `404` from other failures without saying that only the test fake is known to print the wording they look for, and that the header named closed pull requests and a count of 19 checks where the open pull request shows 25. This revision states what the code does at that head for the table, the commands and their flags, the closed paths, the effects on a publisher's own site, the upgrade, the switch and the rollback, the tests and the acceptance, and leaves seven questions about the code under Open questions until the code changes, being the wording of a `404`, the replacing of a secret, the attestation endpoint beneath `/_ts/admin`, the missing tests, four stale texts, the order of the preparers on Axum and the order of the switch. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/13-removals` at `f735e0f85`. Approved, none blocking. This revision adds the changelog entry that still says only the removed key rotation routes read the two store ids to open question 5, says that the sandbox answer in the Fastly entry point exists only in a build with the `reusable-sandbox` feature, and confirms the check counts of the row 13 pull request and the fork's. diff --git a/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md b/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md index 60880eb81f..579f5cd2c2 100644 --- a/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md +++ b/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md @@ -125,11 +125,6 @@ Six things have no test. 5. A selected module with no `lines`. An empty list has a test, which calls `validate` and never `register`, and a table with no `lines` has none. 6. `middleware_for` leaving the module out of the page entry `ts audit` writes. The test of `middleware_for` never hands the function this module's id. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/20-ads-txt` at `de85e19c0`. Issues found, three of them blocking. The three were that the spec said the origin's file is read as it is and that an origin with no file gets the origin's answer, when core replaces the origin's host name in the file and the module writes its lines into an answer with a `text/plain` body of any status, that the spec named closed pull requests and a bare number that belongs to an unrelated pull request, and that the acceptance table left out variable names in either case and a domain with a space while the closing paragraph cited records neither pull request description holds. This revision states what the code does at that head for the origin's answers, the entry's path, the configuration, a reader's data, the adapters, failure and rollout, replaces the closing claims with what the pull request description records, names the rows that add the made file and the `iabtl` names, and lists seven questions that wait for code changes. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/20-ads-txt` at `de85e19c0`. Approved, none blocking. This revision says that the two earlier pull requests were closed without being merged, says definitely when the module's final call carries no bytes, and says the empty list test calls `validate` alone. - ## Revision record | Date | Change | @@ -137,3 +132,8 @@ Six things have no test. | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The header and the related specs, the problem statement, what the module writes, what core does to the file first, each kind of origin answer, a missing file, the entry's path, the configuration, consent and privacy, the adapters, failure, rollout and how soon a crawler sees a change, the rows that hold what this change leaves out, the open questions, the acceptance table and claims, the tests and the cases with none, and the review log. | | 2026-10-16 | Fixes from review round 2. The two earlier pull requests are named as closed without being merged, open question 7 says when the module's final call carries no bytes and why no test reaches that call, and the tests table says the empty list test calls `validate` alone. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/20-ads-txt` at `de85e19c0`. Issues found, three of them blocking. The three were that the spec said the origin's file is read as it is and that an origin with no file gets the origin's answer, when core replaces the origin's host name in the file and the module writes its lines into an answer with a `text/plain` body of any status, that the spec named closed pull requests and a bare number that belongs to an unrelated pull request, and that the acceptance table left out variable names in either case and a domain with a space while the closing paragraph cited records neither pull request description holds. This revision states what the code does at that head for the origin's answers, the entry's path, the configuration, a reader's data, the adapters, failure and rollout, replaces the closing claims with what the pull request description records, names the rows that add the made file and the `iabtl` names, and lists seven questions that wait for code changes. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/20-ads-txt` at `de85e19c0`. Approved, none blocking. This revision says that the two earlier pull requests were closed without being merged, says definitely when the module's final call carries no bytes, and says the empty list test calls `validate` alone. diff --git a/docs/superpowers/specs/2026-10-12-analytics-module-design.md b/docs/superpowers/specs/2026-10-12-analytics-module-design.md index 045ad4b0d0..da7fd6f373 100644 --- a/docs/superpowers/specs/2026-10-12-analytics-module-design.md +++ b/docs/superpowers/specs/2026-10-12-analytics-module-design.md @@ -155,11 +155,6 @@ These are for the reviewer. This revision records what the code does and decides 3. Is it intended that a refused `[analytics]` setting leaves the adapter answering every request with its startup error, when the telemetry guide says that a failure to construct or start telemetry must not change the customer response? 4. Should the message for a section that selects no module offer `module` alone? It says that a section names what runs with `module` or `modules`, and `[analytics]` refuses `modules` as an unknown key (`settings.rs`, lines 2889 to 2892). -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/16-analytics` at `7b19336ec`. Issues found, three of them blocking. The three were that the spec named an accessor, `Settings::telemetry()`, that does not exist, that the first two messages in its refusal table were the wording from before the rename, with the no-module message said to list modules when it lists none, and that the spec credited the refusal of an empty or absent token reference at deploy to the declared secret field, when the function that runs as the settings load refuses it first. This revision states what the code does at that head for the accessor, the messages, the checks at load and at deploy, the single destination, the settings and their defaults, the adapters, what a row holds and when rows are sent, a refused setting, the upgrade order, the tests and the acceptance, and names row 46 as the row that moves the module into a crate, while five open questions wait for code changes, being the adapters that send nothing, the consent gate and the privacy section for rows, the missing test of a stored configuration, the refused setting that leaves the adapter on its startup error and the message that offers `modules`. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/16-analytics` at `7b19336ec`. Issues found, none blocking. This revision cites line 75 for `MeasureAdPerformance`, says that the test of an unselected table checks only the error text, states the claims about rows 46 and 50 as the heads of those rows show them, and says that the June design is in IAB Tech Lab's main. - ## Revision record | Date | Change | @@ -169,3 +164,8 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-16 | Code fixes from review round 1, findings C1 and C2. Rows are built only for a reader whose permissions include `MeasureAdPerformance`, and the Axum, Cloudflare and Spin adapters warn at startup when `[analytics]` selects a module. Open questions 1 and 2 leave the list, apart from the paragraph on `browser_family`, and the tests for both fixes are added. | | 2026-10-16 | Corrections from review round 2. The line cited for `MeasureAdPerformance`, what the test of an unselected table checks, the claims about rows 46 and 50, that IAB Tech Lab's main holds the June design, and the review log. | | 2026-10-17 | For IABTechLab/trusted-server#1148, the telemetry guide's table of the telemetry each adapter sends and a line in the Axum, Cloudflare and Spin guides (`99abf9f4f`), and the Fastly test that each request carries the sink `[analytics]` selects (`b1a47b8fc`). | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/16-analytics` at `7b19336ec`. Issues found, three of them blocking. The three were that the spec named an accessor, `Settings::telemetry()`, that does not exist, that the first two messages in its refusal table were the wording from before the rename, with the no-module message said to list modules when it lists none, and that the spec credited the refusal of an empty or absent token reference at deploy to the declared secret field, when the function that runs as the settings load refuses it first. This revision states what the code does at that head for the accessor, the messages, the checks at load and at deploy, the single destination, the settings and their defaults, the adapters, what a row holds and when rows are sent, a refused setting, the upgrade order, the tests and the acceptance, and names row 46 as the row that moves the module into a crate, while five open questions wait for code changes, being the adapters that send nothing, the consent gate and the privacy section for rows, the missing test of a stored configuration, the refused setting that leaves the adapter on its startup error and the message that offers `modules`. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/16-analytics` at `7b19336ec`. Issues found, none blocking. This revision cites line 75 for `MeasureAdPerformance`, says that the test of an unselected table checks only the error text, states the claims about rows 46 and 50 as the heads of those rows show them, and says that the June design is in IAB Tech Lab's main. diff --git a/docs/superpowers/specs/2026-10-12-attestation-design.md b/docs/superpowers/specs/2026-10-12-attestation-design.md index 4a32d3ae34..07c3e9c80c 100644 --- a/docs/superpowers/specs/2026-10-12-attestation-design.md +++ b/docs/superpowers/specs/2026-10-12-attestation-design.md @@ -203,11 +203,6 @@ Test builds carry no key, so no router test sees signed evidence, and only the h **What no test covers.** The paths listed in section 5, which Open question 2 asks about, and the answers whose headers item 9 says no test checks. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/12-attestation` at `59e09b6cd`. Issues found, four of them blocking. The four were that the `build` fields left out `build.tag`, that the spec said every adapter reports `staging` or `unknown` when Axum, Cloudflare and Spin sign `staging: false`, that the spec counted 37 tests where the module has 40, and that the header and the acceptance named pull requests that have closed and a count of 19 checks where the open pull request shows 25. This revision states what the code does at that head for the evidence, the build inputs, the configuration, the adapters, the answers, the failures, the compatibility of the section, the privacy of the answers, the tests and the acceptance, and leaves eight questions about the code under Open questions until the code changes, being what `staging` says off Fastly, the request preparers on Cloudflare, the build's check of a key, the clock, the missing tests, how a verifier gets the public keys, the rule for new evidence fields and the order of `run` and `tag` in the guide. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/12-attestation` at `59e09b6cd`. Issues found, none blocking. This revision says that the request address's host also stands in for the `Host` header when the header is not readable text, names the scheme headers under "Consent and privacy" and what they change, and names the tests behind Acceptance 4 and 9 and what no test checks. - ## Revision record | Date | Change | @@ -217,3 +212,8 @@ Test builds carry no key, so no router test sees signed evidence, and only the h | 2026-10-16 | Fixes from review round 2. The `host` fallback when the `Host` header is not readable text, the scheme headers and what they change under "Consent and privacy", the tests named in Acceptance 4 and 9 and what no test checks there, and the review log. | | 2026-10-16 | Three fixes in the code, at `694926002`. The handler answers `503` and signs nothing when the clock cannot be read, the build reads every key in the schedule with the function that signs and stops at one that cannot sign, and Cloudflare answers the attestation routes without the request preparers, each with a test. Open questions 2, 3 and 4 of the previous revision leave the list, which is renumbered. | | 2026-10-17 | The guide says how the key schedule reaches a Cloudflare build and a Spin build, and that the file is named by its absolute path (`e5e7fbf88`). | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/12-attestation` at `59e09b6cd`. Issues found, four of them blocking. The four were that the `build` fields left out `build.tag`, that the spec said every adapter reports `staging` or `unknown` when Axum, Cloudflare and Spin sign `staging: false`, that the spec counted 37 tests where the module has 40, and that the header and the acceptance named pull requests that have closed and a count of 19 checks where the open pull request shows 25. This revision states what the code does at that head for the evidence, the build inputs, the configuration, the adapters, the answers, the failures, the compatibility of the section, the privacy of the answers, the tests and the acceptance, and leaves eight questions about the code under Open questions until the code changes, being what `staging` says off Fastly, the request preparers on Cloudflare, the build's check of a key, the clock, the missing tests, how a verifier gets the public keys, the rule for new evidence fields and the order of `run` and `tag` in the guide. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/12-attestation` at `59e09b6cd`. Issues found, none blocking. This revision says that the request address's host also stands in for the `Host` header when the header is not readable text, names the scheme headers under "Consent and privacy" and what they change, and names the tests behind Acceptance 4 and 9 and what no test checks. diff --git a/docs/superpowers/specs/2026-10-12-config-by-host-design.md b/docs/superpowers/specs/2026-10-12-config-by-host-design.md index de09b4f018..fd5c2b65fc 100644 --- a/docs/superpowers/specs/2026-10-12-config-by-host-design.md +++ b/docs/superpowers/specs/2026-10-12-config-by-host-design.md @@ -173,11 +173,6 @@ The behaviours below are stated in the sections above. A change to the code woul 7. **The host of a request made over HTTP/2.** HTTP/2 carries the host in an `:authority` field and not in a `Host` header. The selection reads the `Host` header only, and nothing in this row shows that Fastly fills `Host` from the authority, because the harness sends plain HTTP/1.1 requests. Where is that behaviour shown, or does it need a test of its own? 8. **A selector that misspells the placeholder.** A selector such as `{Host}`, `{hostname}` or `{host` is read as a fixed key for every host, and nothing refuses such a selector. Should a selector like that be refused when it is read? `2026-10-15-fastly-per-host-selector-design.md` (section 2.4) refuses a selector like that as malformed and answers `500`. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/11-config-by-host` at `b8314cb52`. Issues found, 3 of them blocking. The header pointed at closed pull requests and called the fork pull request ready for review, the risk table put three header cells over two-cell rows, and the text said there is no fallback and that a selector that cannot be read fails the request, which is false when the runtime-env store cannot be opened. This revision points the header at the open draft pull request, repairs the table, states the fallback and the removal it needs, and adds the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests, whilst the code behaviours the review found wait for code changes and sit under Open questions. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/11-config-by-host` at `b8314cb52`. Approved, none blocking. This revision names, at the pointer and where this spec describes them, the sections of `2026-10-15-fastly-per-host-selector-design.md` that replace the fallback text (section 2.4) and the push and selector-writing steps (sections 2.6 and 2.8). - ## Revision record | Date | Change | @@ -185,3 +180,8 @@ The behaviours below are stated in the sections above. A change to the code woul | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Round 1 fixes. The header points at the open draft pull request, the risk table is repaired, the fallback to the logical store ID is stated, the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests are added, and the code findings sit under Open questions. The selector store points at the Fastly per-host selector design. | | 2026-10-16 | Round 2 fixes. The pointer to the Fastly per-host selector design names section 2.4, whose `500` answers replace the fallback to the logical store ID, and sections 2.6 and 2.8, which replace the push and selector-writing steps. The same sections are named where this spec describes the fallback and the steps for moving a service. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/11-config-by-host` at `b8314cb52`. Issues found, 3 of them blocking. The header pointed at closed pull requests and called the fork pull request ready for review, the risk table put three header cells over two-cell rows, and the text said there is no fallback and that a selector that cannot be read fails the request, which is false when the runtime-env store cannot be opened. This revision points the header at the open draft pull request, repairs the table, states the fallback and the removal it needs, and adds the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests, whilst the code behaviours the review found wait for code changes and sit under Open questions. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/11-config-by-host` at `b8314cb52`. Approved, none blocking. This revision names, at the pointer and where this spec describes them, the sections of `2026-10-15-fastly-per-host-selector-design.md` that replace the fallback text (section 2.4) and the push and selector-writing steps (sections 2.6 and 2.8). diff --git a/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md b/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md index 50150f8e93..c90145e85a 100644 --- a/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md +++ b/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md @@ -84,11 +84,6 @@ These are for the reviewer. Line numbers are those of `938be5d3b`. 3. Three places in the guides still describe the fetch phase alone or name a tag that is not written ahead of the prompt. `docs/guide/integrations/didomi.md` (lines 82 and 83) and `docs/guide/integrations/sourcepoint.md` (line 52) say that the middleware runs on the pages a `[[fetch]]` entry names. `docs/guide/integrations/inmobi.md` (lines 67 and 68) gives Google Tag Manager as the vendor's tag that the prompt is named ahead of, where Tag Manager writes no head markup (section 3). 4. Should each middleware say whether it writes markup ahead of the script bundle, so that the InMobi module's refusal of a prompt in `[[serve]]` spares a `[[fetch]]` entry whose middleware write none there, such as `tag.google-tag-manager`? Nothing says so today, so the module refuses any middleware that the `[[fetch]]` entry for the page names (section 3). -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/21-cmp-phases` at `938be5d3b`. Issues found, four of them blocking. The blocking issues were that the spec said nothing of a prompt in the serve phase being written after the vendor tags the fetch phase writes, that it said one stored page could serve readers with and without the prompt when neither an entry nor the three modules tell readers apart, that its last paragraph claimed a run of each commit and deliberate faults that no description or commit message records, and that it said a serve entry was refused when the settings load when the refusal comes when the service starts. This revision states where a serve prompt lands in the head, what the serve phase gives and costs, when each refusal comes, what a wrong order of rollout does, what each adapter runs and what the description records as checked, while the missing refusal and test for the order, a prompt named in both phases, the missing serve tests, the weaker stock list test and four stale places in the guides wait for code changes as open questions. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/21-cmp-phases` at `938be5d3b`. Approved, none blocking. This revision narrows where a serve prompt lands to after the fetch markup ahead of the bundle, names the route a page that is not stored takes through the serve entry, confirms the 25 checks on the pull request and adds the Tag Manager example to the open question on the guides. - ## Revision record | Date | Change | @@ -96,3 +91,8 @@ These are for the reviewer. Line numbers are those of `938be5d3b`. | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The header and the pull requests it names, when each refusal comes, what the serve phase gives and costs, which readers get the output, where a prompt lands in the head and what that does to the consent order, what this amends in the InMobi design, failure, rollout and the adapters, Osano, the stock list's weaker test and the row that removes the stock list, what is recorded as checked and what no test covers, the open questions and the review log. | | 2026-10-16 | The InMobi module refuses a prompt in `[[serve]]` for a page whose `[[fetch]]` entry names another middleware and a prompt named for one page in both phases, with tests that drive a page through both phases, and each of the three crates runs its middleware from a `[[serve]]` entry through the registry and a page, which settles the open questions on the order, on both phases and on the serve tests (sections 3, 4, 5, 6 and 7). The configuration and InMobi guides describe the refusals. Fixes from review round 2, being where a serve prompt lands, the route a page that is not stored takes, the checks confirmed and the Tag Manager example in the open questions. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/21-cmp-phases` at `938be5d3b`. Issues found, four of them blocking. The blocking issues were that the spec said nothing of a prompt in the serve phase being written after the vendor tags the fetch phase writes, that it said one stored page could serve readers with and without the prompt when neither an entry nor the three modules tell readers apart, that its last paragraph claimed a run of each commit and deliberate faults that no description or commit message records, and that it said a serve entry was refused when the settings load when the refusal comes when the service starts. This revision states where a serve prompt lands in the head, what the serve phase gives and costs, when each refusal comes, what a wrong order of rollout does, what each adapter runs and what the description records as checked, while the missing refusal and test for the order, a prompt named in both phases, the missing serve tests, the weaker stock list test and four stale places in the guides wait for code changes as open questions. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/21-cmp-phases` at `938be5d3b`. Approved, none blocking. This revision narrows where a serve prompt lands to after the fetch markup ahead of the bundle, names the route a page that is not stored takes through the serve entry, confirms the 25 checks on the pull request and adds the Tag Manager example to the open question on the guides. diff --git a/docs/superpowers/specs/2026-10-12-cookie-constant-design.md b/docs/superpowers/specs/2026-10-12-cookie-constant-design.md index a3fa250a13..01d1896197 100644 --- a/docs/superpowers/specs/2026-10-12-cookie-constant-design.md +++ b/docs/superpowers/specs/2026-10-12-cookie-constant-design.md @@ -94,11 +94,6 @@ git diff 7b19336ec...af8af38f5 | grep __mtm_pref None remain. The four questions of the round 1 revision, being the test of the cookie's name, the README naming the cookie, the "two words" comment and the cookie reaching Prebid Server in `openrtb_only` mode, are fixed as section 2 says. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/17-cookie-constant` at `af8af38f5`. Issues found, three of them blocking. The blocking issues were that the spec said the constant arrived in the module documentation set pull request when the permission model pull request added it, that the header sent reviewers to pull requests that are now closed and never named the open one, and that section 3 implied that a deployment's settings can select the change when no setting does. This revision names the pull requests that exist now, says where the constant arrived, lists the three files and what each changes, states that no setting is involved and that every deployment behaves as before, and adds the adapters, the names left in core, the acceptance checks and the untested paths, while four open questions wait for code changes, being the missing test of the cookie's name, the README's mention of the cookie, one stale comment and the cookie reaching Prebid Server. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/17-cookie-constant` at `af8af38f5`. Approved, none blocking. This revision names IABTechLab/trusted-server#1304, the earlier pull request on this repository, as closed without being merged, and says that the EIDs cookie's constant is renamed `COOKIE_EIDS` and stays in core's constants from row 27 until row 41 deletes it, where the earlier text said that row 41 alone moves the name out of core. - ## Revision record | Date | Change | @@ -107,3 +102,8 @@ None remain. The four questions of the round 1 revision, being the test of the c | 2026-10-15 | Fixes from review round 1. The header and the pull requests it names, where the constant arrived, the three files and what each changes, the reworded comment, why a constant and not a setting, what changes for a deployment and for code, the adapters, the names left in core, the acceptance checks, the untested paths, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. The earlier pull request on this repository, closed without being merged, named in the header, the rows at which the EIDs cookie's constant is renamed and then deleted from core (section 4), and the review log. | | 2026-10-16 | Fixes from code review. `__mtm_pref` is withheld from Prebid Server in `openrtb_only` mode (`389903405`), a test checks the cookie's name against the PMP's `Set-Cookie` strings (`71e039d4c`), core's README no longer names the cookie, and the comment on `valid_signal` and the Permission Signals guide say three words (`f8d0ccfe3`). | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/17-cookie-constant` at `af8af38f5`. Issues found, three of them blocking. The blocking issues were that the spec said the constant arrived in the module documentation set pull request when the permission model pull request added it, that the header sent reviewers to pull requests that are now closed and never named the open one, and that section 3 implied that a deployment's settings can select the change when no setting does. This revision names the pull requests that exist now, says where the constant arrived, lists the three files and what each changes, states that no setting is involved and that every deployment behaves as before, and adds the adapters, the names left in core, the acceptance checks and the untested paths, while four open questions wait for code changes, being the missing test of the cookie's name, the README's mention of the cookie, one stale comment and the cookie reaching Prebid Server. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/17-cookie-constant` at `af8af38f5`. Approved, none blocking. This revision names IABTechLab/trusted-server#1304, the earlier pull request on this repository, as closed without being merged, and says that the EIDs cookie's constant is renamed `COOKIE_EIDS` and stays in core's constants from row 27 until row 41 deletes it, where the earlier text said that row 41 alone moves the name out of core. diff --git a/docs/superpowers/specs/2026-10-12-device-signals-design.md b/docs/superpowers/specs/2026-10-12-device-signals-design.md index 3b0f6060d4..b3aa77cc27 100644 --- a/docs/superpowers/specs/2026-10-12-device-signals-design.md +++ b/docs/superpowers/specs/2026-10-12-device-signals-design.md @@ -102,11 +102,6 @@ These are for the reviewer. This revision records what the code does and decides 1. **Where the score belongs.** Section 2 keeps the signals apart from the attributes by what each answers and by whether core stores each. The score answers a question about a person and not about a browser, and core stores the score nowhere, so the score matches the attributes on both counts, yet the score sits in the signals that `detect` returns. Neither the code nor IABTechLab/trusted-server#1286 says why. What is the reason for placing the score with the signals? 2. **The changelog.** The changelog lists the change under Added and names `DeviceSignals::human_confidence` and `DeviceModule::advertising_attributes`, but it does not name `DeviceInfo::attributes` and gives no breaking-change note for the two new public fields that stop struct literals compiling (`CHANGELOG.md`, line 90). Should it? -## Review log - -1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/28-device-signals` at `b680d0780`. Issues found, five of them blocking. The blocking issues were that the spec described a device type name mapping that the code no longer has, that the spec named two places where core asks the module when the code has three, that the spec's acceptance section claimed a proof by the Axum test and a per-commit record that neither pull request holds, that the spec said the module is handed the call that `detect` is handed and left out that the module sees every header and that the attributes go under no permission check, and that the spec's header pointed at closed pull requests and said Proposed. This revision states what the code does for each, adds the consent and privacy, deployment, adapter and untested-path sections, and leaves seven questions in section 7 for code changes, being the missing test of the three points, the unchecked values, the missing time limit, what the call carries, two comments, where the score belongs, and the guide and the changelog. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/28-device-signals` at `b680d0780`. Issues found, two of them blocking. This revision says that the integration guide describes only the attributes, that the changelog names the score and `DeviceModule::advertising_attributes` but not `DeviceInfo::attributes` and gives no breaking-change note, and that no production code reads the score although two tests do, and it confirms the pull request facts in the header and the acceptance. - ## Revision record | Date | Change | @@ -115,3 +110,8 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-16 | Fixes from review round 1. The header and the pull requests the header names, the removed type name mapping, the three places core asks, what the call carries, the consent and privacy section, what changes for a deployment and for code, the adapters, the acceptance without the process claims, the untested paths, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. What the integration guide describes, what the changelog entry names and leaves out, and the statements that no code reads the score, which now say no production code does. The pull request facts in the header and the acceptance were read against the pull requests and stand. | | 2026-10-16 | Code fixes in the five commits ending at `4badbdaf8`, and open questions 1 to 5 and the guide half of question 7 moved into the body. The call carries the permissions resolved for the request, a backend the module registers in the call has its timers capped at the auction's timeout, core drops device values a bidder cannot read, tests run the three handlers with consent allowed and denied, and the two comments say only what the code does. | + +## Review log + +1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/28-device-signals` at `b680d0780`. Issues found, five of them blocking. The blocking issues were that the spec described a device type name mapping that the code no longer has, that the spec named two places where core asks the module when the code has three, that the spec's acceptance section claimed a proof by the Axum test and a per-commit record that neither pull request holds, that the spec said the module is handed the call that `detect` is handed and left out that the module sees every header and that the attributes go under no permission check, and that the spec's header pointed at closed pull requests and said Proposed. This revision states what the code does for each, adds the consent and privacy, deployment, adapter and untested-path sections, and leaves seven questions in section 7 for code changes, being the missing test of the three points, the unchecked values, the missing time limit, what the call carries, two comments, where the score belongs, and the guide and the changelog. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/28-device-signals` at `b680d0780`. Issues found, two of them blocking. This revision says that the integration guide describes only the attributes, that the changelog names the score and `DeviceModule::advertising_attributes` but not `DeviceInfo::attributes` and gives no breaking-change note, and that no production code reads the score although two tests do, and it confirms the pull request facts in the header and the acceptance. diff --git a/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md b/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md index f156867386..84b3331d5c 100644 --- a/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md +++ b/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md @@ -177,11 +177,6 @@ The paths these tests leave out are listed as questions under Open questions. The fork's CI passed on the series head, `a43a05a33`, all nineteen checks, and the description of IABTechLab/trusted-server#1356 records that, with a link to jwrosewell/trusted-server#10. The same description has every check command of its test plan ticked, being the Fastly and Axum tests and lints, the JavaScript tests, the WebAssembly build of the Fastly adapter and the format commands for the Rust code, the JavaScript and the documentation. A last line for the full workflow set of the repository, run on jwrosewell/trusted-server#10, is ticked as well, and manual testing with `fastly compute serve` is not. The description of IABTechLab/trusted-server#1356 does not say whether anyone also ran the commands by hand on the head. The descriptions of IABTechLab/trusted-server#1356, of IABTechLab/trusted-server#1302 and of jwrosewell/trusted-server#10 hold no record of a lint and a test of each commit on its own or of deliberate faults made one at a time, so this spec claims neither. The description of IABTechLab/trusted-server#1356 also says that the responses the Fastly entry point sends outside the middleware are not exercised, and those are listed under "Not in this change". -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/15-diagnostic-headers` at `a43a05a33`. Issues found, four of them blocking. The four were that the spec covered the three commits of 9 October and none of the five since, that the spec left `X-TS-Version` as it was and left out the breaking change the pull request lists for it, that the rule for paths beneath `/_ts` was wider than the code for an origin's answer while the inspection pages read as if they showed headers nothing writes, and that the acceptance cited a record the pull request descriptions do not hold. This revision states what the code does at that head for all eight commits, the paths beneath `/_ts`, the adapters, the version headers and the `service_version` middleware, the defaults, the rollout, what breaks, the failure and privacy answers, the tests and the acceptance, and lists under open questions what waits for code changes, being the exemption, the list of names, the headers still sent, an origin's version headers, the untested paths and the documents. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/15-diagnostic-headers` at `a43a05a33`. Issues found, none blocking. This revision corrects the acceptance, which said the pull request descriptions hold no record of a run of the whole check on the head although the description of the pull request for this row (IABTechLab/trusted-server#1356) has every check command ticked, and it qualifies three clauses that were only partly true, being what a stored page replays from the origin, what the proxies take off a response and which header the prefix test sends. - ## Revision record | Date | Change | @@ -189,3 +184,8 @@ The fork's CI passed on the series head, `a43a05a33`, all nineteen checks, and t | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The five commits since 9 October, being the fallback removal, the response headers a serve middleware may write, the platform in the middleware context, the `service_version` middleware and the end of the version headers on every adapter, the function names, the paths beneath `/_ts` and what the exemption keeps, the adapters and the responses that skip the finalize step, the defaults and the rollout, what breaks, the failure and privacy answers, the tests, the open questions, the header and the acceptance lines brought to the series head, and the review log. | | 2026-10-16 | Fixes from review round 2. The acceptance says what the description of the pull request for this row records, being the fork's run of all nineteen checks on the head, the ticked check commands and the manual test left unticked, in place of saying that no description records a run of the whole check. Three clauses that were only partly true are qualified, being the headers a stored page replays from the origin, the headers each proxy takes off and the one header the prefix test sends. A new open question asks whether the headers that core or an adapter writes directly should move into a serve middleware. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/15-diagnostic-headers` at `a43a05a33`. Issues found, four of them blocking. The four were that the spec covered the three commits of 9 October and none of the five since, that the spec left `X-TS-Version` as it was and left out the breaking change the pull request lists for it, that the rule for paths beneath `/_ts` was wider than the code for an origin's answer while the inspection pages read as if they showed headers nothing writes, and that the acceptance cited a record the pull request descriptions do not hold. This revision states what the code does at that head for all eight commits, the paths beneath `/_ts`, the adapters, the version headers and the `service_version` middleware, the defaults, the rollout, what breaks, the failure and privacy answers, the tests and the acceptance, and lists under open questions what waits for code changes, being the exemption, the list of names, the headers still sent, an origin's version headers, the untested paths and the documents. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/15-diagnostic-headers` at `a43a05a33`. Issues found, none blocking. This revision corrects the acceptance, which said the pull request descriptions hold no record of a run of the whole check on the head although the description of the pull request for this row (IABTechLab/trusted-server#1356) has every check command ticked, and it qualifies three clauses that were only partly true, being what a stored page replays from the origin, what the proxies take off a response and which header the prefix test sends. diff --git a/docs/superpowers/specs/2026-10-12-google-tag-module-design.md b/docs/superpowers/specs/2026-10-12-google-tag-module-design.md index ea2d2a3edb..e26811587f 100644 --- a/docs/superpowers/specs/2026-10-12-google-tag-module-design.md +++ b/docs/superpowers/specs/2026-10-12-google-tag-module-design.md @@ -106,11 +106,6 @@ CI passed on all 25 checks of the draft pull request at `1940d63f1`, and on all - **The new place in core.** `a_body_open_insert_is_written_once_after_the_opening_body_tag` in `html_processor.rs` drives the serve processor with two inserts and a second ``. The seam probe's middleware writes a marker immediately after the opening body tag, and its page test, `the_middleware_changes_a_page_where_an_entry_names_it`, reads the marker back once, directly after ``, through the fetch processor. - **The ad tag module's loader.** `head_inserts_write_the_loader_tag_when_asked`, `head_inserts_write_the_loader_tag_last` and `head_inserts_write_no_loader_tag_by_default` in `crates/ad-tag/google/src/lib.rs`. -## Review log - -1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/30-google-tag` at `1940d63f1`. Issues found, 5 of them blocking. The five were that the spec said the snippets land at the start of the head when they land before the script bundle's tag and after what a `[[fetch]]` middleware wrote, that it gave a reason for the module's place in the stock list that does not hold, that it never mentioned the `write_loader` setting the pull request adds to the Google ad tag module, that its consent design left out `wait_for_update`, `region` and the states a consent type can take, and that its header called a closed fork pull request ready for review and named a closed pull request as the place of the specs. This revision states what the code does at that head for each of the five, and adds the settings, the failure cases, the adapters, the tests that hold each claim and the code questions the review raised, which wait for code changes under Open questions. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/30-google-tag` at `1940d63f1`. Issues found, none blocking. This revision says what a policy that allows inline scripts by hash does with the snippets and that the consent snippet follows Google's published guide. - ## Revision record | Date | Change | @@ -119,3 +114,8 @@ CI passed on all 25 checks of the draft pull request at `1940d63f1`, and on all | 2026-10-16 | Fixes from review round 1. The header and the pull request pointers, where the head snippets land, the reason for the stock list order, the settings, the consent defaults, the Tag Manager module, the ad tag module's loader setting, failure, adapters, compatibility, the content security policy, the tests, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. The content security policy and the source of the consent snippet. | | 2026-10-16 | Code fixes from review round 1, code findings 1 to 5 and 7 and the case with no head in finding 6. The loaders and the noscript iframe are written only for a reader whose permissions allow storage on the device, a region setting gives every other visitor a default with all four types denied, the example's tag id placeholder fails validation and `AGENTS.md` lists the module. Tests run a page through both modules and a document with no head, and hold the crate's name to its folder. Open questions 1 to 5 and 7 and the case with no head in 6 leave the list. | + +## Review log + +1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/30-google-tag` at `1940d63f1`. Issues found, 5 of them blocking. The five were that the spec said the snippets land at the start of the head when they land before the script bundle's tag and after what a `[[fetch]]` middleware wrote, that it gave a reason for the module's place in the stock list that does not hold, that it never mentioned the `write_loader` setting the pull request adds to the Google ad tag module, that its consent design left out `wait_for_update`, `region` and the states a consent type can take, and that its header called a closed fork pull request ready for review and named a closed pull request as the place of the specs. This revision states what the code does at that head for each of the five, and adds the settings, the failure cases, the adapters, the tests that hold each claim and the code questions the review raised, which wait for code changes under Open questions. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/30-google-tag` at `1940d63f1`. Issues found, none blocking. This revision says what a policy that allows inline scripts by hash does with the snippets and that the consent snippet follows Google's published guide. diff --git a/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md b/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md index dd8a7c2dd4..508a8f4cc2 100644 --- a/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md +++ b/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md @@ -206,11 +206,6 @@ These are for the reviewer. This revision records what the code does and decides 5. **GPP cookies.** InMobi's page on sharing consent between sites names the cookies `euconsent-v2`, `addtl_consent` and `IABGPP_HDR_GppString` ([read on 2026-10-16](https://support.inmobi.com/choice/implementing-cmp-via-code/web-sdk/sharing-consent-between-multiple-sites)), and core reads `__gpp` and `__gpp_sid` for GPP. The page is the only source, because the code cannot show what InMobi's script writes. Does the prompt write a cookie that core reads for GPP, and is a GPP answer meant to reach the permission model? 6. **The deferred loader.** InMobi's page on implementing the web tag says that the tag runs asynchronously, recommends pasting the complete tag into the head without changes and triggering it first, and warns that delaying the CMP can leave IAB vendors unable to read the consent string in time ([read on 2026-10-16](https://support.inmobi.com/choice/getting-started-cmp/implementing-cmp-via-code/web-sdk/basic-implementation)). Only the page says this, and the code cannot show how InMobi's script behaves. The module writes its own stubs and a loader with `defer`. Does InMobi support that form? -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/18-inmobi` at `badd6fb15`. Issues found, 4 of them blocking. The four were that the spec named 51Degrees as the crate's maintainers where the manifest names the Trusted Server maintainers, that the header called a closed fork pull request ready for review and named closed pull requests as the place of the specs and the review, that the acceptance cited a run of every commit and 17 deliberate faults that neither pull request description records, and that the spec said the six rules are what every consent prompt module is held to when the other three modules do not meet them and nothing checks the first. This revision states what the code does at that head for the placement of the prompt, the configuration and when each check runs, consent and privacy, failure, compatibility, the adapters and the tests, replaces the acceptance claims with what the pull requests record, and leaves 8 questions under Open questions that wait for code changes, for the guide or for InMobi's documentation, being which entry covers a page, the gaps in the tests, stored pages after a stub change, the prompt's place against a vendor's tag, the wording of the six things, the sources, the GPP cookies and the deferred loader. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/18-inmobi` at `badd6fb15`. Issues found, none blocking. This revision names `cmp.didomi` and `cmp.sourcepoint` among the stock middleware that write head markup in the fetch phase, adds the Axum job on Windows to the runs of the crate's tests, and says where the design rests on InMobi's support pages alone. - ## Revision record | Date | Change | @@ -218,3 +213,8 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the maintainers, the six things and what holds them, the fetch phase alone and which pages get the prompt, what a vendor's tag is, the configuration and when each check runs, consent and privacy, failure, compatibility, the adapters, the tests and the acceptance, the open questions and the review log. | | 2026-10-16 | Fixes in the code for three faults the review found. Entries for `text/html` that leave a page without the prompt are refused, naming the pages, and so is a `cmp_id` that is not a whole number from 1 to 4095, naming the setting. Each middleware's `template_key` now keys a stored page, so a change to a stub stores pages again. The tests for each are in section 9, and the questions on which entry covers a page and on stored pages after a stub change, and the `cmp_id` part of the one on gaps in the tests, leave Open questions. From review round 2, `cmp.didomi` and `cmp.sourcepoint` are named among the stock middleware that write head markup in the fetch phase, the Axum job on Windows is added to the runs of the crate's tests, and the design says where it rests on InMobi's support pages alone. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/18-inmobi` at `badd6fb15`. Issues found, 4 of them blocking. The four were that the spec named 51Degrees as the crate's maintainers where the manifest names the Trusted Server maintainers, that the header called a closed fork pull request ready for review and named closed pull requests as the place of the specs and the review, that the acceptance cited a run of every commit and 17 deliberate faults that neither pull request description records, and that the spec said the six rules are what every consent prompt module is held to when the other three modules do not meet them and nothing checks the first. This revision states what the code does at that head for the placement of the prompt, the configuration and when each check runs, consent and privacy, failure, compatibility, the adapters and the tests, replaces the acceptance claims with what the pull requests record, and leaves 8 questions under Open questions that wait for code changes, for the guide or for InMobi's documentation, being which entry covers a page, the gaps in the tests, stored pages after a stub change, the prompt's place against a vendor's tag, the wording of the six things, the sources, the GPP cookies and the deferred loader. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/18-inmobi` at `badd6fb15`. Issues found, none blocking. This revision names `cmp.didomi` and `cmp.sourcepoint` among the stock middleware that write head markup in the fetch phase, adds the Axum job on Windows to the runs of the crate's tests, and says where the design rests on InMobi's support pages alone. diff --git a/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md b/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md index 8380744564..ab870e6a34 100644 --- a/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md +++ b/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md @@ -176,11 +176,6 @@ The fork's CI passed on the series head, `497d553c6`, all nineteen checks. That On `33282f0c9`, the last of the five original commits, the whole check also passed locally on Windows: `cargo fmt --all -- --check`, six Clippy runs with warnings denied, `cargo test -p trusted-server-core` (2,615 passed), the Fastly tests under Viceroy (3,626), the Axum alias (900), Cloudflare (59), Spin (93), the seam probe (11), the parity tests (20, three of them new here), the configuration generator's tests (8), the integration tests that need no container (29), the permissions inspector's wasm build, and the docs format, lint and build. The two core-only commits, `5fb5f0793` and `16771b18f`, were each linted with the adapter changes set aside, so each commit builds on its own. Two commits came after `33282f0c9` (`1c02104bd` and `a085a3ce7`), and these counts are not re-measured here. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/9-visibility-endpoints` at `497d553c6`. Issues found, three of them blocking. The design said a deployment whose settings do not select the change behaves as before when all five addresses answer on all four adapters with no setting, named no address, method, `.json` form, status or header for the permissions and configuration endpoints, and described the data page's protection as `Content-Security-Policy: sandbox` when the code sends a longer policy and five more headers and keeps them under an operator's headers. This revision states what the code does at that head for the addresses, the defaults and the one switch, the failure paths, the rollout, the tests and their gaps, and lists five questions that wait for code changes. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/9-visibility-endpoints` at `497d553c6`. Approved, none blocking. This revision says that the modules' request preparers and the request filters never run for these addresses on Fastly and Spin, in place of saying that the answer comes before them, and says where a request with a method other than `GET` goes on Cloudflare, which is the shared catch-all route and then the publisher. - ## Revision record | Date | Change | @@ -188,3 +183,8 @@ On `33282f0c9`, the last of the five original commits, the whole check also pass | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The addresses with their methods, forms, statuses and headers, the defaults and the one switch, what runs around each handler, the data page's full header set and its protection from an operator's headers, the failure paths, the rollout, the tests and the untested paths, the open questions, the header and the acceptance lines brought to the series head, and the review log. | | 2026-10-16 | Fixes from review round 2. The statement that the preparers and filters never run for these addresses on Fastly and Spin, where a request with a method other than `GET` goes on Cloudflare, the matching wording of the second open question, and the review log. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/9-visibility-endpoints` at `497d553c6`. Issues found, three of them blocking. The design said a deployment whose settings do not select the change behaves as before when all five addresses answer on all four adapters with no setting, named no address, method, `.json` form, status or header for the permissions and configuration endpoints, and described the data page's protection as `Content-Security-Policy: sandbox` when the code sends a longer policy and five more headers and keeps them under an operator's headers. This revision states what the code does at that head for the addresses, the defaults and the one switch, the failure paths, the rollout, the tests and their gaps, and lists five questions that wait for code changes. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/9-visibility-endpoints` at `497d553c6`. Approved, none blocking. This revision says that the modules' request preparers and the request filters never run for these addresses on Fastly and Spin, in place of saying that the answer comes before them, and says where a request with a method other than `GET` goes on Cloudflare, which is the shared catch-all route and then the publisher. diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index 593c175eee..390d1e013a 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -168,11 +168,6 @@ These are for the reviewer. This revision records what the code does and decides 1. Core writes `51degrees` in the documentation of `LoadingCacheConfig::module` (`settings.rs` line 3055) and in the test `a_loading_cache_module_is_read_and_written_back` (lines 7036 to 7055). Should those use a name that belongs to no vendor? 2. On rows 27 to 31 `CHANGELOG.md` repeats the robots.txt holding entry (lines 13 and 14 on row 31). The first is an older wording that a merge brought in on row 27, and row 24 does not carry it. Should row 27 drop it? -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/31-caching-release` at `4b6ef9c05`, with the design's code read on `split/24-loading-cache` at `844b01fe2`. Issues found, three of them blocking. The three were that the spec said a feature that names no module gets `NoLoadingCache`, when a stock build gives it the one loading cache module linked, that the spec said loads run on a pool of threads on a native host, when every load runs in its caller and Axum has no key-value store, and that the spec said a module is asked once per refresh however many instances want its rules, when only a store that reserves a key makes other instances wait. This revision states what the code does on both rows, adds the adapters, the configuration, consent and privacy, failure, compatibility, the tests and the pull requests that exist now, and leaves seven code questions under Open questions, which wait for code changes. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/31-caching-release` at `4b6ef9c05`, with the design's code read on `split/24-loading-cache` at `844b01fe2`. Issues found, none blocking. This revision states how the two published crates differ, from a file by file comparison of the crates themselves, and confirms the check counts, the pull request states and the description of IABTechLab/trusted-server#1365 that this spec cites. - ## Revision record | Date | Change | @@ -182,3 +177,8 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-16 | Points from review round 2. How the two published crates differ, from a comparison of the crates themselves, and the date on which the check counts were read. | | 2026-10-16 | Code fixes on row 24. A load is handed services with nothing of the reader in them, each application state has a cache of its own on Cloudflare as elsewhere, `CacheSpec::detachable` and the text about loads that run apart from their caller are gone, the comments and guides say what a table naming no module takes, the deploy check refuses a `module` nothing offers, and tests cover the stock default, a zero memory lifetime, a dropped first caller and a failing store. Open questions 1, 2, 4, 5 and 7, and 3 apart from the repeated changelog entry, move into the text. | | 2026-10-17 | Code fix on row 24. The Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests and a Worker or component without its store fails every request. Who waits for a load, the adapters, a module that fails, what changes for a deployment and the tests say so. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/31-caching-release` at `4b6ef9c05`, with the design's code read on `split/24-loading-cache` at `844b01fe2`. Issues found, three of them blocking. The three were that the spec said a feature that names no module gets `NoLoadingCache`, when a stock build gives it the one loading cache module linked, that the spec said loads run on a pool of threads on a native host, when every load runs in its caller and Axum has no key-value store, and that the spec said a module is asked once per refresh however many instances want its rules, when only a store that reserves a key makes other instances wait. This revision states what the code does on both rows, adds the adapters, the configuration, consent and privacy, failure, compatibility, the tests and the pull requests that exist now, and leaves seven code questions under Open questions, which wait for code changes. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/31-caching-release` at `4b6ef9c05`, with the design's code read on `split/24-loading-cache` at `844b01fe2`. Issues found, none blocking. This revision states how the two published crates differ, from a file by file comparison of the crates themselves, and confirms the check counts, the pull request states and the description of IABTechLab/trusted-server#1365 that this spec cites. diff --git a/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md b/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md index 53dd9b2848..2064691a2d 100644 --- a/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md +++ b/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md @@ -170,11 +170,6 @@ This search prints seven lines, six in `middleware/tests.rs` and one in `publish git grep -n "\.making(" 7723c5e56 -- crates ``` -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/22-creating` at `7723c5e56`. Issues found, four of them blocking. The four were that the spec said the made file is stored when core makes the file again on every request, that the rule left out four behaviors (any path under the prefix makes a file, the origin's `404` body and headers are dropped, an HTML entry makes nothing and the longest path wins), that the header named pull requests that are closed and never the series pull request, and that the acceptance said the pull request description records checks, 6 deliberate faults among them, which neither pull request description records. This revision states what the code does at that head for the trigger, the entry that makes the file, the response, the phases, caching, configuration, the adapters, consent and privacy, failure and the tests, replaces the acceptance claims with what the pull request description records, and leaves four questions about the code under Open questions until the code changes, being an entry whose `path` covers every path, a request with no host, the sellers check that accepts an entry that cannot make `/ads.txt` and the example settings file. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/22-creating` at `7723c5e56`. Approved, none blocking. This revision gives the 25 passing check runs on the pull request and the 19 on the fork's closed pull request separately, says that the description records only the 19, and quotes the description for where the full workflow set ran. - ## Revision record | Date | Change | @@ -182,3 +177,8 @@ git grep -n "\.making(" 7723c5e56 -- crates | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The header and the pull requests the header names, what the change touches, the rule as tables for the trigger, the entry and the response, the empty document, caching, configuration and rollout, the adapters, consent and privacy, failure, the rows that hold what the change leaves out, the open questions, the tests and the cases with none, the acceptance and the review log. | | 2026-10-16 | Fixes from review round 2. The acceptance gives the 25 check runs on the pull request and the 19 on the fork's closed pull request separately, says that the description records only the 19, and quotes the description for where the full workflow set ran. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/22-creating` at `7723c5e56`. Issues found, four of them blocking. The four were that the spec said the made file is stored when core makes the file again on every request, that the rule left out four behaviors (any path under the prefix makes a file, the origin's `404` body and headers are dropped, an HTML entry makes nothing and the longest path wins), that the header named pull requests that are closed and never the series pull request, and that the acceptance said the pull request description records checks, 6 deliberate faults among them, which neither pull request description records. This revision states what the code does at that head for the trigger, the entry that makes the file, the response, the phases, caching, configuration, the adapters, consent and privacy, failure and the tests, replaces the acceptance claims with what the pull request description records, and leaves four questions about the code under Open questions until the code changes, being an entry whose `path` covers every path, a request with no host, the sellers check that accepts an entry that cannot make `/ads.txt` and the example settings file. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/22-creating` at `7723c5e56`. Approved, none blocking. This revision gives the 25 passing check runs on the pull request and the 19 on the fork's closed pull request separately, says that the description records only the 19, and quotes the description for where the full workflow set ran. diff --git a/docs/superpowers/specs/2026-10-12-middleware-media-types-design.md b/docs/superpowers/specs/2026-10-12-middleware-media-types-design.md index 3f235a57d0..0cbc94fbfd 100644 --- a/docs/superpowers/specs/2026-10-12-middleware-media-types-design.md +++ b/docs/superpowers/specs/2026-10-12-middleware-media-types-design.md @@ -139,11 +139,6 @@ Eight things have no test. The fork's CI passed on the series head, `149280a65`, all nineteen checks, as the description of IABTechLab/trusted-server#1360 records. That description ticks the Fastly and Axum tests, the Fastly and Axum Clippy runs, the format check, the JavaScript tests and format, the docs format and the WASM build, and leaves the manual test through `fastly compute serve` unticked. It records that no module of the stack changes a body that is not HTML yet, and that the fixture middleware in core's tests does, through the publisher's own path. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/19-media-types` at `149280a65`. Issues found, four of them blocking. The design listed `image/svg+xml` as a type an entry may cover when the code refuses it, described stored copies that a body that is not HTML never has, said a middleware changes the body alone when a serve middleware's headers also reach the response, and left out that a response declaring a charset other than UTF-8 gets no middleware. This revision states what the code does at that head for the types, the match, the charset, the status codes, caching, the headers, the adapters, failure and compatibility, replaces the acceptance claims with what the pull request description records, and lists five questions that wait for code changes. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/19-media-types` at `149280a65`. Approved, none blocking. This revision qualifies the identity encoding, the one fixture that changes a body, the tests that run Fastly's conversion and the tests that name `application/json`, and says that core's address replacement drops a body's bytes from the first one that is not valid UTF-8. - ## Revision record | Date | Change | @@ -151,3 +146,8 @@ The fork's CI passed on the series head, `149280a65`, all nineteen checks, as th | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The types an entry may cover and the refusals, the match, what no entry can change, the charset, the headers, what a processor is handed, status codes, caching and validators, consent and privacy, the adapters, failure, compatibility, the rows that hold what this change leaves out, the tests and the cases with none, the open questions, the header and the acceptance lines brought to the series head, and the review log. | | 2026-10-16 | Fixes from review round 2. The identity encoding, the one fixture that changes a body, the tests that run Fastly's conversion and the tests that name `application/json` are stated as the code has them, and core's address replacement is said to drop a body's bytes from the first one that is not valid UTF-8. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/19-media-types` at `149280a65`. Issues found, four of them blocking. The design listed `image/svg+xml` as a type an entry may cover when the code refuses it, described stored copies that a body that is not HTML never has, said a middleware changes the body alone when a serve middleware's headers also reach the response, and left out that a response declaring a charset other than UTF-8 gets no middleware. This revision states what the code does at that head for the types, the match, the charset, the status codes, caching, the headers, the adapters, failure and compatibility, replaces the acceptance claims with what the pull request description records, and lists five questions that wait for code changes. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/19-media-types` at `149280a65`. Approved, none blocking. This revision qualifies the identity encoding, the one fixture that changes a body, the tests that run Fastly's conversion and the tests that name `application/json`, and says that core's address replacement drops a body's bytes from the first one that is not valid UTF-8. diff --git a/docs/superpowers/specs/2026-10-12-middleware-prepare-design.md b/docs/superpowers/specs/2026-10-12-middleware-prepare-design.md index 4a18e0f480..9144c5471a 100644 --- a/docs/superpowers/specs/2026-10-12-middleware-prepare-design.md +++ b/docs/superpowers/specs/2026-10-12-middleware-prepare-design.md @@ -174,11 +174,6 @@ git grep -n "prepare_document(" d9fb81a64 -- crates git grep -n "fn prepare<" d9fb81a64 -- crates ``` -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/23-prepare` at `d9fb81a64`. Issues found, three of them blocking. The three were that the example and the key under which an answer is kept were false, because the answer is kept under the phase and the middleware's id and the call is `take_prepared`, that the spec never said what a preparation is called with, so the unstored case read as a bug, and that the header named pull requests that are closed while the acceptance claimed 5 deliberate faults and a check of every commit on its own, which neither pull request description records. This revision states what the code does at that head for the example, the key, what each case is called with, the refusal, the documents prepared, compatibility, configuration, the adapters, consent and privacy, failure, the tests and the acceptance, and leaves six questions about the code under Open questions until the code changes, being the reader's connection in the services of a fetch preparation, the tests of the stored cases, the choice of documents, the checks on a refusal, the wait and the missing answer, and the order against the auction. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/23-prepare` at `d9fb81a64`. Issues found, none blocking. This revision names the rule that the guide and the changelog state, and lists both places that say an answer is kept under the middleware's name, the changelog line as well as the guide. - ## Revision record | Date | Change | @@ -186,3 +181,8 @@ git grep -n "fn prepare<" d9fb81a64 -- crates | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header and the pull requests the header names, the example and the key under which an answer is kept, what the change touches, what each case is called with, the refusal as a table, the documents prepared, compatibility, configuration, the adapters, consent and privacy, failure, the open questions, the tests and the cases with none, the acceptance and the review log. | | 2026-10-16 | Fixes from review round 2. The rule the guide and the changelog state is named, and both places that say an answer is kept under the middleware's name are listed. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/23-prepare` at `d9fb81a64`. Issues found, three of them blocking. The three were that the example and the key under which an answer is kept were false, because the answer is kept under the phase and the middleware's id and the call is `take_prepared`, that the spec never said what a preparation is called with, so the unstored case read as a bug, and that the header named pull requests that are closed while the acceptance claimed 5 deliberate faults and a check of every commit on its own, which neither pull request description records. This revision states what the code does at that head for the example, the key, what each case is called with, the refusal, the documents prepared, compatibility, configuration, the adapters, consent and privacy, failure, the tests and the acceptance, and leaves six questions about the code under Open questions until the code changes, being the reader's connection in the services of a fetch preparation, the tests of the stored cases, the choice of documents, the checks on a refusal, the wait and the missing answer, and the order against the auction. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/23-prepare` at `d9fb81a64`. Issues found, none blocking. This revision names the rule that the guide and the changelog state, and lists both places that say an answer is kept under the middleware's name, the changelog line as well as the guide. diff --git a/docs/superpowers/specs/2026-10-12-module-context-design.md b/docs/superpowers/specs/2026-10-12-module-context-design.md index 5b2b201c88..68a3db999e 100644 --- a/docs/superpowers/specs/2026-10-12-module-context-design.md +++ b/docs/superpowers/specs/2026-10-12-module-context-design.md @@ -136,11 +136,6 @@ These are for the reviewer. This revision records what the code does and decides 4. Should a request filter be handed the location only when the geo module's declared permissions allow it, as the module context does? 5. Should a permission signal module be able to declare permissions at all, when no permission is resolved while the signal modules run? -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/8-module-context` at `bc28b3e7f`. Issues found, three of them blocking. The blocking issues were that section 3 said settings select the change when none does and left out what the Axum, Cloudflare and Spin adapters now do for every module route, that the costs left out the geo lookup and the other work each of those routes now does, and that nothing said what a route answers when a value is withheld or absent. This revision states what the code does for each, adds the adapter table, the answers for a missing value and the untested paths, corrects the partly true claims and the header, and leaves the code questions in section 6 and the missing tests in section 5 for code changes. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/8-module-context` at `bc28b3e7f`. Approved, none blocking. This revision says Cloudflare and Spin resolve the selected Edge Cookie module on every request, as they build their state for each one, and the fork's check runs confirm 19 of 19 on both commits the spec names. - ## Revision record | Date | Change | @@ -148,3 +143,8 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The header, the failure path and the identifier on three adapters in place of the claim that settings select the change, the costs of a module route, the answers for a missing value, the gating defaults, what keeps its own parameter list, the move of a module outside this repository, the acceptance on the head, the untested paths, the open questions and the review log. | | 2026-10-16 | Review round 2, approved, with the module resolution on Cloudflare and Spin stated per request. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/8-module-context` at `bc28b3e7f`. Issues found, three of them blocking. The blocking issues were that section 3 said settings select the change when none does and left out what the Axum, Cloudflare and Spin adapters now do for every module route, that the costs left out the geo lookup and the other work each of those routes now does, and that nothing said what a route answers when a value is withheld or absent. This revision states what the code does for each, adds the adapter table, the answers for a missing value and the untested paths, corrects the partly true claims and the header, and leaves the code questions in section 6 and the missing tests in section 5 for code changes. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/8-module-context` at `bc28b3e7f`. Approved, none blocking. This revision says Cloudflare and Spin resolve the selected Edge Cookie module on every request, as they build their state for each one, and the fork's check runs confirm 19 of 19 on both commits the spec names. diff --git a/docs/superpowers/specs/2026-10-12-page-middleware-design.md b/docs/superpowers/specs/2026-10-12-page-middleware-design.md index 0d3c2e1b7e..d0ce8873b5 100644 --- a/docs/superpowers/specs/2026-10-12-page-middleware-design.md +++ b/docs/superpowers/specs/2026-10-12-page-middleware-design.md @@ -238,11 +238,6 @@ To go back, push the configuration without the entries first and restore the ear Open question 4 lists three things that have no test. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/10-page-middleware` at `0b0320f88`. Issues found, two of them blocking. The two were that section 3 said a deployment that does not select the change behaves as before, when a module that is selected and named by no entry stops changing pages and the fixes bundled with the change alter some pages, and that the header presented the specs, seam and fork pull requests as live when all three were closed on 15 October, and named neither this row's pull request nor the current specs pull request. This revision states the header, the upgrade steps and their order, what each phase is handed, the path rules, consent and privacy, the second parse a serve entry costs, the adapters, the failure cases and the checks a reviewer can run, and leaves five questions under Open questions that wait for code changes, being the middleware that nothing places, the script text written as given, an entry after a path of `/`, the missing tests and the wording of the guides and the description. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/10-page-middleware` at `0b0320f88`. Issues found, none blocking. This revision says that `cargo test-axum` runs eleven of the twelve recording tests because it names no core crate, and confirms the pull request numbers, rows, states and check counts that the spec gives. - ## Revision record | Date | Change | @@ -250,3 +245,8 @@ Open question 4 lists three things that have no test. | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the problem stated once, what each phase is handed, the path rules, consent and privacy, the second parse a serve entry costs, what is removed and what breaks, the adapters, the failure cases and the deploy validation wording, the three faults, the place of DataDome's tag, the section on the first draft of section 3.7, the commit table and its missing commit, the upgrade steps and their order, what later rows build, the open questions, the checks a reviewer can run, the acceptance lines and the review log. | | 2026-10-16 | Fixes from review round 2. The checks table says that `cargo test-axum` runs eleven of the twelve recording tests, because it names no core crate. The pull request numbers, rows, states and check counts were read against the pull requests and stand. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/10-page-middleware` at `0b0320f88`. Issues found, two of them blocking. The two were that section 3 said a deployment that does not select the change behaves as before, when a module that is selected and named by no entry stops changing pages and the fixes bundled with the change alter some pages, and that the header presented the specs, seam and fork pull requests as live when all three were closed on 15 October, and named neither this row's pull request nor the current specs pull request. This revision states the header, the upgrade steps and their order, what each phase is handed, the path rules, consent and privacy, the second parse a serve entry costs, the adapters, the failure cases and the checks a reviewer can run, and leaves five questions under Open questions that wait for code changes, being the middleware that nothing places, the script text written as given, an entry after a path of `/`, the missing tests and the wording of the guides and the description. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/10-page-middleware` at `0b0320f88`. Issues found, none blocking. This revision says that `cargo test-axum` runs eleven of the twelve recording tests because it names no core crate, and confirms the pull request numbers, rows, states and check counts that the spec gives. diff --git a/docs/superpowers/specs/2026-10-12-prefix-module-design.md b/docs/superpowers/specs/2026-10-12-prefix-module-design.md index 72704fd406..7771e30377 100644 --- a/docs/superpowers/specs/2026-10-12-prefix-module-design.md +++ b/docs/superpowers/specs/2026-10-12-prefix-module-design.md @@ -151,11 +151,6 @@ These are for the reviewer. This revision records what the code does and decides 4. Should tests cover the seven cases listed under "What the tests do not prove" in section 4? 5. Should startup check `template_cache_key_cookies` against the prefix the trial resolves? Settings validation refuses a `template_cache_key_cookies` entry that names core's own identity cookies `ts-ec`, `ts-eids` or `sharedId` (`cookies/template_cache_policy.rs` lines 35 to 62, called from `creative_opportunities.rs` line 536). Under another prefix, an entry that names the request's Edge Cookie, such as `_ex-ec`, passes validation and then makes each such request share no template (`publisher.rs` lines 5228 to 5242). -## Review log - -1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/27-prefix-module` at `9a9993423`. Issues found, 5 of them blocking. The five were that the spec named browser storage keys that the code never writes, that it said every shared store key takes the prefix when only the robots.txt answers do, that it said the public prefix carries no domain and the key domain never comes from the request, when the code allows a domain in the public prefix and only core's module follows the rule, that its acceptance section cited per-commit checks and 8 deliberate faults that neither pull request description records and 19 green checks that belong to the fork while a code scanning check fails on the pull request itself, and that its list of what is not in the change held the constant rename the change makes and left out four other changes. This revision states which cookies the public prefix names, the one use of the server key prefix and what keeps its own name, where the domain comes from, when a refused selection shows on each adapter, what changes for code and for a deployment, the adapters and the tests, and leaves five questions for code changes, being the two code scanning alerts, comments and guide text that describe what the code does not do, checks the startup trial does not make, tests that do not exist and the template key cookie check. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/27-prefix-module` at `9a9993423`. Issues found, none blocking. This revision adds the two new public fields to the list of new public items and names the second place the code scanning data flow starts. - ## Revision record | Date | Change | @@ -163,3 +158,8 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The status line and the pull requests it names, the cookies the public prefix names, the one use of the server key prefix and what keeps its own name, where the domain comes from, when a refused selection shows, what changes for code and for a deployment, the adapters, the acceptance rebuilt from the pull request description and the tests, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. The list of new public items and where the code scanning data flow starts. | + +## Review log + +1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/27-prefix-module` at `9a9993423`. Issues found, 5 of them blocking. The five were that the spec named browser storage keys that the code never writes, that it said every shared store key takes the prefix when only the robots.txt answers do, that it said the public prefix carries no domain and the key domain never comes from the request, when the code allows a domain in the public prefix and only core's module follows the rule, that its acceptance section cited per-commit checks and 8 deliberate faults that neither pull request description records and 19 green checks that belong to the fork while a code scanning check fails on the pull request itself, and that its list of what is not in the change held the constant rename the change makes and left out four other changes. This revision states which cookies the public prefix names, the one use of the server key prefix and what keeps its own name, where the domain comes from, when a refused selection shows on each adapter, what changes for code and for a deployment, the adapters and the tests, and leaves five questions for code changes, being the two code scanning alerts, comments and guide text that describe what the code does not do, checks the startup trial does not make, tests that do not exist and the template key cookie check. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/27-prefix-module` at `9a9993423`. Issues found, none blocking. This revision adds the two new public fields to the list of new public items and names the second place the code scanning data flow starts. diff --git a/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md b/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md index a8cbd6885d..d80d0748bc 100644 --- a/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md +++ b/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md @@ -189,11 +189,6 @@ Open question 8 asks whether adapter tests should cover the first four. **CI.** Every check passed, 25 of 25 on IABTechLab/trusted-server#1355 and 19 of 19 on jwrosewell/trusted-server#9, as read on 2026-10-16. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/14-robots-txt` at `35eb5ae83`. Issues found, four of them blocking. The four were that the spec said every answer is held in the key-value store when Fastly and Axum pass a store that holds nothing, that the spec said a contributor is given nothing about who asked when the services it receives carry the crawler's connection, that the spec said every response carries `X-Robots-Tag` when only the responses an adapter finalizes do, and that the header named pull requests that have closed instead of the series. This revision states what the code does at that head for the store each adapter passes, what a contributor is given and what the held key leaves out, the refusal and an empty answer, the steps each adapter runs before the handler, the responses that carry the header, the cache headers, the configuration, rollout and the tests, replaces the acceptance claims with what the pull request description records, names the rows that replace the route and the holding, and leaves eight questions under Open questions that wait for code changes, being what Fastly and Axum hold, what a contributor can read, host, scheme and method, an empty answer, the route on Fastly, the header after the filters, the shape of a contribution and the missing tests. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/14-robots-txt` at `35eb5ae83`. Approved, none blocking. This revision cites `document.rs` line 174 for the line breaks removed from `User-Agent` names, names the changelog's claim that a contributor cannot produce a malformed file among the statements that this spec contradicts, and records the earlier commit that the description says CI passed on. - ## Revision record | Date | Change | @@ -202,3 +197,8 @@ Open question 8 asks whether adapter tests should cover the first four. | 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the settings, the refusal and the empty answer, what a contributor is given and what the key holds, the store each adapter passes, the steps each adapter runs before the handler, the responses that carry `X-Robots-Tag`, the cache headers, consent and privacy, what is refused and when, what changes for a deployment, the rows that replace the route and the holding, the open questions, the tests, the acceptance and the review log. | | 2026-10-16 | Points from review round 2. The citation for the line breaks removed from `User-Agent` names, the changelog's claim that a contributor cannot produce a malformed file, and the earlier commit that the description says CI passed on. | | 2026-10-17 | Which adapters hold answers on this row. None does, Cloudflare and Spin included, because neither declares a store for EdgeZero to open, so the store table, the store column of the route table, the note on the guide and the changelog, and open question 1 say so, and say that row 24 declares the two stores. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/14-robots-txt` at `35eb5ae83`. Issues found, four of them blocking. The four were that the spec said every answer is held in the key-value store when Fastly and Axum pass a store that holds nothing, that the spec said a contributor is given nothing about who asked when the services it receives carry the crawler's connection, that the spec said every response carries `X-Robots-Tag` when only the responses an adapter finalizes do, and that the header named pull requests that have closed instead of the series. This revision states what the code does at that head for the store each adapter passes, what a contributor is given and what the held key leaves out, the refusal and an empty answer, the steps each adapter runs before the handler, the responses that carry the header, the cache headers, the configuration, rollout and the tests, replaces the acceptance claims with what the pull request description records, names the rows that replace the route and the holding, and leaves eight questions under Open questions that wait for code changes, being what Fastly and Axum hold, what a contributor can read, host, scheme and method, an empty answer, the route on Fastly, the header after the filters, the shape of a contribution and the missing tests. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/14-robots-txt` at `35eb5ae83`. Approved, none blocking. This revision cites `document.rs` line 174 for the line breaks removed from `User-Agent` names, names the changelog's claim that a contributor cannot produce a malformed file among the statements that this spec contradicts, and records the earlier commit that the description says CI passed on. diff --git a/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md b/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md index 1f3212d6b4..5db9cb5c01 100644 --- a/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md +++ b/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md @@ -247,12 +247,6 @@ This search prints two lines, both in `publisher.rs` (2052 and 5812), so the pag git grep -n "answers_robots_txt_with_a_page" 16248774e -- crates ``` -## Review log - -1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/25-robots-txt-middleware` at `16248774e`. Issues found, two of them blocking. The two were that the spec said the settings refuse a section only when no `[[fetch]]` or `[[serve]]` entry names the middleware, when the code requires a `[[fetch]]` entry for `text/plain` on exactly `/robots.txt` and refuses an entry anywhere else, and that the spec left out that the origin's answer now decides what a crawler gets, so an unreachable origin, a `401`, a `403`, a `410` or a redirect reaches the crawler even when `refuse_all` is selected. This revision states what the code does at that head for the placement, each answer the origin can give, the file, what a contributor is given, caching and stored copies, configuration, rollout, the adapters and failure, replaces the acceptance claims with what the pull request description records, points the header at the series pull request and the stack issue, and leaves six questions about the code under Open questions until the code changes. - -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/25-robots-txt-middleware` at `16248774e`. Issues found, none blocking. This revision states what the code does with a `304` in case 10 and with cases 7 and 8, cites the default of `[cache.robots-txt] module` where the code sets it, names row 33's change to a test of the holding and the Fastly test file the footprint counts, and checks the row 34 lines against that row's code. - ## Revision record | Date | Change | @@ -265,3 +259,9 @@ git grep -n "answers_robots_txt_with_a_page" 16248774e -- crates | 2026-10-16 | Contributors are asked for the host the request was addressed to, never a forwarded one, which the publisher path hands every preparation context, and Fastly answers a `GET` or `HEAD` of the file before the Edge Cookie steps and the request filters. What a contributor is given, the adapters and five tests. | | 2026-10-17 | One text for both sets of code fixes above. The code that holds both keeps the origin's answer, a `404` among them, for a request that names no host, and hands a contributor nothing of the crawler's connection, so what happens, the steps of a request, case 11, open question 2 and what a contributor is given now say so. | | 2026-10-17 | The kept answer for a failed contributor names the stores the Cloudflare and Spin adapters declare on row 24, `TRUSTED_SERVER_KV` and `default`, and says that Axum keeps none. | + +## Review log + +1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/25-robots-txt-middleware` at `16248774e`. Issues found, two of them blocking. The two were that the spec said the settings refuse a section only when no `[[fetch]]` or `[[serve]]` entry names the middleware, when the code requires a `[[fetch]]` entry for `text/plain` on exactly `/robots.txt` and refuses an entry anywhere else, and that the spec left out that the origin's answer now decides what a crawler gets, so an unreachable origin, a `401`, a `403`, a `410` or a redirect reaches the crawler even when `refuse_all` is selected. This revision states what the code does at that head for the placement, each answer the origin can give, the file, what a contributor is given, caching and stored copies, configuration, rollout, the adapters and failure, replaces the acceptance claims with what the pull request description records, points the header at the series pull request and the stack issue, and leaves six questions about the code under Open questions until the code changes. + +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/25-robots-txt-middleware` at `16248774e`. Issues found, none blocking. This revision states what the code does with a `304` in case 10 and with cases 7 and 8, cites the default of `[cache.robots-txt] module` where the code sets it, names row 33's change to a test of the holding and the Fastly test file the footprint counts, and checks the row 34 lines against that row's code. diff --git a/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md b/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md index 4d04690935..39f6d72d16 100644 --- a/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md +++ b/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md @@ -141,11 +141,6 @@ These are for the reviewer. This revision records what the code does and decides 4. Should a test resolve a shared secret through `settings_from_config_blob_with`, check that an unused one is cleared, and check that `/_ts/config` masks the secret? Only the pieces are tested today. 5. Should push validation refuse a table that no builder carries the mark for? The registry check cannot run at push, because push builds no registry and the CLI links only the builders registered for deploy validation, so a push accepts the table and the service then fails to start. -## Review log - -1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/26-shared-tables` at `5cbd69d20`. Issues found, 2 of them blocking. The two were that the spec called a closed fork pull request ready for review, named a closed pull request as the one holding the design documents, said a table is named by a folder the code never checks and cited an acceptance record the pull request description does not hold, and that the spec never said what a builder run by a table can supply. This revision states what the code does at that head for the table's name, the builder a table runs and what the builder can supply, what keeps running without the table, the secrets, the error text, each refusal and when it runs, the adapters, compatibility and the tests, and lists seven questions that wait for code changes, being key names at push, values in error text, the note on refusals, a mark with a name no table can have, the changelog line for `robots_txt`, a test of the secret path and a table nothing reads at push. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/26-shared-tables` at `5cbd69d20`. Approved, none blocking. This revision confirms the 25 passing checks on IABTechLab/trusted-server#1367, says that the validator quotes a checked value only for a rule without a message of its own, and says that on Axum, Cloudflare and Spin a builder a deployment adds that carries the mark gets key names in place of its shared secrets. - ## Revision record | Date | Change | @@ -154,3 +149,8 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-16 | Fixes from review round 1. The status line and the pull requests it names, the rule for a table's name, the builder a table starts and what keeps running without the table, what a builder a table runs can supply, the secrets at load and at push, the error text, each refusal and when it runs, the adapters, permissions, compatibility, the acceptance rebuilt from the pull request description, the tests, the open questions and the review log. | | 2026-10-16 | Fixes from the code review. A shared table or a module's own table that is refused names the setting at fault and never its value, and push checks that each secret setting a shared table uses names a key, each with a test, which closes the first two open questions. | | 2026-10-16 | Points from review round 2. When the validator quotes a checked value, and what a builder a deployment adds that carries the mark gets on Axum, Cloudflare and Spin. | + +## Review log + +1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/26-shared-tables` at `5cbd69d20`. Issues found, 2 of them blocking. The two were that the spec called a closed fork pull request ready for review, named a closed pull request as the one holding the design documents, said a table is named by a folder the code never checks and cited an acceptance record the pull request description does not hold, and that the spec never said what a builder run by a table can supply. This revision states what the code does at that head for the table's name, the builder a table runs and what the builder can supply, what keeps running without the table, the secrets, the error text, each refusal and when it runs, the adapters, compatibility and the tests, and lists seven questions that wait for code changes, being key names at push, values in error text, the note on refusals, a mark with a name no table can have, the changelog line for `robots_txt`, a test of the secret path and a table nothing reads at push. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/26-shared-tables` at `5cbd69d20`. Approved, none blocking. This revision confirms the 25 passing checks on IABTechLab/trusted-server#1367, says that the validator quotes a checked value only for a rule without a message of its own, and says that on Axum, Cloudflare and Spin a builder a deployment adds that carries the mark gets key names in place of its shared secrets. diff --git a/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md b/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md index dc46ea4d90..9308350cad 100644 --- a/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md +++ b/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md @@ -148,11 +148,6 @@ The questions below are for the reviewer. This revision records what the code do 5. **Mapping calls with no `try` and `catch`.** The calls that follow `defineSlot` in `defineWithViewportSizes` have no `try` and `catch` (`tag/gpt_bootstrap.js` lines 361 to 383, `js/integrations/gpt/index.ts` lines 858 to 877). An exception from them would skip `addService` and the release of the claim on the placement (`tag/gpt_bootstrap.js` lines 706 to 713, `js/integrations/gpt/index.ts` lines 1185 to 1192), so the slot would be defined and not added to the ad service. Should a failure in the mapping calls leave the slot defined with every size instead? Nothing in this row shows GPT throwing from these calls. 6. **Four places that do not describe the row.** First, the comment on `build_ad_slots_script` (`core/publisher.rs`, lines 7526 to 7529) lists the property names of the page data without `viewport_sizes`. Second, `ts config ad-templates --details` prints each format as its size and media type, without its bounds (`crates/trusted-server-cli/src/commands/config/ad_templates.rs`, lines 490 to 497). Third, in the configuration guide the new section sits between the example of `[creative_opportunities]` and the paragraph that begins "The same switch can be overridden", so that paragraph no longer follows the switch it names (`docs/guide/configuration.md`, lines 2772 to 2797). Fourth, the new section of the guide does not say that the server-side auctions are still offered every format. Should each be corrected? -## Review log - -1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/29-viewport-sizes` at `054f3a2a0`. Issues found, two of them blocking. The blocking issues were that the spec never named the refresh auction, which offers a bidder only the sizes of the band, called the bounds browser only and gave no reason why the server-side auctions are still offered every format, and that its last paragraph claimed four deliberate faults that no description or commit message records. This revision states which requests the bands reach and why the server-side auctions are not among them, what a band is and what an empty band means, which slots get a mapping, when the band takes effect, what a wrong order of rollout and rollback does, what each adapter runs, what the refresh auction tells a bidder about the viewport, the checks recorded and the tests that exist, while six open questions wait for code changes, being the size applied after the first paint, the winning bid outside the band, the slot the publisher defined, the paths no test covers, the mapping calls with no `try` and `catch`, and four places that do not describe the row. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/29-viewport-sizes` at `054f3a2a0`. Issues found, none blocking. This revision says what the page-bids request sends and corrects the line where the Spin adapter registers its health route. - ## Revision record | Date | Change | @@ -160,3 +155,8 @@ The questions below are for the reviewer. This revision records what the code do | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The header, the pull requests it names and the related specs, the three requests that ask for a slot's sizes and which of them the bands reach, the refresh auction that the second commit of the row (`9c2e6e802`) added after the first draft, what a band is and what an empty band means, the bounds and the page data, slots the publisher defines, when the band takes effect and who holds the space, failure, rollout, rollback, the adapters and privacy, the checks recorded with the claim of deliberate faults removed, the tests that exist and those that do not, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. What the page-bids request sends and the line of the Spin health route. | + +## Review log + +1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/29-viewport-sizes` at `054f3a2a0`. Issues found, two of them blocking. The blocking issues were that the spec never named the refresh auction, which offers a bidder only the sizes of the band, called the bounds browser only and gave no reason why the server-side auctions are still offered every format, and that its last paragraph claimed four deliberate faults that no description or commit message records. This revision states which requests the bands reach and why the server-side auctions are not among them, what a band is and what an empty band means, which slots get a mapping, when the band takes effect, what a wrong order of rollout and rollback does, what each adapter runs, what the refresh auction tells a bidder about the viewport, the checks recorded and the tests that exist, while six open questions wait for code changes, being the size applied after the first paint, the winning bid outside the band, the slot the publisher defined, the paths no test covers, the mapping calls with no `try` and `catch`, and four places that do not describe the row. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/29-viewport-sizes` at `054f3a2a0`. Issues found, none blocking. This revision says what the page-bids request sends and corrects the line where the Spin adapter registers its health route. diff --git a/docs/superpowers/specs/2026-10-13-middleware-settings-design.md b/docs/superpowers/specs/2026-10-13-middleware-settings-design.md index eb0bb61d97..ee379acd50 100644 --- a/docs/superpowers/specs/2026-10-13-middleware-settings-design.md +++ b/docs/superpowers/specs/2026-10-13-middleware-settings-design.md @@ -166,11 +166,6 @@ Four existing tests list every middleware that the test module supplies, and eac The results of the tests are shown by the checks on the pull request. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/38-middleware-settings` at `0da31d327`. Issues found, two of them blocking. A table under a quoted key that holds a `.` is accepted and never read, and the sentence saying that a module is not selected is missing whenever the check stops first at a first part of the module's name, a table nobody wrote. This revision states both behaviors as the code has them today and puts the code change each needs under Open questions, fills the gaps in the rollout, the failure handling and the module author's notes, and corrects the header and the partly true claims, while the fixes to the code, its comments and its tests that the review advised wait for code changes. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `b63866e4d`, of the round 1 fixes and the new text. No blocking issues. This revision keeps `GET /health` out of the routes a refusal fails on Fastly and Spin and says the `500` on Axum lasts until the process restarts, names the first part of the name in the refusal for a builder a shared table runs, says that only the assertion on the `testing.nothing` message proves the list of supplied names, and names the three code comments left for a later change. - ## Revision record | Date | Change | @@ -178,3 +173,8 @@ The results of the tests are shown by the checks on the pull request. | 2026-10-15 | First draft, written after the code. | | 2026-10-15 | Fixes from review round 1. The status date and the issue, the quoted key and the order of the startup check stated as the code has them with open questions, what runs a module, how often a table is read on each adapter, what a refusal does to every route, the rollout order and the stored pages, the notes for module authors, the crate under `crates/middleware`, and one sentence of the configuration guide. | | 2026-10-15 | Corrections from review round 2. `GET /health` and how long a refusal lasts on each adapter, the name the refusal gives for a builder a shared table runs, what the supplied-names assertion proves, and the three code comments left for a later change. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/38-middleware-settings` at `0da31d327`. Issues found, two of them blocking. A table under a quoted key that holds a `.` is accepted and never read, and the sentence saying that a module is not selected is missing whenever the check stops first at a first part of the module's name, a table nobody wrote. This revision states both behaviors as the code has them today and puts the code change each needs under Open questions, fills the gaps in the rollout, the failure handling and the module author's notes, and corrects the header and the partly true claims, while the fixes to the code, its comments and its tests that the review advised wait for code changes. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `b63866e4d`, of the round 1 fixes and the new text. No blocking issues. This revision keeps `GET /health` out of the routes a refusal fails on Fastly and Spin and says the `500` on Axum lasts until the process restarts, names the first part of the name in the refusal for a builder a shared table runs, says that only the assertion on the `testing.nothing` message proves the list of supplied names, and names the three code comments left for a later change. diff --git a/docs/superpowers/specs/2026-10-13-origin-redirect-design.md b/docs/superpowers/specs/2026-10-13-origin-redirect-design.md index a0ef209ac8..d7803311ce 100644 --- a/docs/superpowers/specs/2026-10-13-origin-redirect-design.md +++ b/docs/superpowers/specs/2026-10-13-origin-redirect-design.md @@ -166,11 +166,6 @@ Nothing tests the following: Sixteen deliberate faults were made, one in each run, and each was caught by a test: a `Location` rewritten on any status, the port left out of the comparison, the origin path not removed, the origin path removed where it does not end at a `/`, a `Location` outside the origin path rewritten with its path whole, the fragment dropped, the query dropped, an address that is not `http` or `https` rewritten, a request with no host rewritten, a `Location` naming no host resolved against the origin, the host of `origin_url` compared where an override is set, a `Location` with the scheme left off not read, the host compared in exact letter case, the reader always sent to `https`, the origin path not passed by the handler, and a `Location` that is the origin path itself rewritten with an empty path and not `/`. All sixteen are in the redirect rule and the handler's use of the rule, and none is recorded for the headers step. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/37-origin-redirect` at `73801ddb7`. Issues found, three of them blocking. The design called the serve middleware headers step outside the change though the series adds it, said the step is asked about every response the publisher path sends though three kinds of response skip it, and said nothing of the reader of a publisher served under a path prefix, who is sent to an address without the prefix. This revision adds a section for the step with its tests, lists the responses the step skips, says a serve middleware and not core puts the prefix back, corrects the partly true claims, fills the gaps in coverage, takes the advice the round gave, and corrects the same sentence in the changelog and the configuration guide. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `9dafb089e`, of the round 1 fixes and the new text. Approved, with no blocking issues. The changelog and the configuration guide now name the two loops and the `421` for a reader under a removed path prefix, and name all three kinds of response the headers step is not called for. - ## Revision record | Date | Change | @@ -179,3 +174,8 @@ Sixteen deliberate faults were made, one in each run, and each was caught by a t | 2026-10-14 | Names its pull request and issue, and points a deployment that serves two names of the site at the serve middleware headers step. | | 2026-10-15 | Fixes from review round 1. The headers step as a section with its tests and the responses it skips, who puts a removed path prefix back, the adapters, the setting, consent, failure and the index file loop, a table of what names the origin, the title now saying host, and the same sentence corrected in the changelog and the configuration guide. | | 2026-10-15 | Corrections from review round 2. The two loops and the `421` under a removed path prefix, and the three kinds of response the headers step is not called for, named in the changelog and the configuration guide. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/37-origin-redirect` at `73801ddb7`. Issues found, three of them blocking. The design called the serve middleware headers step outside the change though the series adds it, said the step is asked about every response the publisher path sends though three kinds of response skip it, and said nothing of the reader of a publisher served under a path prefix, who is sent to an address without the prefix. This revision adds a section for the step with its tests, lists the responses the step skips, says a serve middleware and not core puts the prefix back, corrects the partly true claims, fills the gaps in coverage, takes the advice the round gave, and corrects the same sentence in the changelog and the configuration guide. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `9dafb089e`, of the round 1 fixes and the new text. Approved, with no blocking issues. The changelog and the configuration guide now name the two loops and the `421` for a reader under a removed path prefix, and name all three kinds of response the headers step is not called for. diff --git a/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md b/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md index f6409668af..00e89b1b07 100644 --- a/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md +++ b/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md @@ -137,11 +137,6 @@ The tests of the publisher path are in the `object_storage_origin_tests` module 3. The refusals of a `type` or a field the table does not know, and of an `origin_auth` that is not a table, are checked only for the text `origin_auth`, which the parser's own echo of the document line satisfies, so no test checks the full name `publisher.origin_auth` for them. 4. No test combines a dot segment with `origin_auth`, sets `origin_host_header_override` on a signed origin, gives the closed path check an escape that is not valid UTF-8, or sends a signed read whose query or path holds a character the URL parser encodes. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/36-object-storage` at `f5f8ac2d0`. Issues found, four of them blocking. Section 2 counted the path of `origin_url` as an optional setting that is off unless set, when it changes the requests of every deployment whose document has one, left out that the closed path check now answers `404` to a path still changing after four rounds of percent-decoding on every deployment, said the query is always kept when `origin_query = "strip"` leaves it off, and said no setting is a vendor's when AWS signing is. This revision states both changes as breaking with the upgrade and rollback order, puts `origin_query` in the design, says that this series adds the AWS signing setting where core already kept AWS signing and that a later series moves it, and adds the adapters, the `Host` that is signed, what a signed read sends, what a reader sees when a read is refused, the exact rule for `directory_index`, the tests that are missing and the pull request that holds the stack's design documents. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `1465a29d1`, of the round 1 fixes and the new text. One blocking issue, in the changelog, and three smaller ones. This revision moves the path in `origin_url` and the closed path change into the Changed section of the changelog as breaking entries and corrects the changelog entry's count of settings, says that this series adds the `x-amz-` header rule to core, says that signing rebuilds the address through the URL parser and that a signed origin answers a method other than `GET` or `HEAD` with `405` before the dot segment check, and attributes what the page says about S3 to AWS's documentation. - ## Revision record | Date | Change | @@ -150,3 +145,8 @@ The tests of the publisher path are in the `object_storage_origin_tests` module | 2026-10-14 | Names its draft pull request and issue. | | 2026-10-15 | Fixes from review round 1. The path of `origin_url` and the closed path check as breaking changes with the upgrade and rollback order, `origin_query`, the vendor claim, the adapters and the `Host` that is signed, what a signed read sends and what a reader sees on a refusal, the rule for `directory_index`, the missing tests and the header. | | 2026-10-15 | Corrections from review round 2. The changelog entry, the `x-amz-` rule this series adds, the rebuilt address, the methods the `400` applies to, and the statements about S3 attributed to AWS's documentation. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/36-object-storage` at `f5f8ac2d0`. Issues found, four of them blocking. Section 2 counted the path of `origin_url` as an optional setting that is off unless set, when it changes the requests of every deployment whose document has one, left out that the closed path check now answers `404` to a path still changing after four rounds of percent-decoding on every deployment, said the query is always kept when `origin_query = "strip"` leaves it off, and said no setting is a vendor's when AWS signing is. This revision states both changes as breaking with the upgrade and rollback order, puts `origin_query` in the design, says that this series adds the AWS signing setting where core already kept AWS signing and that a later series moves it, and adds the adapters, the `Host` that is signed, what a signed read sends, what a reader sees when a read is refused, the exact rule for `directory_index`, the tests that are missing and the pull request that holds the stack's design documents. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `1465a29d1`, of the round 1 fixes and the new text. One blocking issue, in the changelog, and three smaller ones. This revision moves the path in `origin_url` and the closed path change into the Changed section of the changelog as breaking entries and corrects the changelog entry's count of settings, says that this series adds the `x-amz-` header rule to core, says that signing rebuilds the address through the URL parser and that a signed origin answers a method other than `GET` or `HEAD` with `405` before the dot segment check, and attributes what the page says about S3 to AWS's documentation. diff --git a/docs/superpowers/specs/2026-10-13-request-addressing-design.md b/docs/superpowers/specs/2026-10-13-request-addressing-design.md index 443b833296..f1ca4eda12 100644 --- a/docs/superpowers/specs/2026-10-13-request-addressing-design.md +++ b/docs/superpowers/specs/2026-10-13-request-addressing-design.md @@ -114,11 +114,6 @@ Tests in `crates/trusted-server-core/src/http_util.rs` cover the host read from Two paths have no test. No test drives `edgezero_main` with a hook registered through `run_with_addressing`, which appears only in a doctest that is not run, and no test covers the `500` that `addressed_router` answers when the continuing path cannot be set on the request. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/35-request-addressing` at `0af81b181`. Issues found, two of them blocking. "Putting the removed path prefix back" left out that a page whose removed path prefix is put back is never stored and is sent `private, no-store`, and section 3 claimed publishers on one host could be kept apart in the browser when the consent cookies, the `sharedId` cookie and the forwarded Cookie header are shared. Both are fixed in this revision, with two of the three partly true claims, the coverage gaps and the advice the round gave. The third partly true claim concerned the copy of this page on `split/6-integration-seam`, which still lists the rules in full where this page points to the guide and the rustdoc, and that copy is brought up to date in the specs pull request IABTechLab/trusted-server#1342. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `d58b3f404`, of the round 1 fixes and the new text. No blocking issues. This revision corrects the claim that every page whose removed path prefix is put back is fetched from the origin for each reader, limits the rollout order to a Fastly service whose selector carries `{host}`, says in the round 1 entry that the copy of this page on `split/6-integration-seam` is brought up to date in the specs pull request, and writes "removed path prefix" wherever the text means it. - ## Revision record | Date | Change | @@ -127,3 +122,8 @@ Two paths have no test. No test drives `edgezero_main` with a hook registered th | 2026-10-15 | Fixes from review round 1. The cost of putting the removed path prefix back, the shared browser origin as a limit, the host each adapter gives the hook, failure handling, rollout, the untested paths and the review log. | | 2026-10-15 | Corrections from review round 2. The origin fetch of a page whose removed path prefix is put back, the rollout order on Fastly, where the copy on `split/6-integration-seam` is brought up to date, and the removed path prefix named in full. | | 2026-10-17 | The host core processes a request for falls back to the URI's authority where no header names one (IABTechLab/trusted-server#1118), with its tests. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/35-request-addressing` at `0af81b181`. Issues found, two of them blocking. "Putting the removed path prefix back" left out that a page whose removed path prefix is put back is never stored and is sent `private, no-store`, and section 3 claimed publishers on one host could be kept apart in the browser when the consent cookies, the `sharedId` cookie and the forwarded Cookie header are shared. Both are fixed in this revision, with two of the three partly true claims, the coverage gaps and the advice the round gave. The third partly true claim concerned the copy of this page on `split/6-integration-seam`, which still lists the rules in full where this page points to the guide and the rustdoc, and that copy is brought up to date in the specs pull request IABTechLab/trusted-server#1342. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `d58b3f404`, of the round 1 fixes and the new text. No blocking issues. This revision corrects the claim that every page whose removed path prefix is put back is fetched from the origin for each reader, limits the rollout order to a Fastly service whose selector carries `{host}`, says in the round 1 entry that the copy of this page on `split/6-integration-seam` is brought up to date in the specs pull request, and writes "removed path prefix" wherever the text means it. diff --git a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md index fb9e6e536c..fc26e91461 100644 --- a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md +++ b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md @@ -304,11 +304,6 @@ Six things have no test. they do not run for `wasm32-wasip1`, although the Fastly adapter is the only one that runs the filter. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/44-client-hints` at `1f6b83d0c`. Issues found, four of them blocking. The four were that the request filter runs wherever the table is present and not only where an entry covers the page, that the cost to existing pages was not stated, that failure handling and the timeout of each adapter were not stated, and that nothing said what each navigation sends to the service or what consent it needs. This revision states each from the code, which also shows that only the Fastly adapter runs request filters, and it corrects the claims about page navigations, the headers and the tests, replaces a related spec that exists in no branch, and corrects the guide, the changelog and the middleware readme. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `126347b00`, of the round 1 fixes and the new text. No blocking issues once the pull request number was corrected. This revision names the pull request that now holds every design document, makes the guide's description of the wait for the first call match the one second connection limit and the limit on each gap between bytes, and lists in section 3 the code comments that still say more than the design does. - ## Revision record | Date | Change | @@ -317,3 +312,8 @@ Six things have no test. | 2026-10-15 | Fixes from review round 1. What the filter runs on and that only the Fastly adapter runs it, the cost to pages, what each navigation sends and the open questions on consent, how each adapter bounds the call, the answers on failure, the settings, what the headers keep, the definition of a page navigation, the untested paths, the related specs and the review log. | | 2026-10-15 | Fixes from review round 2. The pull request that holds the design documents, the wait for the first call in the guide, and the code comments left for a later change. | | 2026-10-15 | Code fixes for the call's scope, what it sends and header merging. The permission question stays open, with what was found, and the answer lifetime is a new open question. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/44-client-hints` at `1f6b83d0c`. Issues found, four of them blocking. The four were that the request filter runs wherever the table is present and not only where an entry covers the page, that the cost to existing pages was not stated, that failure handling and the timeout of each adapter were not stated, and that nothing said what each navigation sends to the service or what consent it needs. This revision states each from the code, which also shows that only the Fastly adapter runs request filters, and it corrects the claims about page navigations, the headers and the tests, replaces a related spec that exists in no branch, and corrects the guide, the changelog and the middleware readme. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `126347b00`, of the round 1 fixes and the new text. No blocking issues once the pull request number was corrected. This revision names the pull request that now holds every design document, makes the guide's description of the wait for the first call match the one second connection limit and the limit on each gap between bytes, and lists in section 3 the code comments that still say more than the design does. diff --git a/docs/superpowers/specs/2026-10-14-auction-protocol-design.md b/docs/superpowers/specs/2026-10-14-auction-protocol-design.md index b7eef81254..3f30f1063b 100644 --- a/docs/superpowers/specs/2026-10-14-auction-protocol-design.md +++ b/docs/superpowers/specs/2026-10-14-auction-protocol-design.md @@ -81,11 +81,6 @@ The protocol crate has a `test-utils` Cargo feature, which turns on core's featu Some behavior has no test. A source on another protocol being a module is shown only by core's `#[cfg(test)]` stand-ins, because the test crate `crates/testing/iabtl` wraps the OpenRTB driver and the only direct implementation of `CompiledDemand` outside core is the driver. No test shows a source that skips the admission helpers being stopped, because nothing in core stops it. No test sends the OpenRTB reader a non-success status other than the `302`, and none sends it a body over 1 MiB, although core's tests send one to a stand-in reader. No test runs an identifier whose `ext` is not an object through `format_eids`. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/45-openrtb` at `82414cad4`. Issues found, three of them blocking. The spec named `ad-server.mock`, which does not exist, said the Prebid EID cookie parser and the consent decoders move in series of their own when they had already moved in the two series below this one, and said a reader in any crate admits a bid by the rules core applies when core only offers the rules and each reader has to call them. All three are fixed in this revision, with the partly true claims, the coverage gaps and the advice the round gave, except that the two stale code comments at `auction/plan.rs:31` and `prebid-server/src/lib.rs:7397` are left for a later change, and whether core should enforce admission is left as an open question. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `6a4416305`, of the round 1 fixes and the new text. No blocking issues. The related specs line no longer points at the closed specs pull request (IABTechLab/trusted-server#1084), the note on an identifier `ext` that is not an object says the `ext` is dropped and the identifier is still sent, and the round 1 entry says the two stale code comments are left for a later change. - ## Revision record | Date | Change | @@ -93,3 +88,8 @@ Some behavior has no test. A source on another protocol being a module is shown | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The module name `ad-server.iabtl`, the title, where `Eid` and the consent names sit and how that differs from two related specs, admission as helpers each reader calls with an open question, new text on consent and privacy, signing, failure, compatibility, adapters and the missing tests, and the review log. | | 2026-10-15 | Corrections from review round 2. The related specs line, the note on an identifier `ext` that is not an object, the round 1 entry's claim about two code comments, and the round 2 entry in the review log. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/45-openrtb` at `82414cad4`. Issues found, three of them blocking. The spec named `ad-server.mock`, which does not exist, said the Prebid EID cookie parser and the consent decoders move in series of their own when they had already moved in the two series below this one, and said a reader in any crate admits a bid by the rules core applies when core only offers the rules and each reader has to call them. All three are fixed in this revision, with the partly true claims, the coverage gaps and the advice the round gave, except that the two stale code comments at `auction/plan.rs:31` and `prebid-server/src/lib.rs:7397` are left for a later change, and whether core should enforce admission is left as an open question. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `6a4416305`, of the round 1 fixes and the new text. No blocking issues. The related specs line no longer points at the closed specs pull request (IABTechLab/trusted-server#1084), the note on an identifier `ext` that is not an object says the `ext` is dropped and the identifier is still sent, and the round 1 entry says the two stale code comments are left for a later change. diff --git a/docs/superpowers/specs/2026-10-14-backend-naming-design.md b/docs/superpowers/specs/2026-10-14-backend-naming-design.md index 1dfd81b1fe..71f28f21ae 100644 --- a/docs/superpowers/specs/2026-10-14-backend-naming-design.md +++ b/docs/superpowers/specs/2026-10-14-backend-naming-design.md @@ -62,12 +62,6 @@ This change makes the first two. The third has no call to move because the comma - Not tested. The bucket count is compared with the distinct timers only at a 2,000 ms ceiling, where the coarse ladder is never reached. At a 60,000 ms ceiling the ladder test bounds the distinct timers at 20 and one assertion checks that two providers fit the budget, but nothing asserts that the count equals the timers the ladder reaches. - Not tested. Axum and Fastly have no startup test of a plan being refused. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/47-backend-naming` at `2dbc2d731`. Issues found, one of them blocking. The opening note and the problem section named the `ts` command as one of three callers of the policy, when the command never called it and the callers were each adapter's startup validation and its registration. This revision names the real callers and fixes the two partly true claims, the coverage gaps and the advice the round gave. - -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `969905300`, of the round 1 fixes and the new text. Approved, with no blocking issues. This revision adds the descriptor changes the compatibility paragraph left out, names the Tech Lab draft pull request, and counts the changed orchestrator assertions as two. - ## Revision record | Date | Change | @@ -76,3 +70,9 @@ This change makes the first two. The third has no call to move because the comma | 2026-10-15 | Fixes from review round 1. The real callers of the policy, the break for crates outside core, the descriptor resting on convention, the timer bucket default, the untested paths, the platform names left in core, what the issue asks, the pull request references and the review log. | | 2026-10-15 | Fixes from review round 2. The descriptor changes the compatibility paragraph left out, the Tech Lab pull request reference and the count of changed orchestrator assertions. | | 2026-10-17 | The certificate check (IABTechLab/trusted-server#1132). The Axum adapter warns once at startup that it does not honor `proxy.certificate_check = false`, with its test, and the guides say which adapters honor the setting. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/47-backend-naming` at `2dbc2d731`. Issues found, one of them blocking. The opening note and the problem section named the `ts` command as one of three callers of the policy, when the command never called it and the callers were each adapter's startup validation and its registration. This revision names the real callers and fixes the two partly true claims, the coverage gaps and the advice the round gave. + +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `969905300`, of the round 1 fixes and the new text. Approved, with no blocking issues. This revision adds the descriptor changes the compatibility paragraph left out, names the Tech Lab draft pull request, and counts the changed orchestrator assertions as two. diff --git a/docs/superpowers/specs/2026-10-14-consent-decoders-design.md b/docs/superpowers/specs/2026-10-14-consent-decoders-design.md index d9f464ad22..230e8efdcc 100644 --- a/docs/superpowers/specs/2026-10-14-consent-decoders-design.md +++ b/docs/superpowers/specs/2026-10-14-consent-decoders-design.md @@ -94,11 +94,6 @@ The running configuration page (`/_ts/config`) shows `[consent]` as `mode` and t 4. A record written by an earlier build. The identity graph test reads a record under the key `scheme-a`, not under `tcf` or `gpp`, and the test of a record carrying removed fields names `scheme-a` where it named the removed field `first`. 5. A module whose `read` is not given the evidence or the settings it names. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/42-consent-decoders` at `564788e6e`. Issues found, three of them blocking. The design said the Cloudflare adapter read core's decoded fields when no adapter's source did, wrote the move of the auction gate to the resolved permissions as a move when it changes the TCF purpose checked and which requests pass, and said core reads `mode` when core only parses it. This revision corrects the three and the claims about unknown keys and Additional Consent, adds the adapters, defaults, failure paths, rollout, cost and public items, lists the tests the change lacks, and corrects the guide sentences that said every module reads `mode` and that core decodes the InMobi cookie. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `f8ffb2170`, of the round 1 fixes and the new text. One blocking issue, a sixth change to the auction gate, and four smaller ones. This revision adds the sixth difference, for an EU signal that only GPP carries, to the gate list, the rollout and the open question, corrects the failure paragraph for an EU section inside a GPP string, cites the open specs pull request, completes the expired string item, lists the stale core comments as left for a later change, and corrects the CHANGELOG entry. - ## Revision record | Date | Change | @@ -107,3 +102,8 @@ The running configuration page (`/_ts/config`) shows `[consent]` as `mode` and t | 2026-10-15 | Fixes from review round 1. The auction gate described as a change with its rollout effect and an open question, the adapters and tools, the defaults and what is no longer checked, the failure paths, the cost, the public items that go, the untested paths and the review log. | | 2026-10-15 | Fixes from review round 2. The sixth gate difference with its rollout effect and the open question, the failure paths for an EU section in a GPP string, the specs pull request cited, the expired string item, the stale core comments listed and the CHANGELOG entry corrected. | | 2026-10-15 | Fixes to the auction gate and to unreadable strings. The gate reads the records again, which carry the permissions a signal consents to, so it asks for Purpose 1, and an expired TC String, an EU signal that only GPP carries, the policy, the Model Terms grant, place rules and opt-outs leave it as the gate it replaced. An EU section of a GPP string that cannot be read refuses every purpose TCF maps. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/42-consent-decoders` at `564788e6e`. Issues found, three of them blocking. The design said the Cloudflare adapter read core's decoded fields when no adapter's source did, wrote the move of the auction gate to the resolved permissions as a move when it changes the TCF purpose checked and which requests pass, and said core reads `mode` when core only parses it. This revision corrects the three and the claims about unknown keys and Additional Consent, adds the adapters, defaults, failure paths, rollout, cost and public items, lists the tests the change lacks, and corrects the guide sentences that said every module reads `mode` and that core decodes the InMobi cookie. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `f8ffb2170`, of the round 1 fixes and the new text. One blocking issue, a sixth change to the auction gate, and four smaller ones. This revision adds the sixth difference, for an EU signal that only GPP carries, to the gate list, the rollout and the open question, corrects the failure paragraph for an EU section inside a GPP string, cites the open specs pull request, completes the expired string item, lists the stale core comments as left for a later change, and corrects the CHANGELOG entry. diff --git a/docs/superpowers/specs/2026-10-14-identifier-modules-design.md b/docs/superpowers/specs/2026-10-14-identifier-modules-design.md index 6597f7e96c..83b38a2b21 100644 --- a/docs/superpowers/specs/2026-10-14-identifier-modules-design.md +++ b/docs/superpowers/specs/2026-10-14-identifier-modules-design.md @@ -90,11 +90,6 @@ These are for the reviewer. This revision records what the code does and decides 5. Should a follow-up move the `sharedId` cookie and the `sharedid.org` source domain out of core? 6. Should a check tie the script's cookie to the Prebid module's, as `the_client_script_reads_the_attribute_the_server_writes` in `crates/trusted-server-core/src/prefix.rs` does for the public prefix attribute? -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/41-prebid-eids` at `e87090be0`. Issues found, four of them blocking. The blocking issues were that decoding moved onto every read of the Edge Cookie state without the spec saying so, that the adapters were unspecified and their wiring untested, that the template cache rule refused names that loaded before with no compatibility analysis, and that the opening sentence said core applies the script's size limit when core accepts 8 KiB and the script writes 3072 bytes. This revision states the per-request decoding and its cost, adds the adapter table and the joins with no test, lists what now fails to load, corrects the size limits, the locations of the cookie's name, `sharedId` and `Eid`, one sentence of the Edge Cookies guide and one cell of the integration guide, and puts the choices it does not make under open questions. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `d6ca6df80`, of the round 1 fixes and the new text. Approved, with no blocking issues. This revision corrects the account of how the cookie is parsed and how the request body treats a long source name, names the requests answered before the read, aligns one paragraph of the integration guide with the spec, and names the pull requests in the header. - ## Revision record | Date | Change | @@ -102,3 +97,8 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The decoding on every read and its cost, the adapter table, the template cache rule's effects, the size limits, the locations of the cookie's name, `sharedId` and `Eid`, the permission order, failure and merge behavior, the untested joins, one sentence of the Edge Cookies guide, one cell of the integration guide, the open questions and the review log. | | 2026-10-15 | Fixes from review round 2. How the cookie is parsed, how the request body treats a long source name, the requests answered before the read, one paragraph of the integration guide, and the pull requests in the header. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/41-prebid-eids` at `e87090be0`. Issues found, four of them blocking. The blocking issues were that decoding moved onto every read of the Edge Cookie state without the spec saying so, that the adapters were unspecified and their wiring untested, that the template cache rule refused names that loaded before with no compatibility analysis, and that the opening sentence said core applies the script's size limit when core accepts 8 KiB and the script writes 3072 bytes. This revision states the per-request decoding and its cost, adds the adapter table and the joins with no test, lists what now fails to load, corrects the size limits, the locations of the cookie's name, `sharedId` and `Eid`, one sentence of the Edge Cookies guide and one cell of the integration guide, and puts the choices it does not make under open questions. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `d6ca6df80`, of the round 1 fixes and the new text. Approved, with no blocking issues. This revision corrects the account of how the cookie is parsed and how the request body treats a long source name, names the requests answered before the read, aligns one paragraph of the integration guide with the spec, and names the pull requests in the header. diff --git a/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md b/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md index fe6ef2cca0..d7dce5f787 100644 --- a/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md +++ b/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md @@ -126,11 +126,6 @@ Not tested: - A name written with its type folder for a module type other than analytics. No core test covers `check_tables` or `Settings::module_config` with such a name, and the Tinybird crate covers them through the settings load and the module's registration only. Nor does any test select a module of another type that way through the deploy check of its secret or the load of a stored configuration. - A declared name that is not a valid header name, which no test declares. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/46-last-names` at `8110daa5b`. Issues found, four of them blocking. The page named `crates/trusted-server-modules`, which no longer exists, said nothing changes for an operator when each adapter now strips a different set of cache headers, said nothing of the fact that only the Fastly adapter sends auction telemetry, and left out the core change that lets a section name a module with its own type folder. This revision states how each adapter and the `ts` tool list the module, which headers each adapter declares and strips with the narrowing put as an open question, that only the Fastly adapter sends, and the core change with the places that still read a table by the name as written, together with the partly true and unsupported claims, the coverage gaps and the advice the round gave, and it corrects the same claims in the changelog and the telemetry guide. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `3286cb534`, of the round 1 fixes and the new text. No blocking issues. This revision says plainly that the full name of the analytics module fails `ts config validate` and the load of a stored configuration today and lists the code fix with the open question, limits the selection test to the settings load and the module's registration, and adds to the names left in core that core reads `fastly-ssl` as a signal of the scheme and removes the header before calling the origin. - ## Revision record | Date | Change | @@ -138,3 +133,8 @@ Not tested: | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The module lists of each adapter and of the `ts` tool, the headers each adapter declares and strips with the narrowing as an open question, the adapters that send telemetry, the table change for a name written with its type folder, the defaults and bounds, what leaves core, the rollout, the names left in core, the untested paths and the review log. | | 2026-10-15 | Corrections from review round 2. The full name of the analytics module failing the deploy check and the load of a stored configuration, the code fix under the open question, what the selection test reaches, and `fastly-ssl` among the names left in core. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/46-last-names` at `8110daa5b`. Issues found, four of them blocking. The page named `crates/trusted-server-modules`, which no longer exists, said nothing changes for an operator when each adapter now strips a different set of cache headers, said nothing of the fact that only the Fastly adapter sends auction telemetry, and left out the core change that lets a section name a module with its own type folder. This revision states how each adapter and the `ts` tool list the module, which headers each adapter declares and strips with the narrowing put as an open question, that only the Fastly adapter sends, and the core change with the places that still read a table by the name as written, together with the partly true and unsupported claims, the coverage gaps and the advice the round gave, and it corrects the same claims in the changelog and the telemetry guide. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `3286cb534`, of the round 1 fixes and the new text. No blocking issues. This revision says plainly that the full name of the analytics module fails `ts config validate` and the load of a stored configuration today and lists the code fix with the open question, limits the selection test to the settings load and the module's registration, and adds to the names left in core that core reads `fastly-ssl` as a signal of the scheme and removes the header before calling the origin. diff --git a/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md b/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md index 6f33c954b7..b90dc83ba7 100644 --- a/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md +++ b/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md @@ -86,11 +86,6 @@ The seam probe is a test fixture that no deployment ships, so its renames reach Three things have no test. The first is the five lists agreeing. Each adapter's order test pins its own list against a copy of the same twenty names, so the four lists agree today, but nothing compares one list with another, and the `ts` list has no order test of its own. Its order is checked only through `build_draft_config_places_the_page_changes_of_the_modules_it_selects`, for the modules that test selects, and the integration tests crate validates its fixtures against the Axum adapter's list alone. Adding a module takes five edits and a missed one passes every test, so a test that gives the same ids in the same order from all five lists is missing. The second is the registry accepting a described middleware that the registration leaves out, because no test has a registration omit a middleware its builder describes. The third is `ts audit` leaving out a middleware described for another media type, because the draft test that checks the entries selects `gpt`, `datadome`, `didomi` and `google_tag_manager`, whose middleware are all described for `text/html`, and `build_page_entries` has no direct test. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/43-no-modules-crate` at `3ad378928`. Approved with no blocking issues. This revision closes the coverage gaps by naming the four adapters and the three list functions with where each is exported, saying what stays the same for a deployment that keeps the stock set, saying that deploy validation never builds the registry so a wrong description is refused only at startup and then stops the whole service, giving the order of work for the renamed selectors, and recording the missing test that ties the five lists together. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `0fdbaf5dc`, of the round 1 fixes and the new text. No blocking issues. This revision stops claiming a test for `ts audit` leaving out a middleware described for another media type and lists the gap as a third untested item, records the three small changes left for a later change, and corrects the two older changelog entries that still named the removed crate. - ## Revision record | Date | Change | @@ -98,3 +93,8 @@ Three things have no test. The first is the five lists agreeing. Each adapter's | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The other users of the old crate, the test that kept its table in step, the renamed packages and the seam probe's names, the adapters and the list functions, what does not change, the startup refusal, the rollout of the renamed selectors, what `ts audit` reads and writes, the missing tests, `iabtl` and the stack defined, the renames in the title, and the pull requests raised. | | 2026-10-15 | Fixes from review round 2. The claimed test for the media types `ts audit` writes, the three changes left for a later change, and the changelog entries that still named the removed crate. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/43-no-modules-crate` at `3ad378928`. Approved with no blocking issues. This revision closes the coverage gaps by naming the four adapters and the three list functions with where each is exported, saying what stays the same for a deployment that keeps the stock set, saying that deploy validation never builds the registry so a wrong description is refused only at startup and then stops the whole service, giving the order of work for the renamed selectors, and recording the missing test that ties the five lists together. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `0fdbaf5dc`, of the round 1 fixes and the new text. No blocking issues. This revision stops claiming a test for `ts audit` leaving out a middleware described for another media type and lists the gap as a third untested item, records the three small changes left for a later change, and corrects the two older changelog entries that still named the removed crate. diff --git a/docs/superpowers/specs/2026-10-14-origin-signing-design.md b/docs/superpowers/specs/2026-10-14-origin-signing-design.md index a312cafb7f..eb7db5986d 100644 --- a/docs/superpowers/specs/2026-10-14-origin-signing-design.md +++ b/docs/superpowers/specs/2026-10-14-origin-signing-design.md @@ -92,11 +92,6 @@ Four cases have no test that can fail: 3. No deploy validation test gives an `s3_sigv4` table an empty `session_token`. The refusal is tested only through the core fixture signer's `token_name`, a setting declared the same way. 4. No test in the `ts` tool's crate names the AWS builder or an `s3_sigv4` table, so nothing fails if the builder leaves the list in `linked_builders()`, and `ts config validate` would then skip every `s3_sigv4` table. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/48-aws-signing` at `a3c17125b`. Issues found, two of them blocking. The section on AWS signing named a modules crate that no longer exists and deploy rules the builder does not carry, and the spec left out that a `type` nothing offers passes `ts config validate` and `ts config push` and only stops the adapter at startup, while it listed the removed `s3_sig_v4` spelling under what is not in the change. This revision corrects both, states which stage refuses each mistake, what a failed signature answers, the rollout, the adapters that run each path, the settings that change and the tests that are missing or cannot fail, leaves closing the gap as an open question, and corrects the changelog, integration guide and configuration guide sentences that repeated the stale claims. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `5c43f6f9a`, of the round 1 fixes and the new text. No blocking issues once the pull request number was corrected. This revision names IABTechLab/trusted-server#1342 as the specs pull request, says that the preflight signs without calling `remove_namespace_headers`, leaves `GET /health` and the Fastly debug probes out of the startup error answer, limits the missing test for an empty `session_token` to deploy validation, and corrects the Tinybird entry in the changelog. - ## Revision record | Date | Change | @@ -104,3 +99,8 @@ Four cases have no test that can fail: | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The stale modules crate and deploy rules claims, the stages that refuse a mistake, the gap and open question for a `type` nothing offers, the breaking removal of `s3_sig_v4`, failure answers, rollout, the adapters that run each path, the missing tests and the review log. | | 2026-10-15 | Corrections from review round 2. The specs pull request number, the preflight's signing without removing namespace headers, the requests an adapter answers whatever its startup error, the deploy validation test for an empty token and the Tinybird changelog entry. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/48-aws-signing` at `a3c17125b`. Issues found, two of them blocking. The section on AWS signing named a modules crate that no longer exists and deploy rules the builder does not carry, and the spec left out that a `type` nothing offers passes `ts config validate` and `ts config push` and only stops the adapter at startup, while it listed the removed `s3_sig_v4` spelling under what is not in the change. This revision corrects both, states which stage refuses each mistake, what a failed signature answers, the rollout, the adapters that run each path, the settings that change and the tests that are missing or cannot fail, leaves closing the gap as an open question, and corrects the changelog, integration guide and configuration guide sentences that repeated the stale claims. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `5c43f6f9a`, of the round 1 fixes and the new text. No blocking issues once the pull request number was corrected. This revision names IABTechLab/trusted-server#1342 as the specs pull request, says that the preflight signs without calling `remove_namespace_headers`, leaves `GET /health` and the Fastly debug probes out of the startup error answer, limits the missing test for an empty `session_token` to deploy validation, and corrects the Tinybird entry in the changelog. diff --git a/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md b/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md index 3ff888787c..630be6452c 100644 --- a/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md +++ b/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md @@ -93,11 +93,6 @@ Seven cases have no test. 6. Any adapter. No adapter route test and no parity test sends a `text/x-component` response, and the shared Next.js fixture they use (`nextjs_auction` in `crates/trusted-server-core/src/test_support.rs`) selects the module with an entry for `text/html` alone. 7. The integration scenario `NextJsRscFlight` (`crates/trusted-server-integration-tests/tests/frameworks/scenarios.rs`), which sends a Flight request to the Next.js test origin. Before this change core rewrote every Flight response. The integration test configuration selects no `[framework]` module and has no `[[fetch]]` entry, so a Flight response in that scenario now meets core's plain replacement and not the middleware. The scenario checks only that the content type is not HTML and that the body holds no injected script, so it would pass with a broken `T` row. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/40-rsc-flight` at `b2124875a`. Issues found, one of them blocking. Section 2 said the middleware rewrites a Flight response as it was rewritten before and as core's plain replacement does, when the middleware builds its addresses from the origin host and so differs from both for a folder in the source's address, for another `Host` and for the `http://` form, and from core's replacement for a bare host inside a longer name. This revision states each difference and why the middleware builds from the host, and it adds the adapters, the configuration refusals, consent and privacy, failure, the rollout order with an open question for deployments that do not select the module, the cases no test covers and the advice the round gave. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `303981e49`, of the round 1 fixes and the new text. Approved, with no blocking issues. This revision corrects the hostname boundary difference, which holds only for a bare host, the `path` paragraph, which now names the validation that checks a `path` and the rule that picks the entry making an empty document, and the template comment, whose order now covers the HTML entries alone. - ## Revision record | Date | Change | @@ -105,3 +100,8 @@ Seven cases have no test. | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The differences from core's plain replacement and why, the alternatives to the new method, adapters, configuration, consent, failure, rollout with its open question, the cases no test covers and the review log. | | 2026-10-15 | Fixes from review round 2. The hostname boundary difference for a bare host alone, the `path` paragraph, the template comment and the review log. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/40-rsc-flight` at `b2124875a`. Issues found, one of them blocking. Section 2 said the middleware rewrites a Flight response as it was rewritten before and as core's plain replacement does, when the middleware builds its addresses from the origin host and so differs from both for a folder in the source's address, for another `Host` and for the `http://` form, and from core's replacement for a bare host inside a longer name. This revision states each difference and why the middleware builds from the host, and it adds the adapters, the configuration refusals, consent and privacy, failure, the rollout order with an open question for deployments that do not select the module, the cases no test covers and the advice the round gave. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `303981e49`, of the round 1 fixes and the new text. Approved, with no blocking issues. This revision corrects the hostname boundary difference, which holds only for a bare host, the `path` paragraph, which now names the validation that checks a `path` and the rule that picks the entry making an empty document, and the template comment, whose order now covers the HTML entries alone. diff --git a/docs/superpowers/specs/2026-10-14-source-modules-design.md b/docs/superpowers/specs/2026-10-14-source-modules-design.md index d31a7af3fb..ef9f31d921 100644 --- a/docs/superpowers/specs/2026-10-14-source-modules-design.md +++ b/docs/superpowers/specs/2026-10-14-source-modules-design.md @@ -158,11 +158,6 @@ Nothing tests these paths: - An `[inspect]` pattern that names an old path. - An old environment variable name through the overlay the `ts` command applies. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/39-source-modules` at `ff33fea74`. Issues found, three of them blocking. The claim that stored pages and assets keep their keys was false for pages and the publisher's assets, the change to the host that a page and every middleware are told as the page's origin host was missing, and the behavior the series adds to asset routes and to modules was missing. This revision corrects the claim and states the other two, and adds the configuration, adapter, consent and privacy, failure and rollout paragraphs, the paths that have no test and a known gap in the spelling of core's source. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `6eabcc695`, of the round 1 fixes and the new text. Approved, with no blocking issues. No change was needed. - ## Revision record | Date | Change | @@ -170,3 +165,8 @@ Nothing tests these paths: | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. What the stored keys hold, the host a page names itself by, the behavior added to asset routes and modules, configuration, adapters, consent and privacy, failure handling, rollout, the untested paths, the spelling of core's source and the review log. | | 2026-10-15 | Review round 2 recorded, with no change to the design. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/39-source-modules` at `ff33fea74`. Issues found, three of them blocking. The claim that stored pages and assets keep their keys was false for pages and the publisher's assets, the change to the host that a page and every middleware are told as the page's origin host was missing, and the behavior the series adds to asset routes and to modules was missing. This revision corrects the claim and states the other two, and adds the configuration, adapter, consent and privacy, failure and rollout paragraphs, the paths that have no test and a known gap in the spelling of core's source. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `6eabcc695`, of the round 1 fixes and the new text. Approved, with no blocking issues. No change was needed. diff --git a/docs/superpowers/specs/2026-10-15-asset-cache-design.md b/docs/superpowers/specs/2026-10-15-asset-cache-design.md index d7b24940b8..afbe9098af 100644 --- a/docs/superpowers/specs/2026-10-15-asset-cache-design.md +++ b/docs/superpowers/specs/2026-10-15-asset-cache-design.md @@ -199,11 +199,6 @@ Joins with no test: 8. **Reordered page checks.** `origin_response_may_be_kept` moves the status check ahead of the cookie and edge header checks in `template_cache_ttl`. A page response with a non-`200` status and a fault found by a later check is now reported as a non-`200` status, and no test was added for the changed order. 9. **A module declaring a cache of core's.** `a_builder_declaring_core_s_own_cache_is_refused` declares `robots-txt` only, so no test has a module declare `assets` or `assets-vary`. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/34-asset-cache` at `67af14ea4`. Issues found, one of them blocking. The asset gate leaves out the page path's cookie rules, so a request that carries a session cookie can read a stored copy or lead a load whose response is stored for every reader, and the spec did not say so. This revision states the cookie rule as the code has it today, with the reason the code gives and the default-on state, puts the choice of cookie rules and an off switch under Open questions, fills the coverage gaps and corrects the partly true claims, while the fixes to the code, its comments and its tests that the review advised wait for code changes. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, checking the round 1 fixes against `split/34-asset-cache` at `e9832a39e`. Issues found, none blocking. This revision states the cookie rule, the off switch and the range check of `[cache.assets]` as the code now has them, says which entries each load writes, removes the claim that no test sends a `Cookie` header through the publisher path, which navigation tests and now the asset gate's tests do, and gives the one linked module as the default in the guide and the code comments. - ## Revision record | Date | Change | @@ -212,3 +207,8 @@ Joins with no test: | 2026-10-15 | Fixes from review round 1. The cookie rules the asset gate leaves out and what that lets through, the reason the code gives, the default-on state and the open questions on cookie rules and an off switch, `Pragma` in the gate, the verdict a response that fails a test leaves for 60 seconds, the module default, where the sizes and the `[cache.assets-vary]` table stand, the key-value store each adapter uses, how far one fetch reaches, what the caches add on Fastly, `[[cache.asset_rules]]`, the headers a route forwards, the cost and limits, the test doubles and the missing tests. | | 2026-10-16 | Fixes to the code, and from review round 2. A request that sends the publisher's origin a cookie neither reads a copy nor leads a load, `[cache.assets] module = "none"` turns both asset caches off, and a `[cache.assets]` lifetime out of range is refused when the settings load, each with its tests. Which entries each load writes, the tests that send a cookie, and the module default in the guide and the code comments. | | 2026-10-17 | The code fix on row 24, where the Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests, and where each application state has its own memory layer, Cloudflare included. How far one fetch reaches, the adapters' shared stores, how long the memory layer lives, and the reads and writes of the shared store say so. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/34-asset-cache` at `67af14ea4`. Issues found, one of them blocking. The asset gate leaves out the page path's cookie rules, so a request that carries a session cookie can read a stored copy or lead a load whose response is stored for every reader, and the spec did not say so. This revision states the cookie rule as the code has it today, with the reason the code gives and the default-on state, puts the choice of cookie rules and an off switch under Open questions, fills the coverage gaps and corrects the partly true claims, while the fixes to the code, its comments and its tests that the review advised wait for code changes. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, checking the round 1 fixes against `split/34-asset-cache` at `e9832a39e`. Issues found, none blocking. This revision states the cookie rule, the off switch and the range check of `[cache.assets]` as the code now has them, says which entries each load writes, removes the claim that no test sends a `Cookie` header through the publisher path, which navigation tests and now the asset gate's tests do, and gives the one linked module as the default in the guide and the code comments. diff --git a/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md b/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md index a05cd3a327..15d22c0540 100644 --- a/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md +++ b/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md @@ -119,11 +119,6 @@ Two pieces of production code build the prefix the writers use, being `for_reque The tests listed under "Tests" in section 2 pass, including `dispatch_set_tester_writes_under_the_removed_prefix` in `crates/trusted-server-adapter-fastly/src/app.rs`, which `cargo test-fastly` runs. `finalize_writes_the_edge_cookie_under_the_removed_prefix` and `resolve_writes_under_the_removed_prefix_of_an_addressed_request` each fail when the production code they cover is reverted, as section 2 says. `cargo test -p trusted-server-core`, `cargo test-fastly`, `cargo clippy -p trusted-server-core --all-targets -- -D warnings`, `cargo clippy-fastly` and `cargo fmt --all -- --check` pass. With no `RemovedPathPrefix` on a request, or one that is empty or `/`, every `Set-Cookie` core writes is as before and the jar keeps the last pair of a name. The existing tests check the unchanged output byte for byte for the Edge Cookie and its expiry, the pull-sync marker and its expiry, and the tester cookie and its clearing, and check the resolved marker and its expiry for their `Path` only. On an addressed request a withdrawal expires the Edge Cookie and the resolved marker under the removed prefix and under `/`, checked byte for byte. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/50-cookies-by-path` at `185557ad2`. Issues found, five of them blocking. The five were the 421 answer for a refused removed prefix, which no adapter gives, the claim that the first response moves visitors' cookies under the publisher's path, an empty removed prefix that writes `Path=/` yet reads the first of two cookies of one name, text the Fastly tester change left stale, and tests that never reach the code that builds the prefix in production. This revision states what the code does today in each case, adds the consent tools that write `Path=/`, the signed origins, the module cookies and the nested prefixes the round found, and leaves the code changes the round asks for under Open questions. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `963bf8bd3`, of the round 1 fixes and the new text. No blocking issues. This revision corrects the claim that no test passes a removed prefix through the `EcContext` readers, names the twelve integrations of the client script that use root-relative paths or write cookies from the page where the text named five, and adds three more untrue statements to the list under Open questions. - ## Revision record | Date | Change | @@ -132,3 +127,8 @@ The tests listed under "Tests" in section 2 pass, including `dispatch_set_tester | 2026-10-15 | Fixes from review round 1. What a refused removed prefix becomes, the rollout for returning visitors, the jar rule for an empty prefix, the adapter table, tests and acceptance for the Fastly tester change, the code no test reaches, the consent tools that write `Path=/`, signed origins, the cookies left out, the open questions and the review log. | | 2026-10-15 | Corrections from review round 2. The tests that pass a removed prefix through the `EcContext` readers, the twelve integrations of the client script that use root-relative paths or write cookies, three more untrue statements under Open questions and the review log. | | 2026-10-15 | Code fixes for open questions 1 to 4. `RequestAddress::new` accepts any removed prefix again and `under_path` holds the one check, a removed prefix that is empty, `/` or not a cookie path reads the jar as no prefix does, tests reach the production wiring through the context and the resolve endpoint, and a withdrawal also expires the Edge Cookie and the resolved marker under `/`. The code comments open question 5 listed are corrected, and the remaining open question keeps the pages. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/50-cookies-by-path` at `185557ad2`. Issues found, five of them blocking. The five were the 421 answer for a refused removed prefix, which no adapter gives, the claim that the first response moves visitors' cookies under the publisher's path, an empty removed prefix that writes `Path=/` yet reads the first of two cookies of one name, text the Fastly tester change left stale, and tests that never reach the code that builds the prefix in production. This revision states what the code does today in each case, adds the consent tools that write `Path=/`, the signed origins, the module cookies and the nested prefixes the round found, and leaves the code changes the round asks for under Open questions. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `963bf8bd3`, of the round 1 fixes and the new text. No blocking issues. This revision corrects the claim that no test passes a removed prefix through the `EcContext` readers, names the twelve integrations of the client script that use root-relative paths or write cookies from the page where the text named five, and adds three more untrue statements to the list under Open questions. diff --git a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md index c41ec2b958..3814736860 100644 --- a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md +++ b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md @@ -132,11 +132,6 @@ Not tested: - Any change on the Axum, Cloudflare and Spin adapters listed in 2.1 to 2.4. No adapter test sends a client's `fastly-ssl`, `fastly-client-ip` or `x-debug-fastly-pop` to those adapters, or selects `module = "platform"` on them. - The strip at entry of a declared header through an adapter. Core's test declares example headers and calls `sanitize_forwarded_headers` directly, and the one adapter that calls the function declares none. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/49-host-features` at `69ba03219`. Issues found, two of them blocking. The page left out the change to `x-debug-fastly-pop` on the Axum, Cloudflare and Spin adapters, and described the spoofable header list as stripping headers although the one adapter that calls the function reading it declares nothing. This revision states both as the code has them today, adds the effects on the other adapters, the scheme change, where each refusal happens, the order for changing the package and the document, the name that stays and the untested paths, and corrects the partly true claims, whereas the two code fixes wait on the open questions in section 2.3. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `b319bf039`, of the round 1 fixes and the new text. No blocking issues. This revision names in the Fastly sentence of 2.3 that `ts config validate` no longer refuses `x-debug-fastly-pop`, corrects when the Axum, Cloudflare and Spin adapters strip `fastly-client-ip`, lists the comment at the top of `platform/request_headers.rs` among the comments left for a later change, and corrects the changelog entry on the spoofable strip while adding the scheme change and the `x-debug-fastly-pop` change on Fastly to it. - ## Revision record | Date | Change | @@ -144,3 +139,8 @@ Not tested: | 2026-10-15 | First draft. | | 2026-10-15 | Fixes from review round 1. The readers of each declared list and the open questions on them, `x-debug-fastly-pop`, the scheme change, the effects on the other adapters, where each refusal happens, the rollout, the narrowed constants, `x-compress-hint`, the file count, the untested paths and the review log. | | 2026-10-15 | Corrections from review round 2. The Fastly sentence of 2.3 and the changelog on the `x-debug-fastly-pop` refusal, the changelog on the spoofable strip and the scheme change, when the Axum, Cloudflare and Spin adapters strip `fastly-client-ip`, and the comment in `request_headers.rs` left for a later change. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/49-host-features` at `69ba03219`. Issues found, two of them blocking. The page left out the change to `x-debug-fastly-pop` on the Axum, Cloudflare and Spin adapters, and described the spoofable header list as stripping headers although the one adapter that calls the function reading it declares nothing. This revision states both as the code has them today, adds the effects on the other adapters, the scheme change, where each refusal happens, the order for changing the package and the document, the name that stays and the untested paths, and corrects the partly true claims, whereas the two code fixes wait on the open questions in section 2.3. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `b319bf039`, of the round 1 fixes and the new text. No blocking issues. This revision names in the Fastly sentence of 2.3 that `ts config validate` no longer refuses `x-debug-fastly-pop`, corrects when the Axum, Cloudflare and Spin adapters strip `fastly-client-ip`, lists the comment at the top of `platform/request_headers.rs` among the comments left for a later change, and corrects the changelog entry on the spoofable strip while adding the scheme change and the `x-debug-fastly-pop` change on Fastly to it. diff --git a/docs/superpowers/specs/2026-10-15-fastly-per-host-selector-design.md b/docs/superpowers/specs/2026-10-15-fastly-per-host-selector-design.md index 73477ae74d..5b6e6e81e5 100644 --- a/docs/superpowers/specs/2026-10-15-fastly-per-host-selector-design.md +++ b/docs/superpowers/specs/2026-10-15-fastly-per-host-selector-design.md @@ -295,11 +295,6 @@ Not tested. - The `500` for a real store that opens and then fails a lookup. No test makes a lookup fail on a store that opens, so the answer is tested through the in-memory lookup and the store that is not linked. - The rollback path of section 2.8 in CI. It depends on a live service's links and is checked by hand before the old entries are removed. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against the stack at `3aaf720c0` and IAB Tech Lab main at `fa9837f2b`. Issues found, 3 of them blocking. The merge statements missed the conflicts row 7 brings to every later row and the `lib.rs` conflicts at rows 11 and 35, the writer relied on `commands/keys.rs` and `config_chunks.rs`, which arrive only at rows 13 and 49, and left out the `--local` push the harness needs, and the Viceroy tests needed several selector states in the one root `fastly.toml`. This revision lists each conflict at the row where a fresh trial merge first meets it, places the writer's local half at row 11 and its remote half at row 13 with the chunk test moving at row 49, replaces the test plan with in-memory unit tests, root-store tests and harness cases, and answers a `Host` that is not UTF-8 and a key over 255 characters with `421` and no lookup. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against the stack at `3aaf720c0` and IAB Tech Lab main at `fa9837f2b`. Issues found, 1 of them blocking. A request addressing hook can address a request by its path and ignore the host, so a `Host` that is not valid UTF-8 could be served from a selector value and then panic in `entry_request` and `EntryRequest::of`, which section 2.4 and section 3 said could not happen. This revision reads `Host` through one function that cannot panic in all four places the adapter reads `Host`, says the row each read changes at, adds the conflict that change meets at row 27 and names the test that addresses a request by its path with such a `Host`, and it also names the comments in core as the only change there, covers main's upgrade checklist in section 2.8, places the `--staging` refusal at argument parsing and sizes the Sourcepoint conflict at about 818 lines of tests on main's side. - ## Revision record | Date | Change | @@ -307,3 +302,8 @@ Not tested. | 2026-10-15 | First draft. | | 2026-10-15 | Round 1 fixes. Merge table rebuilt from a fresh trial merge, writer split between rows 11 and 13 with `--local`, test plan rebuilt, `Host` read without a panic, over-long keys answered `421`, control characters malformed, staging pushes use `__NAME` and `--key`. | | 2026-10-16 | Round 2 fixes. Every read of `Host` in the Fastly adapter goes through one function that cannot panic, with the row of each read, the conflict it meets at row 27 and the test for a request addressed by its path, the comments in core named as the only change there, main's upgrade checklist in section 2.8, the `--staging` refusal placed at argument parsing, and the Sourcepoint conflict sized. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against the stack at `3aaf720c0` and IAB Tech Lab main at `fa9837f2b`. Issues found, 3 of them blocking. The merge statements missed the conflicts row 7 brings to every later row and the `lib.rs` conflicts at rows 11 and 35, the writer relied on `commands/keys.rs` and `config_chunks.rs`, which arrive only at rows 13 and 49, and left out the `--local` push the harness needs, and the Viceroy tests needed several selector states in the one root `fastly.toml`. This revision lists each conflict at the row where a fresh trial merge first meets it, places the writer's local half at row 11 and its remote half at row 13 with the chunk test moving at row 49, replaces the test plan with in-memory unit tests, root-store tests and harness cases, and answers a `Host` that is not UTF-8 and a key over 255 characters with `421` and no lookup. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against the stack at `3aaf720c0` and IAB Tech Lab main at `fa9837f2b`. Issues found, 1 of them blocking. A request addressing hook can address a request by its path and ignore the host, so a `Host` that is not valid UTF-8 could be served from a selector value and then panic in `entry_request` and `EntryRequest::of`, which section 2.4 and section 3 said could not happen. This revision reads `Host` through one function that cannot panic in all four places the adapter reads `Host`, says the row each read changes at, adds the conflict that change meets at row 27 and names the test that addresses a request by its path with such a `Host`, and it also names the comments in core as the only change there, covers main's upgrade checklist in section 2.8, places the `--staging` refusal at argument parsing and sizes the Sourcepoint conflict at about 818 lines of tests on main's side. diff --git a/docs/superpowers/specs/2026-10-15-geo-module-request-design.md b/docs/superpowers/specs/2026-10-15-geo-module-request-design.md index 7f2fc71097..c5e888d9bc 100644 --- a/docs/superpowers/specs/2026-10-15-geo-module-request-design.md +++ b/docs/superpowers/specs/2026-10-15-geo-module-request-design.md @@ -286,11 +286,6 @@ The Cloudflare adapter's tests are unchanged and call its lookup directly. The pull request records the fork's CI green on all 19 checks at 1b8b43858. Its checklist leaves the manual check under `fastly compute serve` unticked. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/33-geo-call` at `c5b04c263`. Issues found, three of them blocking. The three were the claim that a geo module is called with the request as a device module is, when a device module's `detect` is shown at most two headers and five of the six geo callers hand the module every header, the claim that the router path carries the same evidence as the request path, when a request preparer changes the request after the middleware has kept it, and the claim that no header is handed to a module for a response that did not authenticate, when the request path has already handed the module every header. This revision states what the code does in each case, retitles the page, and adds the services the finalize steps get, the response headers that no permission guards, the wait for the module, the missing breaking items and the missing tests, while the decisions the round raises wait under Open questions. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, checking the round 1 fixes against `split/33-geo-call` at `6944f4e70`. Issues found, none blocking. This revision corrects what the page says about why the entry-point step skips the registry, which responses carry `EcFinalizeState`, what the Fastly device module reads and the type of the registration field, names the permissions endpoint for Axum, Cloudflare and Spin, corrects the round 1 sentence on the code changes, and adds the changelog, guide and comment follow-ups to the open questions. - ## Revision record | Date | Change | @@ -298,3 +293,8 @@ The pull request records the fork's CI green on all 19 checks at 1b8b43858. Its | 2026-10-15 | First draft, written after the code. | | 2026-10-15 | Fixes from review round 1. The title, what a device module and a geo module are each handed, the evidence the router path's call carries, the 401 rule, the services the finalize steps get, the response headers that no permission guards, the wait for the module, the breaking items, the missing tests, the open questions and the review log. | | 2026-10-15 | Corrections from review round 2. Why the entry-point step skips the registry, which responses carry `EcFinalizeState`, what the Fastly device module reads, the type of the registration field, the permissions endpoint in Axum, Cloudflare and Spin, the round 1 log sentence on the code changes, and the follow-ups for the changelog, the integration guide and four stale comments. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/33-geo-call` at `c5b04c263`. Issues found, three of them blocking. The three were the claim that a geo module is called with the request as a device module is, when a device module's `detect` is shown at most two headers and five of the six geo callers hand the module every header, the claim that the router path carries the same evidence as the request path, when a request preparer changes the request after the middleware has kept it, and the claim that no header is handed to a module for a response that did not authenticate, when the request path has already handed the module every header. This revision states what the code does in each case, retitles the page, and adds the services the finalize steps get, the response headers that no permission guards, the wait for the module, the missing breaking items and the missing tests, while the decisions the round raises wait under Open questions. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, checking the round 1 fixes against `split/33-geo-call` at `6944f4e70`. Issues found, none blocking. This revision corrects what the page says about why the entry-point step skips the registry, which responses carry `EcFinalizeState`, what the Fastly device module reads and the type of the registration field, names the permissions endpoint for Axum, Cloudflare and Spin, corrects the round 1 sentence on the code changes, and adds the changelog, guide and comment follow-ups to the open questions. diff --git a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md index ced5d85380..30a32a0b25 100644 --- a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md +++ b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md @@ -241,11 +241,6 @@ These joins have no test. 7. **A module loading through a name it did not declare.** Nothing refuses the call, and no test shows what happens for a name no one declared, for a name another module declared, or for `robots-txt`. 8. **A module's cache made once, and shared by a declared cache with no table.** Rule 3 in 2.2 and the sharing in the step for a declared cache with no table have no test. No test checks that two tables naming one module, or a table and a declared cache with no table, are given one cache. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/32-cache-features` at `0ef3ff78d`. Issues found, three of them blocking. The page said a deployment can shorten a declared cache's lifetimes although core applies none of them, said two modules of one publisher never share a copy although nothing checks the names a module loads through, and said the changelog entry names the three changes for crates outside core although the entry left out two of them. This revision states each as the code stands today, adds the order of a push, the other callers of `loading_caches_from`, the names core does not check, the cache made once and the other untested paths, and corrects the changelog entry, whereas the decisions, and the code changes that depend on them, wait on the open questions in 2.10. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, checking the round 1 fixes against `split/32-cache-features` at `be4f47842`. Approved, none blocking. This revision corrects the three statements that were only partly true, being what a module that borrows another module's cache name shares, which section the guide's push-order warning covers and which builders the deploy check is given, corrects the round 1 sentence on the code changes, and adds the guide, rustdoc and visibility follow-ups to 2.10. - ## Revision record | Date | Change | @@ -254,3 +249,8 @@ These joins have no test. | 2026-10-15 | Fixes from review round 1. What core does with a declared table's lifetimes, the names core does not check, the changelog entry and the three changes the entry names, the order of a push, the other callers of `loading_caches_from`, the reserved and unchecked name forms, why the deploy check does not read `module`, the untested paths, the open questions, the review log, and the service call figure in section 1, dropped because it has no source. | | 2026-10-15 | Corrections from review round 2. What a module that borrows another module's cache name shares, the section the guide's push-order warning covers, the builders the deploy check is given, the round 1 log sentence on the code changes, and the follow-ups for the configuration guide, the integration guide, the rustdoc and the visibility of two functions. | | 2026-10-17 | Who can read a copy names the shared store on each adapter, being Fastly's cache, the key-value store the Cloudflare and Spin adapters declare on row 24, `TRUSTED_SERVER_KV` and `default`, and none on Axum. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/32-cache-features` at `0ef3ff78d`. Issues found, three of them blocking. The page said a deployment can shorten a declared cache's lifetimes although core applies none of them, said two modules of one publisher never share a copy although nothing checks the names a module loads through, and said the changelog entry names the three changes for crates outside core although the entry left out two of them. This revision states each as the code stands today, adds the order of a push, the other callers of `loading_caches_from`, the names core does not check, the cache made once and the other untested paths, and corrects the changelog entry, whereas the decisions, and the code changes that depend on them, wait on the open questions in 2.10. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, checking the round 1 fixes against `split/32-cache-features` at `be4f47842`. Approved, none blocking. This revision corrects the three statements that were only partly true, being what a module that borrows another module's cache name shares, which section the guide's push-order warning covers and which builders the deploy check is given, corrects the round 1 sentence on the code changes, and adds the guide, rustdoc and visibility follow-ups to 2.10. diff --git a/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md b/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md index 2b849f4dc7..fb15e35b7e 100644 --- a/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md +++ b/docs/superpowers/specs/2026-10-16-settings-and-consumers-design.md @@ -260,13 +260,6 @@ A table a consumer only reads goes in its `reads` and not in its `runs_when`, so 2. Should a name that no crate the tool links declares refuse at push, as the cache table check does today (`config.rs` line 330), or warn, as section 2.4 proposes and as push leaves a selection unchecked on purpose today (`registry.rs` lines 2163 to 2168)? Refusing catches a misspelt name before the document is written, and warning lets a deployment push, with the stock tool, names that its own crates supply. 3. Should the ads.txt sellers' placement require `path = "/ads.txt"`, as the robots.txt placement requires its path? An entry with no path appends the lines to every `text/plain` file it covers (`crates/ads-txt/iabtl/src/lib.rs` lines 254 to 262), and the module's check counts any `text/plain` entry today (lines 152 to 167), so the declaration keeps that placement until this is decided. -## Review log - -1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/50-cookies-by-path` at `3aaf720c0`. Issues found, two of them blocking. The declaration types could not say that a consumer is code in core, an adapter part, a request filter that feeds no middleware or a preparer that runs without its module, nor that a setting must have a given value or must not be blank, and push and validate had no way to see the parts of the adapter a document is for. This revision gives the types those consumers and conditions, has each adapter declare its parts in a `parts.toml` in its own crate that the tool reads for the adapter `--adapter` names, and applies the other findings. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/50-cookies-by-path` at `3aaf720c0`. Issues found, two of them blocking. A written key whose value failed a consumer's condition counted as a setting no running consumer declares, so defaults such as `certificate_check = true` and `enable_protection = false` were refused, and the doc comment on a core type named two vendors. This revision declares the tables a consumer reads apart from the conditions it runs under, so that the first kind of gap looks only at tables and names, adds a condition for a key that is absent or has another value, takes the media type of core's own middleware from `supplied_middleware`, places the files at the repository root, and gives the doc comment neutral examples. - -3. **Round 3, 2026-10-16.** A short check of the round 2 fixes in a fresh session, against `split/50-cookies-by-path` at `3aaf720c0`. All six were fixed truthfully and nothing blocking remains. This revision makes the fifth kind of gap apply only when the declaration's run conditions hold, so a switch written off gives no line. - ## Revision record | Date | Change | @@ -275,3 +268,10 @@ A table a consumer only reads goes in its `reads` and not in its `runs_when`, so | 2026-10-16 | Round 1 fixes. The types gain consumers for code in core, adapter parts and settings nothing reads, conditions on a key's value, named filters and preparers, and placements for a consumer nothing else places and for a preparer that runs whenever its crate is linked. Each adapter declares its parts in `parts.toml`, which push, diff and a new `ts config validate --adapter` read. Lines have levels, the check runs before any build on the settings as written, the host device case is added, and the claims on the push-time checks, the loading cache default and the one-off notices are corrected. | | 2026-10-16 | Round 2 fixes. A consumer declares the tables it reads apart from the conditions it runs under, and the first kind of gap looks only at tables and names, so a default such as `certificate_check = true` is no longer refused. `KeyIsNot` holds for a key that is absent or has another value, core's own middleware takes its media type from `supplied_middleware`, the doc comment on `SettingsNeed` names no vendor, and the files at the repository root are named as such. | | 2026-10-16 | Round 3 fix. The fifth kind of gap applies only when the declaration's run conditions hold. | + +## Review log + +1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/50-cookies-by-path` at `3aaf720c0`. Issues found, two of them blocking. The declaration types could not say that a consumer is code in core, an adapter part, a request filter that feeds no middleware or a preparer that runs without its module, nor that a setting must have a given value or must not be blank, and push and validate had no way to see the parts of the adapter a document is for. This revision gives the types those consumers and conditions, has each adapter declare its parts in a `parts.toml` in its own crate that the tool reads for the adapter `--adapter` names, and applies the other findings. +2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/50-cookies-by-path` at `3aaf720c0`. Issues found, two of them blocking. A written key whose value failed a consumer's condition counted as a setting no running consumer declares, so defaults such as `certificate_check = true` and `enable_protection = false` were refused, and the doc comment on a core type named two vendors. This revision declares the tables a consumer reads apart from the conditions it runs under, so that the first kind of gap looks only at tables and names, adds a condition for a key that is absent or has another value, takes the media type of core's own middleware from `supplied_middleware`, places the files at the repository root, and gives the doc comment neutral examples. + +3. **Round 3, 2026-10-16.** A short check of the round 2 fixes in a fresh session, against `split/50-cookies-by-path` at `3aaf720c0`. All six were fixed truthfully and nothing blocking remains. This revision makes the fifth kind of gap apply only when the declaration's run conditions hold, so a switch written off gives no line. From 1ee10ab5ca7ce0c909bb8efc44ee789b1485d03d Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 04:57:18 +0100 Subject: [PATCH 145/192] The config migrate design takes review round 1 The design answers each finding of review round 1 and ends with a review log. It gains sections on the adapters, consent and privacy, failure, and compatibility and rollout, and says the command takes the adapter, selects the location lookup only where the host has one and selects no location header. --- .../specs/2026-10-17-config-migrate-design.md | 208 ++++++++++++------ 1 file changed, 136 insertions(+), 72 deletions(-) diff --git a/docs/superpowers/specs/2026-10-17-config-migrate-design.md b/docs/superpowers/specs/2026-10-17-config-migrate-design.md index 587fd8e255..6355fc1eaa 100644 --- a/docs/superpowers/specs/2026-10-17-config-migrate-design.md +++ b/docs/superpowers/specs/2026-10-17-config-migrate-design.md @@ -1,86 +1,95 @@ # Design Spec: `ts config migrate` rewrites a settings document written for main into the shape the stack reads -**Status:** Proposed, 2026-10-17. The top pull request of the stack, not yet raised, for Tech Lab review. +**Status:** Proposed, 2026-10-17, revised 2026-10-18 after review round 1. The top pull request of the stack, not yet raised, for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-14-source-modules-design.md`, `2026-10-13-middleware-settings-design.md`, `2026-10-14-rsc-flight-middleware-design.md`, `2026-10-15-geo-module-request-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 **Related PRs:** none yet **Issues:** none -**Last updated:** 2026-10-17 +**Last updated:** 2026-10-18 -> **Why this spec exists.** The changelog of the stack lists 38 breaking entries under Unreleased, where main lists 10, and a settings document written for Tech Lab main does not load under the stack. Every operator would rewrite their document by hand from the changelog. `ts config migrate` makes the rewrite automatic where the document holds everything the rewrite needs, and says what is left where it does not. +> **Why this spec exists.** The changelog of the stack lists 39 breaking entries under Unreleased, where main lists 10, and a settings document written for Tech Lab main does not load under the stack. Every operator would rewrite their document by hand from the changelog. `ts config migrate` makes the rewrite automatic where the document holds everything the rewrite needs, and says what is left where it does not. ## 1. Problem -A document written for Tech Lab main (`upstream/main` at `5bc388625`) is refused at startup by the stack. `[integrations]` is gone, the origin settings moved to `[source]`, the auction providers moved to `[demand]`, `[[handlers]]` is refused, and a module changes a page only where a `[[fetch]]` or `[[serve]]` entry names its middleware. The changelog lists each change under Unreleased, with 38 entries marked breaking, and the settings code holds a few more that no changelog line names. An operator who upgrades has to read all of them, work out which apply to their document, and rewrite the document by hand without losing the comments they wrote. +A document written for Tech Lab main (`upstream/main` at `5bc388625`) is refused at startup by the stack. `[integrations]` is gone, the origin settings moved to `[source]`, the auction providers moved to `[demand]`, `[[handlers]]` is refused, and a module changes a page only where a `[[fetch]]` or `[[serve]]` entry names its middleware. The changelog lists each change under Unreleased, with 39 entries marked breaking, and the settings code holds a few more that no changelog line names. An operator who upgrades has to read all of them, work out which apply to their document, and rewrite the document by hand without losing the comments they wrote. -Most of the rewrite is mechanical, because the old shape says everything the new one needs. A few changes need a decision only the operator can make, or a value the document does not hold, and a few change behavior in a way no setting can express. +Most of the rewrite is mechanical, because the old shape says everything the new one needs. A few changes need a decision only the operator can make, or a value the document does not hold, a few change behavior in a way no setting can express, and the location settings depend on the adapter the deployment runs on. ## 2. Design ### 2.1 What the command does -`ts config migrate ` reads the document, rewrites every shape the stack refuses or reads differently, and prints the result on stdout. On stderr it prints one line for each change it made, one line for each item left for the operator, and the result of validating the migrated document. +`ts config migrate --adapter ` reads the document, rewrites every shape the stack refuses or reads differently, and prints the result on stdout. On stderr it prints one line for each change it made, one line for each item left for the operator and one line for each change in behavior to check, and then, unless `--check` is given, the result of validating the migrated document. -- `--write` rewrites the file in place, and leaves the file untouched when nothing changes. -- `--check` changes nothing and prints nothing on stdout. It exits 1 when the document needs migrating, so CI can hold a document to the current shape. +- `--adapter` names the adapter the deployment runs on, being `fastly`, `cloudflare`, `axum` or `spin`, as `ts config push` and `ts config diff` take one. It is required, because main's location lookup and location headers differed by adapter, and what each adapter accepts in `[geo]` differs too (§3). +- `--write` rewrites the file in place, and leaves the file untouched when nothing changes. It writes the result even when an item is left to do or the result does not pass validation, because the document did not load before either, and the exit status says so. +- `--check` changes nothing, prints nothing on stdout and does not validate. It exits 1 when the document needs migrating, being a rewrite to make or an item left to do, so CI can hold a document to the current shape. +- `--write` and `--check` cannot be used together. -The command works on the file alone. It reads no `edgezero.toml`, applies no `TRUSTED_SERVER__` overlay and asks no platform anything, because the rewrite is a property of the document. It validates the result as `ts audit generate` validates a document it is about to write, through `TrustedServerAppConfig` and the same push-time validation `ts config validate` runs, with the modules the tool links. +The command works on the file alone. It reads no `edgezero.toml`, applies no `TRUSTED_SERVER__` overlay and asks no platform anything, because the rewrite is a property of the document and the adapter. It validates the result as `ts audit generate` validates a document it is about to write, by parsing the text into `TrustedServerAppConfig` with no manifest and running `validate_excluding_secrets` with the modules the tool links (`crates/trusted-server-cli/src/commands/audit/generate/validate.rs` lines 55 to 66). That is the push-time validation `ts config validate` runs, and `ts config validate` also applies the overlay, checks secrets and runs EdgeZero's adapter checks (EdgeZero v0.0.9, `crates/edgezero-cli/src/config.rs` lines 245 to 277). Neither runs the checks made only when a deployment starts, being core's checks once the settings load (`Settings::finalize_deserialized`, core `settings.rs` lines 4339 to 4365) and each adapter's checks as it builds its state. The command reports the location settings among those as items left to do, because every document written for main gains an Edge Cookie module from its passphrase, and whether that document then starts depends on its geo settings and its adapter (§3). Each line on stderr starts with one of three words. - `changed:` a rewrite the command made. -- `to do:` an item the document cannot load without. It needs a decision or a value the document does not hold, so the command leaves the setting where it is and says what to do. +- `to do:` an item the document cannot load without, or a setting the stack ignores whose value the operator has to carry elsewhere. It needs a decision or a value the document does not hold, so the command leaves the setting where it is, says what to do, and reports the item on every run until the operator has done it. - `check:` a behavior change no setting expresses, printed with the rewrite it follows from, or for a document written for main. Nothing in the document has to change. -The exit status is 0 when nothing is left to do and the migrated document passes validation, 1 when an item is left to do or the result does not pass, and 2 when the file cannot be read or is not TOML. With `--check` it is 0 when the document needs no migrating and 1 otherwise. +The exit status is 0 when nothing is left to do and the migrated document passes validation, 1 when an item is left to do or the result does not pass, and 2 when the file cannot be read, written or parsed as TOML, or the arguments are refused, as `--write` with `--check` is. With `--check` it is 0 when the document needs no migrating and 1 otherwise. ### 2.2 Every change and how it is handled -The table lists every breaking entry in `CHANGELOG.md` under Unreleased that concerns the document, and the changes in the settings code that no changelog line names, marked "settings code". Entries for crates outside core, for the `ts` tool and for request handling with no setting are grouped in the last rows. - -| Change | Handling | -| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `[integrations]` is gone, and each integration is a module selected in the section of its type | Rewritten. An enabled `[integrations.]` becomes the module in its section, `module` or `modules`, with its settings in `[
.]` and `enabled` dropped. One that was not enabled is commented out under its new name. `enabled` follows main's default for each id. | -| `[auction.providers.]` moved to `[demand] modules` and `[demand.]` | Rewritten. The name takes `_` for `-`, `profile` becomes `implementation` by module path, `profile_config` moves into the table, and `protocol`, which main only took as `openrtb-2.6`, is dropped. | -| `[auction.bidders.] provider` is `module`, naming the demand source | Rewritten, with the demand source's new name. | -| `[auction] mediator = "adserver_mock"` and `[integrations.adserver_mock]` moved to `[ad-server]` | Rewritten to `[ad-server] module = "iabtl"` and `[ad-server.iabtl]`. A mock ad server no mediator names is commented out. | -| `[integrations.aps] rendering_mode` moved to the `auction.aps` demand table | Rewritten into each `auction.aps` demand table. Main read it only from an enabled table. | -| `include_mediator_response` is `include_adserver_response` | Rewritten. | -| The origin settings moved from `[publisher]` to `[source]` | Rewritten to `[source] module = "origin"` and `[source.origin]`, with `origin_url` as `url`, `origin_host_header_override` as `host_header`, `origin_auth` as `auth` and `directory_index` unchanged. | -| An asset route's `origin_url` and `auth` moved to its `source` table | Rewritten to `[proxy.asset_routes.source] module = "origin"` and `[proxy.asset_routes.source.origin]`. | -| The signer spelling `s3_sig_v4` is no longer read | Rewritten to `s3_sigv4` in every `auth` table. | -| `[[handlers]]` and HTTP Basic authentication are removed | Rewritten where an entry guards only `/_ts/admin` or `/admin/keys`, where nothing is served now, by removing it. To do for any other entry, below. | -| `[request_signing]` does not take `config_store_id` or `secret_store_id` | Rewritten by removing both. Check, below, for where the ids go. | -| `[ec] passphrase` is read as deprecated (settings code) | Rewritten to `[ec] module = "hmac"` and `[ec.hmac] passphrase`, where the stack documents it. The deprecated key still loads, with a warning at startup, for one release. | -| `[ec.providers.]` is `[ec.]`, and the module names `host-signals` and `client-fixed` are `host_signals` and `client_fixed` | Rewritten, in the tables, the selection and an `implementation` line. | -| `[permission_signal] sources` is `[permission-signal] modules`, with `gpp-sale-opt-out` and `gpp_sale_opt_out` as `gpp` and `us_privacy` as `us-privacy` | Rewritten. | -| `[device] module = "fastly"` is `"platform"` | Rewritten. | -| The host's location lookup runs only where `[geo]` selects it (settings code) | Rewritten for a document written for main by selecting `[geo] module = "platform"`, the lookup main ran on every request. Without it every request is the top of the `permissions.yaml` rules, and an Edge Cookie module with no geo module is refused at startup. | -| No response carries `x-geo-*` unless `headers.geo` is selected and placed | Rewritten for a document written for main to `[headers] modules = ["geo"]`, with `headers.geo` in the `[[serve]]` entry. Check, below, for the pages it covers. | -| `[tinybird]` is `[analytics.tinybird]`, selected by `[analytics] module` | Rewritten, with `enabled` dropped. A table that was not enabled is commented out. Check, below, for the consent gate. | -| A module's page change runs only where a `[[fetch]]` or `[[serve]]` entry names its middleware | Rewritten by adding one `[[fetch]]` and one `[[serve]]` entry for `text/html`, naming the middleware of the selected modules in the order the linked builders supply them, which is the order `trusted-server.example.toml` shows, when the document has no entry at all. | -| The Flight rewriter is the middleware `framework.nextjs.flight` | Rewritten by adding a `[[fetch]]` entry for `text/x-component` where the Next.js module is selected. Check, below, where it is not. | -| The `/robots.txt` route is the `robots_txt` middleware | Rewritten by adding a `[[fetch]]` entry for `text/plain` on `/robots.txt` to a document with `[robots-txt]` and no entry naming `robots_txt`. | -| `ad-server.mock` is `ad-server.iabtl`, `[adserver]` is `[ad-server]`, and `ads-txt.sellers` is `ads-txt.iabtl` | Rewritten in the selection, the table, an `implementation` line and a `[[fetch]]` middleware name. A selected `[ads-txt]` with no entry naming its middleware gets a `[[fetch]]` entry for `text/plain` on `/ads.txt`. | -| A `[[handlers]]` entry that guards other paths | To do. Nothing asks for a password now, so the operator puts one on those paths at the origin or in the hosting platform and then removes the entry. The command never removes it, because the stack refuses the list so that no deployment believes a path is protected. | -| `[consent.gdpr] applies_in` and `[consent.us_states] privacy_states` are not read (settings code) | To do. The places each covers are the `jurisdiction` values in the `permissions.yaml` compiled into the build, so the operator carries any change of theirs there and removes the table. The command keeps the table, because the list is the value the operator has to carry. | -| A `template_cache_key_cookies` entry under the public prefix is refused | To do, naming the setting. | -| A setting the command cannot place | To do, leaving it where it is. These are an `[integrations.]` naming no module, an `enabled` that is not a boolean, `[integration]`, `rendering_mode` with no `auction.aps` source, a mediator with no mock ad server table, an unknown `profile`, a `profile_config` setting named as one of its provider's own, and both forms of a setting. | -| A key in a module table that main ignored and the module now refuses, in the tables of `gpt`, `didomi`, `lockr`, `permutive`, `sourcepoint`, `testlight`, `nextjs`, `js_asset_proxy`, `google_tag_manager`, `adserver_mock` and `[tinybird]` (settings code) | Reported through the validation result, and `ts config validate` names the key. | -| No response carries `x-ts-version` or the platform's version unless `service_version` is named | Check. The command does not name the middleware, because the changelog lists the version's removal under Security. | -| A path in `origin_url` is kept, and the request path is added after it | Check, where `[source.origin] url` has a path, which main dropped. | -| Core no longer moves the origin's address in a Flight response | Check, where the Next.js module is not selected. Selecting it changes every HTML page, so the command does not. | -| The `x-geo-*` headers go only on the pages the `[[serve]]` entry covers | Check. | -| A telemetry row is sent only for a reader whose permissions include `analytics.ad_reporting.measure_ad_performance` | Check, where the table was enabled. | -| The signing store ids go to `ts keys rotate` and `ts keys deactivate` | Check, where the command removed them. | -| Breaking entries for crates outside core: platform headers, staging, config store chunks, backend naming, the auction protocol, consent decoders, identifier modules, geo modules, cookie constants, the page traits, the modules crate, the seam probe and origin signers | Out of scope. They change code, not the document. | -| Breaking entries for the `ts` tool: `ts cache purge` and `ts prebid client` | Out of scope. | -| Breaking entries for requests with no setting: the admin path check and the removed admin, purge and key rotation routes | Out of scope. | -| Environment variable names, such as `TRUSTED_SERVER__SOURCE__ORIGIN__URL` | Out of scope. They live outside the document. | -| Main's own breaking entries the stack keeps: the auction maps, APS OpenRTB, `site.page`, `bid_param_zone_overrides`, string scalars, Sourcepoint and creative sanitization | Nothing to do. A document written for main already meets them, and the maps move again under the second row. | -| Earlier shapes of the stack the changelog does not name, such as `[ec] provider` | Out of scope. They never reached a release. | - -A document is taken as written for main when it has `[publisher]` and every table at its top level is one main read, which are `publisher`, `tester_cookie`, `trusted_client_ip`, `ec`, `integrations`, `handlers`, `response_headers`, `request_signing`, `rewrite`, `auction`, `consent`, `cache`, `proxy`, `creative_opportunities`, `image_optimizer`, `tinybird` and `debug`. Main refused any other table, and every migrated document has `[source]`, so a second run never takes it as written for main. The geo rows, and the checks about `x-ts-version`, Flight responses and the `x-geo-*` headers, apply to such a document alone, so a document already in the stack's shape that leaves `[geo]` out keeps running with no location. +The table lists every breaking entry in `CHANGELOG.md` under Unreleased that concerns the document or how a deployment upgrades, and the changes in the settings code that no changelog line names, marked "settings code". Entries for crates outside core, for the `ts` tool and for request handling with no setting are grouped in the last rows. + +| Change | Handling | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `[integrations]` is gone, and each integration is a module selected in the section of its type | Rewritten. An enabled `[integrations.]` becomes the module in its section, `module` or `modules`, with its settings in `[
.]` and `enabled` dropped. One that was not enabled is commented out under its new name. `enabled` follows main's default for each id. | +| `[auction.providers.]` moved to `[demand] modules` and `[demand.]` | Rewritten. The name takes `_` for `-`, `profile` becomes `implementation` by module path, `profile_config` moves into the table, and `protocol`, which main only took as `openrtb-2.6`, is dropped. | +| `[auction.bidders.] provider` is `module`, naming the demand source | Rewritten, with the demand source's new name. | +| `[auction] mediator = "adserver_mock"` and `[integrations.adserver_mock]` moved to `[ad-server]` | Rewritten to `[ad-server] module = "iabtl"` and `[ad-server.iabtl]`. A mock ad server no mediator names is commented out. | +| `[integrations.aps] rendering_mode` moved to the `auction.aps` demand table | Rewritten into each `auction.aps` demand table. Main read it only from an enabled table. | +| `include_mediator_response` is `include_adserver_response` | Rewritten. | +| The origin settings moved from `[publisher]` to `[source]` | Rewritten to `[source] module = "origin"` and `[source.origin]`, with `origin_url` as `url`, `origin_host_header_override` as `host_header`, `origin_auth` as `auth` and `directory_index` unchanged. Check, below, where `host_header` is set. | +| An asset route's `origin_url` and `auth` moved to its `source` table | Rewritten to `[proxy.asset_routes.source] module = "origin"` and `[proxy.asset_routes.source.origin]`. | +| The signer spelling `s3_sig_v4` is no longer read | Rewritten to `s3_sigv4` in every `auth` table. | +| `[[handlers]]` and HTTP Basic authentication are removed | Rewritten by removing an entry where every path its pattern can match is one the stack closes, where nothing is served now, as the paragraph below the table defines. Main matched a pattern anywhere in the path (main core `settings.rs` line 734), so `^/admin/keys` alone also guarded `/admin/keystore`, which the stack forwards to the origin, and such an entry is to do, below. | +| `[request_signing]` does not take `config_store_id` or `secret_store_id` | Rewritten by removing both. Check, below, for where the ids go. | +| `[ec] passphrase` is read as deprecated (settings code) | Rewritten to `[ec] module = "hmac"` and `[ec.hmac] passphrase`, where the stack documents it. The deprecated key still loads, with a warning at startup, for one release. | +| `[ec.providers.]` is `[ec.]`, and the module names `host-signals` and `client-fixed` are `host_signals` and `client_fixed` | Rewritten, in the tables, the selection and an `implementation` line. | +| `[permission_signal] sources` is `[permission-signal] modules`, with `gpp-sale-opt-out` and `gpp_sale_opt_out` as `gpp` and `us_privacy` as `us-privacy` | Rewritten. | +| `[device] module = "fastly"` is `"platform"` | Rewritten. | +| The host's location lookup runs only where `[geo]` selects it (settings code) | Rewritten for a document written for main on Fastly and Cloudflare by selecting `[geo] module = "platform"`, the lookup main ran on every request there. On Axum and Spin main's lookup located no request, and those adapters refuse the selection at startup, so the command selects no geo module there, and the Edge Cookie module that follows is to do, below. | +| No response carries `x-geo-*` unless `headers.geo` is selected and placed | Check, for a document written for main, saying what main's responses carried on the adapter and, on Fastly, how to keep the headers. The command never selects `headers.geo`, because the headers carry the reader's location (§4). | +| `[tinybird]` is `[analytics.tinybird]`, selected by `[analytics] module` | Rewritten, with `enabled` dropped. A table that was not enabled is commented out. Check, below, for the consent gate. | +| A module's page change runs only where a `[[fetch]]` or `[[serve]]` entry names its middleware | Rewritten for a document written for main by adding one entry for each phase and media type the selected modules supply middleware for, naming that middleware in the order the linked builders supply it, which is the order `trusted-server.example.toml` shows. A document whose modules supply no fetch middleware gets no `[[fetch]]` entry. A document in the stack's shape is left as written, because it loads with no entry, with a warning naming each middleware no entry names (CHANGELOG line 51), and whether its modules change pages is the operator's choice. | +| The Flight rewriter is the middleware `framework.nextjs.flight` | Rewritten by adding a `[[fetch]]` entry for `text/x-component`, among the page entries above, where the Next.js module is selected. Check, below, where it is not. | +| The `/robots.txt` route is the `robots_txt` middleware | Rewritten by adding a `[[fetch]]` entry for `text/plain` on `/robots.txt` to a document with `[robots-txt]` and no entry naming `robots_txt`, which the stack refuses. | +| `ad-server.mock` is `ad-server.iabtl`, `[adserver]` is `[ad-server]` (settings code for the table), and `ads-txt.sellers` is `ads-txt.iabtl` | Rewritten in the selection, the table, an `implementation` line and a `[[fetch]]` middleware name. A selected `[ads-txt]` with no entry naming its middleware gets a `[[fetch]]` entry for `text/plain` on `/ads.txt`, because the module refuses that document (`crates/ads-txt/iabtl/src/lib.rs` lines 154 to 175). | +| A `[[handlers]]` entry whose pattern can match a path the stack does not close | To do. Nothing asks for a password now, so the operator puts one on those paths at the origin or in the hosting platform and then removes the entry. The command never removes it, because the stack refuses the list so that no deployment believes a path is protected. | +| An Edge Cookie module with no geo module, and `[geo] module = "platform"` on Axum or Spin (settings code and the adapters) | To do, for any document. Startup on every adapter refuses the first unless `[geo] assume_single_jurisdiction = true` accepts that every request takes the top of the `permissions.yaml` rules (core `settings.rs` lines 1736 to 1759), and the Axum and Spin adapters refuse the second, because neither host has a location lookup. `ts config validate` checks neither. The operator chooses between a geo module and the acknowledgment, because the stack asks for that choice to be made rather than assumed. | +| `[consent.gdpr] applies_in` and `[consent.us_states] privacy_states` are not read (settings code) | To do, as a setting the stack ignores whose value the operator has to carry elsewhere. The stack loads both tables and reads neither (core `consent_config.rs` lines 15 to 28). The places each covers are the `jurisdiction` values in the `permissions.yaml` compiled into the build, so the operator carries any change of theirs there and removes the table. The command keeps the table, because the list is the value the operator has to carry, so `--check` fails on a document that still carries one, which is intended, as a table nothing reads invites edits that change nothing. | +| A `template_cache_key_cookies` entry under the public prefix is refused | To do, naming the setting. | +| A setting the command cannot place | To do, leaving it where it is. These are an `[integrations.]` naming no module, an `enabled` that is not a boolean, `[integration]`, `rendering_mode` with no `auction.aps` source, a mediator with no mock ad server table, an unknown `profile`, a `profile_config` setting named as one of its provider's own, and both forms of a setting. | +| A key in a module table that main ignored and the module now refuses, in the tables of `gpt`, `didomi`, `lockr`, `permutive`, `sourcepoint`, `testlight`, `nextjs`, `js_asset_proxy`, `google_tag_manager`, `adserver_mock` and `[tinybird]` (settings code) | Reported through the validation result, and `ts config validate` names the key. | +| No response carries `x-ts-version` or the platform's version unless `service_version` is named | Check. The command does not name the middleware, because the changelog lists the version's removal under Security. | +| A path in `origin_url` is kept, and the request path is added after it | Check, where `[source.origin] url` has a path, which main dropped. | +| Where `host_header` is set, the page's own addresses under that host are rewritten, where those under the `url`'s host were (CHANGELOG line 26) | Check, where `origin_host_header_override` moved to `[source.origin] host_header`. | +| Core no longer moves the origin's address in a Flight response | Check, where the Next.js module is not selected. Selecting it changes every HTML page, so the command does not. | +| A telemetry row is sent only for a reader whose permissions include `analytics.ad_reporting.measure_ad_performance` | Check, where the table was enabled. | +| The signing store ids go to `ts keys rotate` and `ts keys deactivate` | Check, where the command removed them. | +| The Cloudflare and Spin adapters require their key-value store | Out of scope. The binding lives in `wrangler.toml` and the Spin manifest, outside the document. Main's `wrangler.toml` declares `TRUSTED_SERVER_KV` and its `spin.toml` grants `default`, and §6 says to keep both. | +| Breaking entries for crates outside core: platform headers, staging, config store chunks, backend naming, the auction protocol, consent decoders, identifier modules, geo modules, cookie constants, the page traits, the modules crate and origin signers | Out of scope. They change code, not the document. | +| The seam probe is `testing.iabtl`, where it was `testing.seam-probe` | Out of scope. It is a test fixture that no production build links (`crates/testing/iabtl/src/lib.rs` lines 15 to 20), so no deployment's document selects it. | +| Breaking entries for the `ts` tool: `ts cache purge` and `ts prebid client` | Out of scope. | +| Breaking entries for requests with no setting: the admin path check and the removed admin, purge and key rotation routes | Out of scope. | +| Environment variable names, such as `TRUSTED_SERVER__SOURCE__ORIGIN__URL` | Out of scope. They live outside the document, and §6 says to rename them. | +| Main's own breaking entries the stack keeps: the auction maps, APS OpenRTB, `site.page`, `bid_param_zone_overrides`, string scalars, Sourcepoint and creative sanitization | Nothing to do. A document written for main already meets them, and the maps move again under the second row. | +| Earlier shapes of the stack that neither the changelog nor the settings code names, such as `[ec] provider` | Out of scope. They never reached a release, and nothing tells an operator about them. | + +A `[[handlers]]` pattern guards only paths the stack closes when it starts `^/_ts/admin`, because every path that begins `/_ts/admin` is closed, or starts `^/admin/keys` followed by `/`, `$`, `(?:/|$)` or `(/|$)`, because of `/admin/keys` only the address itself and the paths below `/admin/keys/` are closed (core `closed_paths.rs` lines 18, 25 and 75 to 81), and when it has no `|` outside a group, which would let it match another path. + +A document is taken as written for main when it has `[publisher]` and every table at its top level is one main read, which are `publisher`, `tester_cookie`, `trusted_client_ip`, `ec`, `integrations`, `handlers`, `response_headers`, `request_signing`, `rewrite`, `auction`, `consent`, `cache`, `proxy`, `creative_opportunities`, `image_optimizer`, `tinybird` and `debug`. Main refused any other table, and every migrated document has `[source]`, so a second run never takes it as written for main. The geo row, the page entry rows, and the checks about `x-ts-version`, the location headers and Flight responses apply to such a document alone, so a document already in the stack's shape keeps its own geo settings and page entries, or the lack of them. + +The rules for `[permission_signal]`, `[ec.providers]`, `[adserver]`, `ad-server.mock`, `ads-txt.sellers`, `[device] module = "fastly"` and `[robots-txt]` with no entry rewrite shapes no document written for main holds. Each was the shape of an earlier pull request of this stack and never of a release, and each is named by a changelog entry or refused with directions by the settings code, so a deployment that ran a build of one of those pull requests holds a document in that shape. The rules save its operator the rewrite by hand, and an earlier shape that nothing names stays out of scope. ### 2.3 How the rewrite keeps what the operator wrote @@ -90,40 +99,95 @@ The command edits the document with `toml_edit`, the editor the tool already use - A new table that follows another, such as `[source]` after `[publisher]`, comes after the comment lines that directly follow that table's last setting, because `toml_edit` holds those lines with the next table. A comment paragraph is the run of comment lines up to a blank line. - A table removed with nothing to replace it, such as an admin handler, takes the paragraph directly above it and leaves the earlier ones, such as the file's opening comments, to the next table. - A module that was not enabled is written as a comment block in its place, under its new name, so the operator runs it by uncommenting the block and selecting the module in its section. A module that has no settings, such as Osano, keeps only the comments above its old table. -- The new sections that replace nothing, `[source]`, `[geo]` and `[headers]`, follow `[publisher]`, and `[ec.hmac]` follows `[ec]`. +- The new sections that replace nothing, `[source]` and `[geo]`, follow `[publisher]`, and `[ec.hmac]` follows `[ec]`. - A setting the stack does not read goes with the comment written directly above it. -- The page entries go at the end of the file, after the comment lines that follow the last table. +- The page entries go after the last table, after the comment lines written directly below its last setting, and ahead of any comment paragraph a blank line separates from that table, such as the comments that close the file. - Comment text is kept as written, so a comment may still name an old setting. - A document with Windows line endings keeps them. -Migrating a migrated document changes nothing, because each rule matches only a shape the stack refuses or a document written for main. +Migrating a migrated document changes nothing, because each rule matches only a shape the stack refuses, a spelling it reads only as deprecated, such as `[ec] passphrase`, or a document written for main, and the migrated document is none of those. An item left to do changes nothing in the document, so a second run reports it again and changes nothing. ### 2.4 What the command never does - It never prints a value from the document. Every line names the setting. The validation result gives the byte offset where reading failed or the phase that refused, as `ts audit generate` does, because a refusal can quote a value, and `ts config validate` prints the refusal whole. -- It never removes a setting the operator still has to decide on, such as a handler that guards a path of the publisher's own. -- It never names `service_version` or selects the Next.js module. It changes a value the document holds only where the stack renamed what the value names, which is a module, a middleware, a signer, an implementation or a demand source. +- It never removes a setting the operator still has to decide on, such as a handler whose pattern can match a path the stack does not close. +- It never sets `[geo] assume_single_jurisdiction`, because the stack asks the operator to accept that every request takes the top of the `permissions.yaml` rules rather than have that assumed. +- It never names `service_version` or selects `headers.geo`, and it never selects the Next.js module for a document that did not enable `[integrations.nextjs]`. It changes a value the document holds only where the stack renamed what the value names, which is a module, a middleware, a signer, an implementation or a demand source. - It never writes the file without `--write`, and never writes it when nothing changes. -## 3. What is not in this change +## 3. The adapters + +The command runs in the tool and does the same on every machine, but the document it writes runs on one adapter, and main's location lookup and location headers differed by adapter. Only the location settings depend on `--adapter`. + +| Adapter | Main's location lookup | What main's responses carried | What a document written for main gets | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | +| Fastly | The platform's lookup, on every request | The `x-geo-*` headers with the reader's location, or `x-geo-info-available: false` where none was found (main `crates/trusted-server-adapter-fastly/src/middleware.rs` lines 202 to 214) | `[geo] module = "platform"`, and a check line saying how to keep the headers | +| Cloudflare | Cloudflare's request headers, on every request (main `crates/trusted-server-adapter-cloudflare/src/platform.rs` lines 668 to 684) | `x-geo-info-available` alone, `true` where `cf-ipcountry` was present and not `XX` (main `crates/trusted-server-adapter-cloudflare/src/middleware.rs` lines 139 to 147) | `[geo] module = "platform"`, and a check line saying that `headers.geo` would add the reader's location | +| Axum | None, as the lookup answered nothing (main `crates/trusted-server-adapter-axum/src/platform.rs` lines 183 to 187) | `x-geo-info-available: false` (main `crates/trusted-server-adapter-axum/src/middleware.rs` lines 133 to 137) | No geo module, an item left to do for the Edge Cookie module, and a check line on the header | +| Spin | None, as the lookup answered nothing (main `crates/trusted-server-adapter-spin/src/platform.rs` lines 757 to 761) | `x-geo-info-available: false` (main `crates/trusted-server-adapter-spin/src/middleware.rs` lines 167 to 175) | As on Axum | + +The stack's Axum and Spin adapters refuse `[geo] module = "platform"` at startup naming the setting, because neither host has a lookup (`crates/trusted-server-adapter-axum/src/app.rs` lines 277 and 326, `crates/trusted-server-adapter-spin/src/app.rs` lines 252 and 286, CHANGELOG line 42). Every document main ran with has an Edge Cookie passphrase (main core `settings.rs` lines 545 and 625 to 633), which becomes `[ec] module = "hmac"`, and startup on every adapter refuses an Edge Cookie module with no geo module unless `[geo] assume_single_jurisdiction = true` accepts that every request takes the top of the `permissions.yaml` rules. On Axum and Spin the operator chooses between that acceptance and a geo module a linked crate offers, so the command leaves the choice to do. Writing `platform` there would report a document as passing that the adapter refuses, and writing the acknowledgment would make the choice the stack asks the operator to make. + +The rest of the document is the same on every adapter. Only the Fastly adapter sends the telemetry an `[analytics]` selection names, as only Fastly's did on main (CHANGELOG lines 20 and 48), and the Cloudflare and Spin adapters need their key-value store (§6). + +## 4. Consent and privacy + +- **No value is printed.** Every line on stderr names a setting and holds no value from the document, because a document can hold secrets, and the validation result gives only the byte offset or the phase that refused. Lines do carry table names the operator chose, such as an integration id or a bidder code (the test `no_line_carries_a_value_from_the_document`). +- **No location header is added.** The command never selects `headers.geo`, which writes the reader's location in the `x-geo-*` headers on the responses a `[[serve]]` entry covers. Main wrote those headers on Fastly alone, and on Cloudflare wrote only `x-geo-info-available`, so selecting the middleware for every document written for main would send the reader's location on Cloudflare, where main sent none. Whether pages carry the reader's location is the operator's decision, as the stack made it one (CHANGELOG line 41), so the check line for each adapter says what main's responses carried and, on Fastly, how to keep the headers. +- **The location the consent rules see.** On Fastly and Cloudflare the document keeps the lookup main ran, so the `permissions.yaml` rules see the reader's location as they did, where without a geo module every request would take the top of those rules. On Cloudflare the stack's lookup also reads the region, from `cf-region-code` where Cloudflare sends it, where main's lookup read none (`crates/trusted-server-adapter-cloudflare/src/platform.rs` lines 709 to 721 and 782 to 787), so the region rules of `permissions.yaml`, such as those for US states, can apply there now. On Axum and Spin main located no request and the document locates none, and an Edge Cookie module runs there only once the operator accepts that with `[geo] assume_single_jurisdiction = true` or selects a geo module. +- **Passwords.** The command removes a `[[handlers]]` entry only where every path its pattern can match is one the stack closes, so a publisher path such as `/admin/keystore` never loses its password without an item left to do. +- **Telemetry.** A telemetry row is sent only for a reader whose permissions include `analytics.ad_reporting.measure_ad_performance`, and a check line says so where `[tinybird]` was enabled. +- **The consent lists.** The places the GDPR and US state rules apply are the `jurisdiction` values in the `permissions.yaml` compiled into the build, so a change the operator made to `applies_in` or `privacy_states` is an item left to do until they carry it there and remove the table. + +## 5. Failure + +- **A file that cannot be read, written or parsed.** The command exits 2. A file that is not TOML is refused naming the line reading stopped at and nothing from it (`migrate` in `rules.rs`, and the test `no_line_carries_a_value_from_the_document`). `--write` rewrites the file in place with one write, so a write that fails part of the way can leave the file damaged, which is one reason §6 asks the operator to keep the original. +- **An item left to do, or a result that does not pass validation.** The command exits 1, and `--write` writes the result anyway, because the document did not load before either and the items say what is left. +- **The checks made only at startup.** The command's validation, like `ts config validate`, leaves out the checks core makes once the settings load and each adapter makes as it builds its state. Of those it reports the location settings as items left to do, for any document. The others, such as a module name that no linked crate supplies, show only when the deployment starts. + +## 6. Compatibility and rollout + +1. **The migrated document goes live with the new build.** A build from before this change refuses it, as main refuses a table it does not know (main core `settings.rs` line 2958), and this build refuses the old one, so no one document starts both builds (CHANGELOG lines 72 and 96). +2. **On Fastly, stage the pair first**, with `ts deploy --staging` and `ts config push --staging`, which read a configuration key of their own (CHANGELOG line 96). +3. **Keep the original document before `--write`.** A rollback restores it with the old build (CHANGELOG line 72), and it holds the `[request_signing]` store ids the command removes and never prints, which `ts keys rotate` and `ts keys deactivate` take as `--config-store-id` and `--secret-store-id`. +4. **Rename the overlay variables.** `ts config push` ignores a variable for a key the document does not hold, so `TRUSTED_SERVER__PUBLISHER__ORIGIN_URL` and `TRUSTED_SERVER__INTEGRATIONS____*` set nothing once the document is migrated. They become `TRUSTED_SERVER__SOURCE__ORIGIN__URL` and `TRUSTED_SERVER__
____*` (CHANGELOG lines 26, 56 and 72). +5. **Keep the key-value store on Cloudflare and Spin.** A Worker without the `TRUSTED_SERVER_KV` binding, or a Spin component that cannot open `default`, now fails every request before any route runs (CHANGELOG line 46). Main's `wrangler.toml` declares the binding and main's `spin.toml` grants the store, so a deployment that kept those files has both. +6. **Expect the stored pages to be fetched again** once after the upgrade, because their keys hold a code worked out from the settings (CHANGELOG line 26). + +The CLI guide's section "Upgrade a configuration written for an earlier release" gives the same steps. + +## 7. What is not in this change - The breaking changes for crates outside core, for the `ts` tool and for request handling, as the table says. -- Pushing the result. The operator runs `ts config validate` and `ts config push` as before. +- Pushing the result. The operator runs `ts config validate` and `ts config push` in the order §6 gives. +- A write through a temporary file, which would keep the original whole when a write fails part of the way. §6 asks the operator to keep the original instead. - Rewriting comment text, which would mean guessing what a comment refers to. -- Earlier shapes of the stack that never reached a release and that the changelog does not name. +- Earlier shapes of the stack that neither the changelog nor the settings code names. -## 4. Tests +## 8. Tests - One test for each rewrite rule in the table, each on a small document, checking the result and the line it reports. -- One test for each to do item, checking that the setting is left in place, that the line names it, and that the exit status says work is left. -- A golden test that migrates main's own `trusted-server.example.toml`, copied from `upstream/main` into `crates/trusted-server-cli/tests/fixtures/main-trusted-server.example.toml` with the commit recorded in its first comment, with its placeholder domain and origin replaced. The result loads and passes validation under the stack's settings code, and migrating it again changes nothing. +- One test for each item left to do, checking that the setting is left in place and that the line names it. Whether a run fails follows from whether any item is left, so one test, `an_item_left_to_do_fails_the_run`, runs the command on a document with an item left and checks that the run fails. +- A golden test that migrates main's own `trusted-server.example.toml`, copied from `upstream/main` byte for byte into `crates/trusted-server-cli/tests/fixtures/main-trusted-server.example.toml` under a header that names the commit. The test replaces its placeholder domain and origin, as an operator's document would have them. The result loads and passes validation under the stack's settings code, and migrating it again changes nothing. +- `the_location_lookup_is_the_one_each_adapter_accepts`, which migrates a document written for main for each adapter. On Fastly and Cloudflare the result selects `platform` and passes core's startup check of the jurisdiction acknowledgment. On Axum and Spin it selects no geo module, that check refuses it and an item left to do says so, and once `[geo] assume_single_jurisdiction = true` is added the check passes and a second run leaves nothing to do. +- `the_location_settings_startup_refuses_are_left_to_do_in_any_document`, for `[geo] module = "platform"` on Axum and Spin and for an Edge Cookie module with no geo module on every adapter. +- `no_location_header_is_selected_for_the_operator`, on every adapter, with the check line each adapter gets. +- `a_handler_is_removed_only_where_every_path_it_matches_is_closed`, with the bounded and the unbounded `/admin/keys` patterns. +- `a_host_header_override_is_reported_to_check`, and `a_document_in_the_stack_shape_with_no_page_entries_gets_none_added`. - A test that the comments of a document survive and its tables keep their order, and one that a document with Windows line endings keeps them. -- A test that no line the command prints carries a value from the document, run on a document whose every value is a marker string. -- Tests of the command itself, that `--check` writes nothing and exits 1 when the document needs migrating and 0 when it does not, that `--write` writes the file, and that a file that is not TOML exits with an error that quotes nothing from it. +- A test that no line the command prints carries a value from the document, on every adapter, run on a document whose every value is a marker string. +- Tests of the command itself, that `--check` writes nothing and exits 1 when the document needs migrating and 0 when it does not, that `--write` writes the file, and that a file that is not TOML exits with an error that quotes nothing from it, and a test that the arguments need `--adapter` and the file and refuse an unknown adapter and `--write` with `--check`. - A test that the page entries name the middleware in the order `trusted-server.example.toml` shows them. +No test starts the Axum or Spin adapter with a migrated document, because the tool links no adapter crate. Their refusal of `platform` is covered by the migrated document selecting no geo module on either. + ## Revision record -| Date | Change | -| ---------- | ------------ | -| 2026-10-17 | First draft. | +| Date | Change | +| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-17 | First draft. | +| 2026-10-18 | Fixes from review round 1. `--adapter`, with the location lookup each adapter accepts and the Edge Cookie and geo settings startup refuses left to do. No location header is selected, a handler is removed only where every path its pattern matches is closed, a host header is reported to check, and page entries are added to a document written for main alone. The flags, the validation's reach, the consent lists, the adapters, consent and privacy, failure, compatibility and rollout, the tests, and 39 breaking entries. | + +## Review log + +1. **Round 1, 2026-10-18.** A review in a fresh session, against `fixpass6/51-config-migrate` at `50c68be1b`. Issues found, four of them blocking. The four were that the Axum and Spin adapters refuse the `[geo] module = "platform"` the command wrote into every document written for main, that the `[headers]` rewrite started sending the reader's location on Cloudflare, where main sent none, that the `[[handlers]]` rule removed a handler on `^/admin/keys`, which also guarded publisher paths such as `/admin/keystore` that the stack forwards to the origin, and that the spec had no compatibility and rollout section. Fixed in `18b7c7d16`, where the command selects `platform` only where the host has a lookup and leaves to do, for any document, an Edge Cookie module with no geo module and `platform` on an adapter with none, in `4465e2ad0`, where it takes `--adapter`, never selects `headers.geo` and prints a check line saying what main's responses carried, in `65bb09b14`, where it removes a handler only where every path its pattern can match is closed, and in this revision, which adds the adapters, consent and privacy, failure, and compatibility and rollout. The non-blocking findings are fixed too, with a host header reported to check in `65814d561` and page entries added to a document written for main alone in `f8dffebdc`. In this revision the consent lists stay items left to do with the definition of `to do:` widened to say so, the seam probe and the key-value store have rows of their own, and the count of breaking entries is 39. The review's statement that a selected `[ads-txt]` with no entry loads with a warning is not borne out, because the module refuses that document (`crates/ads-txt/iabtl/src/lib.rs` lines 154 to 175), so the `/ads.txt` entry is still added to any document. From 9b96efe2e7b6caf37487965afe63b519e949fea5 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 04:57:20 +0100 Subject: [PATCH 146/192] The request filters design answers its first review round It says what a selected device module did on Axum, Cloudflare and Spin before, what DataDome's filter sends and sets with no permission gating it, why no time limit is added to a filter's call, that the Fastly adapter answers all three inspection endpoints before the preparers, which routes stay out of the cycle, which of a crawler's cookies robots.txt reads, when the Cookie check is reached, the configuration, the new tests, the open questions, and ends with its review log. --- ...request-filters-on-every-adapter-design.md | 85 +++++++++++++------ 1 file changed, 61 insertions(+), 24 deletions(-) diff --git a/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md index e23e3d92cd..1cbbd87c75 100644 --- a/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md @@ -2,39 +2,41 @@ **Status:** Proposed, 2026-10-17. A pull request above `ts config migrate`, not yet raised, for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-10-14-51degrees-client-hints-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 +**Related specs:** `2026-10-14-51degrees-client-hints-design.md`, whose sections on the adapters this change amends, and the design documents of the stack in IABTechLab/trusted-server#1342 **Related PRs:** none yet **Issues:** none -**Last updated:** 2026-10-17 +**Last updated:** 2026-10-18 -> **Why this spec exists.** Only the Fastly adapter runs the integration request filters and the device module `[device]` selects. On Axum, Cloudflare and Spin a module's request filter never runs, so the 51Degrees client hints middleware writes nothing and DataDome's protection never asks its service, and no module call carries a device classification. This change runs both on those three adapters where the Fastly adapter runs them. +> **Why this spec exists.** Only the Fastly adapter runs the integration request filters and classifies each request with the device module `[device]` selects. On Axum, Cloudflare and Spin a module's request filter never runs, so the 51Degrees client hints middleware writes nothing and DataDome's protection never asks its service, and a device module is asked only for a bid request's device attributes, so no module call carries a device classification. This change runs both steps on those three adapters where the Fastly adapter runs them. ## 1. Problem The Fastly adapter runs three steps that no other adapter runs. 1. **Request filters.** `run_pre_route_filters` in `crates/trusted-server-adapter-fastly/src/app.rs` calls `IntegrationRegistry::filter_request` for every named route except the inspection endpoints and batch sync, and for the publisher fallback. It runs after the modules' preparers and once the request's Edge Cookie context is read, and it hands every filter the location and the permissions resolved for that context. A filter that answers stands in for the route. A filter that lets the request continue has its request headers applied before the route, and its response headers are applied by the entry point after the response is finalized (`apply_terminal_response_effects` in `src/lib.rs`). -2. **Device detection.** `derive_device_signals` in `src/lib.rs` calls `build_device_module` and `DeviceModule::detect` at the entry point, and `build_ec_request_state` sets the result on the Edge Cookie context. Every module call made from that context carries the classification (`ModuleContext::with_request_state` in core). -3. **robots.txt first.** `dispatch_fallback` answers robots.txt once the preparers have run and before the Edge Cookie steps and the filters, with an empty Edge Cookie context, so no filter's answer stands in for the file and none of the crawler's cookies are read. +2. **Device classification.** For every request, before routing, the entry point's `derive_device_signals` in `src/lib.rs` calls `DeviceModule::detect` on the device module a registration supplies, or on the one `build_device_module` builds where none does, and `build_ec_request_state` sets the result on the Edge Cookie context. A module call made with that context's state carries the classification (`ModuleContext::with_request_state` in core), which covers a module route's call and a serve phase preparation. A request filter is never handed it, because `RequestFilterInput` carries only the location and the permissions, and the fetch phase of a document that may be stored is handed services with nothing of the reader in place of the context's state (`preparation_context` in `crates/trusted-server-core/src/publisher.rs`). +3. **robots.txt first.** `dispatch_fallback` answers robots.txt once the preparers have run and before the Edge Cookie steps and the filters, with an empty Edge Cookie context, so no filter's answer stands in for the file and the crawler's Edge Cookie and consent signals are not read. The entry point has already run the device module by then, so a device module a registration supplies is shown the crawler's cookies. -No code in `crates/trusted-server-adapter-axum`, `-cloudflare` or `-spin` calls `filter_request` or `DeviceModule::detect`. So on those adapters the 51Degrees client hints middleware asks its service nothing and writes no header or tag, DataDome's protection filter never runs, `[device] module` changes nothing, and robots.txt is answered through the publisher path with the crawler's own Edge Cookie context. On Spin the discovery, signature verification and first-party routes and the page-bids preflight also skip the modules' preparers, which every other adapter runs for them. +No source file in `crates/trusted-server-adapter-axum`, `-cloudflare` or `-spin` calls `filter_request` or `DeviceModule::detect`. So on those adapters the 51Degrees client hints middleware asks its service nothing and writes no header or tag, DataDome's protection filter never runs, and robots.txt is answered through the publisher path with the crawler's own Edge Cookie context. A device module `[device]` selects from a registration is not idle there, because each adapter's `services_for_request` puts it on the request's services and core asks it for the bid request's device attributes in every auction, showing it the request's headers and permissions (`attach_device_attributes` in `crates/trusted-server-core/src/auction/formats.rs`, called from `auction/endpoints.rs` and `publisher.rs`). It never classifies a request, so no module call carries a classification. On Spin the discovery, signature verification and first-party routes and the page-bids preflight also skip the modules' preparers, which every other adapter runs for them. ## 2. Design -**One request cycle per adapter, in the order the Fastly adapter runs it.** Each of the three adapters gains the steps the Fastly adapter has, under the Fastly adapter's names, in `src/app.rs`. +**One request cycle per adapter, in the order the Fastly adapter runs it.** Each of the three adapters gains the steps the Fastly adapter has, in `src/app.rs`, with the finalize step in `src/middleware.rs`, under the Fastly adapter's names where it has one. The Edge Cookie step keeps the adapter's own name, `build_ec_context`, which the Fastly adapter calls `build_ec_request_state`, and the response effects are written by `FinalizeResponseMiddleware`, where the Fastly adapter writes them from its entry point. The Fastly adapter and core are unaffected, because no file of either changes. + +Each adapter carries its own copy of the steps. `derive_device_signals`, `run_pre_route_filters`, `attach_request_filter_effects`, `asks_for_robots_txt` and `apply_terminal_response_effects` are the same text in all three, and `answer_robots_txt` and `execute_named` the same in Cloudflare and Spin. One request cycle in core that every adapter calls would replace the copies, but it would also move the Fastly adapter's cycle, which this change leaves as it is, so section 3 leaves one request cycle in core to a later change. | Step | Function | What it does | | --------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Preparers | `execute_named`, `execute_fallback` | `IntegrationRegistry::prepare_request`, once for each request, as before | | robots.txt | `asks_for_robots_txt`, `answer_robots_txt` | Answers a `GET` or `HEAD` of `/robots.txt`, when `[robots-txt]` is present, through the publisher path with `EcContext::default()` and no auction | -| Device classification | `derive_device_signals` | Runs the selected device module, or the built-in one, with the request's module context | +| Device classification | `derive_device_signals` | Runs the device module the request's services carry, or the built-in one, with the request's module context | | Edge Cookie context | `build_ec_context` | `EcContext::read_from_request_resolving_geo`, then `set_device_signals`. An error answers the request | | Request filters | `run_pre_route_filters` | `IntegrationRegistry::filter_request` with the context's location and permissions. An answer, or an error, stands in for the route | | Route | `run_named_route`, `dispatch_fallback` | The handler, given the context the filters saw | | Response effects | `attach_request_filter_effects` | Leaves the filters' response headers on the response for the finalize middleware | | Finalize | `FinalizeResponseMiddleware` | Finalizes the response, then `apply_terminal_response_effects` writes the filters' response headers and restores the privacy guards, as the Fastly entry point does | -**Which routes run the cycle.** The routes the Fastly adapter runs it for, which are every named route but the inspection endpoints, and the publisher fallback, being robots.txt, tsjs, the module routes and the publisher's origin. The inspection endpoints are read only and stay ahead of the Edge Cookie context and the filters, as on Fastly. +**Which routes run the cycle.** The routes the Fastly adapter runs it for, being every named route but the inspection endpoints and batch sync, and the publisher fallback, being robots.txt, tsjs, the module routes and the publisher's origin. These three adapters register no batch sync route, so a request for its path reaches the publisher fallback, as before. The inspection endpoints are read only and stay ahead of the Edge Cookie context and the filters, as on Fastly. Every adapter also answers the attestation pages outside the cycle, for `GET` and `HEAD` and for every method under a prefix the endpoint owns, as on Fastly, and Axum and Spin answer `/health` outside it. | Adapter | Named routes in the cycle | Inspection endpoints | Registration | | ---------- | ----------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- | @@ -44,44 +46,79 @@ No code in `crates/trusted-server-adapter-axum`, `-cloudflare` or `-spin` calls On Spin the discovery, signature verification and first-party routes and the page-bids preflight now run the preparers, because the cycle starts with them. Before this change they ran none. -**Device classification.** `derive_device_signals` follows the Fastly adapter's function of the same name. A device module a registration supplies is shown the `User-Agent` and the `Cookie` header, and the built-in module the `User-Agent` alone, each in a module context carrying the request, that evidence, the settings and the services. Three things differ from the Fastly adapter. +**Device classification.** `derive_device_signals` follows the Fastly adapter's function of the same name. A device module a registration supplies is shown the `User-Agent` and the `Cookie` header, and the built-in module the `User-Agent` alone, each in a module context carrying the request, that evidence, the settings and the services. Four things differ from the Fastly adapter. 1. It runs where the adapter builds the Edge Cookie context, after the preparers, where the Fastly adapter runs it at the entry point before routing. A preparer that removes a reserved cookie therefore removes it before a device module sees it. -2. The registered module is the one the request's services carry (`RuntimeServices::device_module`), which `services_for_request` sets from the registry, and the module is handed those same services, so it reads the platform's key-value store when the services carry one. The Fastly entry point builds services of its own for the call. -3. These adapters supply no TLS or HTTP/2 signals, so `[device] module = "platform"` builds the built-in module, which classifies from the `User-Agent`, because `platform` adds the host's signals only where the platform supplies them. +2. It runs only for the routes in the cycle, so a request for robots.txt, an inspection endpoint or an attestation page is not classified, where the Fastly entry point classifies every request. On Fastly a device module a registration supplies is therefore shown the cookies a crawler sends with a request for robots.txt, and on these adapters it is not. +3. The registered module is the one the request's services carry (`RuntimeServices::device_module`), which `services_for_request` sets from the registry, and the module is handed those same services, so it reads the platform's key-value store when the services carry one. The Fastly entry point builds services of its own for the call. +4. These adapters supply no TLS or HTTP/2 signals, so `[device] module = "platform"` builds the built-in module, which classifies from the `User-Agent`, because `platform` adds the host's signals only where the platform supplies them. + +As on Fastly, a module route's call and a serve phase preparation carry the classification, a request filter is not handed it, and the fetch phase of a document that may be stored is handed services with nothing of the reader. + +**Response effects.** A filter that lets the request continue can name response headers. The route's response carries them in its extensions as `RequestFilterEffects`, and `FinalizeResponseMiddleware` applies them after it writes the finalize headers, so a filter's header is written after an operator's `[response_headers]`, as on Fastly. `apply_terminal_response_effects` then enforces `no-store, private` on a response core marked `TerminalPrivateResponse` and runs `enforce_set_cookie_cache_privacy` and `enforce_uncacheable_cache_privacy`, the steps the Fastly entry point runs before it sends every response. Those steps run on every response, whether or not a filter named a header, so a response core marked private also loses any `ETag`, `Last-Modified`, `Expires` or `Age` that `[response_headers]` writes, as on Fastly. A filter's header is also written after the `X-Robots-Tag` that `[robots-txt]` writes, and `x-robots-tag` is not among the headers a filter is refused (`is_forbidden_filter_header` in `crates/trusted-server-core/src/integrations/registry.rs`), so a filter that names that header replaces the tag, on Fastly as on these adapters. The open questions ask whether a filter should be refused `X-Robots-Tag`. -**Response effects.** A filter that lets the request continue can name response headers. The route's response carries them in its extensions as `RequestFilterEffects`, and `FinalizeResponseMiddleware` applies them after it writes the finalize headers, so a filter's header is written after an operator's `[response_headers]`, as on Fastly. `apply_terminal_response_effects` then enforces `no-store, private` on a response core marked `TerminalPrivateResponse` and runs `enforce_set_cookie_cache_privacy` and `enforce_uncacheable_cache_privacy`, the steps the Fastly entry point runs before it sends every response. +**Configuration.** No setting is added, removed or given a new default. Three settings act on these adapters where they did not, being `[device] module` when it selects a device module a registration supplies, which now classifies each request in the cycle, and DataDome's `enable_protection` and `[shared.51degrees]`, whose filters now run. Two settings these adapters accept are not applied, being DataDome's `timeout_ms`, default `1500`, and `[shared.51degrees] timeout_ms`, default `500`, which only the Fastly adapter applies, as the timeouts of the backend each module registers for its call. No adapter warns about either at startup, the way the Axum adapter warns about a `proxy.certificate_check` it cannot honor, because an adapter names no vendor's setting, and no platform declaration tells core or a module that an adapter does not apply a backend's timeouts. The DataDome and 51Degrees guides say where `timeout_ms` applies. -**Configuration.** None. The filters a deployment's modules register and the module `[device]` selects now run on every adapter. +**Consent and privacy.** Every filter is handed the permissions resolved for the request, as on Fastly, and no shipped filter reads them yet (`docs/guide/permission-model.md`), so no permission gates what the two shipped filters send. -**Consent and privacy.** Every filter is handed the permissions resolved for the request, as on Fastly. The 51Degrees filter sends the reader's `User-Agent` and `Sec-CH-UA` hints to the 51Degrees service for each page navigation an entry naming its middleware covers, whatever the permissions say, which is the behavior and the open question that `2026-10-14-51degrees-client-hints-design.md` describes for Fastly. DataDome's protection filter removes `x-ts-datadome-bypass` from every request it runs for, before its other checks. A device module a registration supplies sees the reader's cookies, as on Fastly. robots.txt is answered with an empty Edge Cookie context, so a crawler's cookies and consent signals are no longer read on these adapters. +- The 51Degrees filter sends the reader's `User-Agent` and `Sec-CH-UA` hints to the 51Degrees service for a page navigation an entry naming its middleware covers. The call goes through the answer cache the 51Degrees modules share, so the service is asked once for each set of those values while an answer is held, and not at all for a request with no `User-Agent` (`crates/shared/51degrees/src/hints_engine.rs`). That is the behavior `2026-10-14-51degrees-client-hints-design.md` describes, and its open question, whether the call needs a permission, stays open, and now concerns four adapters where it concerned one. +- DataDome's protection filter, which DataDome registers only when `enable_protection` is on, removes `x-ts-datadome-bypass` from every request it runs for, before its other checks. For each request it protects, it sends DataDome's Protection API the server side key, the reader's IP address, the request's method, host, path and query, its `User-Agent`, `Accept`, `Accept-Language`, `Accept-Encoding`, `Referer`, `Origin`, `Via`, `X-Forwarded-For`, `True-Client-IP` and `X-Real-IP` headers among others, its `Sec-CH-UA` and `Sec-Fetch` headers, the list of its header names, the lengths of its `Cookie` and `Authorization` headers, and the client id from the `datadome` cookie or the `x-datadome-clientid` header (`build_protection_payload` in `crates/bot-protection/datadome/src/protection.rs`). The filter does not read the permissions. A response the API allows, or a challenge, can carry DataDome's `Set-Cookie`, which sets a cookie on the publisher's domain and keeps the response out of shared caches, as `private, max-age=0` where the response's own policy allowed them (`enforce_set_cookie_cache_privacy`). A reader an IP rule excludes is not checked, and the request is marked personalized, so an HTML page it asks for is served without DataDome's tag and sent `private, no-store`. +- A device module a registration supplies is shown the reader's `User-Agent` and cookies to classify every request in the cycle, with no permission check, because classification runs before the permissions are resolved and startup refuses a supplied device module that declares a permission (`resolve_device_module` in `crates/trusted-server-core/src/integrations/registry.rs`). That is the Fastly adapter's behavior. Before this change these adapters showed the module the request's headers only in an auction, through `advertising_attributes`, which is handed the permissions. +- robots.txt is answered with an empty Edge Cookie context, so the crawler's Edge Cookie and consent signals are not read and no device module is shown its cookies, where before this change these adapters read the crawler's own context. The publisher path still parses the crawler's `Cookie` header, as it does for every request it handles (`handle_publisher_request`), and sends it on to an origin that is not signed. -**Failure handling.** A filter that fails answers the request with the error, logged as `Failed to run integration request filters`, as on Fastly. An Edge Cookie context that cannot be read answers the request with the error before the filters run, which now covers every route in the cycle, including tsjs and the discovery, signature verification and first-party routes. The errors are a selected Edge Cookie module the request cannot build and a `Cookie` header that is not UTF-8, and startup refuses a selection the adapter can never build. A filter's outbound call is bounded only by the adapter's HTTP client, which applies no module's own timeout on these adapters. Axum's client allows 5 seconds to connect and 30 in all, Cloudflare leaves the call to the Workers runtime's limit, and Spin to the Spin runtime's default (`src/platform.rs` of each adapter). +**Failure handling.** A filter that fails answers the request with the error, logged as `Failed to run integration request filters`, as on Fastly. An Edge Cookie context that cannot be read answers the request with the error before the filters run, which now covers every route in the cycle, including tsjs, discovery, signature verification, the first-party routes and the page-bids preflight. The context has two ways to fail, being a selected Edge Cookie module the request cannot build, which startup refuses where the adapter can never build it, and a `Cookie` header it cannot read as text, which answers `400`, because `handle_request_cookies` in `crates/trusted-server-core/src/cookies.rs` reads the header with `HeaderValue::to_str`, which refuses any byte outside visible ASCII, an accented letter in UTF-8 among them. A client's own `Cookie` header never fails that way, because the preparer of the `ad-tag.google.diagnostics` module, which every adapter links and runs for every request the cycle handles, first rebuilds the header from the values it can read (`sanitize_console_cookie` in `crates/ad-tag/google/src/diagnostics.rs`), so only a preparer of a deployment's own module, which runs after it, can leave such a header. A filter's outbound call is bounded only by the adapter's HTTP client, which applies no module's own timeout on these adapters. Axum's client allows 5 seconds to connect and 30 in all, Cloudflare leaves the call to the Workers runtime's limit, and Spin to the Spin runtime's default (`src/platform.rs` of each adapter). A filter that lets the request through when its call fails, as DataDome's and the 51Degrees filter do, therefore delays the request by up to that bound rather than failing it. -**Rollout.** A deployment whose modules register no filter, and that selects no device module, answers as before. It now reads the Edge Cookie context, with its geo lookup where `[geo]` selects a module, and classifies the device with the built-in module, on every route in the cycle, where it read the context for the auction, page-bids, module routes and the publisher alone. A deployment with `[shared.51degrees]` and a `[[serve]]` entry naming `51degrees.client_hints` now has the headers and the tag written on the pages the entry covers, and such a page keeps to the origin path and is sent `private, no-store`, because the filter leaves the hint names in the request's state. A deployment with DataDome's protection on now has its pages checked, and a challenge answers in place of the page. +**Rollout.** + +- A deployment whose modules register no filter, and that selects no device module a registration supplies, answers as before, apart from three things. It reads the Edge Cookie context, with its geo lookup where `[geo]` selects a module, and classifies the device with the built-in module, on every route in the cycle, where it read the context for the auction, page-bids, the module routes and the publisher alone. And a response core marked private loses any `ETag`, `Last-Modified`, `Expires` or `Age` that `[response_headers]` writes, as Response effects says. On Spin, discovery, signature verification, the first-party routes and the page-bids preflight run the modules' preparers, as on the other adapters. A client's `Cookie` header brings no new `400`, as Failure handling says. +- A deployment that selects a device module a registration supplies has that module asked to classify every request in the cycle, being tsjs, discovery, `/verify-signature`, the first-party routes, the auction, page-bids and its preflight, the module routes and the publisher's pages, each time shown the reader's `User-Agent` and cookies, beside the bid request attributes each auction already asks it for. A module that calls a service to classify makes that call for each of those requests. +- A deployment with `[shared.51degrees]` and a `[[serve]]` entry naming `51degrees.client_hints` now has the headers and the tag written on the pages the entry covers, and such a page keeps to the origin path and is sent `private, no-store`, because the filter leaves the hint names in the request's state. +- A deployment with DataDome's protection on now has each request it protects checked by the Protection API before the route, being the publisher's pages and each subresource the static asset rule does not exclude, `/auction` and `GET` on both page-bids paths, none of which is among DataDome's internal paths (`is_internal_path` in `crates/bot-protection/datadome/src/protection.rs`). A challenge then answers in place of a page or an auction call. A response that carries DataDome's cookie is kept out of shared caches, and a reader an IP rule excludes gets pages that keep to the origin path and are sent `private, no-store`. The call has no time limit of DataDome's own on these adapters, so a slow Protection API delays each protected request until the adapter's HTTP client gives up, which is 30 seconds on Axum, before DataDome lets it through. **Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, through the adapter's router built by its composition root, with a stub origin: - `a_filter_that_answers_stands_in_for_the_page`: the answer and its effect header are the response, and the origin is not asked. - `a_filter_s_request_header_reaches_the_route_and_its_effect_is_written_last`: a module route sees the request header a filter added, and the filter's response header replaces the operator's header of the same name. - `the_device_module_s_classification_reaches_the_module_call`: the module `[device]` selects classifies the request, shown its cookies, and a module route's call carries that classification. +- `platform_classifies_the_request_with_the_built_in_module`: with `[device] module = "platform"`, a module route's call carries the built-in module's classification of an iPhone `User-Agent`. +- `an_edge_cookie_context_that_cannot_be_read_answers_before_the_filters`: with a filter that answers every request, a request a module's preparer leaves with an unreadable `Cookie` header is answered `400`, with no filter's effect and no request to the origin, and a client's own unreadable `Cookie` header reaches the filter instead. - `robots_txt_is_answered_before_the_request_filters`: with a filter that answers every request, robots.txt is still the file, and the filter's header is absent. +- `robots_txt_shows_the_device_module_none_of_the_crawler_s_cookies`: the module `[device]` selects, which records the cookies it is shown, never sees the cookies sent with robots.txt, and does see those sent with a module route's request. +- `the_inspection_endpoints_and_the_attestation_pages_skip_the_request_filters`: with a filter that answers every request, the five inspection paths and the two attestation pages are answered by their own handlers with no filter's effect, and a page is the filter's answer. +- `a_filter_cannot_make_a_response_core_made_private_public`: a module route's answer core marks private keeps `no-store, private` when a filter's effect sets `Cache-Control: public, max-age=600`. +- `a_filter_s_cookie_keeps_the_response_from_shared_caches`: a public answer that a filter's effect adds a `Set-Cookie` to is sent `private, max-age=0`. +- `a_filter_cannot_add_a_shared_cache_header_to_a_private_response`: a `Surrogate-Control` that a filter's effect adds to a `private` answer is removed. +- `a_response_core_made_private_loses_the_validators_an_operator_writes`: a module route's answer core marks private, which no filter names a header for, loses the `ETag` and `Last-Modified` that `[response_headers]` writes. + +Taking any one of the three privacy guards out of the Axum adapter's `apply_terminal_response_effects` fails that guard's tests alone, and taking all three out of the Cloudflare or the Spin adapter's fails the four privacy tests. Taking out of the Axum adapter the step each of the other new tests covers fails that test alone. -**Open questions.** None. +No test runs a shipped filter, the 51Degrees middleware or DataDome's protection, through one of these adapters, and none covers `/health`, which runs no module. + +**Open questions.** + +1. Whether the 51Degrees call needs a permission, which `2026-10-14-51degrees-client-hints-design.md` leaves to the reviewer and which now arises on four adapters. +2. Whether a request filter should be refused `X-Robots-Tag`, so that no filter can loosen a site `[robots-txt]` takes out of search, on Fastly as on these adapters. Refusing it is a change to `is_forbidden_filter_header` in core, which this change leaves alone. ## 3. What is not in this change - Creating and writing the Edge Cookie, and the resolve route, on these adapters, which the next pull request of the stack covers. - Asset routes on these adapters, which the pull request after that covers. -- A timeout for a filter's outbound call on Axum, Cloudflare and Spin, which their HTTP clients do not offer. -- Moving the inspection endpoints ahead of the preparers on Axum and Cloudflare, where the Fastly adapter answers two of them before the preparers. +- A time limit for a filter's outbound call on Axum, Cloudflare and Spin. A module's limit reaches the platform only as the `first_byte_timeout` and `between_bytes_timeout` of the `PlatformBackendSpec` it registers, and a request names only its backend (`PlatformHttpRequest`). The Axum backend keeps nothing of those timeouts, naming a backend by its scheme, host, port and discriminator, and its client applies its own two limits, although reqwest's `RequestBuilder::timeout` can bound one request. The Cloudflare backend's name carries the first byte timeout and its client does not read it, although a Worker's fetch can be stopped with `Fetch::send_with_signal` and an `AbortController`. Spin's `spin_sdk::http::send` takes no time limit. Applying a module's limit is a change to each adapter's backend and HTTP client, which every outbound call goes through, the auction's calls to its providers among them, so it is left to a change of its own. +- Moving the inspection endpoints ahead of the preparers on Axum and Cloudflare, where the Fastly adapter answers all three of them, being all five paths, before the preparers. +- One request cycle in core that every adapter calls, in place of the copy each adapter carries, which would move the Fastly adapter's cycle as well. +- A test that runs a shipped filter through one of these adapters, which needs a scripted DataDome Protection API or 51Degrees service and the secret store their keys are read from. ## 4. Acceptance -The four tests pass on each of the three adapters, and the existing tests of each adapter and the cross-adapter parity tests pass unchanged. +The twelve tests pass on each of the three adapters, taking out any of the three privacy guards fails a test, and the existing tests of each adapter and the cross-adapter parity tests pass unchanged. ## Revision record -| Date | Change | -| ---------- | ------------ | -| 2026-10-17 | First draft. | +| Date | Change | +| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-17 | First draft. | +| 2026-10-18 | Fixes from review round 1. What a selected device module did on these adapters before, what DataDome's filter sends and sets, how each filter's call is bounded and why no time limit is added, the inspection endpoints the Fastly adapter answers before the preparers, the routes kept out of the cycle, which of a crawler's cookies robots.txt reads, the `Cookie` check and when it is reached, the configuration, eight new tests on each adapter with the three privacy guards among them, the open questions and the review log. | + +## Review log + +1. **Round 1, 2026-10-18.** A review in a fresh session, against `fixpass6/52-request-filters` at `f0edcd72b`. Issues found, five of them blocking. The five were that the spec said a selected device module did nothing on Axum, Cloudflare and Spin, when core asks it for a bid request's device attributes in every auction there, that nothing said what DataDome's filter sends DataDome or sets, or that no permission gates it, that the client hints design still said only the Fastly adapter runs request filters, that no test proved the privacy guards written after a filter's headers, and that the spec said the Fastly adapter answers two of the inspection endpoints before the preparers, when it answers all three, and that the adapters' HTTP clients cannot bound a call, when the Axum and Cloudflare clients can. Fixed in `7aea7a72b`, which adds the four privacy tests and four more on each adapter, `4549047dc`, which corrects the adapters' comments on robots.txt, `a584b071f`, which corrects the guides, the changelog and the middleware readme, `6b4eb0b8c`, which amends the client hints design, and this revision, which states each from the code and gives the reason a time limit is left to a change of its own. One finding was not borne out. The review said a `Cookie` header holding an accented letter now answers `400` on the routes that newly read the context, but a client's header is rebuilt by the `ad-tag.google.diagnostics` preparer first, which the context test shows, so Rollout names no new `400`. From 0af41749a050f30d75a0a0753d6932a84d9b5699 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 04:57:22 +0100 Subject: [PATCH 147/192] The client hints design says every adapter runs its filter The table of parts, the builder, the timeout_ms setting, consent, failure, rollout, section 3 and the tests no longer say only the Fastly adapter runs request filters, and a table gives what bounds the call on each adapter. --- ...026-10-14-51degrees-client-hints-design.md | 73 +++++++++---------- 1 file changed, 36 insertions(+), 37 deletions(-) diff --git a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md index fc26e91461..2fcae80f29 100644 --- a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md +++ b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md @@ -2,10 +2,10 @@ **Status:** Proposed, 2026-10-14. Raised as draft pull request IABTechLab/trusted-server#1386 (from jwrosewell/trusted-server#39), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-10-12-shared-vendor-tables-design.md` (the `[shared.51degrees]` table), `2026-10-12-loading-cache-design.md` (the `51degrees-answers` cache), `2026-10-12-page-middleware-design.md` (the `[[serve]]` entries) and `2026-10-14-source-modules-design.md` (the source a page is fetched from), all in IABTechLab/trusted-server#1342, which holds every design document of the stack +**Related specs:** `2026-10-12-shared-vendor-tables-design.md` (the `[shared.51degrees]` table), `2026-10-12-loading-cache-design.md` (the `51degrees-answers` cache), `2026-10-12-page-middleware-design.md` (the `[[serve]]` entries), `2026-10-14-source-modules-design.md` (the source a page is fetched from) and `2026-10-17-request-filters-on-every-adapter-design.md` (the request filters every adapter runs), all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** jwrosewell/trusted-server#39, IABTechLab/trusted-server#1386 **Issues:** IABTechLab/trusted-server#1333 -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-18 > **Why this spec exists.** A browser sends its high entropy client hints only when the site asks for them and delegates them, and nothing in the stack writes the headers and the tag that do so for a deployment that uses the 51Degrees service, so every first page view of a session is priced on the User-Agent alone. @@ -50,11 +50,11 @@ own, because a browser keeps an origin's set until a response names another. The three parts run in different places. -| Part | Runs | Decided by | -| --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | -| The filter, which asks the service and leaves names | Before routing, on the Fastly adapter alone, which is the only adapter that runs request filters, and it acts on page navigations only | Any `[[serve]]` entry naming the middleware whose `path` covers the request path, the rule the headers use | -| The `Delegate-CH` tag | In the serve phase, on each page | The first `[[serve]]` entry that covers the page's media type and path | -| `Accept-CH`, `Critical-CH` and `Permissions-Policy` | In the headers step of the serve phase, on each response the publisher path sends, a redirect included | Any `[[serve]]` entry whose `path` covers the request path, whatever its media type | +| Part | Runs | Decided by | +| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------- | +| The filter, which asks the service and leaves names | Before routing, on every adapter, and it acts on page navigations only | Any `[[serve]]` entry naming the middleware whose `path` covers the request path, the rule the headers use | +| The `Delegate-CH` tag | In the serve phase, on each page | The first `[[serve]]` entry that covers the page's media type and path | +| `Accept-CH`, `Critical-CH` and `Permissions-Policy` | In the headers step of the serve phase, on each response the publisher path sends, a redirect included | Any `[[serve]]` entry whose `path` covers the request path, whatever its media type | A page navigation, for the filter, is a `GET` or `HEAD` request whose `Sec-Fetch-Dest` is `document` or whose `Accept` names `text/html`, so a @@ -111,7 +111,7 @@ first navigation from each such browser waits for the service. the vendor's shared table, declares the license key as a secret setting in use whenever the table is present, and registers the filter and the middleware together. The stock list ships it, so every adapter and the `ts` -command line tool take it, although only the Fastly adapter runs its filter. +command line tool take it, and every adapter runs its filter. ```toml [shared.51degrees] @@ -129,28 +129,25 @@ middleware = ["51degrees.client_hints"] | ---------------------------------------------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `[shared.51degrees] endpoint` | Required | A URL. The call goes to the endpoint with its last path segment replaced by `json`, so a resource key written in that segment is never sent, and a query or fragment on the endpoint is dropped. The hints are delegated to the endpoint's origin | | `[shared.51degrees] license_key` | Required | The name of the license key's entry in the secret store. A table with none, or an empty one, is refused by the configuration check and when the application state is built | -| `[shared.51degrees] timeout_ms` | `500` | `1` to `10000` milliseconds. Only the Fastly adapter makes the call, and it applies the value, see "Which adapters run the filter" | +| `[shared.51degrees] timeout_ms` | `500` | `1` to `10000` milliseconds. Only the Fastly adapter applies it, see "What bounds the call on each adapter" | | `[middleware.51degrees.client_hints] critical` | `true` | `true` or `false` | | `[cache.51degrees-answers]` | `memory_seconds` 60, `store_seconds` 604800 and `store_idle_seconds` 86400 | The four settings every declared cache takes. An answer ends after an hour whatever `store_seconds` says | `[shared.51degrees]` and `[middleware.51degrees.client_hints]` each refuse a key they do not read. -**Which adapters run the filter.** Only the Fastly adapter runs request filters -(`docs/guide/permission-model.md`), and no other adapter calls -`IntegrationRegistry::filter_request`. On Axum, Cloudflare and Spin the table -is accepted, the license key is required and the middleware is registered, -but nothing runs the filter, so the service is asked nothing and no headers -and no tag are written. The table gives what `timeout_ms` does to the call -where it is made, and what the adapter's HTTP client would do with it where -it is not. - -| Adapter | Runs the filter | What `timeout_ms` does to the call | -| --------------------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Fastly | Yes | It sets the first byte and between bytes timeouts of the backend, and the connection timeout is fixed at one second (`crates/trusted-server-adapter-fastly/src/backend.rs`) | -| Axum, a development adapter | No | The call is never made. The adapter's HTTP client would ignore `timeout_ms` and allow 5 seconds to connect and 30 in all (`crates/trusted-server-adapter-axum/src/platform.rs`) | -| Cloudflare | No | The call is never made. The adapter's fetch would set no timeout, and the adapter's own note says only the Workers runtime's limit bounds a call (`crates/trusted-server-adapter-cloudflare/src/platform.rs`) | -| Spin | No | The call is never made. `spin_sdk::http::send` takes no timeout, so `timeout_ms` would be ignored, and the adapter's own note says the Spin runtime's default applies (`crates/trusted-server-adapter-spin/src/platform.rs`) | +**What bounds the call on each adapter.** Every adapter runs request filters +(`2026-10-17-request-filters-on-every-adapter-design.md`), so the filter asks +the service on each of them. A module's time limit reaches an adapter only as +the timeouts of the backend it registers for the call, and only the Fastly +adapter applies them, so `timeout_ms` bounds the call there alone. + +| Adapter | What bounds the call | +| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Fastly | `timeout_ms` sets the first byte and between bytes timeouts of the backend, and the connection timeout is fixed at one second (`crates/trusted-server-adapter-fastly/src/backend.rs`) | +| Axum, a development adapter | The adapter's HTTP client ignores `timeout_ms` and allows 5 seconds to connect and 30 in all (`crates/trusted-server-adapter-axum/src/platform.rs`) | +| Cloudflare | The adapter's fetch sets no timeout, and the adapter's own note says only the Workers runtime's limit bounds a call (`crates/trusted-server-adapter-cloudflare/src/platform.rs`) | +| Spin | `spin_sdk::http::send` takes no timeout, so `timeout_ms` is ignored, and the adapter's own note says the Spin runtime's default applies (`crates/trusted-server-adapter-spin/src/platform.rs`) | **What each navigation sends.** For a page navigation an entry covers, the filter asks the cache for the answer to one question, and where none is held @@ -163,7 +160,7 @@ address nor a cookie nor any other header is sent. A request with no `User-Agent` asks nothing. The filter reads only the three hint properties from the answer. -**Consent.** The Fastly adapter hands every request filter the permissions +**Consent.** Every adapter hands every request filter the permissions resolved for the request, so a filter can skip or narrow what it does when a permission it depends on is unset (`docs/guide/permission-model.md`). The 51Degrees filter does not read them and the middleware declares no @@ -214,12 +211,14 @@ hints is held like any other, for up to an hour. On Fastly a call that cannot connect fails after one second, and one that connects waits `timeout_ms` for the first byte and for each gap between bytes, so while the service is down every page navigation an entry covers with no held answer waits for those -limits before its page is built. The other three adapters make no call. - -**Rollout.** A deployment without `[shared.51degrees]` is unchanged. On any -adapter but Fastly, adding the table writes nothing on a page and needs only -the license key. On Fastly, adding the table and an entry naming the -middleware changes the pages that entry covers, served to a browser the service +limits before its page is built. On Axum, Cloudflare and Spin the call waits +until the adapter's HTTP client gives up, which is 30 seconds on Axum and the +runtime's own limit on Cloudflare and Spin, so while the service is down each +such navigation waits that long. + +**Rollout.** A deployment without `[shared.51degrees]` is unchanged. On every +adapter, adding the table and an entry naming the middleware changes the +pages that entry covers, served to a browser the service names hints for. The filter leaves state on the request, so core keeps the request to the origin path (`IntegrationRequestState` in @@ -247,10 +246,10 @@ the middleware covers. - Any change to core. The middleware uses the serve phase, the headers step, the request state and the middleware and shared tables the stack already has. -- **Request filters on Axum, Cloudflare and Spin.** Those adapters run none, - so the middleware asks the service nothing and writes nothing there. A - filter step added to them would call the service through HTTP clients that - apply no `timeout_ms`. +- **`timeout_ms` on Axum, Cloudflare and Spin.** Those adapters' HTTP clients + apply no module's time limit, and + `2026-10-17-request-filters-on-every-adapter-design.md` says why that is + left to a change of its own. - **Asking permission first, or holding answers longer.** The two open questions under "Open questions for the reviewer" decide those. @@ -301,8 +300,7 @@ Six things have no test. state. 6. The two crates' tests run natively under `cargo test-axum`, and their lints under `cargo clippy-axum`. No alias for the Fastly target lists them, so - they do not run for `wasm32-wasip1`, although the Fastly adapter is the only - one that runs the filter. + they do not run for `wasm32-wasip1`, the Fastly adapter's target. ## Revision record @@ -312,6 +310,7 @@ Six things have no test. | 2026-10-15 | Fixes from review round 1. What the filter runs on and that only the Fastly adapter runs it, the cost to pages, what each navigation sends and the open questions on consent, how each adapter bounds the call, the answers on failure, the settings, what the headers keep, the definition of a page navigation, the untested paths, the related specs and the review log. | | 2026-10-15 | Fixes from review round 2. The pull request that holds the design documents, the wait for the first call in the guide, and the code comments left for a later change. | | 2026-10-15 | Code fixes for the call's scope, what it sends and header merging. The permission question stays open, with what was found, and the answer lifetime is a new open question. | +| 2026-10-18 | Every adapter runs the filter, from `2026-10-17-request-filters-on-every-adapter-design.md` and its first review round. The table of parts, the builder, the `timeout_ms` setting, what bounds the call on each adapter, consent, failure, rollout, section 3 and the tests say so. | ## Review log From ba6422e40a70de94a7b37bae693d8454c9b2b65a Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 04:57:24 +0100 Subject: [PATCH 148/192] The Edge Cookie design states what the identifier discloses and that a graph replaces it Consent and privacy says core sends the identifier to the auction providers as user.id and in the auction request's id, and lets module calls read it, behind the storage and personalized-ad selection permissions, and names the two cases where this change discloses more. Rollout says a graph configured later replaces every cookie written without one, and why the graph keeps that rather than adopt such a cookie. The design also says what the bot gate is on these adapters, lists the new tests and ends with a review log of round 1. --- ...-17-edge-cookie-on-every-adapter-design.md | 63 +++++++++++++------ 1 file changed, 45 insertions(+), 18 deletions(-) diff --git a/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md index 8064eac500..040935ff36 100644 --- a/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md @@ -2,12 +2,12 @@ **Status:** Proposed, 2026-10-17. A pull request above the one that runs the request filters on every adapter, not yet raised, for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-10-17-request-filters-on-every-adapter-design.md`, `2026-10-15-cookies-by-path-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 +**Related specs:** `2026-10-17-request-filters-on-every-adapter-design.md`, `2026-10-15-cookies-by-path-design.md`, `2026-07-10-kv-eid-request-snapshot-ec-recovery-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 **Related PRs:** none yet **Issues:** none **Last updated:** 2026-10-18 -> **Why this spec exists.** Only the Fastly adapter creates and writes the Edge Cookie and serves the resolve route. On Axum, Cloudflare and Spin an Edge Cookie module reads the cookie a request carries, but no page view creates one and no response writes one, so the stack's cookies-by-path change is never reached there. This change runs the Fastly adapter's Edge Cookie steps on those three adapters, which open no identity graph, and core writes a created Edge Cookie with or without a graph, so the cookie is written on every host, the graph is optional, and without one the graph's syncs are skipped. +> **Why this spec exists.** Only the Fastly adapter creates and writes the Edge Cookie and serves the resolve route. On Axum, Cloudflare and Spin an Edge Cookie module reads the cookie a request carries and the integration proxy creates one for a navigation to a module route, but no other page view creates one and no response writes one, so the stack's cookies-by-path change is never reached there. This change runs the Fastly adapter's Edge Cookie steps on those three adapters, which open no identity graph, and core writes a created Edge Cookie with or without a graph, so the cookie is written on every host, the graph is optional, and without one the graph's syncs are skipped. ## 1. Problem @@ -18,15 +18,17 @@ The Fastly adapter runs the Edge Cookie lifecycle in four places (`crates/truste 3. Every response a route in the request cycle makes carries an `EcFinalizeState`, and the entry point runs `ec_finalize_response` on it once the response is finalized. That writes a created Edge Cookie, expires one on a withdrawal and reconciles the markers. 4. `POST /_ts/api/v1/ec/resolve` is a row of `NAMED_ROUTES`, answered by `handle_ec_resolve`. -None of the four exists on Axum, Cloudflare or Spin. Spin's `named_fallback_paths` says the resolve route is left out because the identity graph is not wired there. +Axum, Cloudflare and Spin have only the integration proxy's creation in step 2, because each hands the request's Edge Cookie context to `IntegrationRegistry::handle_proxy` as Fastly does, so a navigation to a module route creates an identifier there that the module call can read and no response writes. They have none of the other steps. Spin's `named_fallback_paths` says the resolve route is left out because the identity graph is not wired there. -Core writes a created Edge Cookie only with its identity-graph row. `ec_finalize_response` in `crates/trusted-server-core/src/ec/finalize.rs` skips the write when it is given no graph, so that on a deployment with a graph no browser carries an identifier the graph does not know. The three adapters open no graph, because the graph needs strongly consistent reads and conditional writes (`EcKvStore::key_exists` and `EcKvWriteMode::IfGenerationMatch` in `ec/kv_backend.rs`), which the platform key-value store these adapters can open does not offer (`get_bytes`, `put_bytes`, `put_bytes_with_ttl`, `delete` and `list_keys_page` in `edgezero_core::key_value_store`). Porting the Fastly steps alone would therefore create an Edge Cookie on these adapters and never write it. A Fastly service whose `[ec] ec_store` names no store writes none either, and the resolve route answers `204` with no cookie wherever it has no graph. +Core writes a created Edge Cookie only with its identity-graph row. `ec_finalize_response` in `crates/trusted-server-core/src/ec/finalize.rs` skips the write when it is given no graph, so that on a deployment with a graph no browser carries an identifier the graph does not know. The three adapters open no graph, because the graph needs strongly consistent reads and conditional writes (`EcKvStore::key_exists` and `EcKvWriteMode::IfGenerationMatch` in `ec/kv_backend.rs`), which the platform key-value store these adapters can open, edgezero's `KvStore`, does not offer. It has no strongly consistent read and no conditional write, because its `exists` reads through `get_bytes` and `KvHandle::read_modify_write` is documented as not atomic (`edgezero_core::key_value_store`). Porting the Fastly steps alone would therefore create an Edge Cookie on these adapters and never write it. A Fastly service whose `[ec] ec_store` names no store writes none either, and the resolve route answers `204` with no cookie wherever it has no graph. ## 2. Design -**The Edge Cookie request state.** Each adapter's `build_ec_request_state` reads the context as before, with the device classification, and keeps the bot gate's answer beside it in an `EcRequestState`. A request the gate stops is logged as on Fastly. No identity graph is kept, because none is opened. +**The Edge Cookie request state.** Each adapter's `build_ec_request_state` reads the context as before, with the device classification, and keeps the bot gate's answer beside it in an `EcRequestState`. A request the gate stops is logged with Fastly's message, without the JA4, platform and mobile fields Fastly adds. No identity graph is kept, because none is opened. -**Creating the Edge Cookie.** `dispatch_fallback` calls `EcContext::generate_if_needed` for a real browser's page navigation (`is_navigation_request`) before the publisher path runs, with no graph, as the Fastly adapter does when it has none. The integration proxy creates one for a navigation as before, in core. +**The bot gate on these adapters.** The gate is the answer of the device module `[device]` selects (`DeviceSignals::looks_like_browser`). On these adapters `platform` classifies from the `User-Agent` alone, as the built-in default does, so any client that sends a browser's `User-Agent` passes, where Fastly's `platform` module requires a TLS signal as well as a recognized platform (`DeviceSignals::derive`). Before this change that answer decided nothing about the Edge Cookie on these adapters, and now it decides whether a page view creates one, so a `[device]` module that answers `DeviceSignals::unknown()` stops every created Edge Cookie, there as on Fastly. + +**Creating the Edge Cookie.** `dispatch_fallback` calls `EcContext::generate_if_needed` for a real browser's page navigation (`is_navigation_request`) before the publisher path runs, with no graph, as the Fastly adapter does when it has none. The integration proxy creates one for a navigation to a module route as before, in core, for any client, because it does not read the bot gate, and finalizing writes it only for a request that passed. **Writing it.** Every response a route in the request cycle makes carries an `EcFinalizeState`, holding the context and the request's services. These are the responses the Fastly adapter attaches it to, being the named routes but the inspection endpoints, the fallback's tsjs, module route and publisher answers, and a filter's answer. robots.txt carries none, as on Fastly. `FinalizeResponseMiddleware` takes the state off the response once it has written the finalize headers and runs `ec_finalize_response` with no graph, which applies a module's response headers, writes an Edge Cookie the request created, expires the cookie and its markers on a withdrawal, and reconciles the pull-sync marker. The request filters' response effects are applied after it, as on Fastly. The `sharedId` cookie the Fastly adapter carries to finalizing is read only by the graph's identifier writes, so it is not carried. @@ -42,25 +44,37 @@ Core writes a created Edge Cookie only with its identity-graph row. `ec_finalize | Pull sync after the response is sent | When the graph and the partners allow it | Skipped | | `POST /_ts/api/v1/ec/resolve` | A row of `NAMED_ROUTES` | A named route on each adapter, in the request cycle | -**The resolve route.** Each adapter registers `POST /_ts/api/v1/ec/resolve` as a named route the way it registers its other named routes, so the request passes the preparers, the Edge Cookie context and the request filters, as on Fastly. It is answered by `handle_ec_resolve` with no graph, which answers as it does with one, so its refusals are the Fastly route's, being `403` for a missing or foreign `Origin` and `415` for another content type among them, and a client-side module's created Edge Cookie is set with its marker and no graph row. A request that did not pass the bot gate gets `204` with no cookie, as on Fastly. +**The resolve route.** Each adapter registers `POST /_ts/api/v1/ec/resolve` as a named route the way it registers its other named routes, so the request passes the preparers, the Edge Cookie context and the request filters, as on Fastly. It is answered by `handle_ec_resolve` with no graph, which answers as it does with one, so its refusals are the Fastly route's, being `403` for a missing or foreign `Origin` and `415` for another content type among them, and a client-side module's created Edge Cookie is set with its marker and no graph row. The bot gate is checked after the module has been asked and after the `400` and `409` answers, at the point where, before this change, the Fastly route found no graph for such a request, so a request that did not pass it still drives the module and can still get those answers, and otherwise gets `204` with no cookie, as on Fastly. **One startup warning.** Where `[ec] ec_store` names an identity graph store, each adapter logs once, as it builds its state, that it opens no identity graph, so the Edge Cookie is still written but the data the graph would keep with it, such as partner IDs, is not. Cloudflare and Spin build their state for every request, so a flag keeps the line to one for each instance. -**Configuration.** None. `[ec] ec_store` is accepted on these adapters as before, and logged once. +**Configuration.** No key is added. `[ec] ec_store` is accepted on these adapters as before, and logged once. `[device]` now decides through the bot gate whether a page view creates an Edge Cookie on these adapters, as described above. + +**Consent and privacy.** A created Edge Cookie is written only where core's permission gate allows the Edge Cookie module and the request passed the bot gate, on every host, with a graph or without. A withdrawal expires the cookie and its markers, as on Fastly, and there is no graph row to tombstone. Without a graph no partner identifier is written against the Edge Cookie, and no pull sync, batch sync or identify call reads one. + +The identifier itself leaves the edge without a graph as it does with one. Where the request's permissions set what the Edge Cookie module requires together with storage and personalized-ad selection (`EcContext::ec_sharing_allowed`, being `StoreOnDevice` and `SelectPersonalisedAds`), the publisher path's server-side auction and `GET /_ts/page-bids` send it to each auction provider as the auction request's `user.id`, and as `ts-` in that request's `id`, and `POST /auction` sends it as `user.id` (`publisher.rs`, `auction/endpoints.rs` and `auction/formats.rs` in core). A module call handed the request's state, such as a module route's handler or a document's serve-phase preparation, can read it behind the permissions the Edge Cookie module declares (`ModuleContext::with_request_state`), which for `hmac` is storage alone. That includes the identifier the integration proxy creates for a navigation to a module route by any client whose permissions allow the Edge Cookie, which the route's module call can read even when the bot gate stopped the request, though no response writes it, as on Fastly. The sample rules set storage and personalized-ad selection in the US with no signal. So this change discloses more in two cases. -**Consent and privacy.** A created Edge Cookie is written only where core's permission gate allows the Edge Cookie module and the request passed the bot gate, on every host, with a graph or without. A withdrawal expires the cookie and its markers, as on Fastly, and there is no graph row to tombstone. Without a graph no partner identifier is written against the Edge Cookie, and no pull sync, batch sync or identify call reads one, so the Edge Cookie stays a first-party identifier in the browser. +- On Axum, Cloudflare and Spin the publisher path created no identifier and no response wrote one, so an auction there carried a `user.id` only for a browser that already carried an Edge Cookie, which nothing on these adapters wrote. Now a real browser's first page navigation creates one before the publisher path runs its auction, and the same value reaches the auction providers on that page view and on every later one whose permissions set both, for the year the cookie lasts. +- On a Fastly service with no `ec_store`, `hmac` created a new value on every page view, the HMAC of the client IP with a fresh random suffix, and no response wrote it, so the `user.id` of each page view was a new value whose 64-character HMAC part was the same for every page view from one address. Now one value follows the browser across page views and addresses for the year the cookie lasts. + +This disclosure is intended. It is what a Fastly service with an `ec_store` does for the same browser, because the Edge Cookie is the identifier and the permission model, not the identity graph, decides whether it is shared. **Failure handling.** A failure to create the Edge Cookie is logged and the page is served without one, as on Fastly. A partner registry that cannot be built from `[ec] partners` skips the finalize step with a logged error, as on Fastly. -**Rollout.** A deployment that selects no Edge Cookie module is unchanged. One that selects a server-side module, such as `hmac`, now sets the Edge Cookie on a real browser's first page navigation where the permissions allow it, on these adapters and on a Fastly service with no `ec_store`, and that response is kept out of shared caches because it sets a cookie (`enforce_set_cookie_cache_privacy`). A returning browser keeps its cookie. A client-side module, such as `client_fixed`, gets its cookie from the resolve route on those hosts too. A crate outside core that finalizes the Edge Cookie or answers the resolve route sets the device signals on the context, or no created cookie is written. +**Rollout.** A deployment that selects no Edge Cookie module creates and writes no Edge Cookie, as before, but finalizing now runs on these adapters, so there a withdrawal expires an Edge Cookie or pull-sync marker the browser carries, as on Fastly. One that selects a server-side module, such as `hmac`, now sets the Edge Cookie on a real browser's first page navigation where the permissions allow it, on these adapters and on a Fastly service with no `ec_store`, and that response is kept out of shared caches because it sets a cookie (`enforce_set_cookie_cache_privacy`). A returning browser keeps its cookie while no identity graph sees it. A client-side module, such as `client_fixed`, gets its cookie from the resolve route on those hosts too. + +**A graph configured later replaces these cookies.** When a Fastly service that ran with no `ec_store` names one, or a deployment on one of these adapters moves to a Fastly service with a graph, each browser holding a cookie written without a graph gets a new identifier on the first page navigation it makes there that passes the bot gate and whose permissions allow the Edge Cookie. The publisher path reads the cookie's row for that navigation, because no valid pull-sync marker lets it skip the read and a marker is only set from a row, and finalizing proves the row absent and gives the browser a new identifier with a row (`confirm_then_recover_orphaned_ec` in `ec/finalize.rs`). This design keeps that rather than have the graph adopt such a cookie by creating its row. An identifier a browser carries cannot be authenticated, because the `hmac` module's random suffix is unsigned, so the graph cannot tell a cookie written without a graph from one a client made up or one whose withdrawal tombstone, which lasts 24 hours, has expired, and adopting it would start a graph row from an unverified cookie, which `2026-07-10-kv-eid-request-snapshot-ec-recovery-design.md` rules out. A deployer that adds a graph later therefore resets each reader's identifier once, and the `user.id` the auction providers see changes with it. -**Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, through the adapter's router, with a stub origin and a reader whose services place them in the US, where the sample permission rules grant the Edge Cookie with no signal, and `[geo] module = "platform"`, which leaves that location standing: +**Crates outside core.** A crate outside core that finalizes the Edge Cookie or answers the resolve route sets the device signals on the context, or no created cookie is written. This is breaking for such a crate, and no compiler catches it. `generate_if_needed` does not read the bot gate, so such a crate that passes a graph and sets no device signals gets a graph row for an identifier no response writes. The adapters in this repository cannot reach that, because Fastly passes no graph for a request the gate stops and the others pass none. + +**Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, through the adapter's router, with a stub origin and a reader whose services place them in the US and carry the permission signal modules the adapter links, where the sample permission rules grant the Edge Cookie with no signal, and `[geo] module = "platform"`, which leaves that location standing: - `a_first_page_view_gets_the_edge_cookie_under_the_public_prefix_and_path`: a page view addressed by path gets `ts-ec` with `Path=/publisher.example`, and the origin is asked at the continuing path. - `a_returning_page_view_keeps_its_edge_cookie`: a page view addressed by host gets `ts-ec` under `/`, and the same page view carrying that cookie gets none. - `a_bot_s_page_view_gets_no_edge_cookie`: a page view whose `User-Agent` names a crawler gets none. +- `a_withdrawal_expires_the_edge_cookie_the_browser_carries`: with no `[geo]` module instead, so the reader resolves at the top of the sample rules, where storage needs a signal, a page view carrying `ts-ec` and a TC String that refuses storage gets `ts-ec` expired with `Max-Age=0`. - `the_resolve_route_answers`: a foreign `Origin` gets `403` with `Cache-Control: no-store` and the publisher's gets `204` with no cookie, the answer of a server-side module, which creates nothing from a posted value, and neither reaches the origin. -- `startup_warns_that_this_adapter_opens_no_identity_graph`: building the state with `[ec] ec_store` set logs the warning. +- `startup_warns_that_this_adapter_opens_no_identity_graph`: building the state with `[ec] ec_store` set sets the flag the warning is logged under, and the warning's check with no `ec_store` sets none. The flag is set as the line is logged, so the test shows the warning's branch runs, not that a line reaches the log. The tests' `edge_cookie` helper also fails a response that writes the Edge Cookie more than once, so the first page view shows core alone writes it. @@ -71,26 +85,39 @@ In core: - `resolve_answers_as_it_does_with_a_graph_when_there_is_none` in `ec/resolve.rs`: with no graph the route answers `200` with the Edge Cookie, its marker and `Cache-Control: no-store`, the headers it gives with a graph. - `resolve_creates_nothing_for_a_request_that_did_not_pass_the_bot_gate` in `ec/resolve.rs`: a crawler, or a request with no device signals, gets `204` with no cookie and no row, with a graph and without. - `the_bot_gate_passes_a_browser_and_stops_a_crawler_or_a_request_with_no_signals` in `ec/mod.rs`. +- `finalize_rotates_orphaned_cookie_to_new_backed_ec` in `ec/finalize.rs`, which this change leaves as it is: a well-formed cookie whose row is missing, finalized with a graph on a page view that may recover, gets a new identifier with a row, which is the replacement Rollout describes. + +In the Fastly adapter, under Viceroy, `with_no_ec_store_the_entry_point_writes_a_created_edge_cookie` in `src/app.rs`: with no `[ec] ec_store`, `build_ec_request_state` opens no graph for a browser's page view, `generate_if_needed`, the step `dispatch_fallback` runs for one, creates an Edge Cookie, and `apply_edgezero_ec_finalize`, the entry point's finalize step, writes it once. The test calls the three steps in turn rather than routing a page view through `dispatch_fallback` and `edgezero_main`. -In the Fastly adapter, under Viceroy, `with_no_ec_store_the_entry_point_writes_a_created_edge_cookie` in `src/app.rs`: with no `[ec] ec_store`, a browser's page view builds no graph, creates an Edge Cookie, and the entry point's finalize step writes it once. +No adapter test sees a `200` from the resolve route, because the adapter tests select `hmac`, which creates nothing from a posted value, and the `client_fixed` module is compiled only behind core's `client-fixed-demo` feature, which no adapter passes on. Core's `resolve_answers_as_it_does_with_a_graph_when_there_is_none` covers that answer with no graph, and `the_resolve_route_answers` covers the route on each adapter. **Decisions.** Two questions this design left open, whether the resolve route should write a client-side module's cookie on an adapter that opens no graph and whether a Fastly service with no `ec_store` should write a created Edge Cookie, are both answered yes. The Edge Cookie is the identifier, and the identity graph is one store of the data associated with it. The permission state comes from the permission model, whose sources may be values the graph holds, signals the client sends or any other source, so a missing graph does not stop the cookie. So the Edge Cookie is the identifier, the identity graph is one optional store of the data associated with it, the partner IDs among it, and the permission state comes from the permission model whatever its sources. A missing graph does not stop the cookie. +Where a graph is configured it still replaces an identifier it has no row for, as Rollout describes. That is the graph refusing to start a row from an identifier it cannot verify, not a condition on the cookie, and without a graph nothing replaces a cookie. + ## 3. What is not in this change - An identity graph on Axum, Cloudflare or Spin, which needs a strongly consistent store with conditional writes. - `/_ts/api/v1/identify`, `/_ts/api/v1/batch-sync` and pull sync on those adapters, which need the graph, and the tester routes, which no other adapter registers. - Asset routes on these adapters, which the next pull request of the stack covers. +- A graph adopting an Edge Cookie it has no row for by creating the row instead of replacing the identifier, which needs a way to tell a cookie written without a graph from one a client made up or one whose withdrawal tombstone has expired, such as a signed identifier. +- A bot gate in `generate_if_needed`, which would keep a crate outside core that passes a graph and sets no device signals from writing a row for an identifier no response writes. +- A `200` from the resolve route in the adapters' tests, which needs a client-side module those tests can select. ## 4. Acceptance -The five tests pass on each of the three adapters, the core and Fastly tests above pass, and the existing tests of each adapter, core's Edge Cookie tests and the cross-adapter parity tests pass. +The six tests pass on each of the three adapters, the core and Fastly tests above pass, and the existing tests of each adapter, core's Edge Cookie tests and the cross-adapter parity tests pass. ## Revision record -| Date | Change | -| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-17 | First draft. | -| 2026-10-18 | Core writes a created Edge Cookie with or without an identity graph, behind the bot gate it reads from the device signals, and the resolve route answers without a graph as it does with one, which answers both open questions. | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-17 | First draft. | +| 2026-10-18 | Core writes a created Edge Cookie with or without an identity graph, behind the bot gate it reads from the device signals, and the resolve route answers without a graph as it does with one, which answers both open questions. | +| 2026-10-18 | Fixes from review round 1. The identifier's disclosure to auction providers and module calls with no graph and the two cases where it grows, the replacement of these cookies once a graph sees them and why a graph does not adopt one, the integration proxy's creation the three adapters already ran, the bot gate on these adapters, the order of the resolve route's checks, the store's methods, the log line, a deployment with no module, crates outside core, a test of a withdrawal through each router, the startup warning's case with no `ec_store`, what the Fastly test does, the answer no adapter test covers, what is not in this change and the review log. | + +## Review log + +1. **Round 1, 2026-10-18.** A review in a fresh session, against `fixpass6/53-edge-cookie` at `198ce56db`. Issues found, two of them blocking. The two were that the spec said the Edge Cookie stays a first-party identifier in the browser without a graph, when core sends it to the auction providers as `user.id` and in the auction request's `id` and lets module calls read it, so this change gives the providers one value for a year where they had none or a new one on each page view, and that the spec said a returning browser keeps its cookie without saying that a graph configured later replaces every cookie written without one, which Decisions also left unexplained. Fixed in `960d9c61f`, which corrects the changelog and the Edge Cookie and Fastly guides, in `dbbbcfa14`, which tests a withdrawal through each adapter's router and the startup warning with no `ec_store`, and in this revision, which states the disclosure and why it is intended, states the replacement and why the graph keeps it rather than adopt such a cookie, and fixes the non-blocking findings that were cheap and certain. A graph that adopts such a cookie, a bot gate in `generate_if_needed` and a `200` from the resolve route in the adapters' tests are listed under What is not in this change. From dab7138d72826c9ac5b6f428e95ada6c69f22caa Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 05:28:40 +0100 Subject: [PATCH 149/192] The asset routes design answers review round 1 The design states Spin's refusal of a source's host_header and what Cloudflare does with one, the startup warning for a route that enables the Image Optimizer and why it is a warning, the HEAD a signed route sends, the startup refusal of a source that cannot be built, the per-request failures with their statuses, the body limits and timeouts, and that an asset response neither writes nor expires the Edge Cookie. It lists the new tests, leaves the Cloudflare host_header and HEAD length as open questions, and ends with a review log entry for round 1. --- ...17-asset-routes-on-every-adapter-design.md | 92 +++++++++++++------ 1 file changed, 66 insertions(+), 26 deletions(-) diff --git a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md index dbc81ea696..7f9955d223 100644 --- a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md @@ -1,11 +1,11 @@ # Design Spec: Asset routes are served on every adapter -**Status:** Proposed, 2026-10-17. A pull request above the one that creates and writes the Edge Cookie on every adapter, not yet raised, for Tech Lab review. +**Status:** Proposed, 2026-10-17. Implemented as the draft pull request [IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412), row 54 of the stack, above [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411), which creates and writes the Edge Cookie on every adapter, for Tech Lab review. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-17-request-filters-on-every-adapter-design.md`, `2026-10-17-edge-cookie-on-every-adapter-design.md`, `2026-10-14-source-modules-design.md`, `2026-10-14-origin-signing-design.md`, `2026-10-15-asset-cache-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 -**Related PRs:** none yet +**Related PRs:** [IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412). The request filters run on every adapter in [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410) and the Edge Cookie in [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411) **Issues:** none -**Last updated:** 2026-10-17 +**Last updated:** 2026-10-18 > **Why this spec exists.** Only the Fastly adapter serves `[[proxy.asset_routes]]`. On Axum, Cloudflare and Spin a request under a route's prefix goes to the publisher's origin as an ordinary page request, so a route's own source, its signing and its held answers are never used there. This change dispatches asset routes on those three adapters as the Fastly adapter does. @@ -17,49 +17,89 @@ No code in the Axum, Cloudflare or Spin adapter calls `handle_asset_proxy_reques ## 2. Design -**Dispatch.** Each adapter's `dispatch_fallback` checks a `GET` or `HEAD` against the asset routes where the Fastly adapter does, and `dispatch_asset_fallback` follows the Fastly function of that name: the route's source, `handle_asset_proxy_request`, the response's cache policy in its extensions, and the request filters' effects. An error answers with the error and the `no-store, private` policy, as on Fastly. No Edge Cookie state is attached, so the finalize middleware writes no Edge Cookie on an asset response. +**Dispatch.** Each adapter's `dispatch_fallback` checks a `GET` or `HEAD` against the asset routes where the Fastly adapter does, and `dispatch_asset_fallback` follows the Fastly function of that name: the route's source, `handle_asset_proxy_request`, the response's cache policy in its extensions, and the request filters' effects. An error from the handler answers with the status the error's kind gives and the `no-store, private` policy, as on Fastly. No Edge Cookie state is attached, so the finalize middleware neither writes nor expires an Edge Cookie on an asset response. The Fastly adapter is unaffected, because this change touches no file of the Fastly adapter or of core. -**Finalizing.** `FinalizeResponseMiddleware` takes the cache policy off the response and applies it after the finalize headers with the adapter's own shared cache header (`AssetProxyCachePolicy::apply_after_route_finalization`), before the Edge Cookie and the filters' effects, the order the Fastly entry point uses. +**Finalizing.** `FinalizeResponseMiddleware` takes the cache policy off the response and applies it after the finalize headers with the adapter's own shared cache header (`AssetProxyCachePolicy::apply_after_route_finalization`), before the Edge Cookie and the filters' effects, the order the Fastly entry point uses. Axum and Spin write a shared cache lifetime as `s-maxage` in `Cache-Control`, and Cloudflare writes it in `cloudflare-cdn-cache-control`. -**Held answers and the body.** `handle_asset_proxy_request` keeps and reads its answers through the request's loading caches and value cache, as on Fastly. Cloudflare and Spin rebuild their state for every request and have no value cache, so an answer is held across requests only by a loading cache module that keeps its copies in the platform's key-value store, when the services carry one. None of the three HTTP clients streams a response (`supports_streaming_responses`), so core asks for none and the asset body is held whole, where the Fastly adapter streams it. +**Held answers.** `handle_asset_proxy_request` reads and keeps its copies through the loading caches `[cache.assets]` selects, as on Fastly, and keeps no copy of the answer to a request that carries Image Optimizer metadata. The stock loading cache module keeps a memory layer for each application state and its shared copies in the host's value cache where the services carry one, which only the Fastly adapter supplies, and in the key-value store otherwise (`crates/loading-cache/51degrees/src/stores.rs`). Cloudflare and Spin build their state for every request, so there a copy outlives its request only in the key-value store each declares, `TRUSTED_SERVER_KV` on Cloudflare and `default` on Spin. Axum builds its state once for the process, but its key-value store is the unavailable one, so a copy lives only in memory, for `[cache.assets] memory_seconds`, 5 seconds by default. An asset fetch also asks for the platform's default cache behavior (`PlatformCacheIntent::Default`), which the Cloudflare client leaves to the Workers runtime, whose subrequests are eligible for Cloudflare's cache by default, so on Cloudflare the runtime may hold an answer as well. -**Image Optimizer.** A route that enables `[proxy.asset_routes.image_optimizer]` sets Image Optimizer metadata on the request, which the Fastly adapter translates and the other three cannot. The Cloudflare and Spin HTTP clients already refuse such a request, as the `PlatformHttpRequest` contract asks of a client that cannot honor it, and the Axum client now refuses it too where it would have sent it unchanged. The request then answers with the error. +**The body.** None of the three HTTP clients streams a response (`supports_streaming_responses` is `false` there), so core asks for no stream and each client holds the asset body whole, where the Fastly adapter streams it with no limit. The Cloudflare client refuses a body over 10 MiB (`MAX_PLATFORM_RESPONSE_BODY_BYTES`) and the Spin client one over 8 MiB, before or after it is decompressed (`MAX_DECOMPRESSED_SIZE`), and the Axum client holds any size. The Cloudflare client writes the length of the body it read as the `Content-Length`, so its answer to a `HEAD` says `0`, where the Fastly and Spin adapters keep the origin's length. -| Adapter | Source and signing | Held answers | Body | Image Optimizer | -| ---------- | ------------------ | ------------------------------------------------------ | ---------- | ------------------------------- | -| Fastly | Yes | Loading caches and the value cache | Streamed | Translated | -| Axum | Yes | The loading caches of the process | Held whole | Refused by the client, an error | -| Cloudflare | Yes | A loading cache module's copies in the key-value store | Held whole | Refused by the client, an error | -| Spin | Yes | A loading cache module's copies in the key-value store | Held whole | Refused by the client, an error | +**A source's `Host`.** A route's source can name the `Host` its origin is asked with (`host_header`), which core writes before the signer signs it. The Fastly adapter sends it through its backend's host override, and the Axum client hands it to its HTTP library, which adds a `Host` only to a request that has none. Spin sends the address's own host as the `Host` of every request, because WASI HTTP takes the authority from the address and forbids a `host` header, so a source whose `host_header` names another host would reach its origin with the wrong `Host`, and a signed one with a signature made for another. The Spin backend (`NoopBackend::ensure` in `crates/trusted-server-adapter-spin/src/platform.rs`) therefore refuses to name a backend asked for a `Host` other than the address's own, with its port where that is not the scheme's default, and core answers the request with `502` before anything is sent. The publisher's own source reaches the same backend, so a publisher source with such a `host_header` is refused on Spin too. The Cloudflare client hands the header to `fetch` as given, and Cloudflare documents that a Worker cannot set a `Host` naming a host outside its zone unless the request goes to that host ([Request](https://developers.cloudflare.com/workers/runtime-apis/request/)), so there a `host_header` naming such a host does not reach the origin. -**Configuration.** None. `[[proxy.asset_routes]]` now takes effect on every adapter. +**Image Optimizer.** A route that enables `[proxy.asset_routes.image_optimizer]` sets Image Optimizer metadata on each request whose path is not an SVG or SVGZ file and whose query does not set the profile set's debug parameter to `1` (`asset_path_skips_image_optimizer` and `options_for_asset_request` in core). The Fastly adapter translates the metadata, and the other three cannot. The Cloudflare and Spin HTTP clients already refused such a request on both send paths, as the `PlatformHttpRequest` contract asks of a client that cannot honor it, and the Axum client now refuses it on `send` and `send_async`, where it sent the request unchanged before. A refused request answers `502 Bad Gateway` with `no-store, private`. Before the refusal, a route whose source signs asks its origin for the object with a `HEAD` (`preflight_signed_origin_for_image_optimizer`), and where that `HEAD` fails the request is answered with the origin's own error, with no metadata and with `no-store, private`, as on Fastly. An SVG path and a debug request carry no metadata, so they are served unoptimized and their answers are kept as any other asset's are. -**Consent and privacy.** An asset route forwards only `ASSET_PROXY_FORWARD_HEADERS` to the route's origin, as on Fastly, so the reader's cookies are not sent there. Before this change the request went to the publisher's origin with the headers the publisher path forwards. +| Adapter | Source and signing | A source's `host_header` | Held answers | Body | Image Optimizer | +| ---------- | ------------------ | ---------------------------------- | --------------------------------------------- | ------------------------ | ---------------------------- | +| Fastly | Yes | Sent | Loading caches, copies in the value cache | Streamed, no limit | Translated | +| Axum | Yes | Sent | Loading caches, copies in memory alone | Held whole, no limit | Refused by the client, `502` | +| Cloudflare | Yes | Not sent outside the Worker's zone | Loading caches, copies in `TRUSTED_SERVER_KV` | Held whole, up to 10 MiB | Refused by the client, `502` | +| Spin | Yes | Refused by the backend, `502` | Loading caches, copies in `default` | Held whole, up to 8 MiB | Refused by the client, `502` | -**Failure handling.** A route whose source cannot be built, an origin that cannot be reached and a request the client refuses each answer with the error and `no-store, private`, as on Fastly. +**Configuration.** No key is added or changed, and `[[proxy.asset_routes]]` now takes effect on every adapter. At startup the Axum, Cloudflare and Spin adapters each log one warning naming the prefix of every route that enables the Image Optimizer (`warn_once_when_an_asset_route_enables_the_image_optimizer`), and then serve the route. They warn rather than refuse the settings, because such a route still answers its SVG paths, its debug requests and, where it signs, the origin's own errors, because the deployment's other routes and pages do not depend on it, and because a refusal would stop a Spin or Cloudflare deployment, or an Axum dev server, that shares its settings with a Fastly service whose routes optimize images. -**Rollout.** A deployment with no asset routes is unchanged. One with asset routes now has their requests answered by each route's origin on Axum, Cloudflare and Spin, where the publisher's origin answered them, and a route that enables the Image Optimizer answers with an error there. +**Consent and privacy.** An asset route forwards only `ASSET_PROXY_FORWARD_HEADERS` to the route's origin, plus the `Host` and the signer's headers, as on Fastly, so the reader's cookies are not sent there. Before this change the request went to the publisher's origin with the headers the publisher path forwards. An asset response skips the Edge Cookie steps (`ec_finalize_response` does not run), so it neither writes a created Edge Cookie nor expires the cookie and its markers when the reader has withdrawn permission, which matches Fastly. Before this change the publisher fallback did both for the same requests on these adapters, and a response from the publisher path still does. -**Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, with a stub origin and an asset route whose source signs with the stock `s3_sigv4` signer: +**Failure handling.** -- `a_request_under_an_asset_route_reaches_the_route_s_origin_signed`: the route's origin is asked, at the route's address, with an `AWS4-HMAC-SHA256` `Authorization` header, and its answer is the response. +- A route whose source cannot be built stops the adapter when it builds its state, as the publisher's source does, because the registry builds every source from the settings (`build_sources`). That happens at startup on Axum and for every request on Cloudflare and Spin, and each request is then answered with the startup error response, `500` on Fastly, Axum and Cloudflare and `503` on Spin. `route_source` answers an error for a single request only for a registry built without settings, which no adapter's state builder makes. +- Each of these answers one request with the status its error gives, and with `no-store, private`, on every adapter: + - a source module's `locate` error, with the source's own status, where core's `origin` source never fails + - a request path the route's `path_pattern` does not match, `502`, a rewrite that does not start with `/`, `500`, and a target path with a `.` or `..` segment under a source folder, `400` + - an unknown image profile in `reject` mode, `400` + - a signer that cannot sign, with the signer's status, which for `s3_sigv4` is `400` when a header it signs or writes is not valid + - a backend the platform refuses, `502`, which on Spin is a source whose `host_header` names another host + - an origin that cannot be reached, a body over the client's limit and a request the client refuses, such as one carrying Image Optimizer metadata, `502` +- An origin's own error status is passed through as its answer, and no cache rule is applied to it. +- Core asks the route's backend for a first byte timeout and a between bytes timeout of 15 seconds (`DEFAULT_FIRST_BYTE_TIMEOUT`), which the Fastly adapter applies. The Axum client applies limits of its own instead, 5 seconds to connect and 30 seconds for the whole request, and the Cloudflare and Spin backends only make a name, so there a slow origin is bounded by the runtime's own limits. `2026-10-15-asset-cache-design.md` says the same of the asset caches. + +**Rollout.** + +1. A deployment with no asset routes is unchanged, because `asset_route_index_for_path` answers `None` when there are none. +2. One with asset routes now has their `GET` and `HEAD` requests answered by each route's origin on Axum, Cloudflare and Spin, where the publisher's origin answered them, with the route's signing and cache rule, and with no Edge Cookie written or expired. +3. On those adapters a route that enables the Image Optimizer answers each request it would optimize with `502` and `no-store, private`, where the publisher's origin answered it before, after a signed route has sent its origin the `HEAD`. Its SVG paths and debug requests are served unoptimized, and startup logs a warning naming each such route. +4. On Spin a request to a source whose `host_header` names another host answers `502` and its origin is not asked, a route's source and the publisher's alike, where before the request reached the origin with the address's own host as the `Host`. A Spin deployment whose publisher source sets such a `host_header` therefore answers every page with `502`. +5. A body over 10 MiB on Cloudflare or 8 MiB on Spin answers `502`. + +**Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, with a stub origin: + +- `a_request_under_an_asset_route_reaches_the_route_s_origin_signed`: with a route whose source signs with the stock `s3_sigv4` signer, the route's origin is asked, at the route's address, with an `AWS4-HMAC-SHA256` `Authorization` header, and its answer is the response. - `a_request_outside_the_asset_routes_reaches_the_publisher_s_origin`: the publisher's origin is asked, unsigned. +- `an_asset_route_s_cache_rule_is_written_after_the_finalize_headers`: with a `[[cache.asset_rules]]` rule of a minute for browsers and an hour for shared caches, the answer carries `public, max-age=60, s-maxage=3600` on Axum and Spin, and `public, max-age=60` with `cloudflare-cdn-cache-control: max-age=3600` on Cloudflare. +- `an_asset_route_whose_origin_fails_answers_502_and_is_not_stored`: an origin that fails gives `502` with `cache-control: no-store, private`. +- `a_page_view_under_an_asset_route_gets_no_edge_cookie`: a browser's page navigation under the route, whose permissions would grant an Edge Cookie on a page, gets none, and the route's origin answers it. +- `startup_warns_when_an_asset_route_enables_the_image_optimizer`: building the router with such a route logs the warning. + +The Spin adapter's request cycle tests also send requests through the backend its own services use. In `an_asset_route_whose_source_names_another_host_is_refused_before_its_origin_is_asked` the route whose source names another host answers `502` with `no-store, private` and only the route without one reaches its origin, and in `a_publisher_source_that_names_another_host_is_refused_before_its_origin_is_asked` a page request answers `502` and the publisher's origin is not asked. Its platform tests `a_backend_asked_to_send_another_host_is_refused` and `a_backend_asked_to_send_the_address_s_own_host_is_named` cover another name, another port and the same host written with its default port or in other letter case. The Axum adapter's platform test `a_request_with_image_optimizer_metadata_is_refused` sends through `send` and `send_async`. -The Axum adapter's platform tests gain `a_request_with_image_optimizer_metadata_is_refused`. +Each test was run with the code it guards taken out, and failed. Without the finalize middleware's cache policy step the cache rule test sees `public, max-age=60` and no shared lifetime, and the error test sees no `Cache-Control`, on each adapter, and so does the Spin route refusal test, which also checks `no-store, private`. Without `NoStorePrivate` in the error arm the error test fails on each adapter, an asset response given the publisher path's Edge Cookie state writes a `ts-ec` cookie and fails the Edge Cookie test on each adapter, and a state builder without the warning call fails the warning test on each adapter. With Spin's refusal disabled, `a_backend_asked_to_send_another_host_is_refused` fails and both Spin request cycle tests answer `200`, while `a_backend_asked_to_send_the_address_s_own_host_is_named` still passes, and with the `send_async` check taken out the Axum client test fails at its `send_async` assertion. -**Open questions.** None. +No test sends an asset request through the Cloudflare or Spin HTTP client, whose refusals, body limits and `Content-Length` handling are wasm32 code that the native tests do not run, apart from Spin's `reject_unsupported_request_contracts`, which has native tests of its own. The cross-adapter parity tests have no asset route case. + +**Open questions.** + +1. Should the Cloudflare adapter refuse a source's `host_header` the way Spin now does? Cloudflare documents that a Worker cannot set a `Host` naming a host outside its zone unless the request goes to that host, but the repository has no test of the Workers runtime that shows what it sends for a `host_header` inside the zone, so this change leaves the Cloudflare client handing the header to `fetch` as before. +2. Should the Cloudflare client keep the origin's `Content-Length` on a `HEAD`, as the Fastly and Spin adapters do? The repository does not show whether the Workers runtime corrects the header on the way out, and the origin's length describes the body as it was encoded, which the runtime decodes. ## 3. What is not in this change -- An Image Optimizer on Axum, Cloudflare or Spin. +- An Image Optimizer on Axum, Cloudflare or Spin. Cloudflare offers image transformations to a Worker through the `cf.image` option of `fetch` ([Transform via Workers](https://developers.cloudflare.com/images/optimization/transformations/transform-via-workers/)), and the adapter does not translate the metadata into it. - Streaming an asset body on those adapters, whose HTTP clients hold every response whole. +- Applying the 15-second timeouts core asks for on Axum, Cloudflare and Spin. +- Sending a source's `host_header` on Spin, whose outbound HTTP sends the address's own host. +- A parity test of asset routes. The parity tests build each router with its own services, where the Cloudflare and Spin HTTP clients are unavailable on a native host and the Axum client is a real one, so an asset request reaches no origin there, and the request cycle tests send the same requests through each adapter's composition root with a stub origin instead. ## 4. Acceptance -The tests pass on each of the three adapters, and the existing tests of each adapter and the cross-adapter parity tests pass. +The tests above pass on each of the three adapters, and the existing tests of each adapter pass. The cross-adapter parity tests pass too, though none of them sends an asset request. ## Revision record -| Date | Change | -| ---------- | ------------ | -| 2026-10-17 | First draft. | +| Date | Change | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-17 | First draft. | +| 2026-10-18 | Fixes from review round 1. The pull request this spec belongs to, Spin's refusal of a source's `host_header` and what Cloudflare does with one, the startup warning for a route that enables the Image Optimizer and why it is not a refusal, the `HEAD` a signed route sends and the SVG and debug requests that pass, the Axum client's refusal on `send_async`, the startup refusal of a source that cannot be built, the per-request failures, body limits and timeouts, where answers are held, the Edge Cookie an asset response neither writes nor expires, the tests of the cache rule, the error and the Edge Cookie, the open questions and the review log. | + +## Review log + +1. **Round 1, 2026-10-18.** A review in a fresh session, against `fixpass6/54-asset-routes` at `692ce7b8b`. Issues found, four of them blocking. The four were that the spec gave no configuration decision for a route that enables the Image Optimizer, which those adapters accepted and then answered with `502` request by request with no word at startup, that Spin drops a source's `host_header`, so a route reaches its origin with the wrong `Host` and a signature made for another, that the spec said a source that cannot be built answers per request, when it stops the adapter when the state is built, and left out the body limits, the timeouts and the per-request errors, and that no test proved the finalize step, the `no-store, private` error or that no Edge Cookie is written. Fixed in `2369fc42e`, where the three adapters warn at startup about a route that enables the Image Optimizer, `f1bba5d78`, where Spin refuses a `Host` other than the address's own, this revision for the failure handling, and `ada05a9bf`, which tests the cache rule, the error and the Edge Cookie on each adapter. The non-blocking findings fixed are the Axum client's refusal on `send_async` in `dad986582`, the Cloudflare `Content-Length` comment in `efadc7b7e`, the API reference, the Cloudflare reason and the two designs that contradicted this one in `914800f98` and `ff7ae409a`, and the consent and privacy text and this review log in this revision, which also states what the code does for each issue, corrects the partly true claims on where answers are held, the Image Optimizer and the Axum client, names the pull request this spec belongs to, and leaves the Cloudflare `host_header` and `HEAD` length as open questions, with a parity case, the timeouts and streaming as not in this change. From e0b2bb79ead344e2a18927e117117b4ece94652b Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 05:28:43 +0100 Subject: [PATCH 150/192] The origin signing design says every adapter runs the asset route path The design said asset routes and their preflight run on the Fastly adapter alone, and now says every adapter runs them, with Spin refusing a source whose host_header names another host before it is signed. --- docs/superpowers/specs/2026-10-14-origin-signing-design.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-14-origin-signing-design.md b/docs/superpowers/specs/2026-10-14-origin-signing-design.md index eb7db5986d..3324d3ffc3 100644 --- a/docs/superpowers/specs/2026-10-14-origin-signing-design.md +++ b/docs/superpowers/specs/2026-10-14-origin-signing-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-14-source-modules-design.md`, `2026-10-13-publisher-object-storage-origin-design.md`, `2026-05-19-asset-s3-auth-fastly-io-design.md` and `2026-10-14-no-modules-crate-design.md`, all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** jwrosewell/trusted-server#43, IABTechLab/trusted-server#1391 **Issues:** IABTechLab/trusted-server#1338 -**Last updated:** 2026-10-15 +**Last updated:** 2026-10-18 > **Why this spec exists.** Core signs reads from the publisher's origin and from each asset route's origin with one cloud vendor's signing scheme, and its settings know that scheme's credentials, so an origin that wants another scheme has to add a branch to core. @@ -17,7 +17,7 @@ So core knows one vendor's scheme. A signed read from another vendor's storage, ## 2. Design -**Core keeps an origin authentication interface.** `trusted_server_core::origin_auth` declares `OriginSigner`, which signs the headers of a request for a method on a final address, and removes every header in the scheme's own namespace so a reader's header there cannot reach the origin beside the signature. Core calls it where it signed before, in the asset route path, in the preflight that path runs ahead of the image optimizer, and in the publisher path, through the `OriginSigning` a `SourceLocation` carries. The publisher path is core's own, so it signs on all four adapters. Asset routes and the preflight run on the Fastly adapter alone, because only that adapter calls `handle_asset_proxy_request`. +**Core keeps an origin authentication interface.** `trusted_server_core::origin_auth` declares `OriginSigner`, which signs the headers of a request for a method on a final address, and removes every header in the scheme's own namespace so a reader's header there cannot reach the origin beside the signature. Core calls it where it signed before, in the asset route path, in the preflight that path runs ahead of the image optimizer, and in the publisher path, through the `OriginSigning` a `SourceLocation` carries. The publisher path and the asset route path with its preflight are core's own, and every adapter calls both, so all three sign on all four adapters. On Spin a request to a source whose `host_header` names another host is refused before it is signed, because Spin sends the address's own host as the `Host`. Three things differ from what core did before. First, the `HEAD` preflight, which asked an S3 origin for the object before the image optimizer did, now runs for any signed asset route that uses the optimizer, whatever its signer. Second, removing the headers in the signer's namespace is each signer's duty, so core calls `remove_namespace_headers` before `sign` on the publisher path and the asset route path, and does not check afterwards that none is left. Third, the asset route path's call to `remove_namespace_headers` is new, because that call was made on the publisher path alone. The asset route path starts its outbound headers from eleven named request headers (`ASSET_PROXY_FORWARD_HEADERS` in `crates/trusted-server-core/src/proxy.rs`) and none of them is in the `x-amz-` namespace, so for `s3_sigv4` the call removes nothing. The preflight signs its `HEAD` and its `GET` without calling `remove_namespace_headers`, which also makes no difference for `s3_sigv4`, because it signs copies of those same headers and `Host`. @@ -99,6 +99,7 @@ Four cases have no test that can fail: | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The stale modules crate and deploy rules claims, the stages that refuse a mistake, the gap and open question for a `type` nothing offers, the breaking removal of `s3_sig_v4`, failure answers, rollout, the adapters that run each path, the missing tests and the review log. | | 2026-10-15 | Corrections from review round 2. The specs pull request number, the preflight's signing without removing namespace headers, the requests an adapter answers whatever its startup error, the deploy validation test for an empty token and the Tinybird changelog entry. | +| 2026-10-18 | Where the asset route path and its preflight run, which is every adapter once asset routes are served on every adapter, and Spin's refusal of a `host_header` that names another host. | ## Review log From 7be9589e35c0f33d72116dde0c5ef30575de3820 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 05:28:45 +0100 Subject: [PATCH 151/192] The asset cache design says only the Fastly adapter applies an origin fetch's timeouts The design said an origin fetch has 15-second timeouts, and now says core asks for them and only the Fastly adapter applies them, the Axum client applying its own limits and the Cloudflare and Spin adapters none of their own. --- docs/superpowers/specs/2026-10-15-asset-cache-design.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-15-asset-cache-design.md b/docs/superpowers/specs/2026-10-15-asset-cache-design.md index afbe9098af..4192511641 100644 --- a/docs/superpowers/specs/2026-10-15-asset-cache-design.md +++ b/docs/superpowers/specs/2026-10-15-asset-cache-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-12-loading-cache-design.md`, `2026-09-15-852-template-and-origin-caching-design.md`, `2026-04-28-multi-backend-asset-proxy-design.md`, `2026-05-19-asset-s3-auth-fastly-io-design.md`, `2026-09-08-1138-per-cookie-template-cache-policy-design.md` **Related PRs:** IABTechLab/trusted-server#1375, reviewed on jwrosewell/trusted-server#29 (closed) **Issues:** IABTechLab/trusted-server#1324 -**Last updated:** 2026-10-17 +**Last updated:** 2026-10-18 > **Why this spec exists.** A file that is not a page, whether the publisher's origin sends it or an asset route fetches it, is asked of its origin on every request and by every instance, so a burst of readers for one file is a burst of fetches. @@ -142,7 +142,7 @@ The stock module, `crates/loading-cache/51degrees`, uses a host's value cache wh **Cost and limits.** -- **A slow origin.** For a response that passes the header checks, the request that leads a load waits for the whole body, up to 1 MiB, before its reader is served, and each request waiting on that load waits as long. An origin fetch has a first byte timeout and a between bytes timeout of 15 seconds, on the publisher path (`DEFAULT_PUBLISHER_FIRST_BYTE_TIMEOUT`) and on an asset route (`DEFAULT_FIRST_BYTE_TIMEOUT`). A failed load reaches every request waiting on it and each of them then fetches the origin itself, so a reader can wait for one failed load and then for a fetch of their own. +- **A slow origin.** For a response that passes the header checks, the request that leads a load waits for the whole body, up to 1 MiB, before its reader is served, and each request waiting on that load waits as long. Core asks for a first byte timeout and a between bytes timeout of 15 seconds for an origin fetch, on the publisher path (`DEFAULT_PUBLISHER_FIRST_BYTE_TIMEOUT`) and on an asset route (`DEFAULT_FIRST_BYTE_TIMEOUT`), and the Fastly adapter applies them. The Axum client applies limits of its own instead, 5 seconds to connect and 30 seconds for the whole request, and the Cloudflare and Spin adapters apply none of their own, so their runtimes' limits apply. A failed load reaches every request waiting on it and each of them then fetches the origin itself, so a reader can wait for one failed load and then for a fetch of their own. - **Removals.** When a response no longer varies as its record says, the request that fetched it removes the record, and a lookup that finds an entry it cannot read removes that entry. On Fastly a removal is a purge by surrogate key (`purge_surrogate_key` in the Fastly adapter's `value_cache.rs`), and on a key-value store it is a delete. An origin whose `Vary` differs from one response to the next causes a removal for each response that differs from the record. - **Keys and variants without a limit.** Each query string, each host the reader asked, each scheme, and each combination of values for the headers `Vary` names is a key of its own. A miss writes the copy or the marker to the shared store, and the record too when the address has none, or the record alone when the origin's `Vary` allows no copies. This change sets no limit on the number of keys. `store_seconds` limits how long a copy is used and `store_idle_seconds` how long it lasts unused. Spin's `default` store, which the Spin adapter declares, gives an entry no lifetime, so there an entry that nobody reads again stays in the store, because only a read past its drop time, or a removal, takes it out. - **The fingerprint.** The publisher path computes `template_fingerprint`, a digest of all the serialized settings, for each eligible request, whether or not a copy is found. @@ -207,6 +207,7 @@ Joins with no test: | 2026-10-15 | Fixes from review round 1. The cookie rules the asset gate leaves out and what that lets through, the reason the code gives, the default-on state and the open questions on cookie rules and an off switch, `Pragma` in the gate, the verdict a response that fails a test leaves for 60 seconds, the module default, where the sizes and the `[cache.assets-vary]` table stand, the key-value store each adapter uses, how far one fetch reaches, what the caches add on Fastly, `[[cache.asset_rules]]`, the headers a route forwards, the cost and limits, the test doubles and the missing tests. | | 2026-10-16 | Fixes to the code, and from review round 2. A request that sends the publisher's origin a cookie neither reads a copy nor leads a load, `[cache.assets] module = "none"` turns both asset caches off, and a `[cache.assets]` lifetime out of range is refused when the settings load, each with its tests. Which entries each load writes, the tests that send a cookie, and the module default in the guide and the code comments. | | 2026-10-17 | The code fix on row 24, where the Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests, and where each application state has its own memory layer, Cloudflare included. How far one fetch reaches, the adapters' shared stores, how long the memory layer lives, and the reads and writes of the shared store say so. | +| 2026-10-18 | The timeouts of an origin fetch, which core asks for on every adapter and only the Fastly adapter applies. | ## Review log From 44165f67acbe132f282d3f84d988f5679f852750 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 07:03:45 +0100 Subject: [PATCH 152/192] The prefix module design records the code scanning fix The registry selects SelectedPrefixModule::default() and the trusted_server() constructor is gone, which ends the two code scanning alerts, so the first open question keeps only whether Prefixes::trusted_server should take the publisher domain alone. --- docs/superpowers/specs/2026-10-12-prefix-module-design.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-prefix-module-design.md b/docs/superpowers/specs/2026-10-12-prefix-module-design.md index 7771e30377..85edbdfec2 100644 --- a/docs/superpowers/specs/2026-10-12-prefix-module-design.md +++ b/docs/superpowers/specs/2026-10-12-prefix-module-design.md @@ -139,13 +139,13 @@ When the Fastly state failed to build, the entry point's finalize services carry 6. No test covers the Fastly finalize middleware or the entry point's finalize services (`crates/trusted-server-adapter-fastly/src/middleware.rs` lines 94 to 111 and `crates/trusted-server-adapter-fastly/src/lib.rs` lines 754 to 770). 7. Only the Axum adapter selects a module in a test. The other three adapters test the default. -**How the change was checked.** The description of IABTechLab/trusted-server#1368 lists as run `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter, and marks a manual run with `fastly compute serve` as not done. The description gives CI as 19 of 19 checks green on the fork's pull request for `9a9993423`, and records no run of every commit alone and no deliberate faults. On IABTechLab/trusted-server#1368 itself, 24 of 25 checks pass at this revision and the one that fails is CodeQL (Open question 1). +**How the change was checked.** The description of IABTechLab/trusted-server#1368 lists as run `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter, and marks a manual run with `fastly compute serve` as not done. The description gives CI as 19 of 19 checks green on the fork's pull request for `9a9993423`, and records no run of every commit alone and no deliberate faults. On IABTechLab/trusted-server#1368 itself, 24 of 25 checks passed at `488ab987b` and the one that failed was CodeQL, with two high alerts of the rule "Cleartext logging of sensitive information", [216](https://github.com/IABTechLab/trusted-server/security/code-scanning/216) at `auction/orchestrator.rs` line 1613, which logs a count of planned auction providers, and [217](https://github.com/IABTechLab/trusted-server/security/code-scanning/217) at `module_context/inject.rs` line 158, which logs a module's name and why the module was skipped. Neither line logs anything sensitive. Every data flow behind both started at a call to `SelectedPrefixModule::trusted_server()`, in `resolve_prefix_module` and in `SelectedPrefixModule::default`, because the rule takes a function whose name holds `trusted` to return a secret. The registry now selects `SelectedPrefixModule::default()`, which builds core's own module itself, and that constructor is gone. ## 5. Open questions These are for the reviewer. This revision records what the code does and decides none of them. -1. Should `Prefixes::trusted_server(settings: &Settings)` (`prefix.rs` line 238) take only the field it reads, being `publisher.domain`, in place of the whole settings with its secrets? CodeQL fails on IABTechLab/trusted-server#1368 with two high alerts of the rule "Cleartext logging of sensitive information", [216](https://github.com/IABTechLab/trusted-server/security/code-scanning/216) at `auction/orchestrator.rs` line 1613, which logs a count of planned auction providers, and [217](https://github.com/IABTechLab/trusted-server/security/code-scanning/217) at `module_context/inject.rs` line 158, which logs a module's name and why the module was skipped. Neither line logs anything sensitive. The data flow CodeQL reports for both starts at the call `SelectedPrefixModule::trusted_server()` in `resolve_prefix_module` (`registry.rs` line 1286), which takes no argument. The alert at `module_context/inject.rs` line 158 has a second flow, which starts at the same call in `SelectedPrefixModule::default` (`prefix.rs` line 312). No flow passes through `Prefixes::trusted_server`. Whether to narrow that input, rename the constructors or dismiss both alerts with a reason is not decided here. +1. Should `Prefixes::trusted_server(settings: &Settings)` (`prefix.rs` line 238) take only the field it reads, being `publisher.domain`, in place of the whole settings with its secrets? No code scanning alert depends on it (section 4). 2. Should each of these comments and guide paragraphs say what the code does? `prefix.rs` lines 10, 103 and 155, `html_processor.rs` lines 118 and 1185, `docs/guide/configuration.md` line 949 and `docs/guide/integration-guide.md` line 788 name first-party storage keys, which nothing writes. `docs/guide/configuration.md` line 959 puts the identity-graph rows under the key prefix, where they take the Edge Cookie's identifier. The Fastly `middleware.rs` lines 94 to 96 say the geo module's answers are held under the prefix, and the Fastly `app.rs` line 487 says the publisher a request resolved to comes with "its tenant included", yet no geo code reads the prefix and nothing resolves a tenant. `settings.rs` lines 3997 and 4000, `docs/guide/configuration.md` lines 627 to 649 and `docs/guide/api-reference.md` lines 330 and 363 name `ts-tester` alone. 3. Should the startup trial, or the types, check more? The trial refuses a module only for naming a value its call lacks. `ServerKeyPrefix::new` takes any head and any domain (`platform/server_key.rs` lines 24 to 36). A module has no way to refuse when `PublicPrefix::new` returns `None`, because the trait's error type is `Withheld` (`prefix.rs` lines 161 and 168), so the probe calls `expect` (`crates/testing/seam-probe/src/lib.rs` line 400). An adapter that builds services and never calls `with_prefixes` gets `Prefixes::default()` without a word (`platform/types.rs` line 798). Nothing checks that a module's server key prefix carries the publisher's domain, or that it does not depend on the request's host. 4. Should tests cover the seven cases listed under "What the tests do not prove" in section 4? @@ -158,6 +158,7 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Fixes from review round 1. The status line and the pull requests it names, the cookies the public prefix names, the one use of the server key prefix and what keeps its own name, where the domain comes from, when a refused selection shows, what changes for code and for a deployment, the adapters, the acceptance rebuilt from the pull request description and the tests, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. The list of new public items and where the code scanning data flow starts. | +| 2026-10-18 | The two code scanning alerts. The registry selects `SelectedPrefixModule::default()` and `SelectedPrefixModule::trusted_server()` is gone, so Open question 1 keeps only the input of `Prefixes::trusted_server`. | ## Review log From 49fef62487d71b6e292caa40852aa9cf727d7d08 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 07:03:47 +0100 Subject: [PATCH 153/192] The identifier modules design lists the test of the skipped EID log lines No debug line for a skipped EID carries the source the reader sent, and the line for a partner's EID names the partner's configured domain, with the test that shows it. --- docs/superpowers/specs/2026-10-14-identifier-modules-design.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/superpowers/specs/2026-10-14-identifier-modules-design.md b/docs/superpowers/specs/2026-10-14-identifier-modules-design.md index 83b38a2b21..3ff985562c 100644 --- a/docs/superpowers/specs/2026-10-14-identifier-modules-design.md +++ b/docs/superpowers/specs/2026-10-14-identifier-modules-design.md @@ -65,6 +65,7 @@ The fallback is where each adapter serves the publisher's pages and subresources - Core asks each module in order and keeps their order, passes a module the evidence it names, and skips a module whose call withholds what it names while the next still answers. - The registry hands out every running registration's module in registration order, hands out none for a registration no section selects, and refuses two registrations that declare one module name. - Partner updates are collected from what the modules learned, the body's EIDs win over a module's stale value and ingest a partner the module's cookie trimmed, the `sharedId` cookie is applied last, an unknown source is skipped, an oversized identifier is skipped for a later valid one, and an empty registry touches no store. +- No line logged for a skipped EID carries the source the reader sent, and the line for a partner's EID names the partner's configured domain (`no_log_line_carries_the_source_a_reader_sent` in `ec/identifiers.rs`). - The auction falls back to what the modules learned when the body carries none, prefers the body when it carries EIDs, and resolves nothing when neither has any. - The Prebid module reads the cookie named by the public prefix, learns nothing from a request without it or with one it cannot read, is withheld without the evidence it names, parses both shapes, preserves a vendor-specific `atype`, skips entries with nothing to offer, and refuses invalid base64, invalid JSON and an oversized value. - A template key cookie under the public prefix, and `sharedId`, are refused, and either may be listed for bypass. @@ -97,6 +98,7 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The decoding on every read and its cost, the adapter table, the template cache rule's effects, the size limits, the locations of the cookie's name, `sharedId` and `Eid`, the permission order, failure and merge behavior, the untested joins, one sentence of the Edge Cookies guide, one cell of the integration guide, the open questions and the review log. | | 2026-10-15 | Fixes from review round 2. How the cookie is parsed, how the request body treats a long source name, the requests answered before the read, one paragraph of the integration guide, and the pull requests in the header. | +| 2026-10-18 | No debug line for a skipped EID carries the source the reader sent, and the test for it. | ## Review log From 3caf85892162db913789459eadac27d01b3f3610 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 07:34:13 +0100 Subject: [PATCH 154/192] The config migrate design takes review round 2 A handler is removed only where no quantifier, class or comment follows its prefix or bound, the region the Cloudflare lookup now reads is reported to check, main's jurisdiction check is described as it was, one document run on two adapters is covered, and the review log records round 2. --- .../specs/2026-10-17-config-migrate-design.md | 54 +++++++++++-------- 1 file changed, 32 insertions(+), 22 deletions(-) diff --git a/docs/superpowers/specs/2026-10-17-config-migrate-design.md b/docs/superpowers/specs/2026-10-17-config-migrate-design.md index 6355fc1eaa..bdacb993c1 100644 --- a/docs/superpowers/specs/2026-10-17-config-migrate-design.md +++ b/docs/superpowers/specs/2026-10-17-config-migrate-design.md @@ -1,6 +1,6 @@ # Design Spec: `ts config migrate` rewrites a settings document written for main into the shape the stack reads -**Status:** Proposed, 2026-10-17, revised 2026-10-18 after review round 1. The top pull request of the stack, not yet raised, for Tech Lab review. +**Status:** Proposed, 2026-10-17, revised 2026-10-18 after review rounds 1 and 2. A pull request of the stack, not yet raised, for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-14-source-modules-design.md`, `2026-10-13-middleware-settings-design.md`, `2026-10-14-rsc-flight-middleware-design.md`, `2026-10-15-geo-module-request-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 **Related PRs:** none yet @@ -26,7 +26,7 @@ Most of the rewrite is mechanical, because the old shape says everything the new - `--check` changes nothing, prints nothing on stdout and does not validate. It exits 1 when the document needs migrating, being a rewrite to make or an item left to do, so CI can hold a document to the current shape. - `--write` and `--check` cannot be used together. -The command works on the file alone. It reads no `edgezero.toml`, applies no `TRUSTED_SERVER__` overlay and asks no platform anything, because the rewrite is a property of the document and the adapter. It validates the result as `ts audit generate` validates a document it is about to write, by parsing the text into `TrustedServerAppConfig` with no manifest and running `validate_excluding_secrets` with the modules the tool links (`crates/trusted-server-cli/src/commands/audit/generate/validate.rs` lines 55 to 66). That is the push-time validation `ts config validate` runs, and `ts config validate` also applies the overlay, checks secrets and runs EdgeZero's adapter checks (EdgeZero v0.0.9, `crates/edgezero-cli/src/config.rs` lines 245 to 277). Neither runs the checks made only when a deployment starts, being core's checks once the settings load (`Settings::finalize_deserialized`, core `settings.rs` lines 4339 to 4365) and each adapter's checks as it builds its state. The command reports the location settings among those as items left to do, because every document written for main gains an Edge Cookie module from its passphrase, and whether that document then starts depends on its geo settings and its adapter (§3). +The command works on the file alone. It reads no `edgezero.toml`, applies no `TRUSTED_SERVER__` overlay and asks no platform anything, because the rewrite is a property of the document and the adapter. It validates the result as `ts audit generate` validates a document it is about to write, by parsing the text into `TrustedServerAppConfig` with no manifest and running `validate_excluding_secrets` with the modules the tool links (`crates/trusted-server-cli/src/commands/audit/generate/validate.rs` lines 55 to 66). That is the push-time validation `ts config validate` runs, and `ts config validate` also applies the overlay, checks secrets and runs EdgeZero's adapter checks (EdgeZero v0.0.9, `crates/edgezero-cli/src/config.rs` lines 245 to 277). Neither runs the checks made only when a deployment starts, being the checks core makes once the settings load that push-time validation does not repeat, of the old Edge Cookie layout, the `[ec]`, `[device]` and `[geo]` module selections, the permission policy and the jurisdiction acknowledgment (`Settings::finalize_deserialized`, core `settings.rs` lines 4355, 4356 and 4360 to 4365, where lines 4357 to 4359 run at push time too), and each adapter's checks as it builds its state. The command reports the location settings among those as items left to do, because every document written for main gains an Edge Cookie module from its passphrase, and whether that document then starts depends on its geo settings and its adapter (§3). Each line on stderr starts with one of three words. @@ -51,13 +51,13 @@ The table lists every breaking entry in `CHANGELOG.md` under Unreleased that con | The origin settings moved from `[publisher]` to `[source]` | Rewritten to `[source] module = "origin"` and `[source.origin]`, with `origin_url` as `url`, `origin_host_header_override` as `host_header`, `origin_auth` as `auth` and `directory_index` unchanged. Check, below, where `host_header` is set. | | An asset route's `origin_url` and `auth` moved to its `source` table | Rewritten to `[proxy.asset_routes.source] module = "origin"` and `[proxy.asset_routes.source.origin]`. | | The signer spelling `s3_sig_v4` is no longer read | Rewritten to `s3_sigv4` in every `auth` table. | -| `[[handlers]]` and HTTP Basic authentication are removed | Rewritten by removing an entry where every path its pattern can match is one the stack closes, where nothing is served now, as the paragraph below the table defines. Main matched a pattern anywhere in the path (main core `settings.rs` line 734), so `^/admin/keys` alone also guarded `/admin/keystore`, which the stack forwards to the origin, and such an entry is to do, below. | +| `[[handlers]]` and HTTP Basic authentication are removed | Rewritten by removing an entry where every path its pattern can match is one the stack closes, where nothing is served now, as the paragraph below the table defines. Main matched a pattern anywhere in the path (main core `settings.rs` line 734), so `^/admin/keys` alone also guarded `/admin/keystore`, which the stack forwards to the origin, and so did `^/admin/keys/?`, whose `?` lets the `/` match nothing, and such an entry is to do, below. | | `[request_signing]` does not take `config_store_id` or `secret_store_id` | Rewritten by removing both. Check, below, for where the ids go. | | `[ec] passphrase` is read as deprecated (settings code) | Rewritten to `[ec] module = "hmac"` and `[ec.hmac] passphrase`, where the stack documents it. The deprecated key still loads, with a warning at startup, for one release. | | `[ec.providers.]` is `[ec.]`, and the module names `host-signals` and `client-fixed` are `host_signals` and `client_fixed` | Rewritten, in the tables, the selection and an `implementation` line. | | `[permission_signal] sources` is `[permission-signal] modules`, with `gpp-sale-opt-out` and `gpp_sale_opt_out` as `gpp` and `us_privacy` as `us-privacy` | Rewritten. | | `[device] module = "fastly"` is `"platform"` | Rewritten. | -| The host's location lookup runs only where `[geo]` selects it (settings code) | Rewritten for a document written for main on Fastly and Cloudflare by selecting `[geo] module = "platform"`, the lookup main ran on every request there. On Axum and Spin main's lookup located no request, and those adapters refuse the selection at startup, so the command selects no geo module there, and the Edge Cookie module that follows is to do, below. | +| The host's location lookup runs only where `[geo]` selects it (settings code) | Rewritten for a document written for main on Fastly and Cloudflare by selecting `[geo] module = "platform"`, the lookup main ran on every request there. Check, on Cloudflare, that the lookup now reads the reader's region, which main's did not (§4). On Axum and Spin main's lookup located no request, and those adapters refuse the selection at startup, so the command selects no geo module there, and the Edge Cookie module that follows is to do, below. | | No response carries `x-geo-*` unless `headers.geo` is selected and placed | Check, for a document written for main, saying what main's responses carried on the adapter and, on Fastly, how to keep the headers. The command never selects `headers.geo`, because the headers carry the reader's location (§4). | | `[tinybird]` is `[analytics.tinybird]`, selected by `[analytics] module` | Rewritten, with `enabled` dropped. A table that was not enabled is commented out. Check, below, for the consent gate. | | A module's page change runs only where a `[[fetch]]` or `[[serve]]` entry names its middleware | Rewritten for a document written for main by adding one entry for each phase and media type the selected modules supply middleware for, naming that middleware in the order the linked builders supply it, which is the order `trusted-server.example.toml` shows. A document whose modules supply no fetch middleware gets no `[[fetch]]` entry. A document in the stack's shape is left as written, because it loads with no entry, with a warning naming each middleware no entry names (CHANGELOG line 51), and whether its modules change pages is the operator's choice. | @@ -85,7 +85,13 @@ The table lists every breaking entry in `CHANGELOG.md` under Unreleased that con | Main's own breaking entries the stack keeps: the auction maps, APS OpenRTB, `site.page`, `bid_param_zone_overrides`, string scalars, Sourcepoint and creative sanitization | Nothing to do. A document written for main already meets them, and the maps move again under the second row. | | Earlier shapes of the stack that neither the changelog nor the settings code names, such as `[ec] provider` | Out of scope. They never reached a release, and nothing tells an operator about them. | -A `[[handlers]]` pattern guards only paths the stack closes when it starts `^/_ts/admin`, because every path that begins `/_ts/admin` is closed, or starts `^/admin/keys` followed by `/`, `$`, `(?:/|$)` or `(/|$)`, because of `/admin/keys` only the address itself and the paths below `/admin/keys/` are closed (core `closed_paths.rs` lines 18, 25 and 75 to 81), and when it has no `|` outside a group, which would let it match another path. +A `[[handlers]]` pattern guards only paths the stack closes when it starts `^/_ts/admin`, because every path that begins `/_ts/admin` is closed, or starts `^/admin/keys` followed by `/`, `$`, `(?:/|$)` or `(/|$)`, because of `/admin/keys` only the address itself and the paths below `/admin/keys/` are closed (core `closed_paths.rs` lines 18, 25 and 75 to 81), and when nothing after that prefix or bound widens what it matches, which rules out the following. + +- A `?`, `*` or `{` straight after the prefix or the bound, which starts a quantifier that can let the last character of `^/_ts/admin`, or the whole bound, match nothing, so that `^/_ts/admin?` matched `/_ts/admit` and `^/admin/keys/?` matched `/admin/keystore` under main's search. A `+` is accepted, because it matches what comes before it at least once. +- A `[` or a `#` anywhere after the prefix or the bound, because a character class, or a comment the `(?x)` flag allows, can hide a `|` from the command, as the `(` in `^/_ts/admin[](]|^/members` is a character rather than a group, and that pattern matched `/members`. +- A `|` outside a group, which would let the pattern match another path. + +The command reads the pattern's text and not what the pattern means, so where the text alone cannot show that every path is closed it leaves the entry to do, as it does for `^/admin/keys/?$` and `^/_ts/admin/[a-z]+`, although every path either matches is closed. A document is taken as written for main when it has `[publisher]` and every table at its top level is one main read, which are `publisher`, `tester_cookie`, `trusted_client_ip`, `ec`, `integrations`, `handlers`, `response_headers`, `request_signing`, `rewrite`, `auction`, `consent`, `cache`, `proxy`, `creative_opportunities`, `image_optimizer`, `tinybird` and `debug`. Main refused any other table, and every migrated document has `[source]`, so a second run never takes it as written for main. The geo row, the page entry rows, and the checks about `x-ts-version`, the location headers and Flight responses apply to such a document alone, so a document already in the stack's shape keeps its own geo settings and page entries, or the lack of them. @@ -119,23 +125,25 @@ Migrating a migrated document changes nothing, because each rule matches only a The command runs in the tool and does the same on every machine, but the document it writes runs on one adapter, and main's location lookup and location headers differed by adapter. Only the location settings depend on `--adapter`. -| Adapter | Main's location lookup | What main's responses carried | What a document written for main gets | -| ---------- | --------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | -| Fastly | The platform's lookup, on every request | The `x-geo-*` headers with the reader's location, or `x-geo-info-available: false` where none was found (main `crates/trusted-server-adapter-fastly/src/middleware.rs` lines 202 to 214) | `[geo] module = "platform"`, and a check line saying how to keep the headers | -| Cloudflare | Cloudflare's request headers, on every request (main `crates/trusted-server-adapter-cloudflare/src/platform.rs` lines 668 to 684) | `x-geo-info-available` alone, `true` where `cf-ipcountry` was present and not `XX` (main `crates/trusted-server-adapter-cloudflare/src/middleware.rs` lines 139 to 147) | `[geo] module = "platform"`, and a check line saying that `headers.geo` would add the reader's location | -| Axum | None, as the lookup answered nothing (main `crates/trusted-server-adapter-axum/src/platform.rs` lines 183 to 187) | `x-geo-info-available: false` (main `crates/trusted-server-adapter-axum/src/middleware.rs` lines 133 to 137) | No geo module, an item left to do for the Edge Cookie module, and a check line on the header | -| Spin | None, as the lookup answered nothing (main `crates/trusted-server-adapter-spin/src/platform.rs` lines 757 to 761) | `x-geo-info-available: false` (main `crates/trusted-server-adapter-spin/src/middleware.rs` lines 167 to 175) | As on Axum | +| Adapter | Main's location lookup | What main's responses carried | What a document written for main gets | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Fastly | The platform's lookup, on every request | The `x-geo-*` headers with the reader's location, or `x-geo-info-available: false` where none was found (main `crates/trusted-server-adapter-fastly/src/middleware.rs` lines 202 to 214) | `[geo] module = "platform"`, and a check line saying how to keep the headers | +| Cloudflare | Cloudflare's request headers, on every request (main `crates/trusted-server-adapter-cloudflare/src/platform.rs` lines 668 to 684) | `x-geo-info-available` alone, `true` where `cf-ipcountry` was present and not `XX` (main `crates/trusted-server-adapter-cloudflare/src/middleware.rs` lines 139 to 147) | `[geo] module = "platform"`, a check line saying that the lookup now reads the reader's region, and one saying that `headers.geo` would add the reader's location | +| Axum | None, as the lookup answered nothing (main `crates/trusted-server-adapter-axum/src/platform.rs` lines 183 to 187) | `x-geo-info-available: false` (main `crates/trusted-server-adapter-axum/src/middleware.rs` lines 133 to 137) | No geo module, an item left to do for the Edge Cookie module, and a check line on the header | +| Spin | None, as the lookup answered nothing (main `crates/trusted-server-adapter-spin/src/platform.rs` lines 757 to 761) | `x-geo-info-available: false` (main `crates/trusted-server-adapter-spin/src/middleware.rs` lines 167 to 175) | As on Axum | The stack's Axum and Spin adapters refuse `[geo] module = "platform"` at startup naming the setting, because neither host has a lookup (`crates/trusted-server-adapter-axum/src/app.rs` lines 277 and 326, `crates/trusted-server-adapter-spin/src/app.rs` lines 252 and 286, CHANGELOG line 42). Every document main ran with has an Edge Cookie passphrase (main core `settings.rs` lines 545 and 625 to 633), which becomes `[ec] module = "hmac"`, and startup on every adapter refuses an Edge Cookie module with no geo module unless `[geo] assume_single_jurisdiction = true` accepts that every request takes the top of the `permissions.yaml` rules. On Axum and Spin the operator chooses between that acceptance and a geo module a linked crate offers, so the command leaves the choice to do. Writing `platform` there would report a document as passing that the adapter refuses, and writing the acknowledgment would make the choice the stack asks the operator to make. The rest of the document is the same on every adapter. Only the Fastly adapter sends the telemetry an `[analytics]` selection names, as only Fastly's did on main (CHANGELOG lines 20 and 48), and the Cloudflare and Spin adapters need their key-value store (§6). +A deployment that ran one document on two adapters, as main's quick start pushes one `trusted-server.toml` to the Axum dev server and to Fastly (main `docs/guide/getting-started.md` lines 46 to 88 and 197), runs the command once for each adapter and keeps a document for each. The Axum dev server refuses the `[geo] module = "platform"` the Fastly document gets, and the Axum document, with no geo module, would leave Fastly without the lookup main ran there. + ## 4. Consent and privacy - **No value is printed.** Every line on stderr names a setting and holds no value from the document, because a document can hold secrets, and the validation result gives only the byte offset or the phase that refused. Lines do carry table names the operator chose, such as an integration id or a bidder code (the test `no_line_carries_a_value_from_the_document`). - **No location header is added.** The command never selects `headers.geo`, which writes the reader's location in the `x-geo-*` headers on the responses a `[[serve]]` entry covers. Main wrote those headers on Fastly alone, and on Cloudflare wrote only `x-geo-info-available`, so selecting the middleware for every document written for main would send the reader's location on Cloudflare, where main sent none. Whether pages carry the reader's location is the operator's decision, as the stack made it one (CHANGELOG line 41), so the check line for each adapter says what main's responses carried and, on Fastly, how to keep the headers. -- **The location the consent rules see.** On Fastly and Cloudflare the document keeps the lookup main ran, so the `permissions.yaml` rules see the reader's location as they did, where without a geo module every request would take the top of those rules. On Cloudflare the stack's lookup also reads the region, from `cf-region-code` where Cloudflare sends it, where main's lookup read none (`crates/trusted-server-adapter-cloudflare/src/platform.rs` lines 709 to 721 and 782 to 787), so the region rules of `permissions.yaml`, such as those for US states, can apply there now. On Axum and Spin main located no request and the document locates none, and an Edge Cookie module runs there only once the operator accepts that with `[geo] assume_single_jurisdiction = true` or selects a geo module. -- **Passwords.** The command removes a `[[handlers]]` entry only where every path its pattern can match is one the stack closes, so a publisher path such as `/admin/keystore` never loses its password without an item left to do. +- **The location the consent rules see.** On Fastly and Cloudflare the document keeps the lookup main ran, so the `permissions.yaml` rules see the location main's jurisdiction check saw, where without a geo module every request would take the top of those rules. Main had no `permissions.yaml`. Its check compared the country with `[consent.gdpr] applies_in` and the region with `[consent.us_states] privacy_states` (main core `consent/jurisdiction.rs` lines 41 to 79), and the jurisdiction it found fed the Edge Cookie consent check and the OpenRTB `regs.gdpr` value (main core `ec/consent.rs` lines 11 to 22, `auction/openrtb.rs` lines 652 to 665). On Cloudflare the stack's lookup also reads the region, from `cf-region-code` where Cloudflare sends it, where main's lookup read none (`crates/trusted-server-adapter-cloudflare/src/platform.rs` lines 709 to 721 and 782 to 787), so the region rules of `permissions.yaml` can apply there now, such as the `us-state` jurisdiction it gives each US state it lists where the country gives `non-regulated` (`config/permissions/sample.yaml` lines 421 to 427), and a check line printed with the geo rewrite says so. On Axum and Spin main located no request and the document locates none, and an Edge Cookie module runs there only once the operator accepts that with `[geo] assume_single_jurisdiction = true` or selects a geo module. +- **Passwords.** The command removes a `[[handlers]]` entry only where every path its pattern can match is one the stack closes, read as the paragraph below the §2.2 table says, so a publisher path such as `/admin/keystore` never loses its password without an item left to do, whether the pattern is `^/admin/keys`, `^/admin/keys/?` or one whose character class hides another alternative. The tests check each pattern they name against main's matching and the stack's closed paths (§8). - **Telemetry.** A telemetry row is sent only for a reader whose permissions include `analytics.ad_reporting.measure_ad_performance`, and a check line says so where `[tinybird]` was enabled. - **The consent lists.** The places the GDPR and US state rules apply are the `jurisdiction` values in the `permissions.yaml` compiled into the build, so a change the operator made to `applies_in` or `privacy_states` is an item left to do until they carry it there and remove the table. @@ -143,7 +151,7 @@ The rest of the document is the same on every adapter. Only the Fastly adapter s - **A file that cannot be read, written or parsed.** The command exits 2. A file that is not TOML is refused naming the line reading stopped at and nothing from it (`migrate` in `rules.rs`, and the test `no_line_carries_a_value_from_the_document`). `--write` rewrites the file in place with one write, so a write that fails part of the way can leave the file damaged, which is one reason §6 asks the operator to keep the original. - **An item left to do, or a result that does not pass validation.** The command exits 1, and `--write` writes the result anyway, because the document did not load before either and the items say what is left. -- **The checks made only at startup.** The command's validation, like `ts config validate`, leaves out the checks core makes once the settings load and each adapter makes as it builds its state. Of those it reports the location settings as items left to do, for any document. The others, such as a module name that no linked crate supplies, show only when the deployment starts. +- **The checks made only at startup.** The command's validation, like `ts config validate`, leaves out the checks core makes only once the settings load, which §2.1 lists, and those each adapter makes as it builds its state. Of those it reports the location settings as items left to do, for any document. The others, such as a module name that no linked crate supplies, show only when the deployment starts. ## 6. Compatibility and rollout @@ -152,7 +160,7 @@ The rest of the document is the same on every adapter. Only the Fastly adapter s 3. **Keep the original document before `--write`.** A rollback restores it with the old build (CHANGELOG line 72), and it holds the `[request_signing]` store ids the command removes and never prints, which `ts keys rotate` and `ts keys deactivate` take as `--config-store-id` and `--secret-store-id`. 4. **Rename the overlay variables.** `ts config push` ignores a variable for a key the document does not hold, so `TRUSTED_SERVER__PUBLISHER__ORIGIN_URL` and `TRUSTED_SERVER__INTEGRATIONS____*` set nothing once the document is migrated. They become `TRUSTED_SERVER__SOURCE__ORIGIN__URL` and `TRUSTED_SERVER__
____*` (CHANGELOG lines 26, 56 and 72). 5. **Keep the key-value store on Cloudflare and Spin.** A Worker without the `TRUSTED_SERVER_KV` binding, or a Spin component that cannot open `default`, now fails every request before any route runs (CHANGELOG line 46). Main's `wrangler.toml` declares the binding and main's `spin.toml` grants the store, so a deployment that kept those files has both. -6. **Expect the stored pages to be fetched again** once after the upgrade, because their keys hold a code worked out from the settings (CHANGELOG line 26). +6. **Expect the stored pages and the publisher's stored assets to be fetched again** once after the upgrade, because their keys hold a code worked out from the settings, while an asset route's stored copies keep their keys (CHANGELOG line 26). The CLI guide's section "Upgrade a configuration written for an earlier release" gives the same steps. @@ -162,6 +170,7 @@ The CLI guide's section "Upgrade a configuration written for an earlier release" - Pushing the result. The operator runs `ts config validate` and `ts config push` in the order §6 gives. - A write through a temporary file, which would keep the original whole when a write fails part of the way. §6 asks the operator to keep the original instead. - Rewriting comment text, which would mean guessing what a comment refers to. +- Reading what a `[[handlers]]` pattern means rather than its text, which would also remove a pattern such as `^/admin/keys/?$` whose every path is closed. The command leaves such a pattern to do, and the operator removes it. - Earlier shapes of the stack that neither the changelog nor the settings code names. ## 8. Tests @@ -171,8 +180,8 @@ The CLI guide's section "Upgrade a configuration written for an earlier release" - A golden test that migrates main's own `trusted-server.example.toml`, copied from `upstream/main` byte for byte into `crates/trusted-server-cli/tests/fixtures/main-trusted-server.example.toml` under a header that names the commit. The test replaces its placeholder domain and origin, as an operator's document would have them. The result loads and passes validation under the stack's settings code, and migrating it again changes nothing. - `the_location_lookup_is_the_one_each_adapter_accepts`, which migrates a document written for main for each adapter. On Fastly and Cloudflare the result selects `platform` and passes core's startup check of the jurisdiction acknowledgment. On Axum and Spin it selects no geo module, that check refuses it and an item left to do says so, and once `[geo] assume_single_jurisdiction = true` is added the check passes and a second run leaves nothing to do. - `the_location_settings_startup_refuses_are_left_to_do_in_any_document`, for `[geo] module = "platform"` on Axum and Spin and for an Edge Cookie module with no geo module on every adapter. -- `no_location_header_is_selected_for_the_operator`, on every adapter, with the check line each adapter gets. -- `a_handler_is_removed_only_where_every_path_it_matches_is_closed`, with the bounded and the unbounded `/admin/keys` patterns. +- `no_location_header_is_selected_for_the_operator`, on every adapter, with the check line each adapter gets, and `the_region_the_cloudflare_lookup_now_reads_is_reported_to_check`, on every adapter. +- `a_handler_is_removed_only_where_every_path_it_matches_is_closed`, with the bounded and the unbounded `/admin/keys` patterns and those a quantifier after the prefix or the bound widens, and `a_handler_with_a_class_or_comment_after_its_prefix_is_left_to_do`. Both run each pattern through the `regex` crate main matched with and through core's `closed_path_response` on a set of request paths, so each pattern the command removes is shown to guard no path the stack forwards, each it keeps as unsafe is shown to guard one, and `^/admin/keys/?$` and `^/_ts/admin/[a-z]+` are shown kept although they guard none. - `a_host_header_override_is_reported_to_check`, and `a_document_in_the_stack_shape_with_no_page_entries_gets_none_added`. - A test that the comments of a document survive and its tables keep their order, and one that a document with Windows line endings keeps them. - A test that no line the command prints carries a value from the document, on every adapter, run on a document whose every value is a marker string. @@ -183,11 +192,12 @@ No test starts the Axum or Spin adapter with a migrated document, because the to ## Revision record -| Date | Change | -| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-17 | First draft. | -| 2026-10-18 | Fixes from review round 1. `--adapter`, with the location lookup each adapter accepts and the Edge Cookie and geo settings startup refuses left to do. No location header is selected, a handler is removed only where every path its pattern matches is closed, a host header is reported to check, and page entries are added to a document written for main alone. The flags, the validation's reach, the consent lists, the adapters, consent and privacy, failure, compatibility and rollout, the tests, and 39 breaking entries. | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 2026-10-17 | First draft. | +| 2026-10-18 | Fixes from review round 1. `--adapter`, with the location lookup each adapter accepts and the Edge Cookie and geo settings startup refuses left to do. No location header is selected, a handler is removed only where every path its pattern matches is closed, a host header is reported to check, and page entries are added to a document written for main alone. The flags, the validation's reach, the consent lists, the adapters, consent and privacy, failure, compatibility and rollout, the tests, and 39 breaking entries. Fixes from review round 2. A handler pattern with a quantifier straight after its prefix or bound, or a character class or comment after them, is left to do, and a check line names the region the Cloudflare lookup now reads. The checks made only at startup, the location main's consent saw, a document run on two adapters, the stored assets, the passwords, the tests and the status line. | ## Review log -1. **Round 1, 2026-10-18.** A review in a fresh session, against `fixpass6/51-config-migrate` at `50c68be1b`. Issues found, four of them blocking. The four were that the Axum and Spin adapters refuse the `[geo] module = "platform"` the command wrote into every document written for main, that the `[headers]` rewrite started sending the reader's location on Cloudflare, where main sent none, that the `[[handlers]]` rule removed a handler on `^/admin/keys`, which also guarded publisher paths such as `/admin/keystore` that the stack forwards to the origin, and that the spec had no compatibility and rollout section. Fixed in `18b7c7d16`, where the command selects `platform` only where the host has a lookup and leaves to do, for any document, an Edge Cookie module with no geo module and `platform` on an adapter with none, in `4465e2ad0`, where it takes `--adapter`, never selects `headers.geo` and prints a check line saying what main's responses carried, in `65bb09b14`, where it removes a handler only where every path its pattern can match is closed, and in this revision, which adds the adapters, consent and privacy, failure, and compatibility and rollout. The non-blocking findings are fixed too, with a host header reported to check in `65814d561` and page entries added to a document written for main alone in `f8dffebdc`. In this revision the consent lists stay items left to do with the definition of `to do:` widened to say so, the seam probe and the key-value store have rows of their own, and the count of breaking entries is 39. The review's statement that a selected `[ads-txt]` with no entry loads with a warning is not borne out, because the module refuses that document (`crates/ads-txt/iabtl/src/lib.rs` lines 154 to 175), so the `/ads.txt` entry is still added to any document. +1. **Round 1, 2026-10-18.** A review in a fresh session, against `split/51-config-migrate` at `50c68be1b`. Issues found, four of them blocking. The four were that the Axum and Spin adapters refuse the `[geo] module = "platform"` the command wrote into every document written for main, that the `[headers]` rewrite started sending the reader's location on Cloudflare, where main sent none, that the `[[handlers]]` rule removed a handler on `^/admin/keys`, which also guarded publisher paths such as `/admin/keystore` that the stack forwards to the origin, and that the spec had no compatibility and rollout section. Fixed in `18b7c7d16`, where the command selects `platform` only where the host has a lookup and leaves to do, for any document, an Edge Cookie module with no geo module and `platform` on an adapter with none, in `4465e2ad0`, where it takes `--adapter`, never selects `headers.geo` and prints a check line saying what main's responses carried, in `65bb09b14`, where it removes a handler only where every path its pattern can match is closed, and in this revision, which adds the adapters, consent and privacy, failure, and compatibility and rollout. The non-blocking findings are fixed too, with a host header reported to check in `65814d561` and page entries added to a document written for main alone in `f8dffebdc`. In this revision the consent lists stay items left to do with the definition of `to do:` widened to say so, the seam probe and the key-value store have rows of their own, and the count of breaking entries is 39. The review's statement that a selected `[ads-txt]` with no entry loads with a warning is not borne out, because the module refuses that document (`crates/ads-txt/iabtl/src/lib.rs` lines 154 to 175), so the `/ads.txt` entry is still added to any document. +2. **Round 2, 2026-10-18.** A review in a fresh session, against `split/51-config-migrate` at `6e3c09200`, of the round 1 fixes and the new text. Issues found, one of them blocking. The one was that a quantifier after the `^/admin/keys` bound or the `^/_ts/admin` prefix still let the command remove a handler that guarded publisher paths, with a `changed:` line and no `to do:`, as `^/admin/keys/?`, `^/admin/keys/*` and `^/admin/keys(?:/|$)?` matched `/admin/keystore` and `^/_ts/admin?` matched `/_ts/admit` under main's search. Fixed in `83dd4f1db`, where the command leaves such a pattern to do when `?`, `*` or `{` follows the prefix or the bound, and in `074d454e3`, where it also leaves to do a pattern with a `[` or a `#` after the prefix or the bound, because a character class or a comment the `(?x)` flag allows could hide an alternative such as `|^/members` from its scan, a fault the review did not name, found while checking which other text widens the match. The tests run each pattern through the `regex` crate and core's closed paths. The non-blocking findings are fixed too, with a check line on Cloudflare for the region its lookup now reads in `eb25e6d16`, and in this revision, which names the checks made only at startup, says the consent rules see the location main's jurisdiction check saw through `applies_in` and `privacy_states`, says a document run on two adapters is migrated once for each, adds the step on stored pages to the CLI guide, gives the comment on items left to do the spec's definition, and drops "the top pull request" from the status line. The round 1 commit hashes stay as they are, because `split/51-config-migrate` is merged forward and never rebased, so they resolve once it is pushed. From 9555b83392145d1e5ef33b3d5f523e740d1cb9b5 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 07:55:45 +0100 Subject: [PATCH 155/192] The geo module request design matches the branches The specs copy had stopped at the round 2 review of 15 October. It now carries the x-geo response headers written by the headers.geo serve middleware, the Axum and Spin refusal of the platform lookup, and every adapter calling detect and writing the request filters' response headers, as the branches do, with its review log last. --- .../2026-10-15-geo-module-request-design.md | 174 ++++++++++-------- 1 file changed, 100 insertions(+), 74 deletions(-) diff --git a/docs/superpowers/specs/2026-10-15-geo-module-request-design.md b/docs/superpowers/specs/2026-10-15-geo-module-request-design.md index c5e888d9bc..2534fd5e4e 100644 --- a/docs/superpowers/specs/2026-10-15-geo-module-request-design.md +++ b/docs/superpowers/specs/2026-10-15-geo-module-request-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-12-module-context-design.md`, `2026-10-12-device-signals-design.md`, `2026-08-27-integration-provider-seam-design.md`, `2026-09-07-didomi-geo-design.md`. Every design document of the stack is in IABTechLab/trusted-server#1342. **Related PRs:** IABTechLab/trusted-server#1374, reviewed on jwrosewell/trusted-server#28 (closed) **Issues:** IABTechLab/trusted-server#1321 -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-18 > **Why this spec exists.** A geo module was handed the client address and nothing else of the request, so a vendor whose one service answers location and device from the same request could not answer both from one call. @@ -13,15 +13,17 @@ A geo module was called as `PlatformGeo::lookup(client_ip, services)`, so the module was handed the client address and the services and nothing else of the request. -A device module is called as `DeviceModule::detect(call)`, with a module call that carries the request's view (its method, host, scheme, path and query), the request's evidence, the settings and the services. The evidence is narrow. On Fastly, the only adapter that calls `detect`, the evidence is the client address, the `User-Agent` header and, for a device module that a registration supplies, the `Cookie` header, and the evidence holds no path or query (`derive_device_signals` in `crates/trusted-server-adapter-fastly/src/lib.rs`). A device module's `advertising_attributes`, unlike `detect`, is handed every request header (`attach_device_attributes` in `crates/trusted-server-core/src/auction/formats.rs`). +A device module is called as `DeviceModule::detect(call)`, with a module call that carries the request's view (its method, host, scheme, path and query), the request's evidence, the settings and the services. The evidence is narrow. On Fastly the evidence is the client address, the `User-Agent` header and, for a device module that a registration supplies, the `Cookie` header, and the evidence holds no path or query (`derive_device_signals` in `crates/trusted-server-adapter-fastly/src/lib.rs`). A device module's `advertising_attributes`, unlike `detect`, is handed every request header (`attach_device_attributes` in `crates/trusted-server-core/src/auction/formats.rs`). A vendor whose one service answers location and device from the same request, keyed on the whole request, could not make the geo answer come from the same call as the device answer. The geo module never saw the evidence. The vendor therefore asked its service twice, or the geo answer depended on the device module having run first on the same adapter. -Six places ask for a location with the request in hand, and section 2.2 lists them. Each handed the module the address alone. +Six places asked for a location with the request in hand, and each handed the module the address alone. Section 2.2 lists the four that remain. The other two were the Fastly adapter's finalize steps, which no longer ask (section 2.10). A second fault sat in the Fastly adapter. The step that writes the `x-geo-*` headers on every response built its module with `build_geo_module` alone. That function cannot see the registry, so the function answers the host's own lookup for `platform` and the module that resolves nothing for every other name. A deployment whose `[geo] module` named a module from the registry got no location on that path, and every page carried `x-geo-info-available: false`, while the same module answered the request path. -The issue that asked for this change (IABTechLab/trusted-server#1321) does not ask for the fix to this second fault. The issue asks for the call to be built where each adapter resolves geo, and says nothing of the response headers. The fix changes the `x-geo-*` headers that responses carry on a deployment whose `[geo] module` names a module from the registry, so the open questions at the end of section 2 ask whether the fix is a pull request of its own. +The issue that asked for this change (IABTechLab/trusted-server#1321) does not ask for the fix to this second fault. The issue asks for the call to be built where each adapter resolves geo, and says nothing of the response headers. The fix changes which responses carry the `x-geo-*` headers, so the open questions at the end of section 2 ask whether the fix is a pull request of its own. + +The fix follows the rule that only middleware alters a response. Resolving the location is one operation, through the geo interface and its modules, and it fills in the request's model, which the permission rules, the auction and other modules read. Writing headers into a response is the work of a middleware. So core and the adapters write no `x-geo-*` header, and a serve middleware writes them from the location the request already resolved (section 2.10). ## 2. Design @@ -85,24 +87,18 @@ The method builds the call and calls `locate` with the module's own `id()` and ` With `headers` as `None`, the evidence still carries the address, the path and the query, and has no headers. -One function builds the call, so that a module is handed the same kinds of evidence from every caller. Six callers use that function. +One function builds the call, so that a module is handed the same kinds of evidence from every caller. Four callers use that function. -| Caller | Function | Headers passed | Settings passed | When the module fails | -| -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | ---------------------------- | ------------------------------- | --------------------------------------------------------------------------------- | -| The Edge Cookie state, and the permissions inspection endpoint | `EcContext::read_from_request_resolving_geo` in `crates/trusted-server-core/src/ec/mod.rs` | The request's | Yes | Logged at error level, and every permission resolves to the requires-signal floor | -| The auction endpoint | `handle_auction` in `crates/trusted-server-core/src/auction/endpoints.rs` | The request's | Yes | Logged as a warning, and the auction goes on with no location | -| The Didomi notice loader | `DidomiIntegration::route` in `crates/cmp/didomi/src/lib.rs` | The request's | No, the route is not handed any | The loader answers with a geo failure response | -| The seam probe's report | `SeamProbeProxy::route` in `crates/testing/seam-probe/src/lib.rs` | The request's | No | Logged as a warning, and the report's country is null | -| Fastly response headers, router path | `FinalizeResponseMiddleware::handle` in `crates/trusted-server-adapter-fastly/src/middleware.rs` | The request's, as it arrived | Yes | Logged as a warning, and the response carries `x-geo-info-available: false` | -| Fastly response headers, entry-point path | `apply_entry_point_finalize_headers` in `crates/trusted-server-adapter-fastly/src/lib.rs` | None | Yes | The same | +| Caller | Function | Headers passed | Settings passed | When the module fails | +| -------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | -------------- | ------------------------------- | --------------------------------------------------------------------------------- | +| The Edge Cookie state, and the permissions inspection endpoint | `EcContext::read_from_request_resolving_geo` in `crates/trusted-server-core/src/ec/mod.rs` | The request's | Yes | Logged at error level, and every permission resolves to the requires-signal floor | +| The auction endpoint | `handle_auction` in `crates/trusted-server-core/src/auction/endpoints.rs` | The request's | Yes | Logged as a warning, and the auction goes on with no location | +| The Didomi notice loader | `DidomiIntegration::route` in `crates/cmp/didomi/src/lib.rs` | The request's | No, the route is not handed any | The loader answers with a geo failure response | +| The seam probe's report | `SeamProbeProxy::route` in `crates/testing/seam-probe/src/lib.rs` | The request's | No | Logged as a warning, and the report's country is null | Fastly reads the Edge Cookie state on the fallback (except for a closed path) and on the routes of its named route table other than config, batch sync and data. The read comes after the request preparers have run, except on the permissions route. A request preparer is a function that an integration registers with `with_request_preparer` to clean or change a request before routing, and `IntegrationRegistry::prepare_request` runs each one. Axum, Cloudflare and Spin read the state only in the handlers that need the state, being the auction, page bids, the integration proxy routes, the publisher fallback and the permissions endpoint, which reads it through `handle_permissions` in `crates/trusted-server-core/src/inspect/permissions.rs`. -**How the callers differ.** The six callers do not build the call alike. - -1. The Edge Cookie state, the auction endpoint and the seam probe build the request's view with `ModuleRequest::of`, which reads the host from `Forwarded`, then `X-Forwarded-Host`, then `Host`, and the scheme from the client's TLS metadata, then `Forwarded`, then `X-Forwarded-Proto`, then `fastly-ssl`, and gives `http` when none of them says otherwise. The other three build the view by hand. The Didomi loader and the Fastly middleware read the `Host` header and take the scheme from the request's URI, or `https` when the URI has none. The Fastly entry-point step reads the `Host` header and gives `https` when the `fastly-ssl` header is present and `http` otherwise. -2. The services of the first four callers carry all the client's metadata that the entry point captured. The two Fastly finalize steps set the services' client metadata to the client address alone. -3. The two Fastly finalize steps are given a smaller services graph than the request path (`build_finalize_services` in `crates/trusted-server-adapter-fastly/src/app.rs`). The graph has the config, secret and key-value stores, the value cache, the loading caches, the backend, the HTTP client and the geo module, and each step adds the publisher's prefixes. The graph has no host signals, no device module, no Edge Cookie module, no permission signal modules and no template cache or assembler, and its auction telemetry sink records nothing. A geo module that reads any of these from the services finds nothing there on those two steps. +**How the callers differ.** The Edge Cookie state, the auction endpoint and the seam probe build the request's view with `ModuleRequest::of`, which reads the host from `Forwarded`, then `X-Forwarded-Host`, then `Host`, and the scheme from the client's TLS metadata, then `Forwarded`, then `X-Forwarded-Proto`, then `fastly-ssl`, and gives `http` when none of them says otherwise. The Didomi loader builds the view by hand, reading the `Host` header and taking the scheme from the request's URI, or `https` when the URI has none. The services of all four callers carry all the client's metadata that the entry point captured. ### 2.3 A lookup that needs only the address @@ -131,30 +127,20 @@ The registry, in `crates/trusted-server-core/src/integrations/registry.rs`, hold Every adapter builds the services for a request with `build_geo_module`, wrapping the adapter's own lookup, and then replaces that module with the registry's when `IntegrationRegistry::geo_module()` returns one. -| Adapter | What `platform` selects | `x-geo-info-available` | -| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | -| Fastly | `FastlyPlatformGeo` in `crates/geo/fastly`, which resolves the client address with Fastly's geo lookup | From the module's answer, on the two steps described below | -| Axum | `AxumPlatformGeo`, which resolves nothing | Always `false` | -| Cloudflare | `CloudflareGeo`, built from the request's `cf-ipcountry`, `cf-ipcity`, `cf-ipcontinent`, `cf-iplatitude`, `cf-iplongitude` and `cf-region-code` headers and ignores the client address | `true` when `cf-ipcountry` is present and not `XX`, with no module call | -| Spin | `NullGeo`, which resolves nothing | Always `false` | - -**Fastly.** `build_per_request_services` in `crates/trusted-server-adapter-fastly/src/app.rs` builds the services with `build_geo_module(&state.settings, Arc::new(FastlyPlatformGeo))` and replaces the module with `state.registry.geo_module()` when that returns one. The Edge Cookie state, the auction endpoint and every module that calls `services.geo()` use that module. +| Adapter | What `platform` selects | +| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Fastly | `FastlyPlatformGeo` in `crates/geo/fastly`, which resolves the client address with Fastly's geo lookup | +| Axum | `AxumPlatformGeo`, which resolves nothing, so startup refuses `platform` unless the caller supplies its own services | +| Cloudflare | `CloudflareGeo`, built from the request's `cf-ipcountry`, `cf-ipcity`, `cf-ipcontinent`, `cf-iplatitude`, `cf-iplongitude` and `cf-region-code` headers and ignores the client address | +| Spin | `NullGeo`, which resolves nothing, so startup refuses `platform` unless the caller supplies its own services | -The adapter writes the `x-geo-*` headers on every response, from the location a module resolves for the request. Two steps do this. +On Cloudflare the region comes from `cf-region-code` alone, which Cloudflare's Add visitor location headers managed transform adds. Without it a US visitor is placed in no state, so the rules `permissions.yaml` gives a US state do not apply (`a_us_state_visitor_reaches_the_us_state_jurisdiction_and_its_opt_out` in the adapter's `platform.rs`). -1. `FinalizeResponseMiddleware` finishes every response that passes through the router's middleware chain. The router is given `finalize_geo_module(state)` in `app.rs`, which is the module `IntegrationRegistry::geo_module()` returns, and what `build_geo_module` returns when that call returns `None`. Before the handler takes the request, `handle` keeps the request's method, headers and URI as the request arrived and builds the host and scheme by hand. After the handler, `handle` calls `locate_request` with the kept method, headers and URI, the host and scheme, the settings and the client address alone as the client's metadata. `handle` skips the call when `geo_allowed_for_response` is false, which is the case for a 401 response. The method, headers and URI are copied on every request, before the middleware knows the response's status, and the copy is made for a deployment whose `[geo] module` is unset too. -2. `apply_entry_point_finalize_headers` finishes a response that did not pass through the middleware, such as the 405 the router returns for a method no route accepts. The middleware marks the responses it finished, and the entry point runs this step for those without the mark. The step builds its module with `build_geo_module`, so the module is the host's lookup or the module that resolves nothing, and never one from the registry. The step calls `locate_request` with the request's view from the copy the entry point kept (method, host, scheme, path and query), the settings and the client address alone as the client's metadata. The step passes no headers, because the entry point keeps only the request's view. - -**Why the entry-point step does not use `finalize_geo_module`.** The comment on the step says the module is the host's own lookup or the module that resolves nothing, never one from the registry, and that both read the address alone, so the call carries no headers. The comment gives no reason for leaving the registry out. The entry point holds the application state as an `Option` that is `None` when the state failed to build, and runs the step in that case too, with no registry to ask. When the state exists the entry point holds the registry, so the code could give the step the module `finalize_geo_module` returns, and does not. - -**Why each step asks the module again.** The response already carries the location the request path resolved, inside `EcFinalizeState` in `app.rs`, on the responses whose handler read the Edge Cookie state, apart from two. The permissions route and the asset fallback read the state and attach no `EcFinalizeState`, so their responses carry no location. Neither step reads that location. The middleware keeps the request's headers so that a module that holds its answers by evidence gives the answer it already holds. The headers help only when the middleware's call carries what the request path's call carried, and four differences between the two calls can stop the middleware's call from carrying the same things. +**Fastly.** `build_per_request_services` in `crates/trusted-server-adapter-fastly/src/app.rs` builds the services with `build_geo_module(&state.settings, Arc::new(FastlyPlatformGeo))` and replaces the module with `state.registry.geo_module()` when that returns one. The Edge Cookie state, the auction endpoint and every module that calls `services.geo()` use that module. -1. A request preparer runs inside the handler (`prepare_request` in `execute_named` and `dispatch_fallback`), before the Edge Cookie state is read, and a preparer can change the request. The GPT diagnostics preparer (`crates/ad-tag/google/src/diagnostics.rs`) removes its reserved query parameter and cookie, and `IntegrationRegistry::prepare_request` runs every preparer whether or not its integration is switched on. The middleware keeps the headers and URI as the request arrived, before any preparer. When a preparer changes the request, the middleware's call therefore carries the unchanged request where the request path's call carries the changed one, and a module that holds its answers by the evidence it was given finds no answer held for the middleware's call. -2. The middleware builds the host and scheme by hand, and the request path's call builds them with `ModuleRequest::of` (section 2.2). -3. The middleware's call carries the client address alone as the client's metadata, and the request path's call carries what the entry point captured (section 2.2). -4. The middleware's services graph is the smaller one (section 2.2). +No adapter calls a geo module or writes an `x-geo-*` header when it finishes a response (section 2.10). -**Axum, Cloudflare and Spin.** Each builds the request's services in `build_runtime_services` in its `platform.rs`, and `services_for_request` in its `app.rs` replaces the module with the registry's. This series changes no non-test line in these three adapters, because `build_geo_module` and `RuntimeServices::with_geo` carry the new type. None of the three adapters calls the geo module when writing response headers, so a registry module reaches them only through the core callers in section 2.2. +**Axum, Cloudflare and Spin.** Each builds the request's services in `build_runtime_services` in its `platform.rs`, and `services_for_request` in its `app.rs` replaces the module with the registry's, so a registry module reaches them through the core callers in section 2.2. `build_geo_module` and `RuntimeServices::with_geo` carry the new type, so the only other change to these three adapters is that their finalize step writes no `x-geo-info-available` (section 2.10). ### 2.6 Configuration @@ -171,6 +157,7 @@ No setting is added or changed. The `[geo]` section keeps its two keys. - Two registrations that offer a geo module under one name are refused at startup, and the error names both. - An offered module that the selector does not choose is logged as a warning when the registry is built. - `assume_single_jurisdiction` must be `true` when an Edge Cookie module is configured and `[geo] module` is unset or `none` (`GeoConfig::validate_jurisdiction_acknowledgment` in `settings.rs`). +- On Axum and Spin, whose own lookups resolve nothing, `platform` is refused at startup naming the setting (`refuse_platform_geo` in each adapter's `app.rs`). The selection would locate no request and still pass as a geo module, so the acknowledgement above would never be asked for. Services a caller supplies carry a lookup of their own, which `platform` selects, so the check is skipped for them. **Defaults.** The defaults are unchanged. A deployment with no `[geo] module` resolves no location and makes no host geo call. @@ -186,7 +173,7 @@ No setting is added or changed. The `[geo]` section keeps its two keys. - The services, which a lookup was already handed. - The settings, when the caller has them. -Five of the six callers pass the headers of the request they are serving. `apply_entry_point_finalize_headers` passes none. +All four callers pass the headers of the request they are serving. **What a device module is handed.** On Fastly, the evidence that `detect` is shown holds the client address, the `User-Agent` header and, for a device module that a registration supplies, the `Cookie` header, and holds no path or query. The Fastly device module that `[device] module = "fastly"` selects also reads the TLS and HTTP/2 fingerprints, which the adapter captures into `FastlyHostSignals` and does not pass as headers. Only `advertising_attributes`, which the auction code calls, is handed every request header. A geo module is therefore handed more of the request as evidence than `detect` is, and that includes what the device signals are made from, being the `User-Agent` header and, on Fastly, the TLS and HTTP/2 fingerprint headers. @@ -194,13 +181,13 @@ Five of the six callers pass the headers of the request they are serving. `apply **No permission check guards the call.** The evidence is handed to a geo module without a check. A device module's `detect` is also called without one, before permissions are resolved, but `detect` is handed fewer headers. -**A permission guards the answer, for other modules only.** When a geo module declares permissions, the location is passed on to another module only if that module declares every one of them and the request has been granted them. When `required_permissions` is empty, which is the default, the location is passed to every module. `ModuleContext::with_request_state` in `crates/trusted-server-core/src/module_context.rs` and the permission step of the Edge Cookie state in `crates/trusted-server-core/src/ec/mod.rs` set the requirement from `services.geo().required_permissions()`. `PlatformGeoModule` gives the wrapped lookup's declaration. No permission guards the response headers. On Fastly, `apply_finalize_headers` writes the answer's city, country, continent and coordinates, and the metro code and region when the answer has them, into the `x-geo-*` headers of every response except a 401, whatever permissions the module declares and whatever the request has been granted (`GeoInfo::set_response_headers` in `crates/trusted-server-core/src/geo.rs`). +**A permission guards the answer.** When a geo module declares permissions, the location is passed on to another module only if that module declares every one of them and the request has been granted them. When `required_permissions` is empty, which is the default, the location is passed to every module. `ModuleContext::with_request_state` in `crates/trusted-server-core/src/module_context.rs` and the permission step of the Edge Cookie state in `crates/trusted-server-core/src/ec/mod.rs` set the requirement from `services.geo().required_permissions()`. `PlatformGeoModule` gives the wrapped lookup's declaration. The `x-geo-*` response headers are written by a middleware from the location its module call is handed, so the same gate applies to them (section 2.10). **What the built-in lookups read.** The Fastly lookup reads the client address alone, so a deployment on `platform` or `none` sends no more to a host service than before. The Cloudflare lookup reads the platform's `cf-*` request headers when the services are built, as before. `PlatformGeoModule` hands each lookup the address and nothing else of the evidence. **What a module does with the evidence.** Core filters nothing from the evidence, and a module that sends any of it to another service decides what to send. The operator controls what is sent by choosing the module with `[geo] module`. With the key unset or `none`, the module called uses none of the request's data and answers nothing. -**Responses that get no lookup.** On both Fastly steps, `geo_allowed_for_response` stops the finalize call for a 401 response, so no module is called a second time and the response carries `x-geo-info-available: false` in place of a location. The rule does not stop the call on the request path, and that call has run before the response exists. On Fastly, a request to `/_ts/api/v1/identify` with an invalid token and a request that the origin answers with a 401 have both already handed the module every request header, `Authorization` included, through the Edge Cookie state before the 401 exists. +**No lookup when a response is finished.** No adapter calls a geo module when it finishes a response, so the rule that skipped the Fastly finalize lookup for a 401 went with the lookup. The request path's call runs before the response exists. On Fastly, a request to `/_ts/api/v1/identify` with an invalid token and a request that the origin answers with a 401 have both handed the module every request header, `Authorization` included, through the Edge Cookie state. A 401 page from the origin that a `[[serve]]` entry naming `headers.geo` covers carries the location (open question 2). ### 2.8 Failure handling @@ -210,48 +197,86 @@ The Edge Cookie state resolves permissions at the top node of the `permissions.y - A call with no services reaches `PlatformGeoModule` and answers `Ok(None)`. It is not an error. - `locate_request` always supplies the evidence, so a module called through it always finds the evidence. A call built by hand without evidence withholds it, and the test modules return `PlatformError::Geo` for that. -- Core adds no timeout, retry or cache around the call. Each caller calls once, waits for the answer and uses the answer, so a slow module delays the caller. On Fastly both finalize steps wait before the response headers are sent, so a slow module delays every response except a 401. A module that reaches a service applies its own timeout and holds its own answers. +- Core adds no timeout, retry or cache around the call. Each caller calls once, waits for the answer and uses the answer, so a slow module delays the caller. A module that reaches a service applies its own timeout and holds its own answers. ### 2.9 Rollout for existing deployments -**Deployments.** No settings document and no stored data changes. A deployment on `none` or `platform` behaves as before, apart from the cost below, because the wrapped lookups read the same address and the built-in lookups ignore the added evidence. +**Deployments.** No stored data changes. A deployment on `none` or `platform` resolves the location as before, because the wrapped lookups read the same address and the built-in lookups ignore the added evidence. A deployment on Axum or Spin that selected `platform`, which located nothing there, no longer starts, and leaves `[geo] module` unset, with `assume_single_jurisdiction = true` where it runs an Edge Cookie module, or selects a module a linked crate offers. The integration tests start the Axum dev server that way (`axum_app_config_envelope` in `crates/trusted-server-integration-tests/tests/common/config.rs`), because their shared fixture selects `platform` for Viceroy. -**One difference.** On Fastly, a deployment whose `[geo] module` names a module from the registry now gets that module's location in the `x-geo-*` headers of every response the middleware finishes. Before, those responses carried `x-geo-info-available: false`. Those headers carry the module's city, coordinates, metro code and region as well as its country, and no permission guards them (section 2.7). Responses finished by the entry-point step are unchanged. - -**One cost.** On Fastly, `FinalizeResponseMiddleware` copies the request's method, headers and URI before the handler runs, on every request and whatever `[geo] module` selects, and when the key is unset. +**The headers, a breaking change.** No response carries an `x-geo-*` header until a deployment selects `headers.geo` and names it in a `[[serve]]` entry, and then only the responses the entry covers carry them (section 2.10). Anything that reads the headers from a route of Trusted Server's own, an image or another response no entry covers gets none. On Cloudflare `x-geo-info-available` followed `cf-ipcountry` whatever `[geo] module` selected, and it now follows the location the selected module resolves, so it says `false` with the key unset. **Crates outside core** that use the items below stop compiling until they change, and `CHANGELOG.md` marks the change as breaking for them. -- `RuntimeServicesBuilder::geo`, `RuntimeServices::with_geo`, `IntegrationRegistrationBuilder::with_geo_module` and the Fastly adapter's `FinalizeResponseMiddleware::new` take an `Arc` where they took an `Arc`, and `build_geo_module` returns one. +- `RuntimeServicesBuilder::geo`, `RuntimeServices::with_geo` and `IntegrationRegistrationBuilder::with_geo_module` take an `Arc` where they took an `Arc`, and `build_geo_module` returns one. +- `GeoInfo::set_response_headers` and the `HEADER_X_GEO_*` constants are gone from core (section 2.10). - `RuntimeServices::geo()` returns `&dyn GeoModule`. A caller that called `lookup` on it calls `locate_request`. - `IntegrationRegistry::geo_module()` returns an `Option>`, and the public field `IntegrationRegistration::geo_module` holds an `Option<(&'static str, Arc)>`, being the name `[geo] module` selects the module by and the module, where each had an `Arc` in place of the `Arc`. - A crate that passed a `PlatformGeo` wraps it in `PlatformGeoModule::new(name, module)`. A crate that wants the request implements `GeoModule`. -In this repository the converted code is the six callers in section 2.2, the `ts` command line tool's local services in `crates/trusted-server-cli/src/commands/keys.rs`, the seam probe's registration, the example in the integration guide, and the test services in core, the Fastly adapter, the Axum adapter, the Prebid server crate, the 51Degrees loading cache crate and the Didomi crate. +In this repository the converted code is the four callers in section 2.2, the `ts` command line tool's local services in `crates/trusted-server-cli/src/commands/keys.rs`, the seam probe's registration, the example in the integration guide, and the test services in core, the Fastly adapter, the Axum adapter, the Prebid server crate, the 51Degrees loading cache crate and the Didomi crate. + +### 2.10 The `x-geo-*` response headers + +Core and the four adapters write no `x-geo-*` header, `x-geo-info-available` included, and call no geo module when they finish a response. The module `headers.geo`, in `crates/headers/geo` (package `trusted-server-headers-geo`), supplies a serve middleware of the same name that writes them. The headers are Trusted Server's own and belong to no vendor, so the crate takes the neutral name `geo` under a type of its own, `headers`, and `[geo]` still selects the geo module alone. + +**What a deployment that wants them adds.** + +```toml +[headers] +modules = ["geo"] + +[[serve]] +media_type = "text/html" +middleware = ["headers.geo"] +``` + +A document that selects the module and names it in no `[[serve]]` entry is refused when the settings load and at deploy validation, because the headers would never be written. The middleware works on every media type an entry may cover, so a deployment that wants the headers on JSON or a script writes an entry for that type too. + +**Where the location comes from.** The middleware prepares, and its preparation reads the location from its module call. The serve phase builds that call from the request's Edge Cookie state with `ModuleContext::with_request_state`, on a page fetched for the reader and on a page from the store, so the location is the one `EcContext::read_from_request_resolving_geo` resolved. That one location also chose the permission rules, and the server-side auction on a page reads it as the device's geo (`apply_auction_eids_and_device` in `publisher.rs`). No module is called again. The middleware's `create` writes `x-geo-city`, `x-geo-country`, `x-geo-continent` and `x-geo-coordinates`, `x-geo-metro-code` when the answer has a metro code, `x-geo-region` when the answer has one, and `x-geo-info-available: true`. With no location it writes `x-geo-info-available: false` alone. A value a header cannot carry is skipped with a warning, as before. + +**Which responses carry them.** Only a response that a `[[serve]]` entry naming `headers.geo` covers, being a publisher response of a covered media type and path, from the origin or from the store, whatever its status. Trusted Server's own routes, an integration's routes, an image, a response with no body, the router's 405 and a startup error carry none. A header of one of these names that the origin sends passes through, and `[response_headers]` is applied after the middleware, so an operator's header of one of these names still replaces the middleware's. + +**Permissions.** The location reaches the middleware through the gate in section 2.7. The middleware declares no permission, so where the geo module declares one the middleware is handed no location and writes `x-geo-info-available: false` (open question 3). + +**What is removed.** On Fastly, `FinalizeResponseMiddleware` loses its geo module, its services graph, its registry and the copy of the request it made on every request, and `finalize_geo_module`, `geo_allowed_for_response`, the entry point's `apply_entry_point_finalize_headers` and `EntryRequest` are gone. `build_finalize_services` and `entry_point_finalize_services` are renamed `build_entry_point_services` and `entry_point_services`, because the device module the entry point calls is now their only user. Each adapter's `apply_finalize_headers` takes no location, and the Cloudflare and Spin versions lose their `geo_available` argument. Core's `GeoInfo::set_response_headers` and the seven `HEADER_X_GEO_*` constants move to the crate. + +### 2.11 Other changes to a response on the finalize path + +These still change a response outside middleware, and each could move to a serve middleware in the same way later. + +| Change | Where | Adapters | +| -------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | +| Removes `X-Served-By`, `X-Cache` and `X-Cache-Hits` outside `/_ts` | `strip_deployment_headers` in `crates/trusted-server-core/src/http_util.rs` | All four, in `FinalizeResponseMiddleware` | +| Writes `[response_headers]`, and keeps a response with `Set-Cookie`, or one that is uncacheable, private to a shared cache | `apply_response_headers_with_cache_privacy` in `crates/trusted-server-core/src/response_privacy.rs` | All four, in `apply_finalize_headers` | +| Writes `X-Robots-Tag: noindex, nofollow` when `[robots-txt]` selects `refuse_all` | `apply_response_robots_tag` in `crates/trusted-server-core/src/robots_txt/mod.rs` | All four, in `apply_finalize_headers` | +| Writes `X-TS-ENV: staging` when `FASTLY_IS_STAGING` is `1` | `apply_finalize_headers` in `crates/trusted-server-adapter-fastly/src/middleware.rs` | Fastly | +| Writes an asset route's cache directives again | `AssetProxyCachePolicy::apply_after_route_finalization` in `crates/trusted-server-core/src/proxy.rs` | Fastly, at the entry point | +| Writes the Edge Cookie's `Set-Cookie` and identity headers | `ec_finalize_response` in `crates/trusted-server-core/src/ec/finalize.rs` | Fastly, at the entry point | +| Writes the request filters' response headers, `private, no-store` on a response marked private, and the late cache guards | `apply_terminal_response_effects`, in `crates/trusted-server-adapter-fastly/src/lib.rs` and in each other adapter's `src/middleware.rs` | All four, at the Fastly entry point and in the others' `FinalizeResponseMiddleware` | +| Writes the reusable sandbox's counters on a `private, no-store` response | `attach_sandbox_counters` in `crates/trusted-server-adapter-fastly/src/lib.rs` | Fastly, at the entry point | + +### 2.12 Open questions + +Each of these waits for a decision. + +1. Whether the Didomi loader builds the request's view the way `ModuleRequest::of` does, so that all four callers read the host and scheme by one rule. +2. Whether `headers.geo` leaves the headers off a 401, as the Fastly finalize lookup did. A middleware is not told the response's status, so the rule needs `MiddlewareContext` to carry it, which changes core's middleware contract. +3. Whether `headers.geo` declares permissions, or takes them from its settings, so that a deployment whose geo module declares a permission can still have the headers written. +4. Whether moving the `x-geo-*` headers to `headers.geo`, which the issue for this change (IABTechLab/trusted-server#1321) does not ask for, is a pull request of its own. -**Open questions.** Questions 1 to 6 wait for a decision and for a code change. Items 7 to 12 are follow-ups that need an edit and no decision, each with the file and lines it has in `split/33-geo-call` at `6944f4e70`. This revision changes no code and no document outside this page. +Follow-ups from review round 2 that need an edit and no decision, each with the file and lines it has in `split/33-geo-call` at `6944f4e70`: -1. Whether `FinalizeResponseMiddleware` reads the location that `EcFinalizeState` carries on the response and asks the module only for a response that carries none. Today a request that reads the Edge Cookie state asks the module a second time at the finalize step unless the response is a 401, and the second call carries the differences listed in section 2.5. Reading the carried location would also remove those differences for the responses that carry one. -2. Whether `apply_entry_point_finalize_headers` uses the module `finalize_geo_module` returns when application state was built, so that a response the router did not finish follows `[geo] module` as the middleware's responses do. The step would hand the module the request's view and no headers. -3. Whether the `x-geo-*` response headers are written only when the request has been granted the permissions the module declares, as the location handed to another module is. Today the headers carry the answer's city, coordinates, metro code and region whatever the module declares. -4. Whether the Didomi loader, the Fastly middleware and the Fastly entry-point step build the request's view the way `ModuleRequest::of` does, so that all six callers read the host and scheme by one rule. -5. Whether the middleware copies the request's method, headers and URI only when `[geo] module` selects a module from the registry, since the host's lookup and the module that resolves nothing read the address alone. -6. Whether the fix for the second fault, which changes the `x-geo-*` headers of a deployment's responses and which the issue for this change (IABTechLab/trusted-server#1321) does not ask for, is a pull request of its own. -7. `CHANGELOG.md`, line 12, says a geo module is called with the request's module call "as a device module is". Section 2.7 shows that the two calls carry different evidence, because `detect` is shown less of the request than a geo module is, so the entry should say only that the call has the same type. -8. `docs/guide/integration-guide.md`, line 416 and lines 432 to 433, say a geo module is called with the request "as a device module is", and the same correction applies. -9. The doc comment of `GeoModule` in `crates/trusted-server-core/src/platform/traits.rs`, lines 191 to 195, says no headers survive to the Fastly finalize paths and that both call the host's own lookup, never a module from the registry. Only the entry-point step does that. The middleware keeps the request's headers and calls the module `finalize_geo_module` returns, which can be the registry's (section 2.5). -10. The doc comment of `geo_allowed_for_response` in `crates/trusted-server-adapter-fastly/src/middleware.rs`, lines 178 to 180, says the function only decides, and leaves the lookup to its callers, because `PlatformGeo::lookup` is asynchronous, where both callers now call `locate_request`. -11. The doc comment of `GeoConfig` in `crates/trusted-server-core/src/settings.rs`, lines 1460 to 1462, says `[geo]` selects the module that resolves a client IP, where the module is now called with the request. -12. The `Geo / IP intelligence` row of the capability table in `AGENTS.md`, line 400, names `PlatformGeo` as the trait. `[geo] module` selects a `GeoModule`, and `PlatformGeoModule` wraps a `PlatformGeo`. +1. `CHANGELOG.md`, line 12, says a geo module is called with the request's module call "as a device module is". Section 2.7 shows that the two calls carry different evidence, because `detect` is shown less of the request than a geo module is, so the entry should say only that the call has the same type. +2. `docs/guide/integration-guide.md`, line 416 and lines 432 to 433, say a geo module is called with the request "as a device module is", and the same correction applies. +3. The `Geo / IP intelligence` row of the capability table in `AGENTS.md`, line 400, names `PlatformGeo` as the trait. `[geo] module` selects a `GeoModule`, and `PlatformGeoModule` wraps a `PlatformGeo`. ## 3. What is not in this change - **A geo module in the stock list that reads more than the address.** The host lookups for Fastly, Axum, Cloudflare and Spin stay `PlatformGeo` and are wrapped. The only module this repository offers through the registry is the seam probe's, which wraps a lookup that answers a fixed country. -- **The Fastly entry-point step.** `apply_entry_point_finalize_headers` still builds its module with `build_geo_module` and passes no headers. When `[geo] module` names a registry module, a response that reaches that step carries `x-geo-info-available: false`. -- **Geo in the response headers of Axum, Cloudflare and Spin.** Their `x-geo-info-available` does not follow the module, as the table in section 2.5 shows. +- **Moving the other changes a finalize step makes to a response.** Section 2.11 lists them. - **Making core share one answer between the geo call and the device call.** A vendor crate whose one service answers both holds the answer itself and returns it from `locate` and from `detect`. The device call carries less evidence than the geo call (section 2.7), so a held answer can be keyed only on what both calls carry. - **Checking permissions before the evidence is handed over.** A geo module is handed the evidence without a check, and so is a device module's `detect`, which is handed less (section 2.7). -- **Any change to `PlatformGeo`, `GeoInfo`, the permission rules or the headers written.** +- **Any change to `PlatformGeo`, `GeoInfo`, the permission rules or what the `x-geo-*` headers say.** ## 4. Acceptance @@ -261,8 +286,10 @@ In this repository the converted code is the six callers in section 2.2, the `ts - `crates/trusted-server-core/src/platform/mod.rs` has `runtime_services_geo_lookup_returns_none_for_no_ip`, which calls the services' module with a call. The same file has `build_geo_module_defaults_to_no_geo`, `build_geo_module_none_selects_no_geo_explicitly` and `build_geo_module_uses_host_geo_when_platform_is_selected`, which check the module's name. The first of the three also checks that a call answers nothing. - `crates/trusted-server-core/src/ec/mod.rs` has `a_geo_module_is_called_with_the_request_as_evidence`, where a module answers `FR` only if it reads a header from the call's evidence. The same file has `a_geo_module_reads_a_platform_service_through_the_services_it_is_given` and `a_geo_module_failure_resolves_permissions_at_the_requires_signal_floor`, which run through the wrapper. - `crates/trusted-server-core/src/integrations/registry.rs` has `geo_selector_resolves_the_geo_module_the_selected_module_declares`, `geo_module_none_resolves_no_location` and `an_unset_geo_selector_resolves_the_disabled_module_and_platform_opts_in`. The same file has the refusals `geo_selector_rejects_a_module_that_declares_no_geo_module`, `geo_selector_rejects_a_module_no_section_selects`, `geo_selector_rejects_a_name_no_module_supplies` and `two_registrations_declaring_one_geo_module_name_are_refused`. -- `crates/trusted-server-adapter-fastly/src/middleware.rs` has `finalize_handle_hands_the_geo_module_the_request_headers`, where a module records one header of its evidence. The existing finalize tests in that file now run through the wrapper. +- `crates/headers/geo/src/lib.rs` has the tests of `headers.geo`. `the_location_in_the_module_call_is_written_as_the_headers`, `a_metro_code_of_zero_and_no_region_are_not_written`, `a_value_a_header_cannot_carry_is_skipped`, `no_location_says_none_is_available`, `a_location_whose_use_needs_a_permission_the_middleware_does_not_declare_is_not_written` and `nothing_prepared_says_no_location_is_known` cover what it writes. `a_page_carries_the_location_the_request_resolved_and_the_geo_module_is_asked_once` and `a_page_no_serve_entry_names_the_middleware_on_carries_no_geo_header` send a page through `handle_publisher_request` and the buffered finalizer, with a geo module that counts its calls. `a_document_selecting_the_module_and_naming_it_in_no_serve_entry_is_refused` covers the refusal. +- Each adapter's finalize step is tested to write no `x-geo-*` header, by `finalizing_writes_no_geo_header` in the `middleware.rs` of all four adapters, `finalize_handle_writes_no_geo_header` in the Fastly adapter's `middleware.rs`, `entry_point_finalize_writes_no_geo_header` in its `lib.rs`, `finalize_middleware_writes_the_operators_headers_and_no_geo_header` in the `tests/routes.rs` of Axum, Cloudflare and Spin, and `no_adapter_writes_a_geo_header_on_its_own_routes` in `crates/trusted-server-integration-tests/tests/parity.rs`. `a_finished_response_carries_no_geo_header_and_the_geo_module_is_asked_once`, in the Fastly adapter's `src/app/seam_probe_tests.rs`, sends a request through the router with a registry geo module that answers a location and counts its calls. - `crates/trusted-server-adapter-fastly/src/app/seam_probe_tests.rs` and `crates/trusted-server-adapter-axum/tests/seam_probe.rs` each have `proxy_route_reports_the_modules_geo_and_that_the_preparer_ran`, where a route reaches the registry's module selected by `[geo] module` and gets its country back. Each also has `geo_selector_naming_a_module_without_a_geo_module_fails_at_startup`. +- The `tests/routes.rs` of Axum and of Spin each have `selecting_the_platform_geo_lookup_fails_at_startup`, which checks that `platform` is refused at startup with an error naming the setting and saying the host has no geo lookup. - `crates/trusted-server-adapter-spin/src/platform.rs` has `build_runtime_services_uses_noop_native_stores_without_handles`, which checks that the default module is named `none`. - `crates/trusted-server-integration-tests/tests/documentation_snippets.rs` has `integration_guide_runtime_services_fixture_compiles`, which compiles the runtime services example in the integration guide, the example that wraps `FixtureGeo` in `PlatformGeoModule::new`. @@ -274,15 +301,11 @@ The Cloudflare adapter's tests are unchanged and call its lookup directly. 2. The Didomi loader handing the module the request's headers. The Didomi tests run a stub lookup behind the wrapper, which reads only the address. 3. The seam probe handing the module the request's headers. The seam probe tests show that the registry's module is reached and answers, and its wrapped lookup ignores the evidence. 4. The settings reaching a module through `locate_request`. The tests that call it pass `None`, and no test module reads the settings from the call. -5. `FinalizeResponseMiddleware` handing the module the request's method, host, scheme, path, query and the settings. The new test checks one header only. -6. `finalize_geo_module` choosing the registry's module over the host's. The new Fastly test builds the middleware with a module passed in directly. -7. `apply_entry_point_finalize_headers` calling a module at all. The existing entry-point test covers `geo_allowed_for_response` and `apply_finalize_headers`. -8. `build_geo_module` wrapping the host lookup it is given. The earlier checks compared the two pointers, which the wrapper makes impossible, and the new checks read the module's name. -9. `PlatformGeoModule::required_permissions` returning the lookup's declaration, and the gate on the location that follows from it. The only test of a declaration is for `DisabledGeo`, which declares none. -10. A registry geo module answering a request on Cloudflare or Spin. The seam probe tests that do this run on Fastly and Axum only. -11. The wrapped Axum, Cloudflare and Spin lookups reaching a request through `build_runtime_services`. Their tests call the lookup directly, and the Spin test checks the name of the default module only. -12. A located answer reaching the Fastly response headers. Every finalize test in the Fastly adapter's `middleware.rs` and `lib.rs` has the module answer nothing, so none checks an `x-geo-*` header with a location in it. `proxy_route_reports_the_modules_geo_and_that_the_preparer_ran` passes through `FinalizeResponseMiddleware` with the registry's module selected, and checks the body and not the response's headers. -13. A request that a preparer changes, sent through the router with a geo module that reads its evidence. No test compares the evidence the request path's call carries with the evidence the middleware's call carries. +5. `build_geo_module` wrapping the host lookup it is given. The earlier checks compared the two pointers, which the wrapper makes impossible, and the new checks read the module's name. +6. `PlatformGeoModule::required_permissions` returning the lookup's declaration, and the gate on the location that follows from it. The only test of a declaration is for `DisabledGeo`, which declares none. +7. A registry geo module answering a request on Cloudflare or Spin. The seam probe tests that do this run on Fastly and Axum only. +8. The wrapped Axum, Cloudflare and Spin lookups reaching a request through `build_runtime_services`. Their tests call the lookup directly, and the Spin test checks the name of the default module only. +9. `headers.geo` on a page from the store, through the streaming finalizer, and on a media type other than HTML. Its page tests run the buffered finalizer on a page that is not stored. The pull request records the fork's CI green on all 19 checks at 1b8b43858. Its checklist leaves the manual check under `fastly compute serve` unticked. @@ -293,6 +316,9 @@ The pull request records the fork's CI green on all 19 checks at 1b8b43858. Its | 2026-10-15 | First draft, written after the code. | | 2026-10-15 | Fixes from review round 1. The title, what a device module and a geo module are each handed, the evidence the router path's call carries, the 401 rule, the services the finalize steps get, the response headers that no permission guards, the wait for the module, the breaking items, the missing tests, the open questions and the review log. | | 2026-10-15 | Corrections from review round 2. Why the entry-point step skips the registry, which responses carry `EcFinalizeState`, what the Fastly device module reads, the type of the registration field, the permissions endpoint in Axum, Cloudflare and Spin, the round 1 log sentence on the code changes, and the follow-ups for the changelog, the integration guide and four stale comments. | +| 2026-10-16 | The `x-geo-*` response headers move from core and the adapters to the serve middleware `headers.geo` (section 2.10), so no finalize step calls a geo module, which answers open questions 1, 2, 3 and 5 of the previous revision. Section 2.11 lists the other changes a finalize step makes to a response, and section 4 the tests. | +| 2026-10-17 | Axum and Spin refuse `[geo] module = "platform"` at startup, with a test on each, and the integration tests start the Axum dev server without it (sections 2.5, 2.6, 2.9 and 4). Section 2.5 says Cloudflare's region needs the `cf-region-code` header. | +| 2026-10-18 | Every adapter runs the request filters and calls `detect`, from `2026-10-17-request-filters-on-every-adapter-design.md` and its second review round. Section 1 no longer calls the Fastly adapter the only one that calls `detect`, and section 2.11 says every adapter writes a filter's response headers, `private, no-store` and the late cache guards. | ## Review log From a02fa316bcdc16d3ccab3bc889f895f855e6291f Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 07:55:47 +0100 Subject: [PATCH 156/192] The request filters design answers its second review round Every named route in the request cycle is shown to run the filters on Axum, Cloudflare and Spin, the design names what a device module is shown and what an unreadable Cookie value loses, gives a true reason no shipped filter is tested through an adapter, names the two DataDome designs whose scope this lifts, and its review log records round 2. --- ...request-filters-on-every-adapter-design.md | 39 +++++++++++-------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md index 1cbbd87c75..f762bd27e4 100644 --- a/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md @@ -2,7 +2,7 @@ **Status:** Proposed, 2026-10-17. A pull request above `ts config migrate`, not yet raised, for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-10-14-51degrees-client-hints-design.md`, whose sections on the adapters this change amends, and the design documents of the stack in IABTechLab/trusted-server#1342 +**Related specs:** `2026-10-14-51degrees-client-hints-design.md` and `2026-10-15-geo-module-request-design.md`, whose statements about the adapters this change amends, `2026-06-11-datadome-server-side-protection-design.md` and `2026-08-03-datadome-ip-excluded-client-tag-design.md`, which describe DataDome's protection filter on the Fastly adapter alone, a scope this change lifts (section 1), and the design documents of the stack in IABTechLab/trusted-server#1342 **Related PRs:** none yet **Issues:** none **Last updated:** 2026-10-18 @@ -17,7 +17,9 @@ The Fastly adapter runs three steps that no other adapter runs. 2. **Device classification.** For every request, before routing, the entry point's `derive_device_signals` in `src/lib.rs` calls `DeviceModule::detect` on the device module a registration supplies, or on the one `build_device_module` builds where none does, and `build_ec_request_state` sets the result on the Edge Cookie context. A module call made with that context's state carries the classification (`ModuleContext::with_request_state` in core), which covers a module route's call and a serve phase preparation. A request filter is never handed it, because `RequestFilterInput` carries only the location and the permissions, and the fetch phase of a document that may be stored is handed services with nothing of the reader in place of the context's state (`preparation_context` in `crates/trusted-server-core/src/publisher.rs`). 3. **robots.txt first.** `dispatch_fallback` answers robots.txt once the preparers have run and before the Edge Cookie steps and the filters, with an empty Edge Cookie context, so no filter's answer stands in for the file and the crawler's Edge Cookie and consent signals are not read. The entry point has already run the device module by then, so a device module a registration supplies is shown the crawler's cookies. -No source file in `crates/trusted-server-adapter-axum`, `-cloudflare` or `-spin` calls `filter_request` or `DeviceModule::detect`. So on those adapters the 51Degrees client hints middleware asks its service nothing and writes no header or tag, DataDome's protection filter never runs, and robots.txt is answered through the publisher path with the crawler's own Edge Cookie context. A device module `[device]` selects from a registration is not idle there, because each adapter's `services_for_request` puts it on the request's services and core asks it for the bid request's device attributes in every auction, showing it the request's headers and permissions (`attach_device_attributes` in `crates/trusted-server-core/src/auction/formats.rs`, called from `auction/endpoints.rs` and `publisher.rs`). It never classifies a request, so no module call carries a classification. On Spin the discovery, signature verification and first-party routes and the page-bids preflight also skip the modules' preparers, which every other adapter runs for them. +No source file in `crates/trusted-server-adapter-axum`, `-cloudflare` or `-spin` calls `filter_request` or `DeviceModule::detect`. So on those adapters the 51Degrees client hints middleware asks its service nothing and writes no header or tag, DataDome's protection filter never runs, and robots.txt is answered through the publisher path with the crawler's own Edge Cookie context. A device module `[device]` selects from a registration is not idle there, because each adapter's `services_for_request` puts it on the request's services and core asks it for the bid request's device attributes in every auction, showing it the request's headers, IP address and address and the permissions (`attach_device_attributes` in `crates/trusted-server-core/src/auction/formats.rs`, called from `auction/endpoints.rs` and `publisher.rs`). It never classifies a request, so no module call carries a classification. On Spin the discovery, signature verification and first-party routes and the page-bids preflight also skip the modules' preparers, which every other adapter runs for them. + +The two DataDome designs describe the protection filter as run by the Fastly adapter alone. `2026-06-11-datadome-server-side-protection-design.md` wires request filters into the Fastly adapter, and `2026-08-03-datadome-ip-excluded-client-tag-design.md` keeps its client tag suppression to the Fastly adapter and lists a change to the other adapters' request filter wiring among its non-goals, because only the Fastly adapter ran the filter. This change lifts that scope, so the protection and its tag suppression run on Axum, Cloudflare and Spin as well. ## 2. Design @@ -46,7 +48,7 @@ Each adapter carries its own copy of the steps. `derive_device_signals`, `run_pr On Spin the discovery, signature verification and first-party routes and the page-bids preflight now run the preparers, because the cycle starts with them. Before this change they ran none. -**Device classification.** `derive_device_signals` follows the Fastly adapter's function of the same name. A device module a registration supplies is shown the `User-Agent` and the `Cookie` header, and the built-in module the `User-Agent` alone, each in a module context carrying the request, that evidence, the settings and the services. Four things differ from the Fastly adapter. +**Device classification.** `derive_device_signals` follows the Fastly adapter's function of the same name. The evidence a module is shown holds the client's IP address and the `User-Agent`, with the `Cookie` header as well for a device module a registration supplies, in a module context that also carries the request's method, scheme, host, path and query, the settings and the services, as on Fastly. Four things differ from the Fastly adapter. 1. It runs where the adapter builds the Edge Cookie context, after the preparers, where the Fastly adapter runs it at the entry point before routing. A preparer that removes a reserved cookie therefore removes it before a device module sees it. 2. It runs only for the routes in the cycle, so a request for robots.txt, an inspection endpoint or an attestation page is not classified, where the Fastly entry point classifies every request. On Fastly a device module a registration supplies is therefore shown the cookies a crawler sends with a request for robots.txt, and on these adapters it is not. @@ -55,7 +57,7 @@ On Spin the discovery, signature verification and first-party routes and the pag As on Fastly, a module route's call and a serve phase preparation carry the classification, a request filter is not handed it, and the fetch phase of a document that may be stored is handed services with nothing of the reader. -**Response effects.** A filter that lets the request continue can name response headers. The route's response carries them in its extensions as `RequestFilterEffects`, and `FinalizeResponseMiddleware` applies them after it writes the finalize headers, so a filter's header is written after an operator's `[response_headers]`, as on Fastly. `apply_terminal_response_effects` then enforces `no-store, private` on a response core marked `TerminalPrivateResponse` and runs `enforce_set_cookie_cache_privacy` and `enforce_uncacheable_cache_privacy`, the steps the Fastly entry point runs before it sends every response. Those steps run on every response, whether or not a filter named a header, so a response core marked private also loses any `ETag`, `Last-Modified`, `Expires` or `Age` that `[response_headers]` writes, as on Fastly. A filter's header is also written after the `X-Robots-Tag` that `[robots-txt]` writes, and `x-robots-tag` is not among the headers a filter is refused (`is_forbidden_filter_header` in `crates/trusted-server-core/src/integrations/registry.rs`), so a filter that names that header replaces the tag, on Fastly as on these adapters. The open questions ask whether a filter should be refused `X-Robots-Tag`. +**Response effects.** A filter that lets the request continue can name response headers. The route's response carries them in its extensions as `RequestFilterEffects`, and `FinalizeResponseMiddleware` applies them after it writes the finalize headers, so a filter's header is written after an operator's `[response_headers]`, as on Fastly. `apply_terminal_response_effects` then enforces `no-store, private` on a response core marked `TerminalPrivateResponse` and runs `enforce_set_cookie_cache_privacy` and `enforce_uncacheable_cache_privacy`, the steps the Fastly entry point runs before it sends every response. Those steps run on every response, whether or not a filter named a header, so a response core marked private also loses any `ETag`, `Last-Modified`, `Expires` or `Age` that `[response_headers]` writes, as on Fastly. A filter's header is also written after the `X-Robots-Tag` that `[robots-txt]` writes, and `x-robots-tag` is not among the headers a filter is refused (`is_forbidden_filter_header` in `crates/trusted-server-core/src/integrations/registry.rs`), so a filter that names that header replaces the tag, on Fastly as on these adapters. The open questions ask whether a filter should be refused `X-Robots-Tag`. The guards always remove `Surrogate-Control` and `CDN-Cache-Control` from a private response, but remove a shared cache header an adapter declares for its platform only where core wrote the policy in that header on the same response (`remove_owned_cache_headers` in `crates/trusted-server-core/src/cache_policy.rs`). A filter that names such a header on a private answer therefore keeps it, being `cloudflare-cdn-cache-control` on Cloudflare, where core writes its policy in that header, and `fastly-surrogate-control` on Fastly, while Axum and Spin declare none. This change does not establish whether a platform then stores such an answer, and the open questions ask whether the guards should remove a declared header whoever wrote it. **Configuration.** No setting is added, removed or given a new default. Three settings act on these adapters where they did not, being `[device] module` when it selects a device module a registration supplies, which now classifies each request in the cycle, and DataDome's `enable_protection` and `[shared.51degrees]`, whose filters now run. Two settings these adapters accept are not applied, being DataDome's `timeout_ms`, default `1500`, and `[shared.51degrees] timeout_ms`, default `500`, which only the Fastly adapter applies, as the timeouts of the backend each module registers for its call. No adapter warns about either at startup, the way the Axum adapter warns about a `proxy.certificate_check` it cannot honor, because an adapter names no vendor's setting, and no platform declaration tells core or a module that an adapter does not apply a backend's timeouts. The DataDome and 51Degrees guides say where `timeout_ms` applies. @@ -63,25 +65,26 @@ As on Fastly, a module route's call and a serve phase preparation carry the clas - The 51Degrees filter sends the reader's `User-Agent` and `Sec-CH-UA` hints to the 51Degrees service for a page navigation an entry naming its middleware covers. The call goes through the answer cache the 51Degrees modules share, so the service is asked once for each set of those values while an answer is held, and not at all for a request with no `User-Agent` (`crates/shared/51degrees/src/hints_engine.rs`). That is the behavior `2026-10-14-51degrees-client-hints-design.md` describes, and its open question, whether the call needs a permission, stays open, and now concerns four adapters where it concerned one. - DataDome's protection filter, which DataDome registers only when `enable_protection` is on, removes `x-ts-datadome-bypass` from every request it runs for, before its other checks. For each request it protects, it sends DataDome's Protection API the server side key, the reader's IP address, the request's method, host, path and query, its `User-Agent`, `Accept`, `Accept-Language`, `Accept-Encoding`, `Referer`, `Origin`, `Via`, `X-Forwarded-For`, `True-Client-IP` and `X-Real-IP` headers among others, its `Sec-CH-UA` and `Sec-Fetch` headers, the list of its header names, the lengths of its `Cookie` and `Authorization` headers, and the client id from the `datadome` cookie or the `x-datadome-clientid` header (`build_protection_payload` in `crates/bot-protection/datadome/src/protection.rs`). The filter does not read the permissions. A response the API allows, or a challenge, can carry DataDome's `Set-Cookie`, which sets a cookie on the publisher's domain and keeps the response out of shared caches, as `private, max-age=0` where the response's own policy allowed them (`enforce_set_cookie_cache_privacy`). A reader an IP rule excludes is not checked, and the request is marked personalized, so an HTML page it asks for is served without DataDome's tag and sent `private, no-store`. -- A device module a registration supplies is shown the reader's `User-Agent` and cookies to classify every request in the cycle, with no permission check, because classification runs before the permissions are resolved and startup refuses a supplied device module that declares a permission (`resolve_device_module` in `crates/trusted-server-core/src/integrations/registry.rs`). That is the Fastly adapter's behavior. Before this change these adapters showed the module the request's headers only in an auction, through `advertising_attributes`, which is handed the permissions. +- A device module a registration supplies is shown the reader's IP address, `User-Agent` and cookies, and the request's method, scheme, host, path and query, to classify every request in the cycle, with no permission check, because classification runs before the permissions are resolved and startup refuses a supplied device module that declares a permission (`resolve_device_module` in `crates/trusted-server-core/src/integrations/registry.rs`). That is the Fastly adapter's behavior. Before this change these adapters showed the module the request's headers, IP address and address only in an auction, through `advertising_attributes`, which is handed the permissions. - robots.txt is answered with an empty Edge Cookie context, so the crawler's Edge Cookie and consent signals are not read and no device module is shown its cookies, where before this change these adapters read the crawler's own context. The publisher path still parses the crawler's `Cookie` header, as it does for every request it handles (`handle_publisher_request`), and sends it on to an origin that is not signed. -**Failure handling.** A filter that fails answers the request with the error, logged as `Failed to run integration request filters`, as on Fastly. An Edge Cookie context that cannot be read answers the request with the error before the filters run, which now covers every route in the cycle, including tsjs, discovery, signature verification, the first-party routes and the page-bids preflight. The context has two ways to fail, being a selected Edge Cookie module the request cannot build, which startup refuses where the adapter can never build it, and a `Cookie` header it cannot read as text, which answers `400`, because `handle_request_cookies` in `crates/trusted-server-core/src/cookies.rs` reads the header with `HeaderValue::to_str`, which refuses any byte outside visible ASCII, an accented letter in UTF-8 among them. A client's own `Cookie` header never fails that way, because the preparer of the `ad-tag.google.diagnostics` module, which every adapter links and runs for every request the cycle handles, first rebuilds the header from the values it can read (`sanitize_console_cookie` in `crates/ad-tag/google/src/diagnostics.rs`), so only a preparer of a deployment's own module, which runs after it, can leave such a header. A filter's outbound call is bounded only by the adapter's HTTP client, which applies no module's own timeout on these adapters. Axum's client allows 5 seconds to connect and 30 in all, Cloudflare leaves the call to the Workers runtime's limit, and Spin to the Spin runtime's default (`src/platform.rs` of each adapter). A filter that lets the request through when its call fails, as DataDome's and the 51Degrees filter do, therefore delays the request by up to that bound rather than failing it. +**Failure handling.** A filter that fails answers the request with the error, logged as `Failed to run integration request filters`, as on Fastly. An Edge Cookie context that cannot be read answers the request with the error before the filters run, which now covers every route in the cycle, including tsjs, discovery, signature verification, the first-party routes and the page-bids preflight. The context has two ways to fail, being a selected Edge Cookie module the request cannot build, which startup refuses where the adapter can never build it, and a `Cookie` header it cannot read as text, which answers `400`, because `handle_request_cookies` in `crates/trusted-server-core/src/cookies.rs` reads the header with `HeaderValue::to_str`, which refuses any byte outside visible ASCII, an accented letter in UTF-8 among them. A client's own `Cookie` header never fails that way, because the preparer of the `ad-tag.google.diagnostics` module, which every adapter links and runs for every request the cycle handles, first rebuilds the header from the header values it can read as text (`sanitize_console_cookie` in `crates/ad-tag/google/src/diagnostics.rs`), so only a preparer of a deployment's own module, which runs after it, can leave such a header. The preparer drops a value it cannot read whole rather than repairing it, so one accented cookie removes every cookie sent in the same header value for that request, the Edge Cookie and the consent cookies among them. That held already wherever the preparers ran, and is new on Spin's discovery, signature verification and first-party routes and its page-bids preflight, which ran none before this change. A filter's outbound call is bounded only by the adapter's HTTP client, which applies no module's own timeout on these adapters. Axum's client allows 5 seconds to connect and 30 in all, Cloudflare leaves the call to the Workers runtime's limit, and Spin to the Spin runtime's default (`src/platform.rs` of each adapter). A filter that lets the request through when its call fails, as DataDome's and the 51Degrees filter do, therefore delays the request by up to that bound rather than failing it. **Rollout.** -- A deployment whose modules register no filter, and that selects no device module a registration supplies, answers as before, apart from three things. It reads the Edge Cookie context, with its geo lookup where `[geo]` selects a module, and classifies the device with the built-in module, on every route in the cycle, where it read the context for the auction, page-bids, the module routes and the publisher alone. And a response core marked private loses any `ETag`, `Last-Modified`, `Expires` or `Age` that `[response_headers]` writes, as Response effects says. On Spin, discovery, signature verification, the first-party routes and the page-bids preflight run the modules' preparers, as on the other adapters. A client's `Cookie` header brings no new `400`, as Failure handling says. -- A deployment that selects a device module a registration supplies has that module asked to classify every request in the cycle, being tsjs, discovery, `/verify-signature`, the first-party routes, the auction, page-bids and its preflight, the module routes and the publisher's pages, each time shown the reader's `User-Agent` and cookies, beside the bid request attributes each auction already asks it for. A module that calls a service to classify makes that call for each of those requests. +- A deployment whose modules register no filter, and that selects no device module a registration supplies, answers as before, apart from three things. It reads the Edge Cookie context, with its geo lookup where `[geo]` selects a module, and classifies the device with the built-in module, on every route in the cycle, where it read the context for the auction, page-bids, the module routes and the publisher alone. And a response core marked private loses any `ETag`, `Last-Modified`, `Expires` or `Age` that `[response_headers]` writes, as Response effects says. On Spin, discovery, signature verification, the first-party routes and the page-bids preflight run the modules' preparers, as on the other adapters, so a `Cookie` header value that cannot be read as text is dropped there as well. A client's `Cookie` header brings no new `400`, as Failure handling says. +- A deployment that selects a device module a registration supplies has that module asked to classify every request in the cycle, being tsjs, discovery, `/verify-signature`, the first-party routes, the auction, page-bids and its preflight, the module routes and every request the publisher fallback proxies, the publisher's pages and each image, script, stylesheet and other subresource among them, each time shown the reader's IP address, `User-Agent` and cookies and the request's address, beside the bid request attributes each auction already asks it for. A module that calls a service to classify makes that call for each of those requests. - A deployment with `[shared.51degrees]` and a `[[serve]]` entry naming `51degrees.client_hints` now has the headers and the tag written on the pages the entry covers, and such a page keeps to the origin path and is sent `private, no-store`, because the filter leaves the hint names in the request's state. - A deployment with DataDome's protection on now has each request it protects checked by the Protection API before the route, being the publisher's pages and each subresource the static asset rule does not exclude, `/auction` and `GET` on both page-bids paths, none of which is among DataDome's internal paths (`is_internal_path` in `crates/bot-protection/datadome/src/protection.rs`). A challenge then answers in place of a page or an auction call. A response that carries DataDome's cookie is kept out of shared caches, and a reader an IP rule excludes gets pages that keep to the origin path and are sent `private, no-store`. The call has no time limit of DataDome's own on these adapters, so a slow Protection API delays each protected request until the adapter's HTTP client gives up, which is 30 seconds on Axum, before DataDome lets it through. **Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, through the adapter's router built by its composition root, with a stub origin: - `a_filter_that_answers_stands_in_for_the_page`: the answer and its effect header are the response, and the origin is not asked. +- `a_filter_that_answers_stands_in_for_every_named_route_in_the_cycle`: with a filter that answers every request, discovery, `POST /verify-signature`, `POST /auction`, `GET` and `OPTIONS` on both page-bids paths and the four first-party paths on each method they answer are each the filter's answer with its effect header, and nothing is sent on. - `a_filter_s_request_header_reaches_the_route_and_its_effect_is_written_last`: a module route sees the request header a filter added, and the filter's response header replaces the operator's header of the same name. - `the_device_module_s_classification_reaches_the_module_call`: the module `[device]` selects classifies the request, shown its cookies, and a module route's call carries that classification. - `platform_classifies_the_request_with_the_built_in_module`: with `[device] module = "platform"`, a module route's call carries the built-in module's classification of an iPhone `User-Agent`. -- `an_edge_cookie_context_that_cannot_be_read_answers_before_the_filters`: with a filter that answers every request, a request a module's preparer leaves with an unreadable `Cookie` header is answered `400`, with no filter's effect and no request to the origin, and a client's own unreadable `Cookie` header reaches the filter instead. +- `an_edge_cookie_context_that_cannot_be_read_answers_before_the_filters`: with a filter that answers every request, a page and the discovery route, each of which a module's preparer leaves with an unreadable `Cookie` header, are answered `400`, with no filter's effect and no request to the origin, and a request a client sends with an unreadable `Cookie` header of its own reaches the filter instead. - `robots_txt_is_answered_before_the_request_filters`: with a filter that answers every request, robots.txt is still the file, and the filter's header is absent. - `robots_txt_shows_the_device_module_none_of_the_crawler_s_cookies`: the module `[device]` selects, which records the cookies it is shown, never sees the cookies sent with robots.txt, and does see those sent with a module route's request. - `the_inspection_endpoints_and_the_attestation_pages_skip_the_request_filters`: with a filter that answers every request, the five inspection paths and the two attestation pages are answered by their own handlers with no filter's effect, and a page is the filter's answer. @@ -90,7 +93,7 @@ As on Fastly, a module route's call and a serve phase preparation carry the clas - `a_filter_cannot_add_a_shared_cache_header_to_a_private_response`: a `Surrogate-Control` that a filter's effect adds to a `private` answer is removed. - `a_response_core_made_private_loses_the_validators_an_operator_writes`: a module route's answer core marks private, which no filter names a header for, loses the `ETag` and `Last-Modified` that `[response_headers]` writes. -Taking any one of the three privacy guards out of the Axum adapter's `apply_terminal_response_effects` fails that guard's tests alone, and taking all three out of the Cloudflare or the Spin adapter's fails the four privacy tests. Taking out of the Axum adapter the step each of the other new tests covers fails that test alone. +Taking any one of the three privacy guards out of the Axum adapter's `apply_terminal_response_effects` fails that guard's tests alone, and taking all three out of the Cloudflare or the Spin adapter's fails the four privacy tests. On each of the three adapters, taking `run_pre_route_filters` out of `execute_named` fails the named route test alone, and taking `build_ec_context` out of it fails the context test alone, whose discovery request is then the filter's answer and not `400`, as taking the preparers out of Spin's `execute_named` does. In the Axum adapter, taking out the robots.txt early answer fails both robots.txt tests, taking the classification out of `build_ec_context` fails the classification test, the `platform` test and the robots.txt cookie test, which asserts that a request in the cycle shows the module its cookies, and taking out a filter's answer in the fallback fails the answering test, the context test and the inspection test, each of which asserts that a page is the filter's answer. No test runs a shipped filter, the 51Degrees middleware or DataDome's protection, through one of these adapters, and none covers `/health`, which runs no module. @@ -98,6 +101,7 @@ No test runs a shipped filter, the 51Degrees middleware or DataDome's protection 1. Whether the 51Degrees call needs a permission, which `2026-10-14-51degrees-client-hints-design.md` leaves to the reviewer and which now arises on four adapters. 2. Whether a request filter should be refused `X-Robots-Tag`, so that no filter can loosen a site `[robots-txt]` takes out of search, on Fastly as on these adapters. Refusing it is a change to `is_forbidden_filter_header` in core, which this change leaves alone. +3. Whether the guards should remove a shared cache header an adapter declares, `cloudflare-cdn-cache-control` or `fastly-surrogate-control`, from a private response whoever wrote it, or a filter be refused such a header, so that no filter can leave a private answer carrying a policy the platform's own cache reads. Either is a change to core, to `remove_owned_cache_headers` or to `is_forbidden_filter_header`, which this change leaves alone. ## 3. What is not in this change @@ -106,19 +110,20 @@ No test runs a shipped filter, the 51Degrees middleware or DataDome's protection - A time limit for a filter's outbound call on Axum, Cloudflare and Spin. A module's limit reaches the platform only as the `first_byte_timeout` and `between_bytes_timeout` of the `PlatformBackendSpec` it registers, and a request names only its backend (`PlatformHttpRequest`). The Axum backend keeps nothing of those timeouts, naming a backend by its scheme, host, port and discriminator, and its client applies its own two limits, although reqwest's `RequestBuilder::timeout` can bound one request. The Cloudflare backend's name carries the first byte timeout and its client does not read it, although a Worker's fetch can be stopped with `Fetch::send_with_signal` and an `AbortController`. Spin's `spin_sdk::http::send` takes no time limit. Applying a module's limit is a change to each adapter's backend and HTTP client, which every outbound call goes through, the auction's calls to its providers among them, so it is left to a change of its own. - Moving the inspection endpoints ahead of the preparers on Axum and Cloudflare, where the Fastly adapter answers all three of them, being all five paths, before the preparers. - One request cycle in core that every adapter calls, in place of the copy each adapter carries, which would move the Fastly adapter's cycle as well. -- A test that runs a shipped filter through one of these adapters, which needs a scripted DataDome Protection API or 51Degrees service and the secret store their keys are read from. +- A test that runs a shipped filter through one of these adapters. The pieces such a test needs exist, being the stub client these tests use, which can script the Protection API's or the 51Degrees service's answer, DataDome's key, which is a settings value once resolved, and `HashMapSecretStore` in core's test support. This change proves the cycle with filters written for its tests, and each shipped filter's own tests drive it directly against a scripted service, DataDome's with core's stub client and `HashMapSecretStore` and the 51Degrees filter's with `ScriptedHttpClient` in `crates/shared/51degrees/src/test_services.rs`, so a test of one through an adapter is left to a later change. ## 4. Acceptance -The twelve tests pass on each of the three adapters, taking out any of the three privacy guards fails a test, and the existing tests of each adapter and the cross-adapter parity tests pass unchanged. +The thirteen tests pass on each of the three adapters, taking out any of the three privacy guards fails a test, taking `build_ec_context` or `run_pre_route_filters` out of any adapter's `execute_named` fails a test, and the existing tests of each adapter and the cross-adapter parity tests pass unchanged. ## Revision record -| Date | Change | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-17 | First draft. | -| 2026-10-18 | Fixes from review round 1. What a selected device module did on these adapters before, what DataDome's filter sends and sets, how each filter's call is bounded and why no time limit is added, the inspection endpoints the Fastly adapter answers before the preparers, the routes kept out of the cycle, which of a crawler's cookies robots.txt reads, the `Cookie` check and when it is reached, the configuration, eight new tests on each adapter with the three privacy guards among them, the open questions and the review log. | +| Date | Change | +| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-17 | First draft. | +| 2026-10-18 | Fixes from review round 1. What a selected device module did on these adapters before, what DataDome's filter sends and sets, how each filter's call is bounded and why no time limit is added, the inspection endpoints the Fastly adapter answers before the preparers, the routes kept out of the cycle, which of a crawler's cookies robots.txt reads, the `Cookie` check and when it is reached, the configuration, eight new tests on each adapter with the three privacy guards among them, the open questions and the review log. Fixes from review round 2. A test on each adapter that sends every named route in the cycle through the filters, the discovery route in the context test, the tests each removal fails, the IP address and the request's address a device module is shown, the subresources it classifies, what the diagnostics preparer drops, the two DataDome designs whose scope this change lifts, what the guards do with a platform's own cache header with an open question on it, and the reason no shipped filter is tested through an adapter. | ## Review log -1. **Round 1, 2026-10-18.** A review in a fresh session, against `fixpass6/52-request-filters` at `f0edcd72b`. Issues found, five of them blocking. The five were that the spec said a selected device module did nothing on Axum, Cloudflare and Spin, when core asks it for a bid request's device attributes in every auction there, that nothing said what DataDome's filter sends DataDome or sets, or that no permission gates it, that the client hints design still said only the Fastly adapter runs request filters, that no test proved the privacy guards written after a filter's headers, and that the spec said the Fastly adapter answers two of the inspection endpoints before the preparers, when it answers all three, and that the adapters' HTTP clients cannot bound a call, when the Axum and Cloudflare clients can. Fixed in `7aea7a72b`, which adds the four privacy tests and four more on each adapter, `4549047dc`, which corrects the adapters' comments on robots.txt, `a584b071f`, which corrects the guides, the changelog and the middleware readme, `6b4eb0b8c`, which amends the client hints design, and this revision, which states each from the code and gives the reason a time limit is left to a change of its own. One finding was not borne out. The review said a `Cookie` header holding an accented letter now answers `400` on the routes that newly read the context, but a client's header is rebuilt by the `ad-tag.google.diagnostics` preparer first, which the context test shows, so Rollout names no new `400`. +1. **Round 1, 2026-10-18.** A review in a fresh session, against `split/52-request-filters` at `f0edcd72b`. Issues found, five of them blocking. The five were that the spec said a selected device module did nothing on Axum, Cloudflare and Spin, when core asks it for a bid request's device attributes in every auction there, that nothing said what DataDome's filter sends DataDome or sets, or that no permission gates it, that the client hints design still said only the Fastly adapter runs request filters, that no test proved the privacy guards written after a filter's headers, and that the spec said the Fastly adapter answers two of the inspection endpoints before the preparers, when it answers all three, and that the adapters' HTTP clients cannot bound a call, when the Axum and Cloudflare clients can. Fixed in `7aea7a72b`, which adds the four privacy tests and four more on each adapter, `4549047dc`, which corrects the adapters' comments on robots.txt, `a584b071f`, which corrects the guides, the changelog and the middleware readme, `6b4eb0b8c`, which amends the client hints design, and this revision, which states each from the code and gives the reason a time limit is left to a change of its own. One finding was not borne out. The review said a `Cookie` header holding an accented letter now answers `400` on the routes that newly read the context, but a client's header is rebuilt by the `ad-tag.google.diagnostics` preparer first, which the context test shows, so Rollout names no new `400`. +2. **Round 2, 2026-10-18.** A review in a fresh session, against `split/52-request-filters` at `17136d04b`, of the round 1 fixes and the new text. Issues found, one of them blocking. The one was that no test sent a named route through the request cycle, so taking `build_ec_context` or `run_pre_route_filters` out of any adapter's `execute_named` failed no test, although on Cloudflare and Spin this change replaced each named route's own handler with `execute_named`. Fixed in `7ae3235ad`, which adds a test on each adapter that sends every named route in the cycle to a filter that answers and has the context test's preparer spoil the discovery route as well, so that taking either step, or on Spin the preparers, out of `execute_named` fails a test, `8c9b914c7`, which corrects the `build_ec_context` comments on the `Cookie` check, `233299a06`, which names in the integration guide the IP address and the request's address a device module is shown, `52cfd9fe9`, which amends the geo module request design's table of finalize steps and its line on `detect`, `e7417c898` and `ac0492932`, which fix the client hints design's wait on Axum and its test item 6 and log its round, and this revision, which names the tests each removal fails, the IP address and the request's address, the subresources a device module classifies, what the diagnostics preparer drops, the two DataDome designs whose scope this change lifts and what the guards do with a platform's own cache header, on Fastly as on Cloudflare, with an open question on it, and gives a true reason no shipped filter is tested through an adapter. From 4359837b8944536b3fd5b7556309fd81bccec26f Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 07:55:49 +0100 Subject: [PATCH 157/192] The client hints design logs its third review round The wait on Axum when the service cannot be reached and the targets its tests skip are corrected, and the review log records the review of the change that has every adapter run the filter. --- ...026-10-14-51degrees-client-hints-design.md | 33 +++++++++++-------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md index 2fcae80f29..1fc595b5e0 100644 --- a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md +++ b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md @@ -211,10 +211,13 @@ hints is held like any other, for up to an hour. On Fastly a call that cannot connect fails after one second, and one that connects waits `timeout_ms` for the first byte and for each gap between bytes, so while the service is down every page navigation an entry covers with no held answer waits for those -limits before its page is built. On Axum, Cloudflare and Spin the call waits -until the adapter's HTTP client gives up, which is 30 seconds on Axum and the -runtime's own limit on Cloudflare and Spin, so while the service is down each -such navigation waits that long. +limits before its page is built. On Axum, Cloudflare and Spin only the +adapter's HTTP client bounds the call. On Axum a service that cannot be +reached fails within the client's 5 second connection limit, a refused +connection fails at once, and a service that accepts the connection and then +stalls holds the call for up to 30 seconds in all. On Cloudflare and Spin the +runtime's own limit bounds it. So while the service is slow or cannot be +reached, each such navigation waits up to that bound. **Rollout.** A deployment without `[shared.51degrees]` is unchanged. On every adapter, adding the table and an entry naming the middleware changes the @@ -299,20 +302,24 @@ Six things have no test. `permissions: None`, so nothing shows what the filter does with permission state. 6. The two crates' tests run natively under `cargo test-axum`, and their lints - under `cargo clippy-axum`. No alias for the Fastly target lists them, so - they do not run for `wasm32-wasip1`, the Fastly adapter's target. + under `cargo clippy-axum`. No alias runs their tests for an adapter's + WebAssembly target, being `wasm32-wasip1` for Fastly and Spin and + `wasm32-unknown-unknown` for Cloudflare, because the Fastly aliases do not + list them and `cargo test-cloudflare` and `cargo test-spin` test the + adapter crate alone, on the host. ## Revision record -| Date | Change | -| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-14 | First draft. | -| 2026-10-15 | Fixes from review round 1. What the filter runs on and that only the Fastly adapter runs it, the cost to pages, what each navigation sends and the open questions on consent, how each adapter bounds the call, the answers on failure, the settings, what the headers keep, the definition of a page navigation, the untested paths, the related specs and the review log. | -| 2026-10-15 | Fixes from review round 2. The pull request that holds the design documents, the wait for the first call in the guide, and the code comments left for a later change. | -| 2026-10-15 | Code fixes for the call's scope, what it sends and header merging. The permission question stays open, with what was found, and the answer lifetime is a new open question. | -| 2026-10-18 | Every adapter runs the filter, from `2026-10-17-request-filters-on-every-adapter-design.md` and its first review round. The table of parts, the builder, the `timeout_ms` setting, what bounds the call on each adapter, consent, failure, rollout, section 3 and the tests say so. | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-14 | First draft. | +| 2026-10-15 | Fixes from review round 1. What the filter runs on and that only the Fastly adapter runs it, the cost to pages, what each navigation sends and the open questions on consent, how each adapter bounds the call, the answers on failure, the settings, what the headers keep, the definition of a page navigation, the untested paths, the related specs and the review log. | +| 2026-10-15 | Fixes from review round 2. The pull request that holds the design documents, the wait for the first call in the guide, and the code comments left for a later change. | +| 2026-10-15 | Code fixes for the call's scope, what it sends and header merging. The permission question stays open, with what was found, and the answer lifetime is a new open question. | +| 2026-10-18 | Every adapter runs the filter, from `2026-10-17-request-filters-on-every-adapter-design.md` and its first review round. The table of parts, the builder, the `timeout_ms` setting, what bounds the call on each adapter, consent, failure, rollout, section 3 and the tests say so. Fixes from review round 3, the wait on Axum for a service that cannot be reached and test item 6. | ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/44-client-hints` at `1f6b83d0c`. Issues found, four of them blocking. The four were that the request filter runs wherever the table is present and not only where an entry covers the page, that the cost to existing pages was not stated, that failure handling and the timeout of each adapter were not stated, and that nothing said what each navigation sends to the service or what consent it needs. This revision states each from the code, which also shows that only the Fastly adapter runs request filters, and it corrects the claims about page navigations, the headers and the tests, replaces a related spec that exists in no branch, and corrects the guide, the changelog and the middleware readme. 2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `126347b00`, of the round 1 fixes and the new text. No blocking issues once the pull request number was corrected. This revision names the pull request that now holds every design document, makes the guide's description of the wait for the first call match the one second connection limit and the limit on each gap between bytes, and lists in section 3 the code comments that still say more than the design does. +3. **Round 3, 2026-10-18.** A review in a fresh session, against `split/52-request-filters` at `17136d04b`, of the change that has every adapter run the filter, made with the request filters design's first round. Issues found, none blocking. The two were that When it fails gave 30 seconds on Axum as the wait for a service that cannot be reached, which fails within the client's 5 second connection limit, and that test item 6 named only the Fastly adapter's target, when no alias runs the two crates' tests for the WebAssembly target of any adapter, all of which now run the filter. Fixed in `e7417c898`. From f9a6f2e60924b418d001bd4aa18cb615d121bfb8 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 07:57:32 +0100 Subject: [PATCH 158/192] The RSC Flight middleware design matches its branch The specs copy now carries the 16 October fix, in which the rewriter moves each address once in one pass, with the tests that show it. --- .../specs/2026-10-14-rsc-flight-middleware-design.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md b/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md index 630be6452c..de7c66b4a7 100644 --- a/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md +++ b/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md @@ -48,7 +48,7 @@ middleware = ["framework.nextjs.flight"] **What the middleware is told.** The origin host, the reader's host and the reader's scheme from its context, which is what the module's HTML middleware works from too. The origin host is the `Host` the source sends, being the source's host and port unless the source sets another `Host`. A page names itself by that `Host`, so core tells every middleware that name as the origin host, and a middleware is told no source address. -**What the middleware moves.** From the origin host, the middleware moves an `https://` or `http://` address to the reader's scheme and host, a protocol relative address to the reader's host, and a bare host at hostname boundaries to the reader's host. What the middleware moves is not exactly what core's plain replacement moves, and not exactly what `rsc_flight` moved, the rewriter in core that this change moves out. Core's replacement and `rsc_flight` build the `https://` and `http://` forms from the source's address as written, where the middleware builds them from the host alone, and `rsc_flight` already respected hostname boundaries for a bare host, as the middleware does. The middleware differs from core's replacement in four ways and from `rsc_flight` in the first three. +**What the middleware moves.** From the origin host, the middleware moves an `https://` or `http://` address to the reader's scheme and host, a protocol relative address to the reader's host, and a bare host at hostname boundaries to the reader's host. It moves each address once, in one pass from the start, where `rsc_flight` read each replacement's output again and wrote a request host that begins with the origin host more than once. What the middleware moves is not exactly what core's plain replacement moves, and not exactly what `rsc_flight` moved, the rewriter in core that this change moves out. Core's replacement and `rsc_flight` build the `https://` and `http://` forms from the source's address as written, where the middleware builds them from the host alone, and `rsc_flight` already respected hostname boundaries for a bare host, as the middleware does. The middleware differs from core's replacement in four ways and from `rsc_flight` in the first three. 1. **A folder in the source's address.** For a source at `https://origin.example.com/shop`, core's replacement and `rsc_flight` turn `https://origin.example.com/shop/item` into `https://proxy.example.com/item`. The middleware turns the same text into `https://proxy.example.com/shop/item`, so the folder stays in a rewritten address, as it does in the links of a page. 2. **A `Host` that differs from the address.** Where the source sends another `Host` than its address names, core's replacement and `rsc_flight` look for the address as written and for the `Host`, and the middleware looks for the `Host` alone. An address in a Flight response that names only the source's own address is therefore no longer moved. @@ -78,6 +78,7 @@ The middleware builds from the host because it works from what its context carri ## 4. Tests - The rewriter moves an address in a newline row, moves an `http://` address to the reader's scheme, writes a `T` row's length again, handles a `T` row split across chunks, and leaves an unrelated bare hostname alone. +- The rewriter writes a request host that begins with the origin host once, as a `.localhost` name or with a port, in a `T` row and a newline row over both schemes, writes a hyphenated host and a `www.` host over an apex origin as before, and moves each address once from the start, leaving a row alone when the origin host is empty. - The middleware handles `text/x-component` alone, runs in the fetch phase, says it moves the origin's address itself, and its plan carries one stream processor that rewrites a Flight body. - The module registers the middleware, core leaves `text/x-component` to it and moves the addresses in every other body, a Flight response takes the entry naming it and runs nothing where no entry covers it, and an entry for HTML naming it is refused by name. - In core, a body whose registered middleware moves the address itself is handed to that middleware as the origin sent it, on the buffered path and the streamed path. The same body passes through untouched where no entry names the middleware, tested on the buffered path alone. A `text/plain` body still has its addresses moved before the entry's middleware runs. @@ -100,6 +101,7 @@ Seven cases have no test. | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The differences from core's plain replacement and why, the alternatives to the new method, adapters, configuration, consent, failure, rollout with its open question, the cases no test covers and the review log. | | 2026-10-15 | Fixes from review round 2. The hostname boundary difference for a bare host alone, the `path` paragraph, the template comment and the review log. | +| 2026-10-16 | The rewriter moves each address once, in one pass, and the tests that show it. | ## Review log From b19f1612def52b1d9201787c813483db4a926e63 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 07:57:34 +0100 Subject: [PATCH 159/192] The cookies by path design matches its branch The specs copy now carries the 16 October code fixes for the rollout and privacy finding of review round 1, in which a withdrawal on an addressed request also expires a pull-sync marker under the root path and tombstones the row of each Edge Cookie the request carried, and the two open questions they raise. --- .../2026-10-15-cookies-by-path-design.md | 20 +++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md b/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md index 15d22c0540..94d891d4cc 100644 --- a/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md +++ b/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md @@ -78,46 +78,49 @@ Consent cookies are written by a consent tool in the browser, not by core, and e **Configuration.** None. The path comes from the request, and a deployment that addresses no request by path writes `Path=/` as today. No setting is added or changed. -**Consent and privacy.** Writing core's own cookies under a publisher's path stops the browser sending a publisher's Edge Cookie, resolved marker, pull-sync marker and tester cookie on requests under another publisher's path, so another publisher's origin does not receive the ones written after this change. The change does not isolate consent, because core writes no consent cookie and each consent cookie keeps the `Path` its tool gave it, as above. `Path` is also no security boundary, because every publisher on the host is one origin to the browser. RFC 6265 section 8.5 says a resource from one path might be able to access cookies stored for another, and section 8.6 says the `Path` attribute gives no integrity protection, because a response can set a cookie with any `Path`. A withdrawal expires the Edge Cookie, the resolved marker and the pull-sync marker under the request's removed prefix, the path they were written under, so the expiry reaches the cookies the browser holds for that publisher. `expire_ec_cookie`, which core calls only on a withdrawal, also expires the Edge Cookie and the resolved marker under `/` when the prefix carries a path, so a root Edge Cookie written before this change does not survive the withdrawal. A cookie under `/` is one cookie for every publisher on the host that uses its name, so that expiry reaches each of them, and publishers that keep core's own public prefix, `ts`, all use the name `ts-ec`. The pull-sync marker is expired under the removed prefix only, and a marker under `/` lasts at most an hour (`MARKER_MAX_AGE_SECS` in `ec/pull_sync_marker.rs`). +**Consent and privacy.** Writing core's own cookies under a publisher's path stops the browser sending a publisher's Edge Cookie, resolved marker, pull-sync marker and tester cookie on requests under another publisher's path, so another publisher's origin does not receive the ones written after this change. The change does not isolate consent, because core writes no consent cookie and each consent cookie keeps the `Path` its tool gave it, as above. `Path` is also no security boundary, because every publisher on the host is one origin to the browser. RFC 6265 section 8.5 says a resource from one path might be able to access cookies stored for another, and section 8.6 says the `Path` attribute gives no integrity protection, because a response can set a cookie with any `Path`. A withdrawal expires the Edge Cookie, the resolved marker and the pull-sync marker under the request's removed prefix, the path they were written under, so the expiry reaches the cookies the browser holds for that publisher. When the prefix carries a path, the withdrawal also expires each of the three under `/`, the Edge Cookie and the resolved marker in `expire_ec_cookie`, which core calls only on a withdrawal, and the pull-sync marker in `ec_finalize_response`, so a root cookie written before this change, or by a request addressed by host, does not survive the withdrawal. A cookie under `/` is one cookie for every publisher on the host that uses its name, so that expiry reaches each of them, and publishers that keep core's own public prefix, `ts`, all use the name `ts-ec`. A browser that holds an Edge Cookie under the publisher's path and one of another value under `/` sends both, and the jar keeps one, so on a request under a removed prefix the withdrawal tombstones the identity-graph row of each Edge Cookie the request carried, as it does the row of the one the jar kept (`withdrawal_kv_keys` in `ec/finalize.rs`). A withdrawal on a request addressed by host, such as a call the client script makes at the root of the host, expires only the cookies under `/`, because the browser does not send that request the cookies under a publisher's path and core does not know the path. The cookies under the publisher's path are expired on the next request under that path that carries the withdrawal. **Failure handling.** A removed prefix that is not a cookie path is kept off a cookie by `under_path`, as above, and the request is addressed as its hook answered. A `Set-Cookie` value that fails to render is skipped with a warning, as today, except on the tester routes, which return `TrustedServerError::InvalidHeaderValue`. -**Rollout.** A request with no prefix writes `Path=/` and reads its jar as today, so a deployment that supplies no addressing hook, or one whose hook answers "by host", is unchanged. On a deployment already addressing by path, core writes a cookie under the publisher's path only in a response where it writes that cookie at all. For the Edge Cookie that is when core creates an identifier for a visitor who has none, or replaces an orphaned one, which is an identifier whose row the identity graph has lost. A returning visitor's response writes no Edge Cookie. The root Edge Cookie and resolved marker written before this change therefore stay in that visitor's browser for up to a year (`COOKIE_MAX_AGE` in `ec/cookies.rs`), are still sent to every publisher's pages and origin on the host, and are still read, so the identity continues. Nothing moves that cookie under the publisher's path, and before its `Max-Age` ends only a withdrawal, which expires it under `/` as above, removes it. A browser can hold a root cookie and a cookie under the publisher's path of one name, for example after core writes a new Edge Cookie for a visitor who still has the root one. The jar's first-pair rule then picks the cookie under the publisher's path, provided the browser lists the longer path first. +**Rollout.** A request with no prefix writes `Path=/` and reads its jar as today, so a deployment that supplies no addressing hook, or one whose hook answers "by host", is unchanged. On a deployment already addressing by path, core writes a cookie under the publisher's path only in a response where it writes that cookie at all. For the Edge Cookie that is when core creates an identifier for a visitor who has none, or replaces an orphaned one, which is an identifier whose row the identity graph has lost. A returning visitor's response writes no Edge Cookie. The root Edge Cookie and resolved marker written before this change therefore stay in that visitor's browser for up to a year (`COOKIE_MAX_AGE` in `ec/cookies.rs`), are still sent to every publisher's pages and origin on the host, and are still read, so the identity continues. Nothing moves that cookie under the publisher's path, because a request does not say which path a cookie it carries was written under, and open question 2 sets out what a move would cost. Before its `Max-Age` ends only a withdrawal, which expires it under `/` as above, removes it. A browser can hold a root cookie and a cookie under the publisher's path of one name, for example after core writes a new Edge Cookie for a visitor who still has the root one. The jar's first-pair rule then picks the cookie under the publisher's path, provided the browser lists the longer path first. **Tests.** In `crates/trusted-server-core`: - In `prefix.rs`, a prefix with no path, an empty one or `/` gives `/` and equals the prefix with no path, `for_request` takes the removed prefix, and a prefix holding `;`, a space or non-ASCII, or not starting with `/`, keeps `/`. - In `ec/cookies.rs`, the Edge Cookie, its expiry and the resolved marker carry `Path=/publisher-a.example` under that prefix, and `Path=/` without it. `an_expiry_under_a_removed_prefix_also_expires_the_root_cookies` checks byte for byte that an expiry under that prefix also expires the Edge Cookie and the resolved marker under `/`. - In `ec/pull_sync_marker.rs` and `tester_cookie.rs`, the marker and the tester cookie carry the prefix path. -- In `cookies.rs`, on an addressed request the first of two pairs of one name is read, and with no prefix the last is read, as today, and so is the last with a removed prefix that is empty, `/` or `/a;b`. +- In `cookies.rs`, on an addressed request the first of two pairs of one name is read, and with no prefix the last is read, as today, and so is the last with a removed prefix that is empty, `/` or `/a;b`. `cookie_values_gives_every_value_of_a_name_in_the_order_sent` checks that every pair of one name is returned. - In `evidence.rs`, `RequestInfo::cookie` returns the first of two consent cookies of one name. - In `addressing.rs`, `a_prefix_that_is_not_a_cookie_path_still_addresses_the_request` checks that the hook in the module's rustdoc addresses `/a;b/news` with the selector value `a;b`, the continuing path `/news` and the removed prefix `/a;b`, as before this change, and that core writes that request's cookies under `/`. -- In `ec/mod.rs`, `finalize_writes_the_edge_cookie_under_the_removed_prefix` reads the context from a request with a removed prefix through `EcContext::read_from_request`, creates an identifier, runs `ec_finalize_response` and checks that the Edge Cookie is under the prefix. `a_withdrawal_under_a_removed_prefix_also_expires_the_root_cookies` runs a withdrawal the same way and checks that the Edge Cookie and the resolved marker are expired under the prefix and under `/`, and only under `/` with no prefix. +- In `ec/mod.rs`, `finalize_writes_the_edge_cookie_under_the_removed_prefix` reads the context from a request with a removed prefix through `EcContext::read_from_request`, creates an identifier, runs `ec_finalize_response` and checks that the Edge Cookie is under the prefix. `a_withdrawal_under_a_removed_prefix_also_expires_the_root_cookies` runs a withdrawal the same way and checks that the Edge Cookie and the resolved marker are expired under the prefix and under `/`, and only under `/` with no prefix. `a_withdrawal_reaches_a_returning_visitors_root_edge_cookie` runs a withdrawal for a returning visitor whose only Edge Cookie is under `/`, and checks those four expiries and that the visitor's row is tombstoned. `a_withdrawal_under_a_removed_prefix_also_expires_a_root_pull_sync_marker` checks that a pull-sync marker the request carried is expired under the prefix and under `/`. `a_withdrawal_under_a_removed_prefix_tombstones_each_edge_cookies_row` sends an Edge Cookie under the path and one of another value under `/`, and checks that the rows of both are tombstoned. - In `ec/resolve.rs`, `resolve_writes_under_the_removed_prefix_of_an_addressed_request` reads the context from the resolve request itself, as an adapter does, runs `handle_ec_resolve` and checks that the Edge Cookie and its marker are under the prefix. In `crates/trusted-server-adapter-fastly`, `app.rs` holds `dispatch_set_tester_writes_under_the_removed_prefix`, which sends `/_ts/set-tester` through the router with a removed prefix and checks the cookie's path. The clear route has no addressed test of its own. Two pieces of production code build the prefix the writers use, being `for_request` in `read_from_request_with_geo_status` (`ec/mod.rs`), which all three `read_from_request*` functions of `EcContext` reach, and the use of the context's prefix in `handle_ec_resolve` (`ec/resolve.rs`). Each has a test that fails without it. With the context built from `services.public_prefix().clone()`, `finalize_writes_the_edge_cookie_under_the_removed_prefix` fails with the Edge Cookie at `Path=/`, and with `handle_ec_resolve` writing under `services.public_prefix()`, `resolve_writes_under_the_removed_prefix_of_an_addressed_request` fails with both cookies at `Path=/`. Two other tests pass a removed prefix through a context reader and check no cookie the context writes. In `publisher.rs`, `a_source_answering_per_request_sends_each_request_to_its_own_origin` sends `/a.example` and `/b.example` through `EcContext::read_from_request`, by way of `run_publisher_proxy_with`, and asserts only the origin each request goes to. The Fastly test above reaches `read_from_request_resolving_geo`, because the adapter reads the Edge Cookie context before it runs a tester route, and checks only the tester cookie, whose prefix `run_named_route` builds apart from the context. -**Open questions.** This one waits for a decision. +**Open questions.** These wait for a decision. -1. Whether to correct, with the code, the pages this change makes untrue. The code comments it made untrue are corrected. +1. Whether to correct, with the code, the pages this change makes untrue. The code comments it made untrue, in the rustdoc of `PublicPrefix::for_request` and of `parse_cookies_to_jar_keeping_first` in `cookies.rs`, are corrected. - The request addressing spec (held with every other design document of the stack in IABTechLab/trusted-server#1342) and the section on addressing in `docs/guide/configuration.md` say that core never writes the removed prefix into a response and that publishers on one host share cookies. - The request addressing spec also says that this design "keeps those cookies with the publisher", after naming the consent cookies, `sharedId` and the `Cookie` header forwarded to the origin. This design does not isolate the consent cookies or `sharedId`, and the forwarded header still carries every cookie under `/`. - The description of the Edge Cookie in `docs/guide/edge-cookies.md` says `Path=/`. +2. Whether core should move a root Edge Cookie a returning visitor holds under the publisher's path. A `Cookie` header carries only names and values, and RFC 6265 section 4.2.2 says a server cannot tell from it for which paths a cookie is valid, so core cannot tell a lone root cookie from one under the publisher's path. A move on every returning visitor's response would put a `Set-Cookie` on each of them, and `enforce_set_cookie_cache_privacy` in `response_privacy.rs` makes every response that carries one `private, max-age=0` and removes its edge cache headers, where today a returning visitor's response carries no Edge Cookie. A move made once needs something new to record that it happened, such as a cookie written only under the publisher's path. While the client script calls `/auction` at the root of the host (section 3), a cookie moved under the publisher's path is also no longer sent on that call, so that call's auction would run without the identifier it receives today. Until this is decided a root Edge Cookie stays under `/` until its `Max-Age` ends or a withdrawal expires it, as the rollout above says. +3. Whether the client script's Osano and Sourcepoint mirrors should write the consent cookies under the publisher's path. The Osano mirror writes `us_privacy`, `__gpp`, `__gpp_sid` and `euconsent-v2`, and the Sourcepoint mirror writes `__gpp` and `__gpp_sid`, each with `Path=/` fixed in `integrations/osano/index.ts` and `integrations/sourcepoint/index.ts`, so consent given on one publisher's pages is read on another's. That is a change to the client script, left with its paths for a change of its own (section 3), so this change makes no code change for those cookies. + ## 3. What is not in this change - **Cookies under `/`.** A cookie written under `/`, by an origin, by a module, by a consent tool not set to the publisher's path, or by core before this change, is sent under every publisher's path and forwarded to every origin on the host that is not signed. Core cannot tell which publisher wrote it. A deployment that needs an origin's cookies kept apart rewrites their `Path` with a middleware of its own, as it puts the prefix into links. - **Cookies a module or an integration writes.** A module's own `Set-Cookie` headers pass through core unchanged unless they name a cookie under the public prefix (`reserved_response_effect` in `ec/module.rs`), so their `Path` is the module's choice. The GPT diagnostics cookie `__Host-ts-console`, which `crates/ad-tag/google/src/diagnostics.rs` writes, has to stay `Path=/`, because browsers accept a cookie with the `__Host-` prefix only with `Path=/` (`draft-ietf-httpbis-rfc6265bis`, section 4.1.3.2). - **Nested prefixes.** If one publisher's prefix lies inside another's, such as `/news` and `/news/sport`, the cookies under the outer prefix are also sent on requests under the inner one. - **The client script's paths and cookies.** The script calls paths at the root of the page's host, which carry no publisher part. Its core calls `/auction` (`crates/trusted-server-js/lib/src/core/request.ts`). Of its fourteen integrations, `ec_client_fixed` posts to `/_ts/api/v1/ec/resolve`, `gpt` fetches `/_ts/page-bids` and rewrites its SDK to `/integrations/gpt`, `prebid` defaults its auction endpoint to `/auction`, `creative` calls `/first-party/sign` and `/first-party/proxy-rebuild`, and `aps`, `datadome`, `didomi`, `google_tag_manager`, `lockr`, `permutive` and `sourcepoint` aim their SDK, renderer or API paths at `/integrations//` on the page's origin. Three integrations also write cookies with `Path=/` from the page, being the eids cookie in `prebid` and the consent cookies in `osano` and `sourcepoint`. On a host addressed by path these requests are addressed by host, so the browser no longer sends them the cookies core wrote under the publisher's path, and the cookies core writes in answer to them, such as the resolve endpoint's, take `Path=/`. The cookies the page writes stay shared. The fix is for the server to tell the script the removed prefix in the page, as the server tells the script the public prefix with `data-ts-public-prefix`, and for the script to put the prefix before each path and use it as each cookie's path. That change touches the script's core and twelve of its fourteen integrations, being `aps`, `creative`, `datadome`, `didomi`, `ec_client_fixed`, `google_tag_manager`, `gpt`, `lockr`, `osano`, `permutive`, `prebid` and `sourcepoint`, and is left for a change of its own. -- **Moving root cookies left from before.** Core does not write a root Edge Cookie or resolved marker left from before this change again under the publisher's path, so either stays under `/` until its `Max-Age` ends or a withdrawal expires it. A withdrawal on an addressed request does not expire a pull-sync marker under `/`, which lasts at most an hour. +- **Moving root cookies left from before.** Core does not write a root Edge Cookie or resolved marker left from before this change again under the publisher's path, so either stays under `/` until its `Max-Age` ends or a withdrawal expires it. Open question 2 sets out why and what a move would cost. - **Partitioning other first-party storage.** Local storage and other storage the page script keeps belong to the host, not the path. ## 4. Acceptance -The tests listed under "Tests" in section 2 pass, including `dispatch_set_tester_writes_under_the_removed_prefix` in `crates/trusted-server-adapter-fastly/src/app.rs`, which `cargo test-fastly` runs. `finalize_writes_the_edge_cookie_under_the_removed_prefix` and `resolve_writes_under_the_removed_prefix_of_an_addressed_request` each fail when the production code they cover is reverted, as section 2 says. `cargo test -p trusted-server-core`, `cargo test-fastly`, `cargo clippy -p trusted-server-core --all-targets -- -D warnings`, `cargo clippy-fastly` and `cargo fmt --all -- --check` pass. With no `RemovedPathPrefix` on a request, or one that is empty or `/`, every `Set-Cookie` core writes is as before and the jar keeps the last pair of a name. The existing tests check the unchanged output byte for byte for the Edge Cookie and its expiry, the pull-sync marker and its expiry, and the tester cookie and its clearing, and check the resolved marker and its expiry for their `Path` only. On an addressed request a withdrawal expires the Edge Cookie and the resolved marker under the removed prefix and under `/`, checked byte for byte. +The tests listed under "Tests" in section 2 pass, including `dispatch_set_tester_writes_under_the_removed_prefix` in `crates/trusted-server-adapter-fastly/src/app.rs`, which `cargo test-fastly` runs. `finalize_writes_the_edge_cookie_under_the_removed_prefix` and `resolve_writes_under_the_removed_prefix_of_an_addressed_request` each fail when the production code they cover is reverted, as section 2 says. `cargo test -p trusted-server-core`, `cargo test-fastly`, `cargo clippy -p trusted-server-core --all-targets -- -D warnings`, `cargo clippy-fastly` and `cargo fmt --all -- --check` pass. With no `RemovedPathPrefix` on a request, or one that is empty or `/`, every `Set-Cookie` core writes is as before and the jar keeps the last pair of a name. The existing tests check the unchanged output byte for byte for the Edge Cookie and its expiry, the pull-sync marker and its expiry, and the tester cookie and its clearing, and check the resolved marker and its expiry for their `Path` only. On an addressed request a withdrawal expires the Edge Cookie and the resolved marker under the removed prefix and under `/`, checked byte for byte, expires a pull-sync marker the request carried under both, and tombstones the row of each Edge Cookie the request carried. `a_withdrawal_under_a_removed_prefix_also_expires_a_root_pull_sync_marker` and `a_withdrawal_under_a_removed_prefix_tombstones_each_edge_cookies_row` each fail when the production code they cover is reverted. ## Revision record @@ -127,6 +130,7 @@ The tests listed under "Tests" in section 2 pass, including `dispatch_set_tester | 2026-10-15 | Fixes from review round 1. What a refused removed prefix becomes, the rollout for returning visitors, the jar rule for an empty prefix, the adapter table, tests and acceptance for the Fastly tester change, the code no test reaches, the consent tools that write `Path=/`, signed origins, the cookies left out, the open questions and the review log. | | 2026-10-15 | Corrections from review round 2. The tests that pass a removed prefix through the `EcContext` readers, the twelve integrations of the client script that use root-relative paths or write cookies, three more untrue statements under Open questions and the review log. | | 2026-10-15 | Code fixes for open questions 1 to 4. `RequestAddress::new` accepts any removed prefix again and `under_path` holds the one check, a removed prefix that is empty, `/` or not a cookie path reads the jar as no prefix does, tests reach the production wiring through the context and the resolve endpoint, and a withdrawal also expires the Edge Cookie and the resolved marker under `/`. The code comments open question 5 listed are corrected, and the remaining open question keeps the pages. | +| 2026-10-16 | Code fixes for the rollout and privacy finding of review round 1. On an addressed request a withdrawal also expires a pull-sync marker under `/` and tombstones the row of each Edge Cookie the request carried, and the rustdoc of `for_request` names nested prefixes. Open question 2 records why core does not move a root Edge Cookie, and open question 3 records that the Osano and Sourcepoint mirrors are left with the client script. | ## Review log From b360926dd1c5c7b4988a58f8c64ed5a84bb206fa Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 07:57:36 +0100 Subject: [PATCH 160/192] The Fastly names design matches its branch The specs copy now carries section 2.6, in which core names no hosting platform or platform domain in any line, with the counts, the _vercel prefix, the guard and the tests that moved to the Fastly adapter. --- .../2026-10-15-fastly-names-in-core-design.md | 46 +++++++++++++++++-- 1 file changed, 43 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md index 3814736860..622dbd198f 100644 --- a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md +++ b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md @@ -7,7 +7,7 @@ **Issues:** IABTechLab/trusted-server#777 (its Fastly row) **Last updated:** 2026-10-16 -> **Why this spec exists.** After the series before this one, core still names one hosting platform, Fastly, in small pieces. They are the format of a long configuration document, a staging variable, three request headers, a device module selector and the names of two functions. Each is a place where a change by one platform is a change in core. One more Fastly header name, `x-compress-hint`, stays in core after this change (section 3). +> **Why this spec exists.** After the series before this one, core's shipped code still names one hosting platform, Fastly, in small pieces. They are the format of a long configuration document, a staging variable, three request headers, a device module selector and the names of two functions. Each is a place where a change by one platform is a change in core. Core's comments, tests and README files name Fastly, Cloudflare, Spin, Fermyon, Viceroy and Axum as well, the attestation settings name Vercel, and three tests use a host under Fastly's `edgecompute.app` domain. After this change no line of core names a hosting platform, and a guard outside core keeps it so (section 2.6). One Fastly header, `x-compress-hint`, whose name does not name the platform, is still set by core after this change (section 3). ## 1. Problem @@ -23,6 +23,8 @@ These names remain in core, each with the files that hold it: Each name is small. Each one means that another platform, or a change in how Fastly stores, flags or forwards something, is an edit to core rather than to the adapter beside it. +Core's comments, doc comments, tests and README files named platforms too, and so did one shipped line, the `_vercel` entry among the prefixes an attestation endpoint may not take. Each told a reader that core knows which platform runs it or serves its origin. Section 2.6 counts them. + ## 2. Design Each name moves to the adapter that reads it, and core keeps a neutral declaration or interface in its place. The settings a deployment writes do not change, except the device selector. @@ -108,11 +110,44 @@ The Spin adapter is the only adapter that calls `sanitize_forwarded_headers` (`n **What changes for an operator.** One error message changes with the names. The `Proxy` error for a streaming body in the buffered conversion (`proxy.rs`) now reads `streaming platform response body is not supported by buffered response conversion`, where it named a Fastly response conversion. A `Proxy` error answers 502 with `An internal error occurred`, so the message reaches the log and not the response. Both functions are internal to core. +### 2.6 Core names no hosting platform + +Core is built on EdgeZero and names no hosting platform in any line, whether code, comment, test, document or file name. EdgeZero's own names stay, because EdgeZero is the platform layer core is built on. An adapter may name its own platform. + +**What changes.** + +- Comments and doc comments say what they meant without the platform's name, for example a streaming adapter, an adapter whose platform has a shared cache, or a platform whose `send_async` runs each request eagerly. Where one cited a platform SDK's source, the adapter that maps core's type onto that SDK already documents it (`apply_fastly_cache_intent` in the Fastly adapter's `platform.rs`). +- The `Surrogate-Control` tests in `publisher.rs` test the standard header core reads for every platform, so their names and assertion messages say `Surrogate-Control`, or edge freshness, where they said Fastly. The wildcard exclusion test in `creative.rs` excludes `*.cdn.example.com` where it excluded a platform's domain. A registry test name, a publisher test name and a stub's assertion message drop the platform they named. +- The test that the largest `KvMetadata` serializes within 2048 bytes moves to the tests of the Fastly adapter's store (`ec_kv.rs`), because the limit is the Fastly KV Store's. +- The test that core's code never uses the Fastly SDK moves from core's `migration_guards.rs` to the Fastly adapter's `core_sdk_guard.rs`. A build script in the adapter embeds every Rust source under core's `src` by path, so a file added to core is checked with no edit to the test. Core's build script no longer lists its own sources. +- Core's README and the auction README show the command that runs core's tests on the host and point at `TESTING.md` for the adapter aliases, where they listed each alias by its platform. +- `_vercel` leaves `ORIGIN_ASSET_PREFIXES`, the prefixes a web framework serves its assets from and an attestation endpoint may not take, because it is a hosting platform's prefix and not a framework's. The attestation guide names it instead, as a prefix the origin serves that the operator has to avoid, which the guide already says of every prefix the list does not hold. Three HTML processor tests use `test-publisher-ts.edge.example` as the request host, where they used a host under `edgecompute.app`. +- `scripts/lint-core-platform-names.py` fails on any line or path under `crates/trusted-server-core` that names a hosting platform, a tool named for one, or a domain a platform serves deployments from. It splits text into words at every character that is not a letter and at each change from lower to upper case, so `FASTLY_IS_STAGING` and `edgezero_adapter_cloudflare` each name a platform. Six names, among them `spin`, `axum` and `aws`, count only as words of their own, because ordinary words such as `spinner` and `withdraws` contain them. Eighteen, among them `fastly`, `cloudflare`, `akamai`, `vercel` and `netlify`, also count at the start of a word, so `akamaized` counts. Sixteen domains, among them `edgecompute.app`, `workers.dev` and `fermyon.app`, count where they are not part of a longer host name. It runs its matcher on every name and domain and on known non-names before it reads core, and stops with exit 2 when the matcher is wrong. The Run Format workflow runs it as a job of its own, and `AGENTS.md` lists it among the CI gates. + +**The count.** Lines of `crates/trusted-server-core` that name each platform on this branch before and after this change, counted as a whole word with `git grep -i -w`, and with the guard, which also counts a name inside an identifier such as `FASTLY_IS_STAGING` or `_vercel`. + +| Name | `git grep -i -w` before (files) | Guard before | After, either count | +| -------------------------- | ------------------------------- | ------------ | ------------------- | +| Fastly | 100 (32) | 110 | 0 | +| Cloudflare | 26 (14) | 28 | 0 | +| Axum | 27 (14) | 27 | 0 | +| Spin | 21 (10) | 23 | 0 | +| `edgecompute.app` | 5 (1) | 5 | 0 | +| Viceroy | 4 (3) | 4 | 0 | +| Fermyon | 1 (1) | 1 | 0 | +| Vercel | 0 | 1 | 0 | +| Every other name or domain | 0 | 0 | 0 | + +The guard found 173 lines before this change, some of them naming two platforms, and none after. One of them, the `_vercel` entry, was shipped code, and the rest were comments, documents or test code. + +**What changes for an operator.** An attestation endpoint beneath `/_vercel` is accepted where it was refused, so an operator whose origin is hosted on Vercel has to avoid that prefix, as the attestation guide says. Nothing else that core does when it runs changes. + +**For crates outside core.** `ORIGIN_ASSET_PREFIXES` no longer holds `_vercel`, and nothing else in core's API changes. A change that adds a hosting platform's name to core, in a comment or a test as much as in code, fails the guard. The Fastly adapter gains a build script and a `regex` dev-dependency for the test of core's sources. + ## 3. What is not in this change -- The `x-compress-hint` header, a Fastly compression hint. Core sets it to `on` on the script bundle responses it serves (`HEADER_X_COMPRESS_HINT` in `constants.rs`, used in `publisher.rs`) and lists it among the headers never passed to a third party (`constants.rs`). +- The `x-compress-hint` header, a Fastly compression hint. Core sets it to `on` on the script bundle responses it serves (`HEADER_X_COMPRESS_HINT` in `constants.rs`, used in `publisher.rs`) and lists it among the headers never passed to a third party (`constants.rs`). The header's name does not name the platform, so the guard of 2.6 passes it. - The comment at the top of `platform/request_headers.rs`, which says a TLS or spoofable header is removed at entry, without the precondition in 2.3 that the adapter calls `sanitize_forwarded_headers`. It is a code comment and stays for a later change. -- Comments that mention Fastly, in 29 of core's `.rs` files, and the names and assertion messages of some tests, the publisher's surrogate cache tests among them. None is a shipped line of core. - The Axum, Cloudflare and Spin adapters' comments comparing their behavior with the Fastly adapter's, and their two errors that suggest the Fastly adapter for parallel bidder fan-out. - The `ts` development proxy's removal of `fastly-ssl`. @@ -123,6 +158,8 @@ The Spin adapter is the only adapter that calls `sanitize_forwarded_headers` (`n - A declared TLS header sets the scheme and an undeclared one does not, the declared TLS and spoofable headers are stripped with the standard ones, the auction sees the client's declared TLS header where the origin request does not, a declared internal header is refused in the `auth_header` slot of `[trusted_client_ip]`, and a bare `client-ip` is refused where `example-client-ip` is accepted. - The `platform` device selector builds the module the adapter's factory supplies, and no selector builds the built-in one. - The asset proxy's tests pass unchanged through the renamed conversions. +- The guard of 2.6 passes on core after this change and fails on core before it. Each of its 40 names and domains, planted once on a line of its own in a file git does not yet track, fails it on that line, and so do `_vercel` and the `edgecompute.app` test host put back where they were. It passes `spinner`, `SpinyAI`, `steadfastly`, `denominator`, `withdraws`, `EdgeZero`, `notworkers.dev` and `butterfly.io`. With its word pattern or its domain pattern broken to match nothing, it stops with exit 2 before it reads core. +- The Fastly adapter checks that core's code never uses the Fastly SDK, that the list of core's sources it embeds covers core, and that the largest `KvMetadata` fits the Fastly KV Store's 2048-byte metadata limit. Not tested: @@ -131,6 +168,8 @@ Not tested: - A declared internal header in the `ip_header` slot. The one test uses the `auth_header` slot, and one loop checks both slots. - Any change on the Axum, Cloudflare and Spin adapters listed in 2.1 to 2.4. No adapter test sends a client's `fastly-ssl`, `fastly-client-ip` or `x-debug-fastly-pop` to those adapters, or selects `module = "platform"` on them. - The strip at entry of a declared header through an adapter. Core's test declares example headers and calls `sanitize_forwarded_headers` directly, and the one adapter that calls the function declares none. +- The guard's job in the Run Format workflow, which only a pull request's checks run. +- That the guard's list of names is complete. It holds the platforms the stack has adapters or tools for and other common edge, CDN and hosting platforms, so a platform outside the list has to be added to it. ## Revision record @@ -139,6 +178,7 @@ Not tested: | 2026-10-15 | First draft. | | 2026-10-15 | Fixes from review round 1. The readers of each declared list and the open questions on them, `x-debug-fastly-pop`, the scheme change, the effects on the other adapters, where each refusal happens, the rollout, the narrowed constants, `x-compress-hint`, the file count, the untested paths and the review log. | | 2026-10-15 | Corrections from review round 2. The Fastly sentence of 2.3 and the changelog on the `x-debug-fastly-pop` refusal, the changelog on the spoofable strip and the scheme change, when the Axum, Cloudflare and Spin adapters strip `fastly-client-ip`, and the comment in `request_headers.rs` left for a later change. | +| 2026-10-16 | Core names no hosting platform or platform domain in any line. Section 2.6 with the counts, the `_vercel` prefix and the test host, the guard and the tests that moved to the Fastly adapter, and section 3 no longer leaves core's comments and tests for later. | ## Review log From 030970f85f7f35735d17a7175cc96d4f27a4bfa4 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 07:57:38 +0100 Subject: [PATCH 161/192] The last names design matches its branch The specs copy now carries the 15 October code fixes and the 16 October section in which cache headers follow the stored item's policy, which closes the open question on the cache headers. --- .../2026-10-14-last-names-in-core-design.md | 88 ++++++++++++------- 1 file changed, 56 insertions(+), 32 deletions(-) diff --git a/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md b/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md index d7dce5f787..81155a780c 100644 --- a/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md +++ b/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md @@ -65,40 +65,61 @@ auction_token_secret = "tinybird_auction_append_token" **What the module sends.** Core builds the rows, being the auction's summary, each provider call and each bid, from the same observation as before, and the module posts the batch it is handed without adding a field, so the privacy boundary in `docs/guide/telemetry.md` is unchanged and the rows carry no Edge Cookie identifier. Core hands rows only to the sink `[analytics] module` selected, so a deployment without the section sends nothing. A failure to build or start a send is logged as a warning and never changes the response, and the module drops the pending request without waiting for the Events API. -**A module written with its type folder.** The guide lets a section's selector name a module in full or with its type folder left off (`docs/guide/configuration-rules.md`), as `analytics.tinybird` or `tinybird` in `[analytics]`. Two functions in core now shorten a name written with the section's own type folder before they use it, in every section and not only `[analytics]`, so each takes the module's table to be `[analytics.tinybird]` whichever way the selector wrote the name. `check_tables` in `provider_table.rs` decides whether each table in a section belongs to a selected module, and `Settings::module_config` in `settings.rs` finds the table a module reads. A table written at the full name's parts, as `[analytics.analytics.tinybird]`, is therefore refused, whereas a name under another type's folder, as `testing.testlight` in `[auction]`, keeps its table at the full name's parts. Other places still look a table up by the name as the selector wrote it, among them `source::selected` for a `[source]` module, the demand and ad server tables read in `auction/plan.rs`, `Settings::section_table`, which the deploy check of a module's secret uses, and `module_secrets::clear_unused`, which the load of a stored configuration uses. - -**The full name of the analytics module fails today.** With `[analytics] module = "analytics.tinybird"`, `ts config validate` refuses the document, saying the secret key reference at `analytics.analytics.tinybird.auction_token_secret` must not be empty, because the deploy check looks for the table at `[analytics.analytics.tinybird]` and finds none. The load of a stored configuration fails as well, because the load removes `auction_token_secret` from `[analytics.tinybird]` as unused and the module then refuses the missing token. The short name `tinybird` works through both. Until the deploy check and the load take the table to be at the name without the type folder, as `check_tables` and `Settings::module_config` do, `analytics.tinybird` is not a valid selector for the analytics module, although the guide documents it. The fix is code and is listed under the open question in 2.2. +**A module written with its type folder.** The guide lets a section's selector name a module in full or with its type folder left off (`docs/guide/configuration-rules.md`), as `analytics.tinybird` or `tinybird` in `[analytics]`. Four places in core now shorten a name written with the section's own type folder before they use it, in every section and not only `[analytics]`, so each takes the module's table to be `[analytics.tinybird]` whichever way the selector wrote the name. `check_tables` in `provider_table.rs` decides whether each table in a section belongs to a selected module, `Settings::module_config` in `settings.rs` finds the table a module reads, the deploy check of a module's secret settings in `config.rs` finds the table it checks, and `module_secrets::clear_unused`, which the load of a stored configuration runs, keeps the secret settings of that table in use. `ts config validate` and the load of a stored configuration therefore accept `module = "analytics.tinybird"` as they accept `tinybird`. A table written at the full name's parts, as `[analytics.analytics.tinybird]`, is refused, whereas a name under another type's folder, as `testing.testlight` in `[auction]`, keeps its table at the full name's parts. Other places still look a table up by the name as the selector wrote it, among them `source::selected` for a `[source]` module and the demand and ad server tables read in `auction/plan.rs`. **What leaves core.** The `Settings::analytics` field, `Settings::analytics_module`, `AnalyticsConfig` and core's `TinybirdSettings` are gone, `TinybirdSettings` is now the module crate's, and the Fastly adapter's own `tinybird` module is deleted. `CHANGELOG.md` says what crates outside core use instead. -**Rollout.** A deployment that already writes `[analytics] module = "tinybird"` and `[analytics.tinybird]` changes nothing, in the document or in the form stored. A deployment that writes the full name, `module = "analytics.tinybird"`, writes `tinybird` instead until the code fix under the open question is made. A deployment that lists its own builders adds `trusted_server_analytics_tinybird::builder()` to keep sending, because the registry refuses an `[analytics]` name no listed builder supplies. A deployment on the Axum, Cloudflare or Spin adapter sends nothing before or after. +**Rollout.** A deployment that already writes `[analytics] module = "tinybird"` and `[analytics.tinybird]` changes nothing, in the document or in the form stored. A deployment that lists its own builders adds `trusted_server_analytics_tinybird::builder()` to keep sending, because the registry refuses an `[analytics]` name no listed builder supplies. A deployment on the Axum, Cloudflare or Spin adapter sends nothing before or after. + +### 2.2 The adapter declares its own platform's cache header + +Core named four shared cache headers itself. Two come from standards, `Surrogate-Control` from the Edge Architecture Specification and `CDN-Cache-Control` from RFC 9213, and the other two belong to one platform each, `fastly-surrogate-control` and `cloudflare-cdn-cache-control`. Core could write a policy in three of them, all but `fastly-surrogate-control`, and it stripped all four from every response it made private, on every adapter. It now names only the two standard headers, and section 2.3 says which cache headers it writes or removes. + +**The interface.** An adapter declares the headers its own platform reads a shared cache policy from with `platform::declare_platform_cache_headers`, once, as it builds its state. Core reads the declaration only in `origin_response_may_be_kept` in `publisher.rs`, which keeps an origin response with a policy in a declared header out of the stored copies. The header an adapter has core write its policy in is `EdgeCacheHeader::Platform(name)`, beside `SurrogateControl`, `CdnCacheControl`, the `s-maxage` fallback and none, and `CachePolicy::apply_to_response` records it on the response as a `WrittenPlatformCacheHeader`. A name that is not a valid header name is never written, and a warning is logged each time a policy is rendered for it. The declared list is held for the whole running process, so a name declared twice is listed once, nothing removes a name, and a test that declares an example name leaves it listed for every other test in the same test program. + +**What each adapter declares and writes.** Each adapter holds the header it writes its policy in as `EDGE_CACHE_HEADER` in its own `app.rs`. + +| Adapter | Declares | Writes its policy in | +| ---------- | ------------------------------ | ------------------------------ | +| Fastly | `fastly-surrogate-control` | `Surrogate-Control` | +| Cloudflare | `cloudflare-cdn-cache-control` | `cloudflare-cdn-cache-control` | +| Axum | nothing | `s-maxage` in `Cache-Control` | +| Spin | nothing | `s-maxage` in `Cache-Control` | -### 2.2 The platform declares the header it reads a shared cache policy from +**What leaves core.** `EdgeCacheHeader::CloudflareCdnCacheControl` and the four constants for the Fastly and Cloudflare header names (`HEADER_FASTLY_SURROGATE_CONTROL`, `HEADER_FASTLY_SURROGATE_CONTROL_NAME`, `HEADER_CLOUDFLARE_CDN_CACHE_CONTROL` and `HEADER_CLOUDFLARE_CDN_CACHE_CONTROL_NAME`) are gone from `cache_policy.rs`, and `EDGE_CACHE_HEADER_NAMES` lists only the two standard headers. `EdgeCacheHeader::Platform` takes the variant's place, and the Fastly and Cloudflare adapters hold their own header names as `FASTLY_SURROGATE_CONTROL_HEADER` and `CDN_CACHE_CONTROL_HEADER`. `CHANGELOG.md` says what crates outside core use instead. -Core named four shared cache headers itself. Two come from standards, `Surrogate-Control` from the Edge Architecture Specification and `CDN-Cache-Control` from RFC 9213, and the other two belong to one platform each, `fastly-surrogate-control` and `cloudflare-cdn-cache-control`. Core could write a policy in three of them, all but `fastly-surrogate-control`, which it only stripped, and it stripped all four from every response it made private, on every adapter. It now knows the two headers every platform may read and asks the running platform for the rest. +**What changes for an operator.** Each adapter writes its policy in the header it wrote before. Where core removed all four headers from every response it made private, it now removes `fastly-surrogate-control` from none and `cloudflare-cdn-cache-control` only where core wrote its own policy there, so either header stays where an origin or an operator's `response_headers` wrote it. An origin response with a policy in `fastly-surrogate-control` is kept out of the stored copies on Fastly alone, and one with a policy in `cloudflare-cdn-cache-control` on Cloudflare alone, where every adapter kept both out. Section 2.3 says what a deployment configures to remove such a header. -**The interface.** An adapter declares the headers its platform reads a shared cache policy from with `platform::declare_platform_cache_headers`, once, as it builds its state. `cache_policy::edge_cache_header_names` lists the two core reads and then the declared ones, and that list is what core strips from a response it makes private, skips among an operator's response headers on an uncacheable response, and treats as a policy it does not read when it decides whether an origin page may be kept. The header an adapter writes the policy in is `EdgeCacheHeader::Platform(name)`, beside `SurrogateControl`, `CdnCacheControl`, the `s-maxage` fallback and none. A declared name that is not a valid header name is never written, and a warning is logged each time a policy is rendered for it. The declared list is held for the whole running process, so each adapter appends to the list each time the adapter builds its state, a name declared twice is listed once, and nothing removes a name. A test that declares an example name therefore leaves the name listed for every other test in the same test program, as the tests in `cache_policy.rs`, `response_privacy.rs` and the Google ad tag diagnostics do. +### 2.3 Cache headers follow the stored item's policy -**What each adapter declares and strips.** An adapter writes its policy in the header shown and strips the names shown from a response core makes private. +Only a middleware changes an asset or a response. Everything else changes the model, and the cache policy of an item as stored is model. So removing what an origin sent is the job of a fetch middleware that a deployment chooses, and the cache headers a response is served with mirror the cache policy of the item as stored. Neither core nor an adapter names a platform's cache header in order to remove it. -| Adapter | Declares | Writes its policy in | Strips from a response core makes private | -| ---------- | ------------------------------ | ------------------------------ | ------------------------------------------------------------------------ | -| Fastly | `fastly-surrogate-control` | `Surrogate-Control` | `surrogate-control`, `cdn-cache-control`, `fastly-surrogate-control` | -| Cloudflare | `cloudflare-cdn-cache-control` | `cloudflare-cdn-cache-control` | `surrogate-control`, `cdn-cache-control`, `cloudflare-cdn-cache-control` | -| Axum | nothing | `s-maxage` in `Cache-Control` | `surrogate-control`, `cdn-cache-control` | -| Spin | nothing | `s-maxage` in `Cache-Control` | `surrogate-control`, `cdn-cache-control` | +Core owns `Cache-Control` and the two standard headers, which it reads on every adapter and may write its policy in, and on one response it also owns a platform header it wrote its policy in at the adapter's request. Where the item's policy is private, the response says so in those headers and in no others. Each place the declared list was used becomes the following. -**What leaves core.** `EDGE_CACHE_HEADER_NAMES`, `EdgeCacheHeader::CloudflareCdnCacheControl` and the four constants for the Fastly and Cloudflare header names (`HEADER_FASTLY_SURROGATE_CONTROL`, `HEADER_FASTLY_SURROGATE_CONTROL_NAME`, `HEADER_CLOUDFLARE_CDN_CACHE_CONTROL` and `HEADER_CLOUDFLARE_CDN_CACHE_CONTROL_NAME`) are gone from `cache_policy.rs`. `edge_cache_header_names()` and `EdgeCacheHeader::Platform` take their place, and the Fastly and Cloudflare adapters hold their own header names as `FASTLY_SURROGATE_CONTROL_HEADER` and `CDN_CACHE_CONTROL_HEADER`. `CHANGELOG.md` says what crates outside core use instead. +| Where core used the declared list | What it does now | +| -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Removing them from a response core makes private (`response_privacy.rs`) | Removes `Surrogate-Control`, `CDN-Cache-Control` and a platform header core wrote its policy in on that response, which core records on the response when it writes it. A platform header an origin or an operator wrote stays. | +| Removing them before core writes its own policy (`cache_policy.rs`) | Removes the two standard headers and writes the policy in the header the adapter asks for, replacing what was there. | +| Skipping an operator's `response_headers` entry on an uncacheable response | Skips `Cache-Control`, the two standard headers and a platform header core wrote its policy in. An entry naming another header is written as configured. | +| Not keeping an origin response with a policy core does not read (`publisher.rs`) | Reads `CDN-Cache-Control` and the header the adapter declares for its own platform, because whether an item is kept is model, and keeps a response with a policy in either out of the stored copies. | -**What changes for an operator.** The header each adapter writes its policy in does not change, but what core strips from a response does, because each adapter now strips the two standard names and the ones it declares, where core stripped all four names on every adapter before. The same list decides three things, so each changes the same way: +**What a deployment configures.** A deployment whose origin sends a platform's cache header on a page core may make private names a fetch middleware that removes the header, in a `[[fetch]]` entry for the page's media type. None exists in the stack, and adding one takes two things. Core has to let a fetch middleware remove a header from the origin's response before the item is stored, because today a fetch middleware changes only the body and only a serve middleware writes a header (`MiddlewareAction::response_headers`). Then one crate per platform supplies the middleware, `crates/middleware/fastly` for `fastly-surrogate-control` and `crates/middleware/cloudflare` for `cloudflare-cdn-cache-control`. Until both exist the deployer stops the origin sending the header on such pages. -1. A response core makes private no longer has `cloudflare-cdn-cache-control` removed on the Fastly adapter, `fastly-surrogate-control` removed on the Cloudflare adapter, or either removed on the Axum and Spin adapters. -2. An entry in an operator's `response_headers` for one of those names on the same adapters is no longer skipped on an uncacheable response, so it is applied. -3. An origin response that carries only one of those names on the same adapters is no longer kept out of the stored copies. `origin_response_may_be_kept` in `publisher.rs` decides whether an origin response may be kept as a stored template or a stored asset copy, and the function no longer refuses a response for carrying such a name. Only the Fastly adapter provides a template cache. +**Where core or an adapter still alters the response.** Each place below writes or removes cache headers as a step of its own rather than in a serve middleware writing them from the item's policy, and each is to move there. -**Open question.** Is this narrowing acceptable? The other choice is for every adapter to declare all four names, which strips on each adapter what core stripped before. This design does not choose between them and leaves the choice to the review. One code fix is also still to make, and it is not a question. The full name of the analytics module, `[analytics] module = "analytics.tinybird"`, fails `ts config validate` and the load of a stored configuration today (section 2.1), so the deploy check of a module's secret and the load of a stored configuration need to take the table to be at the name without the type folder, as `check_tables` and `Settings::module_config` do. +| Place | What it does to the response | +| ------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `response_privacy::enforce_private_no_store`, through `enforce_terminal_private_cache_privacy` and `enforce_synthesized_html_cache_privacy` | Writes `Cache-Control: no-store, private` and removes the headers core owns, `ETag`, `Last-Modified`, `Expires` and `Age`. Core calls it on what it answers for one reader, personalized and assembled pages among them, and so do the Google ad tag diagnostics and the Didomi module. | +| `response_privacy::enforce_set_cookie_cache_privacy` | On a response that sets a cookie, removes the headers core owns and writes `Cache-Control: private, max-age=0` unless it already forbids shared storage. | +| `response_privacy::enforce_uncacheable_cache_privacy` | Removes the headers core owns from a response whose `Cache-Control` is private or no-store. | +| `response_privacy::apply_response_headers_with_cache_privacy`, called by every adapter's `apply_finalize_headers` | Writes the operator's `response_headers` between the two steps above, skipping the headers core owns on an uncacheable response. | +| `response_privacy::apply_inactive_ad_stack_browser_cache_policy` | Writes `Cache-Control: private, max-age=60` on an inactive ad template. | +| `cache_policy::CachePolicy::apply_to_response` and `apply_to_headers` | Write `Cache-Control` and the adapter's header, or `s-maxage`, for `/static/tsjs=`, a `[cache]` asset rule and the asset proxy. | +| `cache_policy::apply_no_store_private_to_headers`, also reached through `CacheControlPolicy::NoStorePrivate` | Writes `Cache-Control: no-store, private` and removes the standard headers, for the asset proxy's private answers, the Prebid module's script and the first place in this table. | +| The Fastly adapter's entry point, `lib.rs` | Applies the asset proxy's policy in `Surrogate-Control` after finalization, and runs the first three again after request filter effects and the Edge Cookie step (`apply_terminal_response_effects`). | +| Every adapter's routes | Choose the header core writes the policy in, `Surrogate-Control` on Fastly, `cloudflare-cdn-cache-control` on Cloudflare and `s-maxage` on Axum and Spin. | -### 2.3 Later rows +### 2.4 Later rows The store names, the fingerprint class and the key format follow in the same series, each with its own section here as it lands. @@ -110,29 +131,32 @@ The store names, the fingerprint class and the key format follow in the same ser - The store names and removed store ids, the JA4 fingerprint class and the JWKS key format, which the table above lists and which follow in the same series. - Names the table does not list. Core also names `fastly-client-ip` in the check of `[trusted_client_ip]` (`settings.rs`), lists `fastly-ssl` and `fastly-client-ip` among the headers it strips from a request on entry (`http_util.rs`), reads `fastly-ssl` as a signal of the request's scheme (`http_util.rs`), and removes `fastly-ssl` from the request before calling the origin (`publisher.rs`). - The vendor's name in two places in core, being a comment in `provider_table.rs` and a secret fixture in the tests of `config_payload.rs`. +- The fetch middleware that removes a platform's cache header and the core change it needs, and moving the places section 2.3 lists into a serve middleware. ## 4. Tests -- A declared platform header is stripped with the standard ones by the cache policy and by the response privacy guard, is not restored by an operator header, and a policy is written in the platform's own header when the adapter asks for it. A name declared twice is listed once. -- The module's crate tests its sink (one Events API request per batch, no wait, the row limit, an empty batch sends nothing), its table (the host, the token, the reserved settings, the bounds, an unknown key, the deprecated fields, the documented block in the example configuration), its stored form (the selection and the table survive being stored, with no table at the old name) and its selection (the short name and the full name, the full name through the settings load and the module's registration only, no section, a table the section does not select, the old top-level table, a name the build does not have, the token's key name at deploy). +- Core tests that a declared platform header stays on a private response and on a cookie response, whoever wrote it (`a_declared_platform_header_stays_on_a_private_response`, `a_platform_header_core_did_not_write_is_left_on_a_cookie_response`), that a policy core wrote in a platform header is removed with the headers core owns and an operator header does not restore it (`a_policy_written_in_a_platform_header_is_removed_with_the_owned_headers`, `a_policy_core_wrote_in_a_platform_header_leaves_a_cookie_response`), and that a private policy leaves nothing in the header it is written in (`a_private_policy_empties_the_platform_header_it_is_written_in`). A policy is written in the platform's own header when the adapter asks for it, an origin response with a policy in a declared header is not kept, and a name declared twice is listed once. +- The module's crate tests its sink (one Events API request per batch, no wait, the row limit, an empty batch sends nothing), its table (the host, the token, the reserved settings, the bounds, an unknown key, the deprecated fields, the documented block in the example configuration), its stored form (the selection and the table survive being stored, with no table at the old name) and its selection (the short name and the full name, no section, a table the section does not select, the old top-level table, a name the build does not have, the token's key name at deploy). The full name goes through the settings load and the module's registration, through the deploy check of its token (`the_full_name_s_token_is_checked_at_deploy_in_the_table_the_module_reads`) and through the load of a stored configuration (`the_full_name_loads_from_the_stored_form_with_its_token`). +- Core tests a module of another type selected by its full name, `testing.probe` in `[testing]`, through the deploy check of its secret (`deploy_validation_checks_the_secret_of_a_module_selected_by_its_full_name`), the clearing of secrets not in use (`a_module_selected_by_its_full_name_keeps_the_settings_in_the_table_it_reads`) and the load of a stored configuration (`the_load_looks_up_the_secret_of_a_module_selected_by_its_full_name`). - Core tests the kind: a selected module that supplies no analytics module is refused naming it, a section selecting two names is refused, and no section resolves no sink. - Each of the Axum, Cloudflare, Fastly and Spin adapters tests that its linked modules are listed in hook order with the analytics module among them (`the_linked_modules_are_offered_in_hook_order`), and the Axum adapter's shipped modules test proves the analytics module refuses a setting it does not know. The `ts` tool's list has no test of its own. +- Each of the Axum, Cloudflare, Fastly and Spin adapters builds its state, proves it declares its own platform's header and no other, and proves a response core makes private carries `no-store, private`, loses the policy core wrote in the adapter's header and keeps a platform's header the origin sent (`a_private_response_follows_its_policy_and_keeps_the_origins_header`). Not tested: -- That the Fastly adapter declares `fastly-surrogate-control` and the Cloudflare adapter declares `cloudflare-cdn-cache-control`. No adapter test reads the declared list and the core tests declare an example name of their own, so deleting an adapter's declaration passes every test and leaves its platform's header on a response core makes private. - Any adapter's reading of the sink. No test builds an adapter's state with `[analytics]` selecting the module and checks which sink its services carry, so the Fastly adapter's wiring and the lack of it on the other three adapters are both untested. -- The analytics module's full name through the deploy check of its token and the load of a stored configuration. No test selects the module that way through either, and both fail for the full name today (section 2.1). -- A name written with its type folder for a module type other than analytics. No core test covers `check_tables` or `Settings::module_config` with such a name, and the Tinybird crate covers them through the settings load and the module's registration only. Nor does any test select a module of another type that way through the deploy check of its secret or the load of a stored configuration. -- A declared name that is not a valid header name, which no test declares. +- `Settings::module_config` given a name written with its type folder for a module type other than analytics, and the refusal by `check_tables` of a table at the full name's parts. No test covers either. +- A platform header that is not a valid header name, which no test writes. ## Revision record -| Date | Change | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-14 | First draft. | -| 2026-10-15 | Fixes from review round 1. The module lists of each adapter and of the `ts` tool, the headers each adapter declares and strips with the narrowing as an open question, the adapters that send telemetry, the table change for a name written with its type folder, the defaults and bounds, what leaves core, the rollout, the names left in core, the untested paths and the review log. | -| 2026-10-15 | Corrections from review round 2. The full name of the analytics module failing the deploy check and the load of a stored configuration, the code fix under the open question, what the selection test reaches, and `fastly-ssl` among the names left in core. | +| Date | Change | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-14 | First draft. | +| 2026-10-15 | Fixes from review round 1. The module lists of each adapter and of the `ts` tool, the headers each adapter declares and strips with the narrowing as an open question, the adapters that send telemetry, the table change for a name written with its type folder, the defaults and bounds, what leaves core, the rollout, the names left in core, the untested paths and the review log. | +| 2026-10-15 | Corrections from review round 2. The full name of the analytics module failing the deploy check and the load of a stored configuration, the code fix under the open question, what the selection test reaches, and `fastly-ssl` among the names left in core. | +| 2026-10-15 | Code fixes. The deploy check of a module's secret settings and the load of a stored configuration read the table of a module selected by its full name at the name without its type folder, and the tests of the fix are listed. | +| 2026-10-16 | Cache headers follow the stored item's policy, in a new section 2.3 that closes the open question on the cache headers. Each adapter declares only its own platform's header, which core reads only to decide whether an origin response may be kept, core removes from a private response only the headers it owns, and the page says what a deployment configures and lists where core or an adapter still alters the response. | ## Review log From 94b6879bfb1728c7736a331318b2590efad18da4 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 08:21:09 +0100 Subject: [PATCH 162/192] The Edge Cookie design answers its second review round A graph configured later replaces only a server-side module's cookie, and a client-side module's cookie keeps its identifier with no row, at the cost the design states. The change discloses more in three cases, the client-side module on a host with no graph being the third, each adapter's resolve route is shown to set a client-side module's cookie, and the review log records round 2. --- ...-17-edge-cookie-on-every-adapter-design.md | 27 ++++++++++++------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md index 040935ff36..1573a42d9b 100644 --- a/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-edge-cookie-on-every-adapter-design.md @@ -32,7 +32,7 @@ Core writes a created Edge Cookie only with its identity-graph row. `ec_finalize **Writing it.** Every response a route in the request cycle makes carries an `EcFinalizeState`, holding the context and the request's services. These are the responses the Fastly adapter attaches it to, being the named routes but the inspection endpoints, the fallback's tsjs, module route and publisher answers, and a filter's answer. robots.txt carries none, as on Fastly. `FinalizeResponseMiddleware` takes the state off the response once it has written the finalize headers and runs `ec_finalize_response` with no graph, which applies a module's response headers, writes an Edge Cookie the request created, expires the cookie and its markers on a withdrawal, and reconciles the pull-sync marker. The request filters' response effects are applied after it, as on Fastly. The `sharedId` cookie the Fastly adapter carries to finalizing is read only by the graph's identifier writes, so it is not carried. -**Core writes the cookie on every host.** `ec_finalize_response` writes a created Edge Cookie for a request that passed the bot gate and whose permissions allow the Edge Cookie (`EcContext::ec_allowed`), with or without an identity graph. With one, it records the request's identifiers on the cookie's row and writes the cookie once that row is authoritative, as before. Without one, it skips the graph steps and writes the cookie through `set_ec_cookie_on_response`, which takes the request's public prefix and, on a request addressed by path, the removed prefix as its `Path`, as core writes every cookie it owns. Core reads the bot gate from the device signals the adapter sets on the context (`EcContext::passed_bot_gate`, which a request with no device signals does not pass), because a missing graph no longer stops the cookie and so can no longer stand in for the gate. The Fastly adapter still opens no graph for a request the gate stops, so no graph row is written for it. +**Core writes the cookie on every host.** `ec_finalize_response` writes a created Edge Cookie for a request that passed the bot gate and whose permissions allow the Edge Cookie (`EcContext::ec_allowed`), with or without an identity graph. With one, it records the request's identifiers on the cookie's row and writes the cookie once that row is authoritative, as before. Without one, it skips the graph steps and writes the cookie through `set_ec_cookie_on_response`, which takes the request's public prefix and, on a request addressed by path, the removed prefix as its `Path`, as core writes every cookie it owns. So the invariant of `2026-07-10-kv-eid-request-snapshot-ec-recovery-design.md` that a cookie is emitted only after its backing row exists now holds only where there is a graph, for the resolve route as for finalizing. Core reads the bot gate from the device signals the adapter sets on the context (`EcContext::passed_bot_gate`, which a request with no device signals does not pass), because a missing graph no longer stops the cookie and so can no longer stand in for the gate. The Fastly adapter still opens no graph for a request the gate stops, so no graph row is written for it. | Step | Fastly | Axum, Cloudflare and Spin | | ------------------------------------ | ------------------------------------------------------------------------------------------------- | -------------------------------------------------------- | @@ -52,10 +52,11 @@ Core writes a created Edge Cookie only with its identity-graph row. `ec_finalize **Consent and privacy.** A created Edge Cookie is written only where core's permission gate allows the Edge Cookie module and the request passed the bot gate, on every host, with a graph or without. A withdrawal expires the cookie and its markers, as on Fastly, and there is no graph row to tombstone. Without a graph no partner identifier is written against the Edge Cookie, and no pull sync, batch sync or identify call reads one. -The identifier itself leaves the edge without a graph as it does with one. Where the request's permissions set what the Edge Cookie module requires together with storage and personalized-ad selection (`EcContext::ec_sharing_allowed`, being `StoreOnDevice` and `SelectPersonalisedAds`), the publisher path's server-side auction and `GET /_ts/page-bids` send it to each auction provider as the auction request's `user.id`, and as `ts-` in that request's `id`, and `POST /auction` sends it as `user.id` (`publisher.rs`, `auction/endpoints.rs` and `auction/formats.rs` in core). A module call handed the request's state, such as a module route's handler or a document's serve-phase preparation, can read it behind the permissions the Edge Cookie module declares (`ModuleContext::with_request_state`), which for `hmac` is storage alone. That includes the identifier the integration proxy creates for a navigation to a module route by any client whose permissions allow the Edge Cookie, which the route's module call can read even when the bot gate stopped the request, though no response writes it, as on Fastly. The sample rules set storage and personalized-ad selection in the US with no signal. So this change discloses more in two cases. +The identifier itself leaves the edge without a graph as it does with one. Where the request's permissions set what the Edge Cookie module requires together with storage and personalized-ad selection (`EcContext::ec_sharing_allowed`, being `StoreOnDevice` and `SelectPersonalisedAds`), the publisher path's server-side auction and `GET /_ts/page-bids` send it to each auction provider as the auction request's `user.id`, and as `ts-` in that request's `id`, and `POST /auction` sends it as `user.id` (`publisher.rs`, `auction/endpoints.rs` and `auction/formats.rs` in core). A module call handed the request's state, such as a module route's handler or a document's serve-phase preparation, can read it behind the permissions the Edge Cookie module declares (`ModuleContext::with_request_state`), which for `hmac` is storage alone. That includes the identifier the integration proxy creates for a navigation to a module route by any client whose permissions allow the Edge Cookie, which the route's module call can read even when the bot gate stopped the request, though no response writes it, as on Fastly. The sample rules set storage and personalized-ad selection in the US with no signal. So this change discloses more in three cases. -- On Axum, Cloudflare and Spin the publisher path created no identifier and no response wrote one, so an auction there carried a `user.id` only for a browser that already carried an Edge Cookie, which nothing on these adapters wrote. Now a real browser's first page navigation creates one before the publisher path runs its auction, and the same value reaches the auction providers on that page view and on every later one whose permissions set both, for the year the cookie lasts. +- On Axum, Cloudflare and Spin the publisher path created no identifier and no response wrote one, so an auction there carried a `user.id` only for a browser that already carried an Edge Cookie, which nothing on these adapters wrote. Now a server-side module, such as `hmac`, creates one on a real browser's first page navigation before the publisher path runs its auction, and the same value reaches the auction providers on that page view and on every later one whose permissions set both, for the year the cookie lasts. - On a Fastly service with no `ec_store`, `hmac` created a new value on every page view, the HMAC of the client IP with a fresh random suffix, and no response wrote it, so the `user.id` of each page view was a new value whose 64-character HMAC part was the same for every page view from one address. Now one value follows the browser across page views and addresses for the year the cookie lasts. +- A client-side module, such as `client_fixed`, gave a browser no identifier on Axum, Cloudflare and Spin or on a Fastly service with no `ec_store`, because its `generate` creates nothing on a page navigation, those three adapters had no resolve route, and the Fastly route answered `204` with no graph. Now the resolve route sets the Edge Cookie with no graph for a browser that passes the bot gate where the permissions allow the module, and every later auction whose permissions set both sends that value to the auction providers as `user.id`, for the year the cookie lasts. This disclosure is intended. It is what a Fastly service with an `ec_store` does for the same browser, because the Edge Cookie is the identifier and the permission model, not the identity graph, decides whether it is shared. @@ -63,9 +64,11 @@ This disclosure is intended. It is what a Fastly service with an `ec_store` does **Rollout.** A deployment that selects no Edge Cookie module creates and writes no Edge Cookie, as before, but finalizing now runs on these adapters, so there a withdrawal expires an Edge Cookie or pull-sync marker the browser carries, as on Fastly. One that selects a server-side module, such as `hmac`, now sets the Edge Cookie on a real browser's first page navigation where the permissions allow it, on these adapters and on a Fastly service with no `ec_store`, and that response is kept out of shared caches because it sets a cookie (`enforce_set_cookie_cache_privacy`). A returning browser keeps its cookie while no identity graph sees it. A client-side module, such as `client_fixed`, gets its cookie from the resolve route on those hosts too. -**A graph configured later replaces these cookies.** When a Fastly service that ran with no `ec_store` names one, or a deployment on one of these adapters moves to a Fastly service with a graph, each browser holding a cookie written without a graph gets a new identifier on the first page navigation it makes there that passes the bot gate and whose permissions allow the Edge Cookie. The publisher path reads the cookie's row for that navigation, because no valid pull-sync marker lets it skip the read and a marker is only set from a row, and finalizing proves the row absent and gives the browser a new identifier with a row (`confirm_then_recover_orphaned_ec` in `ec/finalize.rs`). This design keeps that rather than have the graph adopt such a cookie by creating its row. An identifier a browser carries cannot be authenticated, because the `hmac` module's random suffix is unsigned, so the graph cannot tell a cookie written without a graph from one a client made up or one whose withdrawal tombstone, which lasts 24 hours, has expired, and adopting it would start a graph row from an unverified cookie, which `2026-07-10-kv-eid-request-snapshot-ec-recovery-design.md` rules out. A deployer that adds a graph later therefore resets each reader's identifier once, and the `user.id` the auction providers see changes with it. +**A graph configured later replaces a server-side module's cookies.** When a Fastly service that ran with no `ec_store` names one, or a deployment on one of these adapters moves to a Fastly service with a graph, each browser holding a cookie written without a graph by a server-side module such as `hmac`, which creates its identifier at the edge, gets a new identifier on the first page navigation it makes there that passes the bot gate and whose permissions allow the Edge Cookie. The publisher path reads the cookie's row for that navigation, because no valid pull-sync marker lets it skip the read and a marker is only set from a row, and finalizing proves the row absent and gives the browser a new identifier from the module, with a row (`confirm_then_recover_orphaned_ec` in `ec/finalize.rs`). This design keeps that rather than have the graph adopt such a cookie by creating its row. An identifier a browser carries cannot be authenticated, because the `hmac` module's random suffix is unsigned, so the graph cannot tell a cookie written without a graph from one a client made up or one whose withdrawal tombstone, which lasts 24 hours, has expired, and adopting it would start a graph row from an unverified cookie, which `2026-07-10-kv-eid-request-snapshot-ec-recovery-design.md` rules out. A deployer that adds a graph later therefore resets once the identifier of each reader holding such a cookie, and the `user.id` the auction providers see changes with it. -**Crates outside core.** A crate outside core that finalizes the Edge Cookie or answers the resolve route sets the device signals on the context, or no created cookie is written. This is breaking for such a crate, and no compiler catches it. `generate_if_needed` does not read the bot gate, so such a crate that passes a graph and sets no device signals gets a graph row for an identifier no response writes. The adapters in this repository cannot reach that, because Fastly passes no graph for a request the gate stops and the others pass none. +**A client-side module's cookie keeps its identifier with no row.** A client-side module, such as `client_fixed`, creates nothing at the edge, so when finalizing proves such a cookie's row absent, recovery gets no identifier from the module, logs that and leaves the cookie as it is. Nothing else gives that cookie a row. Organic creation does nothing for a request that carries an identifier, a partner ID is not written against a missing row, and the page script posts to the resolve route, where such a module's rows are created, only while the resolved marker the route set beside the cookie, with the same one-year lifetime, is absent. So once a graph is configured, a cookie such a module got without one keeps its identifier with no row, the graph keeps nothing for that reader for as long as the browser keeps the cookie and its marker, and each of that reader's page navigations that passes the bot gate and whose permissions allow the module pays for a second point read and a strongly consistent existence check, a strong-mode list on Fastly, before recovery gives up again. Giving such a cookie its row would start a row from a cookie the graph cannot verify, the reason a graph does not adopt a server-side module's cookie, so it is listed under What is not in this change. + +**Crates outside core.** A crate outside core that finalizes the Edge Cookie or answers the resolve route sets the device signals on the context, or no created cookie is written. This is breaking for such a crate, and no compiler catches it. `generate_if_needed` does not read the bot gate, so such a crate that passes a graph and sets no device signals gets a graph row for an identifier no response writes. The adapters in this repository cannot reach that, because Fastly passes no graph for a request the gate stops and the others pass none. Orphan recovery reads whether the crate let the request recover (`EcContext::set_recovery_eligible`) rather than the bot gate, so such a crate that lets a request recover without its own gate having checked it gets a replacement cookie written with no device signals. In this repository only Fastly lets a request recover, and only for a real browser's publisher navigation. **Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, through the adapter's router, with a stub origin and a reader whose services place them in the US and carry the permission signal modules the adapter links, where the sample permission rules grant the Edge Cookie with no signal, and `[geo] module = "platform"`, which leaves that location standing: @@ -74,6 +77,7 @@ This disclosure is intended. It is what a Fastly service with an `ec_store` does - `a_bot_s_page_view_gets_no_edge_cookie`: a page view whose `User-Agent` names a crawler gets none. - `a_withdrawal_expires_the_edge_cookie_the_browser_carries`: with no `[geo]` module instead, so the reader resolves at the top of the sample rules, where storage needs a signal, a page view carrying `ts-ec` and a TC String that refuses storage gets `ts-ec` expired with `Max-Age=0`. - `the_resolve_route_answers`: a foreign `Origin` gets `403` with `Cache-Control: no-store` and the publisher's gets `204` with no cookie, the answer of a server-side module, which creates nothing from a posted value, and neither reaches the origin. +- `the_resolve_route_sets_a_client_side_module_s_edge_cookie`: with a test module selected by `[ec] module` that creates nothing at the edge and takes its identifier from the posted value, as a client-side module does, the publisher's post of that value gets `200` with `ts-ec`, its `ts-ecr` marker and `Cache-Control: no-store`, and does not reach the origin. - `startup_warns_that_this_adapter_opens_no_identity_graph`: building the state with `[ec] ec_store` set sets the flag the warning is logged under, and the warning's check with no `ec_store` sets none. The flag is set as the line is logged, so the test shows the warning's branch runs, not that a line reaches the log. The tests' `edge_cookie` helper also fails a response that writes the Edge Cookie more than once, so the first page view shows core alone writes it. @@ -86,16 +90,17 @@ In core: - `resolve_creates_nothing_for_a_request_that_did_not_pass_the_bot_gate` in `ec/resolve.rs`: a crawler, or a request with no device signals, gets `204` with no cookie and no row, with a graph and without. - `the_bot_gate_passes_a_browser_and_stops_a_crawler_or_a_request_with_no_signals` in `ec/mod.rs`. - `finalize_rotates_orphaned_cookie_to_new_backed_ec` in `ec/finalize.rs`, which this change leaves as it is: a well-formed cookie whose row is missing, finalized with a graph on a page view that may recover, gets a new identifier with a row, which is the replacement Rollout describes. +- `an_orphan_recovery_that_cannot_rotate_leaves_a_failed_snapshot` in `ec/finalize.rs`, which gains a client-side module's case: a `client_fixed` cookie whose row is missing, finalized with a graph on a page view that may recover, keeps its identifier with no cookie written and a failed snapshot for its key, which is the case Rollout describes for such a module. In the Fastly adapter, under Viceroy, `with_no_ec_store_the_entry_point_writes_a_created_edge_cookie` in `src/app.rs`: with no `[ec] ec_store`, `build_ec_request_state` opens no graph for a browser's page view, `generate_if_needed`, the step `dispatch_fallback` runs for one, creates an Edge Cookie, and `apply_edgezero_ec_finalize`, the entry point's finalize step, writes it once. The test calls the three steps in turn rather than routing a page view through `dispatch_fallback` and `edgezero_main`. -No adapter test sees a `200` from the resolve route, because the adapter tests select `hmac`, which creates nothing from a posted value, and the `client_fixed` module is compiled only behind core's `client-fixed-demo` feature, which no adapter passes on. Core's `resolve_answers_as_it_does_with_a_graph_when_there_is_none` covers that answer with no graph, and `the_resolve_route_answers` covers the route on each adapter. +The adapters' test of a `200` from the resolve route supplies its own client-side module through a registration, as a crate outside core would, because the `client_fixed` module is compiled only behind core's `client-fixed-demo` feature, which no adapter passes on. **Decisions.** Two questions this design left open, whether the resolve route should write a client-side module's cookie on an adapter that opens no graph and whether a Fastly service with no `ec_store` should write a created Edge Cookie, are both answered yes. The Edge Cookie is the identifier, and the identity graph is one store of the data associated with it. The permission state comes from the permission model, whose sources may be values the graph holds, signals the client sends or any other source, so a missing graph does not stop the cookie. So the Edge Cookie is the identifier, the identity graph is one optional store of the data associated with it, the partner IDs among it, and the permission state comes from the permission model whatever its sources. A missing graph does not stop the cookie. -Where a graph is configured it still replaces an identifier it has no row for, as Rollout describes. That is the graph refusing to start a row from an identifier it cannot verify, not a condition on the cookie, and without a graph nothing replaces a cookie. +Where a graph is configured it still replaces a server-side module's identifier it has no row for, and leaves a client-side module's identifier with no row, as Rollout describes. Both are the graph refusing to start a row from an identifier it cannot verify, not a condition on the cookie, and without a graph nothing replaces a cookie. ## 3. What is not in this change @@ -103,12 +108,12 @@ Where a graph is configured it still replaces an identifier it has no row for, a - `/_ts/api/v1/identify`, `/_ts/api/v1/batch-sync` and pull sync on those adapters, which need the graph, and the tester routes, which no other adapter registers. - Asset routes on these adapters, which the next pull request of the stack covers. - A graph adopting an Edge Cookie it has no row for by creating the row instead of replacing the identifier, which needs a way to tell a cookie written without a graph from one a client made up or one whose withdrawal tombstone has expired, such as a signed identifier. +- Giving a client-side module's cookie written without a graph its row once a graph is configured, where recovery has no identifier to replace it with. It needs the same way to tell such a cookie from one a client made up or one whose withdrawal tombstone has expired, because the row would otherwise start from an unverified cookie. - A bot gate in `generate_if_needed`, which would keep a crate outside core that passes a graph and sets no device signals from writing a row for an identifier no response writes. -- A `200` from the resolve route in the adapters' tests, which needs a client-side module those tests can select. ## 4. Acceptance -The six tests pass on each of the three adapters, the core and Fastly tests above pass, and the existing tests of each adapter, core's Edge Cookie tests and the cross-adapter parity tests pass. +The seven tests pass on each of the three adapters, the core and Fastly tests above pass, and the existing tests of each adapter, core's Edge Cookie tests and the cross-adapter parity tests pass. ## Revision record @@ -117,7 +122,9 @@ The six tests pass on each of the three adapters, the core and Fastly tests abov | 2026-10-17 | First draft. | | 2026-10-18 | Core writes a created Edge Cookie with or without an identity graph, behind the bot gate it reads from the device signals, and the resolve route answers without a graph as it does with one, which answers both open questions. | | 2026-10-18 | Fixes from review round 1. The identifier's disclosure to auction providers and module calls with no graph and the two cases where it grows, the replacement of these cookies once a graph sees them and why a graph does not adopt one, the integration proxy's creation the three adapters already ran, the bot gate on these adapters, the order of the resolve route's checks, the store's methods, the log line, a deployment with no module, crates outside core, a test of a withdrawal through each router, the startup warning's case with no `ec_store`, what the Fastly test does, the answer no adapter test covers, what is not in this change and the review log. | +| 2026-10-18 | Fixes from review round 2. A client-side module's cookie, which a graph configured later leaves with no row, and what that costs, the replacement limited to a server-side module's cookie, the third case where the disclosure grows, the 2026-07-10 invariant that now holds only with a graph, orphan recovery for crates outside core, a test of a `200` from the resolve route through each adapter's router, a client-side module's case in core's orphan recovery test, what is not in this change and the review log. | ## Review log -1. **Round 1, 2026-10-18.** A review in a fresh session, against `fixpass6/53-edge-cookie` at `198ce56db`. Issues found, two of them blocking. The two were that the spec said the Edge Cookie stays a first-party identifier in the browser without a graph, when core sends it to the auction providers as `user.id` and in the auction request's `id` and lets module calls read it, so this change gives the providers one value for a year where they had none or a new one on each page view, and that the spec said a returning browser keeps its cookie without saying that a graph configured later replaces every cookie written without one, which Decisions also left unexplained. Fixed in `960d9c61f`, which corrects the changelog and the Edge Cookie and Fastly guides, in `dbbbcfa14`, which tests a withdrawal through each adapter's router and the startup warning with no `ec_store`, and in this revision, which states the disclosure and why it is intended, states the replacement and why the graph keeps it rather than adopt such a cookie, and fixes the non-blocking findings that were cheap and certain. A graph that adopts such a cookie, a bot gate in `generate_if_needed` and a `200` from the resolve route in the adapters' tests are listed under What is not in this change. +1. **Round 1, 2026-10-18.** A review in a fresh session, against `split/53-edge-cookie` at `198ce56db`. Issues found, two of them blocking. The two were that the spec said the Edge Cookie stays a first-party identifier in the browser without a graph, when core sends it to the auction providers as `user.id` and in the auction request's `id` and lets module calls read it, so this change gives the providers one value for a year where they had none or a new one on each page view, and that the spec said a returning browser keeps its cookie without saying that a graph configured later replaces every cookie written without one, which Decisions also left unexplained. Fixed in `960d9c61f`, which corrects the changelog and the Edge Cookie and Fastly guides, in `dbbbcfa14`, which tests a withdrawal through each adapter's router and the startup warning with no `ec_store`, and in `3c9e9f913`, which states the disclosure and why it is intended, states the replacement and why the graph keeps it rather than adopt such a cookie, and fixes the non-blocking findings that were cheap and certain. A graph that adopts such a cookie, a bot gate in `generate_if_needed` and a `200` from the resolve route in the adapters' tests are listed under What is not in this change. +2. **Round 2, 2026-10-18.** A review in a fresh session, against `split/53-edge-cookie` at `3c9e9f913`, of the round 1 fixes and the new text. Issues found, two of them blocking. The two were that the spec, the changelog and both guides said a graph configured later replaces each cookie written without one, when a client-side module creates nothing at the edge, so recovery leaves its cookie with no row and each of that reader's page navigations pays for a second read and an existence check, and that the spec listed two cases where the disclosure grows, leaving out a client-side module on a host with no graph, whose readers now get an identifier from the resolve route where they got none. Fixed in `fa4181b19`, which corrects the changelog, the Edge Cookie, Fastly and API reference guides and the page script's comment, in `fd7227aa1`, which tests a `200` from the resolve route through each adapter's router and a client-side module's orphaned cookie in core, and in this revision, which limits the replacement to a server-side module, states the client-side case and its cost, adds the third case, and fixes the non-blocking findings that were cheap and certain. Giving a client-side module's cookie its row is listed under What is not in this change, beside a graph that adopts a cookie. From cf1a240df982ce3df2722ba336f328f3900c2619 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 08:37:10 +0100 Subject: [PATCH 163/192] The asset cache design no longer names a cache setting that does not exist The design said both caches are built with detachable: false, a field no code has. It now says a load runs in the request that started it, as every load of the loading cache does. --- docs/superpowers/specs/2026-10-15-asset-cache-design.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-10-15-asset-cache-design.md b/docs/superpowers/specs/2026-10-15-asset-cache-design.md index 4192511641..da815483ac 100644 --- a/docs/superpowers/specs/2026-10-15-asset-cache-design.md +++ b/docs/superpowers/specs/2026-10-15-asset-cache-design.md @@ -76,7 +76,7 @@ A stored copy is the origin's response, headers and body, as the origin sent it, **Asset rules.** The `[[cache.asset_rules]]` entries rewrite the `Cache-Control` and edge cache headers of the response a reader is sent, after a copy is read or the origin is asked (`apply_publisher_asset_cache_policy` on the publisher path and the end of `handle_asset_proxy_request` on a route). The asset gate and the copy's lifetime do not read the rules, so the lifetime comes from the origin's response alone, and a rule that gives a reader a longer lifetime does not lengthen the copy's. -**One fetch for many requests.** `get_or_fetch` loads the address's record through `assets-vary`. The request that leads that load fetches the origin and writes its own copy under its own values before the record is published, so a waiting request with the same values finds the copy. A waiting request with other values loads its own copy, and the requests waiting on those values share that load. Both caches are built with `detachable: false`, so a load runs in the request that started it. +**One fetch for many requests.** `get_or_fetch` loads the address's record through `assets-vary`. The request that leads that load fetches the origin and writes its own copy under its own values before the record is published, so a waiting request with the same values finds the copy. A waiting request with other values loads its own copy, and the requests waiting on those values share that load. A load runs in the request that started it, as every load of the loading cache does. **How far one fetch reaches.** The memory layer shares a load among the requests that share the layer. The stock module's layer lives as long as the application state. Fastly serves one request per instance unless the opt-in reusable sandbox is on, and Cloudflare and Spin build the application state for every request (see `AppState` in the three adapters' `app.rs`), so on those three the memory layer is not shared between requests. Across instances only Fastly makes requests share a load. Only the Fastly adapter registers a value cache, and a lookup in that cache makes every other instance at the same location wait for the instance that fills the key (`crates/trusted-server-adapter-fastly/src/value_cache.rs`), so a missing key is loaded once for each location and not once for the service. Cloudflare and Spin share copies only through the key-value store each adapter declares, `TRUSTED_SERVER_KV` and `default`, which makes no request wait, so each request that misses at the same moment fetches the origin itself and later requests read the stored copy, and Axum has no shared store at all. From 893eacc641b6ebcb40c182f29ba0a9e7edd3a6c3 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 08:37:12 +0100 Subject: [PATCH 164/192] The asset routes design answers review round 2 The Cloudflare and Spin adapters install a logger of their own, so the warning for a route that enables the Image Optimizer and the error for Spin's refusal of a host_header naming another host reach each host's log, and the design says where. Spin's refusal is marked breaking for the publisher path too, the warning is weighed against the auction refusal, and the review log records round 2. --- ...17-asset-routes-on-every-adapter-design.md | 50 ++++++++++++------- 1 file changed, 33 insertions(+), 17 deletions(-) diff --git a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md index 7f9955d223..fb69795291 100644 --- a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md @@ -7,7 +7,7 @@ **Issues:** none **Last updated:** 2026-10-18 -> **Why this spec exists.** Only the Fastly adapter serves `[[proxy.asset_routes]]`. On Axum, Cloudflare and Spin a request under a route's prefix goes to the publisher's origin as an ordinary page request, so a route's own source, its signing and its held answers are never used there. This change dispatches asset routes on those three adapters as the Fastly adapter does. +> **Why this spec exists.** Only the Fastly adapter serves `[[proxy.asset_routes]]`. On Axum, Cloudflare and Spin a request under a route's prefix goes to the publisher's origin as an ordinary page request, so a route's own source, its signing and its held answers are never used there. This change dispatches asset routes on those three adapters as the Fastly adapter does. Two parts of it reach beyond asset routes. On Spin a source whose `host_header` names another host is now refused, the publisher's source as well as a route's, so a Spin deployment whose publisher source sets one answers every page with `502`, which is a breaking change. And the Cloudflare and Spin adapters now install a logger, because neither had one and every line they logged was dropped, the warnings this change adds included. ## 1. Problem @@ -15,6 +15,10 @@ No code in the Axum, Cloudflare or Spin adapter calls `handle_asset_proxy_request`. `docs/guide/api-reference.md` lists the route as Fastly's alone, and the Cloudflare HTTP client's note says these requests are not routed to it. +The Spin client sends the address's own host as the `Host` of every request, so a source whose `host_header` names another host reaches its origin with the address's host in place of the one it names. That holds for the publisher's `[source.origin]` as much as for a route's source, so the refusal this change adds on Spin (section 2) changes the publisher path as well as the asset routes. + +Neither the Cloudflare adapter nor the Spin adapter installs a logger. Each hands every request to EdgeZero's `run_app`, whose `init_logger` does nothing on Workers or on Spin at the locked v0.0.9 (`edgezero-adapter-cloudflare/src/lib.rs` and `edgezero-adapter-spin/src/lib.rs`), so the `log` crate drops every line the two adapters write, including the warnings they log as they build their state, a startup error and the report behind each error answer. + ## 2. Design **Dispatch.** Each adapter's `dispatch_fallback` checks a `GET` or `HEAD` against the asset routes where the Fastly adapter does, and `dispatch_asset_fallback` follows the Fastly function of that name: the route's source, `handle_asset_proxy_request`, the response's cache policy in its extensions, and the request filters' effects. An error from the handler answers with the status the error's kind gives and the `no-store, private` policy, as on Fastly. No Edge Cookie state is attached, so the finalize middleware neither writes nor expires an Edge Cookie on an asset response. The Fastly adapter is unaffected, because this change touches no file of the Fastly adapter or of core. @@ -25,7 +29,7 @@ No code in the Axum, Cloudflare or Spin adapter calls `handle_asset_proxy_reques **The body.** None of the three HTTP clients streams a response (`supports_streaming_responses` is `false` there), so core asks for no stream and each client holds the asset body whole, where the Fastly adapter streams it with no limit. The Cloudflare client refuses a body over 10 MiB (`MAX_PLATFORM_RESPONSE_BODY_BYTES`) and the Spin client one over 8 MiB, before or after it is decompressed (`MAX_DECOMPRESSED_SIZE`), and the Axum client holds any size. The Cloudflare client writes the length of the body it read as the `Content-Length`, so its answer to a `HEAD` says `0`, where the Fastly and Spin adapters keep the origin's length. -**A source's `Host`.** A route's source can name the `Host` its origin is asked with (`host_header`), which core writes before the signer signs it. The Fastly adapter sends it through its backend's host override, and the Axum client hands it to its HTTP library, which adds a `Host` only to a request that has none. Spin sends the address's own host as the `Host` of every request, because WASI HTTP takes the authority from the address and forbids a `host` header, so a source whose `host_header` names another host would reach its origin with the wrong `Host`, and a signed one with a signature made for another. The Spin backend (`NoopBackend::ensure` in `crates/trusted-server-adapter-spin/src/platform.rs`) therefore refuses to name a backend asked for a `Host` other than the address's own, with its port where that is not the scheme's default, and core answers the request with `502` before anything is sent. The publisher's own source reaches the same backend, so a publisher source with such a `host_header` is refused on Spin too. The Cloudflare client hands the header to `fetch` as given, and Cloudflare documents that a Worker cannot set a `Host` naming a host outside its zone unless the request goes to that host ([Request](https://developers.cloudflare.com/workers/runtime-apis/request/)), so there a `host_header` naming such a host does not reach the origin. +**A source's `Host`.** A route's source can name the `Host` its origin is asked with (`host_header`), which core writes before the signer signs it. The Fastly adapter sends it through its backend's host override, and the Axum client hands it to its HTTP library, which adds a `Host` only to a request that has none. Spin sends the address's own host as the `Host` of every request, because WASI HTTP takes the authority from the address and forbids a `host` header, so a source whose `host_header` names another host would reach its origin with the wrong `Host`, and a signed one with a signature made for another. The Spin backend (`NoopBackend::ensure` in `crates/trusted-server-adapter-spin/src/platform.rs`) therefore refuses to name a backend asked for a `Host` other than the address's own, with its port where that is not the scheme's default, and core answers the request with `502` before anything is sent. Nothing checks the setting at startup, so a deployment learns of the refusal from the first request refused, through the error line the component logs, which names both hosts (Logging). The publisher's own source reaches the same backend, so a publisher source with such a `host_header` is refused on Spin too. The backend accepts a `host_header` that names the address's own host in other letter case or with the scheme's default port written, and Spin then sends the address's host, while `s3_sigv4` signs the `Host` as the source writes it (`canonical_headers` in `crates/shared/aws/src/sigv4.rs`), so a signed source written that way still fails its origin's signature check. That was so before this change too, and a `host_header` written in lower case without the default port, or none, avoids it. The Cloudflare client hands the header to `fetch` as given, and Cloudflare documents that a Worker cannot set a `Host` naming a host outside its zone unless the request goes to that host ([Request](https://developers.cloudflare.com/workers/runtime-apis/request/)), so there a `host_header` naming such a host does not reach the origin. **Image Optimizer.** A route that enables `[proxy.asset_routes.image_optimizer]` sets Image Optimizer metadata on each request whose path is not an SVG or SVGZ file and whose query does not set the profile set's debug parameter to `1` (`asset_path_skips_image_optimizer` and `options_for_asset_request` in core). The Fastly adapter translates the metadata, and the other three cannot. The Cloudflare and Spin HTTP clients already refused such a request on both send paths, as the `PlatformHttpRequest` contract asks of a client that cannot honor it, and the Axum client now refuses it on `send` and `send_async`, where it sent the request unchanged before. A refused request answers `502 Bad Gateway` with `no-store, private`. Before the refusal, a route whose source signs asks its origin for the object with a `HEAD` (`preflight_signed_origin_for_image_optimizer`), and where that `HEAD` fails the request is answered with the origin's own error, with no metadata and with `no-store, private`, as on Fastly. An SVG path and a debug request carry no metadata, so they are served unoptimized and their answers are kept as any other asset's are. @@ -36,9 +40,13 @@ No code in the Axum, Cloudflare or Spin adapter calls `handle_asset_proxy_reques | Cloudflare | Yes | Not sent outside the Worker's zone | Loading caches, copies in `TRUSTED_SERVER_KV` | Held whole, up to 10 MiB | Refused by the client, `502` | | Spin | Yes | Refused by the backend, `502` | Loading caches, copies in `default` | Held whole, up to 8 MiB | Refused by the client, `502` | -**Configuration.** No key is added or changed, and `[[proxy.asset_routes]]` now takes effect on every adapter. At startup the Axum, Cloudflare and Spin adapters each log one warning naming the prefix of every route that enables the Image Optimizer (`warn_once_when_an_asset_route_enables_the_image_optimizer`), and then serve the route. They warn rather than refuse the settings, because such a route still answers its SVG paths, its debug requests and, where it signs, the origin's own errors, because the deployment's other routes and pages do not depend on it, and because a refusal would stop a Spin or Cloudflare deployment, or an Axum dev server, that shares its settings with a Fastly service whose routes optimize images. +**Logging.** Each of the Cloudflare and Spin adapters now installs a small logger of its own at its entry point, before `run_app` (`init_logger` in `src/logging.rs` of each adapter), and says so through `Hooks::owns_logging`, so `run_app` does not try to install one. The Worker writes each line through the `worker` crate's console macros by level, `console.error`, `console.warn` and, for `info`, `console.log`, with `console.debug` for the two levels below, which it does not write, so `wrangler dev` prints a line, and for a deployed Worker `wrangler tail` and the dashboard's live logs show it, as Workers Logs does where the Wrangler file enables `[observability]`. The Spin component writes each line to standard error, which `spin up` prints in its console and keeps in the `.spin/logs` folder beside a local manifest, and a host that runs the component another way decides where standard error goes. A line gives the level and the module before the message. Both loggers write `info` and above, the level the Axum adapter writes by default. Each is installed once for an instance, because the entry point runs for every request and a call that finds a logger installed changes nothing. With them every adapter logs the full error of a state that cannot be built, which `2026-10-14-origin-signing-design.md` says of a startup failure, and the report behind each error answer, which on Spin names the `Host` refused. + +**Configuration.** No key is added or changed, and `[[proxy.asset_routes]]` now takes effect on every adapter. As it builds its state, each of the Axum, Cloudflare and Spin adapters logs a warning naming the prefix of every route that enables the Image Optimizer (`warn_once_when_an_asset_route_enables_the_image_optimizer`), and then serves the route. A flag keeps the warning to one for each instance of the adapter. Axum builds its state once for the process, so it warns once, at startup. Cloudflare and Spin build their state for every request, so there the flag is what keeps the warning from every request. A Workers instance can serve many requests ([How Workers works](https://developers.cloudflare.com/workers/reference/how-workers-works/)), and Spin can serve several requests from one instance of the component, one after another and at the same time, to a degree it does not guarantee ([HTTP trigger](https://spinframework.dev/v4/http-trigger)), so on Spin the warning appears each time Spin starts an instance, which is every request when an instance serves only one (`spin up --max-instance-reuse-count`). -**Consent and privacy.** An asset route forwards only `ASSET_PROXY_FORWARD_HEADERS` to the route's origin, plus the `Host` and the signer's headers, as on Fastly, so the reader's cookies are not sent there. Before this change the request went to the publisher's origin with the headers the publisher path forwards. An asset response skips the Edge Cookie steps (`ec_finalize_response` does not run), so it neither writes a created Edge Cookie nor expires the cookie and its markers when the reader has withdrawn permission, which matches Fastly. Before this change the publisher fallback did both for the same requests on these adapters, and a response from the publisher path still does. +They warn rather than refuse the settings, for three reasons. Such a route still answers its SVG paths, its debug requests and, where it signs, the origin's own errors. The deployment's other routes and pages do not depend on it. And a refusal would stop a Spin or Cloudflare deployment, or an Axum dev server, that shares its settings with a Fastly service whose routes optimize images. Cloudflare and Spin do refuse at startup an enabled auction with more than one provider (`AuctionPlan::validate_for_target`), although a Fastly service runs one, because such an auction can do nothing there. Their HTTP clients send one request at a time, and the auction orchestrator refuses an auction of more than one provider before any provider is asked, so those settings could run no auction at all, where a route that enables the Image Optimizer still answers its SVG paths and debug requests. The third reason holds for such an auction too, and the refusal stands there because no auction would run either way. + +**Consent and privacy.** An asset route forwards only `ASSET_PROXY_FORWARD_HEADERS` to the route's origin, plus the `Host` and the signer's headers, as on Fastly, so the reader's cookies are not sent there. Before this change the request went to the publisher's origin with the headers the publisher path forwards. A route's origin, which need not be the publisher's, now receives the reader's `User-Agent`, `Accept`, `Accept-Encoding`, `Accept-Language` and `Referer`, with the cache validation and range headers, and not the reader's address, because the forwarding headers are stripped at the edge and not rebuilt for an asset route. An asset response skips the Edge Cookie steps (`ec_finalize_response` does not run), so it neither writes a created Edge Cookie nor expires the cookie and its markers when the reader has withdrawn permission, which matches Fastly. Before this change the publisher fallback did both for the same requests on these adapters, and a response from the publisher path still does. **Failure handling.** @@ -51,15 +59,17 @@ No code in the Axum, Cloudflare or Spin adapter calls `handle_asset_proxy_reques - a backend the platform refuses, `502`, which on Spin is a source whose `host_header` names another host - an origin that cannot be reached, a body over the client's limit and a request the client refuses, such as one carrying Image Optimizer metadata, `502` - An origin's own error status is passed through as its answer, and no cache rule is applied to it. +- Every adapter logs each per-request error listed above at error level with its full report (`http_error`), which on Spin names both hosts of a refused `host_header`, and logs a state that cannot be built with its full error. On Cloudflare and Spin both kinds of error line were dropped before this change, with no logger to write them (Logging). - Core asks the route's backend for a first byte timeout and a between bytes timeout of 15 seconds (`DEFAULT_FIRST_BYTE_TIMEOUT`), which the Fastly adapter applies. The Axum client applies limits of its own instead, 5 seconds to connect and 30 seconds for the whole request, and the Cloudflare and Spin backends only make a name, so there a slow origin is bounded by the runtime's own limits. `2026-10-15-asset-cache-design.md` says the same of the asset caches. **Rollout.** -1. A deployment with no asset routes is unchanged, because `asset_route_index_for_path` answers `None` when there are none. -2. One with asset routes now has their `GET` and `HEAD` requests answered by each route's origin on Axum, Cloudflare and Spin, where the publisher's origin answered them, with the route's signing and cache rule, and with no Edge Cookie written or expired. -3. On those adapters a route that enables the Image Optimizer answers each request it would optimize with `502` and `no-store, private`, where the publisher's origin answered it before, after a signed route has sent its origin the `HEAD`. Its SVG paths and debug requests are served unoptimized, and startup logs a warning naming each such route. -4. On Spin a request to a source whose `host_header` names another host answers `502` and its origin is not asked, a route's source and the publisher's alike, where before the request reached the origin with the address's own host as the `Host`. A Spin deployment whose publisher source sets such a `host_header` therefore answers every page with `502`. +1. A deployment with no asset routes serves its requests as before, because `asset_route_index_for_path` answers `None` when there are none, with two exceptions. On Spin a publisher source whose `host_header` names another host is now refused (item 4), and on Cloudflare and Spin the lines the adapter logs are now written (item 6). +2. One with asset routes now has their `GET` and `HEAD` requests answered by each route's origin on Axum, Cloudflare and Spin, where the publisher's origin answered them, with the route's signing and cache rule, and with no Edge Cookie written or expired. A route reads and keeps copies for a request that carries a cookie, which the publisher path fetches for that request alone, so after the upgrade more requests read and write the key-value store, `TRUSTED_SERVER_KV` on Cloudflare and `default` on Spin. +3. On those adapters a route that enables the Image Optimizer answers each request it would optimize with `502` and `no-store, private`, where the publisher's origin answered it before, after a signed route has sent its origin the `HEAD`. Its SVG paths and debug requests are served unoptimized, and each adapter logs a warning naming each such route, once for each instance (Configuration). +4. On Spin a request to a source whose `host_header` names another host answers `502` and its origin is not asked, a route's source and the publisher's alike, where before the request reached the origin with the address's own host as the `Host`, and the component logs the refusal as an error naming both hosts. A Spin deployment whose publisher source sets such a `host_header` therefore answers every page with `502`, robots.txt included. The changelog gives this as a breaking change, with the check to make before upgrading. 5. A body over 10 MiB on Cloudflare or 8 MiB on Spin answers `502`. +6. On Cloudflare and Spin the lines the adapter logs at `info` and above are written, where every line was dropped, so the warnings above, the identity graph and analytics warnings these adapters already logged, a startup error and the report behind each error answer reach the Worker's console and the component's standard error. **Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, with a stub origin: @@ -68,13 +78,15 @@ No code in the Axum, Cloudflare or Spin adapter calls `handle_asset_proxy_reques - `an_asset_route_s_cache_rule_is_written_after_the_finalize_headers`: with a `[[cache.asset_rules]]` rule of a minute for browsers and an hour for shared caches, the answer carries `public, max-age=60, s-maxage=3600` on Axum and Spin, and `public, max-age=60` with `cloudflare-cdn-cache-control: max-age=3600` on Cloudflare. - `an_asset_route_whose_origin_fails_answers_502_and_is_not_stored`: an origin that fails gives `502` with `cache-control: no-store, private`. - `a_page_view_under_an_asset_route_gets_no_edge_cookie`: a browser's page navigation under the route, whose permissions would grant an Edge Cookie on a page, gets none, and the route's origin answers it. -- `startup_warns_when_an_asset_route_enables_the_image_optimizer`: building the router with such a route logs the warning. +- `startup_warns_when_an_asset_route_enables_the_image_optimizer`: building the router with such a route sets the flag the warning function sets as it logs the warning, and the test reads the flag rather than a log line. The Spin adapter's request cycle tests also send requests through the backend its own services use. In `an_asset_route_whose_source_names_another_host_is_refused_before_its_origin_is_asked` the route whose source names another host answers `502` with `no-store, private` and only the route without one reaches its origin, and in `a_publisher_source_that_names_another_host_is_refused_before_its_origin_is_asked` a page request answers `502` and the publisher's origin is not asked. Its platform tests `a_backend_asked_to_send_another_host_is_refused` and `a_backend_asked_to_send_the_address_s_own_host_is_named` cover another name, another port and the same host written with its default port or in other letter case. The Axum adapter's platform test `a_request_with_image_optimizer_metadata_is_refused` sends through `send` and `send_async`. -Each test was run with the code it guards taken out, and failed. Without the finalize middleware's cache policy step the cache rule test sees `public, max-age=60` and no shared lifetime, and the error test sees no `Cache-Control`, on each adapter, and so does the Spin route refusal test, which also checks `no-store, private`. Without `NoStorePrivate` in the error arm the error test fails on each adapter, an asset response given the publisher path's Edge Cookie state writes a `ts-ec` cookie and fails the Edge Cookie test on each adapter, and a state builder without the warning call fails the warning test on each adapter. With Spin's refusal disabled, `a_backend_asked_to_send_another_host_is_refused` fails and both Spin request cycle tests answer `200`, while `a_backend_asked_to_send_the_address_s_own_host_is_named` still passes, and with the `send_async` check taken out the Axum client test fails at its `send_async` assertion. +The Cloudflare and Spin loggers are tested natively. In the Cloudflare adapter's `src/logging.rs`, `each_level_is_written_with_the_console_method_for_it`, `a_line_gives_the_level_and_target_before_the_message` and `info_and_above_are_written_and_debug_and_trace_are_not` cover the console method, the line and the level, and in the Spin adapter's, `a_record_is_written_as_one_line_with_its_level_and_target` and `info_and_above_are_written_and_debug_and_trace_are_not` cover the line and the level. In `tests/logging.rs` of each adapter, a test binary of its own because a logger is global to a process, `the_logger_is_installed_once_at_info_and_above` calls `init_logger` twice and finds one logger installed that writes `info` and above and leaves debug and trace out, and `the_app_owns_its_logging` checks `Hooks::owns_logging`. The entry points that call `init_logger` compile for wasm32 alone, which `cargo clippy-cloudflare-wasm` checks for the Worker and `cargo clippy-spin-wasm` for the component. + +Each test was run with the code it guards taken out, and failed. Without the finalize middleware's cache policy step the cache rule test sees `public, max-age=60` and no shared lifetime, and the error test sees no `Cache-Control`, on each adapter, and so does the Spin route refusal test, which also checks `no-store, private`. Without `NoStorePrivate` in the error arm the error test fails on each adapter, an asset response given the publisher path's Edge Cookie state writes a `ts-ec` cookie and fails the Edge Cookie test on each adapter, and a state builder without the warning call fails the warning test on each adapter. With Spin's refusal disabled, `a_backend_asked_to_send_another_host_is_refused` fails and both Spin request cycle tests answer `200`, while `a_backend_asked_to_send_the_address_s_own_host_is_named` still passes, and with the `send_async` check taken out the Axum client test fails at its `send_async` assertion. With `init_logger` installing nothing, as EdgeZero's does, `the_logger_is_installed_once_at_info_and_above` fails on each adapter with the level still `Off`, and with a second call that fails rather than being ignored it fails at that call with `SetLoggerError`. With `owns_logging` left at its default, `the_app_owns_its_logging` fails on each adapter. With the level raised to `debug`, the level test fails on each adapter, with `info` written through `console.debug` the Cloudflare console method test fails, and without its line ending the Spin line test fails. -No test sends an asset request through the Cloudflare or Spin HTTP client, whose refusals, body limits and `Content-Length` handling are wasm32 code that the native tests do not run, apart from Spin's `reject_unsupported_request_contracts`, which has native tests of its own. The cross-adapter parity tests have no asset route case. +No test sends an asset request through the Cloudflare HTTP client, whose refusals, body limit and `Content-Length` handling are wasm32 code that the native tests do not run. In the Spin client only the send itself is wasm32 code, and its refusals (`reject_unsupported_request_contracts`), its body limits and its `HEAD` length (`apply_spin_response_policy`) have native tests of their own. The cross-adapter parity tests have no asset route case. **Open questions.** @@ -87,19 +99,23 @@ No test sends an asset request through the Cloudflare or Spin HTTP client, whose - Streaming an asset body on those adapters, whose HTTP clients hold every response whole. - Applying the 15-second timeouts core asks for on Axum, Cloudflare and Spin. - Sending a source's `host_header` on Spin, whose outbound HTTP sends the address's own host. +- A startup check on Spin of each source's `host_header`, which would refuse the settings before the first request rather than each request as it comes. +- A setting for the log level on Cloudflare and Spin, whose loggers write `info` and above. +- A test that reads the Image Optimizer warning's text. The warning tests read the flag the warning function sets as it logs, and of the three adapters only Axum has a log capture in its tests that could read the line. - A parity test of asset routes. The parity tests build each router with its own services, where the Cloudflare and Spin HTTP clients are unavailable on a native host and the Axum client is a real one, so an asset request reaches no origin there, and the request cycle tests send the same requests through each adapter's composition root with a stub origin instead. ## 4. Acceptance -The tests above pass on each of the three adapters, and the existing tests of each adapter pass. The cross-adapter parity tests pass too, though none of them sends an asset request. +The tests above pass on each of the three adapters, and the existing tests of each adapter pass. The cross-adapter parity tests pass too, though none of them sends an asset request. The Worker's entry point passes `cargo clippy-cloudflare-wasm` and the component's passes `cargo clippy-spin-wasm`. ## Revision record -| Date | Change | -| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-17 | First draft. | -| 2026-10-18 | Fixes from review round 1. The pull request this spec belongs to, Spin's refusal of a source's `host_header` and what Cloudflare does with one, the startup warning for a route that enables the Image Optimizer and why it is not a refusal, the `HEAD` a signed route sends and the SVG and debug requests that pass, the Axum client's refusal on `send_async`, the startup refusal of a source that cannot be built, the per-request failures, body limits and timeouts, where answers are held, the Edge Cookie an asset response neither writes nor expires, the tests of the cache rule, the error and the Edge Cookie, the open questions and the review log. | +| Date | Change | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-17 | First draft. | +| 2026-10-18 | Fixes from review round 1. The pull request this spec belongs to, Spin's refusal of a source's `host_header` and what Cloudflare does with one, the startup warning for a route that enables the Image Optimizer and why it is not a refusal, the `HEAD` a signed route sends and the SVG and debug requests that pass, the Axum client's refusal on `send_async`, the startup refusal of a source that cannot be built, the per-request failures, body limits and timeouts, where answers are held, the Edge Cookie an asset response neither writes nor expires, the tests of the cache rule, the error and the Edge Cookie, the open questions and the review log. Fixes from review round 2. The loggers the Cloudflare and Spin adapters now install and where their lines appear, how often each adapter logs the Image Optimizer warning, why such a route gets a warning where those adapters refuse an auction of more than one provider, the Spin `host_header` refusal as a breaking change in the summary, the problem and the rollout, the logged failures, the reader headers a route's origin receives, the rise in key-value traffic, the signed source whose `host_header` names the address's own host in other letter case, what the warning test reads, which Spin client code the native tests reach, the logger tests and what is not in this change. | ## Review log -1. **Round 1, 2026-10-18.** A review in a fresh session, against `fixpass6/54-asset-routes` at `692ce7b8b`. Issues found, four of them blocking. The four were that the spec gave no configuration decision for a route that enables the Image Optimizer, which those adapters accepted and then answered with `502` request by request with no word at startup, that Spin drops a source's `host_header`, so a route reaches its origin with the wrong `Host` and a signature made for another, that the spec said a source that cannot be built answers per request, when it stops the adapter when the state is built, and left out the body limits, the timeouts and the per-request errors, and that no test proved the finalize step, the `no-store, private` error or that no Edge Cookie is written. Fixed in `2369fc42e`, where the three adapters warn at startup about a route that enables the Image Optimizer, `f1bba5d78`, where Spin refuses a `Host` other than the address's own, this revision for the failure handling, and `ada05a9bf`, which tests the cache rule, the error and the Edge Cookie on each adapter. The non-blocking findings fixed are the Axum client's refusal on `send_async` in `dad986582`, the Cloudflare `Content-Length` comment in `efadc7b7e`, the API reference, the Cloudflare reason and the two designs that contradicted this one in `914800f98` and `ff7ae409a`, and the consent and privacy text and this review log in this revision, which also states what the code does for each issue, corrects the partly true claims on where answers are held, the Image Optimizer and the Axum client, names the pull request this spec belongs to, and leaves the Cloudflare `host_header` and `HEAD` length as open questions, with a parity case, the timeouts and streaming as not in this change. +1. **Round 1, 2026-10-18.** A review in a fresh session, against `split/54-asset-routes` at `692ce7b8b`. Issues found, four of them blocking. The four were that the spec gave no configuration decision for a route that enables the Image Optimizer, which those adapters accepted and then answered with `502` request by request with no word at startup, that Spin drops a source's `host_header`, so a route reaches its origin with the wrong `Host` and a signature made for another, that the spec said a source that cannot be built answers per request, when it stops the adapter when the state is built, and left out the body limits, the timeouts and the per-request errors, and that no test proved the finalize step, the `no-store, private` error or that no Edge Cookie is written. Fixed in `2369fc42e`, where the three adapters warn at startup about a route that enables the Image Optimizer, `f1bba5d78`, where Spin refuses a `Host` other than the address's own, `76986f0b0` for the failure handling, and `ada05a9bf`, which tests the cache rule, the error and the Edge Cookie on each adapter. The non-blocking findings fixed are the Axum client's refusal on `send_async` in `dad986582`, the Cloudflare `Content-Length` comment in `efadc7b7e`, the API reference, the Cloudflare reason and the two designs that contradicted this one in `914800f98` and `ff7ae409a`, and the consent and privacy text and this review log in `76986f0b0`, which also states what the code does for each issue, corrects the partly true claims on where answers are held, the Image Optimizer and the Axum client, names the pull request this spec belongs to, and leaves the Cloudflare `host_header` and `HEAD` length as open questions, with a parity case, the timeouts and streaming as not in this change. +2. **Round 2, 2026-10-18.** A review in a fresh session, against `split/54-asset-routes` at `76986f0b0`, of the round 1 fixes and the new text. Issues found, two of them blocking. The two were that the Cloudflare and Spin adapters install no logger, so there the Image Optimizer warning, a startup error and the reason Spin refused a request were never written, and that Spin's refusal of a `host_header` naming another host also breaks a Spin deployment whose publisher source sets one, which the summary and Rollout 1 did not say and the changelog did not mark as breaking. Fixed in `e621e75e4`, where each of the two adapters installs a logger of its own at its entry point, which also makes the origin signing design's statement that a failed startup logs the full error true on every adapter, `9801e2279`, where the changelog gives the Spin refusal a breaking entry of its own and the guides say where each adapter's lines appear, and this revision, which states the publisher path effect in the summary, the problem and Rollout 1, how often each adapter logs the warning, and why a route that enables the Image Optimizer gets a warning where Cloudflare and Spin refuse an auction of more than one provider. The non-blocking findings fixed are the Spin `502` in the API reference's publisher fallback contract in `9801e2279`, the order of the review log in the origin signing and asset cache designs in `08122e33f`, and the reader headers a route's origin receives, the rise in key-value traffic, the signed source whose `host_header` names the address's own host in other letter case or with its default port, what the warning test reads and which Spin client code the native tests reach in this revision, which leaves a test that reads the warning's text, a startup check of `host_header` on Spin and a log level setting as not in this change. From 8084ef071fc47545958314cc47db78202bdc49b3 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 11:57:28 +0100 Subject: [PATCH 165/192] The module cache design takes its four follow-ups out of the open questions The configuration guide says core applies no lifetime of a declared table and when to push one, the integration guide lists .with_cache_features, and the two cache declaration checks are pub(crate), so items 5 to 8 leave the open questions. Section 2.7 and section 3 also say the deploy check refuses a module that offers no loading cache, as it has since the loading cache change. --- ...-10-15-module-cache-declarations-design.md | 29 +++++++++---------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md index 30a32a0b25..ec8416ac79 100644 --- a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md +++ b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-12-loading-cache-design.md`, `2026-10-12-robots-txt-from-modules-design.md`, `2026-08-27-integration-provider-seam-design.md` **Related PRs:** IABTechLab/trusted-server#1373, reviewed on jwrosewell/trusted-server#27 (closed) **Issues:** IABTechLab/trusted-server#1320 -**Last updated:** 2026-10-17 +**Last updated:** 2026-10-18 > **Why this spec exists.** A module that holds its own answers asks core for the cache of the name it chose, finds no `[cache.]` table for that name, and gets the cache that holds nothing, so every request loads the answer again. @@ -153,7 +153,7 @@ This change stores nothing new and reads no request data. The change decides whi | A table names a cache no linked module declares | The deploy check and the state build | The message in 2.4, naming the first such table | | A builder declares `robots-txt` | The same two, with or without a table | A message saying the builder declares the cache `robots-txt`, which is core's own, and naming the builder | | Two builders declare one name | The same two, with or without a table | A message naming both builders and the cache, so linking the two modules fails every deployment that does | -| A table's `module` names a crate that offers no loading cache | The state build only | A message naming the table and the module written, saying no linked crate offers a loading cache from it, and listing the loading cache modules the build has | +| A table's `module` names a crate that offers no loading cache | The deploy check and the state build | A message naming the table and the module written, saying no linked crate offers a loading cache from it, and listing the loading cache modules the build has | | A lifetime out of range | Loading the document and the deploy check | `[cache.answers] store_seconds is 0, and must be from 1 to 31536000` | | A declared cache with no table, and no or several loading cache modules linked | Nothing | The cache holds nothing and loads every time, with no log line | | A module loads through a name it did not declare | Nothing | `LoadingCaches::for_feature` answers `NoLoadingCache` for a name no one declared, and the cache selected for the name where another module or core declared it, with no error either way | @@ -166,7 +166,7 @@ A load that fails is handled as `2026-10-12-loading-cache-design.md` describes. 1. **No document has to change.** `[cache.robots-txt]` is untouched and no other table is needed. A build that links one loading cache module holds a declared cache's answers with no table. 2. **No running deployment gains a cache.** No module that ships declares one, so a deployment holds a new kind of answer only when it links a module that declares a cache. -3. **A document with another `[cache.]` table did not load before.** `CacheSettings` refused the key at parse time, so no running deployment has one. Now the parse accepts the table, and `check_cache_tables` refuses the table at the deploy check and when the state is built unless a linked module declares the name. A tool that parses a document without running the deploy check no longer refuses the table. The order of a push matters, because a binary built before this change still refuses the key. Each adapter loads the document with `Settings::from_json_value` (`crates/trusted-server-core/src/config_payload.rs`) in the function that builds its state, so a service running such a binary answers with its startup error, as 2.7 describes, whenever it loads a document that carries a `[cache.]` table. Deploy the binary with this change to every instance before pushing a document with a table, and push a document without the tables before rolling an instance back to a binary from before this change. The warning "Deploy the code before pushing the config" in `docs/guide/configuration.md` gives the same order for the `[trusted_client_ip]` section alone, and the guide has no such warning for `[cache.]` tables, see 2.10. +3. **A document with another `[cache.]` table did not load before.** `CacheSettings` refused the key at parse time, so no running deployment has one. Now the parse accepts the table, and `check_cache_tables` refuses the table at the deploy check and when the state is built unless a linked module declares the name. A tool that parses a document without running the deploy check no longer refuses the table. The order of a push matters, because a binary built before this change still refuses the key. Each adapter loads the document with `Settings::from_json_value` (`crates/trusted-server-core/src/config_payload.rs`) in the function that builds its state, so a service running such a binary answers with its startup error, as 2.7 describes, whenever it loads a document that carries a `[cache.]` table. Deploy the binary with this change to every instance before pushing a document with a table, and push a document without the tables before rolling an instance back to a binary from before this change. `docs/guide/configuration.md` gives this order in the warning "Deploy the code before pushing the config" under its loading cache settings, as it does for the `[trusted_client_ip]` section. 4. **A module that adds a declaration changes nothing until a deployment links the module.** A deployment then writes a table only to choose a loading cache module other than the one linked, because its build links two or more loading cache modules, or, where the module reads its own table, to set lifetimes other than the module's defaults. 5. **Crates outside core meet three changes.** `CacheSettings::loading_caches` returns a `Vec` where it returned an array, `LoadingCaches::insert` takes `&str` where it took `&'static str`, and `CacheSettings` has a public `declared` field. The `Vec` stops a crate building if the crate uses the result as an array, by type, by pattern or by an array method, or keeps a name as a `&'static str`, because a declared name is borrowed from the settings. The new field stops a crate building if the crate builds `CacheSettings`, or takes `CacheSettings` apart, by listing every field. The wider parameter of `insert` stops no caller. No code in this repository uses `loading_caches()` as an array, keeps a name as a `&'static str` or lists every field of `CacheSettings`. The changelog entry under Changed names all three and does not mark them as breaking, whereas the next entry, for the cookie constants, is marked "Breaking for crates outside core" because each old constant name stops building wherever it is used. Whether to mark these three is an open question, see 2.10. 6. **The documentation.** `docs/guide/configuration.md` describes the declaration under the loading cache settings, and the guide's example module is fictional. @@ -184,16 +184,12 @@ The probe asks only whether the cache keeps values. The probe never loads a valu ### 2.10 Open questions -The first two questions ask for a code change and the next two for a decision. Items 5 to 8 are follow-ups that need an edit and no decision, each with the file and lines it has in `split/32-cache-features` at `be4f47842`. This design chooses none of the answers and changes none of the files the follow-ups name. +The first two questions ask for a code change and the next two for a decision. This design chooses none of the answers. 1. Should core apply a declared table's lifetimes? Today nothing does, see 2.3. One choice is for core to lay them over the lifetimes in the `CacheSpec` a module passes to `load_cached`. Another is a helper that hands each module its table's settings (see 3) and leaves the module to apply them. A third is for a declared table to take only `module`, so that no setting is checked and then left unused. 2. Should core check the name a module loads through? Today nothing does, see 2.6. One choice is for core to refuse a name the calling module did not declare, which needs core to know which module is loading, and `load_cached` carries no module today. The other choice is to keep the convention in 2.1 and leave the names unchecked. 3. Should a deployment be able to turn a declared cache off in a build that links one loading cache module? Today a declared cache with no table is held and `module = "none"` is refused, as for `[cache.robots-txt]`, see 2.6. The other choice is to keep the behavior 2.6 describes. 4. Should the changelog entry mark the three changes in 2.8 as breaking for crates outside core? Today the entry names the three and does not mark them. The other choice is to mark them as the entry for the cookie constants is marked. -5. **The configuration guide.** `docs/guide/configuration.md`, lines 1771 to 1773, offers a declared table "the same four settings" and says nothing of two things this design states. Core applies none of a declared table's lifetimes (2.3), and a document with a `[cache.]` table is pushed only after every instance runs the new binary (2.8). The warning "Deploy the code before pushing the config" at lines 543 to 550 of the same guide covers the `[trusted_client_ip]` section alone. -6. **The integration guide.** `docs/guide/integration-guide.md`, line 364, lists `.with_loading_cache(...)` and has no row for `.with_cache_features(...)`, and lines 841 to 842 say a feature that selects nothing loads through `NoLoadingCache`, which is not so in a build that links one loading cache module. Section 3 describes both gaps. -7. **The rustdoc.** The doc comment of `loading_caches_from` in `crates/trusted-server-core/src/integrations/mod.rs`, lines 897 to 899, and the doc comment of the `module` field of `LoadingCacheConfig` in `crates/trusted-server-core/src/settings.rs`, lines 3059 to 3061, say a table with no `module` keeps no copy. Rule 2 in 2.2 says such a table takes the one loading cache module a build links, and keeps no copy only when the build links none or several. -8. **The visibility of two functions.** `declared_caches` (line 837) and `check_cache_tables` (line 878) in `crates/trusted-server-core/src/integrations/mod.rs` are `pub`, and no crate outside core calls either. The one call from another file is `check_cache_tables` at line 367 of `crates/trusted-server-core/src/config.rs`, which is in core. Both could be `pub(crate)`, unless the design states why a crate outside core needs them. ## 3. What is not in this change @@ -203,8 +199,8 @@ The first two questions ask for a code change and the next two for a decision. I - **A check that a module calls only the names it declared.** `LoadingCaches::for_feature` answers `NoLoadingCache` for a name no one declared, and the cache selected for the name where another module or core declared it, see 2.6. - **A check of a declared name's spelling,** or of whether the name starts with the module's own name. Core reserves only `robots-txt`. `asset_rules` is read as the list of asset rules, so a builder can declare it and no table can then select a module for it. TOML reads `[cache.a.b]` as a table `b` inside a table `a`, so a name with a dot can be written only quoted, as `[cache."a.b"]`. A name with a `/` blurs where the name ends and the key begins in `//`. - **A warning when a declared cache holds nothing** because the build links several loading cache modules and no table names one. -- **A `module` check in the deploy check.** A table whose `module` names a crate that offers no loading cache is still refused only when the state is built, as `[cache.robots-txt] module` was before this change. `loading_caches_from` compares `module` inside the loop that makes each cache, and `check_cache_tables` runs in the deploy check to refuse a table without making a cache. A comparison of `module` with the loading cache modules the builders offer needs the builders that offer them. The deploy check is given core's built-in builders followed by the ones a tool registers, and the adapters are given the stock modules followed by the deployment's own, see 2.4. No built-in builder offers a loading cache, so a loading cache module reaches the deploy check only as a registered builder. The `ts` command registers the stock list, which carries the stock loading cache module, and a deployment's own loading cache module needs a tool that registers it. -- **A row in the integration guide.** The table of builder methods in `docs/guide/integration-guide.md` lists `.with_loading_cache(...)` and does not list `.with_cache_features(...)`. The section "Supplying a loading cache" in the same guide does not mention declared caches, and says a feature that selects nothing loads through `NoLoadingCache`, which is not so in a build that links one loading cache module, see 2.2. +- **A deploy check that knows a deployment's own loading cache module.** The deploy check refuses a table whose `module` names a crate that offers no loading cache, as the state build does, through `check_loading_cache_modules`, which `validate_settings_for_deploy_with` in `crates/trusted-server-core/src/config.rs` calls. It compares `module` with the loading cache modules of the builders it is given, and the deploy check is given core's built-in builders followed by the ones a tool registers, where the adapters are given the stock modules followed by the deployment's own, see 2.4. No built-in builder offers a loading cache, and the `ts` command registers the stock list, which carries the stock loading cache module, so `ts config validate` refuses a table naming a deployment's own loading cache module unless a tool registers that module. +- **Declared caches in the section "Supplying a loading cache".** The integration guide, `docs/guide/integration-guide.md`, lists `.with_cache_features(...)` among the builder methods, and its section "Supplying a loading cache" does not mention declared caches. ## 4. Acceptance @@ -243,12 +239,13 @@ These joins have no test. ## Revision record -| Date | Change | -| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-15 | First draft, written after the code. | -| 2026-10-15 | Fixes from review round 1. What core does with a declared table's lifetimes, the names core does not check, the changelog entry and the three changes the entry names, the order of a push, the other callers of `loading_caches_from`, the reserved and unchecked name forms, why the deploy check does not read `module`, the untested paths, the open questions, the review log, and the service call figure in section 1, dropped because it has no source. | -| 2026-10-15 | Corrections from review round 2. What a module that borrows another module's cache name shares, the section the guide's push-order warning covers, the builders the deploy check is given, the round 1 log sentence on the code changes, and the follow-ups for the configuration guide, the integration guide, the rustdoc and the visibility of two functions. | -| 2026-10-17 | Who can read a copy names the shared store on each adapter, being Fastly's cache, the key-value store the Cloudflare and Spin adapters declare on row 24, `TRUSTED_SERVER_KV` and `default`, and none on Axum. | +| Date | Change | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-15 | First draft, written after the code. | +| 2026-10-15 | Fixes from review round 1. What core does with a declared table's lifetimes, the names core does not check, the changelog entry and the three changes the entry names, the order of a push, the other callers of `loading_caches_from`, the reserved and unchecked name forms, why the deploy check does not read `module`, the untested paths, the open questions, the review log, and the service call figure in section 1, dropped because it has no source. | +| 2026-10-15 | Corrections from review round 2. What a module that borrows another module's cache name shares, the section the guide's push-order warning covers, the builders the deploy check is given, the round 1 log sentence on the code changes, and the follow-ups for the configuration guide, the integration guide, the rustdoc and the visibility of two functions. | +| 2026-10-17 | Who can read a copy names the shared store on each adapter, being Fastly's cache, the key-value store the Cloudflare and Spin adapters declare on row 24, `TRUSTED_SERVER_KV` and `default`, and none on Axum. | +| 2026-10-18 | Items 5 to 8 leave 2.10. The configuration guide says core applies no lifetime of a declared table and when to push one, the integration guide lists `.with_cache_features(...)`, and `declared_caches` and `check_cache_tables` are `pub(crate)`. The rustdoc of `loading_caches_from` and `LoadingCacheConfig::module`, and the integration guide on a table naming no module, already gave rule 2 in 2.2. Section 3 and 2.8 match the guides. 2.7 and section 3 say the deploy check also refuses a `module` that offers no loading cache. | ## Review log From 3d698e50829e5e995e01a65daf5e14b57c3a377a Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 18 Oct 2026 22:04:11 +0100 Subject: [PATCH 166/192] The asset cache design says a failed load is the answer A load that fails asking the origin is the answer to every request that ran or waited on it, so the origin is asked once, and the design's failure handling, the cost of a slow origin and the tests say so. --- .../specs/2026-10-15-asset-cache-design.md | 23 ++++++++++--------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/docs/superpowers/specs/2026-10-15-asset-cache-design.md b/docs/superpowers/specs/2026-10-15-asset-cache-design.md index da815483ac..a714f3c6b4 100644 --- a/docs/superpowers/specs/2026-10-15-asset-cache-design.md +++ b/docs/superpowers/specs/2026-10-15-asset-cache-design.md @@ -19,7 +19,7 @@ The loading cache that core declares (`2026-10-12-loading-cache-design.md`) load ## 2. Design -Core holds the origin's response for a file in two loading caches, `assets-vary` and `assets`, and the one table `[cache.assets]` selects both. The publisher path and the asset routes both call `asset_from_cache` in `publisher.rs`. That function answers with a stored copy, or with the response its own fetch made, or with nothing, and a caller that gets nothing asks the origin as it did before this change. +Core holds the origin's response for a file in two loading caches, `assets-vary` and `assets`, and the one table `[cache.assets]` selects both. The publisher path and the asset routes both call `asset_from_cache` in `publisher.rs`. That function answers with a stored copy, with the response its own fetch made, with the failure of the load the request ran or waited on, or with nothing. A caller given the failure answers as it does when its own origin request fails, and a caller that gets nothing asks the origin as it did before this change. **Where the code is.** @@ -132,9 +132,9 @@ The stock module, `crates/loading-cache/51degrees`, uses a host's value cache wh - **On by default.** Item 3 of the rollout starts the cache on a stock build with no change to its settings, and `[cache.assets] module = "none"` turns it off (item 4). A build that links the loading cache module applies the cookie rule to every eligible asset request from its first deploy. - **Logs.** The lines `asset_cache bypass`, `asset_cache stored` and `asset_cache invalid entry` carry a fixed reason and a byte count, and no URL, key or header. The line `asset_cache load failed` carries the error text of the failed fetch, which includes the platform's error report as the platform formats it, and this change does not filter that text. -**Failure handling.** Each failure ends with the caller asking the origin, as before this change, or with the reader served the response the request's own fetch made. +**Failure handling.** A load that fails asking the origin ends with every request that ran or waited on it answering with that failure. Each other failure ends with the caller asking the origin, as before this change, or with the reader served the response the request's own fetch made. -- **The origin cannot be reached during a load.** `load_asset` returns a `LoadError`. Every request waiting on that load receives it, nothing is stored, `asset_from_cache` logs `asset_cache load failed` at warn level, and each of those requests then asks the origin itself. A failing origin is therefore asked once more by each request that waited. +- **The origin cannot be reached during a load.** `load_asset` returns a `LoadError`. Every request waiting on that load receives it and nothing is stored. `asset_from_cache` logs `asset_cache load failed` at warn level and hands the failure to its caller, which answers as it does when its own origin request fails, with `Failed to proxy request to origin` on the publisher path and `Failed to proxy asset request` on an asset route, both a `502`. A failing origin is therefore asked once for all the requests that share the load, and the next request loads again. - **A stored entry cannot be read.** `AssetEntry::decode` refuses bytes that are short, unreadable or written in another format. A record this build did not write is refused in the same way. In both cases the lookup logs at warn level, removes the entry, and the request asks the origin. A format version is in the key, in the entry and in the record, so a build that changes the format does not read the old entries. - **The shared store fails.** The key-value layer treats a failed read as a miss, so the value is loaded, and logs a failed read, write or removal at debug level. The value cache that Fastly provides reports no errors to the module. - **An error status.** Only a `200` is stored, so the body of an origin error or of a `404` is never held. The verdict is held, as "What is stored" says. For 60 seconds the requests that find the marker ask the origin themselves and none of them stores a copy, so a file the origin begins to answer with a `200` is stored only once the marker ends. @@ -142,7 +142,7 @@ The stock module, `crates/loading-cache/51degrees`, uses a host's value cache wh **Cost and limits.** -- **A slow origin.** For a response that passes the header checks, the request that leads a load waits for the whole body, up to 1 MiB, before its reader is served, and each request waiting on that load waits as long. Core asks for a first byte timeout and a between bytes timeout of 15 seconds for an origin fetch, on the publisher path (`DEFAULT_PUBLISHER_FIRST_BYTE_TIMEOUT`) and on an asset route (`DEFAULT_FIRST_BYTE_TIMEOUT`), and the Fastly adapter applies them. The Axum client applies limits of its own instead, 5 seconds to connect and 30 seconds for the whole request, and the Cloudflare and Spin adapters apply none of their own, so their runtimes' limits apply. A failed load reaches every request waiting on it and each of them then fetches the origin itself, so a reader can wait for one failed load and then for a fetch of their own. +- **A slow origin.** For a response that passes the header checks, the request that leads a load waits for the whole body, up to 1 MiB, before its reader is served, and each request waiting on that load waits as long. Core asks for a first byte timeout and a between bytes timeout of 15 seconds for an origin fetch, on the publisher path (`DEFAULT_PUBLISHER_FIRST_BYTE_TIMEOUT`) and on an asset route (`DEFAULT_FIRST_BYTE_TIMEOUT`), and the Fastly adapter applies them. The Axum client applies limits of its own instead, 5 seconds to connect and 30 seconds for the whole request, and the Cloudflare and Spin adapters apply none of their own, so their runtimes' limits apply. A failed load reaches every request waiting on it, and each of them answers with that failure rather than fetching the origin itself. - **Removals.** When a response no longer varies as its record says, the request that fetched it removes the record, and a lookup that finds an entry it cannot read removes that entry. On Fastly a removal is a purge by surrogate key (`purge_surrogate_key` in the Fastly adapter's `value_cache.rs`), and on a key-value store it is a delete. An origin whose `Vary` differs from one response to the next causes a removal for each response that differs from the record. - **Keys and variants without a limit.** Each query string, each host the reader asked, each scheme, and each combination of values for the headers `Vary` names is a key of its own. A miss writes the copy or the marker to the shared store, and the record too when the address has none, or the record alone when the origin's `Vary` allows no copies. This change sets no limit on the number of keys. `store_seconds` limits how long a copy is used and `store_idle_seconds` how long it lasts unused. Spin's `default` store, which the Spin adapter declares, gives an entry no lifetime, so there an entry that nobody reads again stays in the store, because only a read past its drop time, or a removal, takes it out. - **The fingerprint.** The publisher path computes `template_fingerprint`, a digest of all the serialized settings, for each eligible request, whether or not a copy is found. @@ -179,13 +179,13 @@ The stock module, `crates/loading-cache/51degrees`, uses a host's value cache wh The tests below exist. Those in `publisher.rs` and `proxy.rs` use `StubHttpClient` as the origin and `MemoryLoadingCache` from `crates/trusted-server-core/src/platform/test_support.rs` as the loading cache, apart from the `proxy.rs` test that has no cache and the `publisher.rs` test that selects the caches from the settings with `loading_caches_from`, as an adapter does. Those in `source_cache.rs` use `MemoryLoadingCache` and a counting function as the origin. Those in `asset_cache.rs`, `settings.rs`, `integrations/mod.rs` and the Axum `seam_probe.rs` use neither. The pull request records the fork's CI at `f6e598fda` as 19 of 19 checks green. -| Where | Tests | -| ---------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `asset_cache.rs` | Every input changes the key and none appears in it. An entry reads back as written. A short, foreign or other-format entry is not read. A copy keeps no connection, cookie or length header. A served copy counts its age from when it was made | -| `source_cache.rs` | Record and copy keys follow the publisher. `Vary` names are lowercase, sorted and named once. `*`, `Cookie` and an unreadable `Vary` allow no copies. A record reads back, and a foreign one does not. A variant tells absent, empty and reordered values apart. The first response records its `Vary` and each value gets its own copy. A response that allows no copies has every request fetch its own. A source that changes its `Vary` has its record dropped. A declared token separates copies | -| `publisher.rs`, module `asset_cache_tests` | An image is kept as sent and one fetch serves three requests. A stylesheet is served from one fetch with its age. `Vary: *`, `Vary: Cookie`, `private`, `no-store` and `public, no-store` are never kept. `s-maxage` then `max-age` set the lifetime. A header named in `Vary` separates copies. A body over 1 MiB is served whole from one fetch, with the stub streaming and not. A response whose `Vary` changed is served from its one fetch. A request carrying `Authorization` is fetched for itself. A request carrying a cookie is fetched for itself and its answer kept for no one. An answer that sets a cookie is served only to the reader it was made for. Assets turned off in the settings are fetched for every request, and one fetch serves two requests when the settings name no module. A page navigation never reads a copy | -| `proxy.rs` | A route's answer is held and serves the next request, with its type, an `Age`, its bytes and the origin's lifetime. An answer with `Set-Cookie` is fetched each time. A reader's cookie never reaches a route's origin, so the answer is shared. A range request is fetched. `no-cache`, `no-store` and `min-fresh=10` are fetched. Nothing is held while no cache keeps values | -| `settings.rs`, `integrations/mod.rs`, Axum `seam_probe.rs` | The list of core's caches now holds `assets` and `assets-vary`, in the settings listing, in the refusal that names a misspelt table, and in the Axum deploy check. A lifetime out of range in `[cache.robots-txt]` or `[cache.assets]` is refused, naming the table and the field. Both asset caches take the one module offered when `[cache.assets]` names no module, and `none` turns both off and leaves `robots-txt` as it was | +| Where | Tests | +| ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `asset_cache.rs` | Every input changes the key and none appears in it. An entry reads back as written. A short, foreign or other-format entry is not read. A copy keeps no connection, cookie or length header. A served copy counts its age from when it was made | +| `source_cache.rs` | Record and copy keys follow the publisher. `Vary` names are lowercase, sorted and named once. `*`, `Cookie` and an unreadable `Vary` allow no copies. A record reads back, and a foreign one does not. A variant tells absent, empty and reordered values apart. The first response records its `Vary` and each value gets its own copy. A response that allows no copies has every request fetch its own. A source that changes its `Vary` has its record dropped. A declared token separates copies | +| `publisher.rs`, module `asset_cache_tests` | An image is kept as sent and one fetch serves three requests. A stylesheet is served from one fetch with its age. `Vary: *`, `Vary: Cookie`, `private`, `no-store` and `public, no-store` are never kept. `s-maxage` then `max-age` set the lifetime. A header named in `Vary` separates copies. A body over 1 MiB is served whole from one fetch, with the stub streaming and not. A response whose `Vary` changed is served from its one fetch. A load that fails asking the origin is the answer, a `502` after one origin request, and the next request loads again. A request carrying `Authorization` is fetched for itself. A request carrying a cookie is fetched for itself and its answer kept for no one. An answer that sets a cookie is served only to the reader it was made for. Assets turned off in the settings are fetched for every request, and one fetch serves two requests when the settings name no module. A page navigation never reads a copy | +| `proxy.rs` | A route's answer is held and serves the next request, with its type, an `Age`, its bytes and the origin's lifetime. An answer with `Set-Cookie` is fetched each time. A reader's cookie never reaches a route's origin, so the answer is shared. A range request is fetched. `no-cache`, `no-store` and `min-fresh=10` are fetched. Nothing is held while no cache keeps values. A load that fails asking the origin is the answer, a `502` after one origin request | +| `settings.rs`, `integrations/mod.rs`, Axum `seam_probe.rs` | The list of core's caches now holds `assets` and `assets-vary`, in the settings listing, in the refusal that names a misspelt table, and in the Axum deploy check. A lifetime out of range in `[cache.robots-txt]` or `[cache.assets]` is refused, naming the table and the field. Both asset caches take the one module offered when `[cache.assets]` names no module, and `none` turns both off and leaves `robots-txt` as it was | Joins with no test: @@ -207,6 +207,7 @@ Joins with no test: | 2026-10-15 | Fixes from review round 1. The cookie rules the asset gate leaves out and what that lets through, the reason the code gives, the default-on state and the open questions on cookie rules and an off switch, `Pragma` in the gate, the verdict a response that fails a test leaves for 60 seconds, the module default, where the sizes and the `[cache.assets-vary]` table stand, the key-value store each adapter uses, how far one fetch reaches, what the caches add on Fastly, `[[cache.asset_rules]]`, the headers a route forwards, the cost and limits, the test doubles and the missing tests. | | 2026-10-16 | Fixes to the code, and from review round 2. A request that sends the publisher's origin a cookie neither reads a copy nor leads a load, `[cache.assets] module = "none"` turns both asset caches off, and a `[cache.assets]` lifetime out of range is refused when the settings load, each with its tests. Which entries each load writes, the tests that send a cookie, and the module default in the guide and the code comments. | | 2026-10-17 | The code fix on row 24, where the Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests, and where each application state has its own memory layer, Cloudflare included. How far one fetch reaches, the adapters' shared stores, how long the memory layer lives, and the reads and writes of the shared store say so. | +| 2026-10-18 | A fix to the code. A load that fails asking the origin is the answer to every request that ran or waited on it, so the origin is asked once, with a test on each path. The design, failure handling, the cost of a slow origin and the tests say so. | | 2026-10-18 | The timeouts of an origin fetch, which core asks for on every adapter and only the Fastly adapter applies. | ## Review log From 1126136871e786152c085ff5cf694bb1a60c7234 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 19 Oct 2026 14:22:20 +0100 Subject: [PATCH 167/192] The origin redirect design follows the header rewrite in main The origin redirect design describes the header rewrite merged in main (#1238) with the origin folder that #1378 gives it, in place of the Location rule #1378 no longer has, and records the five deliberate faults its tests catch. The object storage design's sentence on redirects names the mapping it now follows. The same text is on the branch of #1378. --- .../2026-10-13-origin-redirect-design.md | 116 ++++++++---------- ...-publisher-object-storage-origin-design.md | 2 +- 2 files changed, 54 insertions(+), 64 deletions(-) diff --git a/docs/superpowers/specs/2026-10-13-origin-redirect-design.md b/docs/superpowers/specs/2026-10-13-origin-redirect-design.md index d7803311ce..3c1262b2da 100644 --- a/docs/superpowers/specs/2026-10-13-origin-redirect-design.md +++ b/docs/superpowers/specs/2026-10-13-origin-redirect-design.md @@ -1,69 +1,53 @@ -# Design Spec: An origin's redirect to itself keeps the reader on the host they asked for +# Design Spec: A folder origin's addresses in response headers, and a headers step for serve middleware **Status:** Proposed, 2026-10-13. Raised as draft pull request IABTechLab/trusted-server#1378 (from jwrosewell/trusted-server#32, which is closed), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-13-publisher-object-storage-origin-design.md` and `2026-10-13-request-addressing-design.md`, both in IABTechLab/trusted-server#1342, which holds every design document of the stack -**Related PRs:** IABTechLab/trusted-server#1378, jwrosewell/trusted-server#32 (closed) -**Issues:** IABTechLab/trusted-server#1330 -**Last updated:** 2026-10-16 +**Related PRs:** IABTechLab/trusted-server#1378, IABTechLab/trusted-server#1238 (merged, the header rewrite this change extends), jwrosewell/trusted-server#32 (closed) +**Issues:** IABTechLab/trusted-server#1330, IABTechLab/trusted-server#1128 +**Last updated:** 2026-10-19 -> **Why this spec exists.** The publisher proxy relays an origin's redirect exactly as the origin wrote it, so a redirect the origin sends to itself hands the reader the origin's own address and their browser leaves the deployment. The same change gives a serve middleware a way to set headers on a response that holds no page, a redirect among them. +> **Why this spec exists.** The header rewrite merged in main (#1238) maps an address on the origin in a response header to the serving host, and knows the origin only as a host. Since the object storage design, the origin may be a folder of its host, and an address in that folder keeps the folder when it is mapped, so the reader is sent to an address that does not exist. The same change gives a serve middleware a way to set headers on a response that holds no page, a redirect among them. ## 1. Problem -An origin redirects to itself for ordinary reasons, and a request path without its trailing slash is enough. It writes its own name in the `Location` header, because that is the name it was asked for. The publisher proxy relays the header as written, so the reader's browser follows it to the origin. From then on each page is the origin's own answer, with none of the publisher's `[[fetch]]` or `[[serve]]` middleware run on it, at an address the publisher never gave the reader. +The header rewrite (#1238) maps a URL on the origin in `Location`, `Content-Location`, `Refresh`, `Link` and the two Content Security Policy headers to the serving host, keeping its path, on every route of `handle_publisher_request`. The object storage design (`2026-10-13-publisher-object-storage-origin-design.md`) lets `origin_url` name a folder, such as `https://storage.example.com/bucket/site`, and a request for `/news` is sent to the origin as `/bucket/site/news`. An origin that redirects that request to `https://storage.example.com/bucket/site/news/` has the redirect mapped to `https://publisher.example/bucket/site/news/`. The reader's browser asks for that address, which the publisher proxy sends to the origin as `/bucket/site/bucket/site/news/`, an object that does not exist. An address on the origin's host outside the folder, such as `/bucket/other/`, is mapped the same way, though no address on the serving host leads there. The two exceptions the header rewrite makes to avoid a redirect loop compare the path the reader asked for with the path in the header, folder and all, so neither matches under a folder. A second gap sits beside the first. A serve middleware works on the page in a response and writes headers through the action it returns for that page, so a response with no page in it, such as most redirects, reaches no middleware. A deployment that wants a policy of its own for the `Location` of a redirect has nowhere to put the policy. ## 2. Design -There is no setting. When the origin answers a request with a redirect, the publisher proxy reads the `Location` header, and where it names the origin the reader is sent to the same place on the scheme and host they asked for. +There is no setting. The header rewrite is given the path of `origin_url`, the folder the object storage design sends requests into, and maps a URL on the origin only when the URL is in that folder, without the folder. With no path in `origin_url`, or a path of `/`, the mapping is the header rewrite's as merged. -With `origin_url = "https://origin.example"` and a reader who asked for `https://publisher.example/news`: +With `origin_url = "https://storage.example.com/bucket/site"` and a reader who asked for `https://publisher.example/news`: -| The origin answers | The reader receives | -| ----------------------------------------------------------------- | --------------------------------------------------------- | -| `301` with `Location: https://origin.example/news/` | `Location: https://publisher.example/news/` | -| `302` with `Location: https://origin.example/news/?page=2#latest` | `Location: https://publisher.example/news/?page=2#latest` | -| `307` with `Location: //origin.example/news/` | `Location: https://publisher.example/news/` | -| `301` with `Location: https://www.origin.example/news/` | The header as the origin wrote it | -| `301` with `Location: https://elsewhere.example/news/` | The header as the origin wrote it | -| `301` with `Location: /news/` | The header as the origin wrote it | -| `200` or `201` with `Location: https://origin.example/news/` | The header as the origin wrote it | +| The origin answers | The reader receives | +| -------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | +| `301` with `Location: https://storage.example.com/bucket/site/news/` | `Location: https://publisher.example/news/` | +| `302` with `Location: https://storage.example.com/bucket/site/news/?page=2#latest` | `Location: https://publisher.example/news/?page=2#latest` | +| `301` with `Location: https://storage.example.com/bucket/site` | `Location: https://publisher.example/` | +| `301` with `Location: https://storage.example.com/bucket/other/news/` | The header as the origin wrote it | +| `301` with `Location: https://storage.example.com/bucket/sitemap.xml` | The header as the origin wrote it | +| `200` with `Link: ; rel=preload` | `Link: ; rel=preload` | +| `200` with `Content-Security-Policy: img-src https://storage.example.com storage.example.com/bucket/site/img/` | `Content-Security-Policy: img-src https://storage.example.com https://publisher.example storage.example.com/bucket/site/img/ publisher.example/img/` | -**Which responses.** A response with a status from `300` to `399` and a `Location` header. A `Location` on any other status is left as written, so a `201` naming what was created is not touched. +**Which addresses are in the folder.** A URL on the origin, being one whose host and port the header rewrite already takes as the origin's, is in the folder when its path is the folder or beneath it. Dot segments are resolved first, as the reader's browser resolves them, so `/bucket/site/about/../news/` is `/bucket/site/news/` and in the folder, and `/bucket/site/../other/` is outside it. The folder counts only where it ends at a `/`, so `/bucket/site` does not come off `/bucket/sitemap.xml` or `/bucket/site-two/`. Paths are compared exactly, so `/Bucket/Site/` is not the folder. A folder written with a closing `/` is the same folder. The folder itself, with or without its closing `/`, is the serving host's root, `/`. A URL that stops at the origin's host names the host's root, which is outside the folder. -**What names the origin.** The host of the `Location`, with its port, is compared with the `Host` the origin was sent, which is `origin_host_header_override` where one is set and otherwise the host of `origin_url` with its port. Letter case in a host is ignored and the scheme of the `Location` is not compared. The table gives the cases, with `origin_url = "https://origin.example"` unless a row says otherwise. +**What the reader receives.** The URL as the header rewrite maps it, with the folder taken off the path and the query and fragment kept. A protocol-relative URL stays protocol-relative. -| The `Location` names | The origin's name | Because | -| --------------------------------------------------------------------------------------- | ----------------- | --------------------------------------------------------------------------------------- | -| `https://origin.example/news/` | Yes | The host and port are the origin's | -| `https://ORIGIN.example/news/` | Yes | Letter case in a host is ignored | -| `https://origin.example:443/news/` | Yes | The usual port for the scheme counts as no port | -| `http://origin.example/news/` | Yes | The scheme of the `Location` is not compared | -| `//origin.example/news/` | Yes | A `Location` with no scheme names a host and is read on the scheme of the request | -| `https://origin.example:8443/news/` | No | The same host on another port is another origin | -| `https://www.origin.example/news/` | No | A host beneath the origin's is another host | -| `https://origin.example.elsewhere.test/news/` | No | A host that only begins with the origin's name is another host | -| `https://origin.example@elsewhere.test/news/` | No | The text before an `@` is a user name, and the host is `elsewhere.test` | -| `ftp://origin.example/news/` | No | Only an `http` or `https` address counts | -| `https://origin.example/news/`, with `origin_url = "https://origin.example:8443"` | No | The same host without the port is another origin | -| `https://www.example.com/news/`, with `origin_host_header_override = "www.example.com"` | Yes | The override is the name the origin was sent | -| `https://origin.example/news/`, with `origin_host_header_override = "www.example.com"` | No | The host in `origin_url` is where the connection goes and not what the origin is called | +**What is left alone.** A URL on the origin's host outside the folder is left as the origin wrote it, because no address on the serving host leads there, and the reader's browser follows it to the origin's host. -**What the reader receives.** The scheme and host of the request, then the path, query and fragment of the `Location`. The page pipeline writes the same scheme and host in place of the origin's addresses in a page, but the page pipeline takes the host of `origin_url` as the origin's name even where an override is set, whereas this rule takes the override, which is the name the origin was sent. With an override set, the page pipeline and this rule therefore disagree on what the origin is called, and the page pipeline is not changed here. Where the origin's name is the request's own host, the header comes back naming the same host on the request's scheme. A request that names no host has nowhere to send the reader, so its redirect is left as written. +**Content Security Policy sources.** A source expression with no path permits every path on the origin's host, so its serving-host sibling has no path either and permits the whole serving host, as without a folder. A source with a path in the folder gains a sibling without the folder. A source with a path outside the folder gains no sibling. -**The origin path.** Where `origin_url` has a path, the origin is that folder and not the whole host. A `Location` whose path is the origin path, or beneath it, comes back with the origin path removed, which reverses the way the origin address was built. A request for `/news` is sent to `/bucket/site/news`, and a redirect to `/bucket/site/news/` reaches the reader as `/news/`. A `Location` that is the origin path itself reaches the reader as `/`. The origin path counts only where it ends at a `/`, so `/bucket/site` does not come off `/bucket/sitemap.xml`. Paths are compared exactly, so `/Bucket/Site` is not `/bucket/site`. Dot segments in the `Location` are resolved before the comparison, so a path that climbs out of the folder is outside it. A `Location` on the origin's host and outside the origin path is left as written, because no address on the reader's host leads there. +**The loop exceptions.** The header rewrite leaves a `Location` or `Refresh` target unchanged where following it would bring the reader back to the address they asked for (#1238, and "Origin URLs in proxied response headers" in the configuration guide). With a folder, the path the reader asked for is compared with the path of the target without the folder, so both exceptions apply under a folder as they do without one. -**A publisher addressed by a path prefix.** The request addressing design (`2026-10-13-request-addressing-design.md`) lets one host serve several publishers, each under a prefix of its own. The prefix is removed from the path before the request is routed, so the origin is asked for the path without it, and core never writes the removed prefix into a response. The rewritten `Location` is made from the request's scheme and host and the path of the `Location`, and nothing in `publisher.rs` reads the removed prefix, so the reader of such a publisher is sent to an address without the prefix. With the example hook in the request addressing design, on a Fastly service whose `__KEY` selector is `{host}`, a reader of `https://front.example/publisher-a.example/news` whose origin answers `Location: https://origin.example/news/` receives `Location: https://front.example/news/`. The hook reads `news` as the publisher, so the reader is answered `421 Misdirected Request` unless a publisher has that name. Before this change the same reader followed the origin's own address to the origin. A serve middleware puts the prefix back, through the headers step (section 3), and the step is given the rewritten `Location` and the request state a module's request preparer left. Core does not. No module in this repository reads the removed prefix, so none puts it back yet, and no test sends a request addressed by a path prefix through the rule. A root-relative `Location`, such as `/news/`, is resolved by the reader's browser against the root of the host, so it loses the prefix in the same way with or without this change. +**What names the origin.** The header rewrite decides, and this change does not alter it. The origin is the host of `origin_url` with its port, in any letter case, with or without a `Host` override, and the header rewrite maps a `Location` on a response of any status. -**What is left alone.** A `Location` naming any other host is the origin sending the reader somewhere else. A `Location` that names no host, such as `/news/`, `news/` or `?page=2`, is resolved by the reader's browser against the address it asked for, so it stays on the deployment as it is. A value that cannot be read as an address is relayed as it arrived. +**A publisher addressed by a path prefix.** The request addressing design (`2026-10-13-request-addressing-design.md`) lets one host serve several publishers, each under a prefix of its own. The prefix is removed from the path before the request is routed, so the origin is asked for the path without it, and core never writes the removed prefix into a response. The header rewrite writes the request's scheme and host and the path of the URL, and nothing in `publisher.rs` reads the removed prefix, so the reader of such a publisher is sent to an address without the prefix, with or without a folder. With the example hook in the request addressing design, on a Fastly service whose `__KEY` selector is `{host}`, a reader of `https://front.example/publisher-a.example/news` whose origin answers `Location: https://origin.example/news/` receives `Location: https://front.example/news/`. The hook reads `news` as the publisher, so the reader is answered `421 Misdirected Request` unless a publisher has that name. A serve middleware puts the prefix back, through the headers step (section 3), and the step is given the rewritten `Location` and the request state a module's request preparer left. Core does not. No module in this repository reads the removed prefix, so none puts it back yet, and no test sends a request addressed by a path prefix through the header rewrite. A root-relative `Location`, such as `/news/`, is resolved by the reader's browser against the root of the host, so it loses the prefix in the same way. -**When the header cannot be rewritten.** A `Location` with bytes outside printable ASCII, and a rewritten value that is not a valid header value, are each relayed as the origin wrote them. When a response carries several `Location` headers, only the first is read, and when the first names the origin the rewritten value replaces all of them. None of these three cases has a test. +**Where it happens.** In the header rewrite, `rewrite_origin_urls_in_headers` in `response_header_rewrite.rs`, which `handle_publisher_request` in `publisher.rs` calls where #1238 placed it. The folder is the `origin_path` field this change adds to `OriginHeaderRewrite`, which the handler fills from the path of `origin_url`, the path the object storage design builds the origin address from. The call comes after a file that core makes where the origin has none has replaced the origin's answer, and before the template cache gate, so every route the handler returns an origin answer through passes it. A copy of an asset from the asset cache enters at the same point as the origin's own answer. The `502` core sends in place of an unexpected `304` returns before the call, and carries no header from the origin. The serve middleware headers step (section 3) runs straight after the header rewrite. -**Where it happens.** In one place, `handle_publisher_request` in `publisher.rs`, as soon as the origin's response is received. Two things come first and neither reads the `Location`. They are the removal of the three headers that describe the deployment (`X-Cache`, `X-Cache-Hits` and `X-Served-By`) and one debug log line that names the status and the number of headers. The streamed page, the buffered page, a `HEAD` and every other method return that one response, so each carries the rewritten header. A copy of an asset from the asset cache enters at the same point as the origin's own answer. Neither the template cache nor the asset cache keeps a response other than `200`, so no stored copy holds a redirect. The header is written for each request from that request's own scheme and host, after the platform has returned the origin's response to core. The rule itself is `rewritten_origin_redirect`, which takes the `Location`, the origin's host and path and the request's scheme and host, and returns the new value or nothing. - -**Adapters.** No adapter changes. The rule needs a `3xx` to reach core with its `Location` as the origin wrote it, and that holds on all four adapters. The scheme and host the rule writes are the two values `RequestInfo::from_request` reads for the page pipeline, and how each adapter fills them differs. +**Adapters.** No adapter changes. The header rewrite needs a `3xx` to reach core with its `Location` as the origin wrote it, and that holds on all four adapters. The scheme and host it writes are the two values `RequestInfo::from_request` reads for the page pipeline, and how each adapter fills them differs. | Adapter | A `3xx` reaches core because | The scheme comes from | The host comes from | | ---------- | ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- | @@ -74,15 +58,15 @@ With `origin_url = "https://origin.example"` and a reader who asked for `https:/ The Axum and Cloudflare adapters set no TLS signal and strip only the configured client IP headers, so a client of either can choose the host its own redirect names, as it can already choose the host the page pipeline writes into a page. Each adapter already has a test that a redirect from its client is left for core to see, and those tests are not part of this change. -**Consent and privacy.** The rewritten `Location` is made from the request's scheme and host and the path, query and fragment of the `Location`. The rule adds nothing to the address, reads no consent state and makes no request, so it has no effect on consent. Its debug log line is a fixed sentence that prints neither address. +**Consent and privacy.** The mapped URL is made from the request's scheme and host and the path, query and fragment of the origin's URL, and the folder only shortens the path. Nothing is added to the address, no consent state is read and no request is made, so the change has no effect on consent. -**Why core and not a middleware.** The redirect answers the request core made to the one origin `[publisher]` names, and the rule uses only that origin's own name and path and the request's own scheme and host. The rule names no vendor and reads no module's settings. Rewriting the origin's own address is part of the origin fetch, the step that turns the origin's answer into the deployment's. Core already moves the origin's address in a page to the reader's host before any middleware sees the page, and a `Location` naming the origin is the same address in a header. The headers step in section 3 lets a middleware change a header, and three things keep this rule out of a middleware. A middleware is told the host of `origin_url` and no origin path, and with an override set that host is not the name the origin was sent. A middleware runs only where a `[[serve]]` entry covers the request path, so a deployment that named none would still send its readers off the deployment. And the headers step is not asked about every response (section 3). What a deployment decides for itself, such as where a redirect to another name of the site goes or whether to put a removed path prefix back, is a middleware's, through that step. +**Why core and not a middleware.** The folder is part of the origin `[publisher]` names, and the header rewrite that maps the origin's addresses is in core (#1238). A middleware is told the host of `origin_url` and no origin path, and runs only where a `[[serve]]` entry covers the request path, so a deployment that named none would still send its readers to an address that does not exist. What a deployment decides for itself, such as where a redirect to another name of the site goes or whether to put a removed path prefix back, is a middleware's, through the headers step in section 3. -**Why no setting, and what changes for a deployment.** A `Location` naming the origin takes the reader off the deployment that is serving them, which no deployment asks for by naming an origin. A deployment whose origin sends no redirect to itself sees no change. For a deployment whose origin does, the reader stays on the deployment where before the reader's browser left it. Section 4 gives the two cases where the reader's browser loops instead. No setting is added or changed. `origin_host_header_override` gains a second job, being the name a redirect is compared with, and the configuration guide says so under `origin_url` and under the override. Loading the settings already refuses an override that is not a host name, an IPv4 address or a bracketed IPv6 address with an optional port, so the comparison never reads anything else. +**What changes for a deployment.** A deployment whose `origin_url` has no path sees no change. For one with a path, a URL in the folder reaches the reader without the folder, where the header rewrite alone kept it, and a URL on the origin's host outside the folder is relayed as the origin wrote it. No setting is added or changed. ## 3. The serve middleware headers step -A serve middleware may set headers on a response the publisher path sends, a redirect included, except for the responses listed under "Where the step is called" below. The redirect rule in section 2 does not use the step. A deployment's own policy for a redirect does, such as where a `Location` for another name of the site goes, or putting a removed path prefix back. +A serve middleware may set headers on a response the publisher path sends, a redirect included, except for the responses listed under "Where the step is called" below. The header rewrite in section 2 does not use the step. A deployment's own policy for a redirect does, such as where a `Location` for another name of the site goes, or putting a removed path prefix back. **What it adds.** A method on `Middleware`, with a default that returns no headers, and the read only view of the response that the method is given, both in `crates/trusted-server-core/src/middleware.rs`. A middleware written before the method compiles and behaves as before, and a request path no `[[serve]]` entry covers sees no change. @@ -111,7 +95,7 @@ pub struct ResponseView<'a> { **What a middleware may do.** Return headers. Each is written on the response in middleware order, replacing any header of the same name, so a later middleware replaces an earlier one's header of that name. The method cannot change the status or the body. A header the same middleware sets for a page through the action it returns from `create` is written later, in the page pass, and replaces the header set here. -**Where it runs.** In `handle_publisher_request`, once the origin's answer is in hand, after the redirect rule, after a file that core makes where the origin has none has replaced the origin's answer, and before core's cache rules, the modules' `finalize_response` hooks and the adapter's finalizing. A middleware therefore sees the rewritten `Location`, and a cache header it sets can be replaced by the rules that follow, such as the `private, no-store` that core gives a page it has changed for one reader. +**Where it runs.** In `handle_publisher_request`, once the origin's answer is in hand, straight after the header rewrite (section 2), which itself follows a file that core makes where the origin has none, and before core's cache rules, the modules' `finalize_response` hooks and the adapter's finalizing. A middleware therefore sees the headers as the header rewrite left them, and a header it sets is not rewritten. A cache header it sets can be replaced by the rules that follow, such as the `private, no-store` that core gives a page it has changed for one reader. **Where the step is called.** The step is called at three places in `handle_publisher_request`: @@ -127,26 +111,31 @@ The step is not called for three kinds of response: ## 4. What is not in this change -- **Another name for the same site.** An origin at `https://publisher.example` that redirects to `https://www.publisher.example` names a host that is not the one it was asked for, so the header is relayed and the reader follows it. A deployment whose origin does so names the host the origin redirects to, in `origin_url` or in `origin_host_header_override`. A deployment that serves both names, and wants the reader kept on it across that redirect, decides where such a `Location` goes with a serve middleware of its own, through `Middleware::response_headers` (section 3). The step is asked about a redirect with no body, though not about every response the publisher path sends. -- **An origin that redirects away from the scheme it is asked on.** The scheme of the `Location` is not compared, and the reader is sent to their own scheme. An origin that `origin_url` reaches over plain HTTP, and that answers each request with the same address on `https`, sends the reader back to the address they asked for, which gets the same answer again. `origin_url` names the scheme the origin serves its pages on. -- **An origin that redirects an index file back to its folder.** With `directory_index` set, the index file is added to every request path that ends in `/`, so a request for `/news/` is sent to the origin as `/news/index.html`. An origin that answers that request with a redirect to `/news/` sends the reader back to the address they asked for, which gets the same answer again. Before this change the reader followed the origin's own address and left the deployment, and now the reader's browser asks again each time. -- **A `Location` that names no host, from an origin in a folder.** An origin under a path in `origin_url` that answers `Location: /bucket/site/news/` has the header relayed as written, and the reader's browser then asks for a path with the folder in it. +- **Another name for the same site.** An origin at `https://publisher.example` that redirects to `https://www.publisher.example` names a host that is not the origin's, so the header rewrite relays it and the reader follows it. A deployment whose origin does so names the host the origin redirects to in `origin_url`. A deployment that serves both names, and wants the reader kept on it across that redirect, decides where such a `Location` goes with a serve middleware of its own, through `Middleware::response_headers` (section 3). The step is asked about a redirect with no body, though not about every response the publisher path sends. +- **An origin that redirects an index file back to its folder.** With `directory_index` set, the index file is added to every request path that ends in `/`, so a request for `/news/` is sent to the origin as `/bucket/site/news/index.html`. An origin that answers that request with a redirect to `/bucket/site/news/` names the address the reader asked for. Under the header rewrite's second loop exception, a `GET` or `HEAD` redirect that sets no cookie is relayed as the origin wrote it, so the reader's browser leaves the deployment for the origin's address. One that sets a cookie is mapped, so the reader's browser asks again each time. +- **A `Location` that names no host, from an origin in a folder.** An origin under a path in `origin_url` that answers `Location: /bucket/site/news/` has the header relayed as written, as the header rewrite relays any relative URL, and the reader's browser then asks for a path with the folder in it. - **Putting a removed path prefix back.** A serve middleware does so through the headers step, and no module in this repository reads the removed prefix yet. See "A publisher addressed by a path prefix" in section 2. -- **The name the page pipeline gives the origin.** With an override set, the page pipeline still moves the host of `origin_url` in a page and not the override. This change leaves the page pipeline as it was. - **Following a redirect.** The publisher proxy follows no redirect itself. The reader's browser makes the next request. -- **Any header but `Location`.** An address in `Content-Location`, `Link` or `Refresh` is relayed as before. - **Asset routes and the first-party proxy.** Their redirects are handled in `proxy.rs` and are unchanged. ## 5. Acceptance -Tests in `trusted-server-core`, where the code is, for the redirect rule: +Tests in `trusted-server-core`, where the code is, for the folder, in `response_header_rewrite.rs`: + +- A URL in the folder loses the folder and keeps its query and fragment, the folder itself with or without its closing `/` becomes `/`, a dot segment that stays in the folder is resolved, a protocol-relative URL stays protocol-relative, a folder written with a closing `/` is the same folder, and an origin path of `/` takes nothing off (`folder_origin_maps_urls_in_the_folder_without_it`). +- Another folder of the host, a folder whose name only begins with the folder's, a name that begins as the folder does but does not end at a `/`, the host's root, a URL that stops at the host, a dot segment that climbs out of the folder and a path in another letter case are each left unchanged (`folder_origin_leaves_urls_outside_the_folder_unchanged`). +- Under a folder, a scheme change to the address the reader asked for and a cookieless `GET` redirect to it are kept, and a redirect to another address in the folder is mapped (`folder_origin_keeps_redirects_to_the_current_request_that_would_loop`). +- A source with no path gains the whole serving host, a source in the folder gains a sibling without the folder, and a source outside the folder gains none (`folder_origin_csp_sources_follow_the_folder`). + +Tests through `handle_publisher_request` with a scripted origin, in `publisher.rs`: -- A `Location` naming the origin comes back on the request's scheme and host with its path, query and fragment, for an origin on `https` and on plain HTTP, with and without a port, named by an address in brackets, in any letter case, and with the scheme left off. -- With `origin_host_header_override` set, a `Location` naming the override is rewritten and one naming the host of `origin_url` is not. -- A `Location` naming another host, a host beneath the origin's, a host that only starts or ends with the origin's name, or the origin's name on a different port is left as written. So is a path, a query or a fragment alone, an address that is not `http` or `https`, a value that is no address, and any `Location` when the request names no host. -- With a path in `origin_url`, a `Location` at or beneath that path comes back without it, and the origin path itself comes back as `/`. One beside it, above it, in another letter case or climbing out of it is left as written, and the origin path comes off only where it ends at a `/`. -- Through `handle_publisher_request` with a scripted origin, the reader receives the rewritten `Location` on a buffered answer and on a streamed page, for a `GET`, a `HEAD`, a `POST`, a `PUT` and a `DELETE`, for `301`, `302`, `303`, `307` and `308`, and on the request's own scheme. -- Through the same handler, a redirect to another host, a redirect naming no host, a `Location` on a `200`, a `201`, a `404` and a `503`, and a redirect with no `Location` each reach the reader as the origin sent them. +- The reader receives the mapped `Location` on a buffered answer and on a streamed page, for a `GET`, a `HEAD`, a `POST`, a `PUT` and a `DELETE`, for `301`, `302`, `303`, `307` and `308` (`the_reader_receives_the_rewritten_location_on_every_route`), and on the request's own scheme (`the_reader_s_own_scheme_is_used`). +- With a folder in `origin_url`, the origin is asked under the folder, a redirect in the folder reaches the reader without it, and one outside it reaches the reader as written (`a_redirect_under_the_origin_path_reaches_the_reader_without_it`). +- An origin on plain HTTP or on a port is named for the reader, and the same host on another port is not (`an_origin_on_plain_http_or_on_a_port_is_named_for_the_reader`). +- Under a `Host` override, a `Location` naming the host of `origin_url` is mapped and one naming the override is not (`under_a_host_override_the_host_of_origin_url_is_the_origin_s_name`). +- A `Location` on a `200`, a `201`, a `404` and a `503` is mapped on the pass-through, streamed and buffered routes (`a_location_on_an_answer_that_is_no_redirect_is_named_for_the_reader`). +- A redirect to another host and one naming no host reach the reader as the origin wrote them, and a redirect with no `Location` gains none (`a_redirect_that_is_not_the_origin_s_own_reaches_the_reader_unchanged` and `a_redirect_with_no_location_gains_none`). +- The header rewrite's own tests pass on this branch, both its handler tests for the two loop exceptions and its tests in `response_header_rewrite.rs`. Tests in `trusted-server-core` for the headers step: @@ -160,11 +149,11 @@ Nothing tests the following: - The step on the refusal of a streamed page by a middleware's preparation, which is the third place the step is called. - That the step is not called for a refusal made before the origin is asked, for a page from the template cache, or for an answer made from an error. - Several middleware setting a header of one name on one response. -- A `Location` with bytes outside printable ASCII, a rewritten value that is not a valid header value, and several `Location` headers on one response. -- A request addressed by a path prefix, sent through the redirect rule. +- The folder on `Content-Location`, `Refresh` and `Link` through the handler. Each takes the same mapping the folder tests call directly. +- A request addressed by a path prefix, sent through the header rewrite. - Any adapter, because no adapter changes, although each already tests that its client leaves a redirect for core to see. -Sixteen deliberate faults were made, one in each run, and each was caught by a test: a `Location` rewritten on any status, the port left out of the comparison, the origin path not removed, the origin path removed where it does not end at a `/`, a `Location` outside the origin path rewritten with its path whole, the fragment dropped, the query dropped, an address that is not `http` or `https` rewritten, a request with no host rewritten, a `Location` naming no host resolved against the origin, the host of `origin_url` compared where an override is set, a `Location` with the scheme left off not read, the host compared in exact letter case, the reader always sent to `https`, the origin path not passed by the handler, and a `Location` that is the origin path itself rewritten with an empty path and not `/`. All sixteen are in the redirect rule and the handler's use of the rule, and none is recorded for the headers step. +Five deliberate faults were made, one in each run, and each was caught by a test: the handler giving the header rewrite no folder (`a_redirect_under_the_origin_path_reaches_the_reader_without_it`), the folder counted where it does not end at a `/` (`folder_origin_leaves_urls_outside_the_folder_unchanged`), a source with no path taking the folder rule (`folder_origin_csp_sources_follow_the_folder`), the loop exceptions comparing the path with the folder still in it (`folder_origin_keeps_redirects_to_the_current_request_that_would_loop`), and the folder compared before dot segments are resolved (`folder_origin_maps_urls_in_the_folder_without_it`). ## Revision record @@ -174,6 +163,7 @@ Sixteen deliberate faults were made, one in each run, and each was caught by a t | 2026-10-14 | Names its pull request and issue, and points a deployment that serves two names of the site at the serve middleware headers step. | | 2026-10-15 | Fixes from review round 1. The headers step as a section with its tests and the responses it skips, who puts a removed path prefix back, the adapters, the setting, consent, failure and the index file loop, a table of what names the origin, the title now saying host, and the same sentence corrected in the changelog and the configuration guide. | | 2026-10-15 | Corrections from review round 2. The two loops and the `421` under a removed path prefix, and the three kinds of response the headers step is not called for, named in the changelog and the configuration guide. | +| 2026-10-19 | Follows the header rewrite merged in main (#1238) in place of a `Location` rule of its own, and gives the header rewrite the origin folder. The title, sections 1, 2, 4 and 5 and where the headers step runs change to match, and so do the changelog and the configuration guide. | ## Review log diff --git a/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md b/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md index 00e89b1b07..39cbd6a7c5 100644 --- a/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md +++ b/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md @@ -43,7 +43,7 @@ The page that comes back is a publisher page like any other, so it is fetched, c **The origin path.** A path in `origin_url` is kept, and the request path is added after it with exactly one `/` between them, whichever side carries one. The request's query is kept unless `origin_auth` sets `origin_query = "strip"` (see "The query"). An `origin_url` with no path sends the request path as it arrived, apart from the rebuilding of a signed address (see "Signing"). A query in `origin_url` is ignored, as it was. The path is no setting, because `origin_url` is required, and before this change the path was discarded. A deployment whose existing document has a path in `origin_url` therefore sends different requests to its origin, whether or not the document sets the two settings, because each request now goes to the folder the path names and a request path with a `.` or `..` segment is answered `400`. The path is a breaking change for such a deployment. -The publisher proxy builds the origin address in one place, `handle_publisher_request` in `publisher.rs`, and that address now comes from `publisher_origin_path_and_query`. The streamed page, the buffered page, the template cache key, a `HEAD` and every other method take the address built there. The publisher path follows no redirect. A redirect the origin sends to itself has the origin path taken off its `Location`, as `2026-10-13-origin-redirect-design.md` sets out. +The publisher proxy builds the origin address in one place, `handle_publisher_request` in `publisher.rs`, and that address now comes from `publisher_origin_path_and_query`. The streamed page, the buffered page, the template cache key, a `HEAD` and every other method take the address built there. The publisher path follows no redirect. A URL in the folder in a response header, a redirect's `Location` among them, reaches the reader without the folder, and one outside the folder is left as the origin wrote it, as `2026-10-13-origin-redirect-design.md` sets out. **Signing.** `origin_auth` is the table an asset route's `auth` takes, read as the same type, with the same fields, defaults and secret store key references, and it signs through the same function, `apply_asset_origin_auth`. There is one signing path. The request is signed last, once its address and `Host` are final. Signing rebuilds the address through the URL parser, `url::Url`, signs the rebuilt address and sends the request to it, so the two cannot differ. The parser writes the address in its own form. It percent-encodes an apostrophe or a non-ASCII character in the query, and a double quote, a curly brace or a non-ASCII character in the path, all of which `http::Uri` accepts as they are, and it resolves dot segments in the path, which are refused with `400` first when `origin_url` has a path. The table still accepts the deprecated `secret_store` field that an asset route's `auth` accepts. The field is ignored with a logged warning and is never written back. From 912fa093cdf255417908a11d3799352135d85794 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 20 Oct 2026 07:22:48 +0100 Subject: [PATCH 168/192] Add the module scripts design The design of row 55, IABTechLab/trusted-server#1423, joins the specs pull request that carries every design of the stack, with the same text as on that row's branch. --- ...6-10-19-module-scripts-in-crates-design.md | 118 ++++++++++++++++++ 1 file changed, 118 insertions(+) create mode 100644 docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md diff --git a/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md b/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md new file mode 100644 index 0000000000..dfa0793cc8 --- /dev/null +++ b/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md @@ -0,0 +1,118 @@ +# Design Spec: A module's browser script lives in the crate that supplies it + +**Status:** Proposed, 2026-10-19. Implemented as the draft pull request [IABTechLab/trusted-server#1423](https://github.com/IABTechLab/trusted-server/pull/1423), row 55 of the stack, above [IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412), which serves asset routes on every adapter, for Tech Lab review. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** [integration-provider-seam](https://github.com/IABTechLab/trusted-server/blob/feature/6-integration-seam/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md), [page-middleware](https://github.com/IABTechLab/trusted-server/blob/feature/6-integration-seam/docs/superpowers/specs/2026-10-12-page-middleware-design.md), [no-modules-crate](https://github.com/IABTechLab/trusted-server/blob/feature/6-integration-seam/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md), and the design documents of the stack in IABTechLab/trusted-server#1342 +**Related PRs:** [IABTechLab/trusted-server#1423](https://github.com/IABTechLab/trusted-server/pull/1423). [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348), which moves every vendor module's Rust out of core into `crates///` +**Issues:** none +**Last updated:** 2026-10-20 + +> **Why this spec exists.** Each vendor module's Rust lives in its own crate, `crates///`, but its browser script, the script's tests and some of their test data live in `crates/trusted-server-js/lib/src/integrations//` and `lib/test/integrations//`, a folder that belongs to no vendor. Whoever maintains a module works in two places, and the shared browser crate holds every vendor's code. This change moves ten modules' scripts and tests into the nine crates that supply them, makes the build, the tests, the lint and the format checks, and editors, find them there, and moves the Tinybird project into the Tinybird crate. The 14 built bundles are byte for byte the same as before. + +## 1. Problem + +`build-all.mjs` builds a bundle for every folder in `lib/src/integrations/` that holds an `index.ts`, and `expected_module_ids` in `crates/trusted-server-js/build/bundle_set.rs` lists the same folders, so the build script embeds exactly the bundles the Node build wrote. Vitest, ESLint, Prettier and TypeScript are all configured for `crates/trusted-server-js/lib` and see no file outside it. A module's script reaches the shared browser code by relative paths such as `../../core/log`, and its tests reach the script by paths such as `../../../src/integrations/lockr/script_guard`. + +So a module crate such as `crates/identity/lockr` holds the module's Rust, its settings and its page-change fixtures, while the script that runs in the reader's browser and that script's tests sit in another crate, under a folder shared by every vendor. Three tests in the shared crate also use vendor scripts. The shared DOM insertion dispatcher's test imports five vendors' script guards, the Prebid shim's test imports the GPT and GPT diagnostics scripts, and the IIFE state test builds the Permutive bundle. The GPT script imports the APS renderer. + +The Tinybird module has the same split. Its Rust is in `crates/analytics/tinybird`, and the Tinybird project it sends to, its datasources, pipes, fixtures and tests, is in a `tinybird/` folder at the repository root. + +## 2. Design + +### 2.1 Where a module's script lives + +A crate that supplies a browser module keeps the script in `js/src//`, with `index.ts` as the bundle's entry, and the script's tests in `js/test//`. Test data only that crate's tests use goes in `js/test/fixtures/`. The module's name is its folder's name, as before, so one crate can supply more than one module. + +| Module | Crate | +| -------------------- | -------------------------------- | +| `datadome` | `crates/bot-protection/datadome` | +| `didomi` | `crates/cmp/didomi` | +| `google_tag_manager` | `crates/tag/google-tag-manager` | +| `gpt` | `crates/ad-tag/google` | +| `gpt_diagnostics` | `crates/ad-tag/google` | +| `lockr` | `crates/identity/lockr` | +| `osano` | `crates/cmp/osano` | +| `permutive` | `crates/audience/permutive` | +| `sourcepoint` | `crates/cmp/sourcepoint` | +| `testlight` | `crates/testing/testlight` | + +Git detects each move as a rename, because each moved file keeps most of its content, so `git log --follow` and blame keep each file's history. The GPT tests' helper that locates `gpt_bootstrap.js`, `test/fixtures/paths.ts`, moves into `crates/ad-tag/google/js/test/fixtures/`, because only the GPT tests use it. + +A vendor crate can now hold browser code in three ways, and this change adds only the first. + +- **Module scripts**, in `js/src//` and `js/test//`, which the shared build bundles and which the checks in 2.4 cover. +- **Inline page scripts** in `src/*.js`, such as `crates/ad-tag/google/src/gpt_bootstrap.js` and InMobi's `gpp_stub.js` and `tcf_stub.js`, which the crate's Rust embeds into the page. InMobi's are tested by `node --test` in the last step of the Vitest workflow job, as before. +- **A carried script** directly in `js/`, such as `crates/testing/iabtl/js/probe.js`, which the crate embeds with `include_str!` and hands to the registry as a `CarriedJsModule` with its SHA-256. + +So a folder named `js/` holds either module scripts, in `js/src` and `js/test`, or a carried script directly in `js/`. + +### 2.2 How the build finds them + +`moduleSourceDirs()` in `lib/build-module-options.mjs` lists every module's source folder by name, `lib/src/integrations/` for the modules that stay in the shared crate and `crates///js/src/` for the rest. A name found in two places is an error, because bundles and imports are keyed by name, and so is a module named `core`, which is the shared core's bundle. A link whose target is gone counts as no folder. `build-all.mjs` builds a bundle for each listed folder whose `index.ts` is a file. + +`expected_module_ids` in `build/bundle_set.rs` applies the same rule from the Rust side, given the crates folder as well as `lib/src`, so `build.rs` still embeds exactly the bundles the Node build wrote. Each side has tests of a module found in a crate, a name found twice and a module named `core`, in `build/bundle_set.rs` and `lib/test/module-discovery.test.mjs`. Bundles are still named `tsjs-.js`, so the Rust API, the served URLs and every content hash are unchanged. + +`build.rs` asks Cargo to rerun it when `lib/build-module-options.mjs` or any existing `crates///js/src` folder changes, and `js/test` as well when `TSJS_TEST=1` runs the tests. It names only folders that exist, because Cargo reruns a build script on every build while a watched path is missing. A crate's first `js/src` folder is therefore picked up by the next build that runs for another reason, which `crates/trusted-server-js/README.md` says, with touching `lib/package.json` as the way to force it. + +### 2.3 How scripts and tests import + +`lib/build-module-options.mjs` defines `@tsjs/core`, `@tsjs/shared` and `@tsjs/module/` for the build and for Vitest, Vitest's config adds `@tsjs/test`, and `tsconfig.json` maps the same names for TypeScript. + +- `@tsjs/core/...` and `@tsjs/shared/...` reach the shared browser code from a script in any crate. +- `@tsjs/module//...` reaches another module's code by the module's name rather than by the folder it lives in. The GPT script uses it for the APS renderer, and the dispatcher and Prebid tests use it for the vendor scripts they exercise, so an import of a crate script's code by another module can be found by searching for `@tsjs/module/`. Modules kept in `lib` still import each other by relative path, as the Prebid shim imports the APS renderer, and the IIFE state test finds the Permutive bundle's entry through `moduleSourceDirs()`. +- `@tsjs/test/...` reaches test data that more than one module's tests share, which is `test/fixtures/aps-renderer-v1.json`, used by the core, APS, GPT and Prebid tests. + +TypeScript's `paths` cannot match a folder by pattern, so `tsconfig.json` lists each crate's `js/src` folder under `@tsjs/module/*`, and a test checks that the list matches what `moduleSourceRoots()` finds, in the same order. A test imports its own module's script by a short relative path within the crate, for example `../../src/lockr/script_guard`. + +A script in a crate cannot import an npm package. Vite and TypeScript look for a package by walking up from the importing file, and this crate's `node_modules` is not above any other crate. None of the moved scripts imports a package, and a script that needs one stays in `lib` for now, which is one reason the Prebid shim, which imports `prebid.js`, has not moved. + +### 2.4 How the checks and editors find them + +Vitest's `test.dir` is the crates folder, with `trusted-server-js/lib/test/**/*.test.{ts,mjs}` and `*/*/js/test/**/*.test.{ts,mjs}` as the test files, and `server.fs.allow` lets it read the crates folder. The type gate, which checks the GPT diagnostics export contract with `expectTypeOf`, covers `*/*/js/test/**/types.test.ts`, because that test moved with the GPT diagnostics script. `lib/tsconfig.json` includes `../../*/*/js`, maps the `@tsjs/...` names, and maps `vitest` into this crate's `node_modules`, which a test in another crate cannot find by walking up from its own folder. + +ESLint's flat config ignores files outside the folder it runs from, and Prettier and TypeScript look for their settings by walking up from the file they read, so the crates folder holds three small files. `crates/eslint.config.js` applies `lib/eslint.config.js` to `*/*/js/src` and `*/*/js/test` and ignores everything else under the crates folder. ESLint 9 finds it when it runs from the crates folder, as `npm run lint` runs it, and an editor uses it only when it runs ESLint from there with `lib`'s copy of ESLint. `crates/.prettierrc.json` gives files under those folders, and the three files themselves, `lib/.prettierrc.json`'s options and sets nothing for any other file, and a test checks the options are the same. `crates/tsconfig.json` extends `lib/tsconfig.json` for those folders and the shared core's `global.d.ts`, so an editor checks a crate script as `lib`'s TypeScript run does. Vite also reads it when it compiles a crate script, so `build.rs` watches it. + +`npm run lint` lints `lib` as before and then runs ESLint from the crates folder over `*/*/js/{src,test}/**/*.{ts,tsx,js,mjs}`, and `npm run format` checks those folders, each without `--config`, so the command line and an editor apply the same settings. The ESLint config treats `@tsjs/` imports as internal, orders the import groups built-in, external, internal, parent, sibling, index, and gives a crate's `js/test` `.mjs` scripts Node's globals as it gives `lib`'s. The CI jobs still run `npm run lint`, `npm run format`, `npm run build` and `npm test` in `crates/trusted-server-js/lib`, so no workflow changes for the scripts. + +The lint and format checks did not cover inline page scripts or carried scripts before this change, and they still do not. + +### 2.5 The Tinybird project + +`tinybird/` moves to `crates/analytics/tinybird/project/`. Its README's links to the telemetry and configuration guides follow the move, the crate's README points at the project, the telemetry and configuration guides' links follow it, the Markdown format list in `.github/workflows/format.yml` and `AGENTS.md` drops `tinybird/**/*.md`, which `crates/**/*.md` now covers, and the changelog says where the project now is, because an operator runs `tb` from that folder. Because the project is now under `crates/`, a merge that changes only the project starts the documentation proposal workflow, `.github/workflows/docs-proposal.yml`, which runs on changes under `crates/**`. + +### 2.6 Text that named the old places + +Comments and pages that named a moved script by its old path now name the new one, in `crates/ad-tag/google/src/lib.rs`, `crates/cmp/sourcepoint/src/lib.rs`, `crates/trusted-server-core/src/publisher.rs`, the Didomi test and the GPT guide. The old path also appeared in three comment lines in `crates/ad-tag/google/src/gpt_bootstrap.js`, which the GPT module inlines into each page it changes, so readers receive those three comment lines with the new path, and the module's recorded page-change fixture changes with them. That is the only change a reader receives. + +The `tsjs` guide said there are 12 integration source modules and 13 bundles. There are 13 and 14, because it missed `ec_client_fixed`, and it now says so, with where a vendor module's script lives. `AGENTS.md`, the onboarding guide, the error reference's entry for an invalid bundle set and the changelog name both places a module's script can be. + +## 3. What is not in this change + +- **APS's renderer**, `lib/src/integrations/aps/render.ts`, stays in the shared crate, because the shared browser core imports it in `core/auction.ts` and `core/request.ts` and it is compiled into the `core` bundle. It moves when APS rendering leaves the browser core. +- **Prebid**, its shim, its external bundle build `build-prebid-external.mjs` with its tests, and the Prebid code of the `ts prebid client` command in `crates/trusted-server-cli/src/prebid_bundle.rs`, moves together in a later change. The shim imports `prebid.js`, which a script in a crate cannot (2.3), and the CLI runs the bundle build from `lib` with `lib`'s `node_modules`. +- **The shared browser core still names vendors**, GPT, Prebid, APS, Lockr, DataDome and Google Tag Manager in `lib/src/core` and `lib/src/shared`. Taking them out is a later change, as [#1348](https://github.com/IABTechLab/trusted-server/pull/1348) and the pull requests above it did for the Rust core. +- **`creative` and `ec_client_fixed`** are core's own browser features and stay in the shared crate. +- **Vendor fixtures in the integration test crate** and **vendor pages in the docs site** stay, because the integration tests and the docs site build from their own folders. + +## 4. Acceptance + +- Each of the 14 bundles `build-all.mjs` writes has the same SHA-256 as on row 54. +- `npm run lint` and `npm run format` pass, and a planted unused variable in a crate's script makes `npm run lint` fail. +- Every Vitest file passes, including the discovery tests and the checks of the Prettier options and the `@tsjs/module` list, and the type gate covers the moved contract test, which a planted false `expectTypeOf` assertion makes fail. On Windows the two `build-prebid-external` tests that create symlinks fail, as they do on row 54, because Windows refuses symlinks without the privilege. +- `tsc --noEmit -p tsconfig.json` in `lib` reports no error that it does not report on row 54. +- `cargo test` and `cargo clippy -- -D warnings` pass for `trusted-server-js`, including its discovery tests, and for `trusted-server-ad-tag-google`, whose recorded fixture changed, and `cargo fmt --check` passes. +- The repository's workflows pass on the pull request. + +## Revision record + +| Date | Change | +| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-19 | First draft. | +| 2026-10-19 | Fixes from review round 1. TypeScript's `@tsjs/module/*` list and its test, the configuration guide's Tinybird link, the editor settings in the crates folder and their test, the npm package rule, `build.rs` watching `build-module-options.mjs`, discovery refusing `core` and passing over broken links on both sides with JavaScript tests, no unmatched-pattern pass for ESLint, `.mjs` crate tests, the three kinds of browser code a crate can hold, the changelog, `AGENTS.md` and the onboarding guide, and wording throughout. | +| 2026-10-19 | Corrections from review round 2. The `@tsjs/module` search claim, `crates/tsconfig.json` taking in the shared `global.d.ts`, how ESLint finds `crates/eslint.config.js`, `build.rs` watching `crates/tsconfig.json`, the documentation proposal workflow a Tinybird change starts, and Rust discovery passing over a module root that is a file. | +| 2026-10-20 | Names its pull request, IABTechLab/trusted-server#1423. | + +## Review log + +1. **Round 1, 2026-10-19.** A review in a fresh session, against `fixpass11/55-scripts-in-crates` as first committed at `7dc1671fe`, rewritten before it was pushed. Issues found, two of them blocking. TypeScript had no mapping for `@tsjs/module/*`, so the GPT script was no longer type-checked against the APS renderer while this design said TypeScript used the names, and the configuration guide still linked the old Tinybird README. Of the others, editors found no settings for crate scripts, a script in a crate cannot import an npm package and the design did not say so, `build.rs` did not watch `build-module-options.mjs`, the crate lint run passed an empty match, the JavaScript and Rust discovery differed on broken links, an `index.ts` folder and a module named `core`, crate `.mjs` tests were neither run nor given Node's globals, and there was no changelog entry. This revision fixes each of them. +2. **Round 2, 2026-10-19.** A lighter review in a fresh session, against this page at `b89557183`, of the round 1 fixes and the new text. No blocking issues. One round 1 item was only partly fixed, the claim that every import of one module by another uses `@tsjs/module/`, which the Prebid shim's relative import of the APS renderer contradicted. It also found that `crates/tsconfig.json` left out the shared `global.d.ts`, that ESLint 9 finds its config from the folder it runs in rather than from the file, that the build reads `crates/tsconfig.json` without `build.rs` watching it, that Rust discovery failed on a module root that is a file where JavaScript passed over it, that the new configuration files and `.mjs` test were outside the format check, and that a Tinybird change now starts the documentation proposal workflow. This revision fixes each of them, and the design joins the specs pull request when the row is pushed. From a854526e8186c20c4f0320761db6c14cd2cb267f Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Tue, 20 Oct 2026 07:45:41 +0100 Subject: [PATCH 169/192] Bring the module scripts design up to date The design now says why the Rust discovery tests run without a file system, the same text as on the branch of IABTechLab/trusted-server#1423. --- .../specs/2026-10-19-module-scripts-in-crates-design.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md b/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md index dfa0793cc8..c9bbd2d270 100644 --- a/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md +++ b/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md @@ -50,7 +50,7 @@ So a folder named `js/` holds either module scripts, in `js/src` and `js/test`, `moduleSourceDirs()` in `lib/build-module-options.mjs` lists every module's source folder by name, `lib/src/integrations/` for the modules that stay in the shared crate and `crates///js/src/` for the rest. A name found in two places is an error, because bundles and imports are keyed by name, and so is a module named `core`, which is the shared core's bundle. A link whose target is gone counts as no folder. `build-all.mjs` builds a bundle for each listed folder whose `index.ts` is a file. -`expected_module_ids` in `build/bundle_set.rs` applies the same rule from the Rust side, given the crates folder as well as `lib/src`, so `build.rs` still embeds exactly the bundles the Node build wrote. Each side has tests of a module found in a crate, a name found twice and a module named `core`, in `build/bundle_set.rs` and `lib/test/module-discovery.test.mjs`. Bundles are still named `tsjs-.js`, so the Rust API, the served URLs and every content hash are unchanged. +`expected_module_ids` in `build/bundle_set.rs` applies the same rule from the Rust side, given the crates folder as well as `lib/src`, so `build.rs` still embeds exactly the bundles the Node build wrote. Both sides have tests of the module list, a name found twice and a module named `core`, in `build/bundle_set.rs` and `lib/test/module-discovery.test.mjs`, and the JavaScript tests also walk folders in a scratch tree. The Rust tests check `module_ids`, which takes the folders the walk found, because the crate's tests also run as WebAssembly under Viceroy, where there is no file system. Bundles are still named `tsjs-.js`, so the Rust API, the served URLs and every content hash are unchanged. `build.rs` asks Cargo to rerun it when `lib/build-module-options.mjs` or any existing `crates///js/src` folder changes, and `js/test` as well when `TSJS_TEST=1` runs the tests. It names only folders that exist, because Cargo reruns a build script on every build while a watched path is missing. A crate's first `js/src` folder is therefore picked up by the next build that runs for another reason, which `crates/trusted-server-js/README.md` says, with touching `lib/package.json` as the way to force it. @@ -111,6 +111,7 @@ The `tsjs` guide said there are 12 integration source modules and 13 bundles. Th | 2026-10-19 | Fixes from review round 1. TypeScript's `@tsjs/module/*` list and its test, the configuration guide's Tinybird link, the editor settings in the crates folder and their test, the npm package rule, `build.rs` watching `build-module-options.mjs`, discovery refusing `core` and passing over broken links on both sides with JavaScript tests, no unmatched-pattern pass for ESLint, `.mjs` crate tests, the three kinds of browser code a crate can hold, the changelog, `AGENTS.md` and the onboarding guide, and wording throughout. | | 2026-10-19 | Corrections from review round 2. The `@tsjs/module` search claim, `crates/tsconfig.json` taking in the shared `global.d.ts`, how ESLint finds `crates/eslint.config.js`, `build.rs` watching `crates/tsconfig.json`, the documentation proposal workflow a Tinybird change starts, and Rust discovery passing over a module root that is a file. | | 2026-10-20 | Names its pull request, IABTechLab/trusted-server#1423. | +| 2026-10-20 | The Rust discovery tests check `module_ids` without a file system, because the WebAssembly test run under Viceroy has none. | ## Review log From 8057df453a92119213c625c0c71a5794dfa94f5e Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:26 +0000 Subject: [PATCH 170/192] Module endpoints replace the attestation design The module endpoints design, for row 12, says that a module outside core already declares any path through IntegrationProxy::routes(), that the registry refuses two routes on one method and path, and which steps each adapter runs before a module route. Row 12 adds tests of a path outside /integrations/ on all four adapters and a section of the integration guide, and core and the adapters gain no attestation type, setting or route. PlatformIdentity stays in core, in the platform module from row 15. The administration, diagnostic headers, Fastly names and request filters designs no longer describe the attestation document, its endpoint, its prefixes or its pages, and none of their headers calls its pull request a draft. --- ...tration-off-the-publisher-domain-design.md | 20 +- .../specs/2026-10-12-attestation-design.md | 219 ------------------ .../2026-10-12-diagnostic-headers-design.md | 35 ++- .../2026-10-15-fastly-names-in-core-design.md | 23 +- ...request-filters-on-every-adapter-design.md | 14 +- .../2026-10-26-module-endpoints-design.md | 92 ++++++++ 6 files changed, 140 insertions(+), 263 deletions(-) delete mode 100644 docs/superpowers/specs/2026-10-12-attestation-design.md create mode 100644 docs/superpowers/specs/2026-10-26-module-endpoints-design.md diff --git a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md index 19d96373e4..27651fa20e 100644 --- a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md +++ b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md @@ -1,11 +1,11 @@ # Design Spec: Administration leaves the publisher's domain -**Status:** Proposed, 2026-10-12. Raised as draft pull request [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354), row 13 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), answering [IABTechLab/trusted-server#1271](https://github.com/IABTechLab/trusted-server/issues/1271), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#8](https://github.com/jwrosewell/trusted-server/pull/8) and [IABTechLab/trusted-server#1298](https://github.com/IABTechLab/trusted-server/pull/1298), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Raised as pull request [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354), row 13 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), answering [IABTechLab/trusted-server#1271](https://github.com/IABTechLab/trusted-server/issues/1271), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#8](https://github.com/jwrosewell/trusted-server/pull/8) and [IABTechLab/trusted-server#1298](https://github.com/IABTechLab/trusted-server/pull/1298), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-inspection-endpoints-design.md` (row 9, which builds `GET /_ts/data`) and `2026-10-12-attestation-design.md` (row 12, which this row stacks on), in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) -**Related PRs:** [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354) (row 13), [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353) (row 12, attestation, which row 13 stacks on), [IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350) (row 9, the inspection endpoints), [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7, the integration seam, which replaced the closed [IABTechLab/trusted-server#1094](https://github.com/IABTechLab/trusted-server/pull/1094)), [IABTechLab/trusted-server#1377](https://github.com/IABTechLab/trusted-server/pull/1377) (row 36, which later changes the closed path matcher) +**Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-inspection-endpoints-design.md` (row 9, which builds `GET /_ts/data`) and `2026-10-26-module-endpoints-design.md` (row 12, which this row stacks on), in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) +**Related PRs:** [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354) (row 13), [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353) (row 12, which row 13 stacks on), [IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350) (row 9, the inspection endpoints), [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7, the integration seam, which replaced the closed [IABTechLab/trusted-server#1094](https://github.com/IABTechLab/trusted-server/pull/1094)), [IABTechLab/trusted-server#1377](https://github.com/IABTechLab/trusted-server/pull/1377) (row 36, which later changes the closed path matcher) **Issues:** [IABTechLab/trusted-server#1271](https://github.com/IABTechLab/trusted-server/issues/1271), and [IABTechLab/trusted-server#1149](https://github.com/IABTechLab/trusted-server/issues/1149) for a store the development files name. [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-17 +**Last updated:** 2026-10-26 ## 1. Problem @@ -59,11 +59,11 @@ Neither command takes `current-kid` or `active-kids` for a key. They are the two **Closed paths.** `closed_path_response`, in core's new `crates/trusted-server-core/src/closed_paths.rs`, answers `404` to a request whose path begins with `/_ts/admin`, and to `/admin/keys` and the paths beneath it, before the publisher fallback, on all four adapters. The prefix is matched as plain text, so `/_ts/adminec` is closed with `/_ts/admin/ec`, while `/admin/keystore` and `/admin/login` are left to the publisher. The answer is the same for every method, `404` with `{"error":"not found"}`, `Cache-Control: no-store` and `X-Content-Type-Options: nosniff`, where some removed routes answered a method they did not serve with `405`. The answer does not say where administration went, and neither `closed_path_response` nor the adapters' calls to it write a log line of their own. An operator's script that was not updated sends its password and its key payload to a path that answers `404`, and never to the origin. -The matcher is the one that was there before. It tests the path as sent and after each of up to four rounds of percent-decoding (turning `%2F` back into `/`), so a separator encoded up to four times is recognised and one encoded five or more times is not. Row 36 (the publisher origin as a folder of signed object storage, [IABTechLab/trusted-server#1377](https://github.com/IABTechLab/trusted-server/pull/1377)) shares one decoding helper between the origin check and the closed path check, so that a path still changing after the fourth round is closed. +The matcher is the one that was there before. It tests the path as sent and after each of up to four rounds of percent-decoding (turning `%2F` back into `/`), so a separator encoded up to four times is recognized and one encoded five or more times is not. Row 36 (the publisher origin as a folder of signed object storage, [IABTechLab/trusted-server#1377](https://github.com/IABTechLab/trusted-server/pull/1377)) shares one decoding helper between the origin check and the closed path check, so that a path still changing after the fourth round is closed. Fastly, Cloudflare and Spin make the check the first step of their fallback, ahead of the modules' request preparers (each can clean up a query parameter or cookie its module reserves). Axum makes the check in `fallback_handler`, ahead of the preparers, in the same order (`23bde99bc`). -The check sits in the publisher fallback, so a route that the router answers first is not covered by the check. The settings choose one such route, the `[attestation] endpoint`, and the list of addresses an endpoint may not take lost its six administration addresses in this change (`crates/trusted-server-core/src/attestation.rs`, lines 49 to 66). `validate_endpoint` refuses an endpoint whose page or JSON address the closed path check closes, as `reserved_endpoint` naming the address (`df0fc18a0`), so no route is served beneath the prefix, as the API reference says (`docs/guide/api-reference.md`, line 985). +The check sits in the publisher fallback, so a route that the router answers first is not covered by the check. After this change no adapter registers a named route beneath a closed path, so every request for a closed path reaches the check. A module's route is answered in the publisher fallback after the check, so a module route beneath a closed path is never reached (`2026-10-26-module-endpoints-design.md`). **On the publisher's own site.** @@ -89,7 +89,7 @@ The check sits in the publisher fallback, so a route that the router answers fir - A configuration that carries `[[handlers]]` is refused when it is validated and when the service starts, with directions (`crates/trusted-server-core/src/settings.rs`, lines 3521 to 3538, with tests at lines 5649 to 5707). The key itself is refused, so `handlers = []` is refused too. Loading the list and ignoring it would leave an operator who gated a staging site believing the site was still gated. The settings this version writes carry no `handlers` and no store ids. - A `[[handlers]]` entry could also gate a route of this server's own that the application answers, `GET /_ts/config` for instance. Nothing here replaces that gate, so a route of the server's that should not be public stays switched off, and `[inspect] config = false` answers `/_ts/config` with not found. `GET /_ts/debug/ja4` was never gated that way, because the Fastly entry point answers that path before the application is built (`crates/trusted-server-adapter-fastly/src/lib.rs`, lines 465 to 481), and `GET /_ts/debug/sandbox` is answered there too (line 444), but only in a build that has the `reusable-sandbox` feature (line 443). The feature is off by default, and the build command in `fastly.toml` does not pass it. - A configuration that carries `config_store_id` or `secret_store_id` under `[request_signing]` is refused the same way, and enabling request signing no longer needs them. -- A build before this change refuses a configuration without a handler covering `/_ts/admin`, and one with `[request_signing]` and no store ids (row 12, `crates/trusted-server-core/src/settings.rs`, lines 4161 and 4322 to 4329). So no one configuration starts both builds, and the new configuration and the new build go live together. A staged deploy reads a configuration key of its own (`ts deploy --staging`, `ts config push --staging`), which lets the pair be checked together first. +- A build before this change refuses a configuration without a handler covering `/_ts/admin`, and one with `[request_signing]` and no store ids (row 12, in `crates/trusted-server-core/src/settings.rs`). `Settings::validate_admin_coverage` makes the first refusal. For the second, `config_store_id` and `secret_store_id` are required fields of `RequestSigning` with no default, so a `[request_signing]` table without them cannot be read, and `Settings::reject_placeholder_secrets` refuses an empty, placeholder or padded value. So no one configuration starts both builds, and the new configuration and the new build go live together. A staged deploy reads a configuration key of its own (`ts deploy --staging`, `ts config push --staging`), which lets the pair be checked together first. - **The switch.** `ts deploy` and `ts config push` are separate commands, so the build and the configuration do not change in one step. When the settings fail to load, the router that the Fastly adapter builds for a failed startup answers every route, the publisher's pages included, with the startup error (`crates/trusted-server-adapter-fastly/src/app.rs`, lines 75 to 82), which is status `500` for a configuration error. A request that loads the settings between the first of the two changes and the second therefore meets that error. - **Rollback** has the same shape. The earlier build refuses the new configuration, so the earlier configuration has to be at the key when the earlier build is activated, and `ts rollback --rollback-to` re-activates a build version and says nothing of the configuration (`docs/guide/cli.md`, lines 280 to 283). The CHANGELOG says a deployment that linked the `api-keys` store can unlink it and revoke the token. The earlier build reads that token for its key routes, so a rollback after that needs the store and a token again. Open question 7 asks for the order. - `ts cache purge` takes `--service-id` and `FASTLY_API_TOKEN` where it took `--service ` and `TRUSTED_SERVER_ADMIN_PASSWORD`, and no longer takes `--username`. It runs `fastly service purge`, which the Fastly CLI has had since version 14.0.0. The repository pins 15.1.0 (`.tool-versions`). @@ -107,7 +107,7 @@ If refusing is too abrupt for a live deployment, the alternative is small. The r - **The smoke scripts.** The four adapter smoke scripts lose the handler password and its missing-secret case, and no workflow runs them. - **Development store names.** `every_store_a_development_file_names_is_declared`, in the integration tests' `tests/common/config.rs`, checks that every store an uncommented `*_STORE` line of `.env.dev` or `.env.example` names is declared in `fastly.toml` and in the Viceroy template (`e110aa025`). -**Tests added after review.** `rotate_never_replaces_a_secret_stored_under_the_new_kid`, `the_recorded_404_of_the_real_cli_reads_as_not_found` and `the_value_is_read_from_the_fields_the_real_cli_prints` in the CLI, `an_endpoint_beneath_the_closed_administration_prefix_is_refused` in core, and `a_closed_path_is_answered_before_any_preparer_runs` in the Axum adapter's seam probe tests. +**Tests added after review.** `rotate_never_replaces_a_secret_stored_under_the_new_kid`, `the_recorded_404_of_the_real_cli_reads_as_not_found` and `the_value_is_read_from_the_fields_the_real_cli_prints` in the CLI, and `a_closed_path_is_answered_before_any_preparer_runs` in the Axum adapter's seam probe tests. **What the tests leave out.** No test runs the real `fastly` CLI against a store that exists, so a rotation, a retirement, a repeated `--delete` and a purge are not recorded against a real store or service. @@ -115,7 +115,7 @@ If refusing is too abrupt for a live deployment, the alternative is small. The r **CI.** All 25 checks pass on the pull request for this row ([IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354)) at `f735e0f85`. The fork's pull request ([jwrosewell/trusted-server#8](https://github.com/jwrosewell/trusted-server/pull/8)) ran 19 checks on the same commit. -**Local checks.** The description of the pull request records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run, and marks a manual run with `fastly compute serve` as not done. The five commits after review, `11bf7b76d` to `e75e307de`, were checked with rustfmt and Prettier and were not built locally. The two commits for #1149, `1666c38e3` and `e110aa025` on `7ba7d7b55`, were built locally. `every_store_a_development_file_names_is_declared` passes, and with the `opid_store` line put back in `.env.dev` it fails with ``.env.dev sets TRUSTED_SERVER__SYNTHETIC__OPID_STORE to `opid_store`, which fastly.toml does not declare``. Clippy on the integration tests, with every target and warnings refused, passes, and so does `cargo fmt --all -- --check`. +**Local checks.** The description of the pull request records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run, and marks a manual run with `fastly compute serve` as not done. The five commits after review, `11bf7b76d` to `e75e307de`, were checked with rustfmt and Prettier and were not built locally. Among them `df0fc18a0` refused an endpoint the settings chose beneath `/_ts/admin`, and the later `ee544e27f` fixed the Cloudflare test of that endpoint's routes. Both stay in this row's history, and the merge of row 12's removal takes out what each changed. The two commits for #1149, `1666c38e3` and `e110aa025` on `7ba7d7b55`, were built locally. `every_store_a_development_file_names_is_declared` passes, and with the `opid_store` line put back in `.env.dev` it fails with ``.env.dev sets TRUSTED_SERVER__SYNTHETIC__OPID_STORE to `opid_store`, which fastly.toml does not declare``. Clippy on the integration tests, with every target and warnings refused, passes, and so does `cargo fmt --all -- --check`. **The command lines.** Every flag the two commands pass is in the `--help` of Fastly CLI 15.1.0, which `.tool-versions` pins, and of 16.1.0. On 2026-10-16 both versions answered read-only calls for a missing config store, a missing entry, a missing secret store and a missing secret with the wording in section 2, and printed `item_value` as a top level string in `config-store-entry describe --json`. No run has written to a store or purged. @@ -134,6 +134,8 @@ One question is open, about the guides, and this spec does not decide it. | 2026-10-16 | Fixes from review round 2. Open question 5 names the changelog entry on the two store id settings and the checks on row 12 that also read them, the sandbox answer is stated to exist only in a build with the `reusable-sandbox` feature, and the check counts of 25 on the row 13 pull request and 19 on the fork's were read again and stand. | | 2026-10-16 | Fixes from code review. The test fake answers with the 404 text the CLI prints and refuses to replace a secret (`11bf7b76d`), an attestation endpoint beneath `/_ts/admin` is refused (`df0fc18a0`), Axum answers a closed path before the preparers (`23bde99bc`), every adapter test sends the purge path and Fastly's the double-encoded paths (`18aba165c`), and three comments and two changelog entries are corrected (`e75e307de`). Open questions 1 to 6 are answered. | | 2026-10-17 | For IABTechLab/trusted-server#1149, the development files and the Viceroy template no longer name `opid_store` (`1666c38e3`), and a test checks that every store a development file names is declared (`e110aa025`). | +| 2026-10-26 | Row 12 adds no endpoint the settings choose any more, so section 2 no longer names one among the routes the closed path check does not cover, and says that a module route beneath a closed path is never reached. The header names `2026-10-26-module-endpoints-design.md` for row 12. | +| 2026-10-26 | What `df0fc18a0` changed to refuse an endpoint the settings chose beneath `/_ts/admin`, and what `ee544e27f` changed in the Cloudflare test of that endpoint's routes, leave this row in the merge of row 12's removal, while both commits stay in its history. Section 5 drops the test `df0fc18a0` added, and section 6 says what leaves. Section 4 names the two checks of row 12 by function, with the store id fields, and the header no longer calls the pull request a draft. | ## Review log diff --git a/docs/superpowers/specs/2026-10-12-attestation-design.md b/docs/superpowers/specs/2026-10-12-attestation-design.md deleted file mode 100644 index 07c3e9c80c..0000000000 --- a/docs/superpowers/specs/2026-10-12-attestation-design.md +++ /dev/null @@ -1,219 +0,0 @@ -# Design Spec: Attestation - -**Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353), answering [IABTechLab/trusted-server#161](https://github.com/IABTechLab/trusted-server/issues/161). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. -**Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-01-15-attestation-design.md` -**Related PRs:** IABTechLab/trusted-server#1353, which is row 12 of the stack, builds on one app-config blob for each host in [IABTechLab/trusted-server#1352](https://github.com/IABTechLab/trusted-server/pull/1352) and was reviewed on [jwrosewell/trusted-server#7](https://github.com/jwrosewell/trusted-server/pull/7) (closed without merging on 15 October). [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) opens the integration so that a vendor module can live outside core, and [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354), row 13, follows this change -**Issues:** [IABTechLab/trusted-server#161](https://github.com/IABTechLab/trusted-server/issues/161) asks for attestation, [IABTechLab/trusted-server#1235](https://github.com/IABTechLab/trusted-server/issues/1235) asks for attested release binaries, and [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-17 - -> **Why this spec exists.** The attestation issue (#161) asks for a reviewed design before any attestation lands. This spec is that design for the code at `59e09b6cd` and the three fixes of 16 October at `694926002`. It states what the code does, what the code leaves out and which questions about the code are open. - -## 1. Problem - -The Tech Lab's attestation issue (#161) asks for a reviewed design before any attestation lands, and its January proposal (`2026-01-15-attestation-design.md`) names two required core decisions, configuration kept apart from the build and modular integration crates. Nothing in core serves evidence of who operates a deployment and which build it runs, so a reader, a publisher or a regulator has only the operator's word. - -Line references in this document are to commit `59e09b6cd`, the head of `feature/12-attestation`, unless a row is named. The fixes of 16 October, at `694926002`, are named by function rather than by line. - -## 2. Design - -**What the evidence proves, and what it does not.** A signature that verifies proves that whoever made it held the operator's signing key. Under that signature the evidence states the host the request arrived for, the publisher whose settings answered, the platform service and the build that answered, the time, and the nonce when one was sent. A nonce is a value the caller makes up and sends, so that an answer carrying it must have been made after the request. A relying party is a vendor working with the publisher that depends on the evidence, and a verifier is a service that checks the evidence and tells the relying party whether it is genuine. - -It does not prove four things, and the guide (`docs/guide/attestation.md`) says so first. - -1. **What code is running.** A module on a CDN cannot read its own code, so the `build` fields are the operator's own report of what it deployed. -2. **A hardware measurement.** Nothing here takes one. The trust rests on the operator's key alone. -3. **What the site serves.** The evidence says who operates the deployment serving a host, not what pages the origin returns through it. -4. **That production answered.** A staging environment signs with the same key. On Fastly the evidence says so in `platform.staging`. The Axum, Cloudflare and Spin adapters read no staging signal and sign `staging: false` whatever the environment, so on those adapters `false` does not show that production answered. Open question 1 asks whether `false` is the right value there. - -**Configuration.** The `[attestation]` section has four fields and refuses any field it does not read, so a settings document cannot name a signing key. An invalid section is refused when the settings load and when `ts config push` checks the document, with the field and the rule in the message. - -```toml -[attestation] -operator = "Example Operator" -context = "example-attestation:v1" -verify_url = "https://verifier.example/verify?host=publisher.example" -``` - -| Field | Rule | Meaning | -| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | -| `endpoint` | Optional, and `/_ts/attestation` when left out. An absolute path of two or more segments, 64 characters at most, of lower case letters, digits, `_` and `-`. The first segment starts with `_` and has at least one more character. No `.` is allowed, so an endpoint cannot end in `.json`. Neither the page nor its `.json` form may be an address in `RESERVED_PATHS`, and the first segment may not be one of `_app`, `_astro`, `_content`, `_expo`, `_framework`, `_next`, `_nuxt` and `_vercel` | Where the page answers. The JSON form is at the same address plus `.json` | -| `operator` | Required. 1 to 64 characters | Who the evidence says operates the deployment | -| `context` | Required. 1 to 64 characters, none of them a control character | The text signed ahead of the evidence. A verifier must use exactly the same text | -| `verify_url` | Required. An `https` address with a host | Where the page sends a reader to check the claim, and the envelope's `verify` field | - -`RESERVED_PATHS` in `attestation.rs` lists the addresses beneath an underscore prefix that a deployment answers itself, such as `/_ts/config`, `/_ts/permissions`, `/_ts/set-tester` and the addresses under `/_ts/api/v1`. Each of the four adapters has a test that fails when a fixed route beneath an underscore prefix is missing from the list, because two routes at one address stop the router being built. Row 13 removes six `/_ts/admin` addresses from the list, because that row removes those routes. - -| Address | Answer | -| ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `[attestation] endpoint`, which defaults to `/_ts/attestation` | A page stating the claim, linking to the verifier, giving the JSON address to copy and showing the envelope. It says "Do not take this page's word for it." | -| The same address with `.json` | The envelope, as `application/json` | -| The default address and its `.json` form, where the endpoint is not the default | `301` to the endpoint's page or JSON address, with the query kept | -| An endpoint's own prefix, such as `/_ex`, with or without a trailing `/` | A list of the two pages. Any other address beneath that prefix is `404` from the deployment | - -**Methods.** GET and HEAD reach the pages, the list at an owned prefix and the redirect. An endpoint under a prefix of its own, such as `/_ex/attestation`, owns that prefix. Every other method on the two pages and on the prefix itself, and every method on any other address beneath the prefix, is then answered `404` by the deployment and never reaches the publisher's origin. The default prefix `/_ts` is shared with other routes, so under it only GET and HEAD on the attestation addresses are taken, being the two pages and the redirect from the default address, and every other method on those addresses reaches the publisher's origin. Each adapter registers the routes for the seven methods its publisher fallback takes, being GET, POST, HEAD, OPTIONS, PUT, PATCH and DELETE. The routes exist only when the settings carry `[attestation]`, and without the section every attestation address, `/_ts/attestation` included, reaches the publisher's origin as any other path does. An operator has to choose a prefix that the operator's origin serves nothing beneath, because the deployment answers `404` for everything there, and the settings refuse the eight prefixes that common web frameworks serve their assets from. - -The envelope carries the evidence's exact bytes as signed, in base64url (base64 written with `-` and `_` in place of `+` and `/`, and with no `=` padding), the signature with its key id and algorithm, a readable copy of the evidence that no check reads, and the address of the verifier. The evidence has these fields, in this order. - -| Evidence field | Source | -| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `v` | The format version, `1` | -| `operator` | `[attestation] operator` | -| `publisher` | `publisher.domain` in the settings | -| `host` | The request's `Host` header, lower case, with any numeric port removed. The request address's host stands in when there is no `Host` header and when the header is not readable text, meaning that it holds a byte that is neither printable ASCII nor a tab. The field is empty when the request address has no host either. A forwarded host header is never read | -| `platform` | `name`, `serviceId`, `serviceVersion`, `staging` and `pop`, in that order. On Fastly, `name` is `fastly`. `serviceId`, `serviceVersion` and `pop` come from `FASTLY_SERVICE_ID`, `FASTLY_SERVICE_VERSION` and `FASTLY_POP` and read `unknown` when the variable is absent or empty. `staging` is true only when `FASTLY_IS_STAGING` is `1` | -| `build` | `version`, `builtAt`, `commit`, `run` and `tag`, in that order. `version` is the crate version. The other four are the facts the build was given, and `commit`, `run` and `tag` read `unknown` when the build was given none | -| `nonce` | The caller's `nonce` query parameter, only when one was sent | -| `issuedAt` | The platform's clock, RFC 3339 in UTC to the second | - -The Axum, Cloudflare and Spin adapters report their own name in `platform.name`, `unknown` for `serviceId`, `serviceVersion` and `pop`, and `false` for `staging`. - -The signed bytes are the configured context, one line feed, then the payload. The signature is ECDSA on P-256 with SHA-256 (`ES256` in the envelope), made as RFC 6979 describes, so the same evidence always carries the same signature. The envelope holds the signature as the raw 64 byte pair of `r` and `s`, in base64url. The context stops a signature the same key made for any other purpose from passing as evidence. - -A build is given its facts and its keys as build inputs. This change adds the five below. - -| Build input | What it gives | Without it | -| ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | -| `TRUSTED_SERVER_ATTESTATION_KEYS` | A file, the key schedule, with one key on each line as the key id, the Unix second the key comes into force and the private key (base64 of its SEC1 or PKCS #8 DER encoding), separated by tabs. No two lines may share a key id or a start | No key, so the endpoint answers `503` and nothing else changes | -| `TRUSTED_SERVER_COMMIT`, `TRUSTED_SERVER_BUILD_RUN` and `TRUSTED_SERVER_BUILD_TAG` | `build.commit`, `build.run` and `build.tag` | `unknown`, which is also what a value that is empty or holds a control character gives | -| `SOURCE_DATE_EPOCH` | `build.builtAt` | The time of the build, which is also what a value that is not a whole number of seconds gives | - -The build writes the schedule into `attestation_keys.rs` in its output directory, and the attestation module includes that file, so each private key is text inside the binary. The build stops, with the number of the line and the reason, when a schedule line does not hold a key id, a start and a key separated by tabs, when the start is not a number, when a key id or key holds a character other than letters, digits and `-_.:+/=`, when a line repeats the key id or the start of an earlier line, or when a key cannot sign, read with the function that signs (`read` in `signing_key.rs`). The message never repeats the key. A schedule file that cannot be read also stops the build. The shape and the characters are checked in `key_schedule.rs` (lines 24 to 62 at `59e09b6cd`), which then reads each key, however late it starts. - -How a verifier gets the public keys, and the period each is in force, is not part of this change. No tool in this repository exports a public key. The harness (`scripts/attestation-local-test.sh`, described in section 5) makes its keys with OpenSSL and derives the public keys from them (lines 92 to 101), and the unit tests derive them from test keys. Open question 3 asks how a verifier gets the keys. - -**How it meets the scope of #161.** The first eight rows are the items of the proposed scope in #161, as labels. The last three are its "Done when", quoted. - -| #161 asks for | Where it is | -| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| The verifier | Outside this repository. The guide's "Verifying evidence" lists the seven checks a verifier makes and which of them need the relying party's input. The harness below is a small verifier of the signature and the fields | -| The threat model | "What the evidence proves", "Replay and relay" and "Limits" in the guide. Replay onto another domain, relay through another site and a stored answer are each met by a field inside the signed evidence, being `host`, `nonce` and `issuedAt`. A copied page is met only when the relying party builds its own verify address from the domain it is dealing with and does not follow the link on the page, because `verify` sits outside the signed payload and whoever serves a copied page writes it | -| The trust root | The operator's key schedule. A verifier holds the matching public keys and the period each is in force, and how it gets them is not defined here | -| The claim set | The evidence table above, version `1` | -| Freshness and replay | `issuedAt` and the caller's nonce are signed, so a verifier refuses an old time or a nonce that is not its own. Every answer is sent `Cache-Control: no-store` | -| Which configuration changes affect the claims | `publisher.domain` and `[attestation] operator` are claims. `[attestation] context` is signed ahead of the evidence, so a verifier has to use exactly the same text. Nothing else of the settings is claimed, and `/_ts/config`, which the inspection endpoints of row 9 ([IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350)) added, is where a deployment shows the settings it runs | -| Claim creation and signing in the deployment lifecycle | Keys are a build input. The key in force is the latest to have started, so a build carries the next key before it is needed. While the key in force is the last one the build holds, the deployment logs one warning for each process and goes on signing with that key. The proof reaches a verifier in the envelope, which a relying party fetches from the publisher's own address | -| Safe failure and rotation, exposing no secret | No key in force, or a clock that cannot be read, answers `503` at the endpoint alone, and a key that cannot sign stops the build. A settings document cannot name a key, and the section refuses any field it does not read. The private keys are readable by whoever holds the binary or the build's output directory, as "What it costs" below says | -| "A reviewed design identifies the exact claim schema, signer/verifier ownership, configuration inputs, rotation, and failure semantics." | This spec. The claim schema is the evidence table, the signer is the operator's build and the verifier is outside this repository, the configuration inputs are the section and the build inputs, rotation is the key schedule, and failure semantics are the answers in section 3 | -| "Tests prove that an authorized deployment verifies and that tampered, stale, or mismatched claims fail." | The unit tests and the harness verify a signature with the public key alone, and show that a changed host in the payload, the other key of the schedule and another context each fail. A mismatch between the evidence and the domain a relying party asked about is for the verifier to refuse, and no verifier is in this repository. No test makes stale evidence fail. Open question 2 asks which tests to add | -| "The agreed transport is exercised end to end and documented for operators and verifiers, or the investigation records why attestation cannot be supported as proposed." | `scripts/attestation-local-test.sh` exercises the transport on Fastly under Viceroy (Fastly's local runtime) and runs in CI, and `docs/guide/attestation.md` documents it for operators and verifiers. No test exercises signed evidence through Axum, Cloudflare or Spin, because test builds carry no key | - -**Where this meets #1235.** The issue for attested release binaries ([IABTechLab/trusted-server#1235](https://github.com/IABTechLab/trusted-server/issues/1235)) gives its reason as "An attestation only means something if every publisher runs the same bytes", and its Constraints say "No configuration may be compiled into the release binary. Anything a deployment varies goes through `trusted-server.toml` and `ts config push`, or through platform stores. Other features have to fit this rule, not the other way around." This change compiles the operator's signing keys into the build, so the keys do not fit that rule. The two meet here and the choice is one for the task force. The same issue asks for every compile-time input of the release binary to be listed, and this change adds five, being `TRUSTED_SERVER_ATTESTATION_KEYS`, `TRUSTED_SERVER_COMMIT`, `TRUSTED_SERVER_BUILD_RUN`, `TRUSTED_SERVER_BUILD_TAG` and `SOURCE_DATE_EPOCH`. - -- **What compiling the keys buys.** No key sits in a store the settings or the running service can reach. A missing or broken store entry cannot take a site down, where the settings loader refuses the whole configuration when a secret the configuration names is absent. A build with no schedule answers `503` at one address. -- **What it costs.** An operator who attests builds its own binary, so that binary is not the Tech Lab's released bytes. The private keys are also readable by anyone who holds the binary, because they are compiled in as text, and from `target/**/out/attestation_keys.rs`, the file the build script writes them to, which outlives the build and which a CI cache of the target directory keeps. The guide tells an operator to build only where the operator controls the machine, to send the binary only to the operator's own platform account, and to use a target directory and a cache that only the operator can read or to delete the file once the binary is made. -- **The other custody.** Reading the schedule from the secret store keeps one binary for everyone. The signing step is two functions over a schedule, `key_in_force` and `sign`, but moving the schedule is a larger change than those two. The handler is given the settings, the platform identity and the request and no services, so the handler has no store to read from (`handle_prefix`, lines 262 to 266 of `attestation.rs`). The section refuses any field it does not read, so the settings would need a new field naming the secret. Each of the four adapters would also pass its services to the handler. - -The same issue asks "Which build facts the runtime can report for #161, such as version and git commit". Here they are the version, the build time, the commit, the build run and the tag, each the builder's own report. - -**Where this differs from the January proposal.** `docs/superpowers/specs/2026-01-15-attestation-design.md` proposed a signed in-toto statement with a config hash and an integration list at `/.well-known/trusted-server-attestation.json`, and claims carried in each bid request. This change is narrower. The evidence claims who operates the deployment and which build the operator says it runs, in an envelope a verifier checks with one public key and without first writing the JSON into one agreed form, because the bytes are checked exactly as they arrived. Nothing here prevents a config hash or request claims being added to the evidence under a later version. - -## 3. What changes for a deployment - -**Nothing, until the settings carry `[attestation]`.** Without the section no attestation route is registered, so every attestation address reaches the publisher's origin as any other path does. The settings leave the section out of a pushed document when it is absent, so a deployment that never adds the section pushes the same document as before this change. This change adds one dependency to core, the `p256` crate, and the build inputs above. - -**Adding the section.** A binary built from row 11, the row below this one, reads an `[attestation]` table as the section of a module type such as `[cmp]` or `[tag]`, because the settings read every top-level table they do not know as one (`TypeSections` in `settings.rs`). That binary then finds that `context`, `operator` and `verify_url` are not settings a module type's section reads (`SectionModules::from_entries` in `provider_table.rs`) and refuses the whole document. A service can therefore take a document that carries the section only once its binary carries this change. A service rolled back past this change needs a document without the section first, as the settings tests say of any configured section added since the base revision (`settings.rs`, lines 5244 to 5247). - -**The evidence format.** The format version is `1`. `tag` joined the `build` fields on 13 October, after the first draft of this spec, and the version stayed `1` (commit `a656298ec`). A verifier written against the four build fields of that draft still verifies the signature of evidence that carries `tag`, because the bytes are checked as they arrived, and whether such a verifier accepts a field it does not know is the verifier's own rule. The code holds no rule for when a field added to the evidence needs a new version. Open question 4 asks what that rule is. - -**What each adapter reports.** Fastly reads the service's environment, as the evidence table says. Axum, Cloudflare and Spin report their own name, `unknown` as the service id, service version and location, and `false` as `staging` (`PlatformIdentity::named`, lines 147 to 159 of `attestation.rs`). - -**How the adapters answer.** All four adapters register the attestation routes the same way, after the fixed routes and before the catch-all routes that send the rest to the publisher's origin. `attestation_prefix_parity` checks that Axum, Cloudflare and Spin answer the same cases alike, and the Fastly adapter has its own tests of the same cases. On every adapter the handler builds no Edge Cookie context, so it reads, creates and refreshes no Edge Cookie. On Fastly the answer also comes ahead of the request filters, so a verifier fetching the answer from a server is not turned away as a bot (`attestation_route_handler`, lines 1581 to 1597 of the Fastly adapter's `app.rs`). The harness runs the Fastly adapter only. - -**Request preparers.** A request preparer is code that a registered integration runs on a request before routing, for example to remove a query parameter or a cookie that the integration reserves. All four adapters answer the attestation routes without running the preparers, so no preparer can change or refuse the request whose host and nonce are signed. Cloudflare does so through its own `attestation_route_handler` rather than `make_handler`, which runs the preparers first. The Cloudflare route test `attestation_routes_skip_the_request_preparers` registers a preparer that refuses every request, shows that it refuses a request for the publisher's origin, and checks that the list at the prefix, the JSON address and the redirect from the default address answer as they do without it. No test sends a request through a preparer on the attestation routes of Axum, Fastly or Spin. - -**Router middleware.** Each adapter's router wraps every route, the attestation routes included, in its own middleware. That middleware runs basic authentication, so a `[[handlers]]` entry whose `path` pattern matches the endpoint makes the endpoint ask for credentials, and its last step adds the standard response headers to the answer. On Fastly the standard headers include geo headers for the caller's address when a lookup answers, and the operator's `[response_headers]`. - -**Answers.** The 400, 500 and 503 answers are JSON at both addresses. Every answer, the redirect and the errors included, carries `Cache-Control: no-store` and `Access-Control-Allow-Origin: *`, so a relying party's page can read the JSON from a browser and no cache hands an old answer out again. - -| Status | When | Body | -| ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | -| 200 | Evidence was signed | The page, or the envelope at the `.json` address | -| 301 | A read of the default address, or its `.json` form, where the endpoint is not the default | `{"location":""}`, with a `Location` header | -| 400 | The `nonce` is empty, longer than 64 characters or holds a character other than `A` to `Z`, `a` to `z`, `0` to `9`, `_` and `-` | `{"error":"A nonce is 1 to 64 characters from A to Z, a to z, 0 to 9, _ and -."}` | -| 404 | Any other address beneath a prefix the deployment owns, and any method other than GET and HEAD on the pages and on the prefix there | A page pointing to the list at the prefix | -| 404 | The settings carry no `[attestation]` section and the handler is called anyway. No adapter registers a route then, so a request never reaches the handler | `{"error":"This deployment serves no attestation."}` | -| 500 | The evidence could not be written | `{"error":"The evidence could not be written."}` | -| 503 | No key in the build has come into force, or the build carries no key at all | `{"error":"No signing key is built in for this time, so no evidence can be signed."}` | -| 503 | The clock cannot be read as a time after the Unix epoch | `{"error":"The clock could not be read, so no evidence can be signed."}` | -| 503 | The key in force is not base64 or is not a P-256 private key in SEC1 or PKCS #8 form. The build refuses such a key, and the handler keeps this answer for one that still cannot sign when it runs | `{"error":"The signing key could not be read, so no evidence can be signed."}` | - -**Failure.** A failure stays at the attestation addresses, so no other address of the deployment changes. No other key signs in place of a key that cannot be read, because the key in force is the latest to have started. The build stops for the schedule faults listed under the build inputs, a key that cannot sign among them however late it starts, so no build carries a key that would answer `503` from its start time. The build reads each key with the function the handler signs with (`read` in `signing_key.rs`, which the build script and `attestation.rs` both include), and the handler keeps its `503` for a key that still cannot sign when it runs. The handler reads the time with `checked_current_timestamp` and answers `503` and signs nothing when the clock cannot be read as a time after the Unix epoch. - -**Consent and privacy.** The handler reads the request's `Host` header, the request's `nonce` parameter and the headers that name the scheme. The handler takes the scheme from the first of `Forwarded`, `X-Forwarded-Proto` and `Fastly-SSL` that gives one and uses `http` when none does, because the handler reads no TLS details of the connection. Only the JSON address that the page gives to copy uses the scheme, so the evidence never holds the scheme. Fastly and Spin remove those three headers from the request and then set the scheme themselves, Fastly from the TLS connection and Spin from the full URL that the Spin runtime reports. Axum and Cloudflare pass the headers on as they arrived, so on those two adapters the caller, or a proxy in front of the deployment, decides the scheme in the JSON address on the page. The handler reads no cookie, no consent signal and no store, makes no outbound request and sets no cookie. The evidence holds the host and the nonce that the caller sent and facts about the deployment, and nothing else about the caller. The envelope tells whoever asks the operator's name, the publisher's domain, the service id and version, the location, the commit, the build run, the tag and the build time. Every answer allows any origin to read it. - -## 4. What is not in this change - -- **A verifier.** None is in this repository. The guide says what one checks, and the harness is a small one. -- **The key custody**, as above. -- **The platform fields off Fastly.** The Axum, Cloudflare and Spin adapters report their own name, `unknown` for the service id, service version and location, and `false` for `staging`. Open question 1 asks whether `false` is right. -- **A way for a verifier to get the public keys.** No tool in this repository exports a public key, and no document here says where a verifier finds the keys and the period of each. Open question 3 asks where a verifier finds them. -- **The compiled schedule** is checked by the harness alone, because test builds carry no key. The reading of a schedule file is unit tested. - -**Open questions.** Each of these is a question about the code. This spec records what the code does and leaves the questions open. Row 50 ([IABTechLab/trusted-server#1400](https://github.com/IABTechLab/trusted-server/pull/1400), at `3aaf720c0`) has the same code for each question. - -1. Should the Axum, Cloudflare and Spin adapters say that `staging` is unknown, in place of signing `false`? The guide tells a relying party to check `platform.staging` when only answers from production are acceptable (`docs/guide/attestation.md`, lines 472 and 473), and that check passes on those three adapters whatever the environment. -2. Which tests should be added? #161 asks for tests that stale claims fail, and no test makes stale evidence fail, because the harness checks only that fresh evidence is within five minutes of the harness's clock (`scripts/attestation-local-test.sh`, lines 282 to 284). `fastly_platform_identity` has no test, and the harness checks `platform.name` only. The order test leaves out `tag` (`attestation.rs`, lines 1896 to 1914). -3. How does a verifier get the public keys and the period each is in force? No tool in this repository exports them. -4. What rule says when a field may be added to the evidence under `v: 1` and when the version has to change? `tag` was added to `build` after the first draft of this spec with the version left at `1`. -5. Should the guide list `build.run` before `build.tag`? The evidence writes `run` first (`attestation.rs`, lines 167 and 168), and the guide lists `build.tag` first (`docs/guide/attestation.md`, lines 308 and 309). - -## 5. Tests - -At `59e09b6cd` this change adds 54 tests and one harness, and the fixes at `694926002` add three more tests, making 57. - -- **`crates/trusted-server-core/src/attestation.rs` has 40 tests.** They cover the reading of a key schedule, the signature and the key in force, the evidence and the envelope, the nonce, the page, the host, the settings and the endpoint rules, and the routes and methods. The count was 37 before commit `96d8caf89`, which added the tests of a repeated key id or start, of the framework asset prefixes and of the single warning for the last key. The fixes add `a_clock_that_cannot_be_read_signs_nothing` and `a_schedule_key_that_cannot_sign_is_refused_without_showing_it`, making 42. -- **`crates/trusted-server-core/src/build_info.rs` has 3 tests.** The build time is RFC 3339 in UTC, the commit, the run and the tag are never empty, and the build script writes known instants correctly. -- **The Fastly adapter has 5 route tests.** A prefix of the deployment's own is answered there and never by the origin, the default address redirects to the endpoint the settings name, no route is registered without the section, every fixed route beneath an underscore prefix is reserved, and every endpoint the settings accept builds a router. -- **The Axum, Cloudflare and Spin adapters have one test each.** The test `every_fixed_route_beneath_an_underscore_prefix_is_reserved` fails when a fixed route beneath an underscore prefix is missing from `RESERVED_PATHS`. The fixes add `attestation_routes_skip_the_request_preparers` to the Cloudflare route tests, which registers a preparer that refuses every request and checks that the attestation routes answer without it. -- **`crates/trusted-server-integration-tests/tests/parity.rs` has 3 tests** that run the three adapters that can run on a host against each other. They are `attestation_prefix_parity`, `attestation_default_address_redirect_parity` and `attestation_is_not_served_without_its_section_parity`. -- **`scripts/attestation-local-test.sh` is the last step of the Fastly job in CI** (`.github/workflows/test.yml`, lines 80 to 82). It builds the Fastly adapter with a schedule of two keys, runs the adapter under Viceroy against a stub origin and checks the evidence with OpenSSL and the public key alone. It covers Fastly only. - -Test builds carry no key, so no router test sees signed evidence, and only the harness does, on Fastly. - -**What no test covers.** A stale claim failing, the Fastly environment variables read into `platform`, a request preparer on the attestation routes of Axum, Fastly and Spin, and the position of `tag` in the order test. Open question 2 asks about these. The test of the clock hands `answer` a missing time, because a test cannot make the platform's clock fail, so `handle_attestation`, which passes the real clock, has no test of its own. The test of a key that cannot sign calls `constant_source`, which the build script calls, and no test runs a build given such a key. - -## 6. Acceptance - -**Behaviour that can be checked.** Each line names the test or the harness check that fails when the behaviour goes. - -1. With no `[attestation]` section no attestation route is registered, and a read of `/_ts/attestation.json` reaches the publisher's origin (`attestation_is_not_served_without_its_section_parity`, `without_the_section_no_attestation_route_is_registered`). -2. With the section and a key in force, a GET of the endpoint's `.json` address answers `200` with an envelope whose signature verifies, with the public key alone, over the context, a line feed and the payload (the harness, `the_signature_covers_the_context_a_line_feed_and_the_payload`). -3. That signature does not verify with the other key of the schedule, under another context or once the host in the payload changes (the harness, `an_altered_payload_fails_verification`). -4. The evidence names the host the request arrived for, lower case and without a port, and a forwarded host header never replaces it. The publisher comes from the settings (`the_evidence_shown_is_the_payload_signed` and `the_host_falls_back_to_the_request_address` for the case and the port, `forwarded_host_headers_never_become_the_signed_host`, `the_signed_publisher_comes_from_the_settings`, the harness). -5. A nonce is signed back as sent, a malformed nonce answers `400`, and with no nonce the evidence carries none (`a_nonce_is_signed_back_as_sent`, `a_malformed_nonce_is_refused`, `no_nonce_is_signed_unless_asked_for`). -6. A build given no schedule, an empty schedule or blank lines answers `503` at the endpoint, as do a time before the first key, a key that cannot be read and a clock that cannot be read, and nothing is signed then (`a_build_given_no_schedule_compiles_an_empty_one`, `no_usable_key_in_force_answers_service_unavailable`, `a_clock_that_cannot_be_read_signs_nothing`). -7. A schedule line of the wrong shape, one that repeats a key id or a start, or one whose key cannot sign, however late it starts, stops the build without showing the key (`a_schedule_line_of_any_other_shape_is_refused_without_showing_the_key`, `a_schedule_repeating_a_key_id_or_a_start_is_refused`, `a_schedule_key_that_cannot_sign_is_refused_without_showing_it`). -8. The key in force is the latest to have started, and the warning for the last key is given once for each process (`the_key_in_force_is_the_latest_to_have_started`, `the_last_key_warning_is_given_once`). -9. Every answer carries `Cache-Control: no-store`, and the page and the envelope carry `Access-Control-Allow-Origin: *` (`answers_are_never_cached_and_readable_from_any_origin` for both headers on the page and the envelope, the harness for both on the envelope, and `attestation_prefix_parity`, `attestation_default_address_redirect_parity` and the Fastly adapter's tests for `Cache-Control` on the list at a prefix, the redirect, the `404` pages and the `503` answer of a build with no key). No test checks the headers of the `400` and `500` answers, of the `503` answer for a key that cannot be read, or of the `404` answer in JSON that the handler gives when the settings carry no section. -10. An endpoint of the wrong shape, at a reserved address or under a framework asset prefix is refused when the settings load, with the reason (`the_endpoint_is_an_absolute_path_under_an_underscore_prefix`, `an_endpoint_at_a_reserved_address_is_refused_naming_it`, `an_endpoint_under_a_framework_asset_prefix_is_refused_naming_it`), and every fixed route beneath an underscore prefix is in the reserved list on all four adapters. -11. Under a prefix of the deployment's own, every address and method the pages do not serve answers `404` from the deployment on all four adapters, and a read of the default address redirects to the endpoint with the query kept (`attestation_prefix_parity`, `attestation_default_address_redirect_parity` and the Fastly adapter's tests). -12. Cloudflare answers the attestation routes without the request preparers, so a preparer that refuses every request leaves the list at the prefix, the JSON address and the redirect from the default address as they are (`attestation_routes_skip_the_request_preparers`). - -**Checks that can be repeated.** In a clone of IABTechLab/trusted-server at `59e09b6cd`, with the Rust toolchain and the Viceroy version that `.tool-versions` names, these commands pass. - -1. `cargo test-fastly` runs the 40 tests of `attestation.rs`, the 3 of `build_info.rs` and the 5 attestation tests of the Fastly adapter. -2. `cargo test-axum`, `cargo test-cloudflare` and `cargo test-spin` run the test of reserved addresses in each adapter. -3. `cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` runs the three parity tests. -4. `./scripts/attestation-local-test.sh` runs the harness, which needs Viceroy, OpenSSL, `curl` and Python. - -**CI.** At `59e09b6cd`, all 25 checks on IABTechLab/trusted-server#1353 passed, and so did all 19 on the closed jwrosewell/trusted-server#7. - -**Local checks.** The test plan in the description of IABTechLab/trusted-server#1353 ticks `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter, and leaves a manual run with `fastly compute serve` unticked. - -**What no test covers.** The paths listed in section 5, which Open question 2 asks about, and the answers whose headers item 9 says no test checks. - -## Revision record - -| Date | Change | -| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | -| 2026-10-15 | Fixes from review round 1. The header and the pull requests it names, the `build` fields and the `tag`, the configuration fields and the endpoint rules, the methods, the adapters and what each reports for `staging`, the answers and the failures, the compatibility of the section, the privacy of the answers, the scope table and the quotations from the two issues, the key custody and what it costs, the test count and the tests, the acceptance, the open questions and the review log. | -| 2026-10-16 | Fixes from review round 2. The `host` fallback when the `Host` header is not readable text, the scheme headers and what they change under "Consent and privacy", the tests named in Acceptance 4 and 9 and what no test checks there, and the review log. | -| 2026-10-16 | Three fixes in the code, at `694926002`. The handler answers `503` and signs nothing when the clock cannot be read, the build reads every key in the schedule with the function that signs and stops at one that cannot sign, and Cloudflare answers the attestation routes without the request preparers, each with a test. Open questions 2, 3 and 4 of the previous revision leave the list, which is renumbered. | -| 2026-10-17 | The guide says how the key schedule reaches a Cloudflare build and a Spin build, and that the file is named by its absolute path (`e5e7fbf88`). | - -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/12-attestation` at `59e09b6cd`. Issues found, four of them blocking. The four were that the `build` fields left out `build.tag`, that the spec said every adapter reports `staging` or `unknown` when Axum, Cloudflare and Spin sign `staging: false`, that the spec counted 37 tests where the module has 40, and that the header and the acceptance named pull requests that have closed and a count of 19 checks where the open pull request shows 25. This revision states what the code does at that head for the evidence, the build inputs, the configuration, the adapters, the answers, the failures, the compatibility of the section, the privacy of the answers, the tests and the acceptance, and leaves eight questions about the code under Open questions until the code changes, being what `staging` says off Fastly, the request preparers on Cloudflare, the build's check of a key, the clock, the missing tests, how a verifier gets the public keys, the rule for new evidence fields and the order of `run` and `tag` in the guide. -2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/12-attestation` at `59e09b6cd`. Issues found, none blocking. This revision says that the request address's host also stands in for the `Host` header when the header is not readable text, names the scheme headers under "Consent and privacy" and what they change, and names the tests behind Acceptance 4 and 9 and what no test checks. diff --git a/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md b/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md index 84b3331d5c..2ea4bdf735 100644 --- a/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md +++ b/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md @@ -1,11 +1,11 @@ # Design Spec: Headers that describe the deployment stay off a publisher's pages -**Status:** Proposed, 2026-10-12. Raised as draft pull request [IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356), row 15 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#10](https://github.com/jwrosewell/trusted-server/pull/10) and [IABTechLab/trusted-server#1302](https://github.com/IABTechLab/trusted-server/pull/1302), are closed. +**Status:** Proposed, 2026-10-12. Raised as pull request [IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356), row 15 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#10](https://github.com/jwrosewell/trusted-server/pull/10) and [IABTechLab/trusted-server#1302](https://github.com/IABTechLab/trusted-server/pull/1302), are closed. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md`, `2026-10-12-inspection-endpoints-design.md`, `2026-10-12-attestation-design.md` and `2026-10-13-middleware-settings-design.md` (row 38, built after this row), in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342), which holds every design document of the stack, and `2026-09-25-git-version-header-design.md`, which main already holds -**Related PRs:** [IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356), [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355) (row 14, robots.txt from modules, which this row stacks on), [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10, page changes as middleware), [IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350) (row 9, inspection endpoints), [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353) (row 12, attestation), [IABTechLab/trusted-server#1379](https://github.com/IABTechLab/trusted-server/pull/1379) (row 38, middleware settings), [IABTechLab/trusted-server#1213](https://github.com/IABTechLab/trusted-server/pull/1213) (the git version header, merged to main), [jwrosewell/trusted-server#10](https://github.com/jwrosewell/trusted-server/pull/10) (closed), [IABTechLab/trusted-server#1302](https://github.com/IABTechLab/trusted-server/pull/1302) (closed) +**Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md`, `2026-10-12-inspection-endpoints-design.md` and `2026-10-13-middleware-settings-design.md` (row 38, built after this row), in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342), which holds every design document of the stack, and `2026-09-25-git-version-header-design.md`, which main already holds +**Related PRs:** [IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356), [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355) (row 14, robots.txt from modules, which this row stacks on), [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10, page changes as middleware), [IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350) (row 9, inspection endpoints), [IABTechLab/trusted-server#1379](https://github.com/IABTechLab/trusted-server/pull/1379) (row 38, middleware settings), [IABTechLab/trusted-server#1213](https://github.com/IABTechLab/trusted-server/pull/1213) (the git version header, merged to main), [jwrosewell/trusted-server#10](https://github.com/jwrosewell/trusted-server/pull/10) (closed), [IABTechLab/trusted-server#1302](https://github.com/IABTechLab/trusted-server/pull/1302) (closed) **Issues:** [IABTechLab/trusted-server#1273](https://github.com/IABTechLab/trusted-server/issues/1273) -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-26 > **Why this spec exists.** The server passed a reader of any page three headers from the publisher's origin and wrote version headers of its own, so anyone could learn which node answered, whether a cache did and which release ran, without anyone having decided to tell them. This spec says which headers now stay off a response, where the removal runs, what a deployment writes to put the versions back on its pages, and what the change leaves as it was. @@ -25,11 +25,11 @@ Nothing on a page needs any of the five headers. The five describe how the site ## 2. Design -**The rule.** A response leaves this server without `X-Served-By`, `X-Cache` or `X-Cache-Hits` (called the deployment headers here and in the code). The one exemption is a response to a request for a path in the namespace `/_ts`, meaning `/_ts` itself and every path that starts with `/_ts/`, from which the finalize step of each adapter does not remove the three. The finalize step is where an adapter writes headers such as `X-Geo-Info-Available` and applies the operator's `[response_headers]` to a response. The origin's own answer loses the three on that path too, so the exemption keeps a header only when something other than the origin put the header on the response before the finalize step. No adapter writes `X-TS-Version` or `X-TS-Fastly-Version` on any response. What describes a deployment is answered where a deployment is asked about itself, and is left off everything else. +**The rule.** A response leaves this server without `X-Served-By`, `X-Cache` or `X-Cache-Hits` (called the deployment headers here and in the code). The one exemption is a response to a request for a path in the namespace `/_ts`, meaning `/_ts` itself and every path that starts with `/_ts/`, from which the finalize step of each adapter does not remove the three. The finalize step is where an adapter writes headers such as `X-Geo-Info-Available` and applies the operator's `[response_headers]` to a response. The origin's own answer loses the three on that path too, so the exemption keeps a header only when something other than the origin put the header on the response before the finalize step. No adapter writes `X-TS-Version` or `X-TS-Fastly-Version` on any response. The git version and the service's version appear only on the HTML pages a deployment opts into, by naming `service_version` in a `[[serve]]` entry, and no page beneath `/_ts` shows either of them. -**Where a deployment is asked about itself.** The inspection pages (row 9) answer beneath `/_ts`, and so does attestation (row 12) unless the `endpoint` of its `[attestation]` section moves it. What each shows is the publisher's decision. The attestation document carries the build under `build`, and the platform's version number of the service and its staging flag under `platform.serviceVersion` and `platform.staging`. A deployment with no `[attestation]` section has no attestation document. +**Where a deployment is asked about itself.** The inspection pages (row 9) answer beneath `/_ts`, and the one version they give is that of the core crate that answered, under `version`. What each shows is the publisher's decision. -**The commits.** The branch `split/15-diagnostic-headers` holds eight commits and six merges of the row below. Taken together against the row below, the eight commits change 28 files, adding 1,279 lines and removing 443. +**The commits.** The branch `split/15-diagnostic-headers` holds eight commits and six merges of the row below. Taken together against the row below, the eight commits change 28 files, adding 1,279 lines and removing 443. These figures and the first eight rows of the table are those of the series head `a43a05a33`. The last row is the commit "Keep the platform identity in the platform module", made on this row after that head and before row 14 is merged into it, so that the merge of row 14 only deletes `attestation.rs`. | Commit | What the commit does | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -41,6 +41,7 @@ Nothing on a page needs any of the five headers. The five describe how the site | `5cc8fe77a` | A serve middleware may write a response header, and every middleware is told the platform | | `09910474d` | Core supplies the `service_version` middleware, which a `[[serve]]` entry names to write the versions | | `a43a05a33` | No adapter writes a version on a response, core's `version_header` module is removed, and `service_version` writes the git version as well as the platform's | +| New commit | `PlatformIdentity`, `PlatformIdentity::named` and `UNKNOWN` move into core's platform module, and each adapter's report of its platform leaves the attestation code, before row 14 is merged in | The change was first called diagnostic headers, and the name of this file and of the branch still say so. The title, the code, the pull request and the configuration guide say deployment headers, and this spec does too. @@ -69,15 +70,12 @@ Spin reads the path before its `NormalizeMiddleware` runs. Normalizing keeps the - **A path is in the namespace only at a boundary.** `/_ts` and `/_ts/permissions` are. `/_tsx` and `/_ts-archive` are a publisher's pages, and lose the headers like any other. A prefix match would let a page keep the headers by its name alone. - **The removal comes before the finalize step.** A header that finalizing writes is kept, an operator's among them. An operator can set a fixed value for `X-Cache` on every page in `[response_headers]`, and cannot bring back the origin's own value, because the origin's header is gone by then. -- **The namespace is a parameter**, so a test proves the boundary with a namespace of its own. Each adapter passes `ROUTE_NAMESPACE`. The inspection paths (`/_ts/permissions`, `/_ts/config` and `/_ts/data`) and the attestation default (`/_ts/attestation`) are separate literals that nothing ties to the constant. +- **The namespace is a parameter**, so a test proves the boundary with a namespace of its own. Each adapter passes `ROUTE_NAMESPACE`. The inspection paths (`/_ts/permissions`, `/_ts/config` and `/_ts/data`) are separate literals that nothing ties to the constant. - **The exemption keeps nothing today.** No code in the repository sets any of the three on a response, and the publisher fallback removes them from the origin's answer, so a response beneath `/_ts` reaches the finalize step without them. Open questions asks whether the exemption should stay. **No adapter writes a version.** Before this change all four adapters wrote `X-TS-Version`, the git version the build compiled in, and the Fastly adapter also wrote `X-TS-Fastly-Version`, the Fastly service's version number. The `GET /health` probe of Axum, Fastly and Spin carried `X-TS-Version` as well. None of the four adapters writes either header now, on any response. Core's `version_header` module and the constant `HEADER_X_TS_FASTLY_VERSION` are removed. The build still compiles in the git version from `TRUSTED_SERVER__GIT_VERSION` or local git, as `2026-09-25-git-version-header-design.md` describes, and that spec's goals of sending the header from every adapter and of naming the Fastly version `X-TS-Fastly-Version` no longer hold. -The versions stay readable in two places. - -1. The attestation document, for a deployment whose settings carry an `[attestation]` section, under `build` and `platform.serviceVersion`. -2. The HTML pages that a `[[serve]]` entry covers, for a deployment that names core's `service_version` middleware in the entry. +The versions stay readable on the HTML pages that a `[[serve]]` entry covers, for a deployment that names core's `service_version` middleware in the entry. ```toml [[serve]] @@ -112,7 +110,7 @@ pub struct MiddlewareContext<'a> { - Core writes the headers a serve chain returns on the reader's response, each replacing a header of the same name, at each place a serve pass runs, being the assembled copy of a stored page, the streamed copy and the inline copy of a page that is not stored. When several middleware write one name, the one named last wins. A stored page's serve middleware write their headers on each reader's response, on the fill and on the hit. - A fetch middleware works on the origin's copy of a page, which may be stored for every reader, and a header goes on one reader's response. A fetch middleware that writes a header is therefore refused when the chain is planned for a page, which is when a request for that page is handled and not at startup. -- The platform is what the adapter reports for the attestation evidence, being a name, a service id, a service version, a staging flag and the location that answered (`PlatformIdentity`). Each adapter tells the registry once at startup with `IntegrationRegistry::with_platform`, and its attestation route reads the same value from `IntegrationRegistry::platform`, so the two cannot differ. Fastly reports from its environment variables, and the other three report their name only. +- The platform is what the adapter reports about the platform it runs on, being a name, a service id, a service version, a staging flag and the location that answered (`PlatformIdentity`). This change adds `PlatformIdentity`, with `PlatformIdentity::named` and the `UNKNOWN` constant, to core's platform module (`crates/trusted-server-core/src/platform/identity.rs`). Each adapter tells the registry with `IntegrationRegistry::with_platform` when it builds its state, and core hands the same value to every middleware from `IntegrationRegistry::platform`. Fastly reports through `fastly_platform_identity`, which reads `FASTLY_SERVICE_ID`, `FASTLY_SERVICE_VERSION`, `FASTLY_IS_STAGING` and `FASTLY_POP`, and the other three report their name only. **Consent and privacy.** The removal reads the request's path and nothing else from the request, and it only takes headers off. No consent signal, cookie or identifier takes part in the removal, so consent is not affected. The `service_version` middleware reads nothing about the reader and writes two version strings. A deployment that names the middleware shows the git version to every reader of those pages, and the getting started guide advises a neutral value such as the commit hash when a branch name is sensitive. @@ -128,18 +126,17 @@ pub struct MiddlewareContext<'a> { **What breaks.** - An operator who reads `X-Cache` or `X-Served-By` from a page fetched through this server, to see whether the origin's cache answered, no longer sees them. The origin still sends the two headers, which can be read at the origin. -- A check that read `X-TS-Version` or `X-TS-Fastly-Version` from a response no longer finds either, whichever adapter served the response. The probe at `GET /health` carries neither, because `service_version` runs on HTML pages only. The Fastly service's version now has the name `X-TS-Service-Version`. Such a check can read the attestation document instead, where `build` says what the build was made from and `platform.serviceVersion` is the platform's number, or the deployment can name `service_version` in a `[[serve]]` entry. +- A check that read `X-TS-Version` or `X-TS-Fastly-Version` from a response no longer finds either, whichever adapter served the response. The probe at `GET /health` carries neither, because `service_version` runs on HTML pages only. The Fastly service's version now has the name `X-TS-Service-Version`. Such a check can read the versions from an HTML page once the deployment names `service_version` in a `[[serve]]` entry that covers the page. **Rollout.** For a deployment that is already running: 1. The removal and the end of the version headers start on upgrade, on all four adapters, and no setting delays them. 2. A build from the row below refuses at startup a configuration document that names `service_version`, with a message that no module that runs supplies the name. Push such a document only after every instance runs this change. 3. A deployment that wants the versions on its pages names `service_version` in the first `[[serve]]` entry that covers them. -4. A deployment with no `[attestation]` section has no attestation document, so for that deployment `service_version` is the only place the versions appear. **Not in this change.** -- `X-TS-ENV`, which the Fastly adapter writes as `staging` on every response it finalizes when `FASTLY_IS_STAGING` is `1`, that is on a staging deployment. The finalize step writes the header after the removal, and the attestation document reports the same flag as `platform.staging`. +- `X-TS-ENV`, which the Fastly adapter writes as `staging` on every response it finalizes when `FASTLY_IS_STAGING` is `1`, that is on a staging deployment. The finalize step writes the header after the removal. - `X-Geo-Info-Available`, which all four adapters write after the removal. - The six `x-ts-sandbox-*` counters (instance, ordinal, builds, request id, vCPU time and heap size). The Fastly entry point adds them to a private, no-store response after the finalize step when `[debug] sandbox_metrics_enabled` is true, and the setting defaults to false. - `Age`, `CF-Cache-Status` and `X-Cache-Status`, which say the same about a cache as the three removed names. The `ts origin probe-shareability` command reads all three, with `X-Cache`, as signs that a cache answered. @@ -151,11 +148,11 @@ pub struct MiddlewareContext<'a> { 1. Should the exemption for `/_ts` stay while no code beneath it writes the three headers, and if it stays, which code is meant to write them there? The origin's answer beneath `/_ts` already loses them, so the exemption keeps nothing today. 2. Why these three names? `Age`, `CF-Cache-Status` and `X-Cache-Status` say the same about a cache and are not removed. The list is `DEPLOYMENT_HEADERS` in `http_util.rs`, a name is added there, and the code gives no reason for the three. -3. Should `X-TS-ENV` and the six `x-ts-sandbox-*` counters stay on pages? Fastly writes `X-TS-ENV: staging` on every response it finalizes on a staging deployment, and the attestation document already reports the same flag, which was the reason given for removing the Fastly adapter's version header. The counters appear only when sandbox metrics are on, and the entry point adds them after the finalize step, so the removal never sees them. +3. Should `X-TS-ENV` and the six `x-ts-sandbox-*` counters stay on pages? Fastly writes `X-TS-ENV: staging` on every response it finalizes on a staging deployment, so the header describes the deployment on every page, as the version headers did. The counters appear only when sandbox metrics are on, and the entry point adds them after the finalize step, so the removal never sees them. 4. Should an origin's own `X-TS-Version`, `X-TS-Fastly-Version` and `X-TS-Service-Version` be removed? The `version_header` module replaced or removed an origin's `X-TS-Version`, and the Fastly adapter did the same for `X-TS-Fastly-Version`, so that an origin's value was never reported as this server's. No code removes any of the three now, and `service_version` replaces an origin's `X-TS-Version` or `X-TS-Service-Version` only when the middleware has that version to write. 5. Should each adapter's test also cover a path beneath `/_ts` and check the namespace the adapter passes? The four tests use a page path, so an adapter that always removes the headers, or that passes another namespace, passes all four. 6. Should a test send a header name more than once? A chain of caches can send the same name several times and every test inserts one value. The code relies on `HeaderMap::remove` taking every value of a name, which is what the `http` crate version in the lockfile (1.4.2) does. -7. Should a test tie `ROUTE_NAMESPACE` to the inspection paths and the attestation default, which are separate literals? +7. Should a test tie `ROUTE_NAMESPACE` to the inspection paths, which are separate literals? 8. Should one test run a real router with the finalize middleware and the publisher fallback together, with a proxied third-party response among the responses? Today the adapter tests call the finalize middleware with a canned response, and the one fallback test calls the fallback directly. 9. Should `2026-09-25-git-version-header-design.md`, which still reads `Implemented`, and its plan carry a note that this change replaces what they describe? 10. Should the changelog's Security entry for the three headers carry **Breaking:**, as the entry for the version headers under Changed does, given that the pull request description lists the removal under what breaks? @@ -184,6 +181,8 @@ The fork's CI passed on the series head, `a43a05a33`, all nineteen checks, and t | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The five commits since 9 October, being the fallback removal, the response headers a serve middleware may write, the platform in the middleware context, the `service_version` middleware and the end of the version headers on every adapter, the function names, the paths beneath `/_ts` and what the exemption keeps, the adapters and the responses that skip the finalize step, the defaults and the rollout, what breaks, the failure and privacy answers, the tests, the open questions, the header and the acceptance lines brought to the series head, and the review log. | | 2026-10-16 | Fixes from review round 2. The acceptance says what the description of the pull request for this row records, being the fork's run of all nineteen checks on the head, the ticked check commands and the manual test left unticked, in place of saying that no description records a run of the whole check. Three clauses that were only partly true are qualified, being the headers a stored page replays from the origin, the headers each proxy takes off and the one header the prefix test sends. A new open question asks whether the headers that core or an adapter writes directly should move into a serve middleware. | +| 2026-10-26 | Row 12 no longer adds the signed document that reported the build and the service's version, so the versions are readable only where `service_version` writes them, and this change adds `PlatformIdentity` to core's platform module. The header, the pages beneath `/_ts`, the namespace note, the versions, the platform a middleware is told, what breaks, the rollout, `X-TS-ENV` and open questions 3 and 7 say so. | +| 2026-10-26 | The rule says where the git version and the service's version appear, the inspection pages give the core crate's version, the commits table names the series head `a43a05a33` and gains the commit that keeps `PlatformIdentity` in the platform module, the platform bullet names `fastly_platform_identity` and when each adapter reports its platform, and the header no longer calls the pull request a draft. | ## Review log diff --git a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md index 622dbd198f..682d57b38f 100644 --- a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md +++ b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md @@ -1,13 +1,13 @@ # Design Spec: The last Fastly names leave core -**Status:** Proposed, 2026-10-15. Raised as draft pull request IABTechLab/trusted-server#1399 (from jwrosewell/trusted-server#44), for Tech Lab review. +**Status:** Proposed, 2026-10-15. Raised as pull request IABTechLab/trusted-server#1399 (from jwrosewell/trusted-server#44), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-14-last-names-in-core-design.md`, `2026-10-14-backend-naming-design.md` **Related PRs:** IABTechLab/trusted-server#1399, jwrosewell/trusted-server#44 **Issues:** IABTechLab/trusted-server#777 (its Fastly row) -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-26 -> **Why this spec exists.** After the series before this one, core's shipped code still names one hosting platform, Fastly, in small pieces. They are the format of a long configuration document, a staging variable, three request headers, a device module selector and the names of two functions. Each is a place where a change by one platform is a change in core. Core's comments, tests and README files name Fastly, Cloudflare, Spin, Fermyon, Viceroy and Axum as well, the attestation settings name Vercel, and three tests use a host under Fastly's `edgecompute.app` domain. After this change no line of core names a hosting platform, and a guard outside core keeps it so (section 2.6). One Fastly header, `x-compress-hint`, whose name does not name the platform, is still set by core after this change (section 3). +> **Why this spec exists.** After the series before this one, core's shipped code still names one hosting platform, Fastly, in small pieces. They are the format of a long configuration document, a staging variable, three request headers, a device module selector and the names of two functions. Each is a place where a change by one platform is a change in core. Core's comments, tests and README files name Fastly, Cloudflare, Spin, Fermyon, Viceroy and Axum as well, and three tests use a host under Fastly's `edgecompute.app` domain. After this change no line of core names a hosting platform, and a guard outside core keeps it so (section 2.6). One Fastly header, `x-compress-hint`, whose name does not name the platform, is still set by core after this change (section 3). ## 1. Problem @@ -23,7 +23,7 @@ These names remain in core, each with the files that hold it: Each name is small. Each one means that another platform, or a change in how Fastly stores, flags or forwards something, is an edit to core rather than to the adapter beside it. -Core's comments, doc comments, tests and README files named platforms too, and so did one shipped line, the `_vercel` entry among the prefixes an attestation endpoint may not take. Each told a reader that core knows which platform runs it or serves its origin. Section 2.6 counts them. +Core's comments, doc comments, tests and README files named platforms too. Each told a reader that core knows which platform runs it or serves its origin. Section 2.6 counts them. ## 2. Design @@ -121,10 +121,10 @@ Core is built on EdgeZero and names no hosting platform in any line, whether cod - The test that the largest `KvMetadata` serializes within 2048 bytes moves to the tests of the Fastly adapter's store (`ec_kv.rs`), because the limit is the Fastly KV Store's. - The test that core's code never uses the Fastly SDK moves from core's `migration_guards.rs` to the Fastly adapter's `core_sdk_guard.rs`. A build script in the adapter embeds every Rust source under core's `src` by path, so a file added to core is checked with no edit to the test. Core's build script no longer lists its own sources. - Core's README and the auction README show the command that runs core's tests on the host and point at `TESTING.md` for the adapter aliases, where they listed each alias by its platform. -- `_vercel` leaves `ORIGIN_ASSET_PREFIXES`, the prefixes a web framework serves its assets from and an attestation endpoint may not take, because it is a hosting platform's prefix and not a framework's. The attestation guide names it instead, as a prefix the origin serves that the operator has to avoid, which the guide already says of every prefix the list does not hold. Three HTML processor tests use `test-publisher-ts.edge.example` as the request host, where they used a host under `edgecompute.app`. +- Three HTML processor tests use `test-publisher-ts.edge.example` as the request host, where they used a host under `edgecompute.app`. - `scripts/lint-core-platform-names.py` fails on any line or path under `crates/trusted-server-core` that names a hosting platform, a tool named for one, or a domain a platform serves deployments from. It splits text into words at every character that is not a letter and at each change from lower to upper case, so `FASTLY_IS_STAGING` and `edgezero_adapter_cloudflare` each name a platform. Six names, among them `spin`, `axum` and `aws`, count only as words of their own, because ordinary words such as `spinner` and `withdraws` contain them. Eighteen, among them `fastly`, `cloudflare`, `akamai`, `vercel` and `netlify`, also count at the start of a word, so `akamaized` counts. Sixteen domains, among them `edgecompute.app`, `workers.dev` and `fermyon.app`, count where they are not part of a longer host name. It runs its matcher on every name and domain and on known non-names before it reads core, and stops with exit 2 when the matcher is wrong. The Run Format workflow runs it as a job of its own, and `AGENTS.md` lists it among the CI gates. -**The count.** Lines of `crates/trusted-server-core` that name each platform on this branch before and after this change, counted as a whole word with `git grep -i -w`, and with the guard, which also counts a name inside an identifier such as `FASTLY_IS_STAGING` or `_vercel`. +**The count.** Lines of `crates/trusted-server-core` that name each platform on this branch before and after this change, counted as a whole word with `git grep -i -w`, and with the guard, which also counts a name inside an identifier such as `FASTLY_IS_STAGING`. | Name | `git grep -i -w` before (files) | Guard before | After, either count | | -------------------------- | ------------------------------- | ------------ | ------------------- | @@ -135,14 +135,13 @@ Core is built on EdgeZero and names no hosting platform in any line, whether cod | `edgecompute.app` | 5 (1) | 5 | 0 | | Viceroy | 4 (3) | 4 | 0 | | Fermyon | 1 (1) | 1 | 0 | -| Vercel | 0 | 1 | 0 | | Every other name or domain | 0 | 0 | 0 | -The guard found 173 lines before this change, some of them naming two platforms, and none after. One of them, the `_vercel` entry, was shipped code, and the rest were comments, documents or test code. +The guard found 172 lines before this change, some of them naming two platforms, and none after. All 172 were comments, documents or test code. -**What changes for an operator.** An attestation endpoint beneath `/_vercel` is accepted where it was refused, so an operator whose origin is hosted on Vercel has to avoid that prefix, as the attestation guide says. Nothing else that core does when it runs changes. +**What changes for an operator.** Nothing that core does when it runs changes. -**For crates outside core.** `ORIGIN_ASSET_PREFIXES` no longer holds `_vercel`, and nothing else in core's API changes. A change that adds a hosting platform's name to core, in a comment or a test as much as in code, fails the guard. The Fastly adapter gains a build script and a `regex` dev-dependency for the test of core's sources. +**For crates outside core.** Nothing in core's API changes. A change that adds a hosting platform's name to core, in a comment or a test as much as in code, fails the guard. The Fastly adapter gains a build script and a `regex` dev-dependency for the test of core's sources. ## 3. What is not in this change @@ -158,7 +157,7 @@ The guard found 173 lines before this change, some of them naming two platforms, - A declared TLS header sets the scheme and an undeclared one does not, the declared TLS and spoofable headers are stripped with the standard ones, the auction sees the client's declared TLS header where the origin request does not, a declared internal header is refused in the `auth_header` slot of `[trusted_client_ip]`, and a bare `client-ip` is refused where `example-client-ip` is accepted. - The `platform` device selector builds the module the adapter's factory supplies, and no selector builds the built-in one. - The asset proxy's tests pass unchanged through the renamed conversions. -- The guard of 2.6 passes on core after this change and fails on core before it. Each of its 40 names and domains, planted once on a line of its own in a file git does not yet track, fails it on that line, and so do `_vercel` and the `edgecompute.app` test host put back where they were. It passes `spinner`, `SpinyAI`, `steadfastly`, `denominator`, `withdraws`, `EdgeZero`, `notworkers.dev` and `butterfly.io`. With its word pattern or its domain pattern broken to match nothing, it stops with exit 2 before it reads core. +- The guard of 2.6 passes on core after this change and fails on core before it. Each of its 40 names and domains, planted once on a line of its own in a file git does not yet track, fails it on that line, and so does the `edgecompute.app` test host put back where it was. It passes `spinner`, `SpinyAI`, `steadfastly`, `denominator`, `withdraws`, `EdgeZero`, `notworkers.dev` and `butterfly.io`. With its word pattern or its domain pattern broken to match nothing, it stops with exit 2 before it reads core. - The Fastly adapter checks that core's code never uses the Fastly SDK, that the list of core's sources it embeds covers core, and that the largest `KvMetadata` fits the Fastly KV Store's 2048-byte metadata limit. Not tested: @@ -179,6 +178,8 @@ Not tested: | 2026-10-15 | Fixes from review round 1. The readers of each declared list and the open questions on them, `x-debug-fastly-pop`, the scheme change, the effects on the other adapters, where each refusal happens, the rollout, the narrowed constants, `x-compress-hint`, the file count, the untested paths and the review log. | | 2026-10-15 | Corrections from review round 2. The Fastly sentence of 2.3 and the changelog on the `x-debug-fastly-pop` refusal, the changelog on the spoofable strip and the scheme change, when the Axum, Cloudflare and Spin adapters strip `fastly-client-ip`, and the comment in `request_headers.rs` left for a later change. | | 2026-10-16 | Core names no hosting platform or platform domain in any line. Section 2.6 with the counts, the `_vercel` prefix and the test host, the guard and the tests that moved to the Fastly adapter, and section 3 no longer leaves core's comments and tests for later. | +| 2026-10-26 | Row 12 no longer adds `ORIGIN_ASSET_PREFIXES`, so core holds no `_vercel` entry before this change. Section 2.6 counts 172 lines before this change, the count of 2026-10-16 less that entry, all of them comments, documents or test code, and no other section names the prefix. | +| 2026-10-26 | The header no longer calls the pull request a draft. | ## Review log diff --git a/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md index f762bd27e4..53eda0d482 100644 --- a/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md @@ -1,11 +1,11 @@ # Design Spec: Request filters and device detection run on every adapter -**Status:** Proposed, 2026-10-17. A pull request above `ts config migrate`, not yet raised, for Tech Lab review. +**Status:** Proposed, 2026-10-17. Raised as pull request [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410), row 52 of the stack, above `ts config migrate` ([IABTechLab/trusted-server#1409](https://github.com/IABTechLab/trusted-server/pull/1409)), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-14-51degrees-client-hints-design.md` and `2026-10-15-geo-module-request-design.md`, whose statements about the adapters this change amends, `2026-06-11-datadome-server-side-protection-design.md` and `2026-08-03-datadome-ip-excluded-client-tag-design.md`, which describe DataDome's protection filter on the Fastly adapter alone, a scope this change lifts (section 1), and the design documents of the stack in IABTechLab/trusted-server#1342 -**Related PRs:** none yet +**Related PRs:** [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410) **Issues:** none -**Last updated:** 2026-10-18 +**Last updated:** 2026-10-26 > **Why this spec exists.** Only the Fastly adapter runs the integration request filters and classifies each request with the device module `[device]` selects. On Axum, Cloudflare and Spin a module's request filter never runs, so the 51Degrees client hints middleware writes nothing and DataDome's protection never asks its service, and a device module is asked only for a bid request's device attributes, so no module call carries a device classification. This change runs both steps on those three adapters where the Fastly adapter runs them. @@ -38,7 +38,7 @@ Each adapter carries its own copy of the steps. `derive_device_signals`, `run_pr | Response effects | `attach_request_filter_effects` | Leaves the filters' response headers on the response for the finalize middleware | | Finalize | `FinalizeResponseMiddleware` | Finalizes the response, then `apply_terminal_response_effects` writes the filters' response headers and restores the privacy guards, as the Fastly entry point does | -**Which routes run the cycle.** The routes the Fastly adapter runs it for, being every named route but the inspection endpoints and batch sync, and the publisher fallback, being robots.txt, tsjs, the module routes and the publisher's origin. These three adapters register no batch sync route, so a request for its path reaches the publisher fallback, as before. The inspection endpoints are read only and stay ahead of the Edge Cookie context and the filters, as on Fastly. Every adapter also answers the attestation pages outside the cycle, for `GET` and `HEAD` and for every method under a prefix the endpoint owns, as on Fastly, and Axum and Spin answer `/health` outside it. +**Which routes run the cycle.** The routes the Fastly adapter runs it for, being every named route but the inspection endpoints and batch sync, and the publisher fallback, being robots.txt, tsjs, the module routes and the publisher's origin. These three adapters register no batch sync route, so a request for its path reaches the publisher fallback, as before. The inspection endpoints are read only and stay ahead of the Edge Cookie context and the filters, as on Fastly, and Axum and Spin answer `/health` outside the cycle. | Adapter | Named routes in the cycle | Inspection endpoints | Registration | | ---------- | ----------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------- | @@ -51,7 +51,7 @@ On Spin the discovery, signature verification and first-party routes and the pag **Device classification.** `derive_device_signals` follows the Fastly adapter's function of the same name. The evidence a module is shown holds the client's IP address and the `User-Agent`, with the `Cookie` header as well for a device module a registration supplies, in a module context that also carries the request's method, scheme, host, path and query, the settings and the services, as on Fastly. Four things differ from the Fastly adapter. 1. It runs where the adapter builds the Edge Cookie context, after the preparers, where the Fastly adapter runs it at the entry point before routing. A preparer that removes a reserved cookie therefore removes it before a device module sees it. -2. It runs only for the routes in the cycle, so a request for robots.txt, an inspection endpoint or an attestation page is not classified, where the Fastly entry point classifies every request. On Fastly a device module a registration supplies is therefore shown the cookies a crawler sends with a request for robots.txt, and on these adapters it is not. +2. It runs only for the routes in the cycle, so a request for robots.txt or an inspection endpoint is not classified, where the Fastly entry point classifies every request. On Fastly a device module a registration supplies is therefore shown the cookies a crawler sends with a request for robots.txt, and on these adapters it is not. 3. The registered module is the one the request's services carry (`RuntimeServices::device_module`), which `services_for_request` sets from the registry, and the module is handed those same services, so it reads the platform's key-value store when the services carry one. The Fastly entry point builds services of its own for the call. 4. These adapters supply no TLS or HTTP/2 signals, so `[device] module = "platform"` builds the built-in module, which classifies from the `User-Agent`, because `platform` adds the host's signals only where the platform supplies them. @@ -87,7 +87,7 @@ As on Fastly, a module route's call and a serve phase preparation carry the clas - `an_edge_cookie_context_that_cannot_be_read_answers_before_the_filters`: with a filter that answers every request, a page and the discovery route, each of which a module's preparer leaves with an unreadable `Cookie` header, are answered `400`, with no filter's effect and no request to the origin, and a request a client sends with an unreadable `Cookie` header of its own reaches the filter instead. - `robots_txt_is_answered_before_the_request_filters`: with a filter that answers every request, robots.txt is still the file, and the filter's header is absent. - `robots_txt_shows_the_device_module_none_of_the_crawler_s_cookies`: the module `[device]` selects, which records the cookies it is shown, never sees the cookies sent with robots.txt, and does see those sent with a module route's request. -- `the_inspection_endpoints_and_the_attestation_pages_skip_the_request_filters`: with a filter that answers every request, the five inspection paths and the two attestation pages are answered by their own handlers with no filter's effect, and a page is the filter's answer. +- `the_inspection_endpoints_skip_the_request_filters`: with a filter that answers every request, the five inspection paths are answered by their own handlers with no filter's effect, and a page is the filter's answer. - `a_filter_cannot_make_a_response_core_made_private_public`: a module route's answer core marks private keeps `no-store, private` when a filter's effect sets `Cache-Control: public, max-age=600`. - `a_filter_s_cookie_keeps_the_response_from_shared_caches`: a public answer that a filter's effect adds a `Set-Cookie` to is sent `private, max-age=0`. - `a_filter_cannot_add_a_shared_cache_header_to_a_private_response`: a `Surrogate-Control` that a filter's effect adds to a `private` answer is removed. @@ -122,6 +122,8 @@ The thirteen tests pass on each of the three adapters, taking out any of the thr | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-17 | First draft. | | 2026-10-18 | Fixes from review round 1. What a selected device module did on these adapters before, what DataDome's filter sends and sets, how each filter's call is bounded and why no time limit is added, the inspection endpoints the Fastly adapter answers before the preparers, the routes kept out of the cycle, which of a crawler's cookies robots.txt reads, the `Cookie` check and when it is reached, the configuration, eight new tests on each adapter with the three privacy guards among them, the open questions and the review log. Fixes from review round 2. A test on each adapter that sends every named route in the cycle through the filters, the discovery route in the context test, the tests each removal fails, the IP address and the request's address a device module is shown, the subresources it classifies, what the diagnostics preparer drops, the two DataDome designs whose scope this change lifts, what the guards do with a platform's own cache header with an open question on it, and the reason no shipped filter is tested through an adapter. | +| 2026-10-26 | Row 12 adds no route to the adapters any more, so section 2 drops row 12's pages from the routes kept out of the cycle, from the requests a device module does not classify and from the inspection test, which is renamed `the_inspection_endpoints_skip_the_request_filters`. | +| 2026-10-26 | The header names the pull request, IABTechLab/trusted-server#1410, row 52 of the stack, in place of saying that none is raised. | ## Review log diff --git a/docs/superpowers/specs/2026-10-26-module-endpoints-design.md b/docs/superpowers/specs/2026-10-26-module-endpoints-design.md new file mode 100644 index 0000000000..af5f65c27f --- /dev/null +++ b/docs/superpowers/specs/2026-10-26-module-endpoints-design.md @@ -0,0 +1,92 @@ +# Design Spec: A module outside core serves an endpoint of its own + +**Status:** Proposed, 2026-10-26. For the pull request [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353), "Prove a module outside core can serve its own endpoint", row 12 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), for Tech Lab review. One commit removes what the pull request's attestation commits changed, returning the row's files to those of row 11, and the tests and the guide section below come after that commit. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-08-27-integration-provider-seam-design.md` (row 7, the integration seam), `2026-10-12-diagnostic-headers-design.md` (row 15, which keeps `PlatformIdentity` in core), `2026-10-17-request-filters-on-every-adapter-design.md` (row 52) and `2026-10-17-edge-cookie-on-every-adapter-design.md` (row 53), in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) +**Related PRs:** [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353) (row 12), [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7, which opens the integration seam so that a vendor module can live outside core), [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354) (row 13, which closes the administration paths), [IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356) (row 15), [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410) (row 52, the request filters on every adapter) and [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411) (row 53, the Edge Cookie on every adapter) +**Issues:** none. [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack +**Last updated:** 2026-10-26 + +> **Why this spec exists.** A module outside core can already answer a path of its own on every adapter, so a deployment that wants an endpoint core does not have, attestation among them, serves that endpoint from a module and core needs no interface for it. No test shows a module answering a path outside `/integrations/` on all four adapters, and no guide says how a module does so, what runs before its route or which paths it can never take. This change adds those tests and that guide section, and no production code in core or in any adapter changes. + +## 1. What already exists + +Each statement holds at this row and at the top of the stack, unless it names a row. + +**A module declares any path.** `IntegrationProxy::routes()` in `crates/trusted-server-core/src/integrations/registry.rs` returns the module's endpoints, each an `IntegrationEndpoint` holding a method and a path. `IntegrationEndpoint::new(method, path)` and the functions `IntegrationEndpoint::get`, `post`, `put`, `delete` and `patch` keep the path as written. The trait's own methods of the same five names put `proxy_prefix()` in front of the path, and `proxy_prefix()` is `/integrations/` unless the module overrides it. A path ending in `/*` covers every path beneath it. The registry keeps one router for each of `GET`, `POST`, `PUT`, `DELETE`, `PATCH`, `HEAD` and `OPTIONS`, and skips a route for any other method with a warning in the log. A `GET` route answers neither `HEAD` nor `OPTIONS`, so a module that answers either declares a route of its own for that method, as Sourcepoint declares `HEAD` and `OPTIONS` beside its `GET` and `POST` (`crates/cmp/sourcepoint/src/lib.rs`). A request whose method has no module route on its path is handled as if no module route existed. + +**Shipped modules already declare paths outside `/integrations/`.** Prebid declares a `GET` route for each of its `script_patterns`, which are `/prebid.js`, `/prebid.min.js`, `/prebidjs.js` and `/prebidjs.min.js` unless its settings name others (`crates/auction/prebid/src/lib.rs`). Didomi overrides `proxy_prefix()`, so its routes sit beneath the `proxy_path` its settings give in place of `/integrations/didomi/consent` (`crates/cmp/didomi/src/lib.rs`). Core's JS asset proxy declares a `GET` route at the first-party path of each asset that its settings proxy (`crates/trusted-server-core/src/integrations/js_asset_proxy.rs`). + +**A module outside core hands its routes to the adapter in its builder.** A crate outside core gives its `IntegrationBuilder` to the adapter. On Fastly it does so through `run_with` (`crates/trusted-server-adapter-fastly/src/lib.rs`), and from row 35 also through `run_with_addressing`. On Axum, Cloudflare and Spin it does so through `TrustedServerApp::routes_with_registrations`, or through `build_state_with_registrations` for a host that builds its own router around the state, and also through Axum's `routes_with_registrations_and_services` from row 25 and each adapter's `routes_with_addressing` from row 35. The integration guide shows `run_with`, `routes_with_registrations` and `build_state_with_registrations` under "How an adapter composes it in" (`docs/guide/integration-guide.md`), and the configuration guide names the two functions of row 35. A builder's registration, and so its routes, is built only when a section of the module's type names the module in its `module` or `modules` key, unless the builder registers from the auction plan, which runs whether or not a section selects the module (`IntegrationBuilder::with_plan_registration`). Prebid registers that way, whenever the settings carry Prebid's table. The module reads its own table at `[.]` with `Settings::module_config`. The settings read every top-level table they do not know as a module type's section (`TypeSections` in `crates/trusted-server-core/src/settings.rs`), so a module of a type core does not know needs nothing from core to be configured. + +**Two routes cannot share one method and path.** `IntegrationRegistry::with_plan_and_registrations` inserts each route into matchit's router for the route's method. The insert fails for a second route at the same method and path, from the same module or another, for a route matchit cannot tell apart from one already in the router, such as `/foo/{names}` beside `/foo/{name}`, and for a path matchit cannot parse, such as `/src/{*filepath}/x`, whose catch-all is not at the end. Each failure refuses the registry with the message `Integration route registration failed for `, followed by matchit's reason. A stock build, being an adapter built from the repository as it ships with the builders its own list names, can clash from its settings alone. Prebid's default `script_patterns` include `/prebid.js`, and core's JS asset proxy accepts an asset at that path, so when the settings carry Prebid's table and give the JS asset proxy an asset at `/prebid.js` to proxy, both modules declare `GET /prebid.js`. What a refusal does depends on how the builders arrive. + +- The stock entry points compose the shipped modules' builders, being `run` on Fastly and `Hooks::routes()` on Axum, Cloudflare and Spin, and Fastly's `run_with` adds a deployment's own builders to them. When the registry is refused, `startup_error_router` answers every request that reaches the router with the error. The adapter builds its state, and so its registry, for every request in a production Fastly build, a Cloudflare Worker and a Spin component, and once when the Axum dev server starts. A Fastly build with the `reusable-sandbox` feature builds once for each app-config key in each sandbox and keeps no failed build, so the next request for that key builds again. +- On Axum, Cloudflare and Spin a deployment's own builders arrive through `routes_with_registrations` and the other functions above, which return the error to the deployment's code and build no router. The integration guide's example passes the error on with `?`. + +**Where a module route runs.** Every adapter registers its own routes for the methods each serves, then a catch-all for `/` and every other path on the seven methods, whose handler is the publisher fallback. The module routes are not in the adapter's router. The fallback asks the registry for a route at the request's method and path (`IntegrationRegistry::has_route`) and hands the request to the module that declared the route (`IntegrationRegistry::handle_proxy`). At the top of the stack the fallback runs these steps in this order on all four adapters. + +1. The closed path check of row 13, which answers `404` for every path that begins with `/_ts/admin`, and for `/admin/keys` and the paths beneath it. +2. The modules' request preparers. +3. robots.txt, for a `GET` or `HEAD` of `/robots.txt` when `[robots-txt]` is present. +4. The Edge Cookie steps. Axum, Cloudflare and Spin classify the device here, inside `build_ec_context`, and on Fastly the entry point has classified the device before routing. The steps then read the request's Edge Cookie context and resolve the location and the permissions for it. They answer in place of the route when the request's `Cookie` header cannot be read as text, with `400`, or when the selected Edge Cookie module cannot be built for the request. +5. The request filters. A filter that answers stands in for the route. +6. tsjs, for a `GET` beneath `/static/tsjs=`. +7. The module route. +8. The asset routes, then the publisher's origin. + +At this row a module route meets four differences from that order, and the list covers only what differs for a module route. + +1. `deny_admin_diagnostic_fallback` answers a request beneath `/_ts/admin` or `/admin/keys` that reaches the fallback, and row 13 replaces that function with the closed path check. +2. No adapter has robots.txt code at this row. Rows 14 to 24 answer the file with a route of each adapter's own when `[robots-txt]` is present. The fallback answers the file ahead of the other steps from row 25 on Fastly and from row 52 on Axum, Cloudflare and Spin. +3. Axum, Cloudflare and Spin classify no device and run no request filter until row 52, so a module route there runs after the preparers, with an Edge Cookie context read from the request, and that context's errors answer in place of the route as in step 4. Fastly classifies the device at its entry point and runs the preparers, the Edge Cookie steps and the request filters before a module route, as at the top of the stack. +4. Until row 13 removes `[[handlers]]`, each adapter's router asks for the password of a `[[handlers]]` entry whose pattern covers the request's path before any route runs, a module route included. + +**What a module route is handed.** `handle_proxy` removes a caller's `x-ts-ec` header, creates an Edge Cookie identifier for a page navigation that needs one, and calls the module's `handle` with a module call. At this row the call carries the request's method, host, scheme, path and query, the settings, the services, the connection and, from the Edge Cookie context, the permissions, the consent, the location, the device signals and the Edge Cookie identifier. From row 39 the call also carries the request's evidence (`5494af9ca`). A gated value, being the location, the device signals or the Edge Cookie identifier, is passed only to a route that declares and is granted every permission the value's producer declares (`IntegrationProxy::required_permissions`, none by default), and a value whose producer declares none is passed to every route. Nothing gates the route itself, so a route whose use needs a permission checks that the permission is granted. On Fastly the module's response then goes through the Edge Cookie's finalize step, which can write the cookie, and on Axum, Cloudflare and Spin the response goes through that step from row 53. + +**Requests a module route never sees.** Steps 2, 4 and 5 above keep from every module route a request that a preparer refuses, whose Edge Cookie step fails or that a request filter answers. Beyond those, a module route for a method and path never runs when something ahead of it answers the request. + +1. A route of the adapter's own for the same method and path, because the adapter's own route wins over the catch-all. That covers the named routes for the methods each serves, such as `GET /_ts/config` on every adapter, and `GET /health` on Axum and Spin. +2. A request the Fastly entry point answers before its router, being `GET /health`, `GET /_ts/debug/ja4` and, in a build with the `reusable-sandbox` feature, `GET /_ts/debug/sandbox`. +3. A closed path, which the fallback answers first, through `deny_admin_diagnostic_fallback` at this row and the closed path check from row 13. +4. A `GET` or `HEAD` of `/robots.txt` when `[robots-txt]` is present, from row 14, except on Axum, Cloudflare and Spin between rows 25 and 51, where the publisher path answers the file after the module routes. +5. A `GET` beneath `/static/tsjs=`, which the fallback serves as tsjs. +6. From row 35, the path a reader sent, where the deployment's request addressing rewrites the path before routing, because a module route is matched against the path the request continues with. + +Another method on a path of item 1 reaches the fallback as any other request does, so a module route for `POST /_ts/config` runs. The registry does not know the adapter's routes, so the registry accepts a module route on such a path, and nothing reports that the route never runs. + +## 2. What this change adds + +**A route outside `/integrations/` in the seam probe.** The seam probe is the test module that proves the integration seam from outside core, `crates/testing/seam-probe` at this row and `crates/testing/iabtl` from row 43. The probe gains the constant `SEAM_PROBE_OWN_PATH`, which is `/seam-probe/own-path`, and its proxy declares a route at that path for `GET` with `IntegrationEndpoint::get` and for `HEAD` with `IntegrationEndpoint::new(Method::HEAD, ...)`. The proxy's `handle` sends a request for that path to a handler of its own, `SeamProbeProxy::own_path`. The handler answers a `GET` with `200`, `Content-Type: text/plain` and the body `SEAM_PROBE_OWN_BODY`, a constant of the probe, and a `HEAD` with `200`, the same headers and an empty body, which the handler sends itself because nothing in core, the adapters or EdgeZero strips a body from a `HEAD` answer. The handler counts its runs against the token a request carries in `SEAM_PROBE_COUNT_HEADER`, as the preparer counts its own, and `own_path_runs_for` reads that count as `prepare_runs_for` reads the preparer's. `report_route_is_the_documented_path` changes, because it asserts that the probe declares one route, and so does the crate's own doc, which names the Axum and Fastly adapters as the only ones that depend on the probe. + +**Tests on all four adapters.** Each test runs through the adapter's own router, or through the build of the adapter's state where the test expects a refusal. The tests sit beside the probe's other tests on Fastly (`crates/trusted-server-adapter-fastly/src/app/seam_probe_tests.rs`) and Axum (`crates/trusted-server-adapter-axum/tests/seam_probe.rs`), and in the route tests of Cloudflare and Spin (`tests/routes.rs` in each adapter), which gain the probe as a development dependency. + +1. A `GET` of `/seam-probe/own-path` answers `200` with `SEAM_PROBE_OWN_BODY`. +2. A `HEAD` of the path answers `200` with no body. +3. A `POST` of the path, which the probe does not declare, sent with a counting token, leaves `own_path_runs_for` at `0` for that token, and the answer is not the probe's. +4. The probe and a builder of the test's own that also declares `GET` on `/seam-probe/own-path` are refused together, and the error holds `Integration route registration failed for GET /seam-probe/own-path`. +5. A builder of the test's own that declares `GET` and `POST` on `/_ts/config` never runs for the `GET`, which the inspection page answers, and runs for the `POST`. + +**A section of the integration guide.** "Serving an endpoint from a module", in `docs/guide/integration-guide.md`, states only what holds at every row. It says that a module may declare any path through `routes()`, that two routes on one method and path are refused when the registry is built, that a module route is reached only when the adapter answers no route of its own for that method and path, and that the steps that run before a module route are the adapter's request cycle, the same steps a request for one of the publisher's own pages goes through. It does not list those steps, so rows 13, 14, 25, 35, 36, 39, 52 and 53, which change them, leave the section as it is. It tells a module author that a `HEAD` route sends no body, because nothing in core, the adapters or EdgeZero strips one, and that `HEAD` and `OPTIONS` each need a route of their own. The section amends the two places in the guide that tie a module's routes to its prefix, being the rule under "Choose the narrowest hook" that proxy routes should be namespaced and bounded, which is main's text, and the row of the builder table under "What the crate provides" that says the id names the integration's routes under `/integrations//`, which row 7 added. + +## 3. What this change does not add + +- **An early answer.** A module route runs in the publisher fallback, after the Edge Cookie steps and the request filters on every adapter that runs them, and no module route is answered ahead of those steps as the inspection endpoints are. On every adapter that runs request filters, each filter sees the request for a module's endpoint, is told through `RequestFilterInput::is_integration_route` that a module route takes the request, and can answer in the route's place. The stack's own bot filter, DataDome's protection filter, lets every module route through, because `is_request_protected` in `crates/bot-protection/datadome/src/protection.rs` returns false when `is_integration_route` is set. +- **A description of each adapter's request cycle.** No adapter's guide sets out the steps the adapter runs before a module route, at this row or at the top of the stack, and this change adds none. Section 1 gives them. +- **A change to consent or privacy.** No production code changes, so a module route is handed what section 1 says, a gated value still reaches only a route that declares and is granted every permission the value's producer declares, and the tests and the guide section add no new reader of a request. +- **Attestation.** Core and the adapters gain no type, setting or route for attestation. A deployment that wants attestation serves it from a module of its own, as section 1 describes. +- **A platform identity.** `PlatformIdentity` is what an adapter reports about the platform it runs on, being a name, a service id, a service version, a staging flag and the location that answered. The type stays in core. Row 15 ([IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356)) adds `PlatformIdentity` to the platform module, `crates/trusted-server-core/src/platform/identity.rs`, with `PlatformIdentity::named` and the `UNKNOWN` constant, and hands the identity to page middleware so that core's `service_version` middleware can write `X-TS-Service-Version`. Each adapter reports its platform with `IntegrationRegistry::with_platform` when it builds its state. Fastly reports through `fastly_platform_identity`, which reads `FASTLY_SERVICE_ID`, `FASTLY_SERVICE_VERSION`, `FASTLY_IS_STAGING` and `FASTLY_POP`, and the other three report their name only. + +## 4. Tests + +- **Added.** The five checks of section 2 on each of the four adapters, and `report_route_is_the_documented_path` amended for the probe's three routes. +- **Already in the stack.** `proxy_route_reports_the_modules_geo_and_that_the_preparer_ran`, on Fastly and Axum, shows a route of the probe beneath `/integrations/seam_probe` reached through the adapter's router after the preparers. From row 52, `a_filter_s_request_header_reaches_the_route_and_its_effect_is_written_last`, on Axum, Cloudflare and Spin, shows a module route seeing a request header that a filter added. +- **Not tested.** No test declares a module route beneath a closed path, at `/robots.txt` or beneath `/static/tsjs=`. + +## Revision record + +| Date | Change | +| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-26 | First draft. | +| 2026-10-26 | Fixes from review round 1. What each adapter does at startup when two routes clash, and the other routes matchit refuses, the shipped modules that declare paths outside their prefix, what a module call carries at this row and from row 39 and what a gated value needs, where each adapter classifies the device and when the Edge Cookie step answers, the requests a module route never sees, the request filters and DataDome, the guide section and the two places it amends, the probe's handler and counter with five tests, the platform identity and the header. | +| 2026-10-26 | Fixes from review round 2. What a refusal does on the stock entry points and on a deployment's own, with a clash from settings alone, two routes on one method and path in place of two modules on one path, every function that takes outside builders, when each adapter builds its state, the plan registration, the shipped modules' paths outside `/integrations/`, the steps that keep a request from a module route and the rows that change them, and `HEAD` bodies. The empty review log is removed, and this record carries the review rounds. | From 5adc307709156e00b2954128917b2f2b75cde422 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:27 +0000 Subject: [PATCH 171/192] Developer experience follows three journeys The developer experience design follows a module author adding a module, an operator running a deployment from the gold WASM, and Tech Lab's release manager building and signing that WASM, through every step, with the command or file each step takes, what success looks like and how each likely failure reports itself. It lists the changes the journeys rest on, saying which were agreed at the Touchbase and which 51Degrees decided after it, leaves every result and time to a fresh run, and asks Tech Lab seven questions. --- .../2026-10-26-developer-experience-design.md | 191 ++++++++++++++++++ 1 file changed, 191 insertions(+) create mode 100644 docs/superpowers/specs/2026-10-26-developer-experience-design.md diff --git a/docs/superpowers/specs/2026-10-26-developer-experience-design.md b/docs/superpowers/specs/2026-10-26-developer-experience-design.md new file mode 100644 index 0000000000..d157f73a28 --- /dev/null +++ b/docs/superpowers/specs/2026-10-26-developer-experience-design.md @@ -0,0 +1,191 @@ +# Design Spec: The developer experience, step by step, for a module author, an operator and the release manager + +**Status:** Proposed, 2026-10-26. A description of the developer experience of the stack for Tech Lab review, which adds no code of its own. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-07-30-permission-model-design.md` (row 4), `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-inspection-endpoints-design.md` (row 9), `2026-10-12-config-by-host-design.md` and `2026-10-15-fastly-per-host-selector-design.md` (row 11), `2026-10-26-module-endpoints-design.md` (row 12), `2026-10-12-loading-cache-design.md` (row 24), `2026-10-15-module-cache-declarations-design.md` (row 32), `2026-10-15-asset-cache-design.md` (row 34), `2026-10-14-no-modules-crate-design.md` (row 43), `2026-10-17-config-migrate-design.md` (row 51), `2026-10-19-module-scripts-in-crates-design.md` (row 55) and `2026-10-26-integrations-crate-design.md` (row 56), in IABTechLab/trusted-server#1342. Tech Lab's [`2026-06-16-edgezero-based-ts-cli-design.md`](https://github.com/IABTechLab/trusted-server/blob/7aa5c993ba4311fd3b01872f2d2c0ca6e0f532cd/docs/superpowers/specs/2026-06-16-edgezero-based-ts-cli-design.md) on main, its [release attestation design](https://github.com/IABTechLab/trusted-server/blob/a501d1ba6bde2db82546d7e16c93facab8c613e6/docs/superpowers/specs/2026-10-09-1235-release-attestation-design.md) in IABTechLab/trusted-server#1435, and its [split integrations design](https://github.com/IABTechLab/trusted-server/blob/cd0f59823dfaecdc28eed521bef149a5d1064e26/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md) in IABTechLab/trusted-server#1194 +**Related PRs:** the rows the journeys rest on, being [IABTechLab/trusted-server#1345](https://github.com/IABTechLab/trusted-server/pull/1345) (row 4), [IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350) (row 9), [IABTechLab/trusted-server#1352](https://github.com/IABTechLab/trusted-server/pull/1352) (row 11), [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353) (row 12), [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24), [IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373) (row 32), [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375) (row 34), [IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384) (row 43), [IABTechLab/trusted-server#1409](https://github.com/IABTechLab/trusted-server/pull/1409) (row 51), [IABTechLab/trusted-server#1423](https://github.com/IABTechLab/trusted-server/pull/1423) (row 55) and row 56, which this update adds. Tech Lab's [IABTechLab/trusted-server#1262](https://github.com/IABTechLab/trusted-server/pull/1262) and [IABTechLab/trusted-server#1435](https://github.com/IABTechLab/trusted-server/pull/1435), the signed release build, and [IABTechLab/trusted-server#1194](https://github.com/IABTechLab/trusted-server/pull/1194), Tech Lab's draft design for one integrations crate +**Issues:** [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), which lists the stack's pull requests and numbers its rows, [IABTechLab/trusted-server#1055](https://github.com/IABTechLab/trusted-server/issues/1055), Tech Lab's developer experience tracker, which covers the CLI's audits, its human-readable output, its guidance on adapters and tools ([IABTechLab/trusted-server#1060](https://github.com/IABTechLab/trusted-server/issues/1060) to IABTechLab/trusted-server#1062) and the development proxy, [IABTechLab/trusted-server#1062](https://github.com/IABTechLab/trusted-server/issues/1062), which asks for clearer errors when a provider's CLI is missing and for the prerequisites to be written down, and [IABTechLab/trusted-server#1235](https://github.com/IABTechLab/trusted-server/issues/1235), the attested release binaries that the third journey follows +**Last updated:** 2026-10-26 + +> **Why this spec exists.** At the Trusted Server Touchbase meeting of 23 October 2026, IAB Tech Lab asked to review the developer experience of the stack once the permission designs are reviewed. The guides each cover part of it, and Tech Lab's tracker IABTechLab/trusted-server#1055 covers the CLI's audits, its human-readable output, its guidance on adapters and tools and the development proxy, but nothing says what a module author, an operator or the release manager does from the first step to the last. This design follows each of the three through every step, with the command or file the step takes, what success looks like and how each likely failure reports itself. It also says which figures a fresh run still has to measure, and lists the questions that only Tech Lab can answer. + +## 1. The three journeys + +This design treats the developer experience as three journeys. + +1. A module author adds a module to Tech Lab's tree, so that a stock build, being an adapter or the `ts` tool built from the repository as it ships, with the builders that its own list, its stock list, names, can run the module. +2. An operator runs a deployment from the gold WASM, which is one signed release binary that serves any publisher and is configured entirely at runtime. +3. Tech Lab's release manager builds and signs the gold WASM, which is Tech Lab's own work in IABTechLab/trusted-server#1262 and IABTechLab/trusted-server#1435. + +Each journey is a numbered list of steps, quoting the message the code writes wherever the code writes one. A row is one pull request of the stack, numbered as IABTechLab/trusted-server#777 numbers them, from IABTechLab/trusted-server#1342 at row 1 to IABTechLab/trusted-server#1423 at row 55. A branch keeps its own number, which differs from the row's number below row 7, so the permission model, IABTechLab/trusted-server#1345 on the branch `feature/3-permissions`, is row 4. + +The steps were read from the code and the guides of row 55 at `5be114773`, the highest row pushed before this update, from Tech Lab's main at `7aa5c993b`, from EdgeZero, IAB Tech Lab's framework beneath the four adapters and the `ts` tool, and from IABTechLab/trusted-server#1435 at `a501d1ba6`. This update merges Tech Lab's main into the stack, so the stack takes main's pins, being EdgeZero `v0.0.10` (`85d933b80`) where row 55 pins `v0.0.9`, and in `.tool-versions` Rust 1.98.1, the Fastly CLI 16.0.0 and Viceroy 0.21.0, Fastly's local runtime, where row 55 pins 1.95.0, 15.1.0 and 0.17.0. Every EdgeZero flag and message this design quotes is the same at `v0.0.9` and at `v0.0.10`. The steps describe the stack once this update has made the changes in the table below, some agreed with IAB Tech Lab at the Touchbase of 23 October 2026 and the rest decided by 51Degrees after that meeting, and a step that rests on one of those changes names the row that makes it. A blob is a domain's settings as `ts config push` stores them, wrapped in EdgeZero's envelope, which adds the settings' SHA-256, the time the envelope was made and a version. + +| Row | What changes | Agreed or decided | Pull request | +| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | +| 4 | No build carries a permission policy, and a domain given no valid policy does not start. The policy is a file selected and supplied when a service is deployed, carried in the domain's blob with the file's SHA-256, or read from a folder on Axum, and from row 11 Fastly stores each file once, under `permissions_`, however many domains name it. Cloudflare reads its blob from its config store in place of the Worker variable `TRUSTED_SERVER_CONFIG`, and Spin moves from its fixed `503` to the status of the error's kind (journey 2, step 9) | Agreed at the Touchbase that neither the configuration nor the policy is compiled into a build, and that a WASM given no valid policy does not start. 51Degrees decided after the Touchbase how the file is selected, stored and read, and that Spin answers with the status of the error's kind | IABTechLab/trusted-server#1345 | +| 9 | Once row 4 is merged into it, `/_ts/config` shows the policy's name and SHA-256, and never the file's text, and `/_ts/permissions` names the policy whose rules it resolved | Decided by 51Degrees after the Touchbase | IABTechLab/trusted-server#1350 | +| 12 | Attestation leaves the stack, and tests on all four adapters prove that a module outside core serves its own endpoint | Agreed at the Touchbase that attestation leaves the stack. 51Degrees decided after the Touchbase that the row keeps the tests | IABTechLab/trusted-server#1353 | +| 24, 32 and 43 | No stock build links a loading cache module | Decided by 51Degrees after the Touchbase | IABTechLab/trusted-server#1365, IABTechLab/trusted-server#1373 and IABTechLab/trusted-server#1384 | +| 34 | The asset cache becomes a module that no stock build links | Decided by 51Degrees after the Touchbase | IABTechLab/trusted-server#1375 | +| 56 | Every module crate is folded into one crate, `crates/trusted-server-integrations`, with a folder for each module | Asked by Tech Lab at the Touchbase, for the `/` folders kept in one crate, and agreed by 51Degrees. 51Degrees decided after the Touchbase that the crate holds every module, its own among them | Added by this update | + +## 2. Journey 1. A module author adds a module + +The author is a vendor, or a contributor working for one, adding a module to Tech Lab's tree. The integration guide's worked example is the seam probe, `crates/testing/iabtl` at row 55, a fixture crate through which tests drive core's extension points from outside core. The Osano module, `crates/cmp/osano`, whose Rust side only enables its browser script, shows the shape of a module that ships. The steps add a module named `cmp.example`, the name the integration guide's own examples use. They describe row 56, where every module is a folder of one crate, as `2026-10-26-integrations-crate-design.md` defines it, and step 4 says what a crate of its own costs at row 55. + +1. **Open an issue, then write the design.** Tech Lab's [`CONTRIBUTING.md`](https://github.com/IABTechLab/trusted-server/blob/7aa5c993ba4311fd3b01872f2d2c0ca6e0f532cd/CONTRIBUTING.md) on main, as the fix of the contributor documents (IABTechLab/trusted-server#1387) left it, asks for an issue before any feature work, one that describes the problem, the intended behavior, the adapters affected and the constraints on compatibility, opened before an implementation is started. A new integration is a feature, so it then needs a design in `docs/superpowers/specs/YYYY-MM-DD--design.md` that ends in a `## Review log` of at least two review rounds and is linked from the pull request. Success is an issue and a design the maintainers accept. A maintainer moves a feature pull request without a linked design back to draft, and its code is not read until the design is linked. +2. **Make the module's folder and name it.** At row 56 the module is the folder `crates/trusted-server-integrations/src/cmp/example/`, whose `mod.rs` holds what a crate's `src/lib.rs` held. The type folder's `mod.rs` declares the folder with `#[path]`, as `#[path = "example/mod.rs"] pub mod example;`, where the Rust name is the folder's name with `-` written as `_`, so `us-privacy` is `us_privacy`, and the one folder name that starts with a digit, `51degrees`, is `fiftyone_degrees`. A new type folder is declared the same way in the crate's `lib.rs`. The module's name is a constant in `mod.rs`, `pub const MODULE: &str = "cmp.example";`, which a deployment selects with `[cmp] module = "example"` and configures in `[cmp.example]`, and each part of a name is lower case letters, digits, `_` or `-`. The builder's source label is `trusted-server-integrations/cmp/example`, so the registry's messages name the folder. The test `every_folder_names_its_modules_after_itself` compares the folders on disk with a list in the test of each folder and the names it supplies, so the new folder joins that list. The test fails, naming the folder, for a folder the list leaves out, a listed folder that is gone, a name that does not follow its folder, a wrong source label, or any call of `trusted_server_core::module_name!()` in the crate, which row 56 refuses there, because the macro reads Cargo's manifest folder, and after the fold that is the one crate's folder for every module. +3. **Say who maintains it.** The folder's `maintainers.toml` names an `owner` and a `status`, one of `vendor owned`, `seeking vendor owner` and `project owned`, as `owner = "Trusted Server maintainers"` and `status = "seeking vendor owner"` do for a module no vendor has adopted yet. The test `every_folder_declares_its_maintainers` fails, naming the folder, for a folder without the file, an empty `owner` or any other `status`. The file and the test replace row 55's `[package.metadata.maintainers]` table and its test `every_crate_outside_core_declares_its_maintainers`. +4. **Add its dependencies.** A dependency the module needs goes once into `crates/trusted-server-integrations/Cargo.toml`, as an ordinary dependency. A folder gets a Cargo feature, named after the folder as `-`, only for a dependency that not every build can or should carry, or for a module that no stock build links (section 2.8 of `2026-10-26-integrations-crate-design.md`). Five folders have one. The Fastly SDK sits behind `device-fastly` and `geo-fastly`, which the Fastly adapter turns on. `fiftyone-caching` sits behind `loading-cache-51degrees` and the asset cache module behind `asset-cache-51degrees`. No stock build turns either on. A deployment that links the 51Degrees loading cache or the asset cache module turns that module's feature on, and so do the aliases of step 13. The seam probe sits behind `testing-iabtl`, which only the Axum and Fastly adapters' dev-dependencies turn on, so no adapter build compiles the probe. A feature gates the code in its folder and never the folder's name constants or its source label, so the tests of steps 2 and 3 check every folder whatever features are on. Success is the aliases of step 13 building the crate. At row 55 a module is a crate of its own, which joins `members` and `[workspace.dependencies]` in the root `Cargo.toml`, the manifests of the four adapters and of `crates/trusted-server-cli`, and the six aliases in `.cargo/config.toml` that name module crates. Nothing compares those aliases with the workspace, so an alias that leaves a crate out does not lint or test it. That is why the seam probe, which is in no alias at row 55, is tested by a step of its own in `.github/workflows/test.yml`, "Run host-target seam probe integration tests". Row 56 replaces all of those entries with the one crate, the aliases name `-p trusted-server-integrations`, and the native aliases turn on `testing-iabtl`, so that step goes. +5. **Write the builder and its settings.** `mod.rs` gives `builder()`, which returns `IntegrationBuilder::new(ID, "trusted-server-integrations/cmp/example", register, validate).with_module_name(MODULE)`. The settings type takes `#[serde(deny_unknown_fields)]` and implements `IntegrationConfig`. The `validate` function reads `settings.module_config::(MODULE)` and runs for every builder, selected or not, and the `register` function answers `Ok(None)` when no section selects the module. Both should return a refusal as `TrustedServerError::Configuration`, as `settings.module_config` does, because each adapter's startup error router answers with the status of the error's kind, `500` for a configuration or settings error and `503` for a key-value store error, so a refusal of another kind, such as an integration error, would make the service answer `502`. The other builder calls are listed under "What the crate provides" in the [integration guide](https://github.com/IABTechLab/trusted-server/blob/5be114773345a03e207c1da2d3b163a23510985f/docs/guide/integration-guide.md). Success is the module's tests passing (step 13). The refusals a module's settings meet are these, as row 55 words them, each a configuration error that names the table. + - A setting the type does not know, "[cmp.example] could not be read: …", which names the setting and never its value. + - A table its section does not select, "[cmp.example] is configured, but [cmp] module does not select `example`. Add it to module, or remove the table". + - A section that selects nothing, "[cmp] selects no module. A module type's section names what runs with `module` or `modules`, so select one or remove the section". + - At startup, a name that no linked builder supplies, "[cmp] selects `example`, which no module this deployment supplies. The cmp modules it supplies are […]". + - At startup, two builders with one id, "integration id `example` is registered twice, by `` and by ``". +6. **Serve its routes, if it has any.** The registration carries an `IntegrationProxy` through `.with_proxy(...)`, and the proxy's `routes()` lists its endpoints. The proxy's own `self.get("/country")` puts the proxy prefix in front, `/integrations/`, the integration id, so the route is `/integrations/example/country`, and `IntegrationEndpoint::get(path)` keeps a path of the module's own choosing. Row 12 adds tests on all four adapters that a module outside core serves its own endpoint this way, and the integration guide's section "Serving an endpoint from a module". `2026-10-26-module-endpoints-design.md` sets out what such a route meets. + - A `GET` route answers neither `HEAD` nor `OPTIONS`, so a module that answers either declares a route of its own for that method, as Sourcepoint declares `HEAD` and `OPTIONS` beside its `GET` and `POST`. A `HEAD` route sends its empty body itself, because nothing in core, the adapters or EdgeZero strips a body from a `HEAD` answer. + - A module route never sees a request that a request preparer refuses, whose Edge Cookie step fails or that a request filter answers. A request preparer is a function that a module hands its builder with `.with_request_preparer(...)`, and the function runs once for each request before routing and may change the request or refuse it. + - Nor does a module route see six kinds of request that something ahead of the route answers, being a route of the adapter's own for the same method and path, such as `GET /_ts/config`, a request the Fastly entry point answers before its router, such as `GET /health`, a closed path beneath `/_ts/admin` or `/admin/keys`, a `GET` or `HEAD` of `/robots.txt` when `[robots-txt]` is present, a `GET` beneath `/static/tsjs=`, and, from row 35, the path a reader sent where the deployment's request addressing rewrites the path before routing. The registry does not know the adapter's routes, so the registry accepts a route on such a path, and nothing reports that the route never runs. + + Success is the route answering in the adapters' tests. When the registry is built it refuses two routes on one method and path, from one module or two, with `Integration route registration failed for : …`, a configuration error, so every request for the domain answers `500` and the log carries the reason at error level. + +7. **Declare the permissions it needs.** Each part of the module that is handed a request's values, being its route, its middleware and any Edge Cookie, geo, device or robots.txt module it offers, names the permissions it needs in `required_permissions()`, which declares none by default. The location, the device signals and the Edge Cookie identifier each carry the permissions their producer declares, and each is passed only to a module that declares all of those permissions and is granted them on that request. Which permissions a request is granted follows from the request's signals and the operator's policy file, which row 4 has the deployment supply (journey 2, step 4), and never from the module. Success is the module's function being called with the value on a request that grants the permissions. On a request that does not, a function that names the value is not called and the reason goes to the debug log, and a route's `?` answers `403` for a value the request's permissions withhold. Nothing gates a route itself, so a route that must not run without a permission reads `&PermissionState` and refuses. +8. **Declare the caches it loads through.** A module that keeps answers names each cache under itself with `.with_cache_features(&["example-answers"])`, selected in its own `[cache.example-answers]` table, and loads through `services.load_cached(...)`. Rows 24 and 43 leave every stock build, and therefore the gold WASM, with no loading cache linked, so there `load_cached` runs the loader on every call. A module whose load pays off only when its value is kept therefore asks `services.loading_cache_for(name).keeps_values()` before loading, and on such a build the answer is false. The refusals are "`example` declares the cache `robots-txt`, which is core's own. A module names its caches under itself, as modules are named", for `robots-txt`, which is the one cache core declares once row 34 moves the asset caches into the asset cache module, which names them `51degrees-assets` and `51degrees-assets-vary`, then "`example` and `` both declare the cache ``, which would hold both their answers under one set of keys. …", and, for a table that names a cache no linked module declares, `[cache.] names a cache no linked module loads through. The caches this build's modules declare are: …`. +9. **Supply its page changes, if it has any.** A page change is a `Middleware` whose `middleware_id()` is the module's name, or the module's name with a part after it, such as `cmp.example.cleanup`, for a second one, with its `phases()` and `create()`, and `prepare()` where it has to wait on a store or the network. The builder describes each in a `const SUPPLIED_MIDDLEWARE: &[SuppliedMiddleware]`, such as `SuppliedMiddleware::on_html(MODULE, MiddlewarePhase::Fetch)`, handed to `.with_middleware(...)`, so `ts audit` places the middleware without reading the module. The middleware then joins the commented `[[fetch]]` or `[[serve]]` entries in `trusted-server.example.toml` and the table of middleware in `docs/guide/configuration.md`, in the order the registry registers them. Two tests in `crates/trusted-server-adapter-axum/tests/shipped_modules.rs` hold the template's entries and the guide's table in step with the linked modules, and fail with `should document every middleware the linked modules supply for , in the listed order` and "should document the middleware the linked modules supply, each with its phase, in the listed order". At startup the registry refuses a registration whose middleware contradicts its description, and an entry that names a middleware in a phase it does not run in. +10. **Add its browser script, if it has one.** The script is `js/src//index.ts` in the module's folder, with its tests in `js/test//`, as row 55 placed it in each crate and row 56 keeps it in each folder, and the build bundles it as `tsjs-.js`. The folder's `js/src` joins the `@tsjs/module/*` paths in `crates/trusted-server-js/lib/tsconfig.json`, in discovery order. The checks are `npm run lint`, `npm run format`, `npm run build` and `npm test` in `crates/trusted-server-js/lib`. The next build that runs for another reason finds a folder's first `js/src`, and touching `crates/trusted-server-js/lib/package.json` forces such a build. A script outside `lib` cannot import an npm package. The likely failures are these, as row 55 words them. + - The Vitest test "lists every folder of module sources for @tsjs/module, in discovery order" fails when `tsconfig.json` leaves the folder out. + - The build stops with `tsjs: failed to discover modules: module is in both and `, or with `module name core, used by , is the shared core's bundle`. + - A script the module carries itself, through `.with_js_module(CarriedJsModule { source, sha256 })`, is refused at startup when its stated digest is wrong, with "Integration `` carries a browser module whose declared SHA-256 does not match its source (declared …, actual …)", most often because a checkout rewrote the script's line endings. +11. **Link it into the stock build.** At row 56 the builder is `trusted_server_integrations::cmp::example::builder()`, and it joins `shipped_module_builders()` in `src/app.rs` of each of the four adapters and `linked_builders()` in `crates/trusted-server-cli/src/app_config.rs`, in the place its request hooks should run. A permission signal module joins `shipped_signal_modules()` in each adapter instead, and the CLI links no permission signal module. At row 55 the five lists of builders hold the same builders in the same order, and no test compares one list with another, as `2026-10-14-no-modules-crate-design.md` records. A module that is not on the lists is not in the gold WASM, because the stock Fastly binary calls `trusted_server_adapter_fastly::run()`, which links the stock list and nothing else. A module kept in a repository of its own runs only in a deployment that builds its own binary, handing the module's builder to `run_with` on Fastly or to `routes_with_registrations` on Axum, Cloudflare and Spin, and that binary is not the gold WASM. An adapter whose list misses the module refuses it at startup once a section selects it, with "[cmp] selects `example`, which no module this deployment supplies. …", and when the CLI's list misses it, `ts config validate` and `ts config push` skip the module's own rules. Tech Lab's IABTechLab/trusted-server#1194 agrees with the stack on one crate named `trusted-server-integrations`, and section 2.11 of `2026-10-26-integrations-crate-design.md` sets out where the two differ. + - IABTechLab/trusted-server#1194 finds every integration through "one compile-checked catalog" in that crate and makes "External vendor crate injection or adapter-supplied registration" a non-goal, where the stack keeps each adapter's own list and lets a deployment add a crate of its own through `run_with`. + - Rule 6 of IABTechLab/trusted-server#1194 reads "No adapter reconstructs a concrete catalog or imports `aps`, `prebid`, or another integration module directly", so there the integrations are named only in the integrations crate, which puts the catalog together, where the stack also names them in the five lists written by hand. + - IABTechLab/trusted-server#1194 gives each integration one flat folder, `src/integrations//`, where the stack keeps `src///`. + - IABTechLab/trusted-server#1194 moves the browser code into a second crate, `trusted-server-integrations-js`, where the stack keeps each script in its module's folder. +12. **Run it locally against a page.** The author runs the module under Viceroy and looks at a real page through the dev proxy. The author copies `trusted-server.example.toml` to `trusted-server.toml` at the repository root, selects the module with `[cmp] module = "example"` and, from row 4, names a policy with `[permissions] policy = "sample"`. In the same file the author sets `[source.origin] url` to the origin of the page under test, in place of the template's `https://origin.example.com`, and `[publisher] domain` and `cookie_domain` to the host the dev proxy below maps, `www.publisher.example`, so that the page comes from its own origin and its first-party addresses use the mapped host. The push refuses a template value left in place with "Insecure default value for: …", which names each such setting, such as `source.origin.url`. EdgeZero's environment overlay replaces a setting with the value of an environment variable named `TRUSTED_SERVER__` and the setting's path, such as `TRUSTED_SERVER__PUBLISHER__DOMAIN` for `[publisher] domain`, and the overlay only overrides keys that already exist, so a variable that names one of these three keys, as `.env.dev` does for the two hosts, would replace the value in the file. The push therefore takes `--no-env`, which row 4's push honors by leaving the overlay out of its own load, and an author in doubt runs the push in a shell where no `TRUSTED_SERVER__` variable is set. `ts config push --adapter fastly --local --yes --no-diff --no-env --permissions config/permissions/sample.yaml` then writes the settings into the local store in `fastly.toml`, and `fastly compute serve`, or `ts serve --adapter fastly`, serves them under Viceroy at `http://localhost:7676`. The repository's sample policy is for testing and evaluation only, which is what this run needs. `ts dev proxy --map www.publisher.example=localhost:7676 --upstream-plaintext` then serves the mapped host from the local server, on macOS and Linux, to a browser pointed at the proxy. The command leaves out `--rewrite-host`, as the local example of the [dev proxy guide](https://github.com/IABTechLab/trusted-server/blob/5be114773345a03e207c1da2d3b163a23510985f/docs/guide/ts-dev-proxy.md) does, so the local server is asked with `Host: www.publisher.example`, and the Fastly adapter, which strips `X-Forwarded-Host`, keeps the page's first-party addresses on that host. Success is `curl -s http://localhost:7676/_ts/config.json` showing the module's section, and the page showing the module's change through the proxy. A settings refusal at startup makes every request answer `500`, with the reason logged at error level. Tech Lab's IABTechLab/trusted-server#1055 covers the proxy's diagnostics. +13. **Test it.** The module's own tests follow Osano's, in `crates/cmp/osano/src/lib.rs` at row 55. They check that an unselected module registers nothing, that an unknown setting is refused, that `validate_settings_for_deploy_with(&settings, &[builder()])` refuses a bad table, and that a selected module registers through `IntegrationRegistry::with_plan_and_registrations`. The Axum adapter's `every_linked_module_rejects_a_setting_it_does_not_know` then covers every linked module, and fails with "`cmp.example` should refuse a setting it does not know". The integration guide's registration checklist also asks for "source and behavior parity records" beside the tests. Locally, at row 56, `cargo test-axum` runs the integrations crate's tests on the host with the features `loading-cache-51degrees`, `asset-cache-51degrees` and `testing-iabtl`. `cargo test-fastly` runs them as `wasm32-wasip1` under Viceroy, at the version `.tool-versions` pins, with the two Fastly features, which the Fastly adapter's dependencies turn on, `testing-iabtl`, which reaches the alias through that adapter's dev-dependencies, and `asset-cache-51degrees`, which the alias names. `cargo test-cloudflare`, `cargo test-spin`, `cargo fmt --all -- --check` and the clippy aliases follow, with the browser checks of step 10. The [testing guide](https://github.com/IABTechLab/trusted-server/blob/5be114773345a03e207c1da2d3b163a23510985f/docs/guide/testing.md) lists the commands. On the pull request, `.github/workflows/test.yml` runs the four adapters' tests, the cross-adapter parity tests, the CLI's tests and Vitest, and `.github/workflows/format.yml` runs `cargo fmt`, every clippy alias, ESLint, Prettier, the documentation build and `scripts/lint-core-platform-names.py`, which fails when core names a hosting platform. Success is every job passing on the pushed commit. +14. **Document it and open the pull request.** The module gets a page in `docs/guide/integrations/` with an entry in the sidebar of `docs/.vitepress/config.mts`, where the Osano page is `{ text: 'Osano', link: '/guide/integrations/osano' }`, a row in the configuration guide's table of integration sections, a commented block for its table in `trusted-server.example.toml` and an entry in `CHANGELOG.md`. The pull request follows main's `.github/pull_request_template.md`, whose `## Closes` section links the issue of step 1, whose `## Spec` section links the design, and whose `## Test plan` section ticks the checks the author ran, such as `cargo test-fastly && cargo test-axum` and `cargo fmt --all -- --check`. `ts dev install-hooks` installs a check of URL hosts before each commit, `ts dev lint domains`, which flags a host outside the allowlist in `crates/trusted-server-cli/src/commands/dev/lint/domains.rs`, so a vendor's real endpoint is added there or marked with `// allow-domain: `. The documentation build in `format.yml` fails on a dead link. + +## 3. Journey 2. An operator runs a deployment from the gold WASM + +The operator runs Trusted Server on Fastly for one or more publishers from the release Tech Lab signs, and builds nothing apart from the staged rehearsal of step 6. The first phase of IABTechLab/trusted-server#1435 builds `trusted-server-fastly.wasm` for Fastly, `ts` for Linux and macOS and three Axum binaries, and no binary for Cloudflare or Spin, so the steps are Fastly's, and what differs on the other adapters follows them. A domain is a publisher host the service answers. On Fastly each domain has a settings blob of its own, stored under the domain's host, and the service's selector, the `TS__CONFIG__KEY` entry in the store linked as `trusted_server_config`, holds `{host}`, which the runtime replaces with each request's host to find the blob (row 11). The examples use the host `www.publisher-a.example`, its settings file `publisher-a.toml` and the policy file `eu.yaml`. + +The journey needs these. + +- A Fastly Compute service with each publisher's domain added, as the "Create a Compute Service" section of `docs/guide/fastly.md` describes, and each domain's DNS record and TLS certificate set up as that guide's "Domain Configuration" section describes, because the deploy of step 8 passes `--no-default-domain`, which skips creating a default domain, and no step of the journey adds a publisher's domain. +- `gh` 2.102.0, the version of the GitHub CLI that the release verification guide of IABTechLab/trusted-server#1435 names for its offline flags, beside its advice to use the version the release job's log records, with `jq` for its comparison of the SBOM, the software bill of materials that lists the packages a binary was built from, and `shasum` for its checksums. `shasum -a 256` is the one hash tool this journey uses. +- The Fastly CLI at the version the release tag's `.tool-versions` pins, which is 16.0.0 on main. +- Chrome or Chromium, which `ts audit generate` needs. +- `FASTLY_API_TOKEN`, a user token with Global API Access, as `docs/guide/fastly.md` asks, exported in the environment of every command below. A Fastly CLI profile, such as `ts auth login --adapter fastly` creates, does not supply the token that `ts deploy` and the per-host push read (`docs/guide/cli.md`). IABTechLab/trusted-server#1062 asks for clearer errors when a provider's CLI is missing and for these prerequisites to be written down. + +1. **Get the release and check it.** No signed release with binaries exists yet. Tech Lab published two pre-releases on 21 October 2026, `v0.1.0-rc.202609.1` from main and `v0.1.1-rc.202610.1` from the release candidate branch `rc/202610`, and neither carries an asset. The first release this journey can use is one cut after the stack merges into main, because IABTechLab/trusted-server#1435 builds main, which has neither the permission model of row 4 nor the blob for each host of row 11. Once a release with binaries exists, the [release verification guide](https://github.com/IABTechLab/trusted-server/blob/a501d1ba6bde2db82546d7e16c93facab8c613e6/docs/guide/release-verification.md) in IABTechLab/trusted-server#1435 checks it with these commands, for a tag and its approved 40-character source commit, which the guide says to obtain "through an independent source-review process". + - `gh release download "$TAG" --repo IABTechLab/trusted-server --dir .` downloads the assets. + - `test "$(gh api "repos/IABTechLab/trusted-server/releases/tags/$TAG" --jq .immutable)" = true` checks that the release is immutable. + - `gh release verify "$TAG"` checks the release, and `gh release verify-asset` checks every asset. + - `gh attestation verify` checks each binary's provenance, the signed record of the workflow and the source commit that built the binary, and its SBOM, with `--source-digest` and `--signer-digest` set to the approved commit. It runs once with `--signer-workflow` naming `.github/workflows/release-build.yml`, which matches a prefix and names no ref, and once with `--cert-identity`, which pins that workflow at the tag. + - A `jq` comparison checks that the verified SBOM predicate equals the shipped `.spdx.json`, the SBOM written in SPDX, a standard file format for an SBOM. + - `shasum -a 256 --check SHA256SUMS` checks the other assets. + + The verified `ts-` is installed as `ts`. `edgezero.toml` and `fastly.toml` come from a reviewed checkout of the tag, because the CLI reads both and the logical store names come from `edgezero.toml`, and no check above covers either file. Every `ts` command of the journey runs from the root of that checkout, because the CLI loads `edgezero.toml` from the working directory unless `EDGEZERO_MANIFEST` names another file (`docs/guide/cli.md`). Success is every command exiting 0, and the guide says to stop at the first check that fails. The design in IABTechLab/trusted-server#1262, at [`122b78664`](https://github.com/IABTechLab/trusted-server/blob/122b78664b1235c717e9208fd200f9a9b0c335df/docs/superpowers/specs/2026-10-09-1235-release-attestation-design.md), plans `ts release verify --asset --expected-source-commit `, one read-only command for the whole online check, which the first phase of IABTechLab/trusted-server#1435 leaves to the `gh` commands above. + +2. **Create the stores, fill them and link them.** The copy of `fastly.toml` from the tag names a service of its own, so its `service_id` is set to the operator's service, because `ts provision` refuses a `FASTLY_SERVICE_ID` that differs, with "fastly.toml service_id `` conflicts with FASTLY_SERVICE_ID `` …". With the service selected, `ts provision --adapter fastly` creates the stores `edgezero.toml` declares, being the config store `trusted_server_config`, the secret store `trusted_server_secrets` and the key-value (KV) store `trusted_server_kv`, or the physical stores that `EDGEZERO__STORES______NAME` names, such as `ts_secrets` for the secrets. For each store it creates, it prints the command that links the store, `fastly service resource-link create --service-id= --resource-id= --version=latest --autoclone --name=`, with the store's id from `fastly -store list --json`. The template's `[ec] ec_store = "ec_identity_store"` names a fourth store, the KV store that keeps the data held with the Edge Cookie, which `fastly kv-store create --name ec_identity_store` creates, as the section "Create EC KV Store" of `docs/guide/fastly.md` does. That store is linked the same way under the name `ec_identity_store`, because the Fastly adapter opens it by the name `ec_store` gives. The secret store holds each secret that the settings file of step 3 names by key, being `publisher_proxy_secret`, which the template's `[publisher] proxy_secret` names for the secret that signs first-party proxy URLs, and any other key the file names, such as `ec_passphrase` for the `hmac` Edge Cookie module. Each is written with `fastly secret-store-entry create --store-id= --name= --stdin`, as the Fastly guide does with `openssl rand -hex 32 | tr -d '\n' | fastly secret-store-entry create --store-id= --name=ec_passphrase --stdin`, so that the value never appears in the process argument list. The operator runs the link commands, and the latest version they leave, which links each store under its name, is the base version of step 8. The Fastly guide says of the `trusted_server_secrets` link, "Do not create that link by hand". That holds on the managed route, where `ts deploy` makes every link itself, and not on the manual route that the gold WASM takes today, where nothing else makes the links. Success is a version that links all four stores. A version without the `trusted_server_config` or the `trusted_server_secrets` link cannot load any domain's settings, so every publisher route answers `500` while `/health` keeps answering `200`, as the Fastly guide says of a version that lacks the `trusted_server_secrets` link. Without the `ec_identity_store` link the Edge Cookie is still created and written, and its data is not kept, as the Fastly guide says of a service without that store. +3. **Write each domain's settings.** `ts config init --app-config publisher-a.toml` writes the template and prints "Initialized config at publisher-a.toml", and refuses a file that exists with "publisher-a.toml already exists; pass --force to overwrite". The template's placeholder values, `example.com` in `[publisher] domain`, `.example.com` in `cookie_domain` and `https://origin.example.com` in `[source.origin] url`, are refused by `ts config validate` and `ts config push` with "Insecure default value for: …", which names each one left, until the operator replaces them. `ts audit generate https://www.publisher-a.example --config publisher-a.toml` writes a draft from the live page instead. The file selects modules in the sections of their types and places their page changes in `[[fetch]]` and `[[serve]]` entries. Neither command writes anything that chooses a policy, because no policy ships with Trusted Server and no tool chooses one. From row 4 the template names no policy and carries a comment telling the operator to name their own file in `[permissions] policy`, and a push of the template as written is refused with a message naming what is missing, so the operator names the domain's policy in `[permissions] policy = "eu"`. A file written for an earlier release is first copied, for a rollback, and then rewritten in place by `ts config migrate --adapter fastly --write publisher-a.toml`, which prints a `to do:` line for each item it leaves, the policy among them, and exits 1 while one is left. The [configuration rules](https://github.com/IABTechLab/trusted-server/blob/5be114773345a03e207c1da2d3b163a23510985f/docs/guide/configuration-rules.md) give the rules every table follows. +4. **Write the policy files.** A policy is a YAML file in the format of `config/permissions/sample.yaml`, which is a sample for testing and evaluation only, and the file's stem is the policy's name, so `eu.yaml` is the policy `eu`. The section "Groups and rules" of the [permission model guide](https://github.com/IABTechLab/trusted-server/blob/5be114773345a03e207c1da2d3b163a23510985f/docs/guide/permission-model.md) says how to write the file's groups and its tree of rules, and row 4 rewrites the guide's two sentences that say the file is compiled into the build. The push takes only a file whose name ends `.yaml`, and any number of domains can name one file. Row 4 makes the file a deployment input, so no build carries a policy. The file has to reach the push byte for byte, because a checkout that converts line endings changes the file's bytes and so its SHA-256, and no `.gitattributes` in the repository covers the policy files. A domain whose settings name no policy, or name a file that was not supplied, or a file whose SHA-256 differs from the one its blob pins, does not load (step 9). +5. **Validate each domain.** `ts config validate --app-config publisher-a.toml --permissions eu.yaml`, where row 4 adds `--permissions`, which may be repeated and which `ts config diff` and `ts config push` also take. At row 55 success prints "[edgezero] config validate (typed): edgezero.toml + publisher-a.toml OK". A refusal prints after `[ts]` and exits 2, and its message names the table and the fix, such as "[cmp.example] is configured, but [cmp] module does not select `example`. Add it to module, or remove the table". From row 4, given `--permissions`, validation also runs the policy checks of section 3.1 of `2026-07-30-permission-model-design.md`, which refuse settings that name no policy or a policy that no supplied file has, and a file that fails the checks of section 3.3. Without `--permissions`, `ts config validate` says that the policy was not checked. Validation runs the rules of every module the CLI links, which for a `ts` from the same release is the list the gold WASM links. Four selections, `[ec]`, `[geo]`, `[device]` and `[permission-signal]`, are settled only when the service starts, so a passing validation does not prove that a change to those four will start. +6. **Rehearse on staging.** There is no staging for the gold WASM until AC-8 lands, the milestone of IABTechLab/trusted-server#1262 in which a pinned EdgeZero Fastly API deploys the verified bytes to production and to staging. `ts deploy --staging` takes an EdgeZero application release, and the manual route of step 8 stages nothing. The AC-8 plan in IABTechLab/trusted-server#1435 turns staging on, as `ts deploy --release --staging`, only after its task E8 has qualified the route on disposable Fastly services, production first and staging after it, and a release that allows staging has been published (section 6, point 2). Until then a staged rehearsal runs a source build of the same commit, and that build is not the gold WASM. This is the one step of the journey where the operator builds, in a workflow of the operator's own, the application release pipeline that EdgeZero's [GitHub Actions deployment guide](https://github.com/IABTechLab/edgezero/blob/85d933b8044dec29fe9e4685cae194eaf0290911/docs/guide/deploy-github-actions.md) calls the producer. That workflow checks out the commit and builds the application CLI and the Fastly package, as the section on lifecycle commands in `docs/guide/cli.md` says, and EdgeZero's `package-application-release-fastly` action assembles them into the release, which EdgeZero's `deploy-fastly` action or `ts deploy --application-release` then deploys. Such a rehearsal gives staging a physical config store of its own, because a staged push to the production store overwrites the production entry and production instances load that entry on their next start. With `EDGEZERO__STORES__CONFIG__TRUSTED_SERVER_CONFIG__NAME` naming the staging store, each domain is pushed as in step 7 under a key that the staging store's own selector reads, such as the selector `{host}_staging` with `--key www.publisher-a.example_staging` (section 2.5 of `2026-10-15-fastly-per-host-selector-design.md`), or with `--staging` for a service whose one blob sits at the logical key. The CLI refuses `--staging` together with `--key` when it parses the command. `ts deploy --staging` then stages the version, `ts healthcheck --staging` probes it, and `ts rollback --adapter fastly --service-id --version --staging` deactivates it. Success is the staged version passing the checks of step 9. +7. **Push each domain's settings.** For each domain, `ts config push --adapter fastly --key www.publisher-a.example --app-config publisher-a.toml --permissions eu.yaml` runs with `EDGEZERO__STORES__CONFIG__TRUSTED_SERVER_CONFIG__NAME` naming the production store, and then the selector is set once, with `fastly config-store-entry update --upsert --store-id --key TS__CONFIG__KEY --value '{host}'`. The push writes the policy's name and SHA-256 into the domain's blob (row 4), and on Fastly from row 11 it writes each policy file it is given once, as the entry `permissions_` however many domains name it, before the domain's blob. The push always runs the policy checks of step 5, and also refuses settings or an overlay that set the push's own `sha256` or `text`, a file that is not UTF-8 text and two supplied files with one name. When the push has just written a new policy entry it waits 60 seconds before it writes the blob, because Fastly documents no time for a change to reach every location, so the wait is a margin and not a guarantee, and a location that sees the blob before its policy entry answers `500` for that domain and logs the reason (`2026-07-30-permission-model-design.md`, section 3.1). At row 55 a successful push prints "wrote `www.publisher-a.example` (`` bytes) to the Fastly config store ``" for each entry of the blob, and `--dry-run` prints the same lines with "would write" and writes nothing. The likely failures are these. + - "no Fastly config store is named ``", when no store on the account has the name the environment selects. + - "config key `` is `` characters, over the 255-character limit. …", before anything is written. + - `ts config push --key` stores a key as typed, so a key with upper case letters is accepted and never read, because the runtime lowercases each request's host, and every request for that host answers `421`. + - `ts config diff` compares only the blob at the logical key `trusted_server_config`, because EdgeZero refuses any other key on Fastly with "Fastly uses logical config key `trusted_server_config` for every target …", so `--dry-run` previews a domain's push where no diff is available. For the blob it does compare, an edited policy file shows as a changed `sha256` (row 4). +8. **Deploy the WASM.** First, `ts active-version --adapter fastly --service-id ` records the version to roll back to, because once a deploy activates a new version Fastly keeps no record of which one was live. Today the gold WASM reaches Fastly by the manual route of the release verification guide, which IABTechLab/trusted-server#1262 records as AC-9, the manual route for verified bytes to production. In a fresh private folder that holds only the verified WASM and a minimal `fastly.toml`, `fastly compute pack --wasm-binary ./trusted-server-fastly.wasm`, `fastly compute validate`, a `cmp` of the packaged WASM against the download, and `fastly compute deploy --package --service-id --version --no-default-domain` deploy the exact signed bytes. In the guide's words this route "does not create links, push configuration, implement staging, or supply recovery guarantees", so the base version is the one step 2 made, which links the stores. The guide says that `fastly compute deploy` can "change an active/locked base version by cloning it" and asks the operator to "confirm its behavior for the pinned CLI before use", so the operator confirms that the deployed version keeps the base version's links. The managed route, `ts deploy --adapter fastly --service-id --application-release `, with `EDGEZERO_MANIFEST` naming the folder's `edgezero.toml`, takes the folder that an EdgeZero application release is extracted into. That release is an archive holding a CLI, the Fastly package and both manifests, which EdgeZero's `package-application-release-fastly` action assembles, and IABTechLab/trusted-server#1435 publishes no such archive. The [AC-8 plan](https://github.com/IABTechLab/trusted-server/blob/a501d1ba6bde2db82546d7e16c93facab8c613e6/docs/superpowers/plans/2026-10-13-fastly-prebuilt-edgezero-integration.md) in IABTechLab/trusted-server#1435 adds `ts deploy --release` for the verified bytes in its task E7, for production only at first. Success is the new version active on the service. +9. **Check the deployment.** `ts healthcheck --adapter fastly --service-id --version --domain www.publisher-a.example --path /_ts/config` passes only on a `2xx` answer, and `/_ts/config` answers `200` only once that domain's settings have loaded, unless `[inspect] config = false` turns the page off with `404`. The default path, `/`, passes only when the publisher's home page answers `2xx`, and `/health` says nothing about the settings, because the Fastly adapter answers `/health` with `200` before reading any settings. A failed check prints `healthcheck for www.publisher-a.example failed after 3 attempt(s): unhealthy HTTP status `. `ts healthcheck`, `ts active-version` and `ts rollback` work on a version either route deployed, because EdgeZero runs each of them from the flags given and reads no manifest. `curl -s https://www.publisher-a.example/_ts/config.json` then shows the settings the domain runs, with every secret as `XXXX`, and from rows 4 and 9 the policy's name and SHA-256, which `shasum -a 256 eu.yaml` matches, and never the file's text. `curl -s https://www.publisher-a.example/_ts/permissions.json -H "Sec-GPC: 1"` shows the permissions the policy grants that request and the signals that produced them, and from rows 4 and 9 names the policy. Each adapter's startup error router answers with the status of the error's kind, as Fastly, Axum and Cloudflare do at row 55. A configuration or settings error, which includes a domain without a valid policy file and a config store that cannot be read, answers `500`, a key-value store error answers `503`, and the reason is logged. On Fastly each log line goes to the service's logging endpoint named `tslog` and is echoed to standard output, which `fastly log-tail --service-id ` streams. Row 4 moves Spin from its fixed `503` to the status of the error's kind. A check can meet these answers. + + | Case | Answer | + | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | + | A host with no blob, on Fastly, because the service is not configured for that host | `421 Misdirected Request`, marked `private, no-store`, so the host is served as soon as its blob is pushed | + | A domain whose settings do not load, including, from row 4, settings that name no policy, or a policy file that was not pushed or does not match the digest its blob pins | `500` to every request for the domain, the status of a configuration or settings error, with the body "An internal error occurred" and the reason logged at error level as "failed to build application state: …" | + | On Fastly, a config store that cannot be opened or read, or a malformed `TS__CONFIG__KEY` selector | `500` with the body "Internal Server Error", answered by the entry point before its router, with "cannot read the app-config selector: …" or "cannot tell whether … has an app-config blob: …" logged at error level | + | A startup error of another kind | The status of its kind, such as `503` for a key-value store error, with the reason logged at error level | + | Any startup error on Spin before row 4 | `503 Service Unavailable` whatever its kind, with "failed to build application state: …" and "startup failed, serving error fallback: …" logged at error level. Row 4 moves Spin to the status of the error's kind | + | `/health` | `200` on Fastly before any settings are read, and on Spin while its start has failed, so the answer shows that the service runs and not that its settings loaded | + +10. **Change a policy, or roll back.** From row 11 a changed policy file has a new digest, so the push stores it as a new entry under a new key and never edits one in use, and a domain runs the changed file only once its settings are pushed again with it. No command yet removes a policy entry that no blob names, because `ts config gc` removes only chunks that nothing points to, a chunk being one of the entries into which a value longer than a Fastly config store entry's 8,000 characters is split, so such an entry stays in the store and changes no answer (`2026-07-30-permission-model-design.md`, section 3.1). Pushing a domain's previous settings file with its previous policy file rolls the domain's settings back. `ts rollback --adapter fastly --service-id --version --rollback-to ` re-activates the version step 8 recorded, whichever route deployed the new one. EdgeZero's [GitHub Actions deployment guide](https://github.com/IABTechLab/edgezero/blob/85d933b8044dec29fe9e4685cae194eaf0290911/docs/guide/deploy-github-actions.md) asks that settings pushed before a deploy stay readable by the running release until the deploy and its health check finish. The first deployment of this update cannot meet that, because a build of row 55 refuses settings that carry `[permissions]`, with "[permissions] is not a section Trusted Server reads itself, so it is read as the section of a module type, and it has `policy`, which is not a setting it reads. …", and a build of this update refuses settings that name no policy. Until AC-8 brings staging for the gold WASM, the route for that deployment is the one EdgeZero's guide gives when settings cannot stay compatible, "Use versioned physical stores when compatibility cannot be guaranteed". + - `fastly config-store create --name=` creates a second config store. With `EDGEZERO__STORES__CONFIG__TRUSTED_SERVER_CONFIG__NAME` naming it, the pushes of step 7 write every domain's new settings and policy files into it, and `fastly config-store-entry update --upsert --store-id --key TS__CONFIG__KEY --value '{host}'` sets its selector. + - `fastly service version clone --version=active --service-id=` makes a draft version. `fastly service resource-link list --service-id= --version=` gives the id of the draft's `trusted_server_config` link, `fastly service resource-link delete --id= --version= --service-id=` removes that link, and `fastly service resource-link create --resource-id= --version= --service-id= --name=trusted_server_config` links the second store in its place. + - The new WASM is deployed onto that draft version by the manual route of step 8. + + The version that was live still links the first store with the previous settings, so `ts rollback` to that version restores both the build and the settings. The same route serves an operator whose service runs a build of Tech Lab's main, because that build refuses the settings this update's build needs, and this update's build refuses main's settings until `ts config migrate` rewrites them and they name a policy. For an operator whose service runs nothing yet, steps 2 to 8 already follow the route, because the base version links the first config store and no earlier settings have to stay readable. + +**On the other adapters.** IABTechLab/trusted-server#1435 builds three Axum binaries, named `trusted-server-axum-`. Axum reads one settings blob, which `ts config push --adapter axum --local --manifest ./edgezero.toml --app-config ./trusted-server.toml --permissions eu.yaml --no-diff --yes` writes to `.edgezero/local-config-trusted_server_config.json` for the operator to export as `TRUSTED_SERVER_CONFIG_TRUSTED_SERVER_CONFIG_TRUSTED_SERVER_CONFIG`, with each secret as `TRUSTED_SERVER_SECRET_TRUSTED_SERVER_SECRETS_`. From row 4 that blob carries the policy's name and SHA-256 without the file, and Axum reads the file when it starts from the folder that `TRUSTED_SERVER_PERMISSIONS_DIR` names, as `/.yaml`, so the folder must hold the bytes that were pushed. Cloudflare and Spin also read one blob each, and from row 4 the policy travels inside that blob, because only the Fastly adapter chooses a blob by host. Spin reads its blob from the `default` key-value store. From row 4 Cloudflare reads its blob from the Workers KV namespace that `ts config push` writes, in place of the Worker variable `TRUSTED_SERVER_CONFIG`, which, as section 3.1 of `2026-07-30-permission-model-design.md` sets out, cannot hold the 24,231-byte sample, and a Worker without that binding, or with no blob in the namespace, answers `500`. IABTechLab/trusted-server#1435 says that "Cloudflare, Spin, Windows and Linux ARM64 CLI releases follow separately", so an operator of Cloudflare or Spin builds from source today, and that build is not the gold WASM. Staging, `ts healthcheck`, `ts active-version` and `ts rollback` are Fastly's alone. + +## 4. Journey 3. The release manager builds and signs the gold WASM + +This journey is Tech Lab's own, in IABTechLab/trusted-server#1262, a draft pull request, and IABTechLab/trusted-server#1435. Tech Lab cuts release candidate branches named for the month, such as `rc/202608` and `rc/202610`, and on 21 October 2026 published the pre-releases `v0.1.0-rc.202609.1` from main and `v0.1.1-rc.202610.1` from `rc/202610`, neither with binaries. IABTechLab/trusted-server#1435 still lists whether a tag on an rc branch may make a release as a decision for its maintainers, "Decide rc-branch tag eligibility before official use". The steps follow IABTechLab/trusted-server#1435 at `a501d1ba6`, whose workflows build Tech Lab's main, and say where the stack differs. + +1. **Decide which modules the build links.** On main, which IABTechLab/trusted-server#1435 builds, the integrations are in core, and core's `builders()` in `crates/trusted-server-core/src/integrations/mod.rs` lists them, so main has no `shipped_module_builders()`. The stack moves each vendor integration out of core into a module of its own (row 7), and from row 43 lists the builders in each adapter's `shipped_module_builders()` and in the tool's `linked_builders()`. A gold WASM built from the stack links the stock list at the tagged commit, `shipped_module_builders()` in `crates/trusted-server-adapter-fastly/src/app.rs`, and the `ts` binaries link `linked_builders()` in `crates/trusted-server-cli/src/app_config.rs`. The two lists hold the same builders in the same order at row 55, and nothing in the build compares them. A module that is not on the list cannot run on the gold WASM (journey 1, step 11). +2. **Confirm the repository settings.** Before the first production tag, a maintainer enables immutable releases, adds a ruleset that restricts creating, updating and deleting `v*` tags with only the release maintainers' team able to bypass it, and sets the repository variable `RELEASE_PUBLICATION_ENABLED=true`. `gh api "repos/IABTechLab/trusted-server/immutable-releases"` answers `enabled: true` once immutable releases are on and HTTP 404 while they are off, and `gh api "repos/IABTechLab/trusted-server/rulesets?includes_parents=true"` lists the rulesets. Without the variable, signing and publication are skipped. IABTechLab/trusted-server#1435 lists these settings as item D7 of IABTechLab/trusted-server#1235, which is still open. +3. **Rehearse in a repository of its own.** The reviewed commit is rehearsed in a separate public repository with Actions and immutable releases enabled, with a fresh lightweight tag `v0.0.0-rehearsal.N` for each attempt, and no rehearsal tag or release reaches Tech Lab's repository. IABTechLab/trusted-server#1435 records a passing rehearsal at `149754d35`. +4. **Tag the release.** A `v*` tag starts `.github/workflows/release.yml`, which calls `.github/workflows/release-build.yml` at the same commit to build `trusted-server-fastly.wasm`, `ts-x86_64-unknown-linux-gnu`, `ts-aarch64-apple-darwin`, `ts-x86_64-apple-darwin` and three `trusted-server-axum-` binaries, with an SBOM in SPDX 2.3, provenance in the format of SLSA, a framework for recording how a binary was built, and a signed attestation of the SBOM for each, and publishes them as one immutable release of 29 assets with `SHA256SUMS`. The build passes the full 40-character source commit as `TRUSTED_SERVER__GIT_VERSION`, and a tag containing `-` publishes a prerelease. Success is the published immutable release. Publication refuses a tag that already has a release, draft or not. A failure after publication leaves the release as it stands, the release manager checks that release with the read-only commands in the release verification guide, and a fresh release takes a fresh tag. +5. **Check what the build carries.** Section 3.6 of `2026-07-30-permission-model-design.md`, "What the build carries", lists what a release binary of the stack carries once row 4 lands, being the code, the browser scripts, on Cloudflare the Worker glue, the config store's default name from `edgezero.toml`, the git version and the template build digest. It says that no configuration and no policy is among them, and no key once row 12's removal lands. The design in IABTechLab/trusted-server#1435 records the compile-time inputs of a release from an audit against main at `f9281917a`, and the CI of IABTechLab/trusted-server#1435 scans every binary for planted fictional configuration values. Against that audit the stack adds inputs, this update takes some of them out again, and the stack and IABTechLab/trusted-server#1435 change three of the same files. + - It adds the module code, with the page scripts some modules compile in, such as InMobi's `crates/cmp/inmobi/src/tcf_stub.js` and `gpp_stub.js`, and moves others, such as the GPT bootstrap, now `crates/ad-tag/google/src/gpt_bootstrap.js`. It adds the browser scripts in each module's `js/src//`, which the browser build reads with `crates/tsconfig.json` (row 55) and which row 56 moves into the folders of `crates/trusted-server-integrations`. It also adds `crates/trusted-server-adapter-fastly/build.rs`, which lists core's sources for a test and puts nothing into a release binary. + - This update takes out what the stack had added beyond that, being the permission policy compiled in from `config/permissions/sample.yaml` (row 4), and the attestation key schedule and build identity compiled in from `TRUSTED_SERVER_ATTESTATION_KEYS`, `SOURCE_DATE_EPOCH`, `TRUSTED_SERVER_COMMIT`, `TRUSTED_SERVER_BUILD_RUN` and `TRUSTED_SERVER_BUILD_TAG` (row 12). It also takes out of every stock build the 51Degrees loading cache, with its `fiftyone-caching` dependency (rows 24 and 43), and the publisher asset cache, which row 34 as pushed adds to core in `asset_cache.rs` and `source_cache.rs`, two files main does not have, and which this update moves into a module that no stock build links, leaving core's `asset_cache.rs` with the interface alone. So the gold WASM keeps no copy of a publisher's files, and refuses, when its registry is built, an `[asset-cache]` section that selects the module. + - The stack and IABTechLab/trusted-server#1435 meet in three files. Each adds `crates/trusted-server-adapter-fastly/build.rs` and each changes that adapter's `src/main.rs`, and each changes core's `build.rs`, where the stack's change at row 55 is row 12's attestation inputs, which this update takes out. The build scripts of IABTechLab/trusted-server#1435 compile the Cargo target into the Fastly and CLI binaries as `TRUSTED_SERVER__BUILD_TARGET`, and read `TRUSTED_SERVER__RELEASE_BUILD`. So the list of section 3.6 covers the stack's files alone, and a build of both carries those inputs as well, none of them configuration. +6. **Verify the release and keep the record.** The commands of journey 2, step 1, run against the published release, and the release manager keeps their outputs, the reviewed source identity, the bundles and the checksums with the release record, as the release verification guide asks. Success is every check exiting 0. + +## 5. To be measured + +The results and times in the table below are not measured yet. A fresh run of journeys 1 and 2, on a clean checkout of the top of the stack once this update is built, fills in each cell and records the commit it ran on, and until then no cell in the table is a result. Steps that need a Fastly service or a release with binaries run once both exist, and the other steps run on the host and under Viceroy. + +| Step | What is run | Runs on | Result | Time | +| ---------------------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------- | -------------- | -------------- | +| 1.1 Open an issue, then write the design | The issue, and a design in `docs/superpowers/specs/` | The host | To be measured | To be measured | +| 1.2 Make the module's folder and name it | `every_folder_names_its_modules_after_itself` | The host | To be measured | To be measured | +| 1.3 Say who maintains it | `every_folder_declares_its_maintainers` | The host | To be measured | To be measured | +| 1.4 Add its dependencies | `crates/trusted-server-integrations/Cargo.toml` | The host | To be measured | To be measured | +| 1.5 Write the builder and its settings | `mod.rs` and its tests | The host | To be measured | To be measured | +| 1.6 Serve its routes | The adapters' route tests | The host and Viceroy | To be measured | To be measured | +| 1.7 Declare the permissions it needs | `required_permissions()` and its tests | The host | To be measured | To be measured | +| 1.8 Declare the caches it loads through | `.with_cache_features(...)` and its tests | The host | To be measured | To be measured | +| 1.9 Supply its page changes | `crates/trusted-server-adapter-axum/tests/shipped_modules.rs` | The host | To be measured | To be measured | +| 1.10 Add its browser script | `npm run build` and `npm test` | The host | To be measured | To be measured | +| 1.11 Link it into the stock build | The five lists of builders | The host | To be measured | To be measured | +| 1.12 Run it locally against a page | `ts config push --local` and `fastly compute serve` | Viceroy | To be measured | To be measured | +| 1.13 Test it | `cargo test-axum` and `cargo test-fastly` | The host and Viceroy | To be measured | To be measured | +| 1.14 Document it and open the pull request | `ts dev lint domains` and the documentation build | The host | To be measured | To be measured | +| 2.1 Get the release and check it | `gh release verify` and `gh attestation verify` | A release with binaries | To be measured | To be measured | +| 2.2 Create the stores, fill them and link them | `ts provision --adapter fastly` and `fastly service resource-link create` | A Fastly service | To be measured | To be measured | +| 2.3 Write each domain's settings | `ts config init` | The host | To be measured | To be measured | +| 2.4 Write the policy files | `eu.yaml` | The host | To be measured | To be measured | +| 2.5 Validate each domain | `ts config validate --permissions` | The host | To be measured | To be measured | +| 2.6 Rehearse on staging | `ts deploy --staging`, for a source build until AC-8 | A Fastly service | To be measured | To be measured | +| 2.7 Push each domain's settings | `ts config push --key --permissions`, with `--local` under Viceroy | Viceroy, then a Fastly service | To be measured | To be measured | +| 2.8 Deploy the WASM | `fastly compute deploy` | A Fastly service and a release with binaries | To be measured | To be measured | +| 2.9 Check the deployment | `curl` of `/_ts/config.json`, then `ts healthcheck` | Viceroy, then a Fastly service | To be measured | To be measured | +| 2.10 Change a policy, or roll back | `ts config push`, then `ts rollback` | Viceroy, then a Fastly service | To be measured | To be measured | + +Row 56 itself is measured beside row 55, as section 2.10 of `2026-10-26-integrations-crate-design.md` says, with the same tests, the hashes of the 14 bundles, every module name and the time per request, and the results go in the description of row 56's pull request. That design's open question 5 asks whether rebuilding the whole integrations crate after a change to one folder's file is an acceptable cost for a module author, and no figure in this design settles it. + +## 6. Open points for Tech Lab + +The journeys raise seven questions that only Tech Lab can answer. + +1. **How an operator gets the gold WASM.** No signed release with binaries exists yet. The two pre-releases of 21 October 2026 carry no assets, IABTechLab/trusted-server#1435 is open, its item D7, immutable releases and protected `v*` tags, waits on a maintainer, and whether a tag on an rc branch may make a release is still open. The release verification guide asks an operator to obtain the approved tag and its source commit "through an independent source-review process". Where will Tech Lab publish each approved tag and its commit? +2. **When AC-8 ships.** Today the gold WASM reaches Fastly by the manual route that IABTechLab/trusted-server#1262 records as AC-9, which creates no store links and stages nothing. The AC-8 plan in IABTechLab/trusted-server#1435 adds `ts deploy --release` in task E7, for production first, and turns on `ts deploy --release --staging` only after task E8 has qualified production and then staging on disposable services. When does AC-8 ship, and will its staging cover the first deployment of this update, which changes the settings and the build together (journey 2, step 10)? +3. **Where each release lists what it carries.** Each binary of IABTechLab/trusted-server#1435 ships with an SPDX document, `.spdx.json`, and the first phase defers the planned `release-index.json`, whose schema is reserved for a later phase. The list of compile-time inputs is the audit in the design of IABTechLab/trusted-server#1435, made against main at `f9281917a`, and journey 3, step 5, shows how the stack changes it. Will each release publish the list for its own commit, in `release-index.json` or beside it, and who keeps the list current as the stack lands? Should the scan for planted configuration values also look for the text of the files in `config/permissions`, now that row 4 compiles no policy in? +4. **Which modules the gold build links.** A module reaches publishers on the gold WASM only through the stock list in Tech Lab's tree. IABTechLab/trusted-server#1194 and the stack agree on one crate, `trusted-server-integrations`. They differ in how a build finds its modules, being one compile-checked catalog with no registration from outside it in IABTechLab/trusted-server#1194, against each adapter's list and registration through `run_with` in the stack, and in the folders and the browser code (journey 1, step 11). Open question 7 of `2026-10-26-integrations-crate-design.md` asks whether the integrations crate should supply one stock list that the adapters and the tool take, with a deployment that links a different set still writing its own. Which of these will the gold build follow, what does a module need to join its list, for example a maintainers status of `vendor owned`, and will each release publish the list of modules it links? +5. **The caches in the gold build.** With no stock build linking a loading cache (rows 24 and 43), every load through `load_cached` on the gold WASM runs its loader, each robots.txt module's answer included. With the asset cache a module that no stock build links (row 34), the gold WASM fetches every file from its origin, as before row 34, and refuses an `[asset-cache]` section that selects the module. On Fastly the platform's own cache still answers the origin fetch for a file the origin lets that cache hold. Should the gold build link a loading cache module, or the asset cache module, and if so which? +6. **Cloudflare, Spin and the other platforms.** IABTechLab/trusted-server#1435 says that "Cloudflare, Spin, Windows and Linux ARM64 CLI releases follow separately", so operators of Cloudflare and Spin build from source today. When will those have a signed release? +7. **What the release workflow watches.** `.github/workflows/release.yml` in IABTechLab/trusted-server#1435 runs on a pull request only when a path in its filter changes. The filter names no module crate under `crates///`, no `crates/trusted-server-integrations/**`, no `crates/tsconfig.json` and no `trusted-server.example.toml`, which its design lists as embedded in the CLI, so a pull request that changes only one of those runs none of the release checks. Should the filter name them? + +## Revision record + +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-26 | First draft. | +| 2026-10-26 | Fixes from review round 1. The two pre-releases of 21 October and the rc branches, AC-8 and AC-9, staging and the first deployment through versioned physical stores, the store links the manual route needs, the module folders of row 56, IABTechLab/trusted-server#1194, the build and test costs measured on row 56 itself, the issue and the pull request template, a local run under Viceroy, the route traps, the permission signal list, the statuses of startup errors, the pins of main and EdgeZero `v0.0.10`, the files the stack and IABTechLab/trusted-server#1435 both change, the release checks and the prerequisites, the policy entries on every adapter, and links pinned to commits. | +| 2026-10-26 | Fixes from review round 2. Rows are numbered as IABTechLab/trusted-server#777 numbers them, so the permission model is row 4. A folder gets a Cargo feature only for a dependency that not every build can or should carry, or for a module no stock build links. The local page check sets the origin and the hosts in the file, because the overlay only overrides keys that exist, and leaves out `--rewrite-host`. The table of changes says which were agreed at the Touchbase and which 51Degrees decided after the Touchbase. Each startup status is the status of the error's kind, `500` for a configuration or settings error and `503` for a key-value store error. The permission model, integrations crate, asset cache and module endpoints designs are followed where they changed, the file is named for 26 October, and the service and its domains, the secrets, the Edge Cookie store, the operator's one build, the versioned store commands, the clone caution, the two signer checks, main's list of builders and the asset cache in the gold build are stated. | +| 2026-10-26 | Fixes from review round 3. The configuration view never shows a policy's text. The local page check says plainly what it does and keeps the overlay out with `--no-env` or a shell with no `TRUSTED_SERVER__` variable. No stock build turns on the loading cache or asset cache feature, and a feature never gates a folder's source label. Row 32 joins the loading cache change. The operator journey adds the first release it can use, the proxy secret and the other keys, the template's placeholders, where the reason is logged, the guide for writing a policy, the operator's own build workflow, the versioned store route for an operator coming from main or from nothing, the folder every `ts` command runs from, and DNS and certificates. Blob, request preparer, SBOM, provenance, chunks, SPDX, SLSA and stock build are defined, and IABTechLab/trusted-server#1055 is said to cover the CLI's output and its guidance on adapters and tools. | + +## Review log From d619bd42f15d39ef5cc9c1ff93c8f008488a5033 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:28 +0000 Subject: [PATCH 172/192] Review logs record each round of these designs The module endpoints, administration, diagnostic headers, request filters, Fastly names and developer experience designs end with a review log entry for each round they took, as CONTRIBUTING.md asks. The module endpoints design's revision record no longer says that its review log is removed. Item 1 of the config by host design's log takes US spelling and writes its count as a word. --- ...10-12-administration-off-the-publisher-domain-design.md | 2 ++ docs/superpowers/specs/2026-10-12-config-by-host-design.md | 2 +- .../specs/2026-10-12-diagnostic-headers-design.md | 2 ++ .../specs/2026-10-15-fastly-names-in-core-design.md | 2 ++ .../2026-10-17-request-filters-on-every-adapter-design.md | 2 ++ .../specs/2026-10-26-developer-experience-design.md | 4 ++++ .../specs/2026-10-26-module-endpoints-design.md | 7 ++++++- 7 files changed, 19 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md index 27651fa20e..617ef78742 100644 --- a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md +++ b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md @@ -141,3 +141,5 @@ One question is open, about the guides, and this spec does not decide it. 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/13-removals` at `f735e0f85`. Issues found, four of them blocking. The four were that the table named `GET /_ts/data` as the replacement for `GET /_ts/admin/eids` when that page reads only the record of the caller's own `ts-ec` cookie, that the upgrade note said a `[[handlers]]` entry could gate `GET /_ts/debug/ja4` when the Fastly entry point answers that path before any middleware, that the spec said the commands tell a `404` from other failures without saying that only the test fake is known to print the wording they look for, and that the header named closed pull requests and a count of 19 checks where the open pull request shows 25. This revision states what the code does at that head for the table, the commands and their flags, the closed paths, the effects on a publisher's own site, the upgrade, the switch and the rollback, the tests and the acceptance, and leaves seven questions about the code under Open questions until the code changes, being the wording of a `404`, the replacing of a secret, the attestation endpoint beneath `/_ts/admin`, the missing tests, four stale texts, the order of the preparers on Axum and the order of the switch. 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/13-removals` at `f735e0f85`. Approved, none blocking. This revision adds the changelog entry that still says only the removed key rotation routes read the two store ids to open question 5, says that the sandbox answer in the Fastly entry point exists only in a build with the `reusable-sandbox` feature, and confirms the check counts of the row 13 pull request and the fork's. +3. **Round 3.** A review in a fresh session, against `split/13-removals`, of the changes that follow attestation leaving row 12. Issues found, one of them blocking. Section 5 still listed a test of the attestation endpoint beneath `/_ts/admin`, and the test leaves this row with that endpoint. The round also found, as part of a blocking point on the module endpoints design, that the guide section that design plans would go out of date as this row changes the steps before a module route, which that design settles by having the section state only what holds at every row, so this design needs no note about that section. This revision drops the test, says in section 2 that row 12 adds no endpoint the settings choose and that a module route beneath a closed path is never reached, and names by function the two checks of row 12 that section 4 cites, where it gave line numbers that no longer held. +4. **Round 4.** A review in a fresh session, against `split/13-removals`, of the round 3 fixes and the new text. Issues found, none blocking. This revision says that the commit that refused an endpoint the settings chose beneath `/_ts/admin`, and the later commit that fixed the Cloudflare test of that endpoint's routes, stay in this row's history while the merge of row 12's removal takes out what each changed, and says beside the two checks of row 12 that a row 12 build requires `config_store_id` and `secret_store_id` in `[request_signing]`. diff --git a/docs/superpowers/specs/2026-10-12-config-by-host-design.md b/docs/superpowers/specs/2026-10-12-config-by-host-design.md index fd5c2b65fc..ee9d547d81 100644 --- a/docs/superpowers/specs/2026-10-12-config-by-host-design.md +++ b/docs/superpowers/specs/2026-10-12-config-by-host-design.md @@ -183,5 +183,5 @@ The behaviours below are stated in the sections above. A change to the code woul ## Review log -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/11-config-by-host` at `b8314cb52`. Issues found, 3 of them blocking. The header pointed at closed pull requests and called the fork pull request ready for review, the risk table put three header cells over two-cell rows, and the text said there is no fallback and that a selector that cannot be read fails the request, which is false when the runtime-env store cannot be opened. This revision points the header at the open draft pull request, repairs the table, states the fallback and the removal it needs, and adds the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests, whilst the code behaviours the review found wait for code changes and sit under Open questions. +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/11-config-by-host` at `b8314cb52`. Issues found, three of them blocking. The header pointed at closed pull requests and called the fork pull request ready for review, the risk table put three header cells over two-cell rows, and the text said there is no fallback and that a selector that cannot be read fails the request, which is false when the runtime-env store cannot be opened. This revision points the header at the open draft pull request, repairs the table, states the fallback and the removal it needs, and adds the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests, while the code behaviors the review found wait for code changes and sit under Open questions. 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/11-config-by-host` at `b8314cb52`. Approved, none blocking. This revision names, at the pointer and where this spec describes them, the sections of `2026-10-15-fastly-per-host-selector-design.md` that replace the fallback text (section 2.4) and the push and selector-writing steps (sections 2.6 and 2.8). diff --git a/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md b/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md index 2ea4bdf735..9855326a7b 100644 --- a/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md +++ b/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md @@ -188,3 +188,5 @@ The fork's CI passed on the series head, `a43a05a33`, all nineteen checks, and t 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/15-diagnostic-headers` at `a43a05a33`. Issues found, four of them blocking. The four were that the spec covered the three commits of 9 October and none of the five since, that the spec left `X-TS-Version` as it was and left out the breaking change the pull request lists for it, that the rule for paths beneath `/_ts` was wider than the code for an origin's answer while the inspection pages read as if they showed headers nothing writes, and that the acceptance cited a record the pull request descriptions do not hold. This revision states what the code does at that head for all eight commits, the paths beneath `/_ts`, the adapters, the version headers and the `service_version` middleware, the defaults, the rollout, what breaks, the failure and privacy answers, the tests and the acceptance, and lists under open questions what waits for code changes, being the exemption, the list of names, the headers still sent, an origin's version headers, the untested paths and the documents. 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/15-diagnostic-headers` at `a43a05a33`. Issues found, none blocking. This revision corrects the acceptance, which said the pull request descriptions hold no record of a run of the whole check on the head although the description of the pull request for this row (IABTechLab/trusted-server#1356) has every check command ticked, and it qualifies three clauses that were only partly true, being what a stored page replays from the origin, what the proxies take off a response and which header the prefix test sends. +3. **Round 3.** A review in a fresh session, against `split/15-diagnostic-headers`, of the changes that follow attestation leaving row 12. Issues found, none blocking. The design said that what describes a deployment is answered where the deployment is asked about itself, which stops holding once row 12 no longer adds the signed document that reported the build and the service's version, because no page beneath `/_ts` then gives either version, and the design left out `fastly_platform_identity`, which this row adds with `PlatformIdentity`. This revision says that the git version and the service's version appear only on the HTML pages a deployment opts into through `service_version`, that the inspection pages give the core crate's version, and that Fastly reports its platform through `fastly_platform_identity`. +4. **Round 4.** A review in a fresh session, against `split/15-diagnostic-headers`, of the round 3 fixes and the new text. Issues found, none blocking. This revision says that the commit table and its counts are those of the series head, adds a row for the commit that keeps `PlatformIdentity` in core's platform module, and says that each adapter reports its platform when it builds its state. diff --git a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md index 682d57b38f..b013715b6a 100644 --- a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md +++ b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md @@ -185,3 +185,5 @@ Not tested: 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/49-host-features` at `69ba03219`. Issues found, two of them blocking. The page left out the change to `x-debug-fastly-pop` on the Axum, Cloudflare and Spin adapters, and described the spoofable header list as stripping headers although the one adapter that calls the function reading it declares nothing. This revision states both as the code has them today, adds the effects on the other adapters, the scheme change, where each refusal happens, the order for changing the package and the document, the name that stays and the untested paths, and corrects the partly true claims, whereas the two code fixes wait on the open questions in section 2.3. 2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `b319bf039`, of the round 1 fixes and the new text. No blocking issues. This revision names in the Fastly sentence of 2.3 that `ts config validate` no longer refuses `x-debug-fastly-pop`, corrects when the Axum, Cloudflare and Spin adapters strip `fastly-client-ip`, lists the comment at the top of `platform/request_headers.rs` among the comments left for a later change, and corrects the changelog entry on the spoofable strip while adding the scheme change and the `x-debug-fastly-pop` change on Fastly to it. +3. **Round 3.** A review in a fresh session, against `split/49-host-features`, of the changes that follow attestation leaving row 12. No issues found. The round ran the guard's own matcher on core at the commit before this row's guard commits and found 173 lines, one of them the `_vercel` entry of `ORIGIN_ASSET_PREFIXES`, which row 12 no longer adds, so the count of 172 lines before this change, all of them comments, documents or test code, holds. This revision also stops calling the pull request a draft, as IABTechLab/trusted-server#1399 is not one. +4. **Round 4.** A review in a fresh session, checking the round 3 fixes. No issues found. The round found the header right, because IABTechLab/trusted-server#1399 is not a draft, and found nothing else to change in this design. diff --git a/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md index 53eda0d482..7973cfac6d 100644 --- a/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md @@ -129,3 +129,5 @@ The thirteen tests pass on each of the three adapters, taking out any of the thr 1. **Round 1, 2026-10-18.** A review in a fresh session, against `split/52-request-filters` at `f0edcd72b`. Issues found, five of them blocking. The five were that the spec said a selected device module did nothing on Axum, Cloudflare and Spin, when core asks it for a bid request's device attributes in every auction there, that nothing said what DataDome's filter sends DataDome or sets, or that no permission gates it, that the client hints design still said only the Fastly adapter runs request filters, that no test proved the privacy guards written after a filter's headers, and that the spec said the Fastly adapter answers two of the inspection endpoints before the preparers, when it answers all three, and that the adapters' HTTP clients cannot bound a call, when the Axum and Cloudflare clients can. Fixed in `7aea7a72b`, which adds the four privacy tests and four more on each adapter, `4549047dc`, which corrects the adapters' comments on robots.txt, `a584b071f`, which corrects the guides, the changelog and the middleware readme, `6b4eb0b8c`, which amends the client hints design, and this revision, which states each from the code and gives the reason a time limit is left to a change of its own. One finding was not borne out. The review said a `Cookie` header holding an accented letter now answers `400` on the routes that newly read the context, but a client's header is rebuilt by the `ad-tag.google.diagnostics` preparer first, which the context test shows, so Rollout names no new `400`. 2. **Round 2, 2026-10-18.** A review in a fresh session, against `split/52-request-filters` at `17136d04b`, of the round 1 fixes and the new text. Issues found, one of them blocking. The one was that no test sent a named route through the request cycle, so taking `build_ec_context` or `run_pre_route_filters` out of any adapter's `execute_named` failed no test, although on Cloudflare and Spin this change replaced each named route's own handler with `execute_named`. Fixed in `7ae3235ad`, which adds a test on each adapter that sends every named route in the cycle to a filter that answers and has the context test's preparer spoil the discovery route as well, so that taking either step, or on Spin the preparers, out of `execute_named` fails a test, `8c9b914c7`, which corrects the `build_ec_context` comments on the `Cookie` check, `233299a06`, which names in the integration guide the IP address and the request's address a device module is shown, `52cfd9fe9`, which amends the geo module request design's table of finalize steps and its line on `detect`, `e7417c898` and `ac0492932`, which fix the client hints design's wait on Axum and its test item 6 and log its round, and this revision, which names the tests each removal fails, the IP address and the request's address, the subresources a device module classifies, what the diagnostics preparer drops, the two DataDome designs whose scope this change lifts and what the guards do with a platform's own cache header, on Fastly as on Cloudflare, with an open question on it, and gives a true reason no shipped filter is tested through an adapter. +3. **Round 3.** A review in a fresh session, against `split/52-request-filters`, of the changes that follow attestation leaving row 12. Issues found, none blocking. The round found the renamed inspection test, `the_inspection_endpoints_skip_the_request_filters`, right, and found that the header still said the pull request was not yet raised and named no related pull request, although IABTechLab/trusted-server#1410 is open. It also found, as part of a blocking point on the module endpoints design, that the guide section that design plans would go out of date as this row changes the steps before a module route, which that design settles by having the section state only what holds at every row, so this design needs no note about that section. This revision names IABTechLab/trusted-server#1410, row 52 of the stack, in the header. +4. **Round 4.** A review in a fresh session, checking the round 3 fixes. No issues found. The round found the header right, because IABTechLab/trusted-server#1410 is open and not a draft, and found nothing else to change in this design. diff --git a/docs/superpowers/specs/2026-10-26-developer-experience-design.md b/docs/superpowers/specs/2026-10-26-developer-experience-design.md index d157f73a28..763843c4f9 100644 --- a/docs/superpowers/specs/2026-10-26-developer-experience-design.md +++ b/docs/superpowers/specs/2026-10-26-developer-experience-design.md @@ -189,3 +189,7 @@ The journeys raise seven questions that only Tech Lab can answer. | 2026-10-26 | Fixes from review round 3. The configuration view never shows a policy's text. The local page check says plainly what it does and keeps the overlay out with `--no-env` or a shell with no `TRUSTED_SERVER__` variable. No stock build turns on the loading cache or asset cache feature, and a feature never gates a folder's source label. Row 32 joins the loading cache change. The operator journey adds the first release it can use, the proxy secret and the other keys, the template's placeholders, where the reason is logged, the guide for writing a policy, the operator's own build workflow, the versioned store route for an operator coming from main or from nothing, the folder every `ts` command runs from, and DNS and certificates. Blob, request preparer, SBOM, provenance, chunks, SPDX, SLSA and stock build are defined, and IABTechLab/trusted-server#1055 is said to cover the CLI's output and its guidance on adapters and tools. | ## Review log + +1. **Round 1.** A review in a fresh session, against `split/55-scripts-in-crates`, Tech Lab's main and IABTechLab/trusted-server#1435. Issues found, nine of them blocking. The nine were that the design said no release had been published, when Tech Lab had published two pre-releases, that an open point asked Tech Lab for what Tech Lab had already planned as AC-8 and AC-9, and wrongly tied `ts healthcheck` and `ts rollback` to one deployment route, that an operator of the gold WASM could not follow its staging steps, that on the manual route nothing linked the operator's stores, so every request would answer `500`, that its account of row 56 contradicted the integrations crate design, that it never mentioned Tech Lab's split integrations design in IABTechLab/trusted-server#1194, that a table of build and test costs had no record a reader could check, that the module author's journey skipped the issue that Tech Lab's contributor guide asks for first, and that the journey never ran the module. This revision corrects the design on all nine points. It describes the two pre-releases, asks when AC-8 ships, says there is no staging for the gold WASM until then and gives a first deployment through versioned physical stores and the store links the manual route needs, follows the integrations crate design for the module folders, compares the stack with IABTechLab/trusted-server#1194, leaves every cost to a fresh run, and opens the module author's journey with the issue and runs the module under Viceroy against a page. +2. **Round 2.** A review in a fresh session, against `split/55-scripts-in-crates`, of the round 1 fixes and the new text. Issues found, three of them blocking. The three were that the design gave a Cargo feature to every folder with a dependency of its own, where the integrations crate design gives one only for a dependency that not every build can or should carry, or for a module that no stock build links, that the local page check could not pass, because the copied template's origin is fictional and EdgeZero's environment overlay only overrides keys the file already holds, and that the design said Tech Lab agreed changes that 51Degrees decided after the Touchbase. This revision follows the integrations crate design's rule for features, has the local page check set the origin and the hosts in the file, and says in its table of changes which were agreed at the Touchbase and which 51Degrees decided after it. It also gives each startup status as the status of the error's kind, and fills the gaps an outside operator met in the operator journey, from creating the service and its secrets to the commands of the versioned store route. +3. **Round 3.** A review in a fresh session, of the round 2 fixes and the new text. Issues found, one of them blocking. The table of changes said that `/_ts/config` shows the policy's text once row 4 is merged into row 9, when the configuration view never shows it. This revision says that the view shows only the policy's name and SHA-256, has the local page check push with `--no-env`, so that the environment overlay cannot replace the settings in the file, no longer cites the getting-started guide, whose environment files still use the old name of the origin variable, says that no stock build turns on the loading cache or asset cache feature, and adds row 32 to the loading cache change. It also answers nine questions an outside operator would still ask, from the first release they can use to DNS and certificates, defines terms such as blob, SBOM and stock build, and says what IABTechLab/trusted-server#1055 covers. diff --git a/docs/superpowers/specs/2026-10-26-module-endpoints-design.md b/docs/superpowers/specs/2026-10-26-module-endpoints-design.md index af5f65c27f..c4764ef80b 100644 --- a/docs/superpowers/specs/2026-10-26-module-endpoints-design.md +++ b/docs/superpowers/specs/2026-10-26-module-endpoints-design.md @@ -89,4 +89,9 @@ Another method on a path of item 1 reaches the fallback as any other request doe | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-26 | First draft. | | 2026-10-26 | Fixes from review round 1. What each adapter does at startup when two routes clash, and the other routes matchit refuses, the shipped modules that declare paths outside their prefix, what a module call carries at this row and from row 39 and what a gated value needs, where each adapter classifies the device and when the Edge Cookie step answers, the requests a module route never sees, the request filters and DataDome, the guide section and the two places it amends, the probe's handler and counter with five tests, the platform identity and the header. | -| 2026-10-26 | Fixes from review round 2. What a refusal does on the stock entry points and on a deployment's own, with a clash from settings alone, two routes on one method and path in place of two modules on one path, every function that takes outside builders, when each adapter builds its state, the plan registration, the shipped modules' paths outside `/integrations/`, the steps that keep a request from a module route and the rows that change them, and `HEAD` bodies. The empty review log is removed, and this record carries the review rounds. | +| 2026-10-26 | Fixes from review round 2. What a refusal does on the stock entry points and on a deployment's own, with a clash from settings alone, two routes on one method and path in place of two modules on one path, every function that takes outside builders, when each adapter builds its state, the plan registration, the shipped modules' paths outside `/integrations/`, the steps that keep a request from a module route and the rows that change them, and `HEAD` bodies. | + +## Review log + +1. **Round 1.** A review in a fresh session, against `split/12-attestation`. Issues found, seven of them blocking. The seven were that the design said an adapter whose registry is refused serves its startup error, when on Axum, Cloudflare and Spin a deployment's own builders arrive through `routes_with_registrations`, which hands the error back to the deployment's code, that it said a module call carries the request's evidence, which only row 39 adds, that the steps before a module route left out device classification and did not say that the Edge Cookie step can answer in the route's place, that it said a bot filter can answer for a module's endpoint without saying that the stack's own DataDome filter lets every module route through, that the new guide section would contradict the two places in the guide that tie a module's routes to its prefix, that the section would go out of date as later rows change those steps, and that the planned `POST` check could not be built where the design put it. This revision says what each adapter does when two routes clash, what a module call carries at this row and from row 39, where each adapter classifies the device and when the Edge Cookie step answers, which requests a module route never sees, and that every request filter is told when a module route takes the request. The guide section now states only what holds at every row and amends those two places, and the seam probe gains a handler that counts its runs, so the `POST` check asserts that the handler never ran. +2. **Round 2.** A review in a fresh session, against `split/12-attestation`, of the round 1 fixes and the new text. Issues found, two of them blocking. The two were that the design said the stock `Hooks::routes()` takes no builders from outside core, when the stock list holds builders from crates outside core, such as Prebid and Didomi, and did not say what a clash does there, and that the guide section said two modules on one path are refused, when the registry keeps one router for each method. This revision says that the stock entry points compose the shipped modules' builders and answer every request with the startup error when the registry is refused, gives a clash that a stock build reaches from its settings alone, at `/prebid.js`, and says that two routes on one method and path are refused. It also lists every function through which a deployment's own builders arrive, says when each adapter builds its state, and says that nothing in core, the adapters or EdgeZero strips the body from a `HEAD` answer. From 0762b5877e824e57db75b1921fb9393e056fa42c Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:29 +0000 Subject: [PATCH 173/192] Stock builds link no cache, one crate holds modules The loading cache design says that no stock build links a loading cache module, so a stock build loads through NoLoadingCache, and the 51Degrees module is offered only to a deployment that links it. The asset cache design moves the asset cache out of core into the module crates/asset-cache/51degrees, behind one interface in core, and no stock build links that module either. The integrations crate design folds every module crate into crates/trusted-server-integrations, with a folder for each type and vendor, in a new row 56 at the top of the stack. The module cache declarations design says that a build linking one loading cache module holds a declared cache's answers. --- .../specs/2026-10-12-loading-cache-design.md | 187 ++++--- .../specs/2026-10-15-asset-cache-design.md | 509 +++++++++++++----- ...-10-15-module-cache-declarations-design.md | 5 +- .../2026-10-26-integrations-crate-design.md | 328 +++++++++++ 4 files changed, 827 insertions(+), 202 deletions(-) create mode 100644 docs/superpowers/specs/2026-10-26-integrations-crate-design.md diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index 390d1e013a..6b40eeeaf1 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -1,15 +1,15 @@ # Design Spec: The loading cache core declares, and the module that supplies it -**Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365), which is row 24 of the stack, answering [IABTechLab/trusted-server#1282](https://github.com/IABTechLab/trusted-server/issues/1282). The draft pull request [IABTechLab/trusted-server#1372](https://github.com/IABTechLab/trusted-server/pull/1372), row 31, completes the change by moving the pin on the published caching crate. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Revised 2026-10-26, so that no stock build (an adapter or the `ts` tool built from the repository as it ships, with the builders that its own list, its stock list, names) links a loading cache module (section 2, "No stock build links a cache"). Implemented as the pull request [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365), which is row 24 of the stack, answering [IABTechLab/trusted-server#1282](https://github.com/IABTechLab/trusted-server/issues/1282). The pull request [IABTechLab/trusted-server#1372](https://github.com/IABTechLab/trusted-server/pull/1372), row 31, completes the change by moving the pin on the published caching crate. The revision of 2026-10-26 describes a change to row 24, to row 32 ([IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373)) and to row 43 ([IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384)), where the stock list moved into each adapter. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-robots-txt-from-modules-design.md`, `2026-10-12-robots-txt-middleware-design.md`, `2026-10-12-prefix-module-design.md`, `2026-10-15-module-cache-declarations-design.md`, `2026-10-15-asset-cache-design.md` -**Related PRs:** [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24) was reviewed on [jwrosewell/trusted-server#19](https://github.com/jwrosewell/trusted-server/pull/19) (closed) and [IABTechLab/trusted-server#1372](https://github.com/IABTechLab/trusted-server/pull/1372) (row 31) on [jwrosewell/trusted-server#26](https://github.com/jwrosewell/trusted-server/pull/26) (closed). The integration seam is in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348), robots.txt rules from modules in [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355), robots.txt as a middleware in [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366), the prefix module in [IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), module-declared caches in [IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373) and the asset cache in [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375) +**Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-robots-txt-from-modules-design.md`, `2026-10-12-robots-txt-middleware-design.md`, `2026-10-12-prefix-module-design.md`, `2026-10-14-no-modules-crate-design.md`, `2026-10-15-module-cache-declarations-design.md`, `2026-10-15-asset-cache-design.md`, `2026-10-26-integrations-crate-design.md` +**Related PRs:** [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24) was reviewed on [jwrosewell/trusted-server#19](https://github.com/jwrosewell/trusted-server/pull/19) (closed) and [IABTechLab/trusted-server#1372](https://github.com/IABTechLab/trusted-server/pull/1372) (row 31) on [jwrosewell/trusted-server#26](https://github.com/jwrosewell/trusted-server/pull/26) (closed). The integration seam is in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348), robots.txt rules from modules in [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355), robots.txt as a middleware in [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366), the prefix module in [IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), module-declared caches in [IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373), the asset cache in [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375) and each adapter's own list of the modules it links in [IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384) **Issues:** [IABTechLab/trusted-server#1282](https://github.com/IABTechLab/trusted-server/issues/1282). [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-17 +**Last updated:** 2026-10-26 > **Why this spec exists.** Each robots.txt module's answer is read from the key-value store by every request and asked of the module's service by every request that finds no fresh copy, so a burst of requests for one file asks the service once for each request, on every instance. Core has one cache, for page templates, and no interface through which a module can supply a cache. -Line numbers in this document are those of row 31 (`4b6ef9c05`) unless a row is named. The fixes of 2026-10-16 are on row 24 and reach row 31 when it is merged forward, so where this document describes them it names items rather than lines. +Line numbers in this document are those of row 31 at `490a4665d`, the head of IABTechLab/trusted-server#1372, unless a row or a commit is named. Row 24's head, `e66b955a4`, is in row 31's history, so row 31 has every fix that row 24 has. The change to rows 24, 32 and 43 that takes the 51Degrees module out of the stock lists is written here ahead of its code, so where this document describes it, it names files and items, and the line numbers it gives are those of the code before the change. ## 1. Problem @@ -19,9 +19,9 @@ Core had one cache, for page templates (`crates/trusted-server-core/src/platform ## 2. Design -**The interface.** Core declares `PlatformLoadingCache` in `trusted_server_core::platform` (`crates/trusted-server-core/src/platform/loading_cache.rs`). Its `get_or_load` takes `RuntimeServices`, a `CacheSpec`, a `StoreKey` and a `Loader`, and answers the value or a `LoadError`. A missing value is loaded once per key, every caller that asks while that load runs gets its value or its error, and a failed load is stored nowhere. `remove` drops a key from the memory copy and from the shared store. `keeps_values` says whether the cache keeps what it loads, and no code outside tests calls it on rows 24 and 31. A feature loads through the cache with `RuntimeServices::load_cached`, which asks the cache that the spec's name selects and hands it the request's services without the reader, as section 5 describes. `NoLoadingCache` holds nothing, runs every load and answers `false` to `keeps_values`. +**The interface.** Core declares `PlatformLoadingCache` in `trusted_server_core::platform` (`crates/trusted-server-core/src/platform/loading_cache.rs`). Its `get_or_load` takes `RuntimeServices`, a `CacheSpec`, a `StoreKey` and a `Loader`, and answers the value or a `LoadError`. A missing value is loaded once per key, every caller that asks while that load runs gets its value or its error, and a failed load is stored nowhere. `remove` drops a key from the memory copy and from the shared store. `keeps_values` says whether the cache keeps what it loads, and no code outside tests calls it on rows 24 and 31. A feature loads through the cache with `RuntimeServices::load_cached`, which asks the cache that the spec's name selects and hands it the request's services without the connection, as section 5 describes. `NoLoadingCache` holds nothing, runs every load and answers `false` to `keeps_values`. -**The spec and the key.** A `CacheSpec` carries the cache's name, three lifetimes (the brief copy in process memory, the copy in the host's shared store, and how long that shared copy may sit unused) and the most entries the memory copy holds. A `StoreKey` is written `//`, where the prefix is the publisher's `ServerKeyPrefix`, so publishers sharing a store never share a copy. Both rows lowercase the publisher's domain and drop one leading `www.`, and neither builds the domain from the raw `Host` header, because a `Host` header can say anything. Row 24 takes the domain from the loaded settings' `publisher.domain` and row 31 from the publisher the request resolved to, as the table at the end of this section shows. +**The spec and the key.** A `CacheSpec` carries the cache's name, three lifetimes (the brief copy in process memory, the copy in the host's shared store, and how long that shared copy may sit unused) and the most entries the memory copy holds. A `StoreKey` is written `//`, where the prefix is the publisher's `ServerKeyPrefix`, so publishers sharing a store never share a copy. Both rows lowercase the publisher's domain and drop one leading `www.`, and neither builds the domain from the raw `Host` header, because a `Host` header can say anything. Row 24 takes the domain from the loaded settings' `publisher.domain` and row 31 from the publisher the request resolved to, as the table at the end of this section shows. The key after the prefix and the cache name is the feature's own, and section 5 says what robots.txt's is made of. **How a feature selects its cache.** Each feature that holds answers this way owns a `[cache.]` table. On rows 24 and 31 core reads one, `[cache.robots-txt]`, because `[cache]` refuses any other key. Its `module` field names a module that offers a loading cache, by its short name (`51degrees`) or in full (`loading-cache.51degrees`). An adapter resolves every table with `loading_caches_from` when it builds its application state (the settings, modules and caches the adapter keeps to serve requests), and makes a module's cache once however many tables select it. @@ -33,42 +33,59 @@ store_seconds = 604800 store_idle_seconds = 2592000 ``` -**When `module` is absent.** The table takes the one loading cache module that the build links (`loading_caches_from` in `crates/trusted-server-core/src/integrations/mod.rs`). The stock list links exactly one, `loading-cache.51degrees`, so a stock build holds each robots.txt module's answer with no `[cache.robots-txt]` table at all, and every line of the example above can be left out because each value shown is the default. A build that links none, or two or more, and writes no `module`, gives the table `NoLoadingCache`, which holds nothing and runs every load. Nothing is logged when that happens, so a second loading cache module added to the stock list would stop the holding without any message, and the test `a_table_naming_no_module_holds_with_the_stock_loading_cache` fails instead. No setting turns holding off in a stock build, because `module` can only name a linked module. +**No stock build links a cache.** Core declares the interface and supplies `NoLoadingCache`. The repository's one loading cache module written for deployments, the 51Degrees module described below, is offered only to a deployment that links it. The only other is the test cache that row 32 gives the seam probe, a fixture crate through which tests drive core's extension points from outside core, and only tests link it. No adapter's stock list and no list of the `ts` tool names the 51Degrees module, so building an adapter or the tool compiles neither the module crate nor `fiftyone-caching`. At the Touchbase with IAB Tech Lab on 2026-10-23, 51Degrees said it would check that the loading cache could be a module that a deployment links. As decided by 51Degrees after the Touchbase, the module stays in the repository and no stock build links it. Before that decision, row 24 put the module's builder in the stock list in `crates/trusted-server-modules/src/lib.rs` (line 52 at `e66b955a4` and line 53 at `490a4665d`, with the dependency at line 33 of that crate's `Cargo.toml`). From row 43 each adapter's `shipped_module_builders()` and the tool's `linked_builders()` carry it, as at row 55 (`5be114773`), where the Fastly adapter's list holds it under the comment "The loading cache a stock build caches with" (`crates/trusted-server-adapter-fastly/src/app.rs` line 477), and the four adapters' and the tool's `Cargo.toml` name the crate. The change takes the builder out of each of those lists and the crate out of each of those manifests, on row 24 and on row 43, rewrites on row 32 the one test that counts the module as linked, and leaves the module crate in the tree. Section 9 names the tests it changes and the guard it puts beside every stock list, the tool's included. -**A module nothing offers.** A `module` that no linked crate offers a cache from is refused by the deploy check, `ts config validate`, and again when the adapter builds its state. Both use one lookup, `named_cache` in `integrations/mod.rs`, whose message names the table and lists the loading cache modules the build has. The deploy check runs it through `check_loading_cache_modules` from `validate_settings_for_deploy_with`. +**Text the change rewrites.** Besides the lists, the manifests and the tests, the change rewrites each place that says a stock build links the module or holds answers, and adds a `CHANGELOG.md` entry saying that a stock build links no loading cache module and refuses `module = "51degrees"`, so a deployment whose document names the module links it first. -**The implementation.** The implementation is the module crate `crates/loading-cache/51degrees` (package `trusted-server-loading-cache-51degrees`, module `loading-cache.51degrees`), which 51Degrees maintains. The module crate is written with the published [`fiftyone-caching`](https://crates.io/crates/fiftyone-caching) crate ([source](https://github.com/51Degrees/rust/tree/main/caching)). The workspace builds the published crate with its default features off, so the published crate's own Fastly, Cloudflare, Spin and tokio modules are not compiled. The loading logic comes from the published crate, and the stores are written in the module crate (`stores.rs`) over core's `PlatformValueCache` and the host's key-value store. The module crate's `builder()` offers the cache through `IntegrationBuilder::with_loading_cache`, the stock list in `crates/trusted-server-modules` is the only code that names the module crate, and each adapter calls `loading_caches_from` with the builders it is given, so no adapter names the module crate. +- The comment in core above the step for a declared cache with no table, which ends "so a stock build holds a module's answers without a line in the document" (`crates/trusted-server-core/src/integrations/mod.rs` lines 940 to 942 on row 32 at `4a1f03c40`, and lines 1084 to 1086 at row 55). +- The section of the configuration guide on loading caches, which says that a stock build links `loading-cache.51degrees` and holds answers without a line in the document (`docs/guide/configuration.md` lines 1833 to 1837), and the commented example in `trusted-server.example.toml` (lines 458 and 459), whose `module = "51degrees"`, uncommented, a stock build refuses. +- The loading cache entry of `CHANGELOG.md` (line 105), which says that the stock list ships the module, so every adapter takes it. +- The description of IABTechLab/trusted-server#1365, which says in its paragraph on selecting a cache, and again under "What breaks", that a stock build holds answers with no line in the document. +- Row 32's design, `2026-10-15-module-cache-declarations-design.md`, whose sentence in section 2.2 saying that a stock build holds a module's answers is corrected with this revision, and whose lines 202, 226 and 235 describe row 32's code with the 51Degrees module in the stock list. +- Row 43's design, `2026-10-14-no-modules-crate-design.md`, whose list of tests (line 84) says that a feature selects the stock loading cache by name. +- The integration guide, `docs/guide/integration-guide.md`, whose lines 864 and 865 say "The stock list carries `crates/loading-cache/51degrees`" (lines 963 and 964 at row 55). +- From row 44, the 51Degrees shared code's `crates/shared/51degrees/README.md` (lines 12 to 14), the module comment of its `src/engine_cache.rs` (lines 3 to 10), and its `src/client.rs`, in the module comment (lines 11 to 13) and on `CloudPipeline` (lines 102 and 103), which say that a question many requests ask at once is asked of the cloud service once. The 51Degrees guide, `docs/guide/integrations/51degrees.md` (lines 94 to 96), says that a stock build holds the service's answers. +- From row 44, the tests of the client hints middleware, `crates/middleware/51degrees/src/lib.rs` at row 55. The helper `stock_builders()` (lines 1573 to 1575) is documented as "The builders a stock build links for the two 51Degrees modules", and the test `the_shared_table_runs_the_middleware_and_the_stock_cache_holds_its_answers` (lines 1613 to 1641) calls the module the stock loading cache in its name, its comment and its message (line 1638). The link at line 1575 stays, because only those tests use it, and the names and the wording change. + +**When `module` is absent.** A table that names no module takes the one loading cache module the build links (`loading_caches_from` in `crates/trusted-server-core/src/integrations/mod.rs`), and from row 32 so does a cache a module declares with no table at all (`2026-10-15-module-cache-declarations-design.md`). A stock build links none, so in a stock build every feature loads through `NoLoadingCache`, which holds nothing and runs every load, and a table that names a module is refused, as the next paragraph says. A build that links one loading cache module has it taken by every table that names no module, so a deployment that links the 51Degrees module holds each robots.txt module's answer with no `[cache.robots-txt]` table, and every line of the example above can be left out because each value shown is the default. A build that links two or more, and writes no `module`, gives the table `NoLoadingCache` too. Nothing is logged in either case, and the open questions ask whether something should be. A guard test beside each stock list, the `ts` tool's included, fails if the list offers a loading cache (section 9). No setting turns holding off in a build that links one loading cache module, because `module` can only name a linked module. + +**A module nothing offers.** A `module` that no linked crate offers a cache from is refused by the deploy check, `ts config validate`, and again when the adapter builds its state. Both use one lookup, `named_cache` in `integrations/mod.rs`, whose message names the table and lists the loading cache modules the build has. The deploy check runs it through `check_loading_cache_modules` from `validate_settings_for_deploy_with`. A stock build has no such module, so it refuses every `module`, `51degrees` included, and its message ends `The loading cache modules this build has are: none`, because `offered_cache_names` writes `none` for an empty list. + +**The implementation.** The implementation is the module crate `crates/loading-cache/51degrees` (package `trusted-server-loading-cache-51degrees`, module `loading-cache.51degrees`), which 51Degrees maintains and which its manifest marks `vendor owned`. The module crate is written with the published [`fiftyone-caching`](https://crates.io/crates/fiftyone-caching) crate ([source](https://github.com/51Degrees/rust/tree/e2fff6f66097b81d831b7e93c9c35372831e4cc4/caching), at the commit that `4.5.8` was published from). The workspace builds the published crate with its default features off, so the published crate's own Fastly, Cloudflare, Spin and tokio modules are not compiled. The loading logic comes from the published crate, and the stores are written in the module crate (`stores.rs`) over core's `PlatformValueCache` and the host's key-value store. The module crate's `builder()` offers the cache through `IntegrationBuilder::with_loading_cache`. A deployment that wants the cache depends on the module crate and hands `builder()` to its adapter, through `run_with` on Fastly or `routes_with_registrations` on Axum, Cloudflare and Spin. Each of those runs a deployment's builders after the stock ones, and each adapter calls `loading_caches_from` with all of them, so no adapter names the module crate. + +**After the fold.** Row 56 folds every module crate into one crate, `crates/trusted-server-integrations` (`2026-10-26-integrations-crate-design.md`). The module's code moves to `src/loading-cache/51degrees/` in that crate, and the crate compiles the folder's cache, the code that uses `fiftyone-caching`, only with its Cargo feature `loading-cache-51degrees`, which is also the only thing that makes `fiftyone-caching` a dependency. The folder's module name is a constant that every build compiles, so the crate's test of module names checks it whatever features are on. No stock adapter and not the `ts` tool turns the feature on, so a stock build still compiles no `fiftyone-caching`. A deployment that links the module turns the feature on in its own `Cargo.toml` and hands the module's builder to its adapter, as above. **The two layers.** A call goes through two layers. The memory layer is one least recently used store for each cache, shared by every request an adapter serves with one application state, holding up to `memory_entries` values (64 for robots.txt) for `memory_seconds` each. Below the memory layer, a shared store layer is made for each call over the host's shared store, which is the host's value cache where the host gives one, being Fastly's cache, and the host's key-value store otherwise. A read looks in memory, then in the shared store, and only then loads. A copy in the shared store lasts at most `store_seconds`, leaves after `store_idle_seconds` unused, and ends sooner when the loaded value carries a lifetime of its own, which robots.txt sets to the module's refresh. The shared store holds each entry in the `fiftyone-caching` entry format, which carries the time the entry must be dropped, so a store that deletes late, or keeps no lifetimes, never gives back an entry past that time. A write to a key-value store carries a lifetime of at least 60 seconds because Workers KV refuses less, and the entry's own drop time holds the exact lifetime. **Who waits for a load.** Callers that share one memory copy and ask for the same key at the same moment share one load. The first caller runs its own loader inside its own call, because no thread pool or background task exists (`LoadingCaches::new` in `lib.rs` and `Layer` in `layer.rs` of the module crate, and the test `a_load_runs_on_the_thread_of_the_caller_that_leads_it`), and the other callers wait for its value or its error. If the first caller is dropped before the load finishes, a waiting caller takes over and loads. Across processes, other instances wait only where the shared store reserves a missing key for the first instance that asks and makes the rest wait until that instance fills the key. Only Fastly's cache does this, once for each Fastly location. Workers KV and Spin's key-value store, which the Cloudflare and Spin adapters declare, reserve nothing, so instances that miss at the same moment each load, and instances that ask afterwards read the stored copy. Axum's unavailable store keeps nothing, so there the memory copy alone answers. A module's service is therefore asked once per refresh by the requests that share one memory copy, and by all the instances of one Fastly location together, but not by all the instances of a deployment together. -**Why.** The same answer is wanted by every instance and every request, and a cache that lets every miss load at once asks a module's service as many times as there are requests. Loading each key once at a time in a process, with the value shared by every waiter, is the pattern of `LruLoadingCache` in the 51Degrees [.NET](https://github.com/51Degrees/caching-dotnet) and [Java](https://github.com/51Degrees/pipeline-java) pipelines, which the published crate's documentation points to. Core keeps only the interface. Core's cache code depends on no caching package. The only place core writes the module name `51degrees` is `settings.rs`, in the documentation of `LoadingCacheConfig::module` (line 3055) and in the test `a_loading_cache_module_is_read_and_written_back` (lines 7036 to 7055), and the open questions ask about it. +**Why.** The same answer is wanted by every instance and every request, and a cache that lets every miss load at once asks a module's service as many times as there are requests. Loading each key once at a time in a process, with the value shared by every waiter, is the pattern of `LruLoadingCache` in the 51Degrees [.NET](https://github.com/51Degrees/caching-dotnet) and [Java](https://github.com/51Degrees/pipeline-java) pipelines, which the published crate's documentation points to. Core keeps only the interface. Core's cache code depends on no caching package, and once the module leaves the stock lists no stock build depends on one either. The only place core writes the module name `51degrees` is `settings.rs`, in the documentation of `LoadingCacheConfig::module` (line 3102) and in the test `a_loading_cache_module_is_read_and_written_back` (lines 7074 to 7093), and the open questions ask about it. -**The pin.** Row 24 pins `fiftyone-caching` at the pre-release `4.5.8-49` and row 31 at the release `4.5.8`. Row 31 changes `Cargo.toml` and `Cargo.lock` and nothing else, and has no design of its own. The two published crates, [`4.5.8-49`](https://crates.io/crates/fiftyone-caching/4.5.8-49) and [`4.5.8`](https://crates.io/crates/fiftyone-caching/4.5.8), were downloaded from crates.io on 2026-10-16 and compared file by file. The 21 files under `src/`, the 16 files under `tests/` and `README.md` are identical in the two. The other four files differ. `Cargo.toml`, `Cargo.toml.orig` and `Cargo.lock` differ in the crate's own version and in the version of `fiftyone-pipeline-core` that the crate names as an optional dependency, and `Cargo.lock` also holds the checksum of that version of `fiftyone-pipeline-core`. `.cargo_vcs_info.json` names the commit that each crate was published from, [`762578bf6`](https://github.com/51Degrees/rust/commit/762578bf65767856f6f3a5f5945dfb6f8679633a) for `4.5.8-49` and [`e2fff6f66`](https://github.com/51Degrees/rust/commit/e2fff6f66097b81d831b7e93c9c35372831e4cc4) for `4.5.8`, and marks both as published from a working tree with uncommitted changes. Everything this document says about the published crate therefore applies to both rows. +**The pin.** Row 24 pins `fiftyone-caching` at the pre-release `4.5.8-49` and row 31 at the release `4.5.8`. Row 31 changes `Cargo.toml` and `Cargo.lock` and nothing else, and has no design of its own. The two published crates, [`4.5.8-49`](https://crates.io/crates/fiftyone-caching/4.5.8-49) and [`4.5.8`](https://crates.io/crates/fiftyone-caching/4.5.8), were downloaded from crates.io on 2026-10-16 and compared file by file. The 21 files under `src/`, the 16 files under `tests/` and `README.md` are identical in the two. The other four files differ. `Cargo.toml`, `Cargo.toml.orig` and `Cargo.lock` differ in the crate's own version and in the version of `fiftyone-pipeline-core` that the crate names as an optional dependency, and `Cargo.lock` also holds the checksum of that version of `fiftyone-pipeline-core`. `.cargo_vcs_info.json` names the commit that each crate was published from, [`762578bf6`](https://github.com/51Degrees/rust/commit/762578bf65767856f6f3a5f5945dfb6f8679633a) for `4.5.8-49` and [`e2fff6f66`](https://github.com/51Degrees/rust/commit/e2fff6f66097b81d831b7e93c9c35372831e4cc4) for `4.5.8`, and marks both as published from a working tree with uncommitted changes. Everything this document says about the published crate therefore applies to both rows. The published crate's manifest gives its license as `EUPL-1.2`, and it is a dependency, so none of its files is in the repository. The module crate takes the workspace license, Apache-2.0, as every crate under `crates/` does. -**Where row 24 and row 31 differ.** The two rows differ in three places. +**Where row 24 and row 31 differ.** The two rows differ in four places. -| What | Row 24 | Row 31 | -| ---------------------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| The pin on `fiftyone-caching` | The pre-release `4.5.8-49` | The release `4.5.8` | -| Where the key prefix comes from | `ServerKeyPrefix::for_settings`, from the loaded settings' `publisher.domain`, always headed `ts` | The `[prefix]` module (row 27) through `RuntimeServices::server_key_prefix()`, headed `ts` by core's own module, with a trailing dot dropped from the domain | -| What asks the cache for robots.txt | The robots.txt route handler in `robots_txt/mod.rs` | The `robots_txt` middleware's preparation in `robots_txt/middleware.rs` (row 25) | +| What | Row 24 | Row 31 | +| ---------------------------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| The pin on `fiftyone-caching` | The pre-release `4.5.8-49` | The release `4.5.8` | +| Where the key prefix comes from | `ServerKeyPrefix::for_settings`, from the loaded settings' `publisher.domain`, always headed `ts` | The `[prefix]` module (row 27) through `RuntimeServices::server_key_prefix()`, headed `ts` by core's own module, with a trailing dot dropped from the domain | +| What asks the cache for robots.txt | The robots.txt route handler in `robots_txt/mod.rs` | The `robots_txt` middleware's preparation in `robots_txt/middleware.rs` (row 25) | +| What the log keeps of a module's failure | The whole report the module gives | The report with each address in it cut to its scheme and host (row 25), so a key in an address's path, query or user part never reaches the log | -Everything else in this document is the same on both rows, apart from line numbers, the tests named in section 9, the fixes of 2026-10-16 until row 31 is merged forward, and the repeated changelog entry in the open questions. +Everything else in this document is the same on both rows, apart from line numbers and the tests that section 9 names for one row. ## 3. The adapters -Each adapter calls `loading_caches_from` when it builds its application state (Fastly `app.rs` line 347, Cloudflare line 315, Spin line 244, Axum line 214) and hands the result to every request's services. What the cache keeps, and who waits, depends on the stores the adapter gives each request. +Each adapter calls `loading_caches_from` when it builds its application state (Fastly `app.rs` line 352, Cloudflare line 317, Spin line 250, Axum line 219) and hands the result to every request's services. In a stock build the result selects no cache, so every request loads through `NoLoadingCache`. What a deployment that links the 51Degrees module keeps, and who waits for a load, depends on the stores the adapter gives each request. | Adapter | Shared store | Do other instances wait for one load? | Memory copy | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Fastly | Fastly's cache alone, and the key-value store is not used (`stores.rs` lines 61 to 73 of the module crate, `app.rs` line 443 of the Fastly adapter) | Yes, once for each Fastly location, because a missing key is reserved for the first instance that looks it up | One for each application state. Fastly serves one request for each instance unless the opt-in reusable sandbox keeps the state, so by default the copy lasts one request | +| Fastly | Fastly's cache alone, and the key-value store is not used (`stores.rs` lines 61 to 73 of the module crate, `app.rs` line 448 of the Fastly adapter) | Yes, once for each Fastly location, because a missing key is reserved for the first instance that looks it up | One for each application state. Fastly serves one request for each instance unless the opt-in reusable sandbox keeps the state, so by default the copy lasts one request | | Cloudflare | Workers KV, the `TRUSTED_SERVER_KV` binding the adapter declares in `Hooks::stores` (`app.rs` of the Cloudflare adapter). EdgeZero treats a declared store as required, so a Worker without the binding fails every request before any route runs | No | One for each application state, which Cloudflare builds again for every request, so the copy lasts one request and Workers KV holds the copy between requests. `factory()` makes a cache for each state on every host | | Spin | Spin's `default` key-value store, which the adapter declares in `Hooks::stores` and `spin.toml` grants. A component that cannot open it fails every request before any route runs. Spin gives an entry no lifetime, so the entry is written without one and its own drop time ends its use | No | One for each application state, which Spin builds again for every request, so the copy lasts one request and the `default` store holds the copy between requests (see the note below the table) | | Axum | None. The key-value store is `UnavailableKvStore`, which fails every call (`axum/platform.rs` lines 572 to 574, `platform/kv.rs` lines 18 to 24) | Not applicable | One for the process, because Axum, the native development server, builds its application state once. That copy is the only one, and a restart loads again | -Spin builds its application state for every request because the Spin adapter calls `run_app` in [EdgeZero](https://github.com/stackpop/edgezero) (`edgezero-adapter-spin/src/lib.rs`, at the revision the workspace pins), which builds the application on every call. +Spin builds its application state for every request because the Spin adapter calls `run_app` in [EdgeZero](https://github.com/IABTechLab/edgezero) (`edgezero-adapter-spin/src/lib.rs` at the locked `v0.0.10`), which builds the application on every call. A shared store that fails is read as holding nothing. A failed read is a miss, and a failed write is logged at debug level while the value is still served (`stores.rs` lines 207 to 231 for a key-value store, and `value_cache.rs` of the Fastly adapter for Fastly's cache). On Axum the cache therefore keeps the memory copy only. @@ -76,42 +93,54 @@ Every load on every adapter runs in the caller that leads it. The published crat ## 4. Configuration -`[cache.robots-txt]` is the only loading cache table on these rows. Both `CacheSettings` and `LoadingCacheConfig` refuse a key they do not know (`settings.rs` lines 3023 and 3052), so a misspelt field fails when the settings load. +`[cache.robots-txt]` is the only loading cache table on these rows. Both `CacheSettings` and `LoadingCacheConfig` refuse a key they do not know (`settings.rs` lines 3070 and 3099), so a misspelled field fails when the settings load. -| Field | Default | Range | What it does | -| -------------------- | -------------------------------------------- | -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | -| `module` | The one loading cache module the build links | A linked module, by short or full name | Selects the module whose loading cache holds each robots.txt module's answer | -| `memory_seconds` | 60 | 0 to 31536000 | How long the copy in memory is used. Zero keeps no memory copy, and a missing value is still loaded once, as the field says | -| `store_seconds` | 604800 (a week) | 1 to 31536000 | The longest the copy in the shared store is used. The module's own refresh ends that copy sooner | -| `store_idle_seconds` | 2592000 (30 days) | 1 to 31536000 | How long the copy in the shared store lasts unused | +| Field | Default | Range | What it does | +| -------------------- | ----------------------------------------------------------------------- | -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | +| `module` | The one loading cache module the build links, and none in a stock build | A linked module, by short or full name | Selects the module whose loading cache holds each robots.txt module's answer | +| `memory_seconds` | 60 | 0 to 31536000 | How long the copy in memory is used. Zero keeps no memory copy, and a missing value is still loaded once, as the field says | +| `store_seconds` | 604800 (a week) | 1 to 31536000 | The longest the copy in the shared store is used. The module's own refresh ends that copy sooner | +| `store_idle_seconds` | 2592000 (30 days) | 1 to 31536000 | How long the copy in the shared store lasts unused | -The ranges are checked when the settings load and again by the deploy check, which refuses a value out of range and names the field (`settings.rs` lines 3102 to 3105 and 3149 to 3150). The deploy check also refuses a `module` that no linked crate offers a cache from, with the message the adapter gives when it builds its state, as section 2 describes. Without that check a misspelt `module` would pass `ts config validate` and every adapter would answer with its startup error, from its start on Axum and on every request on the other three, because those three build the state for every request. +The ranges are checked when the settings load and again by the deploy check, which refuses a value out of range and names the field (`settings.rs` lines 3150 to 3153 and 3197 to 3198). The deploy check also refuses a `module` that no linked crate offers a cache from, with the message the adapter gives when it builds its state, as section 2 describes. Without that check a misspelled `module` would pass `ts config validate` and every adapter would answer with its startup error, from its start on Axum and on every request on the other three, because those three build the state for every request. A stock build links no loading cache module, so there the deploy check and the adapter refuse every `module`. ## 5. Consent and privacy -**What is stored.** Only what a module answered for the publisher, as the JSON of a robots.txt `Contribution`, and nothing about a reader. The key is `/robots-txt/`, where the hash is the SHA-256, in hexadecimal, of the module's name, a newline and the module's fingerprint (a value that changes whenever the module's answer would, such as a hash of its settings). A fingerprint's contents therefore never appear in a store (`held_key` in `holding.rs`, lines 190 to 202, and the test `the_key_carries_nothing_of_the_fingerprint`). The answer kept for a failure is under `/robots-txt-kept/`. +**What is stored.** Only what a module answered for the publisher, as the JSON of a robots.txt `Contribution`, and nothing about a reader. The key is `/robots-txt/`, where the hash is the SHA-256, in hexadecimal, of the host the request was addressed to, the module's name and the module's fingerprint (a value that changes whenever the module's answer would, such as a hash of its settings), each preceded by its length, so that no two different sets of parts give the same bytes. A fingerprint's contents therefore never appear in a store (`held_key` in `holding.rs`, lines 224 to 240, and the tests `the_key_carries_nothing_of_the_fingerprint` and `parts_that_run_together_do_not_share_a_key`). The host is read from `Host`, or from the request's address where there is no `Host` header, and never from `Forwarded` or `X-Forwarded-Host`, which a client can set to any host it likes (`addressed_host` in `crates/trusted-server-core/src/http_util.rs`). One host's answer is therefore never served for another (the test `an_answer_is_held_for_the_host_it_was_asked_for`). The answer kept for a failure is under `/robots-txt-kept/`. -**What a load is given.** The load of the first caller serves every caller waiting on it and every later reader of the copy, so it is given nothing of the reader whose request started it. `RuntimeServices::without_reader` gives a copy of a request's services with the `ClientInfo` (the client's IP address, TLS details and connection fingerprints) and the host's signals about the client cleared, and the stores, caches, backends, HTTP client and modules kept. `RuntimeServices::load_cached` hands the cache that copy, and for robots.txt the module is called with the request's method, path, host and scheme with the query removed, and with that copy (`holding.rs` builds the call from `ModuleContext::new(request.view()).with_services(&services)`, where the services are `services.without_reader()`). The call carries no reader evidence, connection, location, consent or permissions, and the contract of `RobotsTxtContributor::contribute` (`robots_txt/contributor.rs` lines 61 to 73) says the answer must be the same for every request the publisher receives. The test `the_contributor_is_asked_for_the_request_and_not_who_asked` checks what the call carries, `the_contributor_is_handed_nothing_of_the_reader_who_asks_first` checks that no client address, TLS detail or host signal reaches the module, and `a_cache_is_handed_nothing_of_the_reader_whose_request_loads` checks what the cache is handed. +**What a load is given.** The load of the first caller serves every caller waiting on it and every later reader of the copy, so it is given nothing of the reader whose request started it. `RuntimeServices::without_connection` gives a copy of a request's services with the `ClientInfo` (the client's IP address, TLS details and connection fingerprints) and the host's signals about the client cleared, and the stores, caches, backends, HTTP client and modules kept. `RuntimeServices::load_cached` hands the cache that copy, and for robots.txt the module is called with the request's method, path and scheme, the host the request was addressed to and no query, and with that copy (`holding.rs` builds the call from `ModuleContext::new(request.view()).with_services(&services)`, where the services are `services.clone().without_connection()`). The call carries no reader evidence, connection, location, consent or permissions, and the contract of `RobotsTxtContributor::contribute` (`robots_txt/contributor.rs` lines 61 to 76) says the answer must be the same for every request to the host it was asked for, because it is served for all of them. The test `the_contributor_is_asked_for_the_request_and_not_who_asked` checks what the call carries, `the_contributor_is_handed_nothing_of_the_reader_who_asks_first` checks that no client address, TLS detail or host signal reaches the module, and `a_cache_is_handed_nothing_of_the_reader_whose_request_loads` checks what the cache is handed. -**What the types leave open.** The documentation of `Loader` and of `PlatformLoadingCache::get_or_load` states the rule, and the one caller on these rows, the robots.txt holding, follows it. A `Loader` is a closure that owns whatever it captures, so nothing in the types stops a later caller from capturing a request's own services, and for such a caller the rule holds only in words. +**What the types leave open.** The documentation of `Loader` and of `PlatformLoadingCache::get_or_load` states the rule, and the one caller on these rows, the robots.txt holding, follows it. A `Loader` is a closure that owns whatever it captures, so nothing in the types stops a later caller from capturing a request's own services, and for such a caller the rule holds only in words. Row 44 adds such a caller. The 51Degrees shared code's loader captures the request's own services (`crates/shared/51degrees/src/client.rs` line 206 at row 55), and uses only their backend and HTTP client (lines 252 to 259), so nothing of the reader reaches the cloud service. ## 6. Failure - **A failed load.** The error reaches every caller waiting on that load and nothing is stored, so the next caller loads again. The test `a_failed_load_reaches_every_waiter_and_is_not_kept` checks that behavior with 16 callers. Where the shared store reserves keys, a reservation dropped without being filled passes to another instance, so instances waiting on a failed load do not wait for a timeout (the module crate's test `a_reserving_value_cache_is_filled_by_the_load_and_released_by_a_failure` and the Fastly adapter's test `a_dropped_reservation_passes_the_obligation_on`). - **A first caller that is dropped.** A waiting caller takes over the load with its own loader (the test `a_waiting_caller_loads_when_the_caller_leading_the_load_is_dropped`). - **A shared store that fails.** A failed read is a miss and a failed write is logged while the value is still served, as section 3 says, so the memory copy alone answers (the test `a_store_that_fails_every_call_leaves_the_memory_copy_answering`, with Axum's `UnavailableKvStore`). -- **A held copy that is unreadable.** A copy that no longer parses as a robots.txt `Contribution` is removed from the cache and the module is asked again (`holding.rs` lines 95 to 109, and the test `an_unreadable_held_copy_is_removed_and_the_contributor_asked_again`). -- **A module that fails.** The last answer the module gave is served past its age. Every load that succeeds writes its answer apart from the cache for 30 days, in the host's value cache where it has one and in the key-value store otherwise, and nothing reads that answer unless the module fails (`holding.rs` lines 35 to 40 and 110 to 125). With none kept, the whole file answers `503` with `Retry-After: 600` and is not stored. A value cache may drop an entry early, so on Fastly a kept answer is likely and not certain to be there. Spin's `default` store gives an entry no lifetime, so a kept answer there stays until it is replaced. Axum's key-value store fails every call, so Axum keeps no answer. Because a failure is not stored, every request asks the module again while the module is down, and the requests in one process that arrive during one attempt share that attempt. +- **A held copy that is unreadable.** A copy that no longer parses as a robots.txt `Contribution` is removed from the cache and the module is asked again (`holding.rs` lines 99 to 113, and the test `an_unreadable_held_copy_is_removed_and_the_contributor_asked_again`). +- **A module that fails.** The last answer the module gave is served past its age. Every load that succeeds writes its answer apart from the cache for 30 days, in the host's value cache where it has one and in the key-value store otherwise, and nothing reads that answer unless the module fails (`holding.rs` lines 40 to 45 and 114 to 130). With none kept, the whole file answers `503` with `Retry-After: 600` and is not stored. A value cache may drop an entry early, so on Fastly a kept answer is likely and not certain to be there. Spin's `default` store gives an entry no lifetime, so a kept answer there stays until it is replaced. Axum's key-value store fails every call, so Axum keeps no answer. Because a failure is not stored, every request asks the module again while the module is down, and the requests in one process that arrive during one attempt share that attempt. ## 7. What changes for a deployment -**A stock build holds answers with no new setting.** The default in section 2 means a stock build holds each robots.txt module's answer as soon as the build includes this change. The shortest holding the settings allow is a memory copy of 0 seconds and a shared copy of 1 second. +**A stock build holds nothing, and that has a cost.** With no loading cache module linked, every feature loads through `NoLoadingCache`, which keeps nothing and runs every load. Above these rows the same holds for every cache a module declares. Three things follow. + +1. **The client hints middleware asks the cloud service for every page it is placed for.** A stock build that runs the 51Degrees modules of row 44, through `[shared.51degrees]`, makes one call to the 51Degrees cloud service on each page navigation whose path a `[[serve]]` entry places the client hints middleware on. The middleware's request filter makes the call and waits for it before the page is fetched (at row 55, `crates/middleware/51degrees/src/lib.rs` lines 364 to 390, through `crates/shared/51degrees/src/client.rs` lines 196 to 220). The hour an answer would be held for, `ANSWER_LIFETIME` (`crates/shared/51degrees/src/engine_cache.rs` line 38), saves nothing, and a call that fails leaves the page without hints. +2. **A robots.txt module that a deployment adds is asked, and its answer kept, on every request for the file.** No stock module contributes robots.txt rules, so a stock build with nothing added asks no module. A module that a deployment adds is asked on every request for `/robots.txt`, and each answer it gives is written again as its kept answer, because `keep_answer` (`holding.rs` lines 266 to 286) runs in every load that succeeds, and the request waits for the write. On Cloudflare that is one write to Workers KV for each request for the file, against the published limit of one write a second to one key ([Workers KV limits](https://developers.cloudflare.com/kv/platform/limits/)). A write that fails is logged at debug level and the answer is still served. On Spin it is one write to the `default` store, and on Fastly one to Fastly's cache. +3. **The 51Degrees shared code's README promises what a stock build does not do.** It says that the cloud service's answer is "held in the loading cache so a question many requests ask at once is asked once" (`crates/shared/51degrees/README.md` lines 12 to 14 at row 55), which holds only where a deployment links a loading cache module. The change rewrites the README, as section 2 lists. + +The asset cache is a module that no stock build links (row 34, `2026-10-15-asset-cache-design.md`), so a stock build asks the origin for every file, as before row 34. + +**Linking one module is enough.** A deployment that links the 51Degrees module holds answers with no new setting, because every table that names no module takes the one module linked. The shortest holding the settings allow is a memory copy of 0 seconds and a shared copy of 1 second. + +**A document that names the module needs the module linked.** `module = "51degrees"` in any `[cache.]` table is refused by a build that does not link the module, by `ts config validate` and when the adapter builds its state, with the message section 2 gives. A deployment whose document names the module links the module before it takes a build without the module in its stock lists, or leaves the line out, as the changelog entry the change adds says. + +**The `ts` tool links no loading cache module.** `ts config validate` checks a document against the builders the tool links, being `trusted_server_modules::builders()` on rows 24 to 42 and `linked_builders()` in `crates/trusted-server-cli/src/app_config.rs` from row 43, and neither names the module after the change. The stock tool therefore refuses a document that names it. A deployment that links the module validates with a list that holds the module's builder, registered first through `register_deploy_integrations` in `crates/trusted-server-core/src/config.rs`, where the first call wins, as `2026-10-14-no-modules-crate-design.md` describes for any set of modules other than the stock one. -**Every adapter holds answers where none held any.** Row 23 gave every request `UnavailableKvStore`, Fastly's and Axum's directly and Cloudflare's and Spin's because neither adapter declared a store for EdgeZero to open, so nothing was held and every request asked the module. From row 24 Fastly holds each answer in its cache, shared by the instances at a location, Cloudflare and Spin hold it in the key-value store each declares, and Axum holds each answer in memory for the life of the process. +**Where a module is linked, every adapter holds answers where none held any.** Row 23 gave every request `UnavailableKvStore`, Fastly's and Axum's directly and Cloudflare's and Spin's because neither adapter declared a store for EdgeZero to open, so nothing was held and every request asked the module. From row 24, a deployment that links the 51Degrees module holds each answer in Fastly's cache, shared by the instances at a location, in the key-value store that Cloudflare and Spin each declare, and on Axum in memory for the life of the process. -**Cloudflare and Spin need their store.** EdgeZero treats a declared store as required, so a Worker without the `TRUSTED_SERVER_KV` binding, or a Spin component that cannot open `default`, fails every request before any route runs, the Spin health check included. +**Cloudflare and Spin need their store.** EdgeZero treats a declared store as required, so a Worker without the `TRUSTED_SERVER_KV` binding, or a Spin component that cannot open `default`, fails every request before any route runs, the Spin health check included. The adapters declare the stores whether or not a loading cache module is linked, so this holds for a stock build too. -**The keys change.** Row 23 held a module's answer under `robots-txt:` and a hash of the publisher, the module's name and its fingerprint. Rows 24 and 31 hold the answer under `/robots-txt/`, where the hash covers the module's name and fingerprint and the publisher is in the prefix. A copy written before the change is never read again. The first request for each module after an upgrade asks the module, and a module that fails before it has answered once has no kept answer under the new key, so the file answers `503`. Old entries end by their own 30 day lifetime where the store gives lifetimes, and stay in a store that does not, such as Spin's, until someone deletes them. +**The keys change.** Row 23 held a module's answer under `robots-txt:` and a hash of the publisher, the module's name and its fingerprint. Rows 24 and 31 hold the answer under `/robots-txt/`, where the hash covers the host the request was addressed to, the module's name and its fingerprint, and the publisher is in the prefix. A copy written before the change is never read again. The first request for each module after an upgrade asks the module, and a module that fails before it has answered once has no kept answer under the new key, so the file answers `503`. Old entries end by their own 30 day lifetime where the store gives lifetimes, and stay in a store that does not, such as Spin's, until someone deletes them. **An older binary refuses the table.** `[cache]` refuses a key it does not know, so a binary built before row 24 refuses a settings document that carries `[cache.robots-txt]`. A document that sets nothing under the table serializes without the table (the test `unset_loading_cache_lifetimes_are_not_written_back`), so a stored configuration that does not use the table stays readable by an older binary. @@ -119,64 +148,90 @@ The ranges are checked when the settings load and again by the deploy check, whi - The page template loaded through the loading cache. The page template still goes through the template cache that core has, because moving the page template is a change to the template flow that this change does not make. - One selection for the whole deployment. Each feature names its cache in its own table, because the asset cache and any later feature may choose differently. -- A table for any feature but robots.txt. Row 32 ([IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373), `2026-10-15-module-cache-declarations-design.md`) lets a module declare the caches it loads through, and row 34 ([IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375), `2026-10-15-asset-cache-design.md`) holds the publisher's assets through the loading cache. +- A table for any feature but robots.txt. Row 32 ([IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373), `2026-10-15-module-cache-declarations-design.md`) lets a module declare the caches it loads through, and row 34 ([IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375), `2026-10-15-asset-cache-design.md`) holds the publisher's assets through the loading cache, in a module of its own. +- A cache that keeps values in a stock build. Core supplies only `NoLoadingCache`, and the open questions ask whether it should supply more. - A load against a real host store. The description of IABTechLab/trusted-server#1365 says that none has run against one. ## 9. Acceptance -**Checks that can be repeated.** In a clone of IABTechLab/trusted-server, this search prints two lines, both in `crates/trusted-server-modules`, so no adapter and no part of core names the module crate. +**Checks that can be repeated.** In a clone of IABTechLab/trusted-server, this search prints two lines at row 31 (`490a4665d`), both in `crates/trusted-server-modules`, so no adapter and no part of core names the module crate. ```text -git grep -n -e trusted_server_loading_cache_51degrees -e trusted-server-loading-cache-51degrees 4b6ef9c05 -- crates ':!crates/loading-cache' +git grep -n -e trusted_server_loading_cache_51degrees -e trusted-server-loading-cache-51degrees 490a4665d -- crates ':!crates/loading-cache' ``` -On row 24 after its fixes (`35640194a`), this search prints nothing, so `CacheSpec` has no `detachable` field and nothing uses the published crate's Cloudflare request scope or spawner. +Once the module leaves the stock lists, the same search with the head of row 24, 31, 32 or 43 in place of the commit prints nothing, so no stock list names the module crate either. From row 44 it prints six lines, each a link that only the tests of the 51Degrees modules use, as at row 55 (`5be114773`). + +- `crates/middleware/51degrees/Cargo.toml` line 31, a dev-dependency, and `crates/middleware/51degrees/src/lib.rs` line 1575, in the crate's tests. +- `crates/shared/51degrees/Cargo.toml` line 19, the crate's `test-utils` feature, line 23, the optional dependency that the feature turns on, and line 39, a dev-dependency. +- `crates/shared/51degrees/src/test_services.rs` line 403, which compiles only for tests or with `test-utils`. + +For each adapter and the `ts` tool, the dependency tree then holds no `fiftyone-caching`, which this command shows for the Fastly adapter by printing 0, and the same command with each other adapter's package and target, and with the tool's package, prints 0 as well. This is the check that does not depend on reading the search's output. + +```text +cargo tree -p trusted-server-adapter-fastly -e normal --target wasm32-wasip1 | grep -c fiftyone-caching +``` + +At both heads this search prints nothing, so `CacheSpec` has no `detachable` field and nothing uses the published crate's Cloudflare request scope or spawner. ```text -git grep -n -e detachable -e 'cloudflare::with_context' -e WaitUntil -e local_spawner 35640194a -- crates +git grep -n -e detachable -e 'cloudflare::with_context' -e WaitUntil -e local_spawner 490a4665d -- crates ``` -**Tests.** These tests pin the cache, its selection and its use for robots.txt on row 31, and those marked row 24 came with the fixes of 2026-10-16 and reach row 31 when it is merged forward. Row 24 has every test, apart from `the_held_and_kept_keys_follow_the_publisher`, which arrives with the prefix module on row 27, and row 24 builds the key prefix with `ServerKeyPrefix::for_domain`. +**Tests.** These tests pin the cache, its selection and its use for robots.txt, on both rows unless a row is named. Row 24 builds the key prefix with `ServerKeyPrefix::for_settings` and row 31 with the prefix module of row 27. Up to row 43 a test that needs a cache that keeps values uses core's `MemoryLoadingCache` or sits in the module crate, which is why the search above prints nothing there once the module leaves the stock lists. -- `crates/loading-cache/51degrees/src/tests.rs`, which `cargo test-axum` runs natively and `cargo test-fastly` does not list. `many_callers_on_many_threads_load_a_missing_key_once_in_the_caller` (16 threads, one load), `a_failed_load_reaches_every_waiter_and_is_not_kept`, `the_store_answers_another_process_without_a_load`, `a_store_that_refuses_lifetimes_still_holds_the_entry`, `two_publishers_never_share_an_entry`, `the_memory_copy_is_brief_and_the_store_is_read_after_it`, `an_entry_left_unused_for_its_idle_time_is_loaded_again`, `a_value_with_a_lifetime_of_its_own_ends_by_then`, `a_reserving_value_cache_is_filled_by_the_load_and_released_by_a_failure`, `remove_drops_the_memory_copy_and_the_stored_one`, `the_builder_offers_a_cache_that_keeps_values` and `module_constant_is_the_crate_folder`. Row 24 adds `each_application_state_is_given_a_cache_of_its_own`, `a_load_runs_on_the_thread_of_the_caller_that_leads_it`, `a_memory_lifetime_of_zero_keeps_no_memory_copy_and_still_loads_once` (16 threads, one load, and the store read once the load is done), `a_waiting_caller_loads_when_the_caller_leading_the_load_is_dropped` and `a_store_that_fails_every_call_leaves_the_memory_copy_answering`. +- `crates/loading-cache/51degrees/src/tests.rs`, which `cargo test-axum` runs natively and `cargo test-fastly` does not list. `many_callers_on_many_threads_load_a_missing_key_once_in_the_caller` (16 threads, one load), `a_failed_load_reaches_every_waiter_and_is_not_kept`, `the_store_answers_another_process_without_a_load`, `a_store_that_refuses_lifetimes_still_holds_the_entry`, `two_publishers_never_share_an_entry`, `the_memory_copy_is_brief_and_the_store_is_read_after_it`, `an_entry_left_unused_for_its_idle_time_is_loaded_again`, `a_value_with_a_lifetime_of_its_own_ends_by_then`, `a_reserving_value_cache_is_filled_by_the_load_and_released_by_a_failure`, `remove_drops_the_memory_copy_and_the_stored_one`, `the_builder_offers_a_cache_that_keeps_values`, `module_constant_is_the_crate_folder`, `each_application_state_is_given_a_cache_of_its_own`, `a_load_runs_on_the_thread_of_the_caller_that_leads_it`, `a_memory_lifetime_of_zero_keeps_no_memory_copy_and_still_loads_once` (16 threads, one load, and the store read once the load is done), `a_waiting_caller_loads_when_the_caller_leading_the_load_is_dropped` and `a_store_that_fails_every_call_leaves_the_memory_copy_answering`. - `crates/trusted-server-core/src/integrations/mod.rs`, in `loading_caches_from_tests`. `a_feature_selecting_nothing_takes_the_one_module_offered`, `a_feature_selecting_nothing_keeps_nothing_when_two_are_offered`, `a_feature_selecting_nothing_keeps_nothing_when_none_is_offered`, `a_feature_selects_a_module_by_its_short_name_or_in_full`, `a_feature_naming_a_module_nothing_offers_is_refused_naming_what_is_offered` and `with_nothing_offered_the_refusal_says_so`. -- `crates/trusted-server-core/src/config.rs`, on row 24. `deploy_validation_refuses_a_loading_cache_module_nothing_offers`. -- `crates/trusted-server-core/src/platform/loading_cache.rs`. `two_publishers_never_share_a_key`, `a_key_names_the_publisher_then_the_cache` and `the_default_cache_runs_every_load`. Row 24 adds `a_cache_is_handed_nothing_of_the_reader_whose_request_loads`. +- `crates/trusted-server-core/src/config.rs`. `deploy_validation_refuses_a_loading_cache_module_nothing_offers`. +- `crates/trusted-server-core/src/platform/loading_cache.rs`. `two_publishers_never_share_a_key`, `a_key_names_the_publisher_then_the_cache`, `the_default_cache_runs_every_load` and `a_cache_is_handed_nothing_of_the_reader_whose_request_loads`. - `crates/trusted-server-core/src/platform/server_key.rs`. `a_leading_www_is_folded_and_every_other_label_kept`, and `the_domain_is_the_folded_one` on row 31, where row 24 has `the_prefix_comes_from_the_loaded_settings`. -- `crates/trusted-server-core/src/robots_txt/holding.rs`, which runs the holding against a core test double that does not join loads. `a_contributor_is_asked_once_and_its_answer_is_held_after_that`, `the_held_answer_is_keyed_by_the_publisher`, `the_held_and_kept_keys_follow_the_publisher`, `changed_settings_are_not_answered_from_the_old_copy`, `the_kept_answer_is_served_when_the_contributor_fails`, `the_kept_answer_outlives_the_instance_that_kept_it`, `a_store_without_lifetimes_still_keeps_the_answer`, `with_nothing_kept_a_failure_is_an_error_and_is_not_held`, `an_unreadable_held_copy_is_removed_and_the_contributor_asked_again`, `the_kept_answer_is_in_the_value_cache_where_the_host_has_one`, `the_key_carries_nothing_of_the_fingerprint` and `the_contributor_is_asked_for_the_request_and_not_who_asked`. Row 24 adds `the_contributor_is_handed_nothing_of_the_reader_who_asks_first`. +- `crates/trusted-server-core/src/robots_txt/holding.rs`, which runs the holding against a core test double that does not join loads. `a_contributor_is_asked_once_and_its_answer_is_held_after_that`, `the_held_answer_is_keyed_by_the_publisher`, `an_answer_is_held_for_the_host_it_was_asked_for`, `changed_settings_are_not_answered_from_the_old_copy`, `the_kept_answer_is_served_when_the_contributor_fails`, `the_kept_answer_outlives_the_instance_that_kept_it`, `a_store_without_lifetimes_still_keeps_the_answer`, `with_nothing_kept_a_failure_is_an_error_and_is_not_held`, `an_unreadable_held_copy_is_removed_and_the_contributor_asked_again`, `the_kept_answer_is_in_the_value_cache_where_the_host_has_one`, `the_key_carries_nothing_of_the_fingerprint`, `parts_that_run_together_do_not_share_a_key`, `the_contributor_is_asked_for_the_request_and_not_who_asked` and `the_contributor_is_handed_nothing_of_the_reader_who_asks_first`. Row 31 also has `the_held_and_kept_keys_follow_the_publisher`, which arrives with the prefix module on row 27, and `a_failure_gives_only_the_host_of_an_address_it_names`, which arrives on row 25. - `crates/trusted-server-core/src/settings.rs`. `loading_cache_lifetimes_are_read_with_defaults_for_what_is_unset`, `a_loading_cache_lifetime_out_of_range_is_refused_naming_it`, `unset_loading_cache_lifetimes_are_not_written_back` and `a_loading_cache_module_is_read_and_written_back`. -- `crates/trusted-server-modules/src/lib.rs`. The stock list order, which names `loading-cache.51degrees`, and `a_feature_selects_the_stock_loading_cache_by_name`. Row 24 adds `a_table_naming_no_module_holds_with_the_stock_loading_cache` and `deploy_validation_knows_the_stock_loading_cache`. - `crates/trusted-server-adapter-fastly/src/value_cache.rs`, which `cargo test-fastly` runs under Viceroy, Fastly's local runtime. Seven tests, including `a_missing_key_is_reserved_and_its_fill_is_then_a_hit`, `a_dropped_reservation_passes_the_obligation_on` and `a_stale_entry_is_reserved_rather_than_served`. -- The Cloudflare and Spin adapters, on row 24. `the_declared_store_is_the_key_value_store_a_request_is_given` in each adapter's `tests/key_value_store.rs`, `every_wrangler_manifest_binds_the_declared_key_value_store` in the Cloudflare adapter's `tests/config_defaults.rs`, and `the_declared_key_value_store_is_a_label_spin_grants` in the Spin adapter's `tests/key_value_store.rs`. -- `crates/trusted-server-integration-tests/tests/integration.rs`, on row 24. `test_cloudflare_without_its_kv_binding_fails_every_request`, which CI runs under `wrangler dev` with the namespace binding removed and which expects `500` on every path. +- The Cloudflare and Spin adapters. `the_declared_store_is_the_key_value_store_a_request_is_given` in each adapter's `tests/key_value_store.rs`, `every_wrangler_manifest_binds_the_declared_key_value_store` in the Cloudflare adapter's `tests/config_defaults.rs`, and `the_declared_key_value_store_is_a_label_spin_grants` in the Spin adapter's `tests/key_value_store.rs`. +- The robots.txt route of each adapter, on row 24 alone, because row 25 replaces the route with core's middleware. `with_nothing_held_a_module_that_fails_makes_the_file_unavailable` in the Axum adapter's `tests/seam_probe.rs` and in the Cloudflare and Spin adapters' `tests/routes.rs`, `with_nothing_kept_a_module_that_fails_makes_the_file_unavailable` in the Fastly adapter's `src/app/seam_probe_tests.rs`, and `a_held_answer_is_served_when_its_module_fails` in the Cloudflare and Spin adapters' `tests/routes.rs`. Each gives its module a refresh of zero, so nothing is held through the cache, and the kept answer is what a module that fails is answered from. +- `crates/trusted-server-integration-tests/tests/integration.rs`. `test_cloudflare_without_its_kv_binding_fails_every_request`, which CI runs under `wrangler dev` with the namespace binding removed and which expects `500` on every path. + +**The tests the change touches.** Each reads a list of builders and none reads a build, so the `cargo tree` command above stays the check of what a build compiles. + +- **Row 24, `crates/trusted-server-modules/src/lib.rs`.** The stock list's order test, `stock_modules_are_offered_in_hook_order`, stops naming `loading-cache.51degrees`. The three tests that pin the stock cache, `a_feature_selects_the_stock_loading_cache_by_name`, `a_table_naming_no_module_holds_with_the_stock_loading_cache` and `deploy_validation_knows_the_stock_loading_cache`, give way to `the_stock_build_links_no_loading_cache`, which fails if a builder in the stock list offers a loading cache (`IntegrationBuilder::loading_cache` gives one). +- **Row 32, `crates/trusted-server-adapter-axum/tests/seam_probe.rs`.** `a_cache_the_probe_declares_is_selected_in_its_own_table` (lines 564 to 598 at `4a1f03c40`) counts the stock module as linked. Its first case shows that the table is read only because the build links a second loading cache module, so that a table that is not read would keep nothing. With the stock module gone, the build of that case links the probe's cache alone, which a table that names no module takes as well, so the case would pass whether or not `module` is read. The rewrite therefore links a second loading cache module in that case's build, a test builder offering core's `MemoryLoadingCache`, so that only a table that is read keeps values. In the other two cases the assertions (lines 587 to 591 and 593 to 597) flip, because the build that links both of the probe's builders links one loading cache module, so its case with no table keeps values, and the build that links only `seam_probe::builder()` links none, so that case keeps nothing. The test's comment (lines 564 to 569), which says that the build links the stock module and the probe's, is rewritten. +- **Row 43, each adapter's `src/app.rs`.** `the_linked_modules_are_offered_in_hook_order` stops naming `loading-cache.51degrees` (Axum line 1173, Cloudflare line 1462, Fastly line 3692 and Spin line 1911 at `008f79d86`, and lines 1779, 2044, 3928 and 2397 at row 55), and `the_stock_build_links_no_loading_cache` sits beside it, reading the adapter's `shipped_module_builders()`. The three tests that row 43 moved into `crates/trusted-server-adapter-axum/tests/shipped_modules.rs` give way to the Axum adapter's guard. +- **Row 43, `crates/trusted-server-cli/src/app_config.rs`.** No test pins the tool's `linked_builders()` (lines 36 to 66 at `008f79d86`, and lines 36 to 73 at row 55), so nothing would fail if the tool linked the module again. The change adds `the_stock_build_links_no_loading_cache` beside it, and a test that pins its builders in order, as each adapter's order test pins its own list. -**CI.** Every check passed on the head of each row. That is 19 of 19 on the closed pull requests on the fork, jwrosewell/trusted-server#19 at `844b01fe2` and jwrosewell/trusted-server#26 at `4b6ef9c05`, and 25 of 25 on IABTechLab/trusted-server#1365 and IABTechLab/trusted-server#1372, as read on 2026-10-16. +**CI.** Every check passed at the head of each row before the change, being [32 of 32](https://github.com/IABTechLab/trusted-server/commit/e66b955a43ff4559c109f237d20c3c56e28aca8f/checks) at `e66b955a4` on IABTechLab/trusted-server#1365 and [32 of 32](https://github.com/IABTechLab/trusted-server/commit/490a4665d4931654f798df2ba15ce5759e254c16/checks) at `490a4665d` on IABTechLab/trusted-server#1372. On the fork, where the two rows were reviewed first, [19 of 19](https://github.com/jwrosewell/trusted-server/commit/844b01fe205a4c9c188884233c9ce6d24de2b0b7/checks) passed at `844b01fe2` on jwrosewell/trusted-server#19 and [19 of 19](https://github.com/jwrosewell/trusted-server/commit/4b6ef9c053384b8602b8c13c6213018dbe631ff5/checks) at `4b6ef9c05` on jwrosewell/trusted-server#26. -**Local checks.** The description of IABTechLab/trusted-server#1365 records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run. The description marks a manual run with `fastly compute serve` as not done. For the fixes of 2026-10-16 on row 24, commits `a1931fd7e` to `0d49c4601`, the ten tests they added were run with the fixes reverted and with a second loading cache in the stock list and a one second floor on the memory copy, where the seven those changes break failed and the other three passed, and again with everything restored, where all ten passed. The module crate's 17 tests, 142 tests of core and 5 of the stock list chosen by the code changed, clippy on those three packages with warnings refused, `cargo fmt --all -- --check` and `cargo check-cloudflare` also passed. Neither the full core suite nor `cargo clippy-fastly` was run there, and CI runs both. +**Local checks.** The description of IABTechLab/trusted-server#1365 records `cargo test-fastly` and `cargo test-axum`, `cargo clippy-fastly` and `cargo clippy-axum`, `cargo fmt --all -- --check`, the JavaScript tests and format check, the docs format check and the WebAssembly build of the Fastly adapter as run. The description marks a manual run with `fastly compute serve` as not done. **What no test covers.** 1. **robots.txt through the real cache.** The holding tests use a double that does not join loads, and the module crate's tests use no robots.txt module, so the two never run together. -2. **Any adapter.** No adapter test loads through the cache on rows 24 and 31. The module crate's tests run natively and never on `wasm32-wasip1` or `wasm32-unknown-unknown`. `factory()` has no code that differs by target, so `each_application_state_is_given_a_cache_of_its_own` covers what the factory gives Cloudflare. +2. **Any adapter.** No adapter test checks that a copy is held through the cache. The adapter tests of row 24's robots.txt route give their module a refresh of zero, so nothing is held, and once the module leaves the stock lists no adapter test links it, as a deployment would, through `run_with` or `routes_with_registrations`. The module crate's tests run natively and never on `wasm32-wasip1` or `wasm32-unknown-unknown`. `factory()` has no code that differs by target, so `each_application_state_is_given_a_cache_of_its_own` covers what the factory gives Cloudflare. 3. **Other instances waiting for one load.** The tests use one instance. The behavior comes from Fastly's cache, as the comments in `value_cache.rs` of the Fastly adapter describe. -4. **A robots.txt answer kept in Workers KV.** No test runs a robots.txt module under `wrangler dev`, because the Worker bundle CI builds links the stock modules alone and none of them contributes robots.txt rules. A test would need a contributor in that bundle whose answer differs from one load to the next. +4. **A robots.txt answer kept in Workers KV.** On row 24, `a_held_answer_is_served_when_its_module_fails` keeps the answer in a key-value store that the test holds in memory. No test runs a robots.txt module under `wrangler dev`, because the Worker bundle CI builds links the stock modules alone and none of them contributes robots.txt rules. A test would need a contributor in that bundle whose answer differs from one load to the next. +5. **The `ts` tool with the module registered.** No test registers the module's builder through `register_deploy_integrations` and validates a document that names it. ## 10. Open questions -These are for the reviewer. This revision records what the code does and decides neither of them. +These are for the reviewer. This revision records what the code does, and what the change to rows 24, 32 and 43 makes it do, and decides none of them. -1. Core writes `51degrees` in the documentation of `LoadingCacheConfig::module` (`settings.rs` line 3055) and in the test `a_loading_cache_module_is_read_and_written_back` (lines 7036 to 7055). Should those use a name that belongs to no vendor? -2. On rows 27 to 31 `CHANGELOG.md` repeats the robots.txt holding entry (lines 13 and 14 on row 31). The first is an older wording that a merge brought in on row 27, and row 24 does not carry it. Should row 27 drop it? +1. Core writes `51degrees` in the documentation of `LoadingCacheConfig::module` (`settings.rs` line 3102, and line 2924 at row 55) and in the test `a_loading_cache_module_is_read_and_written_back` (lines 7074 to 7093, and lines 6724 to 6747 at row 55), and from row 32 in the test `a_cache_a_module_declares_is_read_from_its_own_table` (lines 6703 to 6715 at row 55). Core's own deploy check test already names a module that belongs to no vendor, `loading-cache.example` (`config.rs` lines 1529 to 1551, the test `deploy_validation_refuses_a_loading_cache_module_nothing_offers`). Should the others use such a name too? +2. A stock build holds nothing (section 7). Should core supply a cache that keeps values, so that a stock build holds the answers a module would otherwise be asked for on every request, or should every deployment that wants them held link a module? IAB Tech Lab's own draft design for response caching, [IABTechLab/trusted-server#894](https://github.com/IABTechLab/trusted-server/pull/894), keeps the uncached path on the adapters other than Fastly and says an in-process cache there would give misleading parity, because its eviction, isolation, joining of loads and purge would differ from the host's. +3. The stock `ts` tool refuses a document that names the module (section 7). Should the tool take a deployment's builders in some way other than a first call to `register_deploy_integrations`? +4. A table that names no module, and from row 32 a cache that a module declares with no table, load through `NoLoadingCache` when the build links no loading cache module or several, and nothing is logged (at row 55, `integrations/mod.rs` lines 1065 to 1071 and 1084 to 1105). Should that be logged once when the adapter builds its state, or refused? And should a build that links two loading cache modules, with a table that names neither, be refused rather than give that table `NoLoadingCache`? The asset cache design asks the same of its own module in its open question 3 (`2026-10-15-asset-cache-design.md`). ## Revision record -| Date | Change | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | -| 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the default when a table names no module, how loads run, who waits for a load, where rows 24 and 31 differ, the adapters, the configuration, consent and privacy, failure, what changes for a deployment, the tests and what none covers, the open questions and the review log. | -| 2026-10-16 | Points from review round 2. How the two published crates differ, from a comparison of the crates themselves, and the date on which the check counts were read. | -| 2026-10-16 | Code fixes on row 24. A load is handed services with nothing of the reader in them, each application state has a cache of its own on Cloudflare as elsewhere, `CacheSpec::detachable` and the text about loads that run apart from their caller are gone, the comments and guides say what a table naming no module takes, the deploy check refuses a `module` nothing offers, and tests cover the stock default, a zero memory lifetime, a dropped first caller and a failing store. Open questions 1, 2, 4, 5 and 7, and 3 apart from the repeated changelog entry, move into the text. | -| 2026-10-17 | Code fix on row 24. The Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests and a Worker or component without its store fails every request. Who waits for a load, the adapters, a module that fails, what changes for a deployment and the tests say so. | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-16 | Fixes from review round 1. The header and the pull requests it names, the default when a table names no module, how loads run, who waits for a load, where rows 24 and 31 differ, the adapters, the configuration, consent and privacy, failure, what changes for a deployment, the tests and what none covers, the open questions and the review log. | +| 2026-10-16 | Points from review round 2. How the two published crates differ, from a comparison of the crates themselves, and the date on which the check counts were read. | +| 2026-10-16 | Code fixes on row 24. A load is handed services with nothing of the reader in them, each application state has a cache of its own on Cloudflare as elsewhere, `CacheSpec::detachable` and the text about loads that run apart from their caller are gone, the comments and guides say what a table naming no module takes, the deploy check refuses a `module` nothing offers, and tests cover the stock default, a zero memory lifetime, a dropped first caller and a failing store. Open questions 1, 2, 4, 5 and 7, and 3 apart from the repeated changelog entry, move into the text. | +| 2026-10-17 | Code fix on row 24. The Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests and a Worker or component without its store fails every request. Who waits for a load, the adapters, a module that fails, what changes for a deployment and the tests say so. | +| 2026-10-26 | No stock build links a loading cache module. In a stock build every feature with no cache selected loads through `NoLoadingCache`, the 51Degrees module is offered only to a deployment that links it, and after the fold of row 56 its `fiftyone-caching` dependency sits behind a Cargo feature that no stock adapter turns on. The header, the design, the adapters, the configuration, what changes for a deployment, what is not in this change, the checks, the tests, what no test covers and the open questions say so. The services a load is given are named as the code names them, `without_connection`, where this page said `without_reader`, the name an earlier commit on row 24 gave them. The pull requests are no longer drafts, and the header says so. | +| 2026-10-26 | Fixes from review round 3. The change takes in row 32, whose probe test counts the 51Degrees module as linked, and each adapter's order test, and puts a guard beside every stock list, the `ts` tool's included, with a test that pins the tool's list. What a stock build costs is spelled out, being a call to the cloud service on each page the client hints middleware is placed for, a key-value write on each robots.txt request where a deployment adds a robots.txt module, and the README promise of the 51Degrees shared code. The other text that says a stock build holds answers is listed with the change, with a changelog entry for the refused `module = "51degrees"`. Line numbers are those of the current heads, row 31 at `490a4665d` and row 24 at `e66b955a4`, and the design says what the code there does, being the host in each held answer's key, a module's failure logged with each address cut to its scheme and host on row 31, and the adapter robots.txt tests of row 24. The searches give their output from row 44, the open question on a repeated changelog entry, which no row repeats now, is gone, one on falling back to `NoLoadingCache` without a word is added, open question 1 names row 55 and core's `loading-cache.example`, the CI results are linked at their commits, and the counts of the local checks are gone. | +| 2026-10-26 | Fixes from review round 4. Row 32's first case links a second loading cache module, a test builder offering core's `MemoryLoadingCache`, so that the case still shows the table is read. The text the change rewrites adds the integration guide, the 51Degrees client's comments and the client hints middleware's tests. The 51Degrees shared code's loader is named as a caller that captures a request's own services, "stock build" is defined where it is first used, the source link names the commit that `4.5.8` was published from, the test of module names checks the constant whatever features are on, and the decision is placed after the Touchbase. | ## Review log diff --git a/docs/superpowers/specs/2026-10-15-asset-cache-design.md b/docs/superpowers/specs/2026-10-15-asset-cache-design.md index a714f3c6b4..93b2bd8d01 100644 --- a/docs/superpowers/specs/2026-10-15-asset-cache-design.md +++ b/docs/superpowers/specs/2026-10-15-asset-cache-design.md @@ -1,214 +1,455 @@ -# Design Spec: The publisher's assets held through the loading cache +# Design Spec: The publisher's assets held by a module, behind one interface in core -**Status:** Proposed, 2026-10-15. Written after the code, for draft pull request IABTechLab/trusted-server#1375 +**Status:** Proposed, 2026-10-15. Written after the code, for pull request [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375), row 34 of the stack. Rewritten 2026-10-26 for the asset cache as a module, ahead of the code that row 34 changes to follow it. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-10-12-loading-cache-design.md`, `2026-09-15-852-template-and-origin-caching-design.md`, `2026-04-28-multi-backend-asset-proxy-design.md`, `2026-05-19-asset-s3-auth-fastly-io-design.md`, `2026-09-08-1138-per-cookie-template-cache-policy-design.md` -**Related PRs:** IABTechLab/trusted-server#1375, reviewed on jwrosewell/trusted-server#29 (closed) -**Issues:** IABTechLab/trusted-server#1324 -**Last updated:** 2026-10-18 +**Related specs:** `2026-10-12-loading-cache-design.md`, `2026-10-15-module-cache-declarations-design.md`, `2026-10-14-source-modules-design.md`, `2026-10-14-no-modules-crate-design.md`, `2026-10-26-integrations-crate-design.md` (new, added to [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) with this rewrite), `2026-09-15-852-template-and-origin-caching-design.md`, `2026-04-28-multi-backend-asset-proxy-design.md`, `2026-05-19-asset-s3-auth-fastly-io-design.md`, `2026-09-08-1138-per-cookie-template-cache-policy-design.md` +**Related PRs:** IABTechLab/trusted-server#1375, reviewed on [jwrosewell/trusted-server#29](https://github.com/jwrosewell/trusted-server/pull/29) (closed). [IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373) (row 32) lets a module declare the caches it loads through. [IABTechLab/trusted-server#894](https://github.com/IABTechLab/trusted-server/pull/894) is IAB Tech Lab's own draft design for response caching in core +**Issues:** [IABTechLab/trusted-server#1324](https://github.com/IABTechLab/trusted-server/issues/1324) +**Last updated:** 2026-10-26 -> **Why this spec exists.** A file that is not a page, whether the publisher's origin sends it or an asset route fetches it, is asked of its origin on every request and by every instance, so a burst of readers for one file is a burst of fetches. +> **Why this spec exists.** A file that is not a page, whether the publisher's origin sends it or an asset route fetches it, is asked of its origin on every request and by every instance, where no platform cache holds it, so a burst of readers for one file is a burst of fetches. Holding those files is a choice one deployment makes, so the code that holds them belongs in a module, and core offers only the place where a module may answer. -Every design document of the stack is in IABTechLab/trusted-server#1342, including the specs named above. +Every design document of the stack is in IABTechLab/trusted-server#1342, including the specs named above. Line numbers are those of row 34 at `2a449eaf6`, before the change this rewrite describes, unless a row or a commit is named. ## 1. Problem -`handle_publisher_request` in `crates/trusted-server-core/src/publisher.rs` keeps a page as a template in the template cache and sends every other request to the publisher's origin. `handle_asset_proxy_request` in `crates/trusted-server-core/src/proxy.rs` fetches each file of an asset route from the route's origin. Core keeps no copy of either kind of file. A deployment with a host cache in front of core already has its files held there. A deployment with nothing in front of core fetches every file from the origin for every reader, on every instance. On Fastly the platform's readthrough cache answers the origin fetch that core makes, so a file the origin lets that cache hold is already held there (see "What the caches add on Fastly"). +`handle_publisher_request` in `crates/trusted-server-core/src/publisher.rs` keeps a page as a template in the template cache and sends every other request to the publisher's origin. `handle_asset_proxy_request` in `crates/trusted-server-core/src/proxy.rs` fetches each file of an asset route from the route's origin. Before row 34 core kept no copy of either kind of file. A deployment with a host cache in front of core already has its files held there. A deployment with nothing in front of core fetches every file from the origin for every reader, on every instance. On Fastly the platform's readthrough cache answers the origin fetch that core makes, so a file the origin lets that cache hold is already held there (see "What the module adds on Fastly"). -The loading cache that core declares (`2026-10-12-loading-cache-design.md`) loads a missing value once for all the requests that ask for it at the same moment, and gives each of them the value. Each feature that holds answers this way selects its cache in its own `[cache.]` table. The robots.txt answers were the first feature to do so. Assets are the second. +Row 34 as it stands at `2a449eaf6` holds those files in core. It adds 2,771 lines to 10 files of core, among them `asset_cache.rs` (426 lines), `source_cache.rs` (754 lines), the gate and the loader in `publisher.rs` (`handle_publisher_request`, `asset_from_cache`, `load_asset`) and in `proxy.rs` (`handle_asset_proxy_request`, `asset_route_copy_key`), `[cache.assets]` in `settings.rs`, and `declared_caches` in `integrations/mod.rs`, which declares `assets` and `assets-vary` as core's own. A stock build is an adapter or the `ts` tool built from the repository as it ships, with the builders that its own list, its stock list, names. On row 34 a stock build held assets from its first deploy, because every stock list linked one loading cache module. + +At the Touchbase with IAB Tech Lab on 2026-10-23, 51Degrees said it would check that the loading cache could be a module that a deployment links, and asked for advice on whether the publisher asset feature should become a module as well. IAB Tech Lab has its own draft design for response caching in core, IABTechLab/trusted-server#894. As decided by 51Degrees after the Touchbase, the asset cache becomes a module, inside its own row, so that core holds no asset caching code, only one neutral interface, and no stock build links the 51Degrees loading cache (`2026-10-12-loading-cache-design.md`). ## 2. Design -Core holds the origin's response for a file in two loading caches, `assets-vary` and `assets`, and the one table `[cache.assets]` selects both. The publisher path and the asset routes both call `asset_from_cache` in `publisher.rs`. That function answers with a stored copy, with the response its own fetch made, with the failure of the load the request ran or waited on, or with nothing. A caller given the failure answers as it does when its own origin request fails, and a caller that gets nothing asks the origin as it did before this change. +The publisher path and the asset routes each offer a `GET` for a file to the selected asset cache module at the one point where core is about to ask the file's origin. The module answers with a response, with the failure of the origin request it made, or by passing the file back to core. Core holds no copy, no key and no gate of its own. The 51Degrees module, `crates/asset-cache/51degrees`, holds the origin's responses in two loading caches it declares, `51degrees-assets-vary` and `51degrees-assets`, and does what row 34 did in core, as section 2.7 sets out. A deployment that selects no asset cache module has every file asked of its origin, as before row 34. + +### 2.1 The interface in core + +Core declares the module type `asset-cache` in `crates/trusted-server-core/src/asset_cache.rs`, which holds the interface and no caching code. + +```rust +/// A module that may answer for a file before core asks the file's origin. +#[async_trait::async_trait(?Send)] +pub trait AssetCacheModule: Send + Sync { + /// The answer for `file`, at the point core is about to ask its origin. + async fn answer(&self, services: &RuntimeServices, file: AssetFile<'_>) -> AssetAnswer; +} + +/// What core does with the module's answer. +pub enum AssetAnswer { + /// Serves this in place of the origin's response, processed as that + /// response would be. + Serve(Response), + /// The module asked the origin through `AssetFile::origin` and could not + /// reach it, so core answers as it does when its own origin request fails. + OriginFailed(Report), + /// Core asks the origin itself, as it does with no module selected. + AskOrigin, +} + +/// Which of core's two paths offers a file. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum AssetVia { + /// `handle_publisher_request`, for a file the publisher's origin sends. + Publisher, + /// `handle_asset_proxy_request`, for a file an asset route fetches. + Route, +} + +/// A `GET` core is about to send to a file's origin, with what core knows of +/// the reader's request. Core makes it, and a module reads its fields. +#[non_exhaustive] +pub struct AssetFile<'a> { + pub via: AssetVia, + pub carries_authorization: bool, + pub carries_cookie: bool, + pub host: &'a str, + pub scheme: &'a str, + pub is_document: bool, + pub asks_for_origin: bool, + pub origin_url: &'a str, + pub origin_identity: &'a str, + pub origin: OriginRequest, +} + +/// The request core would send to the file's origin, a `GET` with no body. +#[derive(Debug, Clone)] +pub struct OriginRequest { + uri: http::Uri, + headers: http::HeaderMap, + backend_name: String, + cache_intent: PlatformCacheIntent, + stream_response: bool, +} + +impl OriginRequest { + /// The request core would send, built anew for each call. + pub fn platform_request(&self) -> PlatformHttpRequest; + /// The URI the request asks for. + pub fn uri(&self) -> &http::Uri; + /// The headers the request carries, which hold the values `Vary` names. + pub fn headers(&self) -> &http::HeaderMap; +} + +#[cfg(any(test, feature = "test-utils"))] +impl<'a> AssetFile<'a> { + /// A file offered on `via` for `origin_url`, with nothing set that keeps + /// it from a copy, which a module's test changes field by field. + pub fn for_test(via: AssetVia, host: &'a str, origin_url: &'a str, origin: OriginRequest) -> Self; +} + +#[cfg(any(test, feature = "test-utils"))] +impl OriginRequest { + /// A `GET` of `uri` with `headers` to `backend_name`, with the platform's + /// default cache behavior, for a module's tests. + pub fn for_test(uri: http::Uri, headers: http::HeaderMap, backend_name: &str, stream_response: bool) -> Self; +} +``` -**Where the code is.** +`AssetFile`'s fields are public, so a module reads them, and the struct is `#[non_exhaustive]`, so no crate outside core builds one with a struct expression and core can add a field without breaking a module. A module's tests make one with `AssetFile::for_test`, which sets `scheme` to `https`, `origin_identity` to `origin_url` and each flag to the value that keeps nothing from a copy, and then change the public fields they need. `OriginRequest`'s fields are private, so a module can read the request and send it and cannot change what core would send. Both test constructors sit behind core's `test-utils` feature, with the other test doubles. -| File | What it holds | -| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `crates/trusted-server-core/src/source_cache.rs` | `get_or_fetch`, which finds the copy for a request through the record and the copy caches, and the record of which request headers an address varies on | -| `crates/trusted-server-core/src/asset_cache.rs` | `AssetCacheKey`, `AssetCopy` and `AssetEntry`, which say what is stored, how it is written and read, and how a copy is served | -| `crates/trusted-server-core/src/publisher.rs` | The gate in `handle_publisher_request`, then `asset_from_cache`, `load_asset`, `asset_cache_ttl`, `HandBack` and `origin_response_may_be_kept`, which `template_cache_ttl` now calls as well | -| `crates/trusted-server-core/src/proxy.rs` | `asset_route_copy_key`, and the lookup in `handle_asset_proxy_request` | -| `crates/trusted-server-core/src/settings.rs` | `CacheSettings.assets`, `ASSETS_CACHE`, `ASSETS_VARY_CACHE`, `ASSETS_CACHE_OFF`, `CacheSettings::loading_caches`, which now lists both names, and the range check of `[cache.assets]` in `CacheSettings::prepare_runtime` | -| `crates/trusted-server-core/src/integrations/mod.rs` | `declared_caches`, which declares both names as core's own, and `loading_caches_from`, which selects no cache for either name when `[cache.assets]` names `none` | -| `crates/trusted-server-core/src/platform/` | `hash_length_prefixed` in `template_cache.rs`, which `TemplateCacheKey::to_cache_key` and `AssetCacheKey::to_cache_key` share, and `MemoryLoadingCache::held` and `asset_copy_lifetimes` in `test_support.rs` | +**Where core asks.** On the publisher path, `handle_publisher_request` asks the module where row 34 calls `asset_from_cache`. That is after `rewrite_origin_request` has prepared the request for the origin, after any template cache lookup has found no template, and only when no origin request was started early. An origin request starts early only for a navigation, which is always a document (`should_preload_ec_snapshot` requires `is_navigation_request`, and `is_html_document_request` is true for every navigation), so no file the module could answer is lost to it. On an asset route, `handle_asset_proxy_request` asks the module where row 34 calls `asset_from_cache`, after the route's target, its outbound headers and its signing are built. Core offers only a `GET`, the one method a copy can answer, and on a route no request that the platform's image optimizer answers, because the reader is then sent the platform's work and not the origin's response, and the platform holds those answers itself. -**The key.** `AssetCacheKey` has five inputs, and the two callers fill them differently. +**What core hands the module.** `AssetFile` carries what core knows of the file and the reader's request, so the module needs nothing of core's private code. -| Input | Publisher path | Asset route | -| ----------------- | --------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- | -| `url` | The URL asked of the origin, with its query | The route's target URL, after the route's query policy | -| `request_host` | The host the reader asked | The host of the request URI, else its `Host` header | -| `request_scheme` | The scheme the reader asked over | The scheme of the request URI, `https` when it has none | -| `origin_identity` | The publisher's `origin_url` and the `Host` sent to the origin, joined by a NUL byte | The route's `origin_url` and `prefix`, joined by a NUL byte | -| `fingerprint` | `template_fingerprint`, a digest of the build, the script bundle, the public prefix and the serialized settings | Empty | +| Field | Publisher path | Asset route | +| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------- | +| `via` | `AssetVia::Publisher` | `AssetVia::Route` | +| `carries_authorization` | Whether the reader's request carries `Authorization`, which core already reads for the template cache | The same, of the reader's request and not of the request to the route's origin | +| `carries_cookie` | Whether the reader's request carries `Cookie` | The same | +| `host` | The host the reader asked | The host of the request URI, else its `Host` header | +| `scheme` | The scheme the reader asked over | The scheme of the request URI, `https` when it has none | +| `is_document` | `is_html_document_request` on the reader's request | The same | +| `asks_for_origin` | `request_requires_origin`, which is true for a range, a conditional header, a `Cache-Control` or `Pragma` that asks for the origin, state a module left on the request, or a response an integration marked personalized | `request_bypasses_template_cache` on the reader's headers | +| `origin_url` | The URL asked of the origin, with its query | The route's target URL, after the route's query policy | +| `origin_identity` | The publisher's `origin_url` and the `Host` sent to the origin, joined by a NUL byte | The route's `origin_url` and `prefix`, joined by a NUL byte | +| `origin` | The origin request as `rewrite_origin_request` left it | The request to the route's origin, with the route's credentials when it has `auth` | -`AssetCacheKey::to_cache_key` writes a schema version and the five inputs into one SHA-256, each with its length before it, and returns `ts-asset-cache-v1-` and the digest in hex. No input appears in the key. The route's `origin_identity` carries the route's prefix, so two routes to one address keep their own copies. The reader's host and scheme are in the key although the copy is kept as the origin sent it, so two hosts that serve one publisher each keep a copy. +On the publisher path each field is one core already works out for its template cache or its origin request, or whether the reader's request carries a header. On an asset route core works out every field but `via`, `origin_url` and `origin` only to fill `AssetFile`, as row 34 worked out the host, the scheme, the reader's `Authorization`, `asks_for_origin` and `origin_identity` only in `asset_route_copy_key`. -`asset_address` joins that digest to the publisher's prefix (`services.server_key_prefix()`) as a `SourceAddress`. Every key in the store starts with the prefix, so two publishers sharing a store never share a record or a copy. +`asks_for_origin` describes the request as core will send it to the origin. When a middleware needs a file's whole body, core removes a range and the conditional headers that `strip_conditional_and_range_headers` removes before it works the field out, so a reader who asked for a range of such a file is not reported as asking for the origin, because the origin will be asked for the whole file. -An address has one record, in `assets-vary`, and any number of copies, in `assets`. `SourceVary::of` reads the origin's `Vary` into the record as lowercase header names, sorted and each named once. A copy's key is the address and a 128-bit digest (`variant` in `source_cache.rs`) of the request's values for exactly those headers, as RFC 9111 section 4.1 describes. The digest tells apart an absent header, an empty one, and repeated values in a different order. The request headers are those sent to the origin. On the publisher path they are the headers after `rewrite_origin_request` has restricted `Accept-Encoding`. On an asset route they are the headers built in `handle_asset_proxy_request`, which include the route's own credentials when the route has `auth`. +Three rows above row 34 change what core hands the module or how core judges a response, and none of them changes the module. Row 39 ([IABTechLab/trusted-server#1380](https://github.com/IABTechLab/trusted-server/pull/1380), `2026-10-14-source-modules-design.md`) fills `origin_identity` from the source's location, with `location.identity()` on the publisher path and `route_origin_identity` on a route, which adds the `Host` a source sets. Row 46 ([IABTechLab/trusted-server#1389](https://github.com/IABTechLab/trusted-server/pull/1389)) makes `origin_response_may_be_kept` refuse a shared cache header that the platform's adapter declares, and leaves `EDGE_CACHE_HEADER_NAMES` naming only `Surrogate-Control` and `CDN-Cache-Control`, so that one change reaches the template cache and the module together (section 2.2). Row 54 ([IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412)) serves asset routes on every adapter, so from there every adapter offers its routes' files to the module. -**When a stored asset is used.** The publisher path looks for a copy only when every one of these holds, as `request_can_use_asset_copy` in `handle_publisher_request` computes them. Together they are the asset gate. +**Asking the origin.** `AssetFile::origin` is an `OriginRequest`, which owns the prepared request without a body. `OriginRequest::platform_request` builds the request core would send for the file, being a `GET` with the prepared headers to the prepared backend, streaming where the platform offers it, with the cache behavior core would ask for. On the publisher path that is the one `apply_origin_cache_intent` gives, which leaves a request that is not for a document on the platform's default, because only a navigation runs the ad stack, and on an asset route it is the platform's default. It can be called any number of times, and an `OriginRequest` can be cloned and sent across threads, so a module's loader can own one, as a `Loader` owns everything it uses. The module sends the request with the HTTP client of the services it was handed. -1. The method is `GET`. -2. The request is not for a document. `is_html_document_request` reads `Sec-Fetch-Dest`, where `document`, `embed`, `fencedframe`, `frame`, `iframe` and `object` count as documents, and falls back to the navigation check when the header is absent. -3. The request names a host. -4. The request carries no `Authorization` header. -5. The request carries no `Cookie` header, because the origin receives the reader's cookies and nothing shows whether they shaped its answer (see "The cookie rule"). -6. `request_bypasses_template_cache` finds none of `Range`, `If-Range`, `If-Match`, `If-None-Match`, `If-Modified-Since` or `If-Unmodified-Since`, no `Cache-Control` that asks for the origin, being `no-cache`, `no-store`, `min-fresh`, an unreadable value, or a `max-age` that is not zero, and no `Pragma` that is `no-cache` or cannot be read. -7. No module left state on the request, and no integration marked the response as personalized. -8. The loading cache that `[cache.assets]` selects keeps values (`PlatformLoadingCache::keeps_values`). `NoLoadingCache` answers false. A build that links no loading cache module, a build that links several and names no module, and a `[cache.assets]` that names `none` all select it. +**Before and after the origin.** The module answers before core asks the origin, and it sees the origin's answer afterwards by asking the origin itself, through `AssetFile::origin`, inside the load its loading cache joins, so one fetch serves every request waiting on that load. A file the module passes with `AskOrigin` is asked of the origin by core, as with no module selected, and the module does not see that answer, because the module keeps nothing for a file it passed. A second hook called after core's own fetch would not serve, because core's own fetch cannot be joined, so the requests that miss at one moment would each fetch the origin. -An asset route looks for a copy when `asset_route_copy_key` finds the method `GET`, no `Authorization`, none of the conditions in item 6, a host in the request URI or the `Host` header, and a cache that keeps values, and when no image optimizer applies to the request. The route does not check items 2, 5 and 7. A route sends its origin no cookie, so a reader's cookies cannot shape its answer. A request that fails any check is served exactly as before this change. +**What core does with the answer.** `Serve` takes the place of the origin's response, and everything after that point is unchanged, so on the publisher path the response is processed as an origin response is, and on a route `[[cache.asset_rules]]` and the route's response handling apply. `OriginFailed` is answered as core's own origin request failing, with `Failed to proxy request to origin` on the publisher path and `Failed to proxy asset request` on an asset route, both a `502`, so the origin is not asked a second time. `AskOrigin` sends the origin request core prepared, as before row 34. -**What is stored.** `load_asset` fetches the file with `GET` and the prepared request's headers, then tests the response in this order. The first test the response fails ends the checks. The response is then not stored for other readers, and the request whose load fetched it is served that response. +**The selection.** A module supplies its asset cache module through `IntegrationRegistrationBuilder::with_asset_cache_module(name, module)`. `[asset-cache]` is a module type section, read as every section of a type core does not read itself is (`TypeSections` in `settings.rs`), so `module = "51degrees"` runs the builder of `asset-cache.51degrees`. The registry takes the asset cache module only from the registration of the builder that `[asset-cache]` selects, and keeps it as `IntegrationRegistry::asset_cache_module()`. It refuses a registration from any other builder that supplies one, such as a builder a `[shared.]` table or the auction plan runs, naming that builder, and it refuses an `[asset-cache]` section that selects more than one module. Each adapter hands the module to each request's services with `RuntimeServices::with_asset_cache_module`, beside the geo, Edge Cookie and device modules, in `build_per_request_services` in the Fastly adapter's `app.rs` and `services_for_request` in the Axum, Cloudflare and Spin adapters' `app.rs`. Both paths read it with `RuntimeServices::asset_cache_module()`. -1. `Vary` is `*`, names `Cookie`, or cannot be read. -2. `asset_cache_ttl` refuses the response, which happens when the status is not `200`, when the response sets a cookie, when it carries an edge cache header other than `Surrogate-Control`, when its content type is HTML, or when `origin_shared_ttl` finds no usable lifetime. That function refuses `private`, `no-store` and `no-cache`, a repeated or unreadable `max-age` or `s-maxage`, and a response with no lifetime left. Otherwise the lifetime comes from `Surrogate-Control`, else `s-maxage`, else `max-age`, else `Expires` measured from `Date`. The current age is taken off, being the larger of the `Age` header and the age `Date` implies, and `store_seconds` caps the result. The response does not need `public`. -3. `Content-Length` is over 1 MiB (`MAX_ASSET_COPY_BYTES`). -4. A header value is not visible text and so cannot be written down (`AssetCopy::kept_headers`). -5. The body outgrows 1 MiB, or fails, while `read_whole_or_hand_back` reads it. +### 2.2 What stays in core + +1. The interface above, being `AssetCacheModule`, `AssetAnswer`, `AssetVia`, `AssetFile` and `OriginRequest`, with the two constructors for tests behind `test-utils`. +2. The selection, being `with_asset_cache_module`, `IntegrationRegistry::asset_cache_module`, `RuntimeServices::with_asset_cache_module` and `RuntimeServices::asset_cache_module`, and the three lines in each adapter that hand the module on. +3. The two places that ask the module, with the facts in the table above. +4. Two functions of the template cache, made public for the module. `origin_response_may_be_kept` refuses a response whose status is not `200`, that sets a cookie, or that carries a shared cache header core does not read. Row 34 drew it out of `template_cache_ttl`, which calls it, and the module calls it too, so row 46's change to it reaches both callers at once. `origin_shared_ttl` gives how long the origin lets a shared cache keep a response. Both return a new public type, `KeepRefusal`, rather than the template cache's `TemplateCacheBypassReason`, whose sixteen variants and `VaryGap` stay private, and the template cache turns a `KeepRefusal` into its own reason. +5. `MemoryLoadingCache::held` in `platform/test_support.rs`, which a test uses to read the time a held value has left, offered to other crates' tests under `test-utils` with the rest of the test doubles. + +```rust +/// Why an origin's response may not be kept for other readers. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum KeepRefusal { + /// The status is not `200`. + NotOk, + /// The response sets a cookie. + SetsCookie, + /// The response's cache policy keeps it from a shared copy, by `private`, + /// `no-store` or `no-cache`, or by a shared cache header core does not read. + NotShareable, + /// The response's cache policy cannot be read. + MalformedPolicy, + /// The response gives no lifetime, or has none left. + NoLifetime, +} + +pub fn origin_response_may_be_kept(status: StatusCode, headers: &HeaderMap) -> Result<(), KeepRefusal>; +pub fn origin_shared_ttl(headers: &HeaderMap, max_lifetime: Duration) -> Result; +``` + +`origin_response_may_be_kept` gives `NotOk`, `SetsCookie` or `NotShareable`, and `origin_shared_ttl` gives `NotShareable`, `MalformedPolicy` or `NoLifetime`, the three reasons it gave as a `TemplateCacheBypassReason` on row 34. `KeepRefusal` displays as the matching `TemplateCacheBypassReason` does, so the template cache's log lines read as before. + +Core's page path keeps row 34's one change to it. `template_cache_ttl` calls `origin_response_may_be_kept`, which checks the status before the cookie and the shared cache headers, so a page response with a status other than `200` and a fault found by a later check is reported as a status other than `200` (see "What no test covers"). `TemplateCacheKey::to_cache_key` keeps the hashing it has on row 33, because the module hashes its key with a `hash_length_prefixed` of its own. Core's `CacheSettings` gains no `assets` field, core names neither asset cache and reads no `none`, and `declared_caches` declares only `robots-txt` as core's own, as on row 33. + +### 2.3 The module + +The module is the crate `crates/asset-cache/51degrees` (package `trusted-server-asset-cache-51degrees`, module `asset-cache.51degrees`), which 51Degrees maintains. Its manifest names the owner `51Degrees` with the status `vendor owned`, takes the workspace license, Apache-2.0, and depends on core and on the HTTP, hashing and stream crates the moved code already uses. It does not depend on `fiftyone-caching`, because it loads through core's loading cache interface, whichever loading cache module the deployment links. + +| What moves into the module | From | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | +| `MODULE`, `builder()` with `with_module_name(MODULE)` and `with_cache_features(&["51degrees-assets", "51degrees-assets-vary"])`, and the registration that supplies the module | New | +| `AssetCacheKey`, `AssetCopy`, `AssetEntry`, `ASSET_SCHEMA_VERSION` and `MAX_ASSET_COPY_BYTES` | `crates/trusted-server-core/src/asset_cache.rs` | +| `get_or_fetch`, the address record, `SourceVary`, `SourceAddress` and the variant digest, whose tag becomes `ts-asset-variant-v1` | `crates/trusted-server-core/src/source_cache.rs` | +| The publisher path's gate, `request_can_use_asset_copy`, and the route's, `asset_route_copy_key` | `handle_publisher_request` in `publisher.rs`, and `proxy.rs` | +| `asset_from_cache`, `asset_cache`, `asset_address`, `load_asset`, `asset_fetch`, `asset_cache_ttl`, `AssetCacheBypassReason`, `HandBack`, `read_whole_or_hand_back`, `unix_seconds`, `ASSET_DEFAULT_LIFETIMES` and `ASSET_NOT_SHAREABLE_LIFETIME` | `publisher.rs` | +| `hash_length_prefixed` | `platform/template_cache.rs`, where row 34 added it | +| The cache names, `51degrees-assets` and `51degrees-assets-vary`, and the reading of `[cache.51degrees-assets]` | `settings.rs`, where row 34 has `ASSETS_CACHE`, `ASSETS_VARY_CACHE` and `CacheSettings.assets` | +| `asset_copy_lifetimes` | `platform/test_support.rs` | +| The tests, being `asset_cache_tests` in `publisher.rs`, the asset route's holding tests in `proxy.rs`, the tests of `asset_cache.rs` and `source_cache.rs`, and the settings tests of `[cache.assets]` | Core | -A stored copy is the origin's response, headers and body, as the origin sent it, with its `Content-Encoding` left on. `AssetCopy::decoded` is always false. The headers `age`, `connection`, `content-length`, `keep-alive`, `proxy-authenticate`, `proxy-authorization`, `proxy-connection`, `set-cookie`, `te`, `trailer`, `transfer-encoding` and `upgrade` are never written. The copy and the record are held for the lifetime from item 2, less the time the fetch took. The verdict on a response that fails a test is held for 60 seconds (`ASSET_NOT_SHAREABLE_LIFETIME`), as the marker `AssetEntry::NotShareable` under the copy's key, or as the record `NoCopies` when the `Vary` allows no copies, so the requests that follow do not wait on a load that would end the same way. Each of those requests asks the origin itself. A status other than `200` fails test 2, so the body of a `404` is never held, and its verdict is. For those 60 seconds no request that finds the marker stores a copy, even if the origin has begun to answer `200`. +The module carries its own copies of four private helpers of core, rather than core making them public, because each is a few lines and none is an interface core means to keep. Core keeps its own, which its template cache uses. -**What is served.** `AssetCopy::into_response` builds a `200` response from the kept headers, then sets `Age` to the origin's age at the time of the fetch plus the seconds since, and sets `Content-Length`. The kept headers include `Vary`, so a cache further along separates readers as the origin asked. On both paths the copy takes the place of the origin's response, and everything after that point is unchanged. On the publisher path the copy is therefore processed as an origin response is, and the copy is the origin's bytes and not a rewritten body. +- `is_html_content_type`, with the `content_type_contains_ascii_case_insensitive` it calls, 6 lines in all. +- `single_header_value` and `parse_delta_seconds`, 16 lines each, with which the module reads a response's `Age`. +- `std_system_time_now` from `ec/mod.rs`, 6 lines, the clock the module's `unix_seconds` reads, which reads `web_time` as core's does. -**Asset rules.** The `[[cache.asset_rules]]` entries rewrite the `Cache-Control` and edge cache headers of the response a reader is sent, after a copy is read or the origin is asked (`apply_publisher_asset_cache_policy` on the publisher path and the end of `handle_asset_proxy_request` on a route). The asset gate and the copy's lifetime do not read the rules, so the lifetime comes from the origin's response alone, and a rule that gives a reader a longer lifetime does not lengthen the copy's. +The variant digest's tag becomes `ts-asset-variant-v1` in the move, in the form of the key's own `ts-asset-cache` tag. The tag is hashed into every copy's key, so the new tag, with the new cache names (section 2.5) and the key without a fingerprint (section 2.7), changes every key, and a copy kept before simply expires by its own lifetime, never later than `store_seconds`. On Spin, whose `default` store gives an entry no lifetime, those old entries stay in the store until someone deletes them, because nothing reads them again (see "Keys and variants without a limit"). -**One fetch for many requests.** `get_or_fetch` loads the address's record through `assets-vary`. The request that leads that load fetches the origin and writes its own copy under its own values before the record is published, so a waiting request with the same values finds the copy. A waiting request with other values loads its own copy, and the requests waiting on those values share that load. A load runs in the request that started it, as every load of the loading cache does. +`ASSETS_CACHE_OFF`, the `module = "none"` that turned both caches off, and its handling in `loading_caches_from` leave with nothing to replace them, because a deployment turns asset holding off by selecting no asset cache module. -**How far one fetch reaches.** The memory layer shares a load among the requests that share the layer. The stock module's layer lives as long as the application state. Fastly serves one request per instance unless the opt-in reusable sandbox is on, and Cloudflare and Spin build the application state for every request (see `AppState` in the three adapters' `app.rs`), so on those three the memory layer is not shared between requests. Across instances only Fastly makes requests share a load. Only the Fastly adapter registers a value cache, and a lookup in that cache makes every other instance at the same location wait for the instance that fills the key (`crates/trusted-server-adapter-fastly/src/value_cache.rs`), so a missing key is loaded once for each location and not once for the service. Cloudflare and Spin share copies only through the key-value store each adapter declares, `TRUSTED_SERVER_KV` and `default`, which makes no request wait, so each request that misses at the same moment fetches the origin itself and later requests read the stored copy, and Axum has no shared store at all. +The module's load captures the services without the connection (`RuntimeServices::without_connection`), as the `Loader` contract asks. Row 34's `AssetLoad` captures the request's own services. The load uses only the HTTP client, which carries nothing of the connection, so nothing it fetches changes. -**Serving the request whose load fetched.** A response that cannot be shared is not stored for others, yet the request that fetched it must not fetch again. `HandBack` carries the response from the load to that request. `asset_from_cache` makes one `HandBack` for each lookup, `load_asset` puts a response that cannot be shared in the slot by its id, and the lookup ends by taking the slot's response when it finds `AssetEntry::NotShareable` or `Found::NoCopies`. A request that did not lead the load finds the slot empty and asks the origin. Two other cases use the same path. A body that outgrows a copy while it is read is handed back as what was read so far followed by the rest of the stream. A response whose `Vary` no longer matches its record is handled by `copy_loader`, which removes the record, writes the marker with a lifetime of zero seconds, and serves the request what its own fetch returned, so the next request learns the new `Vary`. +**Where the crate is listed.** The workspace `Cargo.toml` names the crate in `members`, as `crates/asset-cache/51degrees`, and in `[workspace.dependencies]`, as `trusted-server-asset-cache-51degrees = { path = "crates/asset-cache/51degrees" }`. In `.cargo/config.toml` the native aliases `clippy-axum` and `test-axum` name `-p trusted-server-asset-cache-51degrees`, as they name the 51Degrees loading cache, and so do `build-fastly`, `check-fastly`, `clippy-fastly` and `test-fastly`, because the module builds for `wasm32-wasip1` with core's dependencies alone and its tests ran under Viceroy, Fastly's local runtime, as core's tests until this rewrite. No adapter and not the `ts` tool depends on the crate. -**Configuration.** One optional table selects and sizes both caches. +**No stock build links the module.** No adapter's stock list and no list of the `ts` tool names the module, as none names the 51Degrees loading cache module (`2026-10-12-loading-cache-design.md`), and the module's caches would keep nothing in a stock build in any case, because a stock build links no loading cache module. A deployment that wants assets held links a loading cache module and this module, and hands both builders to its adapter, through `run_with` on Fastly or `routes_with_registrations` on Axum, Cloudflare and Spin. Any loading cache module serves, because the module loads through core's interface. The open questions ask whether the stock lists should offer the module. + +After row 56 moves every module crate into one crate (`2026-10-26-integrations-crate-design.md`), the module's code is `src/asset-cache/51degrees/` in `crates/trusted-server-integrations`, behind the Cargo feature `asset-cache-51degrees`, which no stock adapter turns on, as the 51Degrees loading cache sits behind `loading-cache-51degrees`. + +### 2.4 Selecting the module ```toml -[cache.assets] +[asset-cache] module = "51degrees" + +[cache.51degrees-assets] memory_seconds = 5 store_seconds = 3600 store_idle_seconds = 86400 ``` -The configuration guide, `docs/guide/configuration.md`, lists the same fields under Loading caches. +`[asset-cache]` names the module by its short name, `51degrees`, or in full, `asset-cache.51degrees`, as every module section does. A section that names a module the build does not link is refused when the adapter builds its registry (`check_section_selections` in `registry.rs`), as every module selection is. Deploy validation does not make that check, because a crate the `ts` tool never links may supply the name. The module reads no table of its own, so it refuses an `[asset-cache.51degrees]` table, as each stock module refuses a setting it does not know. + +### 2.5 The two caches it declares + +`builder()` declares `51degrees-assets` and `51degrees-assets-vary` with `with_cache_features` (row 32, `2026-10-15-module-cache-declarations-design.md`), following the form row 32 asks for, the module's short name and then what the cache holds, as row 44's `51degrees-answers` does. A build that links the module accepts `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]` tables, and a build without it refuses each as a table naming a cache no linked module declares, by the deploy check and when the state is built. A `[cache.assets]` table written for row 34 now names a cache that no module declares, so every build refuses it, and section 2.14 says how such a document changes. + +Each table selects the loading cache module of its own cache, under row 32's rule. A table that names a module takes it. A table that names no module, and a declared cache with no table, take the one loading cache module the build links, and hold nothing when the build links none or several. A table that names a module no linked crate offers is refused. Before this rewrite `[cache.assets]` selected the loading cache module of both caches, and `[cache.assets-vary]` could replace it for the record cache. Now each table selects its own, so a build that links two or more loading cache modules, and names one in `[cache.51degrees-assets]`, names it in `[cache.51degrees-assets-vary]` as well. A build that links one loading cache module needs neither table. + +The module sizes both caches from `[cache.51degrees-assets]`, reading `settings.cache.declared` with `LoadingCacheConfig::lifetimes` and `ASSET_DEFAULT_LIFETIMES` for what the table leaves unset, as row 44's 51Degrees engine cache sizes `51degrees-answers`. Core's `prepare_runtime` checks the ranges of both tables, as it does for every declared table. The lifetimes of `[cache.51degrees-assets-vary]` are checked and not used, as those of `[cache.assets-vary]` were on row 34. The memory layer holds at most 256 records and 32 copies, literals in the module's `asset_cache`. + +`none` is no longer read. A table that names `none` is refused, as one naming any module no linked crate offers is. -| Field | Default | Range | Meaning | -| -------------------- | ---------------------------------------------------------------------- | ------------- | -------------------------------------------------------------------------------- | -| `module` | The one module the build links, and none when it links none or several | | The loading cache module that holds both caches, or `none`, which turns both off | -| `memory_seconds` | `5` | 0 to 31536000 | How long a copy in process memory is used. Zero keeps none | -| `store_seconds` | `3600`, an hour | 1 to 31536000 | The longest a copy in the shared store is used | -| `store_idle_seconds` | `86400`, a day | 1 to 31536000 | How long a copy in the shared store lasts unused | +### 2.6 With no module selected -The defaults are `ASSET_DEFAULT_LIFETIMES` in `publisher.rs`. The memory layer holds at most 256 records and 32 copies. Those two counts are literals in `fn asset_cache` in `publisher.rs`, the limit of 1 MiB on a copy's body is the constant `MAX_ASSET_COPY_BYTES` in `asset_cache.rs`, and none of the three is a setting. `LoadingCacheConfig` refuses a field it does not know. +A deployment that links no asset cache module, or links one and has no `[asset-cache]` section, gets every file asked of its origin, on the publisher path and on every asset route, as before row 34. Each path finds no module in the request's services and sends the origin request it prepared, building no `AssetFile` and no key, so the request the origin receives, and the response the reader is sent, are row 33's. A stock build is such a deployment. A deployment that selects the module in a build that links no loading cache module gets the same, because the module's gate finds that its caches keep nothing and answers `AskOrigin`. -`loading_caches_from` in `integrations/mod.rs` makes the selection. `CacheSettings::loading_caches` lists `robots-txt`, then `assets` and `assets-vary`, both with the settings of `[cache.assets]`, then the tables modules declare. A table that names a module takes that module, by short or full name. A table that names no module, and an absent table, take the one loading cache module the build links, and hold nothing when the build links none or several. A table that names a module no linked crate offers is refused when the service starts, with the modules the build has. `[cache.assets]` may also name `none`, which selects no cache for either name, so `loading_cache_for` gives both `NoLoadingCache`, item 8 of the asset gate fails, and both paths ask the origin for every request as they did before this change. Only `[cache.assets]` reads `none` this way, and another table that names `none` is refused as naming a module no linked crate offers. `declared_caches` declares `assets` and `assets-vary` as core's own, so a module that declares either name is refused, and the error that lists a build's caches now reads `robots-txt, assets, assets-vary` first. +### 2.7 What the module does -The range check, `LoadingCacheConfig::check`, runs in `CacheSettings::prepare_runtime` for `[cache.robots-txt]`, for `[cache.assets]` and for each table a module declares, so settings with a lifetime outside the ranges in the table do not load, and the error names the table and the field, for example `[cache.assets] store_seconds is 0, and must be from 1 to 31536000`. There is no `[cache.assets-vary]` field. A table of that name is read as one a module declares. The deploy check accepts it, because `assets-vary` is a name core declares, and `prepare_runtime` range checks its lifetimes. Its `module`, when it names one, replaces the module `[cache.assets]` selected for the record cache, because `loading_caches_from` handles the table after the entry made from `[cache.assets]` and the later entry wins, unless `[cache.assets]` names `none`, which leaves the record cache unselected whatever the table names. Its lifetimes are never used, because `asset_cache` builds both caches from `[cache.assets]`. +Everything in this section is row 34's behavior, now the module's, apart from four changes this rewrite makes, being the caches' names, the gate's eighth item, the key without a fingerprint and the variant digest's new tag. Where the module reads a fact, it is the field of `AssetFile` named in section 2.1. -**How each adapter takes it.** No adapter code changes. Each adapter calls `loading_caches_from` when it builds its application state and hands the result to each request's services, so the selection reaches `asset_from_cache` through `RuntimeServices::loading_cache_for`. +**The key.** The module builds `AssetCacheKey` from four of the facts core hands it, filled differently on the two paths. -| Adapter | Where the selection is made and handed on | Shared store the stock module uses | How the origin body arrives | -| ---------- | ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ------------------------------------- | -| Fastly | `crates/trusted-server-adapter-fastly/src/app.rs`, handed on with `.loading_caches(...)` | Fastly's cache, through `FastlyValueCache` | Streamed, and read up to 1 MiB | -| Axum | `crates/trusted-server-adapter-axum/src/app.rs`, handed on with `.with_loading_caches(...)` | None. The adapter's key-value store is `UnavailableKvStore`, so only the memory layer holds a copy | Whole, as the adapter does not stream | -| Cloudflare | `crates/trusted-server-adapter-cloudflare/src/app.rs`, with `.with_loading_caches(...)` | The `TRUSTED_SERVER_KV` namespace, which the adapter declares, so a Worker without the binding fails every request | Whole, as the adapter does not stream | -| Spin | `crates/trusted-server-adapter-spin/src/app.rs`, handed on with `.with_loading_caches(...)` | The `default` key-value store, which the adapter declares and which gives an entry no lifetime | Whole, as the adapter does not stream | +| Input | Publisher path | Asset route | +| ----------------- | ------------------------------------------------------------------------------------ | ----------------------------------------------------------- | +| `url` | The URL asked of the origin, with its query | The route's target URL, after the route's query policy | +| `request_host` | The host the reader asked | The host of the request URI, else its `Host` header | +| `request_scheme` | The scheme the reader asked over | The scheme of the request URI, `https` when it has none | +| `origin_identity` | The publisher's `origin_url` and the `Host` sent to the origin, joined by a NUL byte | The route's `origin_url` and `prefix`, joined by a NUL byte | -The stock module, `crates/loading-cache/51degrees`, uses a host's value cache when the services carry one and the key-value store otherwise (`through_store` in `stores.rs`). Only the Fastly adapter registers a value cache. The module's memory layer lives as long as the application state, which is one request on Cloudflare and Spin. Only the Fastly adapter overrides `supports_streaming_responses`, so the others hand `load_asset` a body that is already whole. +`AssetCacheKey::to_cache_key` hashes the tag `ts-asset-cache`, a schema version and the four inputs into one SHA-256, each with its length before it, and returns `ts-asset-cache-v1-` and the digest in hex. No input appears in the key. The route's `origin_identity` carries the route's prefix, so two routes to one address keep their own copies. The reader's host and scheme are in the key although the copy is kept as the origin sent it, so two hosts that serve one publisher each keep a copy. -**What the caches add on Fastly.** The origin fetch asks for the platform's default cache behavior (`PlatformCacheIntent::Default`). The Fastly adapter leaves the request alone for that intent (`apply_fastly_cache_intent`), so Fastly's readthrough cache stays on, and `apply_origin_cache_intent` keeps subresources on that default. Fastly's own cache therefore already holds a subresource for the lifetime the origin gives it, and a load can be answered from that cache and not from the origin. The asset caches add a second copy, held in the value cache for at most `store_seconds`. This change makes no measurement of the gain over the platform's cache on Fastly. The Cloudflare adapter likewise leaves the Workers runtime's default cache behavior in place for that intent (`outbound_cache_mode`). +Row 34's key also held `template_fingerprint`, a digest of the build, the script bundle, the public prefix, each middleware's name and template key, and the serialized settings, so that a build that rewrote bodies differently would never serve a copy an earlier build made. A copy is the origin's bytes and never a rewritten body, because `AssetCopy::decoded` is always false and the fetch phase runs on each reader's copy, so the module leaves the digest out, and the later change that keeps a rewritten body brings it back (section 3). A copy therefore outlives a new build and a change to the settings, as a route's copy already did. -**Consent and privacy.** The asset caches read no consent signal and set none, and they store no identifier of a reader. They do store the origin's response to the request that led a load, which can hold whatever the origin made for that reader from the headers it received (see "The cookie rule"). +`asset_address` joins that digest to the publisher's prefix (`services.server_key_prefix()`) as a `SourceAddress`. Every key in the store starts with the prefix, so two publishers sharing a store never share a record or a copy. + +An address has one record, in `51degrees-assets-vary`, and any number of copies, in `51degrees-assets`. `SourceVary::of` reads the origin's `Vary` into the record as lowercase header names, sorted and each named once. A copy's key is the address and a 128-bit digest (`variant` in the module's `source_cache.rs`, which hashes the tag `ts-asset-variant-v1` first) of the request's values for exactly those headers, as RFC 9111 section 4.1 describes. The digest tells apart an absent header, an empty one, and repeated values in a different order. The request headers are those of the origin request core hands the module. On the publisher path they are the headers after `rewrite_origin_request` has restricted `Accept-Encoding`. On an asset route they are the headers built in `handle_asset_proxy_request`, which include the route's own credentials when the route has `auth`. + +**When a stored asset is used.** On the publisher path the module looks for a copy only when every one of these holds. Together they are the asset gate. + +1. The method is `GET`, which core makes sure of by offering no other method. +2. The request is not for a document. Core computes `is_document` with `is_html_document_request`, which reads `Sec-Fetch-Dest`, where `document`, `embed`, `fencedframe`, `frame`, `iframe` and `object` count as documents, and falls back to the navigation check when the header is absent. +3. The request names a host. +4. `carries_authorization` is false, so the reader's request carries no `Authorization`. +5. `carries_cookie` is false, so the reader's request carries no `Cookie`, because the origin receives the reader's cookies and nothing shows whether they shaped its answer (see "The cookie rule"). +6. `asks_for_origin` is false. Core sets it on the publisher path when `request_bypasses_template_cache` finds, in the request as core will send it (section 2.1), one of `Range`, `If-Range`, `If-Match`, `If-None-Match`, `If-Modified-Since` or `If-Unmodified-Since`, a `Cache-Control` that asks for the origin, being `no-cache`, `no-store`, `min-fresh`, an unreadable value, or a `max-age` that is not zero, or a `Pragma` that is `no-cache` or cannot be read. +7. `asks_for_origin` is also set when a module left state on the request or an integration marked the response as personalized, so that request is served by the origin as well. +8. Both of the module's caches keep values (`PlatformLoadingCache::keeps_values` of `51degrees-assets` and of `51degrees-assets-vary`). `NoLoadingCache` answers false, and a build that links no loading cache module, or links several and names none in the table, gives it. Row 34 checked the copy cache alone, because one table selected both caches. Now that each table selects its own (section 2.5), a record cache that keeps nothing can stand beside a copy cache that keeps values, and then every request runs the record's load, which fetches the origin, so no fetch is saved. The module therefore asks the origin directly in that case. + +On an asset route the module looks for a copy when the method is `GET`, as core makes sure, `carries_authorization` is false, `asks_for_origin` is false, which core sets on a route from the checks in item 6 alone, the request names a host, and both caches keep values. The route does not check items 2, 5 and 7, as on row 34. A route sends its origin no cookie, so a reader's cookies cannot shape its answer. A request that fails any check is answered `AskOrigin` and served exactly as before row 34. + +**What is stored.** `load_asset` asks the origin for the file through the `OriginRequest`, then tests the response in this order. The first test the response fails ends the checks. The response is then not stored for other readers, and the request whose load fetched it is served that response. + +1. `Vary` is `*`, names `Cookie`, or cannot be read. +2. `asset_cache_ttl` refuses the response, which happens when core's `origin_response_may_be_kept` refuses it, for a status other than `200`, a cookie it sets, or a shared cache header other than `Surrogate-Control` that core does not read, when its content type is HTML, or when core's `origin_shared_ttl` finds no usable lifetime. The shared cache headers refused are `Fastly-Surrogate-Control`, `CDN-Cache-Control` and `Cloudflare-CDN-Cache-Control` on row 34, and `CDN-Cache-Control` and those the platform's adapter declares from row 46. `origin_shared_ttl` refuses `private`, `no-store` and `no-cache`, a repeated or unreadable `max-age` or `s-maxage`, and a response with no lifetime left. Otherwise the lifetime comes from `Surrogate-Control`, else `s-maxage`, else `max-age`, else `Expires` measured from `Date`. The current age is taken off, being the larger of the `Age` header and the age `Date` implies, and `store_seconds` caps the result. The response does not need `public`. +3. `Content-Length` is over 1 MiB (`MAX_ASSET_COPY_BYTES`). +4. A header value is not visible text and so cannot be written down (`AssetCopy::kept_headers`). +5. The body outgrows 1 MiB, or fails, while `read_whole_or_hand_back` reads it. + +A stored copy is the origin's response, headers and body, as the origin sent it, with its `Content-Encoding` left on. `AssetCopy::decoded` is always false. The headers `age`, `connection`, `content-length`, `keep-alive`, `proxy-authenticate`, `proxy-authorization`, `proxy-connection`, `set-cookie`, `te`, `trailer`, `transfer-encoding` and `upgrade` are never written. The copy and the record are held for the lifetime from item 2, less the time between the response's arrival and the end of reading its body. The verdict on a response that fails a test is held for 60 seconds (`ASSET_NOT_SHAREABLE_LIFETIME`), as the marker `AssetEntry::NotShareable` under the copy's key, or as the record `NoCopies` when the `Vary` allows no copies, so the requests that follow do not wait on a load that would end the same way. Each of those requests asks the origin itself. A status other than `200` fails test 2, so the body of a `404` is never held, and its verdict is. For those 60 seconds no request that finds the marker stores a copy, even if the origin has begun to answer `200`. + +**What is served.** `AssetCopy::into_response` builds a `200` response from the kept headers, then sets `Age` to the origin's age at the time of the fetch plus the seconds since, and sets `Content-Length`. The kept headers include `Vary`, so a cache further along separates readers as the origin asked. On both paths the module answers `Serve` with the copy, which takes the place of the origin's response, and everything after that point is unchanged. On the publisher path the copy is therefore processed as an origin response is, and the copy is the origin's bytes and not a rewritten body. + +**Asset rules.** The `[[cache.asset_rules]]` entries rewrite the `Cache-Control` and edge cache headers of the response a reader is sent, after the module answers or the origin is asked (`apply_publisher_asset_cache_policy` on the publisher path and the end of `handle_asset_proxy_request` on a route). Core applies them, and the module reads none of them, so a copy's lifetime comes from the origin's response alone, and a rule that gives a reader a longer lifetime does not lengthen the copy's. + +**One fetch for many requests.** `get_or_fetch`, whose one caller is `asset_from_cache` on both paths, loads the address's record through `51degrees-assets-vary`. The request that leads that load fetches the origin and writes its own copy under its own values before the record is published, so a waiting request with the same values finds the copy. A waiting request with other values loads its own copy, and the requests waiting on those values share that load. A load runs in the request that started it, as every load of the loading cache does. + +**How far one fetch reaches.** The memory layer shares a load among the requests that share the layer. The 51Degrees loading cache module's layer lives as long as the application state. Fastly serves one request per instance unless the opt-in reusable sandbox is on, and Cloudflare and Spin build the application state for every request (see `AppState` in the three adapters' `app.rs`), so on those three the memory layer is not shared between requests. Across instances only Fastly makes requests share a load. Only the Fastly adapter registers a value cache, and a lookup in that cache makes every other instance at the same location wait for the instance that fills the key (`crates/trusted-server-adapter-fastly/src/value_cache.rs`), so a missing key is loaded once for each location and not once for the service. Cloudflare and Spin share copies only through the key-value store each adapter declares, `TRUSTED_SERVER_KV` and `default`, which makes no request wait, so each request that misses at the same moment fetches the origin itself and later requests read the stored copy, and Axum has no shared store at all. + +**Serving the request whose load fetched.** A response that cannot be shared is not stored for others, yet the request that fetched it must not fetch again. `HandBack` carries the response from the load to that request. `asset_from_cache` makes one `HandBack` for each lookup, `load_asset` puts a response that cannot be shared in the slot by its id, and the lookup ends by taking the slot's response when it finds `AssetEntry::NotShareable` or `Found::NoCopies`, which the module answers as `Serve`. A request that did not lead the load finds the slot empty and is answered `AskOrigin`, so core asks the origin. Two other cases use the same path. A body that outgrows a copy while it is read is handed back as what was read so far followed by the rest of the stream. A response whose `Vary` no longer matches its record is handled by `copy_loader`, which removes the record, writes the marker with a lifetime of zero seconds, and serves the request what its own fetch returned, so the next request learns the new `Vary`. + +### 2.8 Configuration + +| Table | Field | Default | Range | Meaning | +| ------------------------------- | -------------------- | ---------------------------------------------------------------------- | ------------- | ---------------------------------------------------------------------- | +| `[asset-cache]` | `module` | None, so no file is held | | The asset cache module, `51degrees` or `asset-cache.51degrees` | +| `[cache.51degrees-assets]` | `module` | The one module the build links, and none when it links none or several | | The loading cache module that holds the copies | +| `[cache.51degrees-assets]` | `memory_seconds` | `5` | 0 to 31536000 | How long a copy or a record in process memory is used. Zero keeps none | +| `[cache.51degrees-assets]` | `store_seconds` | `3600`, an hour | 1 to 31536000 | The longest a copy or a record in the shared store is used | +| `[cache.51degrees-assets]` | `store_idle_seconds` | `86400`, a day | 1 to 31536000 | How long a copy or a record in the shared store lasts unused | +| `[cache.51degrees-assets-vary]` | `module` | The one module the build links, and none when it links none or several | | The loading cache module that holds the records | + +The two `[cache.]` tables exist only in a build that links the module. `LoadingCacheConfig` refuses a field it does not know, and the range check names the table and the field, for example `[cache.51degrees-assets] store_seconds is 0, and must be from 1 to 31536000`. The configuration guide, `docs/guide/configuration.md`, lists the same fields under Loading caches, where row 34 adds them, and its text follows this rewrite. + +### 2.9 The adapters + +Each adapter calls `loading_caches_from` with the builders it links when it builds its application state, which include the module's when a deployment adds it, so the module's two caches are selected as every declared cache is. Each adapter also hands the selected asset cache module to each request's services, as section 2.1 says. On row 34 only the Fastly adapter serves asset routes, so on Axum, Cloudflare and Spin the module answers for the publisher path alone until row 54. + +| Adapter | Where the selections are handed on | Shared store the 51Degrees loading cache module uses | How the origin body arrives | +| ---------- | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ------------------------------------- | +| Fastly | `build_per_request_services` in `crates/trusted-server-adapter-fastly/src/app.rs`, with `.loading_caches(...)` | Fastly's cache, through `FastlyValueCache` | Streamed, and read up to 1 MiB | +| Axum | `services_for_request` in `crates/trusted-server-adapter-axum/src/app.rs`, with `.with_loading_caches(...)` | None. The adapter's key-value store is `UnavailableKvStore`, so only the memory layer holds a copy | Whole, as the adapter does not stream | +| Cloudflare | `services_for_request` in `crates/trusted-server-adapter-cloudflare/src/app.rs`, with `.with_loading_caches(...)` | The `TRUSTED_SERVER_KV` namespace, which the adapter declares, so a Worker without the binding fails every request | Whole, as the adapter does not stream | +| Spin | `services_for_request` in `crates/trusted-server-adapter-spin/src/app.rs`, with `.with_loading_caches(...)` | The `default` key-value store, which the adapter declares and which gives an entry no lifetime | Whole, as the adapter does not stream | + +The 51Degrees loading cache module uses a host's value cache when the services carry one and the key-value store otherwise (`through_store` in its `stores.rs`). Only the Fastly adapter registers a value cache. The memory layer lives as long as the application state, which is one request on Cloudflare and Spin. Only the Fastly adapter overrides `supports_streaming_responses`, so the others hand `load_asset` a body that is already whole. + +**What the module adds on Fastly.** The origin fetch asks for the platform's default cache behavior (`PlatformCacheIntent::Default`). The Fastly adapter leaves the request alone for that intent (`apply_fastly_cache_intent`), so Fastly's readthrough cache stays on, and `apply_origin_cache_intent` keeps subresources on that default. Fastly's own cache therefore already holds a subresource for the lifetime the origin gives it, and a load can be answered from that cache and not from the origin. The module adds a second copy, held in the value cache for at most `store_seconds`. This change makes no measurement of the gain over the platform's cache on Fastly. The Cloudflare adapter likewise leaves the Workers runtime's default cache behavior in place for that intent (`outbound_cache_mode`). + +### 2.10 Beside IAB Tech Lab's design for response caching + +IABTechLab/trusted-server#894 (Christian Pavilonis, "Design TS-owned dynamic response caching", a draft holding one design document and no code) holds the publisher's origin HTML in core, for a document navigation that a configured rule matches, through Fastly's HTTP readthrough cache. It is off by default, its rules carry `enabled = false`, it runs on Fastly first, and Axum, Cloudflare and Spin keep the uncached path. It tags each object with the surrogate key `ts-tpl-v1`, a key for all of its objects, and with keys for the site, the policy, the path and any configured groups. It leaves responses that are not documents, such as React Server Component and JSON answers, to later phases, and lists replacing the static asset cache rules among its non-goals. The module sits beside it in six ways. + +1. **They hold different files.** The module answers no document (gate item 2) and keeps no HTML (test 2 of "What is stored"), and #894's first phase holds documents alone, so no file is held by both. +2. **They sit at different places.** #894 rides on core's own origin request, as cache settings on `PlatformHttpRequest` that the Fastly client turns into cache options. The module answers before core's origin request is made, so a file the module answers never reaches that request, and a file it passes reaches it as before, where #894 applies only if the file is a document. +3. **They use different Fastly caches.** #894 chose Fastly's HTTP cache over Fastly's lower-level Core Cache API, because Core Cache would need Trusted Server to define and version four things of its own, being an envelope for a response's status and headers, a writer for inserting entries, a protocol for revalidation, and a way to stream bodies on every runtime. With the 51Degrees loading cache linked, the module holds its copies in Fastly's Core Cache (`fastly::cache::core`, through the adapter's `FastlyValueCache`). It carries the envelope itself, `AssetEntry` with its schema version, outside core, writes through the loading cache's stores, revalidates nothing, because a copy that has ended is fetched again whole, and reads a body whole, up to 1 MiB, rather than streaming it into the store. Each purges by the surrogate keys it writes, #894 by `ts-tpl-v1` and its site, policy, path and group keys, and the Fastly adapter's value cache by `vc-` and a hash of the entry's key, so neither purge reaches the other's entries, and `ts cache purge` reaches no copy the module keeps (see "Cost and limits"). With another loading cache module linked, the copies are wherever that module keeps them, and this item does not apply. +4. **They treat cookies differently.** #894 makes a request that carries cookies eligible. When none of the cookies a rule names for a bypass is present, it removes every `Cookie` field from the canonical origin request, so unknown cookies are stripped and not keyed, and enabling a rule is the operator's assertion that serving the anonymous answer to such a reader is acceptable. The module never changes the origin request, and on the publisher path a request that carries a `Cookie` header neither reads nor leads a copy (see "The cookie rule" and open question 1). +5. **Both are off unless a deployment chooses them.** #894's rules default to `enabled = false`. The module runs only where a deployment links it and selects it in `[asset-cache]`. +6. **They differ on the adapters other than Fastly.** #894 keeps the uncached path there and says an in-process cache would give misleading parity, because its eviction, isolation, joining of loads and purge would differ. The module holds copies on every adapter it is selected on, in the stores section 2.9 lists. A deployment that shares #894's view selects the module only for its Fastly service. + +If a later phase of #894 holds responses that are not documents in core, a file could be offered to both, and the module, asked first, would answer before core's own cache saw the request. The open questions ask how the two should divide such a file. + +### 2.11 Consent and privacy + +The module reads no consent signal and sets none, and it stores no identifier of a reader. It does store the origin's response to the request that led a load, which can hold whatever the origin made for that reader from the headers it received (see "The cookie rule"). - **What is stored** is the origin's response for a `200`, the record of header names, and the marker. No reader's request is stored. The values of the request headers that `Vary` names enter the key only as the 128-bit digest, and no URL, host or header value appears in a key. - **What is never stored for another reader** is a response that sets a cookie, a response whose policy is `private`, `no-store` or `no-cache` or gives no lifetime, a response that varies on `*` or on `Cookie`, and any response to a request that carries `Authorization`, a conditional header, a range, or a `Cache-Control` or `Pragma` that asks for the origin, and on the publisher path any response to a request that carries a `Cookie` header, state a module left or a personalized marker. Those requests also never read a copy. -- **The cookie rule.** No response that sets a cookie, and no answer to a request whose cookies may have shaped it, is stored and served to another reader. On the publisher path `rewrite_origin_request` removes `fastly-ssl`, restricts `Accept-Encoding` and sets `Host`, and `handle_publisher_request` does not remove `Cookie`, so the origin receives the reader's cookies and the cache cannot see whether they shaped the answer. A request that carries a `Cookie` header therefore fails item 5 of the asset gate, so it never reads a copy and never leads a load, and it asks the origin as it did before this change. A response that sets a cookie fails test 2 of "What is stored", so it is served to the request whose load fetched it and to no other, and `set-cookie` is never written with a copy. An asset route sends its origin no cookie (next point), so a route's answer cannot depend on one, and a request to a route may read and lead copies whatever cookies it carries. -- **An asset route sends no cookie, but it sends other headers.** `ASSET_PROXY_FORWARD_HEADERS` lists the eleven request headers an asset origin receives, and `Cookie` and `Authorization` are not among them. An answer from a route's origin therefore cannot depend on a cookie that origin never received. The list does hold `User-Agent`, `Accept`, `Accept-Encoding`, `Accept-Language` and `Referer`. An origin that answers differently on one of those five without naming it in `Vary`, and gives that answer a lifetime another reader may share, has one reader's answer kept and served to others until the lifetime ends. A shared copy opens no path that the route keeps closed, because any reader who asks the route for a path is already given the origin's answer for that path. -- **Other headers on the publisher path.** The origin receives the reader's other headers as well. The key holds only the headers that `Vary` names, so an origin that answers differently on another header without naming it in `Vary`, and gives that answer a lifetime another reader may share, has one reader's answer kept and served to others until the lifetime ends. No setting of this change lets a publisher declare such an input (see section 3). -- **The page path's cookie settings are not read.** Three settings in `[creative_opportunities]` change the page path's cookie rule, being `template_cache_key_cookies`, `template_cache_bypass_cookies` and `origin_is_cookie_independent` (`2026-09-08-1138-per-cookie-template-cache-policy-design.md`), and `evaluate_cookie_policy` in `crates/trusted-server-core/src/cookies/template_cache_policy.rs` applies them. Each describes the origin's HTML, so the asset gate reads none of them. A publisher who sets `origin_is_cookie_independent` to say that cookies do not change its HTML still has every cookie-bearing asset request on the publisher path fetched for itself (see Open questions). -- **The cost.** Browsers send first-party cookies on subresources, which the comment in `apply_origin_cache_intent` in `publisher.rs` says is most repeat-visitor asset traffic. On the publisher path the asset caches therefore serve only requests that carry no cookie, while an asset route serves its eligible requests whatever cookies they carry. -- **On by default.** Item 3 of the rollout starts the cache on a stock build with no change to its settings, and `[cache.assets] module = "none"` turns it off (item 4). A build that links the loading cache module applies the cookie rule to every eligible asset request from its first deploy. -- **Logs.** The lines `asset_cache bypass`, `asset_cache stored` and `asset_cache invalid entry` carry a fixed reason and a byte count, and no URL, key or header. The line `asset_cache load failed` carries the error text of the failed fetch, which includes the platform's error report as the platform formats it, and this change does not filter that text. - -**Failure handling.** A load that fails asking the origin ends with every request that ran or waited on it answering with that failure. Each other failure ends with the caller asking the origin, as before this change, or with the reader served the response the request's own fetch made. - -- **The origin cannot be reached during a load.** `load_asset` returns a `LoadError`. Every request waiting on that load receives it and nothing is stored. `asset_from_cache` logs `asset_cache load failed` at warn level and hands the failure to its caller, which answers as it does when its own origin request fails, with `Failed to proxy request to origin` on the publisher path and `Failed to proxy asset request` on an asset route, both a `502`. A failing origin is therefore asked once for all the requests that share the load, and the next request loads again. -- **A stored entry cannot be read.** `AssetEntry::decode` refuses bytes that are short, unreadable or written in another format. A record this build did not write is refused in the same way. In both cases the lookup logs at warn level, removes the entry, and the request asks the origin. A format version is in the key, in the entry and in the record, so a build that changes the format does not read the old entries. +- **The cookie rule.** No response that sets a cookie, and no answer to a request whose cookies may have shaped it, is stored and served to another reader. On the publisher path `rewrite_origin_request` restricts `Accept-Encoding`, removes `fastly-ssl` and the host's TLS and HTTP/2 signal headers, sets the URI to the origin's and sets `Host`, and `handle_publisher_request` does not remove `Cookie`, so the origin receives the reader's cookies and the module cannot see whether they shaped the answer. A request that carries a `Cookie` header therefore fails item 5 of the asset gate, so it never reads a copy and never leads a load, and core asks the origin as it did before row 34. A response that sets a cookie fails test 2 of "What is stored", so it is served to the request whose load fetched it and to no other, and `set-cookie` is never written with a copy. An asset route sends its origin no cookie (next point), so a route's answer cannot depend on one, and a request to a route may read and lead copies whatever cookies it carries. +- **An asset route sends no cookie, but it sends other headers.** `ASSET_PROXY_FORWARD_HEADERS` in core lists the eleven request headers an asset origin receives, and `Cookie` and `Authorization` are not among them. An answer from a route's origin therefore cannot depend on a cookie that origin never received. The list does hold `User-Agent`, `Accept`, `Accept-Encoding`, `Accept-Language` and `Referer`. An origin that answers differently on one of those five without naming it in `Vary`, and gives that answer a lifetime another reader may share, has one reader's answer kept and served to others until the lifetime ends. A shared copy opens no path that the route keeps closed, because any reader who asks the route for a path is already given the origin's answer for that path. +- **Other headers on the publisher path.** The origin receives the reader's other headers as well. The key holds only the headers that `Vary` names, so an origin that answers differently on another header without naming it in `Vary`, and gives that answer a lifetime another reader may share, has one reader's answer kept and served to others until the lifetime ends. No setting lets a publisher declare such an input (see section 3). +- **The page path's cookie settings are not read.** Three settings in `[creative_opportunities]` change the page path's cookie rule, being `template_cache_key_cookies`, `template_cache_bypass_cookies` and `origin_is_cookie_independent` (`2026-09-08-1138-per-cookie-template-cache-policy-design.md`), and `evaluate_cookie_policy` in `crates/trusted-server-core/src/cookies/template_cache_policy.rs` applies them. Each describes the origin's HTML, so the asset gate reads none of them. A publisher who sets `origin_is_cookie_independent` to say that cookies do not change its HTML still has every cookie-bearing asset request on the publisher path fetched for itself (see the open questions). +- **The cost.** Browsers send first-party cookies on subresources, which the comment in `apply_origin_cache_intent` in `publisher.rs` says is most repeat-visitor asset traffic. On the publisher path the module therefore serves only requests that carry no cookie, while an asset route serves its eligible requests whatever cookies they carry. +- **Off by default.** No stock build links the module, and a build that links it holds nothing until `[asset-cache]` selects it, so the cookie rule applies only to a deployment that has chosen the module. Row 34 as it stands held assets in every stock build from its first deploy. +- **Logs.** The lines `asset_cache bypass`, `asset_cache stored` and `asset_cache invalid entry` carry a fixed reason and a byte count, and no URL, key or header. The line `asset_cache load failed` carries the error text of the failed fetch, which includes the platform's error report as the platform formats it, and the module does not filter that text. + +### 2.12 Failure handling + +A load that fails asking the origin ends with every request that ran or waited on it answering with that failure. Each other failure ends with core asking the origin, as before row 34, or with the reader served the response the request's own fetch made. + +- **The origin cannot be reached during a load.** `load_asset` returns a `LoadError`. Every request waiting on that load receives it and nothing is stored. `asset_from_cache` logs `asset_cache load failed` at warn level and answers `OriginFailed`, which core answers as it does when its own origin request fails, with `Failed to proxy request to origin` on the publisher path and `Failed to proxy asset request` on an asset route, both a `502`. A failing origin is therefore asked once for all the requests that share the load, and the next request loads again. +- **A stored entry cannot be read.** `AssetEntry::decode` refuses bytes that are short, unreadable or written in another format. A record this build did not write is refused in the same way. In both cases the lookup logs at warn level, removes the entry, and answers `AskOrigin`. A format version is in the key, in the entry and in the record, so a build that changes the format does not read the old entries. - **The shared store fails.** The key-value layer treats a failed read as a miss, so the value is loaded, and logs a failed read, write or removal at debug level. The value cache that Fastly provides reports no errors to the module. - **An error status.** Only a `200` is stored, so the body of an origin error or of a `404` is never held. The verdict is held, as "What is stored" says. For 60 seconds the requests that find the marker ask the origin themselves and none of them stores a copy, so a file the origin begins to answer with a `200` is stored only once the marker ends. -- **A body that cannot be read whole.** The reader is served the whole body from the one fetch, as described above. +- **A body that cannot be read whole.** The reader is served the whole body from the one fetch, as section 2.7 describes. +- **A selection the build cannot meet.** `[asset-cache]` naming a module the build does not link, an `[asset-cache]` that selects more than one module, and an asset cache module supplied by a builder other than the one `[asset-cache]` selects are refused when the adapter builds its registry, so every request answers with the adapter's startup error, as for any module selection the registry refuses. A `[cache.51degrees-assets]` or `[cache.51degrees-assets-vary]` table in a build that does not link the module, and a `[cache.assets]` or `[cache.assets-vary]` table in any build, are refused by the deploy check and when the state is built. +- **Caches that keep nothing.** A module selected in a build whose loading caches keep nothing answers `AskOrigin` for every file, and nothing logs that it holds nothing (see the open questions). -**Cost and limits.** +### 2.13 Cost and limits -- **A slow origin.** For a response that passes the header checks, the request that leads a load waits for the whole body, up to 1 MiB, before its reader is served, and each request waiting on that load waits as long. Core asks for a first byte timeout and a between bytes timeout of 15 seconds for an origin fetch, on the publisher path (`DEFAULT_PUBLISHER_FIRST_BYTE_TIMEOUT`) and on an asset route (`DEFAULT_FIRST_BYTE_TIMEOUT`), and the Fastly adapter applies them. The Axum client applies limits of its own instead, 5 seconds to connect and 30 seconds for the whole request, and the Cloudflare and Spin adapters apply none of their own, so their runtimes' limits apply. A failed load reaches every request waiting on it, and each of them answers with that failure rather than fetching the origin itself. +- **A slow origin.** For a response that passes the header checks, the request that leads a load waits for the whole body, up to 1 MiB, before its reader is served, and each request waiting on that load waits as long. Core asks for a first byte timeout and a between bytes timeout of 15 seconds for an origin fetch, on the publisher path (`DEFAULT_PUBLISHER_FIRST_BYTE_TIMEOUT`) and on an asset route (`DEFAULT_FIRST_BYTE_TIMEOUT`), core registers the backend an `OriginRequest` names with them, and the Fastly adapter applies them. The Axum client applies limits of its own instead, 5 seconds to connect and 30 seconds for the whole request, and the Cloudflare and Spin adapters apply none of their own, so their runtimes' limits apply. A failed load reaches every request waiting on it, and each of them answers with that failure rather than fetching the origin itself. - **Removals.** When a response no longer varies as its record says, the request that fetched it removes the record, and a lookup that finds an entry it cannot read removes that entry. On Fastly a removal is a purge by surrogate key (`purge_surrogate_key` in the Fastly adapter's `value_cache.rs`), and on a key-value store it is a delete. An origin whose `Vary` differs from one response to the next causes a removal for each response that differs from the record. -- **Keys and variants without a limit.** Each query string, each host the reader asked, each scheme, and each combination of values for the headers `Vary` names is a key of its own. A miss writes the copy or the marker to the shared store, and the record too when the address has none, or the record alone when the origin's `Vary` allows no copies. This change sets no limit on the number of keys. `store_seconds` limits how long a copy is used and `store_idle_seconds` how long it lasts unused. Spin's `default` store, which the Spin adapter declares, gives an entry no lifetime, so there an entry that nobody reads again stays in the store, because only a read past its drop time, or a removal, takes it out. -- **The fingerprint.** The publisher path computes `template_fingerprint`, a digest of all the serialized settings, for each eligible request, whether or not a copy is found. +- **Keys and variants without a limit.** Each query string, each host the reader asked, each scheme, and each combination of values for the headers `Vary` names is a key of its own. A miss writes the copy or the marker to the shared store, and the record too when the address has none, or the record alone when the origin's `Vary` allows no copies. The module sets no limit on the number of keys. `store_seconds` limits how long a copy is used and `store_idle_seconds` how long it lasts unused. Spin's `default` store, which the Spin adapter declares, gives an entry no lifetime, so there an entry that nobody reads again stays in the store, because only a read past its drop time, or a removal, takes it out. - **Memory.** The memory layer holds at most 256 records and 32 copies, and a copy's body is at most 1 MiB, so the copies take up to about 32 MiB for as long as the layer lives. The request that leads a load also holds the whole body while it reads. - **Reads and writes of the shared store.** A request that misses the memory layer reads the shared store up to twice, once for the record and once for the copy. The record's load writes the record with the copy or the marker, or the record alone when the origin's `Vary` allows no copies, and a copy's load writes only the copy or the marker. The memory layer keeps an entry for at most `memory_seconds`, 5 seconds by default, and it is not shared between requests on Cloudflare, on Spin or, by default, on Fastly, so there every request reads the store. On Cloudflare and Spin the store is the key-value store the adapter declares, `TRUSTED_SERVER_KV` or `default`, and a write to Cloudflare's store is given a lifetime of at least a minute (`KV_SHORTEST`), because Workers KV refuses less, with the entry's own drop time holding the exact lifetime. On Axum the adapter's store is `UnavailableKvStore`, so every read and write fails and is logged at debug level, a failed read counts as a miss, and only the memory layer holds copies. - **Purges.** `ts cache purge` does not reach a copy. It purges by the `ts-template` key or by a page's key, and a copy in Fastly's cache carries neither, only a key made of `vc-` and a hash of its own key (the Fastly adapter's `value_cache.rs`). A copy on another adapter sits in the key-value store or, on Axum, in memory, and `ts cache purge` touches neither. -**Rollout for existing deployments.** +### 2.14 Rollout for existing deployments + +1. **A deployment that does not choose the module sees no change.** Every file is asked of its origin, as on row 33, whether or not a loading cache module is linked. +2. **A deployment that wants assets held** links a loading cache module and this module, hands both builders to its adapter, and writes `[asset-cache] module = "51degrees"`. It then holds shareable assets with no `[cache.51degrees-assets]` table when it links one loading cache module. A change at the origin shows after the lifetime the origin gave the file, and never later than `store_seconds`, which defaults to one hour. The checks in "What is stored" and the cookie rule apply to its asset traffic from the first deploy. +3. **A document written for row 34 before this rewrite** changes in four ways. It adds `[asset-cache]`, without which nothing is held. Its `[cache.assets]` and `[cache.assets-vary]` tables become `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]`, because no module declares the old names, so every build refuses them. A `module = "none"` line is refused, so it is removed. Where its build links two or more loading cache modules, it writes `[cache.51degrees-assets-vary]` naming the module its `[cache.51degrees-assets]` names, because each table now selects its own cache. +4. **The order of a push.** A build without the module refuses a document that carries `[cache.51degrees-assets]` or `[asset-cache] module = "51degrees"`, so a deployment deploys the build that links the module before it pushes such a document, and pushes a document without them before it rolls back to a build without the module. +5. **The `ts` tool.** The stock tool links neither the module nor a loading cache module, so `ts config validate` refuses `[cache.51degrees-assets]` as a table no linked module declares. A deployment that uses the module validates with a list that holds both builders, registered first through `register_deploy_integrations` in core, as `2026-10-14-no-modules-crate-design.md` describes for any set of modules other than the stock one. +6. **A copy outlives a new build and a change to the settings.** The key holds no digest of the build or the settings (section 2.7), so on both paths a copy made before a build or a settings change is found after it, and ends by its own lifetime, never later than `store_seconds`. On Spin, whose `default` store gives an entry no lifetime, the entry itself stays in the store after that, until a read past its drop time or a removal takes it out (see "Keys and variants without a limit"). A change to the publisher's origin URL or to the `Host` sent to it, or to a route's origin or prefix, changes `origin_identity` and so the key. A change to a route's credentials does not. +7. **The changelog.** Row 34's `CHANGELOG.md` entry says that the assets are held through the loading cache selected in `[cache.assets]` and that `module = "none"` holds nothing. It is rewritten for the module, `[asset-cache]`, the two tables and the end of `none`. -1. `[cache.assets]` is optional and is left out when written back (`skip_serializing_if = "LoadingCacheConfig::is_default"`), so settings written before this change read as they did. -2. A build that links no loading cache module holds nothing, and a build that links several and names no module holds nothing. Both behave as before. -3. A stock build links exactly one loading cache module, `loading-cache.51degrees`, so it starts holding shareable assets with no change to its settings. A change at the origin then shows after the lifetime the origin gave the file, and never later than `store_seconds`, which defaults to one hour. The checks in "What is stored" and the cookie rule under Consent and privacy then apply to its asset traffic from the first deploy. -4. `[cache.assets] module = "none"` turns both caches off while a module is linked, and both paths then ask the origin for every asset as they did before this change. A build that links no loading cache module holds nothing as well. -5. Every change to the settings and every new build changes the fingerprint, so a publisher path copy made earlier is not found again and ends by its lifetime. Nothing purges it. The publisher path therefore starts with no copy of any asset after each build or settings change, and each asset is fetched from the origin again. A route's key has no fingerprint, so a route's copy outlives a settings change, including a change to the route's credentials. -6. The deploy check lists the new cache names. The error that names a misspelt table now includes `assets` and `assets-vary`. +### 2.15 Open questions -**Open questions.** One choice is open, and this change does not make it. +These are for the reviewer, and this rewrite decides none of them. -1. A publisher's word for assets. Should a setting let a publisher say that cookies do not shape its assets, as `origin_is_cookie_independent` says of its HTML, so that a request on the publisher path that carries a cookie may read and lead copies (see "The page path's cookie settings are not read" and "The cost")? +1. **A publisher's word for assets.** Should a setting let a publisher say that cookies do not shape its assets, as `origin_is_cookie_independent` says of its HTML, so that a request on the publisher path that carries a cookie may read and lead copies (see "The page path's cookie settings are not read" and "The cost")? #894 takes a related approach for HTML. An operator asserts that a route is anonymous and names the cookies that bypass it, and every other cookie is stripped from the request sent to the origin, so a request that carries cookies stays eligible. +2. **One table for both caches.** Should row 32's declarations let a module say that one table selects two of its caches, as `[cache.assets]` did before this rewrite? +3. **Caches that keep nothing.** Should the module refuse to start, or log once, when it is selected in a build whose loading caches keep nothing, rather than answer `AskOrigin` for every file without a word? +4. **The stock lists.** Should the stock lists offer the module, so that a deployment that links only a loading cache module can select it, or should a deployment keep linking it itself, as section 2.3 has it? +5. **Beside #894's later phases.** If #894 later holds responses that are not documents in core, should core skip its own cache for a file an asset cache module answers, or should a deployment choose one of the two for each kind of file? ## 3. What is not in this change -- **The publisher's declared rule.** An origin can vary on an input it does not name in `Vary`, and only the publisher knows which. `Lookup::declared` in `source_cache.rs` takes a token that enters the copy's key, and one `source_cache` test covers it. Both callers pass `None`, and no setting supplies a value. The rule, as `[publisher.origin_personalization]`, is a later change with its own tests, and until then the origin's own `Vary` alone separates copies. -- **A rewritten body kept once.** The fetch phase runs on each reader's copy. `AssetCopy::decoded` and `SourceVary::without` are for a later change that keeps the rewritten body. Production code leaves `decoded` false and calls `without` nowhere. +- **An asset cache in core.** Core holds the interface and no copy, key, gate or setting of its own. +- **A stock build that holds assets.** No stock list links the module or a loading cache module. +- **A digest of the build and the settings in the key.** The key holds none until a later change keeps a rewritten body, which needs one (section 2.7). +- **The publisher's declared rule.** An origin can vary on an input it does not name in `Vary`, and only the publisher knows which. `Lookup::declared` in the module's `source_cache.rs` takes a token that enters the copy's key, and one test covers it. Its one caller, `asset_from_cache`, passes `None` on both paths, and no setting supplies a value. The rule, as `[publisher.origin_personalization]`, is a later change with its own tests, and until then the origin's own `Vary` alone separates copies. +- **A rewritten body kept once.** The fetch phase runs on each reader's copy. `AssetCopy::decoded` and `SourceVary::without` are for a later change that keeps the rewritten body, and that change brings back the digest the key leaves out. The module leaves `decoded` false and calls `without` nowhere. - **Pages.** A page stays on the template cache. -- **The `/proxy` first-party route.** The route is unchanged and does not use the asset caches. -- **Answers made by the platform's image optimizer.** The platform holds those answers, so core does not keep them. -- **Requests a copy cannot answer.** `HEAD`, other methods, `Authorization`, ranges, conditional requests, and a `Cache-Control` or `Pragma` that asks for the origin always go to the origin. +- **The `/proxy` first-party route.** The route is unchanged and offers nothing to the module. +- **Answers made by the platform's image optimizer.** The platform holds those answers, so core does not offer them to the module. +- **Requests a copy cannot answer.** `HEAD` and every other method but `GET`, which core does not offer, and `Authorization`, ranges, conditional requests, and a `Cache-Control` or `Pragma` that asks for the origin, always go to the origin. - **Files over 1 MiB.** A file over 1 MiB is served as it streams from the origin. -- **A way to purge one asset.** This change adds no purge for the asset caches, and `ts cache purge` does not reach a copy (see Cost and limits). -- **Settings for the memory layer's sizes.** The counts of records and copies are literals in `fn asset_cache`. +- **A way to purge one asset.** This change adds no purge for the module's caches, and `ts cache purge` does not reach a copy (see "Cost and limits"). +- **Settings for the memory layer's sizes.** The counts of records and copies are literals in the module. ## 4. Acceptance -The tests below exist. Those in `publisher.rs` and `proxy.rs` use `StubHttpClient` as the origin and `MemoryLoadingCache` from `crates/trusted-server-core/src/platform/test_support.rs` as the loading cache, apart from the `proxy.rs` test that has no cache and the `publisher.rs` test that selects the caches from the settings with `loading_caches_from`, as an adapter does. Those in `source_cache.rs` use `MemoryLoadingCache` and a counting function as the origin. Those in `asset_cache.rs`, `settings.rs`, `integrations/mod.rs` and the Axum `seam_probe.rs` use neither. The pull request records the fork's CI at `f6e598fda` as 19 of 19 checks green. +**Core's tests.** These are written first, against core alone, and fail until the interface and the two places that ask the module exist. A recording test module keeps each `AssetFile` it is offered and answers `AskOrigin`, so a test reads what core reported. + +| Where | Tests | +| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `publisher.rs` | A deployment selecting no asset cache module asks the origin for every file, two requests making two origin requests. A test module answering `Serve` stops the origin being asked, and its response is processed as an origin response. A test module answering `AskOrigin` leaves one origin request. `OriginFailed` is answered as a failed origin request, a `502`. The recording module shows that a request carrying `Cookie` is offered with `carries_cookie` true, one carrying `Authorization` with `carries_authorization` true, a navigation with `is_document` true, and a range, state a module left and a response an integration marked personalized each with `asks_for_origin` true, that a `HEAD` is not offered, and that `origin.platform_request()` is the request core sends with no module selected, the same in its URI, its headers, its backend, its cache intent and its streaming | +| `proxy.rs` | The same three answers on a route. The recording module shows that a request the image optimizer answers is not offered, that a reader's `Authorization` is reported as `carries_authorization` while the request to the route's origin carries only the headers the route forwards and its own credentials, and that `origin.platform_request()` is the request core sends with no module selected | +| `integrations/mod.rs`, `settings.rs` | `declared_caches` lists only `robots-txt` as core's own, a `[cache.assets]` table is refused, and `CacheSettings` has no `assets` field | +| `registry.rs` | `[asset-cache]` naming a module the build does not link is refused, an `[asset-cache]` selecting two modules is refused, and an asset cache module supplied by a builder other than the one `[asset-cache]` selects is refused, naming that builder | + +The module's tests of the cookie rule build their `AssetFile` with `carries_cookie` set, so they cannot see what core reports, and only core's recording test shows that core reports a reader's `Cookie`. + +**The module's tests.** The tests row 34 has today move into the module with the code they cover, and drive the module through `AssetCacheModule::answer` with `StubHttpClient` as the origin and `MemoryLoadingCache` as the loading cache, both from core's test doubles under `test-utils`, apart from the test that selects the caches from the settings with `loading_caches_from`, as an adapter does. Those of the key, the entry and the source cache use a counting function as the origin, as before. The CI record of the code before this rewrite is IABTechLab/trusted-server#1375 at `2a449eaf6`, where [29 of 29 checks](https://github.com/IABTechLab/trusted-server/commit/2a449eaf62bf34cb9e1e9244cd504216022e571d/checks) succeeded. + +| What | Tests | +| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| The key and the entry | Every input changes the key and none appears in it. An entry reads back as written. A short, foreign or other-format entry is not read. A copy keeps no connection, cookie or length header. A served copy counts its age from when it was made | +| The source cache | Record and copy keys follow the publisher. `Vary` names are lowercase, sorted and named once. `*`, `Cookie` and an unreadable `Vary` allow no copies. A record reads back, and a foreign one does not. A variant tells absent, empty and reordered values apart. The first response records its `Vary` and each value gets its own copy. A response that allows no copies has every request fetch its own. A source that changes its `Vary` has its record dropped. A declared token separates copies | +| The publisher path | An image is kept as sent and one fetch serves three requests. A stylesheet is served from one fetch with its age. `Vary: *`, `Vary: Cookie`, `private`, `no-store` and `public, no-store` are never kept. `s-maxage` then `max-age` set the lifetime. A header named in `Vary` separates copies. A body over 1 MiB is served whole from one fetch, with the stub streaming and not. A response whose `Vary` changed is served from its one fetch. A load that fails asking the origin is the answer, `OriginFailed` after one origin request, and the next request loads again. A request carrying `Authorization` is fetched for itself. A request carrying a cookie is fetched for itself and its answer kept for no one. An answer that sets a cookie is served only to the reader it was made for. One fetch serves two requests when the settings name no loading cache module and one is linked. A document is never answered from a copy | +| An asset route | A route's answer is held and serves the next request, with its type, an `Age`, its bytes and the origin's lifetime. An answer with `Set-Cookie` is fetched each time. A reader's cookie does not keep a route's answer from a copy. A range request is fetched. `no-cache`, `no-store` and `min-fresh=10` are fetched. Nothing is held while the caches keep no values. A load that fails asking the origin is the answer, `OriginFailed` after one origin request | +| The caches and settings | The builder declares `51degrees-assets` and `51degrees-assets-vary`. `[cache.51degrees-assets]` sizes both caches, with the defaults for what it leaves unset. A lifetime out of range in `[cache.51degrees-assets]` is refused, naming the table and the field. A record cache that keeps nothing beside a copy cache that keeps values asks the origin directly. A `[cache.51degrees-assets] module = "none"` table is refused. An `[asset-cache.51degrees]` table is refused | -| Where | Tests | -| ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `asset_cache.rs` | Every input changes the key and none appears in it. An entry reads back as written. A short, foreign or other-format entry is not read. A copy keeps no connection, cookie or length header. A served copy counts its age from when it was made | -| `source_cache.rs` | Record and copy keys follow the publisher. `Vary` names are lowercase, sorted and named once. `*`, `Cookie` and an unreadable `Vary` allow no copies. A record reads back, and a foreign one does not. A variant tells absent, empty and reordered values apart. The first response records its `Vary` and each value gets its own copy. A response that allows no copies has every request fetch its own. A source that changes its `Vary` has its record dropped. A declared token separates copies | -| `publisher.rs`, module `asset_cache_tests` | An image is kept as sent and one fetch serves three requests. A stylesheet is served from one fetch with its age. `Vary: *`, `Vary: Cookie`, `private`, `no-store` and `public, no-store` are never kept. `s-maxage` then `max-age` set the lifetime. A header named in `Vary` separates copies. A body over 1 MiB is served whole from one fetch, with the stub streaming and not. A response whose `Vary` changed is served from its one fetch. A load that fails asking the origin is the answer, a `502` after one origin request, and the next request loads again. A request carrying `Authorization` is fetched for itself. A request carrying a cookie is fetched for itself and its answer kept for no one. An answer that sets a cookie is served only to the reader it was made for. Assets turned off in the settings are fetched for every request, and one fetch serves two requests when the settings name no module. A page navigation never reads a copy | -| `proxy.rs` | A route's answer is held and serves the next request, with its type, an `Age`, its bytes and the origin's lifetime. An answer with `Set-Cookie` is fetched each time. A reader's cookie never reaches a route's origin, so the answer is shared. A range request is fetched. `no-cache`, `no-store` and `min-fresh=10` are fetched. Nothing is held while no cache keeps values. A load that fails asking the origin is the answer, a `502` after one origin request | -| `settings.rs`, `integrations/mod.rs`, Axum `seam_probe.rs` | The list of core's caches now holds `assets` and `assets-vary`, in the settings listing, in the refusal that names a misspelt table, and in the Axum deploy check. A lifetime out of range in `[cache.robots-txt]` or `[cache.assets]` is refused, naming the table and the field. Both asset caches take the one module offered when `[cache.assets]` names no module, and `none` turns both off and leaves `robots-txt` as it was | +Row 34's `assets_turned_off_in_the_settings_are_fetched_for_every_request` and `assets_naming_none_turns_both_asset_caches_off_and_nothing_else` go, with `none`, and core's no-module test takes their place. The seam probe is a fixture crate through which tests drive core's extension points from outside core, and row 34's change to the probe's expected list of caches (`crates/trusted-server-adapter-axum/tests/seam_probe.rs`) goes too, because a build that does not link the module declares only `robots-txt` and the probe's cache. -Joins with no test: +**What no test covers.** 1. **Concurrent requests.** Every test of the asset paths sends requests one after another. The loading cache module has tests for many callers loading a missing key once (`many_callers_on_many_threads_load_a_missing_key_once_in_the_caller`) and for a failed load reaching every waiter (`a_failed_load_reaches_every_waiter_and_is_not_kept`), and those load a generic value and not an asset. -2. **A real shared store.** No test runs the asset caches against Fastly's cache, a key-value store, or two instances. The Axum, Cloudflare and Spin adapters have no test of asset holding. -3. **A `[cache.assets-vary]` table.** No test writes one. -4. **Publisher path cases.** No test covers an HTML response to a non-document request, a `Content-Length` over the limit, a lifetime from `Expires` or `Surrogate-Control`, a request with a conditional header, a range, `Cache-Control` or `Pragma`, request state left by a module, a personalized response, a request with no host, or a header that is not visible text. -5. **Asset route cases.** No test covers an image optimizer route, a route with `auth`, a request with `Authorization`, a method other than `GET`, or a request with no host. -6. **Streaming with a stored copy.** The route tests turn streaming off (`held_asset_stub`), and the publisher tests leave it off except in the oversized body test, which stores nothing. A stored copy served on an adapter that streams has no test. -7. **Two publishers in one cache.** The key test in `source_cache.rs` and the module's `two_publishers_never_share_an_entry` cover the prefix, and no test serves two publishers through either asset path. -8. **Reordered page checks.** `origin_response_may_be_kept` moves the status check ahead of the cookie and edge header checks in `template_cache_ttl`. A page response with a non-`200` status and a fault found by a later check is now reported as a non-`200` status, and no test was added for the changed order. -9. **A module declaring a cache of core's.** `a_builder_declaring_core_s_own_cache_is_refused` declares `robots-txt` only, so no test has a module declare `assets` or `assets-vary`. +2. **A real shared store.** No test runs the module against Fastly's cache, a key-value store, or two instances. The Axum, Cloudflare and Spin adapters have no test of asset holding. +3. **A deployment's build.** No test links the module the way a deployment does, by handing its builder to an adapter's `run_with` or `routes_with_registrations` with a loading cache module beside it. +4. **A `[cache.51degrees-assets-vary]` table.** No test writes one. +5. **Publisher path cases.** No test covers an HTML response to a request that is not for a document, a `Content-Length` over the limit, a lifetime from `Expires` or `Surrogate-Control`, a file offered to the module with `asks_for_origin` true, a request with no host, or a header that is not visible text. Core's recording test reports `asks_for_origin` for a range, module state and a personalized marker, and not for a conditional header, a `Cache-Control` or a `Pragma`. +6. **Asset route cases.** No test covers a route with `auth`, a method other than `GET`, or a request with no host. +7. **Streaming with a stored copy.** The route tests turn streaming off (`held_asset_stub`), and the publisher tests leave it off except in the oversized body test, which stores nothing. A stored copy served on an adapter that streams has no test. +8. **Two publishers in one cache.** The key test of the source cache and the loading cache module's `two_publishers_never_share_an_entry` cover the prefix, and no test serves two publishers through either asset path. +9. **Reordered page checks.** `origin_response_may_be_kept` moves the status check ahead of the cookie and shared cache header checks in `template_cache_ttl`. A page response with a status other than `200` and a fault found by a later check is now reported as a status other than `200`, and no test was added for the changed order. ## Revision record -| Date | Change | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-15 | First draft, written after the code. | -| 2026-10-15 | Fixes from review round 1. The cookie rules the asset gate leaves out and what that lets through, the reason the code gives, the default-on state and the open questions on cookie rules and an off switch, `Pragma` in the gate, the verdict a response that fails a test leaves for 60 seconds, the module default, where the sizes and the `[cache.assets-vary]` table stand, the key-value store each adapter uses, how far one fetch reaches, what the caches add on Fastly, `[[cache.asset_rules]]`, the headers a route forwards, the cost and limits, the test doubles and the missing tests. | -| 2026-10-16 | Fixes to the code, and from review round 2. A request that sends the publisher's origin a cookie neither reads a copy nor leads a load, `[cache.assets] module = "none"` turns both asset caches off, and a `[cache.assets]` lifetime out of range is refused when the settings load, each with its tests. Which entries each load writes, the tests that send a cookie, and the module default in the guide and the code comments. | -| 2026-10-17 | The code fix on row 24, where the Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests, and where each application state has its own memory layer, Cloudflare included. How far one fetch reaches, the adapters' shared stores, how long the memory layer lives, and the reads and writes of the shared store say so. | -| 2026-10-18 | A fix to the code. A load that fails asking the origin is the answer to every request that ran or waited on it, so the origin is asked once, with a test on each path. The design, failure handling, the cost of a slow origin and the tests say so. | -| 2026-10-18 | The timeouts of an origin fetch, which core asks for on every adapter and only the Fastly adapter applies. | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-15 | First draft, written after the code. | +| 2026-10-15 | Fixes from review round 1. The cookie rules the asset gate leaves out and what that lets through, the reason the code gives, the default-on state and the open questions on cookie rules and an off switch, `Pragma` in the gate, the verdict a response that fails a test leaves for 60 seconds, the module default, where the sizes and the `[cache.assets-vary]` table stand, the key-value store each adapter uses, how far one fetch reaches, what the caches add on Fastly, `[[cache.asset_rules]]`, the headers a route forwards, the cost and limits, the test doubles and the missing tests. | +| 2026-10-16 | Fixes to the code, and from review round 2. A request that sends the publisher's origin a cookie neither reads a copy nor leads a load, `[cache.assets] module = "none"` turns both asset caches off, and a `[cache.assets]` lifetime out of range is refused when the settings load, each with its tests. Which entries each load writes, the tests that send a cookie, and the module default in the guide and the code comments. | +| 2026-10-17 | The code fix on row 24, where the Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests, and where each application state has its own memory layer, Cloudflare included. How far one fetch reaches, the adapters' shared stores, how long the memory layer lives, and the reads and writes of the shared store say so. | +| 2026-10-18 | A fix to the code. A load that fails asking the origin is the answer to every request that ran or waited on it, so the origin is asked once, with a test on each path. The design, failure handling, the cost of a slow origin and the tests say so. | +| 2026-10-18 | The timeouts of an origin fetch, which core asks for on every adapter and only the Fastly adapter applies. | +| 2026-10-26 | Rewritten for the asset cache as a module. Core keeps one interface, `AssetCacheModule`, asked where row 34 called `asset_from_cache` on the publisher path and on an asset route, with an origin request the module sends itself, and the module `crates/asset-cache/51degrees` holds everything row 34 put in core. `[asset-cache]` selects it, it declares its caches through row 32, a deployment that selects no module has every file asked of its origin, no stock build links it, and it sits beside IABTechLab/trusted-server#894 as section 2.10 says. The gate also needs the record cache to keep values, `none` goes, each cache table selects its own loading cache module, and the open questions are new. | +| 2026-10-26 | Fixes from review round 3. `origin_response_may_be_kept` stays in core and becomes public beside `origin_shared_ttl`, both returning the new `KeepRefusal`, and row 46's change to it is named beside rows 39 and 54. Core offers only a `GET`, takes the module only from the builder `[asset-cache]` selects, and gains a recording test module for what it reports. `AssetFile`, `AssetVia` and `OriginRequest` are set out with their visibility and test constructors, `path` is `via`, and `fingerprint()` goes, with the key's digest of the build and the settings, until a rewritten body is kept. The caches are `51degrees-assets` and `51degrees-assets-vary`, the variant digest's tag is `ts-asset-variant-v1`, and after row 56 the module sits behind `asset-cache-51degrees`. The helpers the module copies, the crate's workspace and alias entries, what `asks_for_origin` describes, the fourth change to a row 34 document, the changelog, the CI record, #894's keys, its costs of Fastly's Core Cache and its cookie stripping, and smaller corrections. | +| 2026-10-26 | The account of the Touchbase of 2026-10-23 says what 51Degrees said there and what 51Degrees decided after it, and the integrations crate design is named by its file, `2026-10-26-integrations-crate-design.md`. | +| 2026-10-26 | Fixes from review round 4. The pull requests and the issue are linked where first named, and the CI record is linked at its commit. "Stock build", Viceroy and the seam probe are defined where first used. Fastly's Core Cache API is named as #894 names it, the cache names follow the form row 32 asks for, and on Spin the entries that a new key or a lifetime leaves behind stay in the store. The decisions of 51Degrees are placed after the Touchbase. | ## Review log diff --git a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md index ec8416ac79..d694ebd8c6 100644 --- a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md +++ b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-12-loading-cache-design.md`, `2026-10-12-robots-txt-from-modules-design.md`, `2026-08-27-integration-provider-seam-design.md` **Related PRs:** IABTechLab/trusted-server#1373, reviewed on jwrosewell/trusted-server#27 (closed) **Issues:** IABTechLab/trusted-server#1320 -**Last updated:** 2026-10-18 +**Last updated:** 2026-10-26 > **Why this spec exists.** A module that holds its own answers asks core for the cache of the name it chose, finds no `[cache.]` table for that name, and gets the cache that holds nothing, so every request loads the answer again. @@ -62,7 +62,7 @@ A declared name is the cache's name wherever the cache is used. The name is the 2. No `module` selects the one loading cache module the build links, or nothing when the build links none or several. 3. A module's cache is made once however many tables select it. -The new step comes after the tables. A declared cache that has no table at all follows the rule for a table with no `module`, and shares the cache already made for that module when a table made one. A stock build therefore holds a module's answers without a line in the document. The table below shows the result for a declared cache named `answers`. +The new step comes after the tables. A declared cache that has no table at all follows the rule for a table with no `module`, and shares the cache already made for that module when a table made one. A build that links one loading cache module therefore holds a module's answers without a line in the document. Once the 51Degrees module leaves the stock lists (`2026-10-12-loading-cache-design.md`), a stock build links none, so there a declared cache holds nothing, and the Axum test that section 4 describes changes with the stock lists. The table below shows the result for a declared cache named `answers`. | The document | Loading cache modules linked | The `answers` cache | | ---------------------------------- | ---------------------------- | ----------------------------------------------- | @@ -246,6 +246,7 @@ These joins have no test. | 2026-10-15 | Corrections from review round 2. What a module that borrows another module's cache name shares, the section the guide's push-order warning covers, the builders the deploy check is given, the round 1 log sentence on the code changes, and the follow-ups for the configuration guide, the integration guide, the rustdoc and the visibility of two functions. | | 2026-10-17 | Who can read a copy names the shared store on each adapter, being Fastly's cache, the key-value store the Cloudflare and Spin adapters declare on row 24, `TRUSTED_SERVER_KV` and `default`, and none on Axum. | | 2026-10-18 | Items 5 to 8 leave 2.10. The configuration guide says core applies no lifetime of a declared table and when to push one, the integration guide lists `.with_cache_features(...)`, and `declared_caches` and `check_cache_tables` are `pub(crate)`. The rustdoc of `loading_caches_from` and `LoadingCacheConfig::module`, and the integration guide on a table naming no module, already gave rule 2 in 2.2. Section 3 and 2.8 match the guides. 2.7 and section 3 say the deploy check also refuses a `module` that offers no loading cache. | +| 2026-10-26 | 2.2 says that a build linking one loading cache module holds a declared cache's answers without a line in the document, where it said a stock build does, because no stock build links one once the 51Degrees module leaves the stock lists (`2026-10-12-loading-cache-design.md`). | ## Review log diff --git a/docs/superpowers/specs/2026-10-26-integrations-crate-design.md b/docs/superpowers/specs/2026-10-26-integrations-crate-design.md new file mode 100644 index 0000000000..d6ea6dd6ea --- /dev/null +++ b/docs/superpowers/specs/2026-10-26-integrations-crate-design.md @@ -0,0 +1,328 @@ +# Design Spec: One integrations crate holds every module, in a folder for its type and vendor + +**Status:** Proposed, 2026-10-26. For a new row at the top of the stack, row 56, which folds every module crate into one crate, as IAB Tech Lab asked at the Touchbase of 2026-10-23 and as decided by 51Degrees after the Touchbase. No pull request exists for the row yet. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-10-14-no-modules-crate-design.md`, `2026-10-19-module-scripts-in-crates-design.md`, `2026-10-12-loading-cache-design.md`, `2026-10-15-asset-cache-design.md`, `2026-08-27-integration-provider-seam-design.md`, and IAB Tech Lab's `2026-09-17-split-integrations-crates-design.md` on [IABTechLab/trusted-server#1194](https://github.com/IABTechLab/trusted-server/pull/1194) +**Related PRs:** [IABTechLab/trusted-server#1194](https://github.com/IABTechLab/trusted-server/pull/1194) (Aram Grigoryan, draft), [IABTechLab/trusted-server#1423](https://github.com/IABTechLab/trusted-server/pull/1423) (row 55, the top of the stack below this row). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack +**Issues:** [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack +**Last updated:** 2026-10-26 + +> **Why this spec exists.** The stack gives each module a crate of its own, 32 crates under `crates///`. At the Touchbase of 2026-10-23, IAB Tech Lab asked for those folders to be kept inside one crate, as Tech Lab's own design puts every integration in one crate, and 51Degrees agreed. + +Paths, counts and line numbers are those of row 55 at `5be114773` unless a row or a commit is named. + +## 1. Problem + +Every module outside core is a crate of its own at `crates///`, whose package is `trusted-server--`. There are 32. Each manifest names the module's maintainers in `[package.metadata.maintainers]`, being 6 `project owned`, 22 `seeking vendor owner` and 4 `vendor owned`, all four by 51Degrees, being the loading cache, the client hints middleware, the Model Terms signal and the code the 51Degrees modules share. The asset cache module of row 34 (`2026-10-15-asset-cache-design.md`) adds a 33rd, vendor owned by 51Degrees. + +The crates are named one by one in many places. The workspace manifest lists each as a member, and each but the seam probe as a dependency. The seam probe, `crates/testing/iabtl`, is a fixture crate through which tests drive core's extension points from outside core. Each adapter's manifest names between 28 and 31 of the crates, the `ts` tool's names 23, and the permissions inspector under `tools/permissions-inspector/wasm`, a workspace of its own, names four. Each adapter's `shipped_module_builders()` and the tool's `linked_builders()` name each builder by its crate, six aliases in `.cargo/config.toml` list the packages, and two steps of `.github/workflows/test.yml` name a crate or a path below `crates///`. + +IAB Tech Lab's design, `2026-09-17-split-integrations-crates-design.md` on the draft IABTechLab/trusted-server#1194, puts every concrete integration in one crate, `trusted-server-integrations`, each in a flat folder `src/integrations//`, and every integration's browser code in a second crate, `trusted-server-integrations-js`. It sets per-vendor crates aside because they "would multiply dependency, compatibility, release, and governance surfaces without a current external consumer". At the Touchbase of 2026-10-23, IAB Tech Lab asked for one crate that keeps the `/` folders, and 51Degrees agreed. As decided by 51Degrees after the Touchbase, one crate holds every module, its own crates and the asset cache module included, every file 51Degrees puts in the repository takes the repository's license, Apache-2.0, and the 51Degrees loading cache keeps depending on the package `fiftyone-caching`. + +## 2. Design + +### 2.1 The crate and its folders + +Every module crate moves into one crate, `crates/trusted-server-integrations`, package `trusted-server-integrations`, the name #1194 gives it. Each crate's folder becomes a folder of the new crate's `src`, so `crates///` becomes `crates/trusted-server-integrations/src///`, and what was in the crate's own `src/` moves up into that folder, as the table below shows. + +```text +crates/trusted-server-integrations/ + Cargo.toml + src/ + lib.rs declares each type folder + cmp/ + mod.rs declares each vendor folder of the type + inmobi/ + mod.rs was crates/cmp/inmobi/src/lib.rs + maintainers.toml was the manifest's maintainers table + fixtures/ was crates/cmp/inmobi/src/fixtures/ + tests/ InMobi's node tests, as before + didomi/ + mod.rs + maintainers.toml + js/src/didomi/ the module's browser script, as row 55 placed it + js/test/didomi/ + loading-cache/ + mod.rs + 51degrees/ + mod.rs + layer.rs + stores.rs + tests.rs + maintainers.toml +``` + +| In a module crate | In the integrations crate | +| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | +| `src/lib.rs` | `src///mod.rs` | +| Every other file under `src/` | The same name beside `mod.rs`, so `src/fixtures/` is `fixtures/` | +| `js/`, `tests/`, `project/` and `README.md` | The same, in the folder | +| `.gitattributes`, in 13 of the folders | The same, with its patterns read from the folder, so `src/fixtures/page-change.*.html` is `fixtures/page-change.*.html` | +| `[package.metadata.maintainers]` | `maintainers.toml` (section 2.4) | +| The dependencies | The integrations crate's manifest, each once, apart from one tokio feature, placed as below | +| The rest of the manifest | The workspace's edition, license, version and `publish`, with `[lints]` and `doctest = false` as below | + +All 32 manifests set `[lints] workspace = true`, which a crate has to opt into, and 31 set `doctest = false`, so the integrations crate sets both. The seam probe's manifest is the one without `doctest = false`, and its one doctest (`crates/testing/iabtl/src/lib.rs` lines 839 to 843) names its crate, `trusted_server_testing_iabtl`, so that doctest becomes a unit test in the probe's folder, where it neither fails for the missing crate nor stops running under `doctest = false`. + +The 51Degrees loading cache's tests ask tokio for `rt-multi-thread` (`crates/loading-cache/51degrees/Cargo.toml` line 27). tokio 1.52.3 refuses that feature on every WebAssembly target unless `tokio_unstable` is set (its `src/lib.rs` lines 466 to 478), nothing in the repository sets it, and `test-fastly` and `clippy-fastly` build the integrations crate's tests for `wasm32-wasip1`. The feature therefore goes in a table that only a native build reads, and the tests that use threads run on the host only, behind `#[cfg(not(target_arch = "wasm32"))]`. + +```toml +[dev-dependencies] +futures = { workspace = true } +tokio = { workspace = true, features = ["rt", "macros", "time"] } + +[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies] +tokio = { workspace = true, features = ["rt-multi-thread"] } +``` + +The placeholder `crates/edgecookie/README.md`, a type folder with no crate in it yet, moves to `src/edgecookie/README.md`. Each type folder gains a `mod.rs` that declares its vendor folders, and the crate's `lib.rs` declares the type folders. Each file moves once, keeping its content apart from the changes this design lists, so Git follows the moves as renames and `git log --follow` and blame keep each file's history. + +Paths inside the files follow. `include_str!` takes a path from the file that holds it, so a fixture that moves with its source needs no change. The 12 folders whose page change test names its recorded page by a path under `env!("CARGO_MANIFEST_DIR")` name `/src///fixtures/` where they named `/src/fixtures/`, because the manifest folder is now the integrations crate's. A `use` of another module crate, such as the client hints middleware's `trusted_server_shared_51degrees`, becomes a path in the crate, `crate::shared::fiftyone_degrees`. Some paths climb out of a crate's own `src/` or folder, and each of them changes with the move. + +- The seam probe's `include_str!("../js/probe.js")` (`crates/testing/iabtl/src/lib.rs` line 97) becomes `include_str!("js/probe.js")`, because `mod.rs` sits beside `js/`. +- The APS module's tests read the renderer fixture of `trusted-server-js` with `include_str!("../../../trusted-server-js/lib/test/fixtures/aps-renderer-v1.json")` (`crates/auction/aps/src/lib.rs` lines 2641 and 4532), which takes one more `../`, because the file sits one folder deeper below `crates/`. +- InMobi's node tests read `../src/gpp_stub.js` and `../src/tcf_stub.js` (line 9 of each file under `crates/cmp/inmobi/tests/`), which become `../gpp_stub.js` and `../tcf_stub.js`. +- The Google ad tag's script tests find `gpt_bootstrap.js` at `../../../src/gpt_bootstrap.js` (`crates/ad-tag/google/js/test/fixtures/paths.ts` line 14), which becomes `../../../gpt_bootstrap.js`. +- Three `README.md` files link the guides by a relative path, `crates/middleware/51degrees/README.md` line 18, `crates/shared/51degrees/README.md` line 25 and `crates/analytics/tinybird/project/README.md` lines 27 and 28, and each link takes two more `../`. + +Section 3 lists the files outside the crate that name a path in a module's folder. + +### 2.2 Rust names for the folders + +A folder name is what the settings use, and Rust cannot use every folder name as a module name. Each folder keeps its name on disk, and its Rust module takes a name under two rules. + +1. A `-` becomes `_`, so `permission-signal` is `permission_signal` and `us-privacy` is `us_privacy`. +2. The one folder name that starts with a digit, `51degrees`, is `fiftyone_degrees`, the way 51Degrees' own published Rust packages write the number, `fiftyone-caching` among them. + +A name that is a Rust keyword would take a raw name, such as `r#type`, and no folder today is one. The `#[path]` attribute ties each module to its folder. + +```rust +// src/lib.rs +#[path = "permission-signal/mod.rs"] +pub mod permission_signal; + +// src/permission-signal/mod.rs +#[path = "us-privacy/mod.rs"] +pub mod us_privacy; +``` + +| Folder | Rust path in the crate | +| ---------------------------------- | --------------------------------- | +| `src/cmp/inmobi` | `cmp::inmobi` | +| `src/permission-signal/us-privacy` | `permission_signal::us_privacy` | +| `src/tag/google-tag-manager` | `tag::google_tag_manager` | +| `src/auction/prebid-server` | `auction::prebid_server` | +| `src/loading-cache/51degrees` | `loading_cache::fiftyone_degrees` | +| `src/asset-cache/51degrees` | `asset_cache::fiftyone_degrees` | +| `src/shared/51degrees` | `shared::fiftyone_degrees` | + +Other crates write the Rust names in more places than `use` lines and builder calls. Each adapter's `shipped_signal_modules()` makes the permission signal modules by their types (Axum `app.rs` line 160, Cloudflare line 432, Fastly line 426 and Spin line 383), and the Fastly adapter uses the Fastly device and geo types, `FastlyHostSignals` and `FastlyDeviceModule` (its `lib.rs` line 42 and `app.rs` line 149) and `FastlyPlatformGeo` (its `platform.rs` line 668). The tool's `pbs inspect` command reads the Prebid Server module's `MODULE`, and its tests read Prebid's `PrebidIntegrationConfig`. The permissions inspector makes four permission signal modules, and the Axum adapter's tests name constants such as `MODULE` of the Osano module and `COOKIE_MTM_PREF` of the Model Terms signal. A deployment's code writes `trusted_server_integrations::cmp::inmobi::builder()` where it wrote `trusted_server_cmp_inmobi::builder()`. + +### 2.3 Module names do not change + +A module's name is still `.`, with an entry after it where a folder supplies several, such as `ad-tag.google.diagnostics`, so no settings document changes and every section, table and `[[fetch]]` or `[[serve]]` entry reads as before. + +Today `trusted_server_core::module_name!()` derives a name from Cargo's `CARGO_MANIFEST_DIR`, which after the fold is the integrations crate's folder for every module, so the macro can no longer tell two folders apart. At row 55 it is called in 24 tests, the 20 named `module_constant_is_the_crate_folder`, `module_constant_is_the_crate_folder_with_the_entry` in the client hints middleware, `module_constant_names_the_crate_folder_and_the_module_within_it` in the Google ad tag's diagnostics, `the_loader_middleware_is_named_after_the_module` in the Google ad tag and `the_tag_middleware_is_named_after_the_module` in DataDome, and in 8 functions that work a name out when the module runs, being `name()` in each of the five permission signal crates, `vendor()` in the 51Degrees shared code and `module_name()` and `reader_middleware_name()` in the seam probe. These 24 tests sit in 21 of the 32 crates, and the other 11 have no test of their name. + +In the integrations crate every name is a constant written in its folder's `mod.rs`, as most already are (`pub const MODULE: &str = "cmp.inmobi";`), and the 8 functions return constants. One test checks every constant against its folder (section 2.5) and refuses a call of `module_name!` anywhere in the crate. Core keeps the macro for a crate outside the repository, such as a deployment's own, whose name still comes from its folder. Deriving the names from `file!()` was set aside, because the path `file!()` gives depends on how Cargo and rustc were invoked, for example whether the crate is a member of the workspace being built and whether the build remaps source paths, so a name derived from it could differ from one build to the next. + +The builders give the registry a source label for its messages, such as the one that names two builders registering one id. 24 places in 22 of the crates pass `env!("CARGO_PKG_NAME")`, which after the fold would give every builder `trusted-server-integrations`, and two builders write their package name as text, the Google tag's `SOURCE` (`crates/tag/google-tag/src/lib.rs` line 69) and the seam probe's `SEAM_PROBE_SOURCE` (`crates/testing/iabtl/src/lib.rs` line 83). Each folder writes its own label instead, `trusted-server-integrations//`, so the messages still name the folder. Section 4 says what a reader sees change. + +### 2.4 Maintainers + +Each folder holds a `maintainers.toml` with the two values its manifest's table holds today, and the comment beside the table where it has one, with "vendor crate" and "crate" in the comment written "module", as InMobi's below shows (its manifest's comment, lines 27 to 29, says "this vendor crate" and "this crate"). + +```toml +# The visible owner of this module, the way Prebid.js requires a named +# maintainer of every adapter. InMobi has not yet adopted this module, so the +# Trusted Server maintainers own it until a vendor owner steps up. +owner = "Trusted Server maintainers" +status = "seeking vendor owner" +``` + +The counts stay as they are, 6 `project owned`, 22 `seeking vendor owner` and 4 `vendor owned`, with the asset cache module of row 34 a fifth `vendor owned`, by 51Degrees. A folder whose vendor takes ownership changes its own file, and a vendor that wants a crate of its own later takes its one folder, which holds everything the module has. + +The test `every_crate_outside_core_declares_its_maintainers` in `crates/trusted-server-adapter-axum/tests/shipped_modules.rs` (line 548) reads the manifests under `crates//`. It gives way to `every_folder_declares_its_maintainers` in the integrations crate (section 2.5). + +### 2.5 The folder tests + +Two tests in the integrations crate read the crate's own `src` folder, and replace the 24 name tests and the maintainers test. + +1. **`every_folder_declares_its_maintainers`** reads each `src///maintainers.toml` with the `toml` crate, a workspace dependency, and fails for a folder without the file, an empty `owner`, or a `status` that is not `vendor owned`, `seeking vendor owner` or `project owned`, naming the folder. +2. **`every_folder_names_its_modules_after_itself`** compares the folders on disk with a list in the test of each folder and the names it supplies, and fails for a folder the list leaves out or a listed folder that is gone. Each name must be the folder's `.`, that name with an entry after it, such as `ad-tag.google.diagnostics`, or the same with the type left off, as `51degrees.client_hints` is written under `[middleware]`. The one other name the test allows is `loading-cache.iabtl`, the seam probe's test cache, and it allows that name for the probe's folder alone. The test also fails if any file in the crate calls `module_name!`, and it checks each folder's source label. + +Both are host tests, behind `#[cfg(not(target_arch = "wasm32"))]`, because they read the crate's folders, and the crate's tests also run as WebAssembly under Viceroy, Fastly's local runtime, where there is no file system. Row 55 met the same limit in its discovery tests and changed them to work on folders held in memory (`5be114773`, `2026-10-19-module-scripts-in-crates-design.md`), which these two tests cannot do, because the folders on disk are what they check. Before the row is committed, each test is run once with one folder's file or constant broken, to show that it fails and names the folder. + +The list in the second test is the same kind of check as the catalog completeness test that #1194 describes for its flat folders, which fails when a folder is missing from the catalog or a catalog entry has no folder. + +### 2.6 Browser scripts stay in each folder + +A module's browser script stays where row 55 put it, in its folder's `js/src//`, with its tests in `js/test//`, now under `crates/trusted-server-integrations/src///`. `trusted-server-js` still builds every bundle with its one Node project in `crates/trusted-server-js/lib` and embeds them, so the bundles' names, the URLs they are served at and their hashes do not change, and each of the 14 bundles is the same, byte for byte, as on row 55. + +What finds the scripts follows the move. Each place that walks `crates///js/` walks `crates/trusted-server-integrations/src///js/` instead. + +- `moduleSourceRoots` in `crates/trusted-server-js/lib/build-module-options.mjs`, `expected_module_ids` in `crates/trusted-server-js/build/bundle_set.rs`, and `crate_script_folders` in `crates/trusted-server-js/build.rs`, with the folders `build.rs` watches. +- The patterns `*/*/js/...` in `crates/eslint.config.js`, `crates/.prettierrc.json`, `crates/tsconfig.json`, `lib/vitest.config.ts` and the `lint:crates`, `lint:fix`, `format` and `format:write` scripts of `lib/package.json`, which become `trusted-server-integrations/src/*/*/js/...`. The ignore list in `crates/eslint.config.js` (line 8) lets each folder level back in with a negation of its own (`'!*/'`, `'!*/*/'`, `'!*/*/js/'`), so it takes one negation for each level of the new path, from `'!trusted-server-integrations/'` down to `'!trusted-server-integrations/src/*/*/js/'`, ahead of the two for `js/src/**` and `js/test/**`. +- `lib/tsconfig.json`, whose `include` names `../../*/*/js` and whose `@tsjs/module/*` list names each folder's `js/src`, which the test of that list checks against `moduleSourceRoots()`. + +InMobi's `node --test` files move with their folder, and the step of `.github/workflows/test.yml` that runs `crates/*/*/tests/*.test.mjs` runs `crates/trusted-server-integrations/src/*/*/tests/*.test.mjs`. + +**Why the scripts are not a crate of their own.** #1194 moves every integration's browser code into a second crate, `trusted-server-integrations-js`. This design keeps each script in its module's folder, for four reasons. + +1. Whoever maintains a module works in one folder, which holds its Rust, its script and the script's tests, its fixtures and its `maintainers.toml`, so one file names the owner of all of them. A second crate would put the script under a folder whose owner is not the module's. +2. #1194's reason for a browser crate is to take vendor sources out of the neutral browser crate, because "leaving vendor sources in the neutral crate preserves the ownership inversion". Row 55 already moved the ten modules' scripts that could move out of `trusted-server-js` into the folders that supply them, which a second crate would also do. It left vendor code in `trusted-server-js` that #1194's browser crate would take, being the APS renderer, which the shared browser core imports (`lib/src/core/auction.ts` line 5 and `lib/src/core/request.ts` line 2), and Prebid's code, and the shared browser core still names vendors (`2026-10-19-module-scripts-in-crates-design.md`). This row leaves that code where it is, so for the APS renderer and Prebid the inversion #1194 describes remains, until the browser core stops importing the renderer, as #1194's rule 4 asks, and Prebid's code has a folder of its own. +3. #1194's requirement of one Node toolchain, one lockfile and one set of tool settings is met already. Row 55 builds every module's script with the one Node project in `crates/trusted-server-js/lib`, and finds the scripts in the module folders. +4. #1194 asks for each integration to have one Rust folder and, where it has browser code, one browser folder of the same name. Here the two are the same folder. + +What row 55 left in `trusted-server-js` stays there, being the APS renderer, which the shared browser core imports, Prebid's shim, which imports the `prebid.js` package that a script outside `lib` cannot reach, and Prebid's external bundle build, which the `ts` tool runs from `lib`. + +### 2.7 Licenses + +Every file in the integrations crate takes the repository's license, Apache-2.0, the `license` of `[workspace.package]` and the `LICENSE` file at the root, as every crate under `crates/` does at row 55, where each of the 42 manifests takes `license = { workspace = true }`. Two files under `crates/` carry license text of their own, and this row touches neither. `crates/trusted-server-js/lib/src/core/styles/normalize.css` has its MIT header, as IAB Tech Lab's main has it, and `crates/trusted-server-openrtb/proto/openrtb.proto` gives IAB Tech Lab's copyright under the Apache License 2.0 at line 16. The search below, for the license of the 51Degrees package and for license markers, prints nothing on row 55 and must print nothing on row 56. + +```text +git grep -n -i -E 'EUPL|European Union Public|SPDX-License-Identifier' -- crates +``` + +The package `fiftyone-caching` 4.5.8, which the 51Degrees loading cache folder uses, gives its license as `EUPL-1.2` in its manifest and on crates.io. It is a dependency, so none of its files is in the repository, and only that folder uses it, behind the feature of section 2.8. + +### 2.8 Cargo features + +Three folders use a dependency that not every build can or should carry, one holds a module that no stock build links, and one is used only by tests. A stock build is an adapter or the `ts` tool built from the repository as it ships, with the builders that its own list, its stock list, names. Each of the five folders is compiled behind a feature named after the folder, `-`, and the feature gates the code in the folder, never the folder's name constants or its source label, so the tests of section 2.5 check every folder whatever features are on. + +| Feature | What it compiles | Who turns it on | +| ------------------------- | -------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| `device-fastly` | The Fastly device module, with the Fastly SDK, which calls the Fastly host | The Fastly adapter | +| `geo-fastly` | The Fastly geo module, with the Fastly SDK | The Fastly adapter | +| `loading-cache-51degrees` | The 51Degrees loading cache, with `fiftyone-caching` | No stock adapter and not the `ts` tool. A deployment that links the module (`2026-10-12-loading-cache-design.md`) | +| `asset-cache-51degrees` | The 51Degrees asset cache module | No stock adapter and not the `ts` tool. A deployment that links the module, beside a loading cache module (`2026-10-15-asset-cache-design.md`) | +| `testing-iabtl` | The seam probe | The Axum and Fastly adapters' dev-dependencies, the only links to the probe at row 55, so no adapter build compiles it | + +The Fastly SDK's functions are imports that only the Fastly host and Viceroy supply. `fastly-sys` 0.12.1 declares them at its `src/lib.rs` line 298 and guards no other target, so the two Fastly features belong to the builds for `wasm32-wasip1` that the Fastly adapter and its aliases make. + +```toml +[features] +default = [] +loading-cache-51degrees = ["dep:fiftyone-caching"] +asset-cache-51degrees = [] + +[dependencies] +fiftyone-caching = { workspace = true, optional = true } +``` + +So `cargo tree -p -e normal --target ` finds no `fiftyone-caching` for any of the four adapters, nor for the `ts` tool, and finds the Fastly SDK under the integrations crate only for the Fastly adapter. The aliases in `.cargo/config.toml` name `-p trusted-server-integrations` where they named each module package. `clippy-fastly`, which builds for `wasm32-wasip1`, lints with every feature, as its `--all-features` already gives. The Fastly adapter's dependencies turn on the two Fastly features in `build-fastly`, `check-fastly` and `test-fastly`, and its dev-dependencies turn on `testing-iabtl` in `test-fastly`. The three also name `asset-cache-51degrees`, so that the asset cache module still builds and checks for `wasm32-wasip1` and its tests still run under Viceroy, as `2026-10-15-asset-cache-design.md` has the four Fastly aliases list the module's crate on row 34. None of the three names `loading-cache-51degrees`, as no Fastly alias lists the 51Degrees loading cache at row 55. The native aliases, `test-axum` and `clippy-axum`, turn on `loading-cache-51degrees`, `asset-cache-51degrees` and `testing-iabtl` and leave the two Fastly features to the Fastly aliases, so `clippy-axum` names the features it wants where it passes `--all-features` today. With `testing-iabtl` on, `test-axum` runs the seam probe's tests, so the step of `.github/workflows/test.yml` that tests the probe's package on the host (lines 133 to 139) goes. + +The two features the module crates have today go. `test-utils` of the OpenRTB auction protocol served the APS and Prebid Server crates' tests, and `test-utils` of the 51Degrees shared code served the client hints middleware's tests, and those tests are now in the same crate, so they use the helpers under `#[cfg(test)]`. A test that holds an answer between two calls, as tests of the 51Degrees shared code and the client hints middleware do with the 51Degrees loading cache today, either uses core's `MemoryLoadingCache` or runs only with `loading-cache-51degrees` on, which the native test alias turns on. + +The asset cache module needs no dependency the other folders lack. It sits behind `asset-cache-51degrees` because no stock build links it, as the 51Degrees loading cache sits behind its own feature. + +### 2.9 What a deployment outside the repository changes + +A deployment that builds on the repository through a path to a checkout of it replaces each module package it names with one dependency on the integrations crate, and turns on the features of the folders it links that have one. + +```toml +# Before +[dependencies] +trusted-server-adapter-fastly = { path = "trusted-server/crates/trusted-server-adapter-fastly" } +trusted-server-cmp-inmobi = { path = "trusted-server/crates/cmp/inmobi" } +trusted-server-loading-cache-51degrees = { path = "trusted-server/crates/loading-cache/51degrees" } + +# After +[dependencies] +trusted-server-adapter-fastly = { path = "trusted-server/crates/trusted-server-adapter-fastly" } +trusted-server-integrations = { path = "trusted-server/crates/trusted-server-integrations", features = ["loading-cache-51degrees"] } +``` + +Its code changes each `use` and builder call from the module's package to its path in the crate, by the names of section 2.2, so `trusted_server_loading_cache_51degrees::builder()` is `trusted_server_integrations::loading_cache::fiftyone_degrees::builder()`. Its settings documents do not change, because no module name does. A crate of its own that supplies modules keeps its package and its name, because `module_name!()` still names such a crate by its folder, and it hands its builders to the adapter as before. A git dependency changes in the same way, with `git` and `rev` where the example writes `path`. + +### 2.10 Why one new row at the top + +The crates were made in eleven rows and are folded in one, row 56, above row 55. + +| Row | Crates it made | +| --- | ------------------------------------------------------------------------------------------------------------ | +| 2 | `device/fastly`, `geo/fastly` | +| 3 | The five under `permission-signal/` | +| 7 | Sixteen, from `ad-server/mock` to `testing/testlight`, as the integration seam moved each vendor out of core | +| 18 | `cmp/inmobi` | +| 20 | `ads-txt/sellers` | +| 24 | `loading-cache/51degrees` | +| 30 | `tag/google-tag` | +| 33 | `headers/geo` | +| 44 | `middleware/51degrees`, `shared/51degrees` | +| 46 | `analytics/tinybird` | +| 48 | `shared/aws` | + +Row 43 renamed three of them, `ad-server/mock`, `ads-txt/sellers` and `testing/seam-probe`, to `iabtl`, and row 34 makes the asset cache module. Folding in each of those rows was set aside for four reasons. + +1. **Each fold would travel through the rows above it.** A fold in row 2 changes every row above it that touches a moved file, and 31 of the 48 rows above row 7 change a file in a module crate's folder, so the moves would be merged up through as many as 52 rows, each merge meeting the next row's own changes to the moved files. One fold at the top moves each file once, from the place the rows below leave it. +2. **The rows below stay as reviewed.** 41 of the other design documents in IABTechLab/trusted-server#1342 name a path under `crates///`, as do each row's description and code. A fold in each row would change all of them in rows already reviewed. With one fold at the top, each row's design describes the tree that row has, and this design gives the map from the old paths to the new. +3. **The fold changes how modules are built and not what they do, and one row shows it.** Its review asks one question, whether anything but the paths and the labels of section 4 changed, which this row answers by the same tests passing, the 14 bundles keeping their hashes and every module keeping its name. Row 56 is measured beside row 55, with the same tests, the hashes of the 14 bundles, every module name and the time per request, and the results go in the description of row 56's pull request. +4. **Tech Lab's own move fits in its place.** #1194 also moves the crate boundary in a milestone of its own that preserves behavior, ahead of any change to the configuration. A row at the top can be replaced by Tech Lab's move, or by a later form of this one, without a change to the rows below it. + +### 2.11 Beside IAB Tech Lab's split design + +| What | #1194 | This design | +| --------------------------- | --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| The crate | `trusted-server-integrations` | The same | +| The folders | `src/integrations//`, one level | `src///`, as Tech Lab asked on 2026-10-23 | +| Browser code | A second crate, `trusted-server-integrations-js` | In each module's folder, built by `trusted-server-js` (section 2.6) | +| Module names and settings | One ordered `[integrations]` inventory, in a milestone after the crate move | Unchanged `.` names and the module sections the stack has | +| Owners | No maintainer record | `maintainers.toml` in each folder | +| A crate outside the project | "External vendor crate injection or adapter-supplied registration" is a non-goal | A deployment's own crates still hand builders to the adapters, through `run_with` or `routes_with_registrations` | +| Finding every folder | A static catalog in `lib.rs` and a test that the catalog and the folders agree | Each adapter's own list (row 43) and the folder test of section 2.5 | +| Where vendor names live | Only the integrations crate, which puts the catalog together, as no adapter imports an integration (its rule 6) | Also five lists written by hand, each adapter's `shipped_module_builders()` and the tool's `linked_builders()` | + +Rule 6 of #1194 reads "No adapter reconstructs a concrete catalog or imports `aps`, `prebid`, or another integration module directly." Here each adapter's `shipped_signal_modules()` and the permissions inspector also name the permission signal modules, and section 2.2 lists the other places outside the crate that name a module. + +## 3. What else changes in the repository + +1. The workspace `Cargo.toml` lists one member and one dependency where it listed each module crate, and `Cargo.lock` follows. +2. Each adapter's `Cargo.toml` and the `ts` tool's names `trusted-server-integrations`, with the features of section 2.8, and each `shipped_module_builders()`, `shipped_signal_modules()` and `linked_builders()` names each builder or module by its path in the crate, in the same order, as do the other places section 2.2 lists. +3. The permissions inspector, `tools/permissions-inspector/wasm`, names the integrations crate where it named four permission signal crates. Its build for `wasm32-unknown-unknown` then compiles every folder that needs no feature. The Cloudflare adapter's build for that target compiles each of those folders' crates at row 55, the 51Degrees shared code through the client hints middleware and the others because it names them. +4. In `.cargo/config.toml` the aliases change as section 2.8 says. In `.github/workflows/test.yml` the step that runs InMobi's node tests changes as section 2.6 says, and the step that tests the seam probe's package on the host goes, as section 2.8 says. +5. `crates/fastly.toml` goes. It is the Viceroy manifest for crates three folders deep, because the runner in `.cargo/config.toml` starts Viceroy with `-C ../../fastly.toml` from the crate's folder. From `crates/trusted-server-integrations`, two folders deep, that path reaches the root `fastly.toml`, and nothing else uses `crates/fastly.toml`. +6. `crates/trusted-server-integration-tests/browser/initial-render/run.cjs` reads the Google ad tag's `gpt_bootstrap.js` by its path (lines 70, 166 and 477), which becomes `crates/trusted-server-integrations/src/ad-tag/google/gpt_bootstrap.js`. `.github/workflows/integration-tests.yml` runs that script (line 211) and needs no change. +7. The doc comments in core that call a module's name a crate's path under `crates`, on `IntegrationBuilder::with_module_name` and `with_cache_features`, on `IntegrationRegistrationBuilder::with_geo_module` and at the head of `module_name.rs`, say a folder's path under the integrations crate's `src` as well. The same goes for the documentation of a permission signal's name in `permission_signal/mod.rs` (line 195) and of a middleware's name in `middleware.rs` (line 521), and for `permission_signal/README.md`, whose line 232 names `module_name!()` and whose line 256 says that five crates ship under `crates/permission-signal/`. A test comment in `publisher.rs` (line 7143) names the Google ad tag's `crates/ad-tag/google/js/src/gpt/index.ts`. No code in core changes. +8. The 20 guides under `docs/guide` that name a crate path or package, among them the integration guide, the onboarding guide and `docs/guide/integrations/inmobi.md`, the contributor documents that name one, the sample permission policy `config/permissions/sample.yaml`, whose comment at line 514 names `crates/permission-signal/tcf`, and `CHANGELOG.md`, which says what a deployment changes (section 2.9). + +## 4. What is not in this change + +- **A change of behavior.** No module, middleware, route or setting behaves differently, so there is no consent or privacy effect, and the stock lists hold the same builders in the same order. Two things a reader sees do change with the paths. The source label each builder gives the registry becomes `trusted-server-integrations//` (section 2.3), so the message for two builders registering one id names that label, and the Axum test that expects `trusted-server-identity-lockr` in it (`crates/trusted-server-adapter-axum/tests/routes.rs` line 781) changes with it. Every module's log target becomes its path in the crate, such as `trusted_server_integrations::cmp::inmobi`, and the Fastly adapter's logger prints only the last part of a target (`target_label` in `crates/trusted-server-adapter-fastly/src/logging.rs` lines 11 to 17). On Fastly a line logged in a folder's `mod.rs` therefore carries only the last part of the folder's Rust path, so the three `iabtl` folders log as `iabtl`, the 51Degrees folders as `fiftyone_degrees` and the two Fastly folders as `fastly`, where each crate logged under its own name. +- **A change to module names or settings.** No section, table or entry changes. +- **IAB Tech Lab's `[integrations]` inventory.** #1194 changes the configuration in a milestone of its own, and this row does not. +- **A crate for browser code.** Section 2.6 says why. +- **A crate for each vendor.** A vendor that takes ownership of its module may later ask for one, and its folder is what such a crate would hold. + +## 5. Acceptance + +1. Every test alias passes on row 56. The native aliases run the tests they ran on row 55, apart from the 24 name tests and the maintainers test that section 2.5 replaces, and they also run the seam probe's tests, which the workflow step this row removes ran on row 55. `test-fastly` also runs, under Viceroy, the tests of eight folders it did not list on row 55, being the five permission signals, the 51Degrees shared code, the client hints middleware and the seam probe, because those folders are now in a crate it tests. The ninth it did not list, the 51Degrees loading cache, stays out, because its feature is off there. A test among them that cannot run under Viceroy is marked for the host, such as the 51Degrees shared code's test that asks from two threads at once (`crates/shared/51degrees/src/client.rs` line 706). +2. Each of the 14 bundles `build-all.mjs` writes has the same SHA-256 as on row 55. +3. `every_folder_declares_its_maintainers` and `every_folder_names_its_modules_after_itself` pass, and each fails, naming the folder, when one folder's `maintainers.toml` is removed or one name constant is changed. +4. `cargo tree -e normal` finds no `fiftyone-caching` for any adapter, on its own target, or for the `ts` tool, and finds the Fastly SDK under the integrations crate only for the Fastly adapter. +5. The license search of section 2.7 prints nothing. +6. `git log --follow` on a moved file, such as `crates/trusted-server-integrations/src/cmp/inmobi/mod.rs`, shows the file's history from before the move. +7. No settings document, fixture or example in the repository changes, apart from the paths that sections 2.1 and 3 list and paths in comments. + +## 6. Open questions + +These are for the reviewer, and this design decides none of them. + +1. **The Rust name of `51degrees`.** `fiftyone_degrees` writes the number the way 51Degrees' own packages do. Another rule, such as a leading `_` (`_51degrees`) or the vendor's own short form (`fiftyone`), would serve, and the choice reaches every deployment's `use` lines. +2. **The seam probe.** It folds into `src/testing/iabtl/` behind `testing-iabtl`. Kept as a crate of its own instead, it would go on proving that a crate beside the integrations crate, as a deployment's own is, plugs into the adapters. Which do reviewers prefer? +3. **Where the folder tests live.** They sit in the integrations crate, as host tests. The maintainers test sits today in the Axum adapter's tests, which run only on the host. Should they stay together there? +4. **Which crate embeds the bundles.** `trusted-server-js` still embeds every bundle, the vendor ones included, as on row 55, and core depends on `trusted-server-js`, so core's build carries every vendor's browser script. Should the integrations crate embed its folders' bundles itself, as #1194 has its browser crate do? +5. **A rebuild after one vendor file changes.** A change to one folder's file rebuilds the whole integrations crate, where today it rebuilds that module's crate and what depends on it. Is that an acceptable cost for module authors, or should a folder whose vendor works on it often be extracted into a crate of its own? +6. **Features for every folder.** Only the five folders of section 2.8 have a feature. Should every folder have one, so that a deployment compiles only the modules it links? +7. **One stock list.** The four adapters and the tool each name every builder in a list written by hand, and `2026-10-14-no-modules-crate-design.md` says that no test checks that the five lists agree. In #1194 the integrations crate composes the catalog and no adapter imports an integration directly. Should the integrations crate supply one stock list that the adapters and the tool take, with a deployment that links a different set still writing its own? +8. **Log labels on Fastly.** After the fold, folders that share a vendor share a log label on Fastly (section 4). Should each folder give its log lines a target of its own, or should the Fastly logger print more of the target? + +## Revision record + +| Date | Change | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-26 | First draft. | +| 2026-10-26 | Fixes from review round 1. tokio's `rt-multi-thread`, which the loading cache's tests ask for, goes in a table that only native builds read, with the threaded tests on the host. The paths that climb out of a crate's folder are listed with their fixes, and `run.cjs`, `crates/fastly.toml`, the seam probe's workflow step and the core documents that name a crate's path join section 3. The browser scripts' second reason says that the APS renderer and Prebid's code stay in `trusted-server-js`, where #1194 would move them. Row 56's own measurement goes in its pull request. The manifest's lint and doctest settings, the source labels written as text, the log labels on Fastly, the places the Rust names are used, the rule for `51degrees`, the probe's one name of another type, the Fastly SDK's host imports, the files with license text of their own, the count of rows a fold would travel and where vendor names live beside #1194 are stated. Open questions 4 and 5 are reworded and 7 and 8 added. The Touchbase of 2026-10-23 says what Tech Lab asked, and that 51Degrees decided, and the seam probe and Viceroy are defined where first used. | +| 2026-10-26 | Fixes from review round 2. The split design's pull request names its author, Aram Grigoryan. The account of the Touchbase gives Tech Lab's request as the record has it, and places the decisions of 51Degrees after the Touchbase. "Stock build" is defined where first used, the split design's integrations crate is the one that puts the catalog together, and a feature gates neither a folder's name constants nor its source label. `build-fastly`, `check-fastly` and `test-fastly` name `asset-cache-51degrees` and none of them names `loading-cache-51degrees`, each `maintainers.toml` comment says "module" where the manifest says "crate", and the sample permission policy joins section 3. | + +## Review log From 8ce9f744463204dd07658fb472bb820235717bcf Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:30 +0000 Subject: [PATCH 174/192] Review logs record each round of the cache designs The loading cache, asset cache, integrations crate and module cache declarations designs end with a review log entry for each round they took, as CONTRIBUTING.md asks. --- docs/superpowers/specs/2026-10-12-loading-cache-design.md | 2 ++ docs/superpowers/specs/2026-10-15-asset-cache-design.md | 2 ++ .../specs/2026-10-15-module-cache-declarations-design.md | 2 ++ docs/superpowers/specs/2026-10-26-integrations-crate-design.md | 3 +++ 4 files changed, 9 insertions(+) diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index 6b40eeeaf1..4b5fe21492 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -237,3 +237,5 @@ These are for the reviewer. This revision records what the code does, and what t 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/31-caching-release` at `4b6ef9c05`, with the design's code read on `split/24-loading-cache` at `844b01fe2`. Issues found, three of them blocking. The three were that the spec said a feature that names no module gets `NoLoadingCache`, when a stock build gives it the one loading cache module linked, that the spec said loads run on a pool of threads on a native host, when every load runs in its caller and Axum has no key-value store, and that the spec said a module is asked once per refresh however many instances want its rules, when only a store that reserves a key makes other instances wait. This revision states what the code does on both rows, adds the adapters, the configuration, consent and privacy, failure, compatibility, the tests and the pull requests that exist now, and leaves seven code questions under Open questions, which wait for code changes. 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/31-caching-release` at `4b6ef9c05`, with the design's code read on `split/24-loading-cache` at `844b01fe2`. Issues found, none blocking. This revision states how the two published crates differ, from a file by file comparison of the crates themselves, and confirms the check counts, the pull request states and the description of IABTechLab/trusted-server#1365 that this spec cites. +3. **Round 3.** A review in a fresh session, against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. Issues found, one of them blocking. The change that takes the 51Degrees module out of the stock lists named only rows 24 and 43, when row 32's probe test counts the module as linked and each adapter's order test names it, and the change's guard test read only Axum's list, when five stock lists, the `ts` tool's among them, could each link the module again. This revision takes row 32 into the change with those tests, puts a guard test beside every stock list, with a test that pins the tool's list, and spells out what a stock build then costs, being a call to the cloud service on each page the client hints middleware is placed for and a key-value write on each robots.txt request where a deployment adds a robots.txt module. It also lists the other text that says a stock build holds answers, asks under Open questions whether loading through `NoLoadingCache` with nothing logged should be logged or refused, and gives the design's line numbers at the current heads of rows 24 and 31. +4. **Round 4.** A review in a fresh session, checking the round 3 fixes against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. Issues found, one of them blocking. The design said that once the 51Degrees module leaves the stock lists, the first case of row 32's probe test still shows the table selecting the module it names, when that case's build would then link only the probe's cache, which a table naming no module takes anyway. This revision has that case link a second loading cache module, a test builder offering core's `MemoryLoadingCache`, so that the case still shows that the table is read, adds the integration guide, the 51Degrees client's comments and the client hints middleware's tests to the text the change rewrites, defines stock build where it is first used, and links the source of `fiftyone-caching` at the commit that `4.5.8` was published from. diff --git a/docs/superpowers/specs/2026-10-15-asset-cache-design.md b/docs/superpowers/specs/2026-10-15-asset-cache-design.md index 93b2bd8d01..3d3ac6c26d 100644 --- a/docs/superpowers/specs/2026-10-15-asset-cache-design.md +++ b/docs/superpowers/specs/2026-10-15-asset-cache-design.md @@ -455,3 +455,5 @@ Row 34's `assets_turned_off_in_the_settings_are_fetched_for_every_request` and ` 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/34-asset-cache` at `67af14ea4`. Issues found, one of them blocking. The asset gate leaves out the page path's cookie rules, so a request that carries a session cookie can read a stored copy or lead a load whose response is stored for every reader, and the spec did not say so. This revision states the cookie rule as the code has it today, with the reason the code gives and the default-on state, puts the choice of cookie rules and an off switch under Open questions, fills the coverage gaps and corrects the partly true claims, while the fixes to the code, its comments and its tests that the review advised wait for code changes. 2. **Round 2, 2026-10-15.** A lighter review in a fresh session, checking the round 1 fixes against `split/34-asset-cache` at `e9832a39e`. Issues found, none blocking. This revision states the cookie rule, the off switch and the range check of `[cache.assets]` as the code now has them, says which entries each load writes, removes the claim that no test sends a `Cookie` header through the publisher path, which navigation tests and now the asset gate's tests do, and gives the one linked module as the default in the guide and the code comments. +3. **Round 3.** A review in a fresh session, against `split/34-asset-cache`. Issues found, three of them blocking. The three were that row 46 changes the keep check that this rewrite moves out of core, and the design never named row 46, that core's planned tests no longer covered what core reports for the cookie rule once that rule's end-to-end test moves into the module, and that the tag the variant digest hashes needed a new name in the move. This revision keeps `origin_response_may_be_kept` in core as a public function, so that row 46 changes it once for core and the module, adds core tests with a recording test module that show what core reports, and gives the variant digest the tag `ts-asset-variant-v1`, in the form of the key's `ts-asset-cache` tag, which changes every copy key, so a copy kept before simply expires. It also has core offer the module only `GET` requests and take the module only from the builder that `[asset-cache]` selects, sets out `AssetFile`, `AssetVia` and `OriginRequest`, names the caches `51degrees-assets` and `51degrees-assets-vary`, and puts the module behind the `asset-cache-51degrees` feature after row 56. +4. **Round 4.** A review in a fresh session, checking the round 3 fixes against `split/34-asset-cache`. Issues found, none blocking. This revision links each pull request and the issue where first named, and the CI record at its commit, defines stock build, Viceroy and the seam probe where first used, names Fastly's Core Cache API as IABTechLab/trusted-server#894 names it, calls the form of the cache names the form row 32 asks for, where it called that form a rule, and says that on Spin the entries a new key or a lifetime leaves behind stay in the store. diff --git a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md index d694ebd8c6..4d320e2db0 100644 --- a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md +++ b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md @@ -252,3 +252,5 @@ These joins have no test. 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/32-cache-features` at `0ef3ff78d`. Issues found, three of them blocking. The page said a deployment can shorten a declared cache's lifetimes although core applies none of them, said two modules of one publisher never share a copy although nothing checks the names a module loads through, and said the changelog entry names the three changes for crates outside core although the entry left out two of them. This revision states each as the code stands today, adds the order of a push, the other callers of `loading_caches_from`, the names core does not check, the cache made once and the other untested paths, and corrects the changelog entry, whereas the decisions, and the code changes that depend on them, wait on the open questions in 2.10. 2. **Round 2, 2026-10-15.** A lighter review in a fresh session, checking the round 1 fixes against `split/32-cache-features` at `be4f47842`. Approved, none blocking. This revision corrects the three statements that were only partly true, being what a module that borrows another module's cache name shares, which section the guide's push-order warning covers and which builders the deploy check is given, corrects the round 1 sentence on the code changes, and adds the guide, rustdoc and visibility follow-ups to 2.10. +3. **Round 3.** A review in a fresh session, against `split/32-cache-features`, of the loading cache change in `2026-10-12-loading-cache-design.md`, which reaches this row. Issues found, one of them blocking. The loading cache change named only rows 24 and 43, while section 2.2 of this design said that a stock build holds a module's answers without a line in the document, and the Axum test that section 4 describes counts the stock loading cache module as linked, so both stop being true once no stock build links a loading cache module. This revision says that a build linking one loading cache module holds those answers, that a stock build then links none, so a declared cache there holds nothing, and that the Axum test changes with the stock lists, as the loading cache design sets out. +4. **Round 4.** A review in a fresh session, checking the round 3 fixes against `split/32-cache-features`. Approved for this design, none blocking. The round's one blocking point on the loading cache change was that the first case of the Axum test in section 4 would then pass whether or not the table's `module` is read, and the loading cache design now has that case link a second loading cache module, a test builder offering core's `MemoryLoadingCache`, so that the case still shows that the table is read. diff --git a/docs/superpowers/specs/2026-10-26-integrations-crate-design.md b/docs/superpowers/specs/2026-10-26-integrations-crate-design.md index d6ea6dd6ea..1b8432ef40 100644 --- a/docs/superpowers/specs/2026-10-26-integrations-crate-design.md +++ b/docs/superpowers/specs/2026-10-26-integrations-crate-design.md @@ -326,3 +326,6 @@ These are for the reviewer, and this design decides none of them. | 2026-10-26 | Fixes from review round 2. The split design's pull request names its author, Aram Grigoryan. The account of the Touchbase gives Tech Lab's request as the record has it, and places the decisions of 51Degrees after the Touchbase. "Stock build" is defined where first used, the split design's integrations crate is the one that puts the catalog together, and a feature gates neither a folder's name constants nor its source label. `build-fastly`, `check-fastly` and `test-fastly` name `asset-cache-51degrees` and none of them names `loading-cache-51degrees`, each `maintainers.toml` comment says "module" where the manifest says "crate", and the sample permission policy joins section 3. | ## Review log + +1. **Round 1.** A review in a fresh session, against `split/55-scripts-in-crates`. Issues found, three of them blocking. The three were that one manifest would carry the loading cache's test dependency on tokio's `rt-multi-thread`, which tokio refuses to build for WebAssembly, so `test-fastly` and `clippy-fastly` would stop, that three paths climb out of a crate's folder and break after the move, and that the design said a separate browser crate would move no more vendor code out, when the APS renderer and Prebid's code stay in `trusted-server-js`, where IABTechLab/trusted-server#1194 would move them. This revision puts that dependency in a table that only native builds read, with the threaded tests on the host, lists every path that climbs out of a folder with its fix, and says where the APS renderer and Prebid's code stay. It also states the manifest's lint and doctest settings, the source labels written as text, the log labels on Fastly and where vendor names live beside IABTechLab/trusted-server#1194, and leaves row 56's own measurement to that row's pull request. +2. **Round 2.** A review in a fresh session, checking the round 1 fixes against `split/55-scripts-in-crates`. Issues found, none blocking. This revision names the author of Tech Lab's split design, Aram Grigoryan, gives Tech Lab's request at the Touchbase as the record has it and places the decisions of 51Degrees after the Touchbase, defines stock build where first used, and calls the split design's integrations crate the crate that puts the catalog together. It also says that a feature gates neither a folder's name constants nor its source label, that `build-fastly`, `check-fastly` and `test-fastly` name `asset-cache-51degrees` and none of them names `loading-cache-51degrees`, and adds the sample permission policy to section 3. From 1fce02556eedfedfcdbf4e830db229ee82f202f7 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:31 +0000 Subject: [PATCH 175/192] Four designs follow the cache module change The module cache declarations, asset routes, no modules crate and client hints designs no longer say that a stock build links a loading cache module or that core holds the publisher's assets, following the loading cache and asset cache designs. The ts command's list carries no loading cache module, the Axum probe test links core's MemoryLoadingCache beside the probe's module and a fourth case links none, the asset routes offer a GET to the module [asset-cache] selects and keep no copy in core, and the 51Degrees cloud answer is held only where a deployment links a loading cache module. --- ...026-10-14-51degrees-client-hints-design.md | 25 +++++++++++-------- .../2026-10-14-no-modules-crate-design.md | 5 ++-- ...-10-15-module-cache-declarations-design.md | 7 +++--- ...17-asset-routes-on-every-adapter-design.md | 5 ++-- 4 files changed, 25 insertions(+), 17 deletions(-) diff --git a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md index 1fc595b5e0..cc89256a77 100644 --- a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md +++ b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-12-shared-vendor-tables-design.md` (the `[shared.51degrees]` table), `2026-10-12-loading-cache-design.md` (the `51degrees-answers` cache), `2026-10-12-page-middleware-design.md` (the `[[serve]]` entries), `2026-10-14-source-modules-design.md` (the source a page is fetched from) and `2026-10-17-request-filters-on-every-adapter-design.md` (the request filters every adapter runs), all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** jwrosewell/trusted-server#39, IABTechLab/trusted-server#1386 **Issues:** IABTechLab/trusted-server#1333 -**Last updated:** 2026-10-18 +**Last updated:** 2026-10-26 > **Why this spec exists.** A browser sends its high entropy client hints only when the site asks for them and delegates them, and nothing in the stack writes the headers and the tag that do so for a deployment that uses the 51Degrees service, so every first page view of a session is priced on the User-Agent alone. @@ -88,19 +88,23 @@ every server side call presents, and the refusal when the table has none. The one call to the service for a request's evidence, a form POST to the service's `json` endpoint over the platform's HTTP client, naming the properties each module registered so one call carries them all, and its -answer held in the loading cache so a question many requests ask at once is -asked once. The evidence a request gives, being the `User-Agent` and the -`Sec-CH-UA` hints and nothing else. The names the service asked for, as the -type a module reads them from the request's state by. +answer loaded through the loading cache, so that where a deployment links a +loading cache module a question many requests ask at once is asked once, and +in a stock build, which links none, every call asks the service +(`2026-10-12-loading-cache-design.md`, section 7). The evidence a request +gives, being the `User-Agent` and the `Sec-CH-UA` hints and nothing else. The +names the service asked for, as the type a module reads them from the +request's state by. **The cache is a declared one.** The builder declares `51degrees-answers`, and a deployment selects the loading cache that holds it and sizes its copies -in `[cache.51degrees-answers]`, as every declared cache is selected. A stock -build holds the answers with its one loading cache module without a line in -the document. A build that links no loading cache module, or several with no +in `[cache.51degrees-answers]`, as every declared cache is selected. A build +that links one loading cache module holds the answers without a line in the +document. A build that links no loading cache module, or several with no `[cache.51degrees-answers]` table naming one, holds nothing, so every page -navigation an entry covers calls the service. On Fastly the copies are held in -the host's value cache, shared by every instance at a location, with a brief +navigation an entry covers calls the service, and every stock build links +none. On Fastly the copies are held in the host's value cache, shared by every +instance at a location, with a brief copy in process memory in front. An answer is held for an hour at most. The key is a hash of the endpoint, the license key, the properties and the evidence sent, being the `User-Agent` and the `Sec-CH-UA` hints, so one held @@ -317,6 +321,7 @@ Six things have no test. | 2026-10-15 | Fixes from review round 2. The pull request that holds the design documents, the wait for the first call in the guide, and the code comments left for a later change. | | 2026-10-15 | Code fixes for the call's scope, what it sends and header merging. The permission question stays open, with what was found, and the answer lifetime is a new open question. | | 2026-10-18 | Every adapter runs the filter, from `2026-10-17-request-filters-on-every-adapter-design.md` and its first review round. The table of parts, the builder, the `timeout_ms` setting, what bounds the call on each adapter, consent, failure, rollout, section 3 and the tests say so. Fixes from review round 3, the wait on Axum for a service that cannot be reached and test item 6. | +| 2026-10-26 | The service's answer is held only where a deployment links a loading cache module, and a stock build, which links none, calls the service on every page navigation an entry covers (`2026-10-12-loading-cache-design.md`, section 7). The table of parts and the declared cache say so. | ## Review log diff --git a/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md b/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md index b90dc83ba7..a3697f2021 100644 --- a/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md +++ b/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-14-source-modules-design.md`, in IABTechLab/trusted-server#1342, which replaced IABTechLab/trusted-server#1084 as the one place for every design document of the stack of pull requests listed in IABTechLab/trusted-server#777 **Related PRs:** IABTechLab/trusted-server#1384, jwrosewell/trusted-server#38 **Issues:** IABTechLab/trusted-server#1334 -**Last updated:** 2026-10-15 +**Last updated:** 2026-10-26 > **Why this spec exists.** One crate held the list of the modules a stock build ships on every adapter's behalf, with a table saying which middleware each supplies, so a deployment that wanted a different set edited a crate that was not its own, and a module's description of itself lived in a crate that was not the module's. The same change renames the three modules the project owns itself, and those renames are the only part that changes what an operator writes in settings. @@ -81,7 +81,7 @@ The seam probe is a test fixture that no deployment ships, so its renames reach - A description the registration contradicts is refused at startup, naming the module, the middleware and the difference, for a wrong phase and for a wrong media type, and a description the registration bears out is accepted. - Each stock module that supplies middleware, core's script proxy and the seam probe describe exactly what they register under their own fixture settings, in registration order. - Each adapter lists its builders in the order their hooks run, and `module_builders_with` puts a deployment's builders after them. -- The Axum adapter's linked builders give what the stock list gave: a feature selects the stock loading cache by name, every page module refuses a setting it does not know, the diagnostics module's request preparer is attached, the template's entries and the guide's table name the middleware the linked modules supply in order, the documented tables validate when uncommented, deploy validation runs a module's own rules and lists its secret settings, the ad server keeps an APS winner's identities, and every crate outside core declares its maintainers. +- The Axum adapter's linked builders give what the stock list gave: no loading cache module is linked, so every feature loads through `NoLoadingCache` unless a deployment links one (`2026-10-12-loading-cache-design.md`), every page module refuses a setting it does not know, the diagnostics module's request preparer is attached, the template's entries and the guide's table name the middleware the linked modules supply in order, the documented tables validate when uncommented, deploy validation runs a module's own rules and lists its secret settings, the ad server keeps an APS winner's identities, and every crate outside core declares its maintainers. - `ts audit` finds a module's selection by its integration id from the builders the tool links, and writes `text/html` entries that run the selected modules' middleware in registration order. Three things have no test. The first is the five lists agreeing. Each adapter's order test pins its own list against a copy of the same twenty names, so the four lists agree today, but nothing compares one list with another, and the `ts` list has no order test of its own. Its order is checked only through `build_draft_config_places_the_page_changes_of_the_modules_it_selects`, for the modules that test selects, and the integration tests crate validates its fixtures against the Axum adapter's list alone. Adding a module takes five edits and a missed one passes every test, so a test that gives the same ids in the same order from all five lists is missing. The second is the registry accepting a described middleware that the registration leaves out, because no test has a registration omit a middleware its builder describes. The third is `ts audit` leaving out a middleware described for another media type, because the draft test that checks the entries selects `gpt`, `datadome`, `didomi` and `google_tag_manager`, whose middleware are all described for `text/html`, and `build_page_entries` has no direct test. @@ -93,6 +93,7 @@ Three things have no test. The first is the five lists agreeing. Each adapter's | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The other users of the old crate, the test that kept its table in step, the renamed packages and the seam probe's names, the adapters and the list functions, what does not change, the startup refusal, the rollout of the renamed selectors, what `ts audit` reads and writes, the missing tests, `iabtl` and the stack defined, the renames in the title, and the pull requests raised. | | 2026-10-15 | Fixes from review round 2. The claimed test for the media types `ts audit` writes, the three changes left for a later change, and the changelog entries that still named the removed crate. | +| 2026-10-26 | The Axum adapter's tests in section 4 say that its linked builders include no loading cache module, so every feature loads through `NoLoadingCache` unless a deployment links one (`2026-10-12-loading-cache-design.md`), where they said a feature selects the stock loading cache by name. | ## Review log diff --git a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md index 4d320e2db0..95e5d49639 100644 --- a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md +++ b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md @@ -199,7 +199,7 @@ The first two questions ask for a code change and the next two for a decision. T - **A check that a module calls only the names it declared.** `LoadingCaches::for_feature` answers `NoLoadingCache` for a name no one declared, and the cache selected for the name where another module or core declared it, see 2.6. - **A check of a declared name's spelling,** or of whether the name starts with the module's own name. Core reserves only `robots-txt`. `asset_rules` is read as the list of asset rules, so a builder can declare it and no table can then select a module for it. TOML reads `[cache.a.b]` as a table `b` inside a table `a`, so a name with a dot can be written only quoted, as `[cache."a.b"]`. A name with a `/` blurs where the name ends and the key begins in `//`. - **A warning when a declared cache holds nothing** because the build links several loading cache modules and no table names one. -- **A deploy check that knows a deployment's own loading cache module.** The deploy check refuses a table whose `module` names a crate that offers no loading cache, as the state build does, through `check_loading_cache_modules`, which `validate_settings_for_deploy_with` in `crates/trusted-server-core/src/config.rs` calls. It compares `module` with the loading cache modules of the builders it is given, and the deploy check is given core's built-in builders followed by the ones a tool registers, where the adapters are given the stock modules followed by the deployment's own, see 2.4. No built-in builder offers a loading cache, and the `ts` command registers the stock list, which carries the stock loading cache module, so `ts config validate` refuses a table naming a deployment's own loading cache module unless a tool registers that module. +- **A deploy check that knows a deployment's own loading cache module.** The deploy check refuses a table whose `module` names a crate that offers no loading cache, as the state build does, through `check_loading_cache_modules`, which `validate_settings_for_deploy_with` in `crates/trusted-server-core/src/config.rs` calls. It compares `module` with the loading cache modules of the builders it is given, and the deploy check is given core's built-in builders followed by the ones a tool registers, where the adapters are given the stock modules followed by the deployment's own, see 2.4. No built-in builder offers a loading cache, and the `ts` command registers the stock list, which carries no loading cache module once the 51Degrees module leaves the stock lists (`2026-10-12-loading-cache-design.md`), so `ts config validate` refuses a table naming any loading cache module unless a tool registers that module. - **Declared caches in the section "Supplying a loading cache".** The integration guide, `docs/guide/integration-guide.md`, lists `.with_cache_features(...)` among the builder methods, and its section "Supplying a loading cache" does not mention declared caches. ## 4. Acceptance @@ -223,7 +223,7 @@ In `crates/trusted-server-core/src/settings.rs`: In `crates/trusted-server-adapter-axum/tests/seam_probe.rs`, through the Axum router: -- `a_cache_the_probe_declares_is_selected_in_its_own_table` shows three cases. A table naming `seam-probe` makes the cache keep values, and the build links the stock loading cache module as well as the probe's, so the cache would keep nothing if the table's `module` were not read. The probe's cache and the stock cache both keep values, so the case does not show which of them the table selected. The same two modules and no table make the cache keep nothing. The stock loading cache module alone, with no table, is taken and keeps values. +- `a_cache_the_probe_declares_is_selected_in_its_own_table` shows four cases. A table naming `seam-probe` makes the cache keep values, and the build links a second loading cache module beside the probe's, a test builder offering core's `MemoryLoadingCache`, so the cache would keep nothing if the table's `module` were not read. The probe's cache and that second cache both keep values, so the case does not show which of them the table selected. The same two modules and no table make the cache keep nothing. The probe's loading cache module alone, with no table, is taken and keeps values. No loading cache module and no table, which is the stock list with the probe, make the cache keep nothing. - `a_cache_table_no_module_declares_is_refused_by_the_deploy_check_and_at_startup` shows `validate_settings_for_deploy_with` and `routes_with_registrations` refusing a misspelled table with the same message, listing `robots-txt, seam-probe-answers`. These joins have no test. @@ -232,7 +232,7 @@ These joins have no test. 2. **`ts config validate` and `TrustedServerAppConfig::validate`.** The Axum test calls `validate_settings_for_deploy_with`, the function that `ts config validate` and `TrustedServerAppConfig::validate` both reach. No test registers a builder that declares a cache through `register_deploy_integrations`. 3. **A range error on a declared table.** The range test covers `cache.robots-txt` only, so the loop over `CacheSettings::declared` in `prepare_runtime` has no test. 4. **A declared table's lifetimes reaching a cache.** Nothing in core applies them (see 2.3), so no test shows `memory_seconds`, `store_seconds` or `store_idle_seconds` of a declared table reaching a cache. -5. **A declared table choosing one of several loading cache modules.** The core test links one module and the Axum test links two whose caches both keep values, so no test shows which module a declared table selected. Only `[cache.robots-txt]` has a test that tells two modules apart, `a_feature_selects_a_module_by_its_short_name_or_in_full`. No test in `crates/loading-cache/51degrees` or `crates/trusted-server-modules` selects the stock loading cache module for a declared cache by name, and the stock module is reached for a declared cache only as the one module linked, in the third case of the Axum test. +5. **A declared table choosing one of several loading cache modules.** The core test links one module and the Axum test links two whose caches both keep values, so no test shows which module a declared table selected. Only `[cache.robots-txt]` has a test that tells two modules apart, `a_feature_selects_a_module_by_its_short_name_or_in_full`. No test in `crates/loading-cache/51degrees` or `crates/trusted-server-modules` selects the 51Degrees loading cache module for a declared cache by name, and no case of the Axum test links that module, because no stock build links it (`2026-10-12-loading-cache-design.md`), so no test reaches it for a declared cache. 6. **The deploy check's builder list against the adapters'.** No test covers a built-in builder that declares a cache. Only the deploy check would see that cache. 7. **A module loading through a name it did not declare.** Nothing refuses the call, and no test shows what happens for a name no one declared, for a name another module declared, or for `robots-txt`. 8. **A module's cache made once, and shared by a declared cache with no table.** Rule 3 in 2.2 and the sharing in the step for a declared cache with no table have no test. No test checks that two tables naming one module, or a table and a declared cache with no table, are given one cache. @@ -247,6 +247,7 @@ These joins have no test. | 2026-10-17 | Who can read a copy names the shared store on each adapter, being Fastly's cache, the key-value store the Cloudflare and Spin adapters declare on row 24, `TRUSTED_SERVER_KV` and `default`, and none on Axum. | | 2026-10-18 | Items 5 to 8 leave 2.10. The configuration guide says core applies no lifetime of a declared table and when to push one, the integration guide lists `.with_cache_features(...)`, and `declared_caches` and `check_cache_tables` are `pub(crate)`. The rustdoc of `loading_caches_from` and `LoadingCacheConfig::module`, and the integration guide on a table naming no module, already gave rule 2 in 2.2. Section 3 and 2.8 match the guides. 2.7 and section 3 say the deploy check also refuses a `module` that offers no loading cache. | | 2026-10-26 | 2.2 says that a build linking one loading cache module holds a declared cache's answers without a line in the document, where it said a stock build does, because no stock build links one once the 51Degrees module leaves the stock lists (`2026-10-12-loading-cache-design.md`). | +| 2026-10-26 | Section 3, the Axum test in section 4 and the fifth join with no test follow the stock lists linking no loading cache module (`2026-10-12-loading-cache-design.md`). The `ts` command's list carries none, so `ts config validate` refuses a table naming any loading cache module unless a tool registers it, the Axum test's first two cases link core's `MemoryLoadingCache` beside the probe's module and a fourth case links none, and no test reaches the 51Degrees module for a declared cache. | ## Review log diff --git a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md index fb69795291..0ffd46f231 100644 --- a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-17-request-filters-on-every-adapter-design.md`, `2026-10-17-edge-cookie-on-every-adapter-design.md`, `2026-10-14-source-modules-design.md`, `2026-10-14-origin-signing-design.md`, `2026-10-15-asset-cache-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 **Related PRs:** [IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412). The request filters run on every adapter in [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410) and the Edge Cookie in [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411) **Issues:** none -**Last updated:** 2026-10-18 +**Last updated:** 2026-10-26 > **Why this spec exists.** Only the Fastly adapter serves `[[proxy.asset_routes]]`. On Axum, Cloudflare and Spin a request under a route's prefix goes to the publisher's origin as an ordinary page request, so a route's own source, its signing and its held answers are never used there. This change dispatches asset routes on those three adapters as the Fastly adapter does. Two parts of it reach beyond asset routes. On Spin a source whose `host_header` names another host is now refused, the publisher's source as well as a route's, so a Spin deployment whose publisher source sets one answers every page with `502`, which is a breaking change. And the Cloudflare and Spin adapters now install a logger, because neither had one and every line they logged was dropped, the warnings this change adds included. @@ -25,7 +25,7 @@ Neither the Cloudflare adapter nor the Spin adapter installs a logger. Each hand **Finalizing.** `FinalizeResponseMiddleware` takes the cache policy off the response and applies it after the finalize headers with the adapter's own shared cache header (`AssetProxyCachePolicy::apply_after_route_finalization`), before the Edge Cookie and the filters' effects, the order the Fastly entry point uses. Axum and Spin write a shared cache lifetime as `s-maxage` in `Cache-Control`, and Cloudflare writes it in `cloudflare-cdn-cache-control`. -**Held answers.** `handle_asset_proxy_request` reads and keeps its copies through the loading caches `[cache.assets]` selects, as on Fastly, and keeps no copy of the answer to a request that carries Image Optimizer metadata. The stock loading cache module keeps a memory layer for each application state and its shared copies in the host's value cache where the services carry one, which only the Fastly adapter supplies, and in the key-value store otherwise (`crates/loading-cache/51degrees/src/stores.rs`). Cloudflare and Spin build their state for every request, so there a copy outlives its request only in the key-value store each declares, `TRUSTED_SERVER_KV` on Cloudflare and `default` on Spin. Axum builds its state once for the process, but its key-value store is the unavailable one, so a copy lives only in memory, for `[cache.assets] memory_seconds`, 5 seconds by default. An asset fetch also asks for the platform's default cache behavior (`PlatformCacheIntent::Default`), which the Cloudflare client leaves to the Workers runtime, whose subrequests are eligible for Cloudflare's cache by default, so on Cloudflare the runtime may hold an answer as well. +**Held answers.** `handle_asset_proxy_request` offers each `GET` that the platform's Image Optimizer does not answer to the asset cache module `[asset-cache]` selects before asking the route's origin, as on Fastly, and keeps no copy itself (`2026-10-15-asset-cache-design.md`), so a deployment that selects no module, a stock build among them, asks the origin for every answer. The 51Degrees asset cache module holds its copies, and its records of what each address varies on, through the loading caches `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]` select, and holds nothing in a build that links no loading cache module. The 51Degrees loading cache module keeps a memory layer for each application state and its shared copies in the host's value cache where the services carry one, which only the Fastly adapter supplies, and in the key-value store otherwise (`crates/loading-cache/51degrees/src/stores.rs`). Cloudflare and Spin build their state for every request, so there a copy outlives its request only in the key-value store each declares, `TRUSTED_SERVER_KV` on Cloudflare and `default` on Spin. Axum builds its state once for the process, but its key-value store is the unavailable one, so a copy lives only in memory, for `[cache.51degrees-assets] memory_seconds`, 5 seconds by default. An asset fetch also asks for the platform's default cache behavior (`PlatformCacheIntent::Default`), which the Cloudflare client leaves to the Workers runtime, whose subrequests are eligible for Cloudflare's cache by default, so on Cloudflare the runtime may hold an answer as well. **The body.** None of the three HTTP clients streams a response (`supports_streaming_responses` is `false` there), so core asks for no stream and each client holds the asset body whole, where the Fastly adapter streams it with no limit. The Cloudflare client refuses a body over 10 MiB (`MAX_PLATFORM_RESPONSE_BODY_BYTES`) and the Spin client one over 8 MiB, before or after it is decompressed (`MAX_DECOMPRESSED_SIZE`), and the Axum client holds any size. The Cloudflare client writes the length of the body it read as the `Content-Length`, so its answer to a `HEAD` says `0`, where the Fastly and Spin adapters keep the origin's length. @@ -114,6 +114,7 @@ The tests above pass on each of the three adapters, and the existing tests of ea | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-17 | First draft. | | 2026-10-18 | Fixes from review round 1. The pull request this spec belongs to, Spin's refusal of a source's `host_header` and what Cloudflare does with one, the startup warning for a route that enables the Image Optimizer and why it is not a refusal, the `HEAD` a signed route sends and the SVG and debug requests that pass, the Axum client's refusal on `send_async`, the startup refusal of a source that cannot be built, the per-request failures, body limits and timeouts, where answers are held, the Edge Cookie an asset response neither writes nor expires, the tests of the cache rule, the error and the Edge Cookie, the open questions and the review log. Fixes from review round 2. The loggers the Cloudflare and Spin adapters now install and where their lines appear, how often each adapter logs the Image Optimizer warning, why such a route gets a warning where those adapters refuse an auction of more than one provider, the Spin `host_header` refusal as a breaking change in the summary, the problem and the rollout, the logged failures, the reader headers a route's origin receives, the rise in key-value traffic, the signed source whose `host_header` names the address's own host in other letter case, what the warning test reads, which Spin client code the native tests reach, the logger tests and what is not in this change. | +| 2026-10-26 | Held answers follow the asset cache as a module (`2026-10-15-asset-cache-design.md`). `handle_asset_proxy_request` offers a `GET` to the module `[asset-cache]` selects and keeps no copy itself, the 51Degrees module holds its copies through `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]`, and no stock build links that module or a loading cache module. | ## Review log From 4fd5a475b18be2684163eed46975f14a77026152 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:32 +0000 Subject: [PATCH 176/192] Two cache designs say their code is built The loading cache design describes the change to rows 24, 32 and 43 as built, with line numbers still those of the code before it, and the asset cache design's header says that row 34 now holds the module, where both said they were written ahead of the code. --- docs/superpowers/specs/2026-10-12-loading-cache-design.md | 3 ++- docs/superpowers/specs/2026-10-15-asset-cache-design.md | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index 4b5fe21492..a77a8b87da 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -9,7 +9,7 @@ > **Why this spec exists.** Each robots.txt module's answer is read from the key-value store by every request and asked of the module's service by every request that finds no fresh copy, so a burst of requests for one file asks the service once for each request, on every instance. Core has one cache, for page templates, and no interface through which a module can supply a cache. -Line numbers in this document are those of row 31 at `490a4665d`, the head of IABTechLab/trusted-server#1372, unless a row or a commit is named. Row 24's head, `e66b955a4`, is in row 31's history, so row 31 has every fix that row 24 has. The change to rows 24, 32 and 43 that takes the 51Degrees module out of the stock lists is written here ahead of its code, so where this document describes it, it names files and items, and the line numbers it gives are those of the code before the change. +Line numbers in this document are those of row 31 at `490a4665d`, the head of IABTechLab/trusted-server#1372, unless a row or a commit is named. Row 24's head, `e66b955a4`, is in row 31's history, so row 31 has every fix that row 24 has. This document describes a change to rows 24, 32 and 43 that takes the 51Degrees module out of the stock lists, so where it describes that change it names files and items, and the line numbers it gives are those of the code before the change. ## 1. Problem @@ -232,6 +232,7 @@ These are for the reviewer. This revision records what the code does, and what t | 2026-10-26 | No stock build links a loading cache module. In a stock build every feature with no cache selected loads through `NoLoadingCache`, the 51Degrees module is offered only to a deployment that links it, and after the fold of row 56 its `fiftyone-caching` dependency sits behind a Cargo feature that no stock adapter turns on. The header, the design, the adapters, the configuration, what changes for a deployment, what is not in this change, the checks, the tests, what no test covers and the open questions say so. The services a load is given are named as the code names them, `without_connection`, where this page said `without_reader`, the name an earlier commit on row 24 gave them. The pull requests are no longer drafts, and the header says so. | | 2026-10-26 | Fixes from review round 3. The change takes in row 32, whose probe test counts the 51Degrees module as linked, and each adapter's order test, and puts a guard beside every stock list, the `ts` tool's included, with a test that pins the tool's list. What a stock build costs is spelled out, being a call to the cloud service on each page the client hints middleware is placed for, a key-value write on each robots.txt request where a deployment adds a robots.txt module, and the README promise of the 51Degrees shared code. The other text that says a stock build holds answers is listed with the change, with a changelog entry for the refused `module = "51degrees"`. Line numbers are those of the current heads, row 31 at `490a4665d` and row 24 at `e66b955a4`, and the design says what the code there does, being the host in each held answer's key, a module's failure logged with each address cut to its scheme and host on row 31, and the adapter robots.txt tests of row 24. The searches give their output from row 44, the open question on a repeated changelog entry, which no row repeats now, is gone, one on falling back to `NoLoadingCache` without a word is added, open question 1 names row 55 and core's `loading-cache.example`, the CI results are linked at their commits, and the counts of the local checks are gone. | | 2026-10-26 | Fixes from review round 4. Row 32's first case links a second loading cache module, a test builder offering core's `MemoryLoadingCache`, so that the case still shows the table is read. The text the change rewrites adds the integration guide, the 51Degrees client's comments and the client hints middleware's tests. The 51Degrees shared code's loader is named as a caller that captures a request's own services, "stock build" is defined where it is first used, the source link names the commit that `4.5.8` was published from, the test of module names checks the constant whatever features are on, and the decision is placed after the Touchbase. | +| 2026-10-26 | The change to rows 24, 32 and 43 is described as built, so the line numbers paragraph no longer says this page is written ahead of its code, and its line numbers stay those of the code before the change. | ## Review log diff --git a/docs/superpowers/specs/2026-10-15-asset-cache-design.md b/docs/superpowers/specs/2026-10-15-asset-cache-design.md index 3d3ac6c26d..184bb8b9b1 100644 --- a/docs/superpowers/specs/2026-10-15-asset-cache-design.md +++ b/docs/superpowers/specs/2026-10-15-asset-cache-design.md @@ -1,6 +1,6 @@ # Design Spec: The publisher's assets held by a module, behind one interface in core -**Status:** Proposed, 2026-10-15. Written after the code, for pull request [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375), row 34 of the stack. Rewritten 2026-10-26 for the asset cache as a module, ahead of the code that row 34 changes to follow it. +**Status:** Proposed, 2026-10-15. Written after the code, for pull request [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375), row 34 of the stack. Rewritten 2026-10-26 for the asset cache as a module, which row 34 now holds. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-12-loading-cache-design.md`, `2026-10-15-module-cache-declarations-design.md`, `2026-10-14-source-modules-design.md`, `2026-10-14-no-modules-crate-design.md`, `2026-10-26-integrations-crate-design.md` (new, added to [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) with this rewrite), `2026-09-15-852-template-and-origin-caching-design.md`, `2026-04-28-multi-backend-asset-proxy-design.md`, `2026-05-19-asset-s3-auth-fastly-io-design.md`, `2026-09-08-1138-per-cookie-template-cache-policy-design.md` **Related PRs:** IABTechLab/trusted-server#1375, reviewed on [jwrosewell/trusted-server#29](https://github.com/jwrosewell/trusted-server/pull/29) (closed). [IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373) (row 32) lets a module declare the caches it loads through. [IABTechLab/trusted-server#894](https://github.com/IABTechLab/trusted-server/pull/894) is IAB Tech Lab's own draft design for response caching in core @@ -450,6 +450,7 @@ Row 34's `assets_turned_off_in_the_settings_are_fetched_for_every_request` and ` | 2026-10-26 | Fixes from review round 3. `origin_response_may_be_kept` stays in core and becomes public beside `origin_shared_ttl`, both returning the new `KeepRefusal`, and row 46's change to it is named beside rows 39 and 54. Core offers only a `GET`, takes the module only from the builder `[asset-cache]` selects, and gains a recording test module for what it reports. `AssetFile`, `AssetVia` and `OriginRequest` are set out with their visibility and test constructors, `path` is `via`, and `fingerprint()` goes, with the key's digest of the build and the settings, until a rewritten body is kept. The caches are `51degrees-assets` and `51degrees-assets-vary`, the variant digest's tag is `ts-asset-variant-v1`, and after row 56 the module sits behind `asset-cache-51degrees`. The helpers the module copies, the crate's workspace and alias entries, what `asks_for_origin` describes, the fourth change to a row 34 document, the changelog, the CI record, #894's keys, its costs of Fastly's Core Cache and its cookie stripping, and smaller corrections. | | 2026-10-26 | The account of the Touchbase of 2026-10-23 says what 51Degrees said there and what 51Degrees decided after it, and the integrations crate design is named by its file, `2026-10-26-integrations-crate-design.md`. | | 2026-10-26 | Fixes from review round 4. The pull requests and the issue are linked where first named, and the CI record is linked at its commit. "Stock build", Viceroy and the seam probe are defined where first used. Fastly's Core Cache API is named as #894 names it, the cache names follow the form row 32 asks for, and on Spin the entries that a new key or a lifetime leaves behind stay in the store. The decisions of 51Degrees are placed after the Touchbase. | +| 2026-10-26 | The header says that row 34 now holds the module, where it said the rewrite was ahead of the code. | ## Review log From f6f5247c707e90163254efa395a52fc1b6e2d00d Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:33 +0000 Subject: [PATCH 177/192] Count the designs naming a type and vendor path 43 of the other design documents name a path under crates/// at this revision, not 41, because the developer experience and module endpoints designs name such paths too. The design gives the search that produces the count. --- docs/superpowers/specs/2026-10-26-integrations-crate-design.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-10-26-integrations-crate-design.md b/docs/superpowers/specs/2026-10-26-integrations-crate-design.md index 1b8432ef40..86b3dd360c 100644 --- a/docs/superpowers/specs/2026-10-26-integrations-crate-design.md +++ b/docs/superpowers/specs/2026-10-26-integrations-crate-design.md @@ -256,7 +256,7 @@ The crates were made in eleven rows and are folded in one, row 56, above row 55. Row 43 renamed three of them, `ad-server/mock`, `ads-txt/sellers` and `testing/seam-probe`, to `iabtl`, and row 34 makes the asset cache module. Folding in each of those rows was set aside for four reasons. 1. **Each fold would travel through the rows above it.** A fold in row 2 changes every row above it that touches a moved file, and 31 of the 48 rows above row 7 change a file in a module crate's folder, so the moves would be merged up through as many as 52 rows, each merge meeting the next row's own changes to the moved files. One fold at the top moves each file once, from the place the rows below leave it. -2. **The rows below stay as reviewed.** 41 of the other design documents in IABTechLab/trusted-server#1342 name a path under `crates///`, as do each row's description and code. A fold in each row would change all of them in rows already reviewed. With one fold at the top, each row's design describes the tree that row has, and this design gives the map from the old paths to the new. +2. **The rows below stay as reviewed.** 43 of the other design documents in IABTechLab/trusted-server#1342 name a path under `crates///`, as do each row's description and code. The count is of the files that `git grep -l -E 'crates/(ad-server|ad-tag|ads-txt|analytics|asset-cache|auction|auction-protocol|audience|bot-protection|cmp|device|framework|geo|headers|identity|loading-cache|middleware|permission-signal|shared|tag|testing)/[a-z0-9][a-z0-9-]*' docs/superpowers/specs` lists at this revision of IABTechLab/trusted-server#1342, leaving out this document. A fold in each row would change all of them in rows already reviewed. With one fold at the top, each row's design describes the tree that row has, and this design gives the map from the old paths to the new. 3. **The fold changes how modules are built and not what they do, and one row shows it.** Its review asks one question, whether anything but the paths and the labels of section 4 changed, which this row answers by the same tests passing, the 14 bundles keeping their hashes and every module keeping its name. Row 56 is measured beside row 55, with the same tests, the hashes of the 14 bundles, every module name and the time per request, and the results go in the description of row 56's pull request. 4. **Tech Lab's own move fits in its place.** #1194 also moves the crate boundary in a milestone of its own that preserves behavior, ahead of any change to the configuration. A row at the top can be replaced by Tech Lab's move, or by a later form of this one, without a change to the rows below it. @@ -324,6 +324,7 @@ These are for the reviewer, and this design decides none of them. | 2026-10-26 | First draft. | | 2026-10-26 | Fixes from review round 1. tokio's `rt-multi-thread`, which the loading cache's tests ask for, goes in a table that only native builds read, with the threaded tests on the host. The paths that climb out of a crate's folder are listed with their fixes, and `run.cjs`, `crates/fastly.toml`, the seam probe's workflow step and the core documents that name a crate's path join section 3. The browser scripts' second reason says that the APS renderer and Prebid's code stay in `trusted-server-js`, where #1194 would move them. Row 56's own measurement goes in its pull request. The manifest's lint and doctest settings, the source labels written as text, the log labels on Fastly, the places the Rust names are used, the rule for `51degrees`, the probe's one name of another type, the Fastly SDK's host imports, the files with license text of their own, the count of rows a fold would travel and where vendor names live beside #1194 are stated. Open questions 4 and 5 are reworded and 7 and 8 added. The Touchbase of 2026-10-23 says what Tech Lab asked, and that 51Degrees decided, and the seam probe and Viceroy are defined where first used. | | 2026-10-26 | Fixes from review round 2. The split design's pull request names its author, Aram Grigoryan. The account of the Touchbase gives Tech Lab's request as the record has it, and places the decisions of 51Degrees after the Touchbase. "Stock build" is defined where first used, the split design's integrations crate is the one that puts the catalog together, and a feature gates neither a folder's name constants nor its source label. `build-fastly`, `check-fastly` and `test-fastly` name `asset-cache-51degrees` and none of them names `loading-cache-51degrees`, each `maintainers.toml` comment says "module" where the manifest says "crate", and the sample permission policy joins section 3. | +| 2026-10-26 | The count of the other design documents naming a path under `crates///` is 43, with the search that gives it. | ## Review log From 9037462d046af742b284deca94f0d232721d8f9b Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:34 +0000 Subject: [PATCH 178/192] Three designs take one more review round --- docs/superpowers/specs/2026-10-12-loading-cache-design.md | 1 + docs/superpowers/specs/2026-10-26-developer-experience-design.md | 1 + docs/superpowers/specs/2026-10-26-module-endpoints-design.md | 1 + 3 files changed, 3 insertions(+) diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index a77a8b87da..81f865e865 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -240,3 +240,4 @@ These are for the reviewer. This revision records what the code does, and what t 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/31-caching-release` at `4b6ef9c05`, with the design's code read on `split/24-loading-cache` at `844b01fe2`. Issues found, none blocking. This revision states how the two published crates differ, from a file by file comparison of the crates themselves, and confirms the check counts, the pull request states and the description of IABTechLab/trusted-server#1365 that this spec cites. 3. **Round 3.** A review in a fresh session, against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. Issues found, one of them blocking. The change that takes the 51Degrees module out of the stock lists named only rows 24 and 43, when row 32's probe test counts the module as linked and each adapter's order test names it, and the change's guard test read only Axum's list, when five stock lists, the `ts` tool's among them, could each link the module again. This revision takes row 32 into the change with those tests, puts a guard test beside every stock list, with a test that pins the tool's list, and spells out what a stock build then costs, being a call to the cloud service on each page the client hints middleware is placed for and a key-value write on each robots.txt request where a deployment adds a robots.txt module. It also lists the other text that says a stock build holds answers, asks under Open questions whether loading through `NoLoadingCache` with nothing logged should be logged or refused, and gives the design's line numbers at the current heads of rows 24 and 31. 4. **Round 4.** A review in a fresh session, checking the round 3 fixes against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. Issues found, one of them blocking. The design said that once the 51Degrees module leaves the stock lists, the first case of row 32's probe test still shows the table selecting the module it names, when that case's build would then link only the probe's cache, which a table naming no module takes anyway. This revision has that case link a second loading cache module, a test builder offering core's `MemoryLoadingCache`, so that the case still shows that the table is read, adds the integration guide, the 51Degrees client's comments and the client hints middleware's tests to the text the change rewrites, defines stock build where it is first used, and links the source of `fiftyone-caching` at the commit that `4.5.8` was published from. +5. **Round 5.** A review in a fresh session, checking the round 4 fixes against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. Issues found, none blocking. Row 32's probe test now links a second loading cache module in its first case, a test builder offering core's `MemoryLoadingCache`, so the case shows that the table is read, and the guard test sits beside every stock list, the `ts` tool's included, with the tool's list pinned in order. The search of section 9 prints nothing on rows 24, 31, 32 and 43 and six lines at the top of the stack, the manifests, the comment in core, the guides, the template, the changelog and the 51Degrees crates say that a stock build links no loading cache module, and the line numbers and tests the design names were read at the commits it cites. Three minor points stay, that the search is said to print six lines at row 55 as at `5be114773`, where it prints those six and the ten stock-list lines the change removes, that the text the change rewrites describes two sibling designs in the present tense when both now follow the change, and that the account of row 32's rewritten test gives three cases where the test keeps four, the case of two linked modules and no table staying with the test builder in place of the 51Degrees module. diff --git a/docs/superpowers/specs/2026-10-26-developer-experience-design.md b/docs/superpowers/specs/2026-10-26-developer-experience-design.md index 763843c4f9..a5e048b28a 100644 --- a/docs/superpowers/specs/2026-10-26-developer-experience-design.md +++ b/docs/superpowers/specs/2026-10-26-developer-experience-design.md @@ -193,3 +193,4 @@ The journeys raise seven questions that only Tech Lab can answer. 1. **Round 1.** A review in a fresh session, against `split/55-scripts-in-crates`, Tech Lab's main and IABTechLab/trusted-server#1435. Issues found, nine of them blocking. The nine were that the design said no release had been published, when Tech Lab had published two pre-releases, that an open point asked Tech Lab for what Tech Lab had already planned as AC-8 and AC-9, and wrongly tied `ts healthcheck` and `ts rollback` to one deployment route, that an operator of the gold WASM could not follow its staging steps, that on the manual route nothing linked the operator's stores, so every request would answer `500`, that its account of row 56 contradicted the integrations crate design, that it never mentioned Tech Lab's split integrations design in IABTechLab/trusted-server#1194, that a table of build and test costs had no record a reader could check, that the module author's journey skipped the issue that Tech Lab's contributor guide asks for first, and that the journey never ran the module. This revision corrects the design on all nine points. It describes the two pre-releases, asks when AC-8 ships, says there is no staging for the gold WASM until then and gives a first deployment through versioned physical stores and the store links the manual route needs, follows the integrations crate design for the module folders, compares the stack with IABTechLab/trusted-server#1194, leaves every cost to a fresh run, and opens the module author's journey with the issue and runs the module under Viceroy against a page. 2. **Round 2.** A review in a fresh session, against `split/55-scripts-in-crates`, of the round 1 fixes and the new text. Issues found, three of them blocking. The three were that the design gave a Cargo feature to every folder with a dependency of its own, where the integrations crate design gives one only for a dependency that not every build can or should carry, or for a module that no stock build links, that the local page check could not pass, because the copied template's origin is fictional and EdgeZero's environment overlay only overrides keys the file already holds, and that the design said Tech Lab agreed changes that 51Degrees decided after the Touchbase. This revision follows the integrations crate design's rule for features, has the local page check set the origin and the hosts in the file, and says in its table of changes which were agreed at the Touchbase and which 51Degrees decided after it. It also gives each startup status as the status of the error's kind, and fills the gaps an outside operator met in the operator journey, from creating the service and its secrets to the commands of the versioned store route. 3. **Round 3.** A review in a fresh session, of the round 2 fixes and the new text. Issues found, one of them blocking. The table of changes said that `/_ts/config` shows the policy's text once row 4 is merged into row 9, when the configuration view never shows it. This revision says that the view shows only the policy's name and SHA-256, has the local page check push with `--no-env`, so that the environment overlay cannot replace the settings in the file, no longer cites the getting-started guide, whose environment files still use the old name of the origin variable, says that no stock build turns on the loading cache or asset cache feature, and adds row 32 to the loading cache change. It also answers nine questions an outside operator would still ask, from the first release they can use to DNS and certificates, defines terms such as blob, SBOM and stock build, and says what IABTechLab/trusted-server#1055 covers. +4. **Round 4.** A review in a fresh session, against `split/55-scripts-in-crates`, Tech Lab's main, IABTechLab/trusted-server#1435 and EdgeZero `v0.0.10`, of the round 3 fixes and the whole text. Issues found, none blocking. The configuration view is now said to show the policy's name and SHA-256 and never its text, and `--no-env` was checked against row 4's push, which leaves the overlay out of its own load. The round also checked every message the journeys quote, in the stack's code and in EdgeZero's, the pins of main and of row 55, the Fastly guide's sections and commands, the contributor guide's rules on issues and drafts, the pull request template, main's list of builders, the quotes from IABTechLab/trusted-server#1194, IABTechLab/trusted-server#1262 and IABTechLab/trusted-server#1435 and the release workflows, the agreements and decisions in the table of changes, and the sections of the integrations crate, per-host selector and permission model designs that the journeys cite, the last against that design as revised for row 4. One minor point stays, that the operator journey gives the sample policy as 24,231 bytes, its size at row 55, where section 3.1 of the permission model design gives 24,271 bytes, its size at row 4. diff --git a/docs/superpowers/specs/2026-10-26-module-endpoints-design.md b/docs/superpowers/specs/2026-10-26-module-endpoints-design.md index c4764ef80b..ca0db7d8d1 100644 --- a/docs/superpowers/specs/2026-10-26-module-endpoints-design.md +++ b/docs/superpowers/specs/2026-10-26-module-endpoints-design.md @@ -95,3 +95,4 @@ Another method on a path of item 1 reaches the fallback as any other request doe 1. **Round 1.** A review in a fresh session, against `split/12-attestation`. Issues found, seven of them blocking. The seven were that the design said an adapter whose registry is refused serves its startup error, when on Axum, Cloudflare and Spin a deployment's own builders arrive through `routes_with_registrations`, which hands the error back to the deployment's code, that it said a module call carries the request's evidence, which only row 39 adds, that the steps before a module route left out device classification and did not say that the Edge Cookie step can answer in the route's place, that it said a bot filter can answer for a module's endpoint without saying that the stack's own DataDome filter lets every module route through, that the new guide section would contradict the two places in the guide that tie a module's routes to its prefix, that the section would go out of date as later rows change those steps, and that the planned `POST` check could not be built where the design put it. This revision says what each adapter does when two routes clash, what a module call carries at this row and from row 39, where each adapter classifies the device and when the Edge Cookie step answers, which requests a module route never sees, and that every request filter is told when a module route takes the request. The guide section now states only what holds at every row and amends those two places, and the seam probe gains a handler that counts its runs, so the `POST` check asserts that the handler never ran. 2. **Round 2.** A review in a fresh session, against `split/12-attestation`, of the round 1 fixes and the new text. Issues found, two of them blocking. The two were that the design said the stock `Hooks::routes()` takes no builders from outside core, when the stock list holds builders from crates outside core, such as Prebid and Didomi, and did not say what a clash does there, and that the guide section said two modules on one path are refused, when the registry keeps one router for each method. This revision says that the stock entry points compose the shipped modules' builders and answer every request with the startup error when the registry is refused, gives a clash that a stock build reaches from its settings alone, at `/prebid.js`, and says that two routes on one method and path are refused. It also lists every function through which a deployment's own builders arrive, says when each adapter builds its state, and says that nothing in core, the adapters or EdgeZero strips the body from a `HEAD` answer. +3. **Round 3.** A review in a fresh session, against `split/12-attestation`, of the round 2 fixes and the whole text. None blocking. The stock entry points are now said to compose the shipped modules' builders and to answer every request with the startup error when the registry is refused, with a clash at `/prebid.js` that the settings alone reach, and the guide section says that two routes on one method and path are refused. The round read the row's code for the probe's own path, its handler and its counter, the five checks on each of the four adapters, the development dependencies that Cloudflare and Spin gain, the crate's own doc, the guide section and the two places it amends, the registry's one router for each of seven methods and its refusal, the plan registration, the Fastly entry point's early answers, the closed path check, Didomi's prefix, the functions through which a deployment's own builders arrive at the top of the stack, the two tests section 4 names there and row 15's platform identity, and found each as the design states it. From 741f693cbde82219c29dbab60fe433748137a280 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:35 +0000 Subject: [PATCH 179/192] Four minor points from the confirmation round The developer experience design gives the sample policy as 24,271 bytes, its size at row 4. The loading cache design says that the section 9 search prints six lines at the head of row 55 and, at the published commit it takes line numbers from, those six and the ten stock-list lines the change removes, that the change rewrote the two sibling designs on this branch, and that row 32's rewritten probe test has four cases. The round 4 and round 5 entries of their review logs end with none blocking and no minor point. --- .../specs/2026-10-12-loading-cache-design.md | 10 +++++----- .../specs/2026-10-26-developer-experience-design.md | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index 81f865e865..33c8877334 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -41,8 +41,8 @@ store_idle_seconds = 2592000 - The section of the configuration guide on loading caches, which says that a stock build links `loading-cache.51degrees` and holds answers without a line in the document (`docs/guide/configuration.md` lines 1833 to 1837), and the commented example in `trusted-server.example.toml` (lines 458 and 459), whose `module = "51degrees"`, uncommented, a stock build refuses. - The loading cache entry of `CHANGELOG.md` (line 105), which says that the stock list ships the module, so every adapter takes it. - The description of IABTechLab/trusted-server#1365, which says in its paragraph on selecting a cache, and again under "What breaks", that a stock build holds answers with no line in the document. -- Row 32's design, `2026-10-15-module-cache-declarations-design.md`, whose sentence in section 2.2 saying that a stock build holds a module's answers is corrected with this revision, and whose lines 202, 226 and 235 describe row 32's code with the 51Degrees module in the stock list. -- Row 43's design, `2026-10-14-no-modules-crate-design.md`, whose list of tests (line 84) says that a feature selects the stock loading cache by name. +- Row 32's design, `2026-10-15-module-cache-declarations-design.md`. Its sentence in section 2.2 said that a stock build holds a module's answers, and its lines 202, 226 and 235 described row 32's code with the 51Degrees module in the stock list. The change rewrote all four, and row 32's copy of the design takes the same text. +- Row 43's design, `2026-10-14-no-modules-crate-design.md`, whose list of tests (line 84) said that a feature selects the stock loading cache by name. The change rewrote it, and row 43's copy of the design takes the same text. - The integration guide, `docs/guide/integration-guide.md`, whose lines 864 and 865 say "The stock list carries `crates/loading-cache/51degrees`" (lines 963 and 964 at row 55). - From row 44, the 51Degrees shared code's `crates/shared/51degrees/README.md` (lines 12 to 14), the module comment of its `src/engine_cache.rs` (lines 3 to 10), and its `src/client.rs`, in the module comment (lines 11 to 13) and on `CloudPipeline` (lines 102 and 103), which say that a question many requests ask at once is asked of the cloud service once. The 51Degrees guide, `docs/guide/integrations/51degrees.md` (lines 94 to 96), says that a stock build holds the service's answers. - From row 44, the tests of the client hints middleware, `crates/middleware/51degrees/src/lib.rs` at row 55. The helper `stock_builders()` (lines 1573 to 1575) is documented as "The builders a stock build links for the two 51Degrees modules", and the test `the_shared_table_runs_the_middleware_and_the_stock_cache_holds_its_answers` (lines 1613 to 1641) calls the module the stock loading cache in its name, its comment and its message (line 1638). The link at line 1575 stays, because only those tests use it, and the names and the wording change. @@ -160,7 +160,7 @@ The asset cache is a module that no stock build links (row 34, `2026-10-15-asset git grep -n -e trusted_server_loading_cache_51degrees -e trusted-server-loading-cache-51degrees 490a4665d -- crates ':!crates/loading-cache' ``` -Once the module leaves the stock lists, the same search with the head of row 24, 31, 32 or 43 in place of the commit prints nothing, so no stock list names the module crate either. From row 44 it prints six lines, each a link that only the tests of the 51Degrees modules use, as at row 55 (`5be114773`). +Once the module leaves the stock lists, the same search with the head of row 24, 31, 32 or 43 in place of the commit prints nothing, so no stock list names the module crate either. From row 44 it prints six lines, each a link that only the tests of the 51Degrees modules use, and at the head of row 55 after the change it prints those six, listed below with their line numbers at `5be114773`, where it prints those six and the ten stock-list lines the change removes, two for each adapter and two for the tool. - `crates/middleware/51degrees/Cargo.toml` line 31, a dev-dependency, and `crates/middleware/51degrees/src/lib.rs` line 1575, in the crate's tests. - `crates/shared/51degrees/Cargo.toml` line 19, the crate's `test-utils` feature, line 23, the optional dependency that the feature turns on, and line 39, a dev-dependency. @@ -195,7 +195,7 @@ git grep -n -e detachable -e 'cloudflare::with_context' -e WaitUntil -e local_sp **The tests the change touches.** Each reads a list of builders and none reads a build, so the `cargo tree` command above stays the check of what a build compiles. - **Row 24, `crates/trusted-server-modules/src/lib.rs`.** The stock list's order test, `stock_modules_are_offered_in_hook_order`, stops naming `loading-cache.51degrees`. The three tests that pin the stock cache, `a_feature_selects_the_stock_loading_cache_by_name`, `a_table_naming_no_module_holds_with_the_stock_loading_cache` and `deploy_validation_knows_the_stock_loading_cache`, give way to `the_stock_build_links_no_loading_cache`, which fails if a builder in the stock list offers a loading cache (`IntegrationBuilder::loading_cache` gives one). -- **Row 32, `crates/trusted-server-adapter-axum/tests/seam_probe.rs`.** `a_cache_the_probe_declares_is_selected_in_its_own_table` (lines 564 to 598 at `4a1f03c40`) counts the stock module as linked. Its first case shows that the table is read only because the build links a second loading cache module, so that a table that is not read would keep nothing. With the stock module gone, the build of that case links the probe's cache alone, which a table that names no module takes as well, so the case would pass whether or not `module` is read. The rewrite therefore links a second loading cache module in that case's build, a test builder offering core's `MemoryLoadingCache`, so that only a table that is read keeps values. In the other two cases the assertions (lines 587 to 591 and 593 to 597) flip, because the build that links both of the probe's builders links one loading cache module, so its case with no table keeps values, and the build that links only `seam_probe::builder()` links none, so that case keeps nothing. The test's comment (lines 564 to 569), which says that the build links the stock module and the probe's, is rewritten. +- **Row 32, `crates/trusted-server-adapter-axum/tests/seam_probe.rs`.** `a_cache_the_probe_declares_is_selected_in_its_own_table` (lines 564 to 598 at `4a1f03c40`) counts the stock module as linked. Its first case shows that the table is read only because the build links a second loading cache module, so that a table that is not read would keep nothing. With the stock module gone, the build of that case links the probe's cache alone, which a table that names no module takes as well, so the case would pass whether or not `module` is read. The rewrite therefore links a second loading cache module in the builds of the first two cases, a test builder offering core's `MemoryLoadingCache`, so that only a table that is read keeps values, and the case of two linked modules and no table (lines 587 to 591) keeps nothing as before, with that test builder in place of the stock module. The test has four cases where it had three. Its third case links both of the probe's builders and no table, which is now one loading cache module, so the cache keeps values, and its fourth links only `seam_probe::builder()`, as the old third case did (lines 593 to 597), which is now no loading cache module, so the cache keeps nothing where it kept values. The test's comment (lines 564 to 569), which says that the build links the stock module and the probe's, is rewritten. - **Row 43, each adapter's `src/app.rs`.** `the_linked_modules_are_offered_in_hook_order` stops naming `loading-cache.51degrees` (Axum line 1173, Cloudflare line 1462, Fastly line 3692 and Spin line 1911 at `008f79d86`, and lines 1779, 2044, 3928 and 2397 at row 55), and `the_stock_build_links_no_loading_cache` sits beside it, reading the adapter's `shipped_module_builders()`. The three tests that row 43 moved into `crates/trusted-server-adapter-axum/tests/shipped_modules.rs` give way to the Axum adapter's guard. - **Row 43, `crates/trusted-server-cli/src/app_config.rs`.** No test pins the tool's `linked_builders()` (lines 36 to 66 at `008f79d86`, and lines 36 to 73 at row 55), so nothing would fail if the tool linked the module again. The change adds `the_stock_build_links_no_loading_cache` beside it, and a test that pins its builders in order, as each adapter's order test pins its own list. @@ -240,4 +240,4 @@ These are for the reviewer. This revision records what the code does, and what t 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/31-caching-release` at `4b6ef9c05`, with the design's code read on `split/24-loading-cache` at `844b01fe2`. Issues found, none blocking. This revision states how the two published crates differ, from a file by file comparison of the crates themselves, and confirms the check counts, the pull request states and the description of IABTechLab/trusted-server#1365 that this spec cites. 3. **Round 3.** A review in a fresh session, against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. Issues found, one of them blocking. The change that takes the 51Degrees module out of the stock lists named only rows 24 and 43, when row 32's probe test counts the module as linked and each adapter's order test names it, and the change's guard test read only Axum's list, when five stock lists, the `ts` tool's among them, could each link the module again. This revision takes row 32 into the change with those tests, puts a guard test beside every stock list, with a test that pins the tool's list, and spells out what a stock build then costs, being a call to the cloud service on each page the client hints middleware is placed for and a key-value write on each robots.txt request where a deployment adds a robots.txt module. It also lists the other text that says a stock build holds answers, asks under Open questions whether loading through `NoLoadingCache` with nothing logged should be logged or refused, and gives the design's line numbers at the current heads of rows 24 and 31. 4. **Round 4.** A review in a fresh session, checking the round 3 fixes against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. Issues found, one of them blocking. The design said that once the 51Degrees module leaves the stock lists, the first case of row 32's probe test still shows the table selecting the module it names, when that case's build would then link only the probe's cache, which a table naming no module takes anyway. This revision has that case link a second loading cache module, a test builder offering core's `MemoryLoadingCache`, so that the case still shows that the table is read, adds the integration guide, the 51Degrees client's comments and the client hints middleware's tests to the text the change rewrites, defines stock build where it is first used, and links the source of `fiftyone-caching` at the commit that `4.5.8` was published from. -5. **Round 5.** A review in a fresh session, checking the round 4 fixes against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. Issues found, none blocking. Row 32's probe test now links a second loading cache module in its first case, a test builder offering core's `MemoryLoadingCache`, so the case shows that the table is read, and the guard test sits beside every stock list, the `ts` tool's included, with the tool's list pinned in order. The search of section 9 prints nothing on rows 24, 31, 32 and 43 and six lines at the top of the stack, the manifests, the comment in core, the guides, the template, the changelog and the 51Degrees crates say that a stock build links no loading cache module, and the line numbers and tests the design names were read at the commits it cites. Three minor points stay, that the search is said to print six lines at row 55 as at `5be114773`, where it prints those six and the ten stock-list lines the change removes, that the text the change rewrites describes two sibling designs in the present tense when both now follow the change, and that the account of row 32's rewritten test gives three cases where the test keeps four, the case of two linked modules and no table staying with the test builder in place of the 51Degrees module. +5. **Round 5.** A review in a fresh session, checking the round 4 fixes against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. None blocking. Row 32's probe test now links a second loading cache module in its first case, a test builder offering core's `MemoryLoadingCache`, so the case shows that the table is read, and the guard test sits beside every stock list, the `ts` tool's included, with the tool's list pinned in order. The search of section 9 prints nothing on rows 24, 31, 32 and 43 and six lines at the top of the stack, the manifests, the comment in core, the guides, the template, the changelog and the 51Degrees crates say that a stock build links no loading cache module, and the line numbers and tests the design names were read at the commits it cites. diff --git a/docs/superpowers/specs/2026-10-26-developer-experience-design.md b/docs/superpowers/specs/2026-10-26-developer-experience-design.md index a5e048b28a..56e4ce1a2e 100644 --- a/docs/superpowers/specs/2026-10-26-developer-experience-design.md +++ b/docs/superpowers/specs/2026-10-26-developer-experience-design.md @@ -118,7 +118,7 @@ The journey needs these. The version that was live still links the first store with the previous settings, so `ts rollback` to that version restores both the build and the settings. The same route serves an operator whose service runs a build of Tech Lab's main, because that build refuses the settings this update's build needs, and this update's build refuses main's settings until `ts config migrate` rewrites them and they name a policy. For an operator whose service runs nothing yet, steps 2 to 8 already follow the route, because the base version links the first config store and no earlier settings have to stay readable. -**On the other adapters.** IABTechLab/trusted-server#1435 builds three Axum binaries, named `trusted-server-axum-`. Axum reads one settings blob, which `ts config push --adapter axum --local --manifest ./edgezero.toml --app-config ./trusted-server.toml --permissions eu.yaml --no-diff --yes` writes to `.edgezero/local-config-trusted_server_config.json` for the operator to export as `TRUSTED_SERVER_CONFIG_TRUSTED_SERVER_CONFIG_TRUSTED_SERVER_CONFIG`, with each secret as `TRUSTED_SERVER_SECRET_TRUSTED_SERVER_SECRETS_`. From row 4 that blob carries the policy's name and SHA-256 without the file, and Axum reads the file when it starts from the folder that `TRUSTED_SERVER_PERMISSIONS_DIR` names, as `/.yaml`, so the folder must hold the bytes that were pushed. Cloudflare and Spin also read one blob each, and from row 4 the policy travels inside that blob, because only the Fastly adapter chooses a blob by host. Spin reads its blob from the `default` key-value store. From row 4 Cloudflare reads its blob from the Workers KV namespace that `ts config push` writes, in place of the Worker variable `TRUSTED_SERVER_CONFIG`, which, as section 3.1 of `2026-07-30-permission-model-design.md` sets out, cannot hold the 24,231-byte sample, and a Worker without that binding, or with no blob in the namespace, answers `500`. IABTechLab/trusted-server#1435 says that "Cloudflare, Spin, Windows and Linux ARM64 CLI releases follow separately", so an operator of Cloudflare or Spin builds from source today, and that build is not the gold WASM. Staging, `ts healthcheck`, `ts active-version` and `ts rollback` are Fastly's alone. +**On the other adapters.** IABTechLab/trusted-server#1435 builds three Axum binaries, named `trusted-server-axum-`. Axum reads one settings blob, which `ts config push --adapter axum --local --manifest ./edgezero.toml --app-config ./trusted-server.toml --permissions eu.yaml --no-diff --yes` writes to `.edgezero/local-config-trusted_server_config.json` for the operator to export as `TRUSTED_SERVER_CONFIG_TRUSTED_SERVER_CONFIG_TRUSTED_SERVER_CONFIG`, with each secret as `TRUSTED_SERVER_SECRET_TRUSTED_SERVER_SECRETS_`. From row 4 that blob carries the policy's name and SHA-256 without the file, and Axum reads the file when it starts from the folder that `TRUSTED_SERVER_PERMISSIONS_DIR` names, as `/.yaml`, so the folder must hold the bytes that were pushed. Cloudflare and Spin also read one blob each, and from row 4 the policy travels inside that blob, because only the Fastly adapter chooses a blob by host. Spin reads its blob from the `default` key-value store. From row 4 Cloudflare reads its blob from the Workers KV namespace that `ts config push` writes, in place of the Worker variable `TRUSTED_SERVER_CONFIG`, which, as section 3.1 of `2026-07-30-permission-model-design.md` sets out, cannot hold the 24,271-byte sample, and a Worker without that binding, or with no blob in the namespace, answers `500`. IABTechLab/trusted-server#1435 says that "Cloudflare, Spin, Windows and Linux ARM64 CLI releases follow separately", so an operator of Cloudflare or Spin builds from source today, and that build is not the gold WASM. Staging, `ts healthcheck`, `ts active-version` and `ts rollback` are Fastly's alone. ## 4. Journey 3. The release manager builds and signs the gold WASM @@ -193,4 +193,4 @@ The journeys raise seven questions that only Tech Lab can answer. 1. **Round 1.** A review in a fresh session, against `split/55-scripts-in-crates`, Tech Lab's main and IABTechLab/trusted-server#1435. Issues found, nine of them blocking. The nine were that the design said no release had been published, when Tech Lab had published two pre-releases, that an open point asked Tech Lab for what Tech Lab had already planned as AC-8 and AC-9, and wrongly tied `ts healthcheck` and `ts rollback` to one deployment route, that an operator of the gold WASM could not follow its staging steps, that on the manual route nothing linked the operator's stores, so every request would answer `500`, that its account of row 56 contradicted the integrations crate design, that it never mentioned Tech Lab's split integrations design in IABTechLab/trusted-server#1194, that a table of build and test costs had no record a reader could check, that the module author's journey skipped the issue that Tech Lab's contributor guide asks for first, and that the journey never ran the module. This revision corrects the design on all nine points. It describes the two pre-releases, asks when AC-8 ships, says there is no staging for the gold WASM until then and gives a first deployment through versioned physical stores and the store links the manual route needs, follows the integrations crate design for the module folders, compares the stack with IABTechLab/trusted-server#1194, leaves every cost to a fresh run, and opens the module author's journey with the issue and runs the module under Viceroy against a page. 2. **Round 2.** A review in a fresh session, against `split/55-scripts-in-crates`, of the round 1 fixes and the new text. Issues found, three of them blocking. The three were that the design gave a Cargo feature to every folder with a dependency of its own, where the integrations crate design gives one only for a dependency that not every build can or should carry, or for a module that no stock build links, that the local page check could not pass, because the copied template's origin is fictional and EdgeZero's environment overlay only overrides keys the file already holds, and that the design said Tech Lab agreed changes that 51Degrees decided after the Touchbase. This revision follows the integrations crate design's rule for features, has the local page check set the origin and the hosts in the file, and says in its table of changes which were agreed at the Touchbase and which 51Degrees decided after it. It also gives each startup status as the status of the error's kind, and fills the gaps an outside operator met in the operator journey, from creating the service and its secrets to the commands of the versioned store route. 3. **Round 3.** A review in a fresh session, of the round 2 fixes and the new text. Issues found, one of them blocking. The table of changes said that `/_ts/config` shows the policy's text once row 4 is merged into row 9, when the configuration view never shows it. This revision says that the view shows only the policy's name and SHA-256, has the local page check push with `--no-env`, so that the environment overlay cannot replace the settings in the file, no longer cites the getting-started guide, whose environment files still use the old name of the origin variable, says that no stock build turns on the loading cache or asset cache feature, and adds row 32 to the loading cache change. It also answers nine questions an outside operator would still ask, from the first release they can use to DNS and certificates, defines terms such as blob, SBOM and stock build, and says what IABTechLab/trusted-server#1055 covers. -4. **Round 4.** A review in a fresh session, against `split/55-scripts-in-crates`, Tech Lab's main, IABTechLab/trusted-server#1435 and EdgeZero `v0.0.10`, of the round 3 fixes and the whole text. Issues found, none blocking. The configuration view is now said to show the policy's name and SHA-256 and never its text, and `--no-env` was checked against row 4's push, which leaves the overlay out of its own load. The round also checked every message the journeys quote, in the stack's code and in EdgeZero's, the pins of main and of row 55, the Fastly guide's sections and commands, the contributor guide's rules on issues and drafts, the pull request template, main's list of builders, the quotes from IABTechLab/trusted-server#1194, IABTechLab/trusted-server#1262 and IABTechLab/trusted-server#1435 and the release workflows, the agreements and decisions in the table of changes, and the sections of the integrations crate, per-host selector and permission model designs that the journeys cite, the last against that design as revised for row 4. One minor point stays, that the operator journey gives the sample policy as 24,231 bytes, its size at row 55, where section 3.1 of the permission model design gives 24,271 bytes, its size at row 4. +4. **Round 4.** A review in a fresh session, against `split/55-scripts-in-crates`, Tech Lab's main, IABTechLab/trusted-server#1435 and EdgeZero `v0.0.10`, of the round 3 fixes and the whole text. None blocking. The configuration view is now said to show the policy's name and SHA-256 and never its text, and `--no-env` was checked against row 4's push, which leaves the overlay out of its own load. The round also checked every message the journeys quote, in the stack's code and in EdgeZero's, the pins of main and of row 55, the Fastly guide's sections and commands, the contributor guide's rules on issues and drafts, the pull request template, main's list of builders, the quotes from IABTechLab/trusted-server#1194, IABTechLab/trusted-server#1262 and IABTechLab/trusted-server#1435 and the release workflows, the agreements and decisions in the table of changes, and the sections of the integrations crate, per-host selector and permission model designs that the journeys cite, the last against that design as revised for row 4. From 65145efe04375504322197ccb92ac7d5bbf76130 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:36 +0000 Subject: [PATCH 180/192] The scripts design's review log cites no unpublished commit The round 1 entry no longer names the first commit of fixpass11/55-scripts-in-crates, which was rewritten before it was pushed and is on no branch. --- .../specs/2026-10-19-module-scripts-in-crates-design.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md b/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md index c9bbd2d270..5fe1e9e716 100644 --- a/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md +++ b/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md @@ -115,5 +115,5 @@ The `tsjs` guide said there are 12 integration source modules and 13 bundles. Th ## Review log -1. **Round 1, 2026-10-19.** A review in a fresh session, against `fixpass11/55-scripts-in-crates` as first committed at `7dc1671fe`, rewritten before it was pushed. Issues found, two of them blocking. TypeScript had no mapping for `@tsjs/module/*`, so the GPT script was no longer type-checked against the APS renderer while this design said TypeScript used the names, and the configuration guide still linked the old Tinybird README. Of the others, editors found no settings for crate scripts, a script in a crate cannot import an npm package and the design did not say so, `build.rs` did not watch `build-module-options.mjs`, the crate lint run passed an empty match, the JavaScript and Rust discovery differed on broken links, an `index.ts` folder and a module named `core`, crate `.mjs` tests were neither run nor given Node's globals, and there was no changelog entry. This revision fixes each of them. +1. **Round 1, 2026-10-19.** A review in a fresh session, against `fixpass11/55-scripts-in-crates` as first committed, before it was rewritten and pushed. Issues found, two of them blocking. TypeScript had no mapping for `@tsjs/module/*`, so the GPT script was no longer type-checked against the APS renderer while this design said TypeScript used the names, and the configuration guide still linked the old Tinybird README. Of the others, editors found no settings for crate scripts, a script in a crate cannot import an npm package and the design did not say so, `build.rs` did not watch `build-module-options.mjs`, the crate lint run passed an empty match, the JavaScript and Rust discovery differed on broken links, an `index.ts` folder and a module named `core`, crate `.mjs` tests were neither run nor given Node's globals, and there was no changelog entry. This revision fixes each of them. 2. **Round 2, 2026-10-19.** A lighter review in a fresh session, against this page at `b89557183`, of the round 1 fixes and the new text. No blocking issues. One round 1 item was only partly fixed, the claim that every import of one module by another uses `@tsjs/module/`, which the Prebid shim's relative import of the APS renderer contradicted. It also found that `crates/tsconfig.json` left out the shared `global.d.ts`, that ESLint 9 finds its config from the folder it runs in rather than from the file, that the build reads `crates/tsconfig.json` without `build.rs` watching it, that Rust discovery failed on a module root that is a file where JavaScript passed over it, that the new configuration files and `.mjs` test were outside the format check, and that a Tinybird change now starts the documentation proposal workflow. This revision fixes each of them, and the design joins the specs pull request when the row is pushed. From b4699bbf482d9e025a308af19891a1159c6f76bf Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:37 +0000 Subject: [PATCH 181/192] The designs' headers call no ready pull request a draft Forty-one Status and header lines in 38 of our designs called a pull request a draft when it is ready for review. Each now says pull request. Lines naming a pull request that is still a draft keep the word, being the fork's jwrosewell/trusted-server#36 and #43, row 55 (#1423), and Tech Lab's #1194 and #894. Tech Lab's own documents are not touched. --- .../specs/2026-07-30-client-cycle-ec-resolve-design.md | 6 +++--- .../specs/2026-07-30-pluggable-providers-design.md | 2 +- .../specs/2026-08-27-integration-provider-seam-design.md | 2 +- docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md | 2 +- .../superpowers/specs/2026-10-12-analytics-module-design.md | 2 +- .../specs/2026-10-12-consent-prompt-phases-design.md | 2 +- docs/superpowers/specs/2026-10-12-cookie-constant-design.md | 2 +- docs/superpowers/specs/2026-10-12-device-signals-design.md | 2 +- .../specs/2026-10-12-google-tag-module-design.md | 4 ++-- .../specs/2026-10-12-inmobi-choice-module-design.md | 2 +- .../2026-10-12-middleware-creates-a-missing-file-design.md | 2 +- .../specs/2026-10-12-middleware-media-types-design.md | 2 +- .../specs/2026-10-12-middleware-prepare-design.md | 2 +- docs/superpowers/specs/2026-10-12-module-context-design.md | 2 +- docs/superpowers/specs/2026-10-12-page-middleware-design.md | 2 +- docs/superpowers/specs/2026-10-12-prefix-module-design.md | 2 +- .../specs/2026-10-12-robots-txt-from-modules-design.md | 2 +- .../specs/2026-10-12-robots-txt-middleware-design.md | 2 +- .../specs/2026-10-12-shared-vendor-tables-design.md | 2 +- docs/superpowers/specs/2026-10-12-viewport-sizes-design.md | 2 +- .../specs/2026-10-13-middleware-settings-design.md | 2 +- docs/superpowers/specs/2026-10-13-origin-redirect-design.md | 2 +- .../2026-10-13-publisher-object-storage-origin-design.md | 2 +- .../specs/2026-10-13-request-addressing-design.md | 2 +- .../specs/2026-10-14-51degrees-client-hints-design.md | 2 +- .../superpowers/specs/2026-10-14-auction-protocol-design.md | 2 +- docs/superpowers/specs/2026-10-14-backend-naming-design.md | 2 +- .../superpowers/specs/2026-10-14-consent-decoders-design.md | 2 +- .../specs/2026-10-14-identifier-modules-design.md | 4 ++-- .../specs/2026-10-14-last-names-in-core-design.md | 2 +- .../superpowers/specs/2026-10-14-no-modules-crate-design.md | 2 +- docs/superpowers/specs/2026-10-14-origin-signing-design.md | 2 +- .../specs/2026-10-14-rsc-flight-middleware-design.md | 2 +- docs/superpowers/specs/2026-10-14-source-modules-design.md | 2 +- docs/superpowers/specs/2026-10-15-cookies-by-path-design.md | 2 +- .../specs/2026-10-15-geo-module-request-design.md | 2 +- .../specs/2026-10-15-module-cache-declarations-design.md | 2 +- .../2026-10-17-asset-routes-on-every-adapter-design.md | 2 +- 38 files changed, 42 insertions(+), 42 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md index 7fd9e37d46..ef6454bde2 100644 --- a/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md +++ b/docs/superpowers/specs/2026-07-30-client-cycle-ec-resolve-design.md @@ -1,7 +1,7 @@ # Design Spec: Client-Cycle Edge Cookie Modules and the Resolve Endpoint **Status:** Proposed. The implementation is the series branch -`split/4-client-resolve`, carried by draft pull request +`split/4-client-resolve`, carried by pull request IABTechLab/trusted-server#1346, and is not merged to main. The anti-replay reservation of §3.9 and the vendor envelope verification it serves are deliberately not in v1. They land with the first real vendor @@ -12,8 +12,8 @@ normative rather than deferred. **Issue references:** #778 (series), successor spec of the 2026-07-31 draft **Related specs:** `2026-07-30-pluggable-providers-design.md`, `2026-07-30-permission-model-design.md` -**Related PRs:** IABTechLab/trusted-server#1346 (Tech Lab draft) carries the -implementation. IABTechLab/trusted-server#1345 (Tech Lab draft) carries the +**Related PRs:** IABTechLab/trusted-server#1346 (Tech Lab pull request) carries the +implementation. IABTechLab/trusted-server#1345 (Tech Lab pull request) carries the permission model that the implementation builds on. IABTechLab/trusted-server#1342 holds every design document of the stack. **Last updated:** 2026-10-16 diff --git a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md index d7fb92fe7f..baedcb4505 100644 --- a/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md +++ b/docs/superpowers/specs/2026-07-30-pluggable-providers-design.md @@ -1,6 +1,6 @@ # Design Spec: Pluggable Edge Cookie, Device, and Geo Modules -**Status:** Proposed. Raised as draft pull requests +**Status:** Proposed. Raised as pull requests IABTechLab/trusted-server#1343 (row 2 of the stack, the Edge Cookie module seam) and IABTechLab/trusted-server#1344 (row 3, device and geo selection), neither of which is merged to main. Both were reviewed earlier, as pull diff --git a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md index 58ad6736b7..48a1f930bb 100644 --- a/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md +++ b/docs/superpowers/specs/2026-08-27-integration-provider-seam-design.md @@ -1,6 +1,6 @@ # Design Spec: The Integration Seam -**Status:** Proposed, 2026-08-27, revised 2026-08-28, 2026-10-06, 2026-10-07, 2026-10-08, 2026-10-12, 2026-10-15 and 2026-10-16. Draft +**Status:** Proposed, 2026-08-27, revised 2026-08-28, 2026-10-06, 2026-10-07, 2026-10-08, 2026-10-12, 2026-10-15 and 2026-10-16. PR #1342, row 1 of the stack in #777, carries this spec with every design document of the stack and the URL host linter's allowlist for the modules the stack ships, and targets `main` directly, so the whole normative set is diff --git a/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md b/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md index 579f5cd2c2..7986e63f64 100644 --- a/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md +++ b/docs/superpowers/specs/2026-10-12-ads-txt-sellers-design.md @@ -1,6 +1,6 @@ # Design Spec: The sellers a deployment adds to the publisher's ads.txt -**Status:** Proposed, 2026-10-12. Raised as draft pull request IABTechLab/trusted-server#1361, row 20 of the stack in IABTechLab/trusted-server#777, answering IABTechLab/trusted-server#1278, for Tech Lab review. The two earlier pull requests for this change, IABTechLab/trusted-server#1307 and jwrosewell/trusted-server#15, were closed on 2026-10-15 without being merged. +**Status:** Proposed, 2026-10-12. Raised as pull request IABTechLab/trusted-server#1361, row 20 of the stack in IABTechLab/trusted-server#777, answering IABTechLab/trusted-server#1278, for Tech Lab review. The two earlier pull requests for this change, IABTechLab/trusted-server#1307 and jwrosewell/trusted-server#15, were closed on 2026-10-15 without being merged. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md`, `2026-10-12-middleware-media-types-design.md`, `2026-10-12-middleware-creates-a-missing-file-design.md` and `2026-10-14-no-modules-crate-design.md`, all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** IABTechLab/trusted-server#1361, IABTechLab/trusted-server#1307 (closed without being merged), jwrosewell/trusted-server#15 (closed without being merged), IABTechLab/trusted-server#1360 (row 19, middleware on a media type other than HTML), IABTechLab/trusted-server#1363 (row 22, a file the origin does not have), IABTechLab/trusted-server#1384 (row 43, which renames the project's own modules `iabtl`) and IABTechLab/trusted-server#1348 (row 7, the integration seam, which replaces the closed IABTechLab/trusted-server#1094) diff --git a/docs/superpowers/specs/2026-10-12-analytics-module-design.md b/docs/superpowers/specs/2026-10-12-analytics-module-design.md index da7fd6f373..bcf94094e8 100644 --- a/docs/superpowers/specs/2026-10-12-analytics-module-design.md +++ b/docs/superpowers/specs/2026-10-12-analytics-module-design.md @@ -1,6 +1,6 @@ # Design Spec: Auction telemetry is a module [analytics] selects -**Status:** Proposed, 2026-10-12. Raised as draft pull request [IABTechLab/trusted-server#1357](https://github.com/IABTechLab/trusted-server/pull/1357), row 16 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), for Tech Lab review, answering [IABTechLab/trusted-server#1274](https://github.com/IABTechLab/trusted-server/issues/1274). The two earlier pull requests for the same commits, [jwrosewell/trusted-server#11](https://github.com/jwrosewell/trusted-server/pull/11) and [IABTechLab/trusted-server#1303](https://github.com/IABTechLab/trusted-server/pull/1303), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Raised as pull request [IABTechLab/trusted-server#1357](https://github.com/IABTechLab/trusted-server/pull/1357), row 16 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), for Tech Lab review, answering [IABTechLab/trusted-server#1274](https://github.com/IABTechLab/trusted-server/issues/1274). The two earlier pull requests for the same commits, [jwrosewell/trusted-server#11](https://github.com/jwrosewell/trusted-server/pull/11) and [IABTechLab/trusted-server#1303](https://github.com/IABTechLab/trusted-server/pull/1303), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-diagnostic-headers-design.md` (row 15, which this row stacks on) and `2026-10-12-administration-off-the-publisher-domain-design.md` (row 13), all in IABTechLab/trusted-server#1342, and `2026-06-22-auction-prebid-metrics-tinybird-grafana-design.md`, which IAB Tech Lab's main already holds and whose configuration table this spec supersedes **Related PRs:** [IABTechLab/trusted-server#1357](https://github.com/IABTechLab/trusted-server/pull/1357), [IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356) (row 15, headers that describe the deployment, which this row stacks on), [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354) (row 13, administration leaves the publisher's domain), [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7, the integration seam), [jwrosewell/trusted-server#11](https://github.com/jwrosewell/trusted-server/pull/11) (closed), [IABTechLab/trusted-server#1303](https://github.com/IABTechLab/trusted-server/pull/1303) (closed) diff --git a/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md b/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md index c90145e85a..32d2e26d37 100644 --- a/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md +++ b/docs/superpowers/specs/2026-10-12-consent-prompt-phases-design.md @@ -1,6 +1,6 @@ # Design Spec: A consent prompt runs in the phase the publisher chooses -**Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1362](https://github.com/IABTechLab/trusted-server/pull/1362), row 21 of the stack, answering [IABTechLab/trusted-server#1279](https://github.com/IABTechLab/trusted-server/issues/1279). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Implemented as the pull request [IABTechLab/trusted-server#1362](https://github.com/IABTechLab/trusted-server/pull/1362), row 21 of the stack, answering [IABTechLab/trusted-server#1279](https://github.com/IABTechLab/trusted-server/issues/1279). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md` (the two phases and where a serve middleware's head markup goes), `2026-10-12-inmobi-choice-module-design.md` (amended in section 3) **Related PRs:** [IABTechLab/trusted-server#1362](https://github.com/IABTechLab/trusted-server/pull/1362) builds on the sellers module in [IABTechLab/trusted-server#1361](https://github.com/IABTechLab/trusted-server/pull/1361) (row 20), replaces [IABTechLab/trusted-server#1308](https://github.com/IABTechLab/trusted-server/pull/1308) (closed, an earlier pull request for this change against `main`) and was reviewed on [jwrosewell/trusted-server#16](https://github.com/jwrosewell/trusted-server/pull/16) (closed). The InMobi module is in [IABTechLab/trusted-server#1359](https://github.com/IABTechLab/trusted-server/pull/1359) (row 18), page changes as middleware are in [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10), and the change that removes the stock list of modules is [IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384) (row 43) diff --git a/docs/superpowers/specs/2026-10-12-cookie-constant-design.md b/docs/superpowers/specs/2026-10-12-cookie-constant-design.md index 01d1896197..b9f31fafed 100644 --- a/docs/superpowers/specs/2026-10-12-cookie-constant-design.md +++ b/docs/superpowers/specs/2026-10-12-cookie-constant-design.md @@ -1,6 +1,6 @@ # Design Spec: The Model Terms cookie name is declared by the module that reads it -**Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1358](https://github.com/IABTechLab/trusted-server/pull/1358), answering [IABTechLab/trusted-server#1275](https://github.com/IABTechLab/trusted-server/issues/1275). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Implemented as the pull request [IABTechLab/trusted-server#1358](https://github.com/IABTechLab/trusted-server/pull/1358), answering [IABTechLab/trusted-server#1275](https://github.com/IABTechLab/trusted-server/issues/1275). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-07-30-permission-model-design.md` **Related PRs:** IABTechLab/trusted-server#1358, which is row 17 of the stack, builds on auction telemetry as a module in [IABTechLab/trusted-server#1357](https://github.com/IABTechLab/trusted-server/pull/1357) and was reviewed on [jwrosewell/trusted-server#12](https://github.com/jwrosewell/trusted-server/pull/12) (closed). [IABTechLab/trusted-server#1304](https://github.com/IABTechLab/trusted-server/pull/1304), the earlier pull request on this repository for the same commit, is closed without being merged, and the review continues on IABTechLab/trusted-server#1358. The permission model in [IABTechLab/trusted-server#1345](https://github.com/IABTechLab/trusted-server/pull/1345) added the constant this change moves, and the integration seam is in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) diff --git a/docs/superpowers/specs/2026-10-12-device-signals-design.md b/docs/superpowers/specs/2026-10-12-device-signals-design.md index b3aa77cc27..52f146ec05 100644 --- a/docs/superpowers/specs/2026-10-12-device-signals-design.md +++ b/docs/superpowers/specs/2026-10-12-device-signals-design.md @@ -1,6 +1,6 @@ # Design Spec: A device module may score a person and describe the device -**Status:** Approved on 2026-10-14, having been proposed on 2026-10-12. James Rogers approved the change at `b680d0780` on the fork's pull request [jwrosewell/trusted-server#23](https://github.com/jwrosewell/trusted-server/pull/23), which is now closed. The draft pull request [IABTechLab/trusted-server#1369](https://github.com/IABTechLab/trusted-server/pull/1369), row 28 of the stack, carries `b680d0780` with the row before it merged in, is open for Tech Lab review and answers [IABTechLab/trusted-server#1286](https://github.com/IABTechLab/trusted-server/issues/1286). The earlier pull request for the same commits, [IABTechLab/trusted-server#1316](https://github.com/IABTechLab/trusted-server/pull/1316), is closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Approved on 2026-10-14, having been proposed on 2026-10-12. James Rogers approved the change at `b680d0780` on the fork's pull request [jwrosewell/trusted-server#23](https://github.com/jwrosewell/trusted-server/pull/23), which is now closed. The pull request [IABTechLab/trusted-server#1369](https://github.com/IABTechLab/trusted-server/pull/1369), row 28 of the stack, carries `b680d0780` with the row before it merged in, is open for Tech Lab review and answers [IABTechLab/trusted-server#1286](https://github.com/IABTechLab/trusted-server/issues/1286). The earlier pull request for the same commits, [IABTechLab/trusted-server#1316](https://github.com/IABTechLab/trusted-server/pull/1316), is closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-07-30-pluggable-providers-design.md`, `2026-10-12-module-context-design.md` and `2026-07-30-permission-model-design.md` **Related PRs:** [IABTechLab/trusted-server#1369](https://github.com/IABTechLab/trusted-server/pull/1369) builds on the prefix module in [IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), and the next row is [IABTechLab/trusted-server#1370](https://github.com/IABTechLab/trusted-server/pull/1370). The module context that the change relies on is in [IABTechLab/trusted-server#1349](https://github.com/IABTechLab/trusted-server/pull/1349), and the permission model is in [IABTechLab/trusted-server#1345](https://github.com/IABTechLab/trusted-server/pull/1345). Three later rows change what this document describes, being row 43 in [IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384), which renames the seam probe crate to `iabtl`, row 45 in [IABTechLab/trusted-server#1388](https://github.com/IABTechLab/trusted-server/pull/1388), which moves the OpenRTB driver out of core, and row 49 in [IABTechLab/trusted-server#1399](https://github.com/IABTechLab/trusted-server/pull/1399), which renames the `fastly` value of `[device] module` to `platform` diff --git a/docs/superpowers/specs/2026-10-12-google-tag-module-design.md b/docs/superpowers/specs/2026-10-12-google-tag-module-design.md index e26811587f..038f57d00f 100644 --- a/docs/superpowers/specs/2026-10-12-google-tag-module-design.md +++ b/docs/superpowers/specs/2026-10-12-google-tag-module-design.md @@ -1,6 +1,6 @@ # Design Spec: A module writes Google's tags on the page from the settings -**Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1371](https://github.com/IABTechLab/trusted-server/pull/1371), row 30 of the stack, answering [IABTechLab/trusted-server#1288](https://github.com/IABTechLab/trusted-server/issues/1288). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Implemented as the pull request [IABTechLab/trusted-server#1371](https://github.com/IABTechLab/trusted-server/pull/1371), row 30 of the stack, answering [IABTechLab/trusted-server#1288](https://github.com/IABTechLab/trusted-server/issues/1288). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md` (the two phases and where a serve middleware's head markup goes), `2026-10-12-consent-prompt-phases-design.md` (a consent prompt may run in either phase) **Related PRs:** [IABTechLab/trusted-server#1371](https://github.com/IABTechLab/trusted-server/pull/1371) builds on sizes by viewport width in [IABTechLab/trusted-server#1370](https://github.com/IABTechLab/trusted-server/pull/1370) (row 29), replaces [IABTechLab/trusted-server#1318](https://github.com/IABTechLab/trusted-server/pull/1318) (closed, an earlier pull request for this change against `main`) and was reviewed on [jwrosewell/trusted-server#25](https://github.com/jwrosewell/trusted-server/pull/25) (closed). Modules register through the integration seam in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7), and page changes as middleware are in [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10) @@ -92,7 +92,7 @@ At least one of the three ids is required, because a table naming none asks the ## 4. Acceptance -CI passed on all 25 checks of the draft pull request at `1940d63f1`, and on all 19 checks of the fork's closed pull request at the same commit. The description of the pull request lists the checks that were run, says that Google's own handling of the snippets was not exercised, and leaves its manual test with `fastly compute serve` unticked. The code fixes of 2026-10-16 sit on top of that commit and have not yet been through CI. Locally the crate's 41 tests pass, clippy on the crate with warnings denied passes and `cargo fmt --all -- --check` passes, and each fix's new tests fail with that fix reverted. The module's 41 tests, one test in core, one in the seam probe and three in the ad tag module hold the claims in section 2 as follows. +CI passed on all 25 checks of the pull request at `1940d63f1`, and on all 19 checks of the fork's closed pull request at the same commit. The description of the pull request lists the checks that were run, says that Google's own handling of the snippets was not exercised, and leaves its manual test with `fastly compute serve` unticked. The code fixes of 2026-10-16 sit on top of that commit and have not yet been through CI. Locally the crate's 41 tests pass, clippy on the crate with warnings denied passes and `cargo fmt --all -- --check` passes, and each fix's new tests fail with that fix reverted. The module's 41 tests, one test in core, one in the seam probe and three in the ad tag module hold the claims in section 2 as follows. - **Each snippet is Google's text with only the id substituted.** For the tag snippet, `the_tag_snippet_is_googles_own_with_the_id_substituted`. For the AdSense loader, `the_adsense_loader_is_googles_own_with_the_account_substituted`. For the container script, `the_container_script_is_googles_own_with_the_id_substituted`. For the noscript iframe, `the_container_noscript_is_googles_own_with_the_id_substituted`. The consent snippet has no such test. - **The consent defaults.** `every_consent_type_defaults_to_denied`, `the_tag_waits_for_the_platform_s_update_before_it_fires` (500 and no region), `a_publisher_may_scope_the_defaults_to_regions_and_set_the_wait`, `a_publisher_may_grant_a_consent_type_up_front`, `a_region_that_is_not_a_code_is_refused_by_name` and `a_region_setting_leaves_every_other_visitor_with_all_four_denied`, which holds the second command with no region. diff --git a/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md b/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md index 508a8f4cc2..f01b5344d3 100644 --- a/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md +++ b/docs/superpowers/specs/2026-10-12-inmobi-choice-module-design.md @@ -1,6 +1,6 @@ # Design Spec: The InMobi Choice consent prompt as a module -**Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1359](https://github.com/IABTechLab/trusted-server/pull/1359), row 18 of the stack, answering [IABTechLab/trusted-server#1276](https://github.com/IABTechLab/trusted-server/issues/1276). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Implemented as the pull request [IABTechLab/trusted-server#1359](https://github.com/IABTechLab/trusted-server/pull/1359), row 18 of the stack, answering [IABTechLab/trusted-server#1276](https://github.com/IABTechLab/trusted-server/issues/1276). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md` (the entries and the two phases), `2026-10-12-consent-prompt-phases-design.md` (lets a consent prompt run in the serve phase) **Related PRs:** [IABTechLab/trusted-server#1359](https://github.com/IABTechLab/trusted-server/pull/1359) builds on the Model Terms cookie name in [IABTechLab/trusted-server#1358](https://github.com/IABTechLab/trusted-server/pull/1358) (row 17), replaces [IABTechLab/trusted-server#1305](https://github.com/IABTechLab/trusted-server/pull/1305) (closed, an earlier pull request for this change against `main`) and was reviewed on [jwrosewell/trusted-server#13](https://github.com/jwrosewell/trusted-server/pull/13) (closed). The change that lets a vendor module live outside core is [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7), page changes as middleware are in [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10), middleware on other media types is in [IABTechLab/trusted-server#1360](https://github.com/IABTechLab/trusted-server/pull/1360) (row 19) and a consent prompt in the serve phase is in [IABTechLab/trusted-server#1362](https://github.com/IABTechLab/trusted-server/pull/1362) (row 21) diff --git a/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md b/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md index 2064691a2d..2404e28d79 100644 --- a/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md +++ b/docs/superpowers/specs/2026-10-12-middleware-creates-a-missing-file-design.md @@ -1,6 +1,6 @@ # Design Spec: A file the origin does not have is made by the middleware named for its path -**Status:** Proposed, 2026-10-12. Raised as draft pull request [IABTechLab/trusted-server#1363](https://github.com/IABTechLab/trusted-server/pull/1363), row 22 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#17](https://github.com/jwrosewell/trusted-server/pull/17) and [IABTechLab/trusted-server#1309](https://github.com/IABTechLab/trusted-server/pull/1309), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Raised as pull request [IABTechLab/trusted-server#1363](https://github.com/IABTechLab/trusted-server/pull/1363), row 22 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#17](https://github.com/jwrosewell/trusted-server/pull/17) and [IABTechLab/trusted-server#1309](https://github.com/IABTechLab/trusted-server/pull/1309), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md`, `2026-10-12-middleware-media-types-design.md`, `2026-10-12-ads-txt-sellers-design.md` and `2026-10-12-robots-txt-middleware-design.md` (row 25, built after this row), all in IABTechLab/trusted-server#1342 **Related PRs:** IABTechLab/trusted-server#1363 builds on the consent prompt phases in [IABTechLab/trusted-server#1362](https://github.com/IABTechLab/trusted-server/pull/1362) (row 21). The `[[fetch]]` and `[[serve]]` entries come from [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10, page changes as middleware), the entries for a media type other than HTML from [IABTechLab/trusted-server#1360](https://github.com/IABTechLab/trusted-server/pull/1360) (row 19) and the sellers module from [IABTechLab/trusted-server#1361](https://github.com/IABTechLab/trusted-server/pull/1361) (row 20). [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366) (row 25) makes robots.txt a middleware and comes after [IABTechLab/trusted-server#1364](https://github.com/IABTechLab/trusted-server/pull/1364) (row 23) and [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24) diff --git a/docs/superpowers/specs/2026-10-12-middleware-media-types-design.md b/docs/superpowers/specs/2026-10-12-middleware-media-types-design.md index 0cbc94fbfd..f1ae42f6b6 100644 --- a/docs/superpowers/specs/2026-10-12-middleware-media-types-design.md +++ b/docs/superpowers/specs/2026-10-12-middleware-media-types-design.md @@ -1,6 +1,6 @@ # Design Spec: A middleware may run on a media type other than HTML -**Status:** Proposed, 2026-10-12. Raised as draft pull request IABTechLab/trusted-server#1360, row 19 of the stack in IABTechLab/trusted-server#777, for Tech Lab review. The two earlier pull requests for this change, IABTechLab/trusted-server#1306 and jwrosewell/trusted-server#14, are closed. +**Status:** Proposed, 2026-10-12. Raised as pull request IABTechLab/trusted-server#1360, row 19 of the stack in IABTechLab/trusted-server#777, for Tech Lab review. The two earlier pull requests for this change, IABTechLab/trusted-server#1306 and jwrosewell/trusted-server#14, are closed. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md`, `2026-10-12-ads-txt-sellers-design.md`, `2026-10-12-middleware-creates-a-missing-file-design.md`, `2026-10-12-robots-txt-middleware-design.md` and `2026-10-14-rsc-flight-middleware-design.md`, all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** IABTechLab/trusted-server#1360, IABTechLab/trusted-server#1306 (closed), jwrosewell/trusted-server#14 (closed) diff --git a/docs/superpowers/specs/2026-10-12-middleware-prepare-design.md b/docs/superpowers/specs/2026-10-12-middleware-prepare-design.md index 9144c5471a..0360ca4bb5 100644 --- a/docs/superpowers/specs/2026-10-12-middleware-prepare-design.md +++ b/docs/superpowers/specs/2026-10-12-middleware-prepare-design.md @@ -1,6 +1,6 @@ # Design Spec: A middleware may prepare, with the request's services, and may refuse -**Status:** Proposed, 2026-10-12. Raised as draft pull request [IABTechLab/trusted-server#1364](https://github.com/IABTechLab/trusted-server/pull/1364), row 23 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#18](https://github.com/jwrosewell/trusted-server/pull/18) and [IABTechLab/trusted-server#1311](https://github.com/IABTechLab/trusted-server/pull/1311), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Raised as pull request [IABTechLab/trusted-server#1364](https://github.com/IABTechLab/trusted-server/pull/1364), row 23 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#18](https://github.com/jwrosewell/trusted-server/pull/18) and [IABTechLab/trusted-server#1311](https://github.com/IABTechLab/trusted-server/pull/1311), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-module-context-design.md`, `2026-10-12-page-middleware-design.md`, `2026-10-12-middleware-media-types-design.md`, `2026-10-12-middleware-creates-a-missing-file-design.md` and `2026-10-12-robots-txt-middleware-design.md` (row 25, built after this row), all in IABTechLab/trusted-server#1342 **Related PRs:** IABTechLab/trusted-server#1364 builds on the file made from the middleware named for its path in [IABTechLab/trusted-server#1363](https://github.com/IABTechLab/trusted-server/pull/1363) (row 22). The module context a preparation is called with comes from [IABTechLab/trusted-server#1349](https://github.com/IABTechLab/trusted-server/pull/1349) (row 8) and the `[[fetch]]` and `[[serve]]` entries from [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10, page changes as middleware). The loading cache in [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24) comes next, and [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366) (row 25, robots.txt as a middleware) is the first middleware that prepares. diff --git a/docs/superpowers/specs/2026-10-12-module-context-design.md b/docs/superpowers/specs/2026-10-12-module-context-design.md index 68a3db999e..45f6d84983 100644 --- a/docs/superpowers/specs/2026-10-12-module-context-design.md +++ b/docs/superpowers/specs/2026-10-12-module-context-design.md @@ -1,6 +1,6 @@ # Design Spec: One module context -**Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1349](https://github.com/IABTechLab/trusted-server/pull/1349), answering [IABTechLab/trusted-server#1267](https://github.com/IABTechLab/trusted-server/issues/1267). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Implemented as the pull request [IABTechLab/trusted-server#1349](https://github.com/IABTechLab/trusted-server/pull/1349), answering [IABTechLab/trusted-server#1267](https://github.com/IABTechLab/trusted-server/issues/1267). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md` **Related PRs:** IABTechLab/trusted-server#1349, which builds on the integration seam in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) and was reviewed on [jwrosewell/trusted-server#3](https://github.com/jwrosewell/trusted-server/pull/3) (closed) diff --git a/docs/superpowers/specs/2026-10-12-page-middleware-design.md b/docs/superpowers/specs/2026-10-12-page-middleware-design.md index d0ce8873b5..59a81fe5b0 100644 --- a/docs/superpowers/specs/2026-10-12-page-middleware-design.md +++ b/docs/superpowers/specs/2026-10-12-page-middleware-design.md @@ -1,6 +1,6 @@ # Design Spec: Page changes as middleware -**Status:** Proposed, 2026-10-12. Raised as draft pull request [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351), row 10 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), answering [IABTechLab/trusted-server#1269](https://github.com/IABTechLab/trusted-server/issues/1269), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#5](https://github.com/jwrosewell/trusted-server/pull/5) and [IABTechLab/trusted-server#1295](https://github.com/IABTechLab/trusted-server/pull/1295), were closed on 2026-10-15 without being merged. So were the pull request that carried the design documents, [IABTechLab/trusted-server#1084](https://github.com/IABTechLab/trusted-server/pull/1084), and the integration seam pull request that this row builds on, [IABTechLab/trusted-server#1094](https://github.com/IABTechLab/trusted-server/pull/1094). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) now carries every design document of the stack in one place, and [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) is the integration seam pull request. +**Status:** Proposed, 2026-10-12. Raised as pull request [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351), row 10 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), answering [IABTechLab/trusted-server#1269](https://github.com/IABTechLab/trusted-server/issues/1269), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#5](https://github.com/jwrosewell/trusted-server/pull/5) and [IABTechLab/trusted-server#1295](https://github.com/IABTechLab/trusted-server/pull/1295), were closed on 2026-10-15 without being merged. So were the pull request that carried the design documents, [IABTechLab/trusted-server#1084](https://github.com/IABTechLab/trusted-server/pull/1084), and the integration seam pull request that this row builds on, [IABTechLab/trusted-server#1094](https://github.com/IABTechLab/trusted-server/pull/1094). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) now carries every design document of the stack in one place, and [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) is the integration seam pull request. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-middleware-media-types-design.md` (row 19), `2026-10-12-middleware-prepare-design.md` (row 23) and `2026-10-13-middleware-settings-design.md` (row 38), all in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342), which holds every design document of the stack **Related PRs:** [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351), which is row 10 of the stack, stacks on the inspection endpoints in [IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350) (row 9) and builds on the integration seam in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7). jwrosewell/trusted-server#5, IABTechLab/trusted-server#1295, IABTechLab/trusted-server#1094 and IABTechLab/trusted-server#1084 are closed. diff --git a/docs/superpowers/specs/2026-10-12-prefix-module-design.md b/docs/superpowers/specs/2026-10-12-prefix-module-design.md index 85edbdfec2..6c25f0db14 100644 --- a/docs/superpowers/specs/2026-10-12-prefix-module-design.md +++ b/docs/superpowers/specs/2026-10-12-prefix-module-design.md @@ -1,6 +1,6 @@ # Design Spec: A module names the browser's cookies and the shared store keys -**Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), row 27 of the stack, answering [IABTechLab/trusted-server#1285](https://github.com/IABTechLab/trusted-server/issues/1285). The two earlier pull requests for the same commits, [jwrosewell/trusted-server#22](https://github.com/jwrosewell/trusted-server/pull/22) and [IABTechLab/trusted-server#1315](https://github.com/IABTechLab/trusted-server/pull/1315), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Implemented as the pull request [IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), row 27 of the stack, answering [IABTechLab/trusted-server#1285](https://github.com/IABTechLab/trusted-server/issues/1285). The two earlier pull requests for the same commits, [jwrosewell/trusted-server#22](https://github.com/jwrosewell/trusted-server/pull/22) and [IABTechLab/trusted-server#1315](https://github.com/IABTechLab/trusted-server/pull/1315), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-loading-cache-design.md` (row 24), `2026-10-13-request-addressing-design.md` (row 35) and `2026-10-15-cookies-by-path-design.md` (row 50), all in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342), which holds every design document of the stack **Related PRs:** [IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), [IABTechLab/trusted-server#1367](https://github.com/IABTechLab/trusted-server/pull/1367) (row 26, a vendor's shared table, which this row stacks on), [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24, the loading cache, which adds the server key prefix), [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (the integration seam), [IABTechLab/trusted-server#1376](https://github.com/IABTechLab/trusted-server/pull/1376) (row 35, request addressing), [IABTechLab/trusted-server#1400](https://github.com/IABTechLab/trusted-server/pull/1400) (row 50, cookies by path), [IABTechLab/trusted-server#1382](https://github.com/IABTechLab/trusted-server/pull/1382) (row 41, the Prebid crate reads the cookie its script writes), [jwrosewell/trusted-server#22](https://github.com/jwrosewell/trusted-server/pull/22) (closed), [IABTechLab/trusted-server#1315](https://github.com/IABTechLab/trusted-server/pull/1315) (closed) diff --git a/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md b/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md index d80d0748bc..c34c866835 100644 --- a/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md +++ b/docs/superpowers/specs/2026-10-12-robots-txt-from-modules-design.md @@ -1,6 +1,6 @@ # Design Spec: robots.txt from modules -**Status:** Proposed, 2026-10-12. Raised as the draft pull request [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355), row 14 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), answering [IABTechLab/trusted-server#1272](https://github.com/IABTechLab/trusted-server/issues/1272), for Tech Lab review. The two earlier pull requests for the same commits, [IABTechLab/trusted-server#1301](https://github.com/IABTechLab/trusted-server/pull/1301) and [jwrosewell/trusted-server#9](https://github.com/jwrosewell/trusted-server/pull/9), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Raised as the pull request [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355), row 14 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), answering [IABTechLab/trusted-server#1272](https://github.com/IABTechLab/trusted-server/issues/1272), for Tech Lab review. The two earlier pull requests for the same commits, [IABTechLab/trusted-server#1301](https://github.com/IABTechLab/trusted-server/pull/1301) and [jwrosewell/trusted-server#9](https://github.com/jwrosewell/trusted-server/pull/9), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-module-context-design.md` (row 8), `2026-10-12-loading-cache-design.md` (row 24) and `2026-10-12-robots-txt-middleware-design.md` (row 25), all in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342), which holds every design document of the stack **Related PRs:** [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355), which is row 14 of the stack, stacks on the removal of administration from the publisher's domain in [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354) (row 13) and asks contributors through the module context of [IABTechLab/trusted-server#1349](https://github.com/IABTechLab/trusted-server/pull/1349) (row 8). The loading cache in [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24) changes how answers are held, and the middleware in [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366) (row 25) replaces the route. [IABTechLab/trusted-server#1301](https://github.com/IABTechLab/trusted-server/pull/1301) and [jwrosewell/trusted-server#9](https://github.com/jwrosewell/trusted-server/pull/9) are closed, and so are [IABTechLab/trusted-server#1094](https://github.com/IABTechLab/trusted-server/pull/1094) and [jwrosewell/trusted-server#3](https://github.com/jwrosewell/trusted-server/pull/3), which this spec named before and which the integration seam in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7) and the module context in IABTechLab/trusted-server#1349 replace diff --git a/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md b/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md index 5db9cb5c01..9475603708 100644 --- a/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md +++ b/docs/superpowers/specs/2026-10-12-robots-txt-middleware-design.md @@ -1,6 +1,6 @@ # Design Spec: robots.txt as a middleware -**Status:** Proposed, 2026-10-12. Raised as draft pull request [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366), row 25 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), answering [IABTechLab/trusted-server#1283](https://github.com/IABTechLab/trusted-server/issues/1283), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#20](https://github.com/jwrosewell/trusted-server/pull/20) and [IABTechLab/trusted-server#1313](https://github.com/IABTechLab/trusted-server/pull/1313), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Raised as pull request [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366), row 25 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), answering [IABTechLab/trusted-server#1283](https://github.com/IABTechLab/trusted-server/issues/1283), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#20](https://github.com/jwrosewell/trusted-server/pull/20) and [IABTechLab/trusted-server#1313](https://github.com/IABTechLab/trusted-server/pull/1313), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md` (row 10), `2026-10-12-robots-txt-from-modules-design.md` (row 14), `2026-10-12-middleware-media-types-design.md` (row 19), `2026-10-12-middleware-creates-a-missing-file-design.md` (row 22), `2026-10-12-middleware-prepare-design.md` (row 23), `2026-10-12-loading-cache-design.md` (row 24), `2026-10-12-shared-vendor-tables-design.md` (row 26, built after this row), `2026-10-12-prefix-module-design.md` (row 27, built after this row) and `2026-10-15-asset-cache-design.md` (row 34, built after this row), all in IABTechLab/trusted-server#1342 **Related PRs:** [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366) builds on the loading cache in [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24). The preparation step comes from [IABTechLab/trusted-server#1364](https://github.com/IABTechLab/trusted-server/pull/1364) (row 23), the rule that makes a file the origin does not have from [IABTechLab/trusted-server#1363](https://github.com/IABTechLab/trusted-server/pull/1363) (row 22), the entries for a media type other than HTML from [IABTechLab/trusted-server#1360](https://github.com/IABTechLab/trusted-server/pull/1360) (row 19), the `[[fetch]]` and `[[serve]]` entries from [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10) and the robots.txt contributors from [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355) (row 14). The publisher's assets held through the loading cache are in [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375) (row 34), which comes after this row. diff --git a/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md b/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md index 39f6d72d16..21fc49145a 100644 --- a/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md +++ b/docs/superpowers/specs/2026-10-12-shared-vendor-tables-design.md @@ -1,6 +1,6 @@ # Design Spec: A vendor's modules share one table -**Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1367](https://github.com/IABTechLab/trusted-server/pull/1367), row 26 of the stack, answering [IABTechLab/trusted-server#1284](https://github.com/IABTechLab/trusted-server/issues/1284). The two earlier pull requests for the same commits, [jwrosewell/trusted-server#21](https://github.com/jwrosewell/trusted-server/pull/21) and [IABTechLab/trusted-server#1314](https://github.com/IABTechLab/trusted-server/pull/1314), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Implemented as the pull request [IABTechLab/trusted-server#1367](https://github.com/IABTechLab/trusted-server/pull/1367), row 26 of the stack, answering [IABTechLab/trusted-server#1284](https://github.com/IABTechLab/trusted-server/issues/1284). The two earlier pull requests for the same commits, [jwrosewell/trusted-server#21](https://github.com/jwrosewell/trusted-server/pull/21) and [IABTechLab/trusted-server#1314](https://github.com/IABTechLab/trusted-server/pull/1314), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md` and `2026-10-14-51degrees-client-hints-design.md` (row 44, the first vendor that reads a shared table), both in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342), which holds every design document of the stack **Related PRs:** [IABTechLab/trusted-server#1367](https://github.com/IABTechLab/trusted-server/pull/1367), [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366) (row 25, robots.txt as a middleware, which this row stacks on), [IABTechLab/trusted-server#1386](https://github.com/IABTechLab/trusted-server/pull/1386) (row 44, 51Degrees client hints, the first vendor that reads a shared table), [jwrosewell/trusted-server#21](https://github.com/jwrosewell/trusted-server/pull/21) (closed), [IABTechLab/trusted-server#1314](https://github.com/IABTechLab/trusted-server/pull/1314) (closed) diff --git a/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md b/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md index 9308350cad..6294efe51f 100644 --- a/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md +++ b/docs/superpowers/specs/2026-10-12-viewport-sizes-design.md @@ -1,6 +1,6 @@ # Design Spec: A format can name the viewport widths at which the page asks for it -**Status:** Proposed, 2026-10-12. Implemented as the draft pull request [IABTechLab/trusted-server#1370](https://github.com/IABTechLab/trusted-server/pull/1370), row 29 of the stack, answering [IABTechLab/trusted-server#1287](https://github.com/IABTechLab/trusted-server/issues/1287). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Implemented as the pull request [IABTechLab/trusted-server#1370](https://github.com/IABTechLab/trusted-server/pull/1370), row 29 of the stack, answering [IABTechLab/trusted-server#1287](https://github.com/IABTechLab/trusted-server/issues/1287). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-07-24-prebid-refresh-gam-path-opt-out-design.md` (the wrapper on `refresh()` that runs the refresh auction, whose sizes this change takes from the band) and `2026-07-24-prevent-duplicate-gpt-slot-requests-design.md` (how the head bootstrap and the bundle define a slot on the publisher's div), both on `main` of IABTechLab/trusted-server **Related PRs:** [IABTechLab/trusted-server#1370](https://github.com/IABTechLab/trusted-server/pull/1370) builds on the device signals in [IABTechLab/trusted-server#1369](https://github.com/IABTechLab/trusted-server/pull/1369) (row 28), replaces [IABTechLab/trusted-server#1317](https://github.com/IABTechLab/trusted-server/pull/1317) (closed, an earlier pull request for this change against `main`) and was reviewed on [jwrosewell/trusted-server#24](https://github.com/jwrosewell/trusted-server/pull/24) (closed) diff --git a/docs/superpowers/specs/2026-10-13-middleware-settings-design.md b/docs/superpowers/specs/2026-10-13-middleware-settings-design.md index ee379acd50..c0995df105 100644 --- a/docs/superpowers/specs/2026-10-13-middleware-settings-design.md +++ b/docs/superpowers/specs/2026-10-13-middleware-settings-design.md @@ -1,6 +1,6 @@ # Design Spec: A middleware reads its settings from a table of its own -**Status:** Proposed, 2026-10-15. Raised as draft pull request IABTechLab/trusted-server#1379 (from jwrosewell/trusted-server#33), for Tech Lab review. +**Status:** Proposed, 2026-10-15. Raised as pull request IABTechLab/trusted-server#1379 (from jwrosewell/trusted-server#33), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md`, `2026-10-12-shared-vendor-tables-design.md`, `2026-10-14-51degrees-client-hints-design.md` **Related PRs:** jwrosewell/trusted-server#33, IABTechLab/trusted-server#1379, IABTechLab/trusted-server#1342 diff --git a/docs/superpowers/specs/2026-10-13-origin-redirect-design.md b/docs/superpowers/specs/2026-10-13-origin-redirect-design.md index 3c1262b2da..3b2bed2218 100644 --- a/docs/superpowers/specs/2026-10-13-origin-redirect-design.md +++ b/docs/superpowers/specs/2026-10-13-origin-redirect-design.md @@ -1,6 +1,6 @@ # Design Spec: A folder origin's addresses in response headers, and a headers step for serve middleware -**Status:** Proposed, 2026-10-13. Raised as draft pull request IABTechLab/trusted-server#1378 (from jwrosewell/trusted-server#32, which is closed), for Tech Lab review. +**Status:** Proposed, 2026-10-13. Raised as pull request IABTechLab/trusted-server#1378 (from jwrosewell/trusted-server#32, which is closed), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-13-publisher-object-storage-origin-design.md` and `2026-10-13-request-addressing-design.md`, both in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** IABTechLab/trusted-server#1378, IABTechLab/trusted-server#1238 (merged, the header rewrite this change extends), jwrosewell/trusted-server#32 (closed) diff --git a/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md b/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md index 39cbd6a7c5..0f4e8c09c3 100644 --- a/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md +++ b/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md @@ -1,6 +1,6 @@ # Design Spec: A publisher's site served from a folder of object storage -**Status:** Proposed, 2026-10-13. Raised as draft pull request IABTechLab/trusted-server#1377 (from jwrosewell/trusted-server#31, which is closed), for Tech Lab review. +**Status:** Proposed, 2026-10-13. Raised as pull request IABTechLab/trusted-server#1377 (from jwrosewell/trusted-server#31, which is closed), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-12-page-middleware-design.md`, `2026-10-12-middleware-creates-a-missing-file-design.md`, `2026-04-28-multi-backend-asset-proxy-design.md` and `2026-10-14-origin-signing-design.md`, with every design document of the stack held in IABTechLab/trusted-server#1342 **Related PRs:** IABTechLab/trusted-server#1377, jwrosewell/trusted-server#31 (closed) diff --git a/docs/superpowers/specs/2026-10-13-request-addressing-design.md b/docs/superpowers/specs/2026-10-13-request-addressing-design.md index f1ca4eda12..ad2149550f 100644 --- a/docs/superpowers/specs/2026-10-13-request-addressing-design.md +++ b/docs/superpowers/specs/2026-10-13-request-addressing-design.md @@ -1,6 +1,6 @@ # Design Spec: A deployment may address a request by its path before its configuration is chosen -**Status:** Proposed, 2026-10-13. Raised as draft pull request IABTechLab/trusted-server#1376 (from jwrosewell/trusted-server#30, which is closed), for Tech Lab review. +**Status:** Proposed, 2026-10-13. Raised as pull request IABTechLab/trusted-server#1376 (from jwrosewell/trusted-server#30, which is closed), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-12-config-by-host-design.md`, `2026-10-12-middleware-prepare-design.md`, `2026-10-12-prefix-module-design.md`, `2026-08-27-integration-provider-seam-design.md` and `2026-10-15-cookies-by-path-design.md`, all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** IABTechLab/trusted-server#1376, jwrosewell/trusted-server#30 (closed) diff --git a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md index cc89256a77..b68fd0838a 100644 --- a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md +++ b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md @@ -1,6 +1,6 @@ # Design Spec: A deployment using 51Degrees gets client hints from one middleware -**Status:** Proposed, 2026-10-14. Raised as draft pull request IABTechLab/trusted-server#1386 (from jwrosewell/trusted-server#39), for Tech Lab review. +**Status:** Proposed, 2026-10-14. Raised as pull request IABTechLab/trusted-server#1386 (from jwrosewell/trusted-server#39), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-12-shared-vendor-tables-design.md` (the `[shared.51degrees]` table), `2026-10-12-loading-cache-design.md` (the `51degrees-answers` cache), `2026-10-12-page-middleware-design.md` (the `[[serve]]` entries), `2026-10-14-source-modules-design.md` (the source a page is fetched from) and `2026-10-17-request-filters-on-every-adapter-design.md` (the request filters every adapter runs), all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** jwrosewell/trusted-server#39, IABTechLab/trusted-server#1386 diff --git a/docs/superpowers/specs/2026-10-14-auction-protocol-design.md b/docs/superpowers/specs/2026-10-14-auction-protocol-design.md index 3f30f1063b..426fd2eade 100644 --- a/docs/superpowers/specs/2026-10-14-auction-protocol-design.md +++ b/docs/superpowers/specs/2026-10-14-auction-protocol-design.md @@ -1,6 +1,6 @@ # Design Spec: OpenRTB is an auction protocol module, and core sends the request a demand source builds -**Status:** Proposed, 2026-10-14. Raised as draft pull request IABTechLab/trusted-server#1388 (from jwrosewell/trusted-server#40), for Tech Lab review. +**Status:** Proposed, 2026-10-14. Raised as pull request IABTechLab/trusted-server#1388 (from jwrosewell/trusted-server#40), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-03-19-auction-orchestration-flow-design.md` (on the main branch), `2026-10-14-consent-decoders-design.md` (IABTechLab/trusted-server#1383), `2026-10-14-identifier-modules-design.md` (IABTechLab/trusted-server#1382) and `2026-10-14-no-modules-crate-design.md` (IABTechLab/trusted-server#1384), with every design document of the stack in IABTechLab/trusted-server#1342 **Related PRs:** IABTechLab/trusted-server#1388, jwrosewell/trusted-server#40 diff --git a/docs/superpowers/specs/2026-10-14-backend-naming-design.md b/docs/superpowers/specs/2026-10-14-backend-naming-design.md index 71f28f21ae..769127dd3e 100644 --- a/docs/superpowers/specs/2026-10-14-backend-naming-design.md +++ b/docs/superpowers/specs/2026-10-14-backend-naming-design.md @@ -1,6 +1,6 @@ # Design Spec: Each adapter names its own backends -**Status:** Proposed, 2026-10-14. Raised as draft pull request IABTechLab/trusted-server#1390 (from jwrosewell/trusted-server#42), for Tech Lab review. +**Status:** Proposed, 2026-10-14. Raised as pull request IABTechLab/trusted-server#1390 (from jwrosewell/trusted-server#42), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-10-config-first-auction-provider-architecture-design.md`, `2026-03-19-edgezero-migration-design.md`, `2026-10-14-last-names-in-core-design.md` **Related PRs:** jwrosewell/trusted-server#42, IABTechLab/trusted-server#1390 diff --git a/docs/superpowers/specs/2026-10-14-consent-decoders-design.md b/docs/superpowers/specs/2026-10-14-consent-decoders-design.md index 230e8efdcc..f83a58b352 100644 --- a/docs/superpowers/specs/2026-10-14-consent-decoders-design.md +++ b/docs/superpowers/specs/2026-10-14-consent-decoders-design.md @@ -1,6 +1,6 @@ # Design Spec: The consent decoders live in the crates named for their standards -**Status:** Proposed, 2026-10-14. Raised as draft pull request IABTechLab/trusted-server#1383 (from jwrosewell/trusted-server#37), for Tech Lab review. +**Status:** Proposed, 2026-10-14. Raised as pull request IABTechLab/trusted-server#1383 (from jwrosewell/trusted-server#37), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-14-source-modules-design.md`, in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** IABTechLab/trusted-server#1383, jwrosewell/trusted-server#37 diff --git a/docs/superpowers/specs/2026-10-14-identifier-modules-design.md b/docs/superpowers/specs/2026-10-14-identifier-modules-design.md index 3ff985562c..c913b0c41b 100644 --- a/docs/superpowers/specs/2026-10-14-identifier-modules-design.md +++ b/docs/superpowers/specs/2026-10-14-identifier-modules-design.md @@ -1,9 +1,9 @@ # Design Spec: A module hands core the identifiers it learned for a request -**Status:** Proposed, 2026-10-14. Raised as draft pull requests jwrosewell/trusted-server#36 and IABTechLab/trusted-server#1382, for Tech Lab review. +**Status:** Proposed, 2026-10-14. Raised as draft pull request jwrosewell/trusted-server#36 and as pull request IABTechLab/trusted-server#1382, for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-07-10-kv-eid-request-snapshot-ec-recovery-design.md`, `2026-09-08-1138-per-cookie-template-cache-policy-design.md` -**Related PRs:** jwrosewell/trusted-server#36 (fork draft) and IABTechLab/trusted-server#1382 (Tech Lab draft). IABTechLab/trusted-server#1342 holds every design document of the stack. +**Related PRs:** jwrosewell/trusted-server#36 (fork draft) and IABTechLab/trusted-server#1382 (Tech Lab pull request). IABTechLab/trusted-server#1342 holds every design document of the stack. **Issues:** IABTechLab/trusted-server#1337 **Last updated:** 2026-10-15 diff --git a/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md b/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md index 81155a780c..b559ce5dcb 100644 --- a/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md +++ b/docs/superpowers/specs/2026-10-14-last-names-in-core-design.md @@ -1,6 +1,6 @@ # Design Spec: Small vendor, standard and platform names leave core -**Status:** Proposed, 2026-10-14. Raised as draft pull request IABTechLab/trusted-server#1389 (from jwrosewell/trusted-server#41), for Tech Lab review. +**Status:** Proposed, 2026-10-14. Raised as pull request IABTechLab/trusted-server#1389 (from jwrosewell/trusted-server#41), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-14-source-modules-design.md`, `2026-10-14-no-modules-crate-design.md` and `2026-10-13-request-addressing-design.md` **Related PRs:** IABTechLab/trusted-server#1389, jwrosewell/trusted-server#41 diff --git a/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md b/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md index a3697f2021..e2a1bad947 100644 --- a/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md +++ b/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md @@ -1,6 +1,6 @@ # Design Spec: Each adapter lists the modules it links, each module describes its middleware, and the project's own modules are renamed iabtl -**Status:** Proposed, 2026-10-14. Raised as draft pull request IABTechLab/trusted-server#1384 (from jwrosewell/trusted-server#38), for Tech Lab review. +**Status:** Proposed, 2026-10-14. Raised as pull request IABTechLab/trusted-server#1384 (from jwrosewell/trusted-server#38), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-14-source-modules-design.md`, in IABTechLab/trusted-server#1342, which replaced IABTechLab/trusted-server#1084 as the one place for every design document of the stack of pull requests listed in IABTechLab/trusted-server#777 **Related PRs:** IABTechLab/trusted-server#1384, jwrosewell/trusted-server#38 diff --git a/docs/superpowers/specs/2026-10-14-origin-signing-design.md b/docs/superpowers/specs/2026-10-14-origin-signing-design.md index 3324d3ffc3..62c3a54445 100644 --- a/docs/superpowers/specs/2026-10-14-origin-signing-design.md +++ b/docs/superpowers/specs/2026-10-14-origin-signing-design.md @@ -1,6 +1,6 @@ # Design Spec: An origin signer is a module a source's auth table names -**Status:** Proposed, 2026-10-14. Raised as draft pull request jwrosewell/trusted-server#43, and as draft pull request IABTechLab/trusted-server#1391, for Tech Lab review. +**Status:** Proposed, 2026-10-14. Raised as draft pull request jwrosewell/trusted-server#43, and as pull request IABTechLab/trusted-server#1391, for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-14-source-modules-design.md`, `2026-10-13-publisher-object-storage-origin-design.md`, `2026-05-19-asset-s3-auth-fastly-io-design.md` and `2026-10-14-no-modules-crate-design.md`, all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** jwrosewell/trusted-server#43, IABTechLab/trusted-server#1391 diff --git a/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md b/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md index de7c66b4a7..c3e1c98fda 100644 --- a/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md +++ b/docs/superpowers/specs/2026-10-14-rsc-flight-middleware-design.md @@ -1,6 +1,6 @@ # Design Spec: The Flight rewriter is a middleware of the Next.js module -**Status:** Proposed, 2026-10-14. Raised as draft pull request IABTechLab/trusted-server#1381 (from jwrosewell/trusted-server#35), for Tech Lab review. +**Status:** Proposed, 2026-10-14. Raised as pull request IABTechLab/trusted-server#1381 (from jwrosewell/trusted-server#35), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-09-07-850-parser-aware-body-hold-nextjs-streaming-design.md`, `2026-10-14-source-modules-design.md` **Related PRs:** IABTechLab/trusted-server#1381, jwrosewell/trusted-server#35 diff --git a/docs/superpowers/specs/2026-10-14-source-modules-design.md b/docs/superpowers/specs/2026-10-14-source-modules-design.md index ef9f31d921..c2a99715e5 100644 --- a/docs/superpowers/specs/2026-10-14-source-modules-design.md +++ b/docs/superpowers/specs/2026-10-14-source-modules-design.md @@ -1,6 +1,6 @@ # Design Spec: A source module says where the publisher path and the asset routes fetch from -**Status:** Proposed, 2026-10-14. Raised as draft pull request IABTechLab/trusted-server#1380 (from jwrosewell/trusted-server#34), for Tech Lab review. +**Status:** Proposed, 2026-10-14. Raised as pull request IABTechLab/trusted-server#1380 (from jwrosewell/trusted-server#34), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-13-publisher-object-storage-origin-design.md`, `2026-10-13-origin-redirect-design.md`, `2026-10-13-request-addressing-design.md`, `2026-04-28-multi-backend-asset-proxy-design.md` **Related PRs:** jwrosewell/trusted-server#34, IABTechLab/trusted-server#1380 diff --git a/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md b/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md index 94d891d4cc..943aed18b9 100644 --- a/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md +++ b/docs/superpowers/specs/2026-10-15-cookies-by-path-design.md @@ -1,6 +1,6 @@ # Design Spec: Core keeps a publisher's cookies under that publisher's path when one host serves several -**Status:** Proposed, 2026-10-15. Raised as draft pull request IABTechLab/trusted-server#1400 (from jwrosewell/trusted-server#45), for Tech Lab review. +**Status:** Proposed, 2026-10-15. Raised as pull request IABTechLab/trusted-server#1400 (from jwrosewell/trusted-server#45), for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-13-request-addressing-design.md`, `2026-10-12-prefix-module-design.md` **Related PRs:** IABTechLab/trusted-server#1400, jwrosewell/trusted-server#45 diff --git a/docs/superpowers/specs/2026-10-15-geo-module-request-design.md b/docs/superpowers/specs/2026-10-15-geo-module-request-design.md index 2534fd5e4e..6f8a9869cf 100644 --- a/docs/superpowers/specs/2026-10-15-geo-module-request-design.md +++ b/docs/superpowers/specs/2026-10-15-geo-module-request-design.md @@ -1,6 +1,6 @@ # Design Spec: A geo module is called with the request -**Status:** Proposed, 2026-10-15. Written after the code, for draft pull request IABTechLab/trusted-server#1374 +**Status:** Proposed, 2026-10-15. Written after the code, for pull request IABTechLab/trusted-server#1374 **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-12-module-context-design.md`, `2026-10-12-device-signals-design.md`, `2026-08-27-integration-provider-seam-design.md`, `2026-09-07-didomi-geo-design.md`. Every design document of the stack is in IABTechLab/trusted-server#1342. **Related PRs:** IABTechLab/trusted-server#1374, reviewed on jwrosewell/trusted-server#28 (closed) diff --git a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md index 95e5d49639..5d04a5bf28 100644 --- a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md +++ b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md @@ -1,6 +1,6 @@ # Design Spec: A module declares the caches it loads through -**Status:** Proposed, 2026-10-15. Written after the code, for draft pull request IABTechLab/trusted-server#1373. Every design document of the stack is in IABTechLab/trusted-server#1342. +**Status:** Proposed, 2026-10-15. Written after the code, for pull request IABTechLab/trusted-server#1373. Every design document of the stack is in IABTechLab/trusted-server#1342. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-12-loading-cache-design.md`, `2026-10-12-robots-txt-from-modules-design.md`, `2026-08-27-integration-provider-seam-design.md` **Related PRs:** IABTechLab/trusted-server#1373, reviewed on jwrosewell/trusted-server#27 (closed) diff --git a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md index 0ffd46f231..75bf1e8c5b 100644 --- a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md @@ -1,6 +1,6 @@ # Design Spec: Asset routes are served on every adapter -**Status:** Proposed, 2026-10-17. Implemented as the draft pull request [IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412), row 54 of the stack, above [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411), which creates and writes the Edge Cookie on every adapter, for Tech Lab review. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-17. Implemented as the pull request [IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412), row 54 of the stack, above [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411), which creates and writes the Edge Cookie on every adapter, for Tech Lab review. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-17-request-filters-on-every-adapter-design.md`, `2026-10-17-edge-cookie-on-every-adapter-design.md`, `2026-10-14-source-modules-design.md`, `2026-10-14-origin-signing-design.md`, `2026-10-15-asset-cache-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 **Related PRs:** [IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412). The request filters run on every adapter in [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410) and the Edge Cookie in [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411) From 693b8c451baff44497f69e9de5488cfa4e6e7f7b Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:38 +0000 Subject: [PATCH 182/192] The permission designs take the policy file chosen when a service is deployed No build compiles a permission policy in. A domain's settings name the publisher's policy file in `[permissions] policy` and record its SHA-256, `ts config push --permissions ` supplies the file, and a domain whose file is missing, unnamed or a different file from the one pushed answers 500 and logs the reason. - The permission model design says how the file travels on each adapter, in the blob from IABTechLab/trusted-server#1345 and as one shared entry per file on Fastly from #1352, the 60 seconds a live push waits between a new entry and the blob from #1354, what a release build carries, and what reading the file costs. - The config by host design adds the shared policy entries, one for each file under its SHA-256, each written before the blobs that name it. - The inspection endpoints design shows the policy's name and digest and never its text. - The config migrate design leaves naming the policy file to do, on every adapter, and a document written for main may name it before or after it is migrated. - The provider migration rollout and response header hook designs follow the same model. --- ...integration-response-header-hook-design.md | 79 +- .../2026-07-30-permission-model-design.md | 972 +++++++++++++++--- ...07-30-provider-migration-rollout-design.md | 289 ++++-- .../specs/2026-10-12-config-by-host-design.md | 37 +- .../2026-10-12-inspection-endpoints-design.md | 31 +- .../specs/2026-10-17-config-migrate-design.md | 143 +-- 6 files changed, 1179 insertions(+), 372 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 791d27d4a0..2cbb12e9fa 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -4,24 +4,26 @@ `split/5-response-hook-docs` (§1a). The pull request that carried the documentation set ([#1047](https://github.com/IABTechLab/trusted-server/pull/1047)) was closed -without merging on 2026-10-15, and draft pull request -[#1347](https://github.com/IABTechLab/trusted-server/pull/1347) carries the -same set. This spec is held with every other design -document of the stack in +without merging on 2026-10-15, and +[#1347](https://github.com/IABTechLab/trusted-server/pull/1347), one of the +stack's pull requests, carries the same set. This spec is held with every +other design document of the stack in [#1342](https://github.com/IABTechLab/trusted-server/pull/1342). The hook was removed from the earlier draft of the documentation set because it has no consumer, which is this spec's own §-rule for speculative surface. The spec is retained as the design bar for the hook when its first consumer arrives (an integration that must set response headers such as Accept-CH or detection -results). Revised 2026-08-25, on 2026-10-06 for the module wording, and on -2026-10-15 and 2026-10-16 after its round 1 and round 2 reviews (Review log). +results). Revised 2026-08-25, on 2026-10-06 for the module wording, on +2026-10-15 and 2026-10-16 after its round 1 and round 2 reviews (Review log), +and on 2026-10-26 for the permission policy, which is selected when a service +is deployed. **Author:** Engineering **Issue references:** #782 **Related specs:** `2026-07-30-pluggable-providers-design.md`, `2026-07-30-permission-model-design.md`, `2026-07-30-provider-migration-rollout-design.md` and the baseline DataDome design `2026-06-11-datadome-server-side-protection-design.md` -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-26 > **Context.** Issue #782 already specifies this feature well; its done-when > is the contract. PR #838 shipped the trait and registry wiring with **no @@ -30,8 +32,9 @@ design `2026-06-11-datadome-server-side-protection-design.md` > restates the contract plus the two details the issue left open (ordering > and collision policy), and adds the rule that prevents a repeat, which is > that the hook lands with a consumer or not at all. The series (PRs -> #1043-#1047, closed on 2026-10-15 and carried now by drafts #1343-#1347) -> applied that rule to this spec itself, see the Status above and §7. +> #1043-#1047, closed on 2026-10-15 and carried now by the stack's pull +> requests #1343-#1347) applied that rule to this spec itself, see the +> Status above and §7. --- @@ -62,11 +65,11 @@ This section states the code at `split/5-response-hook-docs` (commit that design is built. A search of the code and the guides at that commit finds none of the names below, and the same search at `split/7-integration-seam-impl`, `split/31-caching-release` and -`split/50-cookies-by-path` (the tip of the stack) finds none either. The names -are `IntegrationResponseMutator`, `with_response_mutator`, -`behavior_revision`, `HOOK_INVARIANT_REVISION`, the seven fields of the cache -revision tuple in §3, `SecurityUse`, `DataDomeSecurityRequestFilter`, -`RedactedRequestView`, `PartnerEgress` and `vary_digest_key_secret_name`. +`split/50-cookies-by-path` (row 50) finds none either. The names are +`IntegrationResponseMutator`, `with_response_mutator`, `behavior_revision`, +`HOOK_INVARIANT_REVISION`, the seven fields of the cache revision tuple in §3, +`SecurityUse`, `DataDomeSecurityRequestFilter`, `RedactedRequestView`, +`PartnerEgress` and `vary_digest_key_secret_name`. **Not built, and what each part waits for.** @@ -74,16 +77,25 @@ revision tuple in §3, `SecurityUse`, `DataDomeSecurityRequestFilter`, mutator and no revision number, and no adapter has an apply point. - **The cache revision tuple (§2, §3 and §3a).** Four of its seven fields (`active_policy_digest`, `active_policy_ordinal`, `model_epoch` and - `activation_generation`) come from the permission model's policy - activation. Section 5.5 of `2026-07-30-permission-model-design.md` is headed - "Policy revision activation (deferred)" and begins "Not implemented", the - record in its section 11 says "None of it exists", and row 19 of §8 of - `2026-07-30-provider-migration-rollout-design.md` says "Not implemented". - Each build embeds `permissions.yaml`, so the deployed build is the policy. - The other three fields (`integration_registry_revision`, - `effective_config_revision` and `hook_invariant_revision`) have no code - either. The `pre_epic_v1` test in §4 item 8, the serve fence in §4a and the - effective-config digest in §4a.3 depend on the same activation. + `activation_generation`) come from the permission model's policy activation. + Section 5.5 of `2026-07-30-permission-model-design.md` is headed "Policy + revision activation (deferred)" and begins "Not implemented", the record in + its section 11 says "None of it exists", and row 19 of §8 of + `2026-07-30-provider-migration-rollout-design.md` says "Not implemented". At + this commit each build embeds the repository's sample, + `config/permissions/sample.yaml` (`DEFAULT_PERMISSION_RULES`, + `permissions.rs` line 494), so the deployed build is the policy. Row 4 of + the stack (#1345) builds section 3.1 of the permission model design, and at + row 4 no build carries a policy and a domain's blob, the settings as + `ts config push` stores them, names its policy file by name and SHA-256. + Once row 4 is merged forward into this row, the pushed blob is the policy + identity and `active_policy_digest` takes that `sha256`, while + `active_policy_ordinal`, `model_epoch` and `activation_generation` still + have nothing to come from. The remaining three fields + (`integration_registry_revision`, `effective_config_revision` and + `hook_invariant_revision`) have no code either. The `pre_epic_v1` test in §4 + item 8, the serve fence in §4a and the effective-config digest in §4a.3 + depend on the same activation. - **The keyed digest of `Vary` request values (§2).** The setting `[cache] vary_digest_key_secret_name` does not exist. `CacheSettings` in `settings.rs` holds `asset_rules` only and refuses an unknown key, and no @@ -100,7 +112,7 @@ revision tuple in §3, `SecurityUse`, `DataDomeSecurityRequestFilter`, its table is `[integrations.datadome]`. From `split/7-integration-seam-impl` on, DataDome is the crate `crates/bot-protection/datadome` and its table is `[bot-protection.datadome]`. None of the settings that §4a.3 adds exists in - the tree at either branch or at the tip of the stack. + the tree at either branch or at `split/50-cookies-by-path`. **What exists.** @@ -1519,10 +1531,10 @@ first consumer, §7) merges: response-header hook. The hook was removed from that PR before the series was presented, so PR #1047 carried the documentation set only, and no hook trait, registration builder, or adapter call site exists in the series' - tree (§1a). PR #1047 was closed without merging on 2026-10-15, and draft - PR #1347 carries the same set at the same commit (see the Status). This is - exactly the outcome §4 item 3 and §5 require for a feature with no - consumer, applied to the feature's own spec. + tree (§1a). PR #1047 was closed without merging on 2026-10-15, and one of + the stack's pull requests, PR #1347, carries the same set at the same + commit (see the Status). This is exactly the outcome §4 item 3 and §5 + require for a feature with no consumer, applied to the feature's own spec. - The hook returns together with its first consumer, meaning an integration that must set response headers, for example `Accept-CH` client-hint requests or detection results such as §4a's security channel. The @@ -1563,6 +1575,14 @@ changes no code and decides none of them. activation exists. The tuple in §3 names four fields from that activation and none has code. The template cache checks `template_fingerprint`, which hashes the build digest, the browser bundles and the whole typed settings. + At `61b1399c6` the build digest does not hash + `config/permissions/sample.yaml`, because core's build script hashes only + its `src` folder and the files it lists, so a change to the compiled sample + leaves the fingerprint as it was. Row 4 of the stack (#1345) builds section + 3.1 of the permission model design, and at row 4 the policy's `sha256` sits + in the typed settings, so a policy change changes the fingerprint, which + leaves the policy's `text` out because `sha256` already names the file + (`template_fingerprint`, `publisher.rs` lines 2116 to 2145 at row 4). 3. Whether `Edge-Control` joins `EDGE_CACHE_HEADER_NAMES`. The code strips four names and has no `edge-control`. If it joins, `remove_edge_cache_headers`, `is_edge_cache_header_name` and the tests of both change with it. @@ -1610,6 +1630,7 @@ changes no code and decides none of them. | 2026-10-06 | Module wording. | | 2026-10-15 | Fixes from review round 1. The status and the closed pull request, a new section 1a on what the code does today, the cookie rule, the shared finalization code and the DataDome section name stated as the code has them, the reserved CDN cache fields taken from `EDGE_CACHE_HEADER_NAMES`, the three ids the id rule refuses, the template cache and the cache revision tuple marked as unreconciled or not built, the changes to shipped DataDome behavior in section 4a.4, the order of sections 4 and 4a, the sign-off rows, `must-revalidate` precedence, the duplicate-ID test in the done-when, the existing tests, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. What the providers design now says (section 1a), the two core functions that every adapter's `apply_finalize_headers` calls (sections 1a and 2), the closed pull request that carried the documentation set (#1047, section 7), the revision dates and the review log. | +| 2026-10-26 | Section 1a keeps that each build embeds the repository's sample at its commit, and says that row 4 of the stack (#1345) builds section 3.1 of the permission model design, so that once row 4 is merged forward a domain's blob, which names its policy file by name and SHA-256, is the policy identity and gives `active_policy_digest` its value. Open question 2 says that row 4's policy `sha256` reaches the template fingerprint, which leaves the policy's text out and which the compiled sample never reached. The stack's pull requests are named without calling them drafts. | ## Review log diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 0e37b86676..4487032d80 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -1,26 +1,37 @@ # Design Spec: Jurisdiction Permission Model **Status:** Proposed. The implementation is on the branch -`split/3-permissions`, which is row 4 of the stack of draft pull requests that -builds this design. The stack starts at the pull request that carries the -design documents (IABTechLab/trusted-server#1342), and the draft for row 4 -(IABTechLab/trusted-server#1345) is not yet merged to main. The pull request -that carried the implementation before (#1045) was closed on 2026-10-15 -without merging. Revised against that implementation, 2026-08-25. Revised -again on 2026-09-01 for the `rules:` tree, which replaces the flat rule keys -and retires `[geo] default_country` (§3.2, §5.4, and the §12 record). Revised -again on 2026-09-14 for permission signal modules, which replace the fixed -signal precedence with a configured order (§4, and the §13 record). Revised on +`split/3-permissions` of the fork jwrosewell/trusted-server, which is row 4 of +the stack's pull requests that build this design, numbered as in +IABTechLab/trusted-server#777. Row 4's pull request +(IABTechLab/trusted-server#1345) carries the same commits on the branch +`feature/3-permissions` and is not yet merged to main. The stack starts at the +pull request that carries the design documents +(IABTechLab/trusted-server#1342). The pull request that carried the +implementation before (#1045) was closed on 2026-10-15 without merging. +Revised against that implementation, 2026-08-25. Revised again on 2026-09-01 +for the `rules:` tree, which replaces the flat rule keys and retires +`[geo] default_country` (§3.2, §5.4, and the §12 record). Revised again on +2026-09-14 for permission signal modules, which replace the fixed signal +precedence with a configured order (§4, and the §13 record). Revised on 2026-10-06 to say module for what an operator selects, and on 2026-10-07 to name a module by its crate folder, so the selector reads `[permission-signal] modules`. Revised on 2026-10-15 after a review of this page against the code at `split/3-permissions` (see the Review log), so that -§1 to §8 and the records in §10 to §13 state what that code does. +§1 to §8 and the records in §10 to §13 state what that code does. Revised on +2026-10-26 for the policy file, which is never compiled into a build, as +agreed with IAB Tech Lab at the Trusted Server Touchbase on 2026-10-23, and +which is selected and supplied when a service is deployed, as decided by +51Degrees after the Touchbase (§3.1, §3.6 and §3.7). Row 4 builds that design, +and where this page says how the policy file reaches a deployment, it states +the code of row 4's pull request, IABTechLab/trusted-server#1345. Where this +page says what rows 9 and 11 add, it states the code of their pull requests, +IABTechLab/trusted-server#1350 and IABTechLab/trusted-server#1352. **Author:** Engineering **Issue references:** #779 **Related specs:** `2026-07-30-pluggable-providers-design.md`, `2026-07-30-provider-migration-rollout-design.md` -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-26 > **Context.** PR #838 proposed a permission model whose review surfaced two > classes of defect this spec exists to prevent. The first was silent @@ -36,11 +47,11 @@ page against the code at `split/3-permissions` (see the Review log), so that > `crates/trusted-server-core/src/permission_signal/mod.rs`, > `crates/trusted-server-core/src/ec/consent.rs` and > `crates/trusted-server-adapter-axum/tests/permission_signals.rs`. One -> structural position of the 2026-07-31 draft was not adopted: policy remains -> a build-time-embedded -> `permissions.yaml`, not a `[permissions]` section of `trusted-server.toml`, -> because the runtime config push and activation apparatus the draft assumed -> does not exist yet (§3.1). Every other draft position that was narrowed, +> structural position of the 2026-07-31 draft is adopted in part. The policy +> stays a `permissions.yaml` file rather than rules in a `[permissions]` +> section of `trusted-server.toml`, and `[permissions]` names that file, so +> the file is selected and supplied when a service is deployed and never +> compiled into a build (§3.1). Every other draft position that was narrowed, > simplified, or deferred is recorded in §11, and the move to configured > precedence in §13. @@ -128,7 +139,7 @@ the vocabulary in two tiers: signal therefore reaches eleven of the fifty-three under the sample policy, and the other forty-two keep their configured baseline. They exist for completeness, testing, and demonstration, and where no informed policy - decision has been made the shipped file sets them `denied`. + decision has been made the repository's sample sets them `denied`. The two Data Uses that carry enforcement weight today are `necessary.operations.storage` (TCF Purpose 1, storage) and @@ -141,7 +152,7 @@ This is a deliberate departure from the draft's rule that a permission appears only when it has both a signal mapping and an enforcement point. The eleven named Data Uses all have the signal mapping, and the fifty-three further Data Uses are declared policy rather than enforced policy, as no module gates -on them. The file header of `permissions.yaml` states this plainly, and the +on them. The header of the repository's sample states this plainly, and the `denied` default means an undeployed flag cannot silently authorize anything. Policy validation still **rejects** any rule or flag that references an identifier outside the modeled vocabulary, so a policy cannot name a Data Use @@ -168,33 +179,433 @@ spelling.) ## 3. Policy -### 3.1 Location: `permissions.yaml`, compiled into the build +### 3.1 The policy file is selected when a service is deployed Policy lives in a human-editable permissions YAML document, which this spec -calls `permissions.yaml` after the name the code and the guides use. The -repository file the build compiles in is `config/permissions/sample.yaml`, a -sample for testing and evaluation only, and no other file or setting supplies -policy on this row. The build embeds the file with `include_str!`, and the -embedded text is parsed once per instance (cached behind a `OnceLock` in -`PermissionMaps::standard`). A deployer edits or replaces the file and -rebuilds to change policy. The file is not read at runtime. - -This keeps the mechanism the draft rejected, for a reason the draft's own -premise no longer supports. The draft required policy to flow through the -runtime config pipeline (`ts config push`, staged activation, §5.5), and that -activation apparatus does not exist. Publishing a `[permissions]` TOML section -with no activation protocol would reintroduce exactly the mixed-revision and -lazy-validation hazards the draft cataloged. Until the runtime pipeline -exists, the embedded file is the safer home, and moving policy to runtime -configuration is recorded as deferred follow-up (§11), not abandoned. - -Within the embedded model, the draft's specific complaints are answered: - -- The parse runs once per instance, and the embedded file is a build-time - constant covered by unit tests, so a malformed committed file fails the - test suite rather than surfacing as a per-request failure. The documented - panic on a malformed embedded file is a build defect signal, not a runtime - condition. +calls `permissions.yaml` after the name the code and the guides use. No +build carries a policy and no policy ships with Trusted Server, so the only +default is no policy at all. A service is given the policy for each domain +when it is deployed, and a domain without a valid policy file does not +start. On 2026-10-23, at the Touchbase, it was agreed with IAB Tech Lab +that neither the configuration nor the policy is ever compiled into the WASM +binary, so that Tech Lab can release one binary for every publisher, and +that a binary starts no domain unless it is given a valid policy. The +attestation issue, IABTechLab/trusted-server#1235, gives the reason as "An +attestation only means something if every publisher runs the same bytes." +How the file reaches a service, as this section describes it, was decided +by 51Degrees after the Touchbase. §3.6 lists what a build does carry. + +Row 4 of the stack (IABTechLab/trusted-server#1345) builds this design, and +this section states the code of that pull request. Where this section says +what rows 9 and 11 add, it states the code of their pull requests, +IABTechLab/trusted-server#1350 and IABTechLab/trusted-server#1352, whose +branches carry row 4's commits. Row numbers here are those of +IABTechLab/trusted-server#777, and a branch keeps its own number. No build +carries a policy, because `DEFAULT_PERMISSION_RULES`, +`PermissionMaps::standard`, `validate_default_policy` and +`GeoConfig::validate_permission_policy`, which compiled the repository's +sample into core, checked it and parsed it, are gone. Each adapter keeps the +policy its domain loaded in its application state and hands it to every +request through `RuntimeServices::permission_policy` (`platform/types.rs` line +340). The Edge Cookie context passes that policy's rules (`ec/mod.rs` +line 461) to `default_jurisdiction`, `assemble_permissions` and +`storage_acquisition` in `ec/consent.rs`, and through +`ConsentPipelineInput::permission_maps` (`consent/mod.rs` line 91) to +`detect_jurisdiction` in `consent/jurisdiction.rs`, each of which takes the +rules as an argument. The services builder requires the policy as it requires +the stores (`platform/types.rs` lines 636 to 638), and each adapter refuses to +build its state from settings that carry none, through +`Settings::required_permission_policy` (`settings.rs` line 3936), so no +request is decided without a policy and nothing stands in for a missing one. +The tests read the sample through `PermissionPolicy::sample()` and +`PermissionMaps::sample()`, which only test builds and core's `test-utils` +feature compile (`permissions.rs` lines 870 to 918), and the `gpc`, `gpp`, +`tcf` and `us-privacy` crates list core with `test-utils` under their +`[dev-dependencies]` for them. + +**Blobs, pointers and chunks.** A blob is a domain's settings as +`ts config push` stores them, wrapped in EdgeZero's envelope (`BlobEnvelope`, +which holds the settings as `data` with their SHA-256, the time the envelope +was made and a version). A Fastly config store entry holds at most 8,000 +characters, so a longer value is split into chunks of 7,000 characters, each +an entry of its own (`CHUNK_PAYLOAD_TARGET`, `chunked_config.rs` line 42 in +`edgezero-adapter-fastly`), and a pointer, the entry at the value's own key +that names each chunk with its length and SHA-256. + +**What row 4 builds.** + +1. **The file.** The publisher's `permissions.yaml`, in the format it has + today (§3.2). The push takes only a file whose name ends `.yaml`, and the + policy's name is the file's stem, so `standard.yaml` is `standard` + (`policy_name` in + `crates/trusted-server-cli/src/commands/config/policy.rs`, called + `policy.rs` below). The repository's sample is for testing and evaluation + only, and no tool supplies it in place of the publisher's file. +2. **Selected when the service is deployed.** A domain's settings name the + policy the domain runs in `[permissions] policy = ""` + (`PermissionsConfig`, `settings.rs` lines 1354 to 1364). `ts` wraps + EdgeZero's argument types for `ts config push`, `ts config validate` and + `ts config diff` in types of its own that add `--permissions `, once + for each file (`policy.rs` lines 45 to 79), and `ts config validate` gains + no `--adapter`. Each of the three loads the settings with the + `TRUSTED_SERVER__` environment overlay applied, being the environment + variables named `TRUSTED_SERVER__` and a setting's path, which replace that + setting, through EdgeZero's `deserialize_app_config_with_options` and its + raw loader (`load_settings` and `prepare`, lines 263 to 331). An + operator's `--no-env` leaves the overlay out of both loads, because + `load_settings` turns the overlay off in its load options (line 270) and + `prepare` hands the same options to the raw loader (line 302). The push + runs the policy checks, adds to `[permissions]` `sha256`, the SHA-256 of + the exact bytes of the named file in lower-case hexadecimal + (`policy_sha256`, `permissions.rs` line 768), and, except on Axum (item 4), + `text`, the file itself. It hands the result to EdgeZero with `no_env` set, + whether or not the operator gave `--no-env` (line 110), so the overlay is + applied at most once, and the digest is taken after it and before EdgeZero + builds the envelope (`run_push_with`, lines 97 to 118). + EdgeZero `v0.0.10` offers no public function that takes the settings as a + document, because its typed push, validate and diff read them from the + `--app-config` path (`config.rs` lines 452 to 470 in `edgezero-cli`), so + `ts` writes the result to a temporary file under the system temp folder, + never in the working tree, which is removed when the command ends, unless + the command is interrupted at EdgeZero's confirmation prompt + (`write_settings`, lines 439 to 451). The push then prints the policy's + name and digest. + + The policy checks are that the settings name a policy and that a supplied + file has that name (`named_policy`, line 410), that every supplied file is + a `.yaml` policy in UTF-8 text that parses and that no two share a name + (`read_policy_files`, line 352), and that neither the settings nor the + overlay set `sha256` or `text`, which only the push writes + (`refuse_fields_the_push_writes`, line 334). The push always runs them. + `ts config validate` runs them when `--permissions` is given, leaving the + text out because validation writes nothing, and without it says "the policy + was not checked, because no --permissions file was given" + (`run_validate_with`, lines 136 to 168). `ts config diff` compares the + settings as the push would write them, digest included, so when the + settings name a policy it refuses to run without `--permissions` + (`run_diff`, lines 181 to 212). Every command refuses a `sha256` or `text` + that the settings or the overlay set, with or without the flag, because + each loads the settings through `load_settings`. `ts config migrate` and + `ts audit generate` never run the policy checks, and migrate prints its + `to do:` line instead (`2026-10-17-config-migrate-design.md`). The runtime + loader checks the file against the blob's digest (item 7), and a later edit + of the file never changes what an existing blob runs. + +3. **Carried inside the one blob.** At row 4 a service reads one blob, at the + logical store ID (on Fastly `RuntimeStoreConfig::logical()`, `main.rs` line + 330), so on Fastly, Cloudflare and Spin the policy's text travels inside + it. EdgeZero's own push writes the blob on all three, and on Fastly splits + it into chunks as it does any long blob. The repository's sample at row 4 + is 24,271 bytes with the SHA-256 + `1c432b6153ba1a74eb9c28441f0edf7fbd96c08701d1e9d3de01fe56b06572b5`, and + written as a JSON string it adds 24,824 characters, so it adds three or + four chunks to a blob already in chunks, and turns a blob that fitted in + one entry into a pointer and four or five chunks. Each adapter reads the + blob through the path it reads every blob, so row 4 brings no second writer + and no reader of its own, and hands the runtime loader `PolicyOrigin::Blob` + (the Fastly adapter's `app.rs` lines 232 to 238, Spin's `app.rs` line 137 + and Cloudflare's `app.rs` line 144). The loader, + `settings_from_config_blob` (`config_payload.rs` lines 51 to 83), takes + `text` out of the settings, checks its SHA-256 against `sha256`, parses it, + and keeps the policy with the settings, which keep only its name and digest + (`load_domain_policy`, lines 89 to 129), so a blob there without `text` + fails to load. +4. **On Axum, a folder.** On Axum the push writes `policy` and `sha256` + without `text`, and the deployment names a folder of policy files in the + environment variable `TRUSTED_SERVER_PERMISSIONS_DIR`, where an empty value + names none. `AxumPolicyFolder` (the Axum adapter's `platform.rs` lines 160 + to 216) reads `/.yaml`, and refuses a name that is not a + single file name inside the folder, a file it does not hold and a file that + is not UTF-8 text. Axum's `build_state` hands it to the runtime loader as + `PolicyOrigin::Source` when the process starts (`app.rs` line 106), so the + loader checks the file's SHA-256 against `sha256`, the folder has to hold + the bytes that were pushed, and Axum never uses a `text` that its blob + carries. +5. **Cloudflare reads its blob from the config store.** A Worker variable + holds at most 5 KB on every plan (Cloudflare's Workers + [Limits](https://developers.cloudflare.com/workers/platform/limits/) page), + and the sample alone is 24,271 bytes, so the policy cannot travel in the + `TRUSTED_SERVER_CONFIG` variable that main's Cloudflare adapter reads its + settings from (main's `src/lib.rs` line 27). EdgeZero's Cloudflare config + store is a Workers KV namespace (the header of `config_store.rs` in + `edgezero-adapter-cloudflare`), which holds values of up to 25 MiB + (Cloudflare's Workers KV + [Limits](https://developers.cloudflare.com/kv/platform/limits/) page), and + `ts config push` writes the blob there (`push_config_entries` in that + crate's `cli.rs`, through `wrangler kv bulk put`). So the Worker reads its + blob from that namespace. A Workers KV read is asynchronous, and EdgeZero's + `run_app` builds the application with the synchronous `Hooks::build_app`, + so the Worker's fetch entry point calls the adapter's own `build_app`, + which resolves the binding and the key as the push does, from + `EDGEZERO__STORES__CONFIG__TRUSTED_SERVER_CONFIG__NAME` and `__KEY` with + `trusted_server_config` as both defaults, awaits the read, builds the + application from the settings, and serves it with EdgeZero's `dispatch_app` + (the Cloudflare adapter's `lib.rs` lines 19 to 40, and `app.rs` lines 87 to + 159). A binding that does not open, a read that fails and a namespace with + nothing at the key are each a configuration error, so the Worker answers + 500 from the startup error router, which logs the reason from row 54 (item + 7), and the synchronous `Hooks::routes` serves that router too, because it + cannot read the store (`app.rs` lines 490 to 496). `wrangler.toml` and + `wrangler.ci.toml` bind the config store as a namespace of its own, + `trusted_server_config`, beside `TRUSTED_SERVER_KV`, and `cloudflare.toml` + names the same binding. The variable bridge is gone from the adapter's + code, manifests and `README.md`, whose test + `no_wrangler_manifest_carries_settings_in_a_variable` checks that no + manifest carries settings in a variable (`tests/config_defaults.rs`), from + the integration tests' Cloudflare environment, which stores its blob in the + local namespace with `wrangler kv key put`, from + `scripts/smoke-cloudflare.sh`, which pushes into the namespace with + `ts config push --adapter cloudflare --local`, and from + `docs/guide/cloudflare.md`, with its hand copy and double encoding. +6. **One way in.** Core declares `PlatformPolicySource` + (`platform/policy_source.rs` lines 13 to 23), a trait with one method, + `policy_text`, that returns the text of the file a reference names, the + reference being `policy` and `sha256` (`PolicyReference`, `permissions.rs` + line 775), and `PolicyOrigin` (lines 27 to 32), which tells the runtime + loader whether a domain's file comes from its blob or from a source. At row + 4 only Axum implements the trait, over the folder, because elsewhere the + text arrives inside the blob. `load_policy` (`permissions.rs` line 821) + checks the SHA-256 of the text against `sha256`, parses it, and keeps one + parsed copy for each digest for the life of the instance, under whatever + name it is loaded (`parsed_policy`, lines 847 to 866), so two domains that + pin one digest share one copy. +7. **Never started without a valid file.** A domain fails to load when its + settings name no policy, or name one with no `sha256`, when the file is + absent, which on Fastly, Cloudflare and Spin means a blob without `text`, + when on Axum the file cannot be read from its folder or + `TRUSTED_SERVER_PERMISSIONS_DIR` is unset, and when the file's SHA-256 does + not match `sha256` or its YAML fails the checks of §3.3 + (`load_domain_policy` and `load_policy`). Each adapter's startup router + answers with the status of the error's kind (`status_code` in `error.rs`), + 500 for a configuration or settings error and 503 for a key-value store + error (`TrustedServerError::KvStore`), and logs the reason at error level, + so nothing is served from the origin. Core reports each of these failures + as `TrustedServerError::Configuration`, with a reason that names the + policy, so every request for the domain answers 500. Every failed read of + the config store is a configuration error as well, a store that cannot be + opened included (`read_config_entry`, `settings_data.rs` lines 98 to 109), + and Spin reports a store that does not open the same way (the Spin + adapter's `app.rs` lines 112 to 121), so a config store that cannot be read + and an entry or a chunk that is absent each answer 500. Row 4 moves Spin's + startup router from a fixed 503 to the status of the error's kind (`app.rs` + lines 601 to 612), so its configuration errors answer 500 too. Fastly and + Axum log the reason, and Cloudflare and Spin log nothing until row 54, as + the configuration guide says (`docs/guide/configuration.md` lines 784 to + 788), because EdgeZero's `init_logger` does nothing on either (`lib.rs` + lines 39 to 41 in `edgezero-adapter-cloudflare`, and lines 78 to 80 in + `edgezero-adapter-spin`), and row 54 gives each adapter a logger of its + own. On Fastly the `GET /health` probe reads no settings and answers 200. + Spin builds its state, the settings and the policy included, before it + answers any request (`app.rs` lines 653 and 654), and its startup router + still answers `GET /health` with 200. +8. **Changes to main's behavior.** Row 4 changes two behaviors that main has, + and its entries in `CHANGELOG.md` list both. Cloudflare reads its blob from + the config store in place of the `TRUSTED_SERVER_CONFIG` variable (on main, + line 27 of the Cloudflare adapter's `src/lib.rs`). Spin's startup router + answers with the status of the error's kind in place of a fixed 503, so its + configuration errors answer 500 (on main, line 514 of the Spin adapter's + `src/app.rs`). +9. **Kept once read.** A digest never changes what it names, so a parsed copy + is safe to keep while the instance lives, and an instance that already + holds the digest a blob names parses nothing more. Each instance keeps its + copies in memory. Whether Fastly also keeps the parsed form in its value + cache (`PlatformValueCache`, a store of bytes that every instance at a + location shares), which arrives at row 24, is decided by a measurement on + Fastly staging (§3.7). +10. **Seen from outside.** From row 9 (IABTechLab/trusted-server#1350), + `/_ts/config` shows `permissions` as the policy's name and `sha256` only, + never its `text`, and `/_ts/permissions` names the policy whose rules it + resolved, in the field `policy` that `permissions_payload` adds + (`inspect/permissions.rs`, and + `2026-10-12-inspection-endpoints-design.md`). Every place that checks + `[inspect]` patterns goes through `validate_patterns` + (`inspect/config.rs`), being `ts config validate`, `ts config push`, + `ts config diff`, `ts audit generate` and the load, and from row 51 + `ts config migrate`. `validate_patterns` refuses a pattern under + `permissions.text`, which the view never shows, and judges every other + pattern against the settings as the view shows a domain the push wrote + and the loader loaded, with `text` gone and `sha256` present, because the + push always writes it (`as_the_view_sees_a_pushed_domain`). So a pattern + that names `permissions.sha256` is judged alike before the push and at the + load. A name is a file's stem, which two versions of a file share, so the + SHA-256 is what matches a domain to one exact copy of the file. + +**What rows 9 and 11 add over row 4.** + +- **Row 9** (IABTechLab/trusted-server#1350) adds the inspection endpoints. + `permissions_payload` (`inspect/permissions.rs`) gives the permissions + answer a field `policy`, which names the file whose rules the answer + resolved, as `[permissions] policy` names it. The configuration view, which + is the masked settings, shows `policy` and `sha256` and no `text`, because + row 4's runtime loader takes the text out of the settings when it loads the + policy (item 3). `validate_patterns` (`inspect/config.rs`) checks + `[inspect]` patterns against that view (item 10). +- **Row 11** (IABTechLab/trusted-server#1352) adds a blob for each host on + Fastly, the shared policy entries, and the 421 answer for a host with no + blob, which stays 421 and is never 500. Only the Fastly adapter chooses a + blob by host (`config_selection.rs`, and the keyed writer `keyed_push.rs`, + which "Pushes an app-config blob to Fastly under a key other than the + logical store ID"). The keyed writer writes a host's blob without `text`. + For a blob at any key but the logical store ID, the Fastly adapter reads + the policy from the host's policy entry (`blob_carries_its_policy` and + `load_settings_from_config_store` in the adapter's `app.rs`). + `FastlyPolicyEntries` (the adapter's `platform.rs`) implements + `PlatformPolicySource` over that entry, and reads the entry through + `FastlyPlatformConfigStore::get` and core's `resolve_fastly_chunk_pointer` + (from row 49 the adapter's `config_chunks.rs`). Core's `policy_entry_text` + takes the file out of the entry's envelope. A blob at + the logical store ID, which EdgeZero's push writes, keeps `text`. One + function in core, `policy_key` (`platform/policy_source.rs`), forms the key + `permissions_` that the keyed writer and the Fastly adapter use, so + the command line tool and the adapter share one key format, and row 4 has + no key function. Cloudflare and Spin keep the text in their one blob, and + Axum reads the file from its folder when it starts. + +**What later rows add, once each is merged forward over row 4.** + +- **Row 13** adds the live keyed push. At row 11 the keyed writer writes + only the local server's store and refuses a live push (`run` in + `keyed_push.rs`), and from row 13 it writes the live store through the + `fastly` command line tool. +- **Row 24** adds `PlatformValueCache`, the host's value cache, which only + the Fastly adapter registers. The policy uses it only if the measurement + on Fastly staging shows a need (§3.7), and core then uses it wherever an + adapter registers one, through no branch that names a platform. +- **Row 49** adds the Fastly adapter's chunk reader (`config_chunks.rs`), + which takes over from core's `resolve_fastly_chunk_pointer` + (`settings_data.rs`), so from then on the chunks of a blob, and of a + policy entry, are put back together in the Fastly adapter and not in core. +- **Row 51** adds `ts config migrate`. Its merge with row 4 adds the `to do:` + line for a document that names no policy, points the `to do:` line on the + consent lists at the publisher's policy file, and keeps the policy checks + out of the command's validation, as they are kept out of `ts audit generate` + (`2026-10-17-config-migrate-design.md`). +- **Row 54** gives the Cloudflare and Spin adapters loggers of their own + (item 7). + +**The shared policy entries, from row 11.** A host's blob carries `policy` and +`sha256` without `text`, and the file is stored once, as a policy entry under +the key `permissions_`, however many hosts' blobs pin that digest. The +keyed writer loads and checks the settings through `prepare` in `policy.rs`, +as a push to the logical store ID does, `--no-env` included, and leaves the +text out of the blob. It validates the blob as settings and wraps it in +EdgeZero's envelope (the `envelope` function in `keyed_push.rs`). For each +policy file it builds the envelope itself, with `data` holding the policy's +`name` and `text` (`PolicyEntry` in core's `platform/policy_source.rs`), and +splits it into chunks and a pointer as it does a blob (`policy_entries`). The +entry is therefore in EdgeZero's own format, which EdgeZero's Fastly config +store would accept too, because it accepts any value it puts back together +that parses as an envelope and passes `verify` (`chunked_config.rs` lines 586 +to 611). The Fastly adapter reads the entry as it reads a blob (see row 11 +above) and hands the whole value to core's `policy_entry_text`, which parses +the envelope, checks it with `verify` and returns `text`. The runtime loader +then checks `text` against the digest (`load_policy`). Before the push skips +writing an entry that already exists, it reads the entry back and checks that +it holds the same file, and it rewrites one that does not (`holds_policy`). +The push writes an entry for each file `--permissions` supplies that the store +does not already hold, before the host's blob (`run`). At row 11 it writes +only the local server's store in `fastly.toml`, all in one write, so nothing +has to wait. From row 13 (IABTechLab/trusted-server#1354), a live push that +has just written a policy entry waits `POLICY_PROPAGATION_MARGIN_SECONDS`, 60 +seconds, before it writes the blob (`push_remote` in `keyed_push.rs`). +Fastly documents no time for a change to reach every location (its guide +[Working with config stores](https://www.fastly.com/documentation/guides/compute/edge-data-storage/working-with-config-stores) +says only that changes "immediately impact all service versions"), and +EdgeZero notes that the store "is eventually consistent ACROSS keys" +(`config_store.rs` in `edgezero-adapter-fastly`), so the wait is a margin and +not a guarantee. A location that still sees a host's blob before its policy +entry fails to load that host, which answers 500 and logs the reason (item 7). +A hundred hosts that name three files store three policy entries, and each +file the size of the sample is a pointer and four chunks, so fifteen entries +in all. They share the 500 entries that a store of a paid Fastly account holds +(the same guide) with the hundred hosts' own blobs, which take one entry each +where a blob fits in one entry. A changed file is a new entry under a new key, +never an edit of an entry in use, so an instance still serving a previous blob +still finds the file that blob names. The underscore in the key keeps a `Host` +header from naming a policy entry under a selector that is `{host}` alone, +because a host name holds only letters, digits, hyphens and dots +(`2026-10-12-config-by-host-design.md`). + +**While a blob propagates, at row 4.** At row 4 the policy's text changes with +its blob, under one key, on Fastly, Cloudflare and Spin, so it has no +propagation window of its own, and Axum reads its folder when it starts. On +Cloudflare and Spin the blob is one value. On Fastly a blob longer than an +entry is a pointer and chunks, and a location can see a new pointer before all +of its chunks. The Fastly adapter reads its blob itself, through +`FastlyPlatformConfigStore::get` (`platform.rs`) and core's +`resolve_fastly_chunk_pointer` (`settings_data.rs`), and not through +EdgeZero's Fastly config store, which answers a missing chunk with a 503 to +retry (`config_store.rs` in `edgezero-adapter-fastly`, lines 60 to 80 and 108 +to 110). A chunk the store does not yet hold is "not found", which core +reports as `TrustedServerError::Configuration` (`read_config_entry` in +`settings_data.rs`), so the adapter's startup router answers 500, and not +EdgeZero's 503, with the reason logged (`app.rs` lines 1181 to 1183 and 1427). +A failed build is not kept, so the next request at that location tries again +(`main.rs` lines 390 to 395, where `initialize` keeps only a successful +build). From row 49 the adapter's own chunk reader reports a missing chunk as +"not found" through the same `read_config_entry`, so the answer stays 500. + +**The push, further.** + +- `--permissions` may be given once for each file. At row 4 a push writes + one blob, which names one policy, so one file is enough, and any other + file given is parsed and checked but not written. From row 11 a script + that pushes many hosts' blobs can give each push every file the hosts use, + and the keyed push writes each missing policy entry before that host's + blob. +- A checkout that converts line endings changes a file's bytes and so its + digest, and the repository has no `.gitattributes` to prevent it, so the + file must reach the push, and Axum's folder, byte for byte. +- `ts config init` writes the repository's example settings + (`trusted-server.example.toml`). The example names no policy and carries a + comment telling the operator to name their own file in + `[permissions] policy`, and a push of the example as written is refused + with a message naming what is missing. No tool ever chooses a policy. +- No text in row 4's tree says that a build carries a policy. The sample's + header (lines 17 to 23), the permission model guide, the example settings + (`trusted-server.example.toml` lines 153 to 156 and 223 to 235) and the + permissions inspector's `README.md` and `index.html` say that no build + carries one and that a domain's settings name the publisher's own file. The + inspector reads the sample with `include_str!` and parses it with + `PermissionMaps::from_yaml` (`tools/permissions-inspector/wasm/src/lib.rs` + lines 45 and 51). + +**Removing an entry that no blob names is open.** EdgeZero's clean-up +command, `ts config gc` (`ConfigGcArgs` in +`crates/trusted-server-cli/src/run.rs`), does not do it. At EdgeZero +`v0.0.10`, the version the stack pins, the command runs on Fastly only, and +every other adapter answers that it does not implement the command. On +Fastly it removes only chunks, being entries whose key holds the reserved +`.__edgezero_chunks.` part, that no entry in the store points to and that +are older than the window `--older-than` asserts. It keeps every entry at an +ordinary key as one that may still be read, a policy entry among them, +because it cannot tell which entries a blob names (`classify_store_entries` +in EdgeZero's `crates/edgezero-adapter-fastly/src/cli.rs`). Removing a +policy entry needs a command that reads every blob in the store, collects +the digests the blobs name, and removes the other `permissions_` entries +once no instance can still be serving a blob that names them, which is the +assertion `--older-than` makes for chunks. No such command exists. Until one +does, an entry that no blob names stays in the store and changes no answer, +because no blob leads an instance to it. + +The draft wanted policy in `[permissions]`, flowing through `ts config push` +with the settings so that it is validated, compared and pushed with them +(§11). Settings now name the policy in `[permissions]` and the push supplies +the file, so the policy gets all three, while the file keeps the YAML form a +policy owner reads. The draft's staged activation (§5.5) is still not built, +so a push reaches instances as any settings push does, and §5.5 says what +that means while a domain's policy changes. + +The draft's specific complaints are answered: + +- The parse runs once per instance for each digest, when a domain's settings + load, and a file that does not parse fails that load (item 7). Fastly, + Cloudflare and Spin load the settings for each request unless an instance + keeps its application, so such a domain answers 500 on every request until + a valid file is pushed, and no request is ever served under a policy that + did not parse. `ts config push` runs the same parse first, as + `ts config validate` does when `--permissions` is given, so a malformed + file is refused before it reaches a store. - The draft's complaint about unknown fields is only partly answered, because the parser does not reject unknown keys as such. Any key beside `group`, `jurisdiction` and `permissions` in a node of the `rules:` tree is read as a @@ -208,23 +619,28 @@ Within the embedded model, the draft's specific complaints are answered: misspelled key (Open questions, item 3). - Two rule keys naming the same location in different case are rejected at parse, so one spelling cannot silently overwrite another. -- Auditability lives where the draft placed it, in version control. The file - ships in the repository, and its history is the change log. - -The `include_str!` path still reaches above the crate root, so the crate -packaging concern the draft raised remains open and moves with the runtime -follow-up. - -**Fallback posture.** The file is always present, so there is no "no policy" -state. Every location resolves to a group, because the tree's top node is -required and states the baseline for a request whose country cannot be -resolved (§3.2). A failed geo lookup is the separate state, and it resolves -every permission to the **requires-signal floor**, where nothing is set without -a signal that grants it. That floor also covers the belt-and-braces case where -resolution somehow finds no node at all. Absence of an applicable node is -always safe, and there is no fail-open default. The -draft's `regime = "gdpr"` component of the protective profile has no -implemented counterpart because no regime concept exists (§3.2). +- Auditability. The publisher keeps the file under version control or as + they choose, and the SHA-256 in each blob ties what a domain runs to one + exact version of it, which `/_ts/config` shows (item 10). + +No build compiles the sample in, so core's library code reads no policy file +above the crate root, which settles the packaging concern the draft raised +about the `include_str!` path. Only test builds and core's `test-utils` +feature read the sample (`permissions.rs` lines 870 and 871, and §3.5), and +core's build script still reads the workspace's `Cargo.toml`, `Cargo.lock` and +`edgezero.toml` above the crate root (`WORKSPACE_INPUTS` in `build.rs`). + +**Fallback posture.** A domain that serves requests always has a policy, +because a domain whose settings name no valid file never loads (item 7), so +there is no "no policy" state. Every location resolves to a group, because +the tree's top node is required and states the baseline for a request whose +country cannot be resolved (§3.2). A failed geo lookup is the separate state, +and it resolves every permission to the **requires-signal floor**, where +nothing is set without a signal that grants it. That floor also covers the +case where resolution finds no node at all, which should never happen. +Absence of an applicable node is always safe, and there is no fail-open +default. The draft's `regime = "gdpr"` component of the protective profile +has no implemented counterpart because no regime concept exists (§3.2). ### 3.2 Format @@ -237,8 +653,8 @@ Uses. Each permission resolves to an **acquisition rule**: - `denied`, never set, even when a signal grants it. ```yaml -# permissions.yaml (abbreviated). The shipped file lists every Data Use in -# every group so each group's meaning is fully explicit. +# permissions.yaml (abbreviated). The repository's sample lists every Data +# Use in every group so each group's meaning is fully explicit. groups: gdpr-eu: necessary.operations.storage: requires_signal @@ -287,7 +703,7 @@ Format rules, as implemented: - A group is a flat map of Data Use to acquisition flag, with an optional `default` key covering any permission the group omits. A group without `default` must list **every** modeled permission exactly once, or the - parse fails (`IncompleteGroup`). The shipped groups list every Data Use. + parse fails (`IncompleteGroup`). The sample's groups list every Data Use. - `rules` is one tree. Every node has a `group`, and children are optional. Child keys are place codes, so the first level holds ISO 3166-1 alpha-2 country codes and the levels beneath a country hold ISO 3166-2 region codes @@ -376,32 +792,46 @@ Validation runs where the policy actually enters the system: `Jurisdiction` type cannot represent, `us-state` on the top node or a country node, a place written under a region, and a `revokes` keyword other than `all`. An unknown key is not rejected (§3.1). -- **At startup**, the top node of the compiled `permissions.yaml` must carry - both a `group` naming a defined group and a `jurisdiction`, and the no-geo - acknowledgment must be present where required (§5.3). A file whose top node - is missing either key is rejected exactly as a missing - `[geo] default_country` was rejected before. Both are settings-construction - failures, never per-request failures. A name in `[permission-signal] modules` - that the adapter does not link, or a name given twice, fails when the - adapter builds its application state (§4), and the adapter then serves its - startup error router in place of the application. Settings validation - cannot check the names, because only an adapter knows which crates it links, +- **At push and at load**, the same parse runs on the file a domain's settings + name. `ts config push` always parses every file `--permissions` supplies and + refuses settings whose `[permissions] policy` names no supplied file, and + `ts config validate` and `ts config diff` do the same when `--permissions` + is given, while `ts config diff` refuses to run without it when the settings + name a policy (§3.1, item 2). `ts config migrate` and `ts audit generate` + never run these checks. When a domain's settings load, core parses the file + the adapter read for it, once its SHA-256 matches the blob. Either way the + top node must carry both a `group` naming a defined group and a + `jurisdiction`, and a file whose top node is missing either key is rejected + exactly as a missing `[geo] default_country` was rejected before. +- **At startup**, the no-geo acknowledgment must be present where required + (§5.3). A file refused at load and a missing acknowledgment fail when the + settings are built, not while a request is handled, so where an adapter + builds its settings for each request, every request fails the same way. A + name in `[permission-signal] modules` that the adapter does not link, or a + name given twice, fails when the adapter builds its application state (§4). + Each of these failures is a configuration error, so after any of them every + request for the domain answers 500, and the reason is logged at error level, + so nothing is served from the origin. Row 4 moves Spin from a fixed 503 to + the status of the error's kind, as the other adapters answer, and Cloudflare + and Spin log nothing until row 54 (§3.1, item 7). Settings validation cannot + check the module names, because only an adapter knows which crates it links, and the `ts` command line tool links none of them (§6.1). Not implemented from the draft's list, and recorded as future hardening (§11): checking rule-key country parts against the assigned ISO 3166-1 list, checking region parts against assigned ISO 3166-2 subdivisions, and the group identifier grammar. A mistyped country key (`DL` for `DK`) therefore still -parses. For the shipped table the EU and EEA coverage test (§3.5) closes the -consequence the draft cared about, a member state silently dropping to the -fallback. +parses. For the repository's sample the EU and EEA coverage test (§3.5) +closes the consequence the draft cared about, a member state silently +dropping to the fallback. A publisher's own file is checked by the parse +alone, and no test of its coverage runs. ### 3.4 One source of jurisdiction truth (adopted 2026-09-01) The runtime lists `consent.gdpr.applies_in` and `consent.us_states.privacy_states` retire into the `rules:` tree, so one document now carries the permission baselines and the regime applicability -together. The shipped policy file states the same 31 GDPR countries, being the +together. The repository's sample states the same 31 GDPR countries, being the EU 27 plus Iceland, Liechtenstein, Norway and the United Kingdom, which inherit `gdpr` from the top node, and the same 20 US states with a comprehensive privacy law, written as region children of `US` carrying @@ -415,21 +845,224 @@ consent subsystem's gate rather than a policy regime class (§7.4), so unifying the last consumer travels with the dispatch migration. Documentation that pointed operators at the two `[consent]` lists points at the tree instead. -### 3.5 Shipped-table coverage +### 3.5 Sample coverage -Implemented as a unit test -(`every_eu_and_eea_member_requires_a_signal_for_storage` in -`permissions.rs`): every one of the 27 EU member states plus the three EEA -members (IS, LI, NO), 30 codes in all, must have a rule, and each must -resolve `necessary.operations.storage` as `requires_signal`. A mistyped -member-state key fails this test rather than silently diverting the country -to the top node's group. The shipped table maps the EU 27 and EEA to +Implemented as a unit test, +`every_eu_and_eea_member_requires_a_signal_for_storage` in `permissions.rs`, +which requires every one of the 27 EU member states plus the three EEA +members (IS, LI, NO), 30 codes in all, to have a rule, and each to resolve +`necessary.operations.storage` as `requires_signal`. The test reads the +repository's sample, so it guards that file and no publisher's own. A +mistyped member-state key fails this test rather than silently diverting the +country to the top node's group. The sample maps the EU 27 and EEA to `gdpr-eu`, the UK to `gdpr-uk` (storage `granted`, a baseline that row 4 of the rollout design's behavior-preservation matrix, §2, flags for the task force to confirm with its citation, everything else opt-in), and the US and Australia to `us-opt-out`. Countries with no node of their own fall to the tree's top `group` (§5.4). +### 3.6 What the build carries + +IABTechLab/trusted-server#1235 asks for every compile-time input of the +release binary to be listed. Each statement below names the row it describes, +being row 4 (IABTechLab/trusted-server#1345), or row 55 +(IABTechLab/trusted-server#1423), which carries every row's changes, row 4's +included. At both rows a release binary of any adapter carries these, which +are the same for every publisher. + +1. **The code.** The source of core, of the adapter and of every module crate + the adapter links, at the dependency versions `Cargo.lock` pins, built by + the toolchain `rust-toolchain.toml` names (1.98.1 at both rows) with the + Cargo features the build turns on. The Fastly adapter's opt-in + `reusable-sandbox` feature, which lets one Fastly instance (a sandbox) + serve several requests, changes what the binary does at both rows, and so + does core's `client-fixed-demo` at row 55. The crates an adapter links fix + which modules a deployment can select, and the settings choose among them. +2. **The browser scripts.** The bundles that + `crates/trusted-server-js/build.rs` builds from the TypeScript in + `crates/trusted-server-js/lib/src`, and at row 55 in each module crate's + `js/src` (`crate_script_folders`), with the bundler files the script + watches, which its `main` names first. The script runs the `npm` it finds + on the `PATH` (`build_bundles`), with whatever Node.js that `npm` runs. + `.tool-versions` pins Node.js at 24.20.0 for a machine that reads that + file, and the build does not check it. The script installs the packages + `lib/package-lock.json` pins with `npm ci` only when `lib/node_modules` is + missing (`install_dependencies_if_missing`), so a folder already there is + used as it stands, unless `ensure_dependencies_fresh` finds its + `node_modules/.package-lock.json` older than `package-lock.json` or + missing, when the build stops. That check compares modification times, not + contents. In place of a build it copies the bundles from the folder + `TSJS_PREBUILT_DIR` names, which must hold the same set. The scripts that + crates carry whole with `include_str!` are the Google ad tag bootstrap + (`GPT_BOOTSTRAP_JS`), in core's `integrations/gpt.rs` at row 4 and in + `crates/ad-tag/google` at row 55, and at row 55 also InMobi's + `gpp_stub.js` and `tcf_stub.js` (`GPP_STUB_SCRIPT` and `TCF_STUB_SCRIPT` in + `crates/cmp/inmobi/src/lib.rs`). +3. **The Worker glue on Cloudflare.** Cloudflare's `build.sh` installs + `worker-build` at the version of the `worker` crate that `Cargo.lock` pins + and runs it (lines 21 to 39 at both rows). `worker-build` downloads the + `wasm-bindgen` command line tool that matches the locked `wasm-bindgen` + crate, which writes the JavaScript glue the Worker loads beside the WASM, + so a Cloudflare release carries that glue as well. `worker-build --release` + at the locked 0.8.5 also downloads binaryen version 130 and runs its + `wasm-opt -O --all-features --debuginfo` over the WASM, which rewrites the + WASM itself, and downloads esbuild 0.28.1 to bundle and minify the + JavaScript shim. The sources are in workers-rs `v0.8.5`, being + `worker-build/src/versions.rs` lines 11 and 12, `build/mod.rs` lines 309 + and 310, 455 to 470 and 558 and 559, `build/manifest.rs` lines 181 to 192 + and 299 to 305, and `main.rs` lines 123 and 124. The adapter's `Cargo.toml` + turns none of this off. `worker-build` also reads `CUSTOM_SHIM`, + `COREDUMP`, `RUN_TO_COMPLETION`, `COREDUMP_FLAGS` and `NO_MINIFY` + (`main.rs` lines 88, 100, 176, 237 and 363), and a `_BIN` path in + place of each tool it would download (`binary.rs` lines 22 to 24), and the + `.env` files that `build.sh` exports could set any of them. +4. **The config store's default name.** Core's `build.rs` reads the default id + of `[stores.config]` in `edgezero.toml` and passes it on (its `main`), and + `config_payload.rs` compiles it in as `DEFAULT_CONFIG_STORE_ID`. An + operator maps a store of their own to that name, so the repository's + `edgezero.toml` is the same for every publisher. +5. **The git version.** `TRUSTED_SERVER__GIT_VERSION`, which the deploy + pipeline sets or core's `build.rs` reads from local git + (`emit_git_version`), compiled in as `TS_GIT_VERSION` in `constants.rs`. +6. **The template build digest.** A SHA-256 that core's `build.rs` takes over + core's sources, its build script and `Cargo.toml`, and the workspace's + `Cargo.toml`, `Cargo.lock` and `edgezero.toml` (`template_build_digest`), + compiled in as `TEMPLATE_BUILD_DIGEST` in `publisher.rs`. It is part of the + fingerprint a stored page template is kept under, so a template stored by + other code is never served. + +No configuration, no policy and no key is among them at either row. + +**No build-time read takes configuration or a policy.** At both rows no build +script reads configuration or a policy into the build. The sample reaches only +test builds and core's `test-utils` feature (`SAMPLE_POLICY_TEXT` in +`permissions.rs`), and the permissions inspector, a tool of its own, which +embeds it on purpose. + +- Core's build script reads by name only `TRUSTED_SERVER__GIT_VERSION` + (`emit_git_version`), at both rows, besides the directories Cargo sets, the + files it hashes, `edgezero.toml` and local git. +- The JavaScript crate's build script reads by name only `TSJS_PREBUILT_DIR`, + `TSJS_SKIP_BUILD` and `TSJS_TEST`, at both rows, besides the directories + Cargo sets and the `PATH` it finds `npm` on. +- At row 55 the Fastly adapter has a build script as well, which writes the + list of core's Rust sources that a test of the adapter reads to check that + core never uses the Fastly SDK. Only a test build compiles that list in + (`core_sdk_guard`, which the adapter's `lib.rs` declares for tests alone), + and the script reads no variable by name beyond the directories Cargo sets. +- Vite runs with no configuration file and no `define` (`moduleBuildOptions` + in `lib/build-module-options.mjs`, at both rows), and no browser source + reads `import.meta.env` or `process.env`, the module crates' `js/src` + included at row 55, so no environment file reaches a bundle. +- EdgeZero's runtime crates read nothing while they compile, and no adapter + uses EdgeZero's `app!` macro, the one place where EdgeZero reads a file at + compile time (`edgezero-macros/src/app.rs`). +- Cloudflare's `build.sh` exports the root `.env` and the adapter's + `.env.cloudflare.dev`, neither of which is committed, into the build's + environment (lines 8 to 15 at both rows). The committed `.env.dev` and + `.env.example` hold `TRUSTED_SERVER__` settings, so such a file can carry + settings, but no build-time read takes one of them. + +**A check that no release WASM carries a policy.** +`scripts/check-no-compiled-policy.sh` reads each release WASM it is given and +fails when one carries text from a policy file in `config/permissions`, being +the file's first comment line or its `name:` value, which no code carries. It +skips either when it is shorter than 16 characters (`MIN_MARKER_LENGTH`), +because text that short, such as a bare `#`, could be in any WASM, and it +fails too when an artifact is missing or cannot be read, or when a policy file +has nothing long enough to look for. It reads the artifact, so it sees what +the linker kept rather than what the source says. At both rows CI runs it on +the Fastly and Spin release WASMs the test workflow builds, and on the +Cloudflare release WASM the integration workflow builds with `worker-build`, +in the steps named "Check that no permission policy is compiled into" each +WASM (`.github/workflows/test.yml` and +`.github/workflows/integration-tests.yml`). One more step of the test +workflow, "Check that the policy guard refuses a file that carries a policy", +runs the script on `config/permissions/sample.yaml` itself and fails the job +unless the script refuses that file with exit 1, so a guard that stops finding +the markers fails CI rather than passing every WASM. + +### 3.7 Cost of the fetch + +The policy is read when an instance loads a domain's settings, and what an +instance is differs by adapter. + +| Adapter | Where the policy is read from | What an instance is | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Fastly | Inside the blob at row 4, read with it, where the sample adds three to five reads (§3.1, item 3). From row 11, for a host's blob, the entry `permissions_`, being one read for the pointer and one for each chunk, so five for the 24,271-byte sample | One request, unless the adapter is built with the opt-in `reusable-sandbox` feature, when a sandbox serves several requests and, from row 11, keeps the applications of up to eight hosts | +| Cloudflare | Inside the blob, which the Worker reads from the Workers KV namespace with one awaited read from row 4 | A Worker instance, which builds its state for each request and serves requests until Cloudflare starts another | +| Spin | Inside the blob, in the `default` key-value store the settings come from, read with it | A component instance, which builds its state for each request and serves one request or several as Spin decides | +| Axum | `/.yaml` | The process, which loads its settings once when it starts | + +An instance takes the SHA-256 of the text on every load, and parses the text +the first time it meets the digest. An instance that already holds the digest +a blob names parses nothing, but it still reads the file and hashes it. The +runtime loader asks its source for the text before anything looks for a +parsed copy (`load_domain_policy`, `config_payload.rs` lines 131 to 171 at +IABTechLab/trusted-server#1352), and `load_policy` takes the SHA-256 of the +text it is handed before `parsed_policy` finds the copy the instance holds +(`permissions.rs`). So from row 11 a host's blob costs an instance that +already holds its digest the reads of its policy entry, through +`FastlyPolicyEntries::policy_text` (the Fastly adapter's `platform.rs`), +being the pointer and its chunks, five for the sample, and a hash of the +text, and no parse. Checking the per-digest cache before asking the source is +a change left for the measurement on Fastly staging that this section names. +The parsed copies stay until the instance ends, with no lifetime of their +own, because a digest never changes what it names. An instance keeps one +copy for each digest its domains' blobs pin, so the hundred hosts that name +three files in §3.1 cost an instance at most three copies while no file +changes, and each changed file adds a copy for the digest it brings. + +At row 4 a settings load parses a policy file only the first time the +instance meets its digest (`parsed_policy` in `permissions.rs`), so on Fastly, +where an instance serves one request unless the opt-in feature is built, a +request parses its domain's file once, where before row 4 it parsed the +compiled sample twice, once to check it and once on first use. What is new on +Fastly is the digest, taken on every load, and the three to five reads that a +blob carrying the sample adds, and from row 11, for a host's blob, five reads +of its policy entry on every load, whether or not the instance already holds +the digest. + +On Cloudflare, at row 4 (IABTechLab/trusted-server#1345), the Worker's fetch +entry point `main` (the Cloudflare adapter's `lib.rs`) calls `build_app` for +every request, and `load_startup_settings` (the adapter's `app.rs`) opens the +config store's KV namespace and awaits one read of the blob, which the +runtime loader loads with the policy inside it. So every request reads the +blob and hashes the policy text (`load_policy`), and the parsed policy stays +in the Worker instance by digest, in the `PARSED` map of `parsed_policy`, for +as long as Cloudflare keeps that instance, so a request that meets a held +digest parses nothing. Before row 4, `main` built the application for every +request as well, through EdgeZero's `run_app`, from the settings in the +`TRUSTED_SERVER_CONFIG` variable, so what row 4 adds to each request is the +one KV read and the hash. The adapter's guide, `docs/guide/cloudflare.md`, +states the per-request read in its opening paragraph. + +`template_fingerprint` serializes and hashes the typed settings on every +request that can use a shared page template, and leaves `permissions.text` out +of what it hashes, because `sha256` already names the file, so no request +hashes the file (`template_fingerprint` and its call in `publisher.rs`, lines +2116 to 2145 and 4881 at row 4). + +The numbers are not measured yet. A measurement on Fastly staging will record, +for the sample, the time and the compute an instance spends from reading a +domain's blob to holding its policy, beside the same for a scratch build that +compiles the sample in. It will decide whether Fastly also keeps a serialized +copy of the parsed policy in its value cache (`PlatformValueCache`, from row +24, which only the Fastly adapter registers), shared by every instance at a +location, so that a new instance decodes one cached value in place of reading +and parsing the file. The value cache holds bytes, so each instance would +still decode its own copy. The numbers will be filled in here. Viceroy, +Fastly's local test server, serves its config store from `fastly.toml` on the +local machine, so a local run cannot show the cost of a read from Fastly's +store. The Fastly integration tests' Viceroy configuration stores the blob as +chunks of at most 7,000 bytes and a pointer, as +`ts config push --adapter fastly` does, because Viceroy, like Fastly, accepts +at most 8,000 characters in one config store value (`config_store_entries` in +`crates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rs`, +lines 14 to 20 and 205 to 248). Cloudflare, Spin and Axum are not measured, +because the measurement decides only Fastly's value cache. From row 4 every +Cloudflare request waits for one Workers KV read, where main read a variable. + ## 4. Signal precedence (normative, implemented) Precedence is the **order a deployment configures**, not a rule in code. The @@ -592,8 +1225,8 @@ at `EcContext` construction (`storage_acquisition`), never a group label. The implemented trigger, exhaustively (nothing else withdraws): -1. **A TCF record refusing storage (Purpose 1) withdraws iff the baseline is - not `granted`, and only when the refusal is carried by the live +1. **A TCF record refusing storage (Purpose 1) withdraws only when the + baseline is not `granted`, and only when the refusal is carried by the live request.** Under a `requires_signal` (or `denied`) baseline the refusal is the visitor declining the very signal storage depends on, so it withdraws. Where the baseline is `granted`, storage never depended on @@ -614,10 +1247,11 @@ The implemented trigger, exhaustively (nothing else withdraws): (§4). Destruction requires an affirmative, decodable signal. 4. **Absence of signal never destroys identity.** A visitor who has not yet made a choice is never stripped of an existing identity. -5. **A policy change is not a user signal.** There are no runtime policy - edits (§3.1), and a rebuild that tightens a baseline does not itself - tombstone, because withdrawal still requires the affirmative refusal above on a - live request. +5. **A policy change is not a user signal.** A domain's policy changes only + when its settings are pushed again with another file or an edited one + (§3.1), and a push that tightens a baseline does not itself tombstone, + because withdrawal still requires the affirmative refusal above on a live + request. The draft's additional trigger, an explicit storage-withdrawal or authenticated deletion request honored in every jurisdiction, has no @@ -646,7 +1280,7 @@ grant of storage. ### 4.3 Withdrawal durability (largely deferred) -Implemented behavior (`ec/finalize.rs`): when the request carries the +Implemented behavior (`ec/finalize.rs`). When the request carries the withdrawal signal and the client presented a cookie, the response expires the EC cookie, and the identity-graph tombstone is written for each presented identifier the module accepts (the incoming cookie value and @@ -705,7 +1339,7 @@ raw strings that were read and used. The same list reaches the page as `signals` (§13). The draft's declared reordering, expiry filtering **before** conflict -resolution, was **not implemented**: conflict resolution still runs first, +resolution, was **not implemented**. Conflict resolution still runs first, so the pre-epic order stands. Recorded in §11. **Persisted-KV consent (not built).** The draft's fallback to the consent @@ -734,7 +1368,8 @@ questions, item 2, asks whether that is intended. ### 4.5 US signal field mapping -Implemented sources, as declared in the shipped `signals` section: +Implemented sources, as the repository's sample declares them in its +`signals` section: | Source | Effect | | -------------------------------------------------- | ---------------------------------------- | @@ -795,7 +1430,7 @@ mode does not affect it. | `non-marketing` | None | None | The word `non-marketing` is a recorded answer that changes no permission, so -the baseline stands. Under the shipped `gdpr-eu` and `gdpr-uk` groups the +the baseline stands. Under the sample's `gdpr-eu` and `gdpr-uk` groups the baseline grants no marketing Data Use, and under `us-opt-out` the baseline grants them. The module also: @@ -852,7 +1487,7 @@ case-insensitively. Implemented in ### 5.2 Lookup failure -Implemented, with the failure state carried explicitly: +Implemented, with the failure state carried explicitly. The type `GeoStatus { Located, NoLocation, Failed }` (in `ec/consent.rs`) separates "the module resolved no location" from "the lookup errored". A **failed** lookup resolves every permission to the **requires-signal floor** and its @@ -899,7 +1534,7 @@ a static constant, which is only honest when the operator can genuinely assert single-jurisdiction traffic. Constraint, implemented in -`GeoConfig::validate_jurisdiction_acknowledgment`: **startup fails** when +`GeoConfig::validate_jurisdiction_acknowledgment`. **Startup fails** when an Edge Cookie module is configured and no geo module is selected, unless the operator sets `[geo] assume_single_jurisdiction = true`. This makes the dangerous migration config (a permissive top `group` with @@ -935,23 +1570,30 @@ draft's `rules.default` policy entry does not exist either, so as "unresolved". The separation the draft treated as safety-critical is the one the implementation does keep, where a **failed** lookup never reaches the top node and resolves at the requires-signal floor instead (§5.2). -With no top `group` or no `jurisdiction` startup fails, and in the unreachable -belt-and-braces case where resolution still finds no node, the floor -applies. +With no top `group` or no `jurisdiction` startup fails, and if resolution +still finds no node, which should never happen, the floor applies. ### 5.5 Policy revision activation (deferred) -Not implemented, and currently moot. Policy is compiled into the binary -(§3.1), so the deployed artifact is the policy identity and there is no -runtime activation to coordinate. The draft's activation design, covering -the JCS-canonical policy digest and ordinal pair, the activation register -and candidate protocol, fleet readiness and quiescence, admission leases, -the hash-linked activation journal with store-clock retention, and the -model-epoch transition, is the reference design for the runtime-policy -follow-up recorded in §11. Its guarantee that mixed-revision irreversible -behavior is prohibited is honored today by construction, because a -deployment runs exactly one embedded policy and destructive withdrawal -requires a live user signal (§4.2), never a policy change. +Not implemented. A domain's policy changes when its blob is pushed with +another file or an edited one (§3.1), and the pushed blob, which carries the +file's name and SHA-256, is the policy identity. §3.1 also says what a +location answers while a blob, or from row 11 a policy entry, has not yet +reached it. Nothing coordinates the moment every instance has moved to a new +blob. While a push propagates, and for as long as an instance keeps a domain's +application, such as a Fastly sandbox built with `reusable-sandbox`, one +instance can run the previous policy while another runs the new one. The +draft's activation design, covering the JCS-canonical policy digest and +ordinal pair, the activation register and candidate protocol, fleet readiness +and quiescence, admission leases, the hash-linked activation journal with +store-clock retention, and the model-epoch transition, is the reference design +for ending that overlap, recorded in §11. Its guarantee that mixed-revision +irreversible behavior is prohibited is therefore not honored by construction. +Each instance runs one policy for a domain, the one its blob names, and +destructive withdrawal requires a live user signal (§4.2), never a policy +change. A push that changes a storage baseline can still let the same refusal +withdraw on an instance that runs one policy and not on an instance that runs +the other, until the overlap ends. ## 6. Failure-mode matrix (normative, implemented) @@ -964,12 +1606,25 @@ requires a live user signal (§4.2), never a policy change. | Top node missing `group` or `jurisdiction` | Startup failure (§3.3) | | EC module configured, no geo, no acknowledgment | Startup failure (§5.3) | | Module name the adapter does not link, or a name given twice | Startup failure when the adapter builds its application state, and not at settings validation (§3.3) | -| Malformed `permissions.yaml` | Parse error at settings load, once per instance, because the embedded file is a build-time constant, never per-request | +| Malformed `permissions.yaml` | Refused by `ts config push`, and by `ts config validate` when `--permissions` is given, and again at load, where the domain fails to load and answers 500 (§3.1, §3.3) | +| Settings name no policy, or a policy no supplied file has | Refused by `ts config push`, and by `ts config validate` when `--permissions` is given, and a blob pushed without a policy fails to load (§3.1) | +| Policy file absent, or its SHA-256 differs from the blob | The domain fails to load and answers 500, as it does when Axum cannot read the file from its folder (§3.1) | +| Blob names a policy with no `sha256` | The domain fails to load and answers 500 (§3.1) | +| Axum with `TRUSTED_SERVER_PERMISSIONS_DIR` unset | The domain fails to load and answers 500 (§3.1) | | Undecodable record present (TCF, GPP, or USP) | Read by the module for that scheme as the refusal or opt-out it may have carried, on the Data Uses that scheme covers, and never a withdrawal. Core answers nothing ahead of the modules, so the order decides against the other schemes (§4) | | Expired TCF record | Distinct state, not unreadable. The decoded record is cleared, the `tcf` module answers `Neutral`, and the baseline applies | | Signals contradict (opt-out plus consent) | The later module in the configured order decides, so under the default order a TCF consent amends an earlier opt-out and a PMP answer read by `mtm` amends both (§4) | | No EC module selected | Identity fails closed, with nothing created and an incoming cookie value never used or egressed (§7) | +A startup failure and a domain that fails to load have the same answer. The +startup router answers with the status of the error's kind, which is 500 for +every startup failure in the table, because each is a configuration error, as +a config store that cannot be opened or read is. The reason is logged at error +level, so nothing is served from the origin. Row 4 moves Spin from a fixed 503 +to the same rule, and Cloudflare and Spin log nothing until row 54 (§3.1, item +7). The rows on the policy file state the code of row 4, +IABTechLab/trusted-server#1345. + The posture is fail-closed. Every ambiguous state resolves to the configured baseline or more restrictive. @@ -994,16 +1649,24 @@ migration for `[ec]`, `[device]` and `[geo]`. For the permission model: names the new one. 5. Check the names in `modules` against the crates the adapter links. Only an adapter finds a wrong or repeated name, because settings validation and the - `ts` command line tool link none of the module crates. When an adapter cannot - build its state it serves its startup error router in place of the - application. + `ts` command line tool link none of the module crates. When an adapter + cannot build its state, every request answers with the status of the + error's kind, 500 for a wrong or repeated name, which is a configuration + error, and the reason is logged at error level, so nothing is served from + the origin. Row 4 moves Spin from a fixed 503 to the same rule, and + Cloudflare and Spin log the reason only from row 54 (§3.1, item 7). 6. Decide the order. The default order lets a TCF answer amend an opt-out, which is the opposite of the earlier fixed rule, and a deployment that wants the earlier outcome lists the opt-outs after `tcf` (§4). -7. Put the policy in the build. The build compiles in - `config/permissions/sample.yaml`, a sample for testing and evaluation, and - no other file or setting supplies policy on this row (§3.1). A deployment - replaces that file and rebuilds. +7. Name the policy and supply its file. `[permissions] policy` names the + publisher's file by its stem, and `ts config push --permissions ` + supplies it, as `ts config validate` and `ts config diff` take it (§3.1). + No policy ships with Trusted Server and no tool chooses one, so + `ts config migrate` names no policy in a document that has none and reports + the missing policy as an item left to do + (`2026-10-17-config-migrate-design.md`). A deployment that skips this step + does not start, with the reason "the configuration names no permission + policy" (`docs/guide/error-reference.md`). ## 7. Enforcement points @@ -1049,7 +1712,7 @@ Consumers of the resolved set in the implementation: | Pull sync (browser-request-scoped) | `ec_sharing_allowed`, from the live request resolution | | Batch sync (context-free S2S) | Authenticated, and withdrawn or missing rows are ineligible | | KV EID resolution for auctions | `ec_allowed`, then the EID pair gate on the result | - | Publisher navigation and page-bids `user.id` | `ec_sharing_allowed` (the storage plus personalised-ads pair) | + | Publisher navigation and page-bids `user.id` | `ec_sharing_allowed` (the storage plus personalized-ads pair) | With **no EC module configured**, identity fails closed, meaning the gate is closed rather than open by default (`ec_allowed` is false), so a cookie @@ -1141,7 +1804,7 @@ module crates, the page script, and groups, case-insensitive duplicate rule keys, unknown revoke keywords, the jurisdiction rules and places written under a region. No test covers a misspelled key in a node, or `us-state` under a country other than `US`. -- **Shipped-table coverage.** All 30 EU/EEA codes lock storage to +- **Sample coverage.** All 30 EU/EEA codes lock storage to requires-signal (§3.5). - **Vocabulary breadth.** A TCF record grants or revokes every one of the eleven mapped purposes, not only storage and personalized ads. @@ -1157,6 +1820,35 @@ module crates, the page script, and - **Page script.** `crates/trusted-server-js/lib/test/core/permissions.test.ts` covers how the snapshot, with its signals and terms, reaches page code in every order of arrival. +- **The policy file (row 4).** The tests in `policy.rs` show that the push + writes the digest and the exact text of the named file, that Axum and + validation leave the text out, that the digest is taken after the + environment overlay and that `--no-env` leaves the overlay out + (`the_digest_is_taken_after_the_environment_overlay`), and that the + settings handed to EdgeZero are removed when the command is done. They + refuse a policy no supplied file has, a digest or text the settings set, a + file that is not a `.yaml` policy in UTF-8 text, two files that supply one + policy, the example settings as + written, and a diff without the file when the settings name a policy, and + they check every supplied file whether or not it is named. The command line + tool's own tests show that `ts config validate` without the file says the + policy was not checked and that a push of the example settings is refused + (`crates/trusted-server-cli/tests/config_store_defaults.rs`). The tests in + `config_payload.rs` show that a domain naming no policy, a policy with no + digest, a blob without its file where the file travels in the blob, a file + that is not the pushed file, a pushed file that is not a valid policy and a + policy the source cannot read each fail to load, that only the policy's name + and digest stay in the settings, that a source supplies the file and any + text in the blob is unused, and that two loads that pin one digest parse the + file once. Each adapter's tests show each request given the policy its + domain loaded. Those of Fastly, Cloudflare and Spin show the startup router + answering a policy load error with 500, Axum's show that startup without the + pushed policy file answers 500 on every route, and Cloudflare's show that + each manifest binds the config store as a namespace of its own and carries + no settings in a variable. `the_loaded_policy_decides_and_not_the_sample` in + `ec/consent.rs`, `the_policy_file_counts_by_its_digest_and_not_its_text` in + `publisher.rs`, and the Viceroy configuration's tests, which load the + generated chunks through the runtime loader, complete the set. - **Not covered by a test.** The order between `mtm` and `tcf` or an opt-out in one request. A withdrawal beside an `mtm` grant of storage. The startup warning with the real module names. Proxy mode through `EcContext` and @@ -1176,15 +1868,16 @@ their deferred features. - The §4.3 durability protocol, §4.5 full field mapping and registry snapshot, §5.5 activation apparatus, §7.4 dispatch migration, and §7.1 - contextual projection: deferred follow-ups, recorded in §11 with the + contextual projection are deferred follow-ups, recorded in §11 with the 2026-07-31 draft as their reference design. Deferred, not rejected. -- Runtime policy configuration (a `[permissions]` config section): - deferred until the config push and activation pipeline exists (§3.1). +- Rules written into `[permissions]` itself are not adopted, because + `[permissions]` names the policy file, which keeps the YAML form a policy + owner reads (§3.1). - Per-signal jurisdiction scoping (honoring GPC only where a law defines - it): rejected, as in the draft. The opt-out signal layer is + it) is rejected, as in the draft. The opt-out signal layer is jurisdiction-free in the implementation, and the country baseline decides only what a revoke has to drop. -- An authenticated deletion or explicit storage-withdrawal endpoint: +- An authenticated deletion or explicit storage-withdrawal endpoint is deferred (§4.2). **Open questions.** Each of these waits for a decision and for a code change, @@ -1236,32 +1929,36 @@ acceptance contract, not two: | -------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Unmatched countries fall to `default_country` | Adopted, with a changed mechanism since 2026-09-01. Unmatched and unresolved requests both fall to the required top node of the `rules:` tree, and a **failed** lookup floors instead (§5, §12) | The failure state is the one that must never reach a permissive default, and the draft's `rules.default` split was not kept | | The full TCF purpose vocabulary is modeled | Adopted and extended. All eleven purposes are signal-resolved, and the full Privacy Taxonomy is carried as declared baseline (§2) | The joint taxonomy work made whole-taxonomy declaration the goal, and `denied` defaults keep undeclared uses inert | -| Policy is an embedded file | Adopted. `permissions.yaml` is compiled into the build (§3.1), and runtime configuration is deferred follow-up | The runtime push and activation pipeline does not exist, and version control is the audit trail meanwhile | +| The legal policy is the deployer's to set and is not baked into Trusted Server | Adopted. The policy is the publisher's file, selected and supplied when the service is deployed, and never compiled into a build (§3.1) | Agreed with IAB Tech Lab on 2026-10-23, so that one released binary serves every publisher and no domain starts without a valid policy file | +| A default deployment needs no policy interaction at all | Not adopted. Every domain names a policy file, which the push supplies, because no policy ships with Trusted Server and a domain without a valid file does not start (§3.1) | Agreed with IAB Tech Lab on 2026-10-23, so that a binary carries no policy to fall back on | | Permission sources are open-ended (#777: publisher interaction, external services may grant) | Adopted as a seam. A source is a permission signal module, a crate implementing `PermissionSignalModule`, listed by each adapter in `shipped_signal_modules()` and asked in the order `[permission-signal] modules` gives (§1, §4) | The five shipped schemes already use it from outside core, so the interface has real consumers, and publisher interaction or an external service arrives as another module | ## 11. Revision record vs the 2026-07-31 draft -One row per divergence between the draft and the implementation this -revision was verified against (branch `split/3-permissions` at `a028c00da`, -row 4 of the stack). +One row per divergence between the draft and the implementation this revision +was verified against (branch `split/3-permissions` at `a028c00da`, row 4 of +the stack, which pins EdgeZero at `683202c`, where §3.1 cites +EdgeZero `v0.0.10`, the version the stack pins now). The rows on where the +policy lives state the code of row 4, IABTechLab/trusted-server#1345 (§3.1), +and the row on its activation records the deferral of §5.5. | Draft position | Implemented position | Why | | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Vocabulary is two TCF-purpose identifiers, enforced permissions only (§2) | IAB Privacy Taxonomy Data Uses, with eleven named purposes all signal-resolved, plus 53 further taxonomy Data Uses that no module gates on and that a signal reaches for eleven of them under the sample policy (§2) | The joint taxonomy adoption postdates the draft, and whole-taxonomy declaration serves completeness and demonstration, with `denied` defaults keeping unenforced flags inert | -| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Policy is `permissions.yaml`, compiled into the build with `include_str!`, parsed once and covered by tests | The runtime config push and activation apparatus does not exist, so publishing runtime policy without it would recreate the hazards the draft cataloged. Runtime policy is deferred follow-up | +| Policy lives in `[permissions]` in `trusted-server.toml`, published via `ts config push` (§3.1) | Adopted in part. `[permissions] policy` names the publisher's `permissions.yaml`, which keeps its YAML form and is pinned in each blob by its SHA-256, and Fastly stores each file once from row 11. `ts config push`, `validate` and `diff` take it with `--permissions` (§3.1) | The draft asked for `[permissions]` so that policy is validated, compared and pushed with the settings, and the file it names gets all three while keeping the YAML form a policy owner reads | | Every group carries a required `regime` class read by auction dispatch (§3.2) | No regime field exists | Its only consumer, regime-gated dispatch, was not migrated, and the field returns with that work | | Overrides name explicit acquisition rules, replacing the `+`/`-` sigils (§3.2) | Adopted. A detailed rule's `permissions` map assigns `granted`, `requires_signal`, or `denied` per Data Use | The draft's requirement, expressed in the YAML schema. `requires_signal` is now expressible per rule | | Two fallbacks: policy `rules.default` for unmatched countries, `default_country` only for the static no-geo mode (§5.4) | One required fallback covers unmatched and unresolved requests in every mode, startup-validated, and a failed lookup floors separately. It was `[geo] default_country` until 2026-09-01 and is now the top node of the `rules:` tree (§12) | One deployer knob is simpler, and the safety-critical separation kept is failure vs absence, carried by `GeoStatus` | -| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and the jurisdiction values. The parser does not reject an unknown key in a node (§3.1) | Smaller surface shipped first, and the EU/EEA coverage test guards the shipped table against the typo class. ISO-assignment checks are future hardening | -| Three-class signal taxonomy with regime-scoped grant acceptance, where the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | Of the shipped modules TCF and `mtm` are the grant sources, the US posture is a `granted` baseline that opt-outs revoke, and explicit non-opt-out values grant nothing | A simpler two-signal model without regimes. The cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the shipped file | +| Validation checks assigned ISO 3166-1 codes, assigned subdivisions, and a group identifier grammar (§3.3) | Validation covers unknown groups, permissions, acquisitions, revoke rules, incomplete groups, case-duplicate rule keys, and the jurisdiction values. The parser does not reject an unknown key in a node (§3.1) | A smaller surface shipped first, and the EU/EEA test guards the sample against typos. ISO checks are future hardening, and matter more now that a publisher's own file has no such test | +| Three-class signal taxonomy with regime-scoped grant acceptance, where the US posture is `requires_signal` with GPP/USP non-opt-out values as grants (§4) | Of the shipped modules TCF and `mtm` are the grant sources, the US posture is a `granted` baseline that opt-outs revoke, and explicit non-opt-out values grant nothing | A simpler two-signal model without regimes. The cost, no-signal US traffic is allowed by baseline rather than blocked pending a signal, is a deliberate policy choice in the sample | | Malformed-present blocks grants per record family and mapped section (§4.4, §4.5) | Each module answers for its own scheme's unreadable record, as a refusal or opt-out on the Data Uses that scheme covers, and core answers nothing ahead of the modules (§4) | The scope is the Data Uses of the scheme whose record could not be read, because per-section scoping needs the full §4.5 decoder work | | Normalization runs expiry before conflict resolution, a declared change (§4.4) | Conflict resolution still runs before the expiry check | The reordering was not implemented, though the expired state itself (distinct from unreadable, absent for acquisition) was adopted | | Persisted-KV consent flows through the full normalization pipeline with an explicit TTL comparison (§4.4) | Not built. No persisted-KV consent exists on this row, the consent pipeline reads the live request only, and `consent_store` is refused (§4.4) | The legacy consent store was removed (IABTechLab/trusted-server#883), so live consent is interpreted on each request | | Proxy mode gains minimal opt-out extraction (§4.4) | Proxy mode still skips decoding, so a present TC, GPP or US Privacy string is unreadable to its module and revokes the Data Uses that scheme covers, and the GPC header opt-out is honored without decoding. By reading, no raw string reaches the auction and publisher paths (§4.4) | The permission-layer outcome is equally or more restrictive with no new decode paths, and this is revisited with the §4.5 decoder work | | Withdrawal has four triggers including an explicit storage-withdrawal or authenticated deletion request (§4.2) | The TCF Purpose 1 refusal under a non-granted baseline is the only trigger, and opt-outs, unreadable records, absence, and policy changes never withdraw (adopted) | No deletion endpoint exists to carry the extra trigger, so the narrowest destructive surface shipped first | | §4.3 durability protocol: family records first, suppression and authority-state records, outbox, breaker, strong reads | Cookie expiry plus best-effort identity-graph tombstones per presented identifier, with failures logged | The protocol requires storage primitives (linearizable CAS, independent durability domains) the adapters do not yet qualify, so it is deferred with the providers-spec storage work | -| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out. The revoke set is policy-declared, and the shipped file lists device storage, sale, sharing, third-party sale, targeted advertising and profiling (§4.5) | The full decoder and registry vendoring are their own project, and the policy-declared revoke set gives deployers the scoping lever meanwhile | -| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists, and the built binary is the policy identity | With no runtime policy there is nothing to activate, and the draft remains the reference design for the runtime-config follow-up | +| §4.5 field mapping and §4.5.1 vendored registry snapshot (sharing/targeted opt-outs, embedded GPC, applicability, derived `gpp_sid`) | Opt-out sources are the GPC header, a GPP sale opt-out, and a USP sale opt-out. The revoke set is policy-declared, and the repository's sample lists device storage, sale, sharing, third-party sale, targeted advertising and profiling (§4.5) | The full decoder and registry vendoring are their own project, and the policy-declared revoke set gives deployers the scoping lever meanwhile | +| §5.5 activation: JCS policy digests, ordinals, activation register, journal, drains, admission leases | None of it exists. Each blob names its policy file by name and SHA-256, so the pushed blob is the policy identity (§5.5) | A push reaches instances as any settings push does, and the draft remains the reference design for moving every instance at once | | §3.4 single jurisdiction truth, and §7 dispatch gated on the policy regime with a contextual projection | Half adopted. Since 2026-09-01 the `rules:` tree carries regime applicability and the two `[consent]` lists retire into it (§3.4), while auction dispatch still keeps the consent-subsystem gate (effective TCF Purpose 1 for GDPR or unknown jurisdictions), with no contextual view | Dispatch migration is follow-up. The legacy-list drift risk the draft named still stands and is recorded rather than resolved | | Every raw-EC egress path is pair-gated, with per-row tests and a denylist check (§7) | Pair gating is centralized in `ec_sharing_allowed` (auction endpoint `user.id`, publisher navigation and page-bids `user.id`, identify, pull sync) and `gate_eids_by_permissions` (EIDs everywhere). Batch sync checks row state only | Partial adoption. Aligning the remaining paths, the S2S stored-provenance authority, and the inventory tests is recorded follow-up | | Identity rows never store raw consent strings, only normalized provenance and a digest (§1) | The identity-graph entry stores the raw TCF and GPP strings that a module vouched for with the row | The normalized provenance schema belongs to the providers-spec storage work, and until then rows carry the raw strings | @@ -1286,7 +1983,7 @@ change. | Startup rejects a missing or unresolvable `default_country` | Startup rejects a top node missing `group` or `jurisdiction`, and reserved keys cannot collide with place codes because ISO codes are short | The same loud failure, moved to where the policy now lives | | `jurisdiction` is written at the top of the tree only | Any node may carry `jurisdiction:` beside `group:`, and a node carrying none inherits the nearest ancestor that does. The top node must carry both, so the inheritance always terminates | A country or a state often needs its own regime, and inheritance states the common case once while leaving the exceptions visible where a reader meets them | | The US-state value carries the state code, as `us-state/` | The value is bare `us-state`, valid only on a region node, and rejected at the top and on a country node. The parser does not check the country above the region (§3.2) | The matched node already names the state, so repeating the code in the value invites the two disagreeing | -| `consent.gdpr.applies_in` and `consent.us_states.privacy_states` in the operator TOML carry regime applicability | Both retire into the tree, where the shipped file holds the same 31 GDPR countries inheriting `gdpr` and the same 20 privacy-law states as region children of `US` with `jurisdiction: us-state` (§3.4) | One document carries the baselines and the applicability together, so a policy owner keeps one list rather than two in step, and the drift risk §3.4 recorded has nothing left to drift against | +| `consent.gdpr.applies_in` and `consent.us_states.privacy_states` in the operator TOML carry regime applicability | Both retire into the tree, where the sample file holds the same 31 GDPR countries inheriting `gdpr` and the same 20 privacy-law states as region children of `US`, with `jurisdiction: us-state` (§3.4) | One document carries the baselines and the applicability together, so a policy owner keeps one list rather than two in step, and the drift risk §3.4 recorded has nothing left to drift against | This revision leaves unchanged the group vocabulary and flags, the signal precedence of §4, and the requires-signal floor for a failed geo lookup, which @@ -1326,6 +2023,7 @@ include the changes that followed the first revision of this section on | 2026-10-07 | A module is named by its crate folder, so the selector reads `[permission-signal] modules`. | | 2026-10-15 | Fixes from review round 1. An unreadable record answered by its own module with nothing ahead of the modules, the fifth module `mtm` with its own section, registration through each adapter, the four members of the page state, the sharing pair on navigation and page-bids, the persisted consent, resolve endpoint and device module check named as built by other rows or not at all, what the policy parser does not reject, the Data Uses a signal reaches, the raw strings removed after assembly, a table of what each adapter runs, the upgrade list, the open questions and the review log. | | 2026-10-16 | Review round 2, approved with one minor point recorded in the Review log. | +| 2026-10-26 | The policy file is never compiled into a build, as agreed with IAB Tech Lab at the Touchbase on 2026-10-23, and is selected and supplied when a service is deployed, as decided by 51Degrees after the Touchbase (§3.1). What row 4 builds and later rows add, Cloudflare reading its blob from the config store, the push, answers that follow the error's kind, and a blob's propagation (§3.1). What the build carries (§3.6), the cost of the fetch (§3.7), the failure modes (§6), the upgrade list (§6.1), the tests (§8), the records in §10 and §11, and the sample named as the sample. | ## Review log diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index b1f7e7d6cc..4ada6c2731 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -1,26 +1,28 @@ # Design Spec: Module and Permission Model, Migration and Rollout -**Status:** Proposed. The implementation is carried by the series' draft PRs -#1343 to #1347 (rows 2 to 6 of the stack in #777), none of which is yet merged -to main, and this document was revised against that series on 2026-08-25. The -§8 sign-off rows remain the series' -decision ledger, and rows the implementation now satisfies are marked with -their PR so the task force can ratify rather than re-litigate. Updated on -2026-09-01, where `[geo] default_country` is retired and the fallback baseline -for a request with no resolvable country moves to the top node of the `rules:` -tree in `permissions.yaml`, where `jurisdiction` is a per-node inherited -attribute and the `consent.gdpr.applies_in` and -`consent.us_states.privacy_states` lists retire into the same tree -(permission-model spec §3.2, §3.4 and §12). Updated on 2026-10-06 to say -module where it said provider, with `provider` kept for the two auction -selectors (pluggable spec §2.1). +**Status:** Proposed. The implementation is carried by the stack's pull +requests #1343 to #1347 (rows 2 to 6 of the stack in #777), none of which is +yet merged to main, and this document was revised against that series on +2026-08-25. The §8 sign-off rows remain the series' decision ledger, and +rows the implementation now satisfies are marked with their PR so the task +force can ratify rather than re-litigate. Updated on 2026-09-01, where +`[geo] default_country` is retired and the fallback baseline for a request +with no resolvable country moves to the top node of the `rules:` tree in +`permissions.yaml`, where `jurisdiction` is a per-node inherited attribute +and the `consent.gdpr.applies_in` and `consent.us_states.privacy_states` +lists retire into the same tree (permission-model spec §3.2, §3.4 and §12). +Updated on 2026-10-06 to say module where it said provider, with `provider` +kept for the two auction selectors (pluggable spec §2.1). Updated on +2026-10-26, where the permission policy is the publisher's file, selected +and supplied when a service is deployed and never compiled into a build +(permission-model spec §3.1), which row 4 of the stack (#1345) builds. **Author:** Engineering **Issue references:** #777-#781 (epic) **Related specs:** `2026-07-30-pluggable-providers-design.md`, `2026-07-30-permission-model-design.md`, `2026-07-30-integration-response-header-hook-design.md` (not implemented in the series, see its status) -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-26 > **Context.** The module/permission epic is a breaking change to a live > identity system. PR #838's review showed that the riskiest part of such a @@ -68,23 +70,27 @@ integration module seam spec on top of the series and changes no code: Whether the host-signal surface stays is an **open review question** (sign-off row 22), not a settled decision. 3. **PR #1045, the permission model.** Permission names follow the IAB - Privacy Taxonomy Data Uses (`permissions.yaml`, compiled into the - build). Signal precedence is fixed in code, most restrictive first, - meaning an opt-out (Sec-GPC, GPP sale opt-out, US Privacy) suppresses - the Data Uses the policy revokes even against a consenting TCF record, - a present but undecodable record blocks baseline grants (fail-closed), - and only then does a TCF record decide its mapped Data Uses. + Privacy Taxonomy Data Uses (`permissions.yaml`, which that pull request + compiled into the build, and which the permission-model spec now has + selected when a service is deployed, §3.1). Signal precedence is fixed + in code, most restrictive first, meaning an opt-out (Sec-GPC, GPP sale + opt-out, US Privacy) suppresses the Data Uses the policy revokes even + against a consenting TCF record, a present but undecodable record + blocks baseline grants (fail-closed), and only then does a TCF record + decide its mapped Data Uses. Row 4 of the stack (#1345) replaces that + fixed order with the order the deployment configures in + `[permission-signal] modules` (permission-model spec §4 and §13). Destructive withdrawal is narrow, where only a TCF record refusing storage in a jurisdiction whose baseline did not grant storage expires the cookie and writes the identity-graph tombstone, and opt-outs never destroy an issued identifier. Sharing beyond the edge (bidstream `user.id`, the identify response, partner pull sync) requires storage - plus personalised-ad selection, the same pair that gates bidstream + plus personalized-ad selection, the same pair that gates bidstream EIDs. A fallback baseline for a request with no resolvable country becomes required, shipped as `[geo] default_country` and carried by the - rules tree's top node since 2026-09-01, a failed geo lookup - resolves at the requires-signal floor instead of that baseline, and a - no-geo deployment running an Edge Cookie module must set + rules tree's top node since 2026-09-01, a failed geo lookup resolves at + the requires-signal floor instead of that baseline, and a no-geo + deployment running an Edge Cookie module must set `[geo] assume_single_jurisdiction = true`. 4. **PR #1046, the hardened client-cycle resolve endpoint.** `POST /_ts/api/v1/ec/resolve` enforces publisher-origin `Origin` (403), @@ -132,33 +138,35 @@ for the follow-up work that will implement them. | --- | -------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | EU/GDPR request, no TCF consent → no EC | Same (opt-in baseline) | Preserved. Implemented (#1045), where `gdpr-eu` requires a signal for storage, with an EU-27 plus EEA coverage test | | 2 | US-state request, sale opt-out → no EC, existing EC expired and tombstoned even beside a consenting TCF string | Opt-out suppresses use but does not delete the identity. Lifting the opt-out restores the identity. The positive contextual-auction projection (permission §7's `ContextualAuctionView`) remains the design for what may still dispatch | **Changed deliberately.** The separation is implemented (#1045), with opt-outs stripping egress and never tombstoning. The contextual projection is not implemented (sign-off 4 open), and today EIDs and `user.id` are withheld | -| 3 | US-state request, no signals at all → no EC (fail-closed) | The draft's recipe kept this via a `requires_signal` US rule with an extended grant-signal class | **Changed by the shipped default (#1045), flagged.** `permissions.yaml` ships US as a `granted` baseline whose uses drop on any opt-out, so a no-signal US request receives an EC. A deployer edits the yaml to differ | +| 3 | US-state request, no signals at all → no EC (fail-closed) | The draft's recipe kept this via a `requires_signal` US rule with an extended grant-signal class | **Changed by the repository's sample (#1045), flagged.** It gives US a `granted` baseline that opt-outs revoke, so a no-signal US request gets an EC. From row 4 no build carries the sample, so each publisher's file decides | | 3a | US-state request, explicit not-opted-out GPP/USP value → EC allowed | No US signal grants anything. N/A, absent, reserved, and unknown values grant nothing | Implemented (#1045) in a stricter form than drafted, where signals only revoke, so the drafted "explicit not-opted-out may grant" class does not exist (sign-offs 3, 17) | -| 3b | US-state request, TCF record refusing Purpose 1, no US opt-out → no EC | Refusal beats coexisting non-TCF grant signals | Implemented (#1045), where an authoritative TCF refusal revokes its mapped uses. Under the shipped `granted` US baseline the refusal suppresses without tombstoning | +| 3b | US-state request, TCF record refusing Purpose 1, no US opt-out → no EC | Refusal beats coexisting non-TCF grant signals | Implemented (#1045), where an authoritative TCF refusal revokes its mapped uses. Under the sample's `granted` US baseline the refusal suppresses without tombstoning | | 3c | Consent-record conflict modes, expiry, KV fallback, proxy mode | Per the permission spec §4.4 matrix, whose changed row is that malformed-present blocks acquisition | Malformed-present fail-closed is implemented (#1045). The rest of the consent normalization pipeline is carried forward unchanged by the series | | 3d | Valid plus expired consent records, where conflict resolution can select the expired record | Expired sources drop before conflict resolution | Open. Not addressed by the series | -| 3e | Only the GPP sale field (and USP) is consulted, with sharing/targeted opt-outs ignored | Sale, sharing, and targeted-advertising opt-outs deny the personalised-ads uses, and none affects storage or destroys identity | Partially implemented (#1045), where sale, USP, and Sec-GPC are honored and never destructive. The sharing and targeted-advertising GPP fields are not yet decoded (sign-off 3 open) | -| 3f | Non-privacy-state US traffic (for example Wyoming) is non-regulated → EC allowed | Country-level `US` is a protective floor, region rules may be stricter, and regionless traffic never degrades to non-regulated | Implemented (#1045), where `permissions.yaml` maps country `US` to `us-opt-out`, with state rows able to override | +| 3e | Only the GPP sale field (and USP) is consulted, with sharing/targeted opt-outs ignored | Sale, sharing, and targeted-advertising opt-outs deny the personalized-ads uses, and none affects storage or destroys identity | Partially implemented (#1045), where sale, USP, and Sec-GPC are honored and never destructive. The sharing and targeted-advertising GPP fields are not yet decoded (sign-off 3 open) | +| 3f | Non-privacy-state US traffic (for example Wyoming) is non-regulated → EC allowed | Country-level `US` is a protective floor, region rules may be stricter, and regionless traffic never degrades to non-regulated | Implemented (#1045), where the repository's sample maps country `US` to `us-opt-out`, with state rows able to override. From row 4 no build carries the sample, so each publisher's file decides | | 3g | Graph rows persist JA4 class, H2 signal hash, and buyer-facing quality metadata | The draft discontinued these for new rows | **Contradicted by the series, flagged.** Rows still persist device signals, including signal hash prefixes, when the opt-in modules run (#1044/#1046). Tied to the open host-signal question (sign-off 22) | -| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB with citation and sign-off | **The shipped default adopts `granted` storage for GB (#1045), flagged.** The citation and sign-off the draft required do not exist yet. The task force owns this row | +| 4 | UK request, no TCF record → no EC | Same, unless the policy deliberately adopts a `granted` storage baseline for GB with citation and sign-off | **The repository's sample adopts `granted` storage for GB (#1045), flagged.** The citation and sign-off the draft required do not exist yet. The task force owns this row | | 5 | No country resolvable (geo failure) → no EC (fail-closed) | Protective failure profile, where permissions resolve at the requires-signal floor and `default_country` is reserved for unmatched requests in acknowledged static-jurisdiction mode | Implemented (#1045), where a failed lookup resolves at the requires-signal floor, logged at error level, and never falls back to `default_country` (sign-off 18) | | 6 | Non-regulated country, TCF record refusing Purpose 1 → EC still created, identity never tombstoned | Refusal blocks new grants everywhere, and existing identity is never tombstoned where the baseline is `granted` | Implemented (#1045), where an authoritative refusal revokes its mapped uses everywhere and withdrawal is scoped to non-granted baselines | | 7 | Country resolved but in no regulation list → EC created, EIDs pass through | Governed by the deployment's default rule. The implementation expresses this as the required `[geo] default_country`, naming the `permissions.yaml` rule for unmatched requests | Implemented (#1045) with a changed mechanism, since no `rules.default` entry exists and the fallback is required and validated at startup. Restructured on 2026-09-01, where the fallback is the top node of the `rules:` tree | | 8 | Opt-out signal outside US states → ignored today | Mapped use restrictions are honored globally, and opt-outs never tombstone identity | Implemented (#1045), where the signal mapping is jurisdiction-free and suppresses even TCF-consented uses, without destruction (sign-off 1) | | 9 | Fastly bot gate requires JA4 plus platform class before KV-backed EC writes | The draft deferred host signal processing and startup-failed `[device] module = "fastly"` | **Contradicted by the series, flagged.** The `fastly` device module ships opt-in with `builtin` (UA-only) as the default (#1044). Whether the host-signal surface stays is sign-off 22, open | | 10 | Fastly always resolves geo per request | Only with `[geo] module = "platform"`. The neutral default flips only together with the permission model's jurisdiction guard, never in an intermediate step | Implemented as sequenced (#1044 kept the platform default, and #1045 flipped geo off by default together with `default_country` and the acknowledgment guard) | -| 11a | Raw EC egress on jurisdiction-gated paths today (`user.id`, EIDs, identify, pull sync) | Gated by the sharing pair (storage plus personalised-ads), at least as strict as today for every path | Implemented (#1045), where `ec_sharing_allowed` gates `user.id`, the identify response, and pull sync, and `gate_eids_by_permissions` gates EIDs, all on the same pair | +| 11a | Raw EC egress on jurisdiction-gated paths today (`user.id`, EIDs, identify, pull sync) | Gated by the sharing pair (storage plus personalized-ads), at least as strict as today for every path | Implemented (#1045), where `ec_sharing_allowed` gates `user.id`, the identify response, and pull sync, and `gate_eids_by_permissions` gates EIDs, all on the same pair | | 11b | Proxy / click / Testlight forwarding extract the raw EC cookie/header without today's jurisdiction gate | Gated by the egress inventory (both purposes) | Open. Not implemented by the series (sign-off 8) | | 11c | Batch sync only authenticates the S2S caller and checks row state | Gated by stored-provenance recompute, with legacy rows failing closed until backfilled | Open. Not implemented. Batch sync still checks caller and row state, with missing or withdrawn rows collapsing to ineligible (sign-offs 7, 13, 25) | | 12 | EC generation succeeds without a configured identity-graph store | The draft required an openable graph store at startup for a creating module | **Changed from the draft (#1043/#1046).** No startup requirement ships. The implemented rule is per request, where no available graph means no identifier is created (no phantom cookies), organic and resolve paths alike | | 13 | Cookies created by graphless deployments have no graph row | Rowless proof under the graphless-migration flag, capped `w` records, disclosed cookie-only expiry | Open. None of the rowless machinery is implemented. A cookie without a row is simply never shared (sign-offs 21, 29, 30) | -Rows 3, 4, and 7 are policy decisions, not code decisions. In the -implemented series they live in `permissions.yaml`, which is compiled into -the build precisely so the policy stays visible and reviewable in version -control, made explicitly by maintainers rather than implied by an -implementation. The shipped defaults for rows 3 and 4 are flagged above for -the task force. +Rows 3, 4, and 7 are policy decisions, not code decisions. They live in the +publisher's `permissions.yaml`, which a domain's settings name and which is +supplied when the service is deployed (permission-model spec §3.1), so the +policy stays visible and reviewable as a file of its own, made explicitly by +the publisher rather than implied by an implementation. No policy ships with +Trusted Server, so nothing decides these rows for a publisher, and the +values the repository's sample gives rows 3 and 4 are flagged above for the +task force. ## 3. Identity stability guarantee @@ -230,6 +238,14 @@ Requirements, each marked with its implementation state: release that rejects `[ec] passphrase` outright with a message naming the new location is scheduled, not yet in the tree. + The permission policy has no such dual-read release. A build with + permission-model spec §3.1, which row 4 of the stack builds, starts no + domain whose blob names no policy, and a build without it, as main is, + refuses `[permissions]` as an unknown field (`deny_unknown_fields` on + `Settings`, main's `settings.rs` line 2964), so no one blob starts both + builds. The build and the blob that names its policy therefore go live + together, and on Fastly the pair is staged first (§6 item 1). + The draft's full N+1/N+2 interim (the negative-record read/write matrix, the `permissions_v2` model promotion, the `m00` mirror, and the rollback-floor machinery) is **deferred** together with the durable @@ -291,13 +307,18 @@ Requirements, each marked with its implementation state: example ships Edge Cookie identity off by default, with each selector and its block documented together and commented together, and the fallback baseline stated in `permissions.yaml` rather than left to the - reader. The draft wanted the happy path - uncommented. The series instead closes PR #838's silent-stateless trap - by validation (requirement 6), so a half-uncommented configuration - refuses to start rather than running stateless. Static-geo examples - pair the rules tree's top node with the commented - `assume_single_jurisdiction` acknowledgment, which startup enforces - whenever an EC module runs with no geo module (#1045). + reader. The draft wanted the happy path uncommented. The series instead + closes PR #838's silent-stateless trap by validation (requirement 6), so + a half-uncommented configuration refuses to start rather than running + stateless. Static-geo examples pair the rules tree's top node with the + commented `assume_single_jurisdiction` acknowledgment, which startup + enforces whenever an EC module runs with no geo module (#1045). From row + 4 of the stack the example names no policy, and a comment tells the + operator to name their own file in `[permissions] policy`, because no + tool chooses a policy. `ts config init` writes the example as the + operator's settings, and a push of it as written is refused with a + message naming what is missing (`trusted-server.example.toml` lines 223 + to 235 at row 4). 10. Every misconfiguration in the providers spec §6 table fails at **startup**. Implemented for configuration errors (#1043-#1045: selector/block mismatches, unknown keys, unknown selector values, @@ -311,12 +332,15 @@ Requirements, each marked with its implementation state: 11. Validation is split into two named layers. Partially implemented. Structural validation runs at `ts config push` / `ts config validate` (the CLI deserializes and validates the typed settings, with a - regression test covering environment overlays) and again at startup, - and startup additionally validates deployment facts only startup can - see (module availability in the build, the top node of the rules tree - in the compiled `permissions.yaml`, the acknowledgment rule). The - machine-readable adapter capability profile for push-time deployment - pre-checks is deferred with requirement 2. + regression test covering environment overlays) and again at startup, and + startup additionally validates deployment facts only startup can see + (module availability in the build, the acknowledgment rule). From row 4 + of the stack the policy file the settings name, the top node of its + rules tree included, is validated at both, at push against the files + `--permissions` supplies and at load against the file the service reads + (permission-model spec §3.1 and §3.3). The machine-readable adapter + capability profile for push-time deployment pre-checks is deferred with + requirement 2. ## 5. Minimal-divergence migration recipe (operator-facing) @@ -325,46 +349,64 @@ name says so. The divergences that actually shipped in the series, each a matrix row: global honoring of mapped opt-outs (row 8), an authoritative TCF refusal blocking new grants everywhere (row 6), the protective geo-failure floor (row 5), country-wide protective US handling (row 3f), -malformed-present blocking acquisition (row 3c), and the shipped `granted` +malformed-present blocking acquisition (row 3c), and the sample's `granted` US and GB storage baselines (rows 3 and 4, flagged for ratification). Divergences the draft listed that have **not** shipped and remain deferred: the sharing/targeted GPP fields (row 3e), the grant-signal class (row 3a, now stricter instead), and the batch-sync provenance gate (row 11c). -Policy now lives in a permissions YAML document (the repository sample is -`config/permissions/sample.yaml`, for testing and evaluation only), -compiled into the build. There is -no `[permissions]` block in `trusted-server.toml`, so the draft's +Policy lives in the publisher's permissions YAML document (the repository +sample is `config/permissions/sample.yaml`, for testing and evaluation +only), which `[permissions] policy` in the settings names and +`ts config push --permissions` supplies when the service is deployed, and +no build carries it. On Axum the file sits in the folder +`TRUSTED_SERVER_PERMISSIONS_DIR` names. Row 4 of the stack builds that +design (permission-model spec §3.1). In the document the operator writes, +`[permissions]` holds only that name, never rules, so the draft's partial-policy trap (a TOML table containing only a permissive default) -cannot be written at all. A deployer edits the yaml and rebuilds, which -keeps every policy change reviewable in version control. The shipped -default policy maps the EU-27 plus the EEA states to `gdpr-eu` (a signal -required for every modeled use), GB to `gdpr-uk` (storage `granted`, -flagged in row 4), and US and AU to `us-opt-out` (a `granted` baseline -where all granted uses drop on any opt-out signal), leaves every unmodeled -Data Use `denied`, and sends unmatched requests to the group named at the top -of the required rules tree. +still cannot be written. A blob is the settings as `ts config push` stores +them, and the push adds to the blob's `[permissions]` the file's SHA-256 +and, where the policy travels inside the blob, the file's text. A deployer +edits the yaml and pushes the settings again, which keeps every policy +change reviewable in the file's own history, and each blob pins the exact +file it runs by its SHA-256. The repository's sample maps the EU-27 plus +the EEA states to `gdpr-eu`, which needs a signal for each of the eleven +named Data Uses and grants 23 of the 64 Data Uses without one, GB to +`gdpr-uk` (storage `granted`, flagged in §2 row 4), and US and AU to +`us-opt-out`, which grants 45 of the 64, the eleven named ones among them, +until an opt-out revokes the seven Data Uses its revoke list names. It +sends unmatched requests to the group named at the top of the required +rules tree. The migrated operator configuration is: - `[ec] module = "hmac"` with its `[ec.hmac]` table naming the secret that holds the 32-plus character passphrase, carried over verbatim for - identity stability (§3); + identity stability (§3). - `[device]` left at the `builtin` default, with `fastly` as the opt-in - documented alongside the open sign-off 22 question; -- `[geo] module = "platform"` where the adapter supplies a host lookup - (Fastly and Cloudflare in production, the Axum dev server in - development, while Spin resolves nothing either way). A selected - module's lookup failure resolves at the requires-signal floor and - never at the fallback baseline. A static deployment instead states its - group at the top of the rules tree together with - `assume_single_jurisdiction = true`; + documented alongside the open sign-off 22 question. +- `[geo] module = "platform"` where the adapter has a host lookup, which is + Fastly and Cloudflare. The Axum and Spin lookups resolve nothing, and + from row 33 of the stack both adapters refuse the selection at startup. A + selected module's lookup failure resolves at the requires-signal floor + and never at the fallback baseline. A static deployment instead states + its group at the top of the rules tree together with + `assume_single_jurisdiction = true`. - the `permissions.yaml` rules tree carrying a top node with a `group` and a `jurisdiction` (required), with each country or region below stating its own `jurisdiction` only where it differs from the node above. The operator no - longer carries `consent.gdpr.applies_in` or `consent.us_states.privacy_states`, - because the shipped tree holds the same 31 GDPR countries and the same 20 - privacy-law states. + longer carries `consent.gdpr.applies_in` or + `consent.us_states.privacy_states`, because the publisher's own tree + states its countries and states. The repository's sample lists the 31 + GDPR countries and the 20 privacy-law states main's lists held, and its + top node carries `jurisdiction: gdpr`, so every country it does not list + resolves to `gdpr` as well. Carrying a changed list into the publisher's + file is an item `ts config migrate` leaves to do + (`2026-10-17-config-migrate-design.md`). +- `[permissions] policy` naming that file, which + `ts config push --permissions` supplies with the settings, and on Axum + the same file in the folder `TRUSTED_SERVER_PERMISSIONS_DIR` names + (permission-model spec §3.1). The draft's committed per-adapter fixture files (`docs/guide/fixtures/migration-preserving-.toml`, CI-validated @@ -375,11 +417,20 @@ signals is unconditional in the shipped model. ## 6. Rollout sequence and observability -1. Implementation PRs land in the epic's order. **Done as specified.** - The series landed modules first with the geo default held at today's +1. Implementation PRs land in the epic's order. **Done as specified.** The + series landed modules first with the geo default held at today's behavior (#1044), and the permission model PR flipped the default together with its jurisdiction guard (#1045). Each PR's description - states what that PR changes. + states what that PR changes. With permission §3.1 the build and the + blob that names its policy go live together (§4 requirement 1). On + Fastly the pair is staged first, with `ts deploy --staging` and + `ts config push --staging --permissions `. From row 11 of the + stack a service that chooses its blob by host stages each host's blob + with `ts config push --key --permissions `, with + `EDGEZERO__STORES__CONFIG__TRUSTED_SERVER_CONFIG__NAME` naming the + staging store, because EdgeZero refuses `--staging` beside `--key` + (`2026-10-15-fastly-per-host-selector-design.md` section 2.5), and the + host's policy entries go to that store. 2. **Adapter qualification as a release gate.** **Deferred** with §4 requirement 2. The series' posture is the declared stateless-identity path for adapters without the identity endpoints. The response hook @@ -389,9 +440,11 @@ signals is unconditional in the shipped model. blob envelope and validates the configuration. The immutable `push_sequence` envelope, prepare/commit activation, admission lease, quiescence barrier, and scheduled-unavailability protocol are not - implemented. Policy in the shipped series changes by rebuilding - (`permissions.yaml`) or by pushing configuration, both taking effect - on restart/reload rather than through a fleet-wide activation CAS. + implemented. Policy changes by pushing the settings again with an + edited file, or another one, which the push pins under a new SHA-256 + (permission-model spec §3.1), and the change takes effect as each + instance next loads its settings rather than through a fleet-wide + activation CAS. 4. Before/after deploy, operators watch **EC issuance rate** and EID attachment rate as the canary metrics, because the failure mode of a bad migration is a silent drop to zero (or a silent grant to @@ -418,29 +471,45 @@ signals is unconditional in the shipped model. sense, where selection is configuration, so reverting to the previous configuration on the previous binary restores the previous behavior, and the same compiled binary switches modules through the - `TRUSTED_SERVER__ec__provider` / `TRUSTED_SERVER__device__provider` / - `TRUSTED_SERVER__geo__provider` overrides applied when the operator - publishes configuration through `ts config push`. The durable - artifacts today are exactly the **identity-graph withdrawal - tombstones** written by explicit storage withdrawal (no recovery, that - is their purpose). The draft's longer durable list (model epoch tuple, - use-opt-out suppressions, negative outbox, safety breaker) describes - the deferred durable-suppression design (sign-offs 11, 16, 19, 20). - The two documented-not-automated procedures (cleanup after a policy - tightening, legacy-reader retirement) remain for the migration guide. + `TRUSTED_SERVER__ec__module` / `TRUSTED_SERVER__device__module` / + `TRUSTED_SERVER__geo__module` overrides applied when the operator + publishes configuration through `ts config push`. Pushing the previous + settings again with the previous policy file restores the previous + policy, because a blob pins its file by name and SHA-256 and the push + pins whatever exact bytes `--permissions` supplies. A rollback + therefore needs the previous file's exact bytes, whether or not a store + entry for them survives, and on Axum the folder must hold that file + (permission-model spec §3.1). The durable artifacts today are exactly + the **identity-graph withdrawal tombstones** written by explicit + storage withdrawal (no recovery, that is their purpose). The draft's + longer durable list (model epoch tuple, use-opt-out suppressions, + negative outbox, safety breaker) describes the deferred + durable-suppression design (sign-offs 11, 16, 19, 20). The two + documented-not-automated procedures (cleanup after a policy tightening, + legacy-reader retirement) remain for the migration guide. ### 6.1 Operator CLI delta for this epic -The pre-existing CLI design remains unchanged and is what the series -uses. `ts config push` validates and publishes the operator configuration +The pre-existing CLI design is what the series uses. `ts config push` +validates and publishes the operator configuration (with typed environment overlays covered by a CLI regression test), and -`ts config validate` runs the same structural validation standalone. +`ts config validate` runs the same structural validation standalone. Once +the policy is selected when a service is deployed (permission-model spec +§3.1), both commands and `ts config diff` also take `--permissions `, +once for each policy file, which supplies the files the settings name. The +push always checks the policy, validate and diff check it when +`--permissions` is given, and validate without it says the policy was not +checked. The draft's normative delta to `ts config push` (never-reused `push_sequence`, immutable envelope identity, candidate CAS, §5.5 -promotion protocol) and the `ts config gc` command are **not +promotion protocol) and the draft's `ts config gc` command are **not implemented**. They are deferred together with sign-off 19 and remain the design bar for the durable-suppression and staged-activation follow-up. +EdgeZero has a `ts config gc` of its own, which reclaims only chunks, +the entries a value too long for one entry is split into, that no other +entry points to, and only on Fastly (permission-model spec §3.1), so it is +not the draft's command. Nothing in the series authorizes a generic raw metadata command or a model-transition command. @@ -463,11 +532,13 @@ model-transition command. `#[cfg(test)]`. The core test-only helper survives as legacy, and the runtime path does not use that helper.) - The permission model page states the precedence rules the code - implements. **Done** (#1045/#1047). The permission-model guide is in - the docs navigation, and the precedence is fixed in code (opt-out over - TCF, malformed-present fail-closed, then TCF), with pinning tests per - opt-out source against a consenting TCF record. Operator docs and - normative spec must not diverge on precedence. + implements. **Done** (#1045/#1047). The permission-model guide is in the + docs navigation, and the precedence is fixed in code (opt-out over TCF, + malformed-present fail-closed, then TCF), with pinning tests per opt-out + source against a consenting TCF record. In #1345 the order is the one + the deployment configures in `[permission-signal] modules` + (permission-model spec §4 and §13). Operator docs and normative spec + must not diverge on precedence. ## 8. Product decisions requiring explicit sign-off @@ -489,13 +560,13 @@ the code with the row open. | # | Recommended resolution | Where | Decision record | Status | | --- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | Honor mapped use opt-outs globally. Destructive identity effects are limited to explicit storage withdrawal, authenticated deletion, or a qualifying live TCF Purpose 1 refusal. | permission §4, §4.2 | (none) | open. Implemented by #1045 (jurisdiction-free suppression, withdrawal only on a live TCF storage refusal under a non-granted baseline). Ratify | -| 2 | GPP/USP sale opt-outs suppress the personalised-ads uses only. They neither revoke storage nor delete the identity. | permission §4.5 | (none) | open. Partially implemented by #1045 (no opt-out deletes or destructively revokes). The shipped default maps opt-outs to `revokes: all` rather than personalised-ads only, a `permissions.yaml` choice to ratify | -| 3 | Sharing/targeted-advertising opt-outs suppress the personalised-ads uses. An explicit applicable not-opted-out value may grant them. Neither affects storage. | permission §4.5 | (none) | open. Not implemented (the sharing and targeted-advertising GPP fields are not decoded, and no US signal grants anything in the shipped model) | -| 4 | US auction dispatch may continue while personalised-ads is unset only through permission §7's positive `ContextualAuctionView` and its sole normative inline manifest. | permission §7.1 | (none) | open. The strip half is implemented by #1045 (EIDs and `user.id` withheld when the pair is unset). The positive contextual projection is not implemented | -| 5 | Country-only and regionless US traffic use a protective country-wide `us-opt-out` floor. State rules may be stricter. | permission §3.4 | (none) | open. Implemented by #1045 (`permissions.yaml` maps country `US` to `us-opt-out` with state overrides available). Ratify | +| 2 | GPP/USP sale opt-outs suppress the personalized-ads uses only. They neither revoke storage nor delete the identity. | permission §4.5 | (none) | open. Partially implemented by #1045 (no opt-out deletes or destructively revokes). The repository's sample revokes seven Data Uses rather than personalized ads only, and from row 4 each publisher's file decides | +| 3 | Sharing/targeted-advertising opt-outs suppress the personalized-ads uses. An explicit applicable not-opted-out value may grant them. Neither affects storage. | permission §4.5 | (none) | open. Not implemented (the sharing and targeted-advertising GPP fields are not decoded, and no US signal grants anything in the shipped model) | +| 4 | US auction dispatch may continue while personalized-ads is unset only through permission §7's positive `ContextualAuctionView` and its sole normative inline manifest. | permission §7.1 | (none) | open. The strip half is implemented by #1045 (EIDs and `user.id` withheld when the pair is unset). The positive contextual projection is not implemented | +| 5 | Country-only and regionless US traffic use a protective country-wide `us-opt-out` floor. State rules may be stricter. | permission §3.4 | (none) | open. Implemented by #1045 (the repository's sample maps country `US` to `us-opt-out` with state overrides available). Ratify, and from row 4 each publisher's file decides | | 6 | Raw regulatory strings reach only the positively registered OpenRTB field that requires each source. All other destinations default deny. Identity rows retain normalized provenance/digests, not raw consent snapshots. | permission §7; modules §6.3 | (none) | open. Not addressed by the series | | 7 | Reject legacy batch-sync traffic until live-browser provenance backfill makes the row re-evaluable. | rollout §6 item 6; permission §7 | (none) | open. Not implemented (no provenance exists to recompute) | -| 8 | Gate proxy, click, and Testlight identity forwarding on the sharing pair (storage plus personalised-ads). | §2 row 11b | (none) | open. Not implemented by the series | +| 8 | Gate proxy, click, and Testlight identity forwarding on the sharing pair (storage plus personalized-ads). | §2 row 11b | (none) | open. Not implemented by the series | | 9 | Defer integration-owned cookie operations from the v1 response hook, and require a complete read/use/withdraw lifecycle before admission. | hook §3 | (none) | open. Overtaken (#1047 removed the whole hook from the series, so no cookie surface shipped). The deferral returns with the hook's first consumer | | 10 | Do not create a blanket session-cookie exemption. Every cookie must be covered by an approved permission or narrowly defined security-use authority. | hook §3 | (none) | open. Hook not shipped, unaffected | | 11 | Require a durable per-family negative-intent outbox in a failure domain independent of its strong target and checked freshly by every identity consumer, with a globally visible breaker over positive identity operations when neither can commit. | permission §4.3 | (none) | open. Not implemented (part of the durable-suppression follow-up) | @@ -529,16 +600,16 @@ the code with the row open. | --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Epic #777-#781 pending sign-off, no implementation | Implemented as five stacked PRs #1043-#1047, with §8 staying the ledger and rows marked for ratification | The series shipped the seam, selection, permission model, resolve endpoint, and docs, so the task force ratifies rather than re-litigates | | `[device] module = "fastly"` startup-fails pending a separate host-signal design (§2 row 9, row 22) | The `fastly` device module and host-signal EC module ship opt-in, and device signals persist in graph rows. The identifier-collision defect the review found (host-signal identifiers shared the HMAC grammar and keyspace) is fixed by the mandatory module-code envelope, so host-signal identifiers are `hs00~` namespaced. The policy question of row 22 is unchanged by the collision fix | Implementation choice, deliberately flagged as the open review question rather than presented as settled | -| US recipe: `requires_signal` with an extended grant-signal class (§2 rows 3/3a) | Shipped default: `granted` US baseline, `revokes: all` on any opt-out, and no US signal grants anything | The shipped signal model is revoke-only, which is simpler and stricter on grants, and the baseline choice is deployer-editable yaml, flagged in rows 3 and 5 | -| `[permissions]` TOML policy published at runtime | `permissions.yaml` compiled into the build, no `[permissions]` block | Policy stays reviewable in version control and the partial-policy trap cannot be written | +| US recipe: `requires_signal` with an extended grant-signal class (§2 rows 3/3a) | The repository's sample gives US a `granted` baseline and revokes seven Data Uses on any opt-out, and no US signal grants anything | The series' signal model is revoke-only, which is simpler and stricter on grants, and the baseline is each publisher's own decision, flagged in rows 3 and 5 | +| `[permissions]` TOML policy published at runtime | The policy stays the publisher's `permissions.yaml`, which `[permissions] policy` names and `ts config push --permissions` supplies when the service is deployed, stored once and pinned in each blob by its SHA-256. The operator's `[permissions]` holds that name and never rules (permission-model spec §3.1) | The file stays reviewable on its own, the digest ties each blob to one exact version, and the partial-policy trap still cannot be written | | `rules.default` worldwide default entry (§2 row 7) | Required fallback baseline naming the group for unmatched requests, validated at startup. Shipped as `[geo] default_country` and carried by the rules tree's top node since 2026-09-01 | Same role, one mechanism, loud when missing | | Graph store mandatory at startup for creating modules (§2 row 12, §4.4b) | No startup requirement, and with no graph no identifier is created, per request | The phantom-cookie rule holds without a breaking startup change, and graphless deployments run identity-less | | `legacy_providers` reader chain for module switches (§4.8) | Not implemented, so a switch restarts identity | Deferred until a second server-side module makes switching real | | N+1/N+2 negative-record machinery, model epochs, `m00` mirror (§4.1, row 20) | Deferred with rows 11, 16, 19, 20. The shipped dual-read is one release of `[ec] passphrase` mapping with mixed forms rejected | Nothing durable beyond pre-existing withdrawal tombstones ships, so binary rollback strands no new state | -| Staged activation, `push_sequence`, quiescence, `ts config gc` (§6.3, §6.1) | Basic `ts config push` / `ts config validate` only | The activation protocol belongs to the deferred durable-suppression rollout (row 19) | +| Staged activation, `push_sequence`, quiescence, `ts config gc` (§6.3, §6.1) | Basic `ts config push` / `ts config validate` only. EdgeZero's own `ts config gc` reclaims only unreferenced chunks on Fastly and is not the draft's command | The activation protocol belongs to the deferred durable-suppression rollout (row 19) | | Migration guide with committed per-adapter fixtures and a full gated metric set (§5, §6.4) | `configuration.md` / `edge-cookies.md` document the migrated shape, while fixtures, the guide page, and metrics remain outstanding | Documentation shipped for configuration, and the operational guide is the remaining deliverable before a release | | Client cycle descoped and hook shipped headers-only (row 15) | The client cycle shipped hardened (#1046) and the hook shipped not at all (#1047) | Hardening replaced descoping, and the hook had no consumer, which is the hook spec's own admission rule | | Example config ships the migrated happy path uncommented (§4.9) | Identity off by default with selector and block commented together, and validation closes the silent-stateless trap | Loud startup validation, not an uncommented default, is what prevents PR #838's silent-stateless state | | Environment override documented but `#[cfg(test)]`-only in PR #838 (§7) | Override applied as typed EdgeZero app-config overlays at `ts config push`; the existing CLI overlay test covers the mechanism, and a module-specific override test is still to write | The same compiled binary switches modules at deployment through the published configuration | | Pinned known-answer HMAC vectors committed (§3) | Stability tested per inputs, and the pinned vector is still to commit | The cross-version CI pin remains open work under §3 | -| GB storage baseline change only with citation and sign-off (§2 row 4) | The shipped `permissions.yaml` adopts `granted` storage for GB without a recorded citation | Flagged in row 4, since the task force owns the decision and its record | +| GB storage baseline change only with citation and sign-off (§2 row 4) | The repository's sample adopts `granted` storage for GB without a recorded citation | Flagged in row 4, since the task force owns the decision and its record | diff --git a/docs/superpowers/specs/2026-10-12-config-by-host-design.md b/docs/superpowers/specs/2026-10-12-config-by-host-design.md index ee9d547d81..5c32fb63a9 100644 --- a/docs/superpowers/specs/2026-10-12-config-by-host-design.md +++ b/docs/superpowers/specs/2026-10-12-config-by-host-design.md @@ -1,15 +1,15 @@ # Design Spec: One app-config blob for each host -**Status:** Proposed, 2026-10-12. Raised as draft pull request IABTechLab/trusted-server#1352 (Add one app-config blob for each host). Pull request IABTechLab/trusted-server#1342 carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Raised as IABTechLab/trusted-server#1352 (Add one app-config blob for each host), one of the stack's pull requests. Pull request IABTechLab/trusted-server#1342 carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-03-19-edgezero-migration-design.md` and `2026-10-15-fastly-per-host-selector-design.md`, which replaces where this design reads the selector from **Related PRs:** IABTechLab/trusted-server#1348, IABTechLab/trusted-server#1352 **Issues:** IABTechLab/trusted-server#1270 -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-26 > **Why this spec exists.** An operator who runs several publishers needs a Fastly service for each. This row lets one service read the settings of the publisher whose host a request names. -Line references are at `b8314cb52`, the head of `split/11-config-by-host`, which is row 11 of the stack. A file named without a folder, such as `lib.rs`, is in `crates/trusted-server-adapter-fastly/src`. References to EdgeZero are at `683202c`, the version the stack pins. +Line references are at `b8314cb52`, which was the head of `split/11-config-by-host`, row 11 of the stack, when this spec was reviewed. What this spec says about the policy file states the code of IABTechLab/trusted-server#1352, cited by file and function. A file named without a folder, such as `lib.rs`, is in `crates/trusted-server-adapter-fastly/src`. References to EdgeZero are at `683202c`, the version `b8314cb52` pins. The stack now pins EdgeZero `v0.0.10`, and the text says so where it names that version. ## 1. Problem @@ -34,11 +34,11 @@ fastly config-store-entry update --upsert \ --value '{host}' ``` -The selector entry sits in the `edgezero_runtime_env` store, which this row reads through EdgeZero's `runtime_env_config`. IAB Tech Lab's main has moved EdgeZero to `v0.0.9`, a version without that store, so `2026-10-15-fastly-per-host-selector-design.md` (the selector design) replaces where this row reads the selector from with an entry in the application's own config store (section 2.1 of the selector design). The placeholder and the answers below stay as they are, apart from three groups of changes in the selector design. +The selector entry sits in the `edgezero_runtime_env` store, which this row reads through EdgeZero's `runtime_env_config`. IAB Tech Lab's main has since moved EdgeZero to `v0.0.9` and then to `v0.0.10`, neither of which has that store, so `2026-10-15-fastly-per-host-selector-design.md` (the selector design) replaces where this row reads the selector from with an entry in the application's own config store (section 2.1 of the selector design). The placeholder and the answers below stay as they are, apart from three groups of changes in the selector design. 1. Sections 2.2 to 2.4 change how a `Host` that is not UTF-8 text, a key that is too long and a selector that misspells the placeholder are answered, and which key `Hooks::routes()` reads. Open questions 1, 2, 6 and 8 in section 5 describe the four changes. 2. Section 2.4 also answers `500` for a blank selector and for a store that cannot be opened, which replaces the fallback to the logical store ID that this spec describes under "Why the placeholder goes in the selector that already exists" and in the risk table. A selector entry that is absent still reads the logical store ID. -3. Sections 2.6 and 2.8 replace the push and selector-writing steps, being the two `ts config push` commands and the `fastly config-store-entry update` command above and the steps for moving a service below. EdgeZero `v0.0.9` refuses any Fastly config key other than the logical store ID, so section 2.6 changes how a host's blob is pushed, and section 2.8 sets the order of the steps, with the selector written to its new entry (section 2.1). +3. Sections 2.6 and 2.8 replace the push and selector-writing steps, being the two `ts config push` commands and the `fastly config-store-entry update` command above and the steps for moving a service below. EdgeZero `v0.0.10` refuses any Fastly config key other than the logical store ID, so section 2.6 changes how a host's blob is pushed, and section 2.8 sets the order of the steps, with the selector written to its new entry (section 2.1). The entry point puts the request's host in the placeholder's place before anything reads settings, so the debug probes, the application build and the finalizers all read the blob of the publisher the request is for (`lib.rs:432`, before the reads at `:446`, `:467`, `:507` and `:646`). @@ -52,7 +52,7 @@ The placeholder is the text `{host}` in lower case, and every occurrence is repl Keys are pushed in lower case. The runtime lowercases the host and `ts config push --key` stores a key as typed (`resolve_config_key`, `crates/edgezero-cli/src/config.rs:1394-1408` in EdgeZero), so a blob pushed under capitals is never read and its host is answered `421`. Only the guide asks for lower case, and no code checks the case of a pushed key. -Under a selector that carries the placeholder, each request for a host name makes one lookup in the config store before the application is built or found, whether or not a sandbox already holds that application. The lookup reads the entry at the key alone (`blob_exists`, `config_selection.rs:96-105`), so a blob split into chunks counts as present when the entry at the blob's key is present. +Under a selector that carries the placeholder, each request for a host name makes one lookup in the config store before the application is built or found, whether or not a sandbox already holds that application. The lookup reads the entry at the key alone (`blob_exists`, `config_selection.rs:96-105`), so a blob split into chunks counts as present when the entry at the blob's key is present. Chunks are the entries a value too long for one entry is split into, and the entry at the value's own key is then a pointer that names each chunk. ### The answers @@ -77,7 +77,7 @@ A selector without the placeholder is the key as it stands and nothing is looked A switch of its own beside the sandbox bounds is the obvious design. EdgeZero's staged deploy is why this is not built that way. -At the EdgeZero version this stack pins, a staged deploy copies every runtime entry of the service into a store of its own and replaces only the config key selectors, pointing each at `_staging` ([`staging_entries_from_production`](https://github.com/stackpop/edgezero/blob/683202c66948146ec360f490126f1d60f920288b/crates/edgezero-adapter-fastly/src/cli.rs#L3579-L3603)). A separate switch would be copied with the rest, and a staged version would then choose by host and read the production blobs. The selector is replaced, so a staged version reads the one staging blob for every host and never a production one. `2026-10-15-fastly-per-host-selector-design.md` (section 2.5) says how staging works under the EdgeZero version Tech Lab main uses. +At `683202c`, a staged deploy copies every runtime entry of the service into a store of its own and replaces only the config key selectors, pointing each at `_staging` ([`staging_entries_from_production`](https://github.com/stackpop/edgezero/blob/683202c66948146ec360f490126f1d60f920288b/crates/edgezero-adapter-fastly/src/cli.rs#L3579-L3603)). A separate switch would be copied with the rest, and a staged version would then choose by host and read the production blobs. The selector is replaced, so a staged version reads the one staging blob for every host and never a production one. EdgeZero `v0.0.10`, which the stack now pins, has no `staging_entries_from_production`, and `2026-10-15-fastly-per-host-selector-design.md` (section 2.5) says how staging works under it. The selector is also read by [`runtime_env_config`](https://github.com/stackpop/edgezero/blob/683202c66948146ec360f490126f1d60f920288b/crates/edgezero-adapter-fastly/src/lib.rs#L259-L282), and that function does not always fail when the selector cannot be read. A lookup that fails panics, so the request fails. Where the `edgezero_runtime_env` store cannot be opened, the function logs a warning and returns no entries, and an entry that is blank or carries a control character is read as absent ([`EnvConfig::store_key`](https://github.com/stackpop/edgezero/blob/683202c66948146ec360f490126f1d60f920288b/crates/edgezero-core/src/env_config.rs#L113-L122)). In both cases the key becomes the logical store ID (`app.rs:189`), so a blob left at that key answers every host. Section 2.4 of the selector design replaces both cases with a `500` answer, for a blank selector and for a store that cannot be opened, and an entry that is absent still reads the logical store ID. A service that chooses by host therefore holds no blob at the logical store ID, and the steps for moving a service (below) remove any blob there. @@ -86,11 +86,11 @@ The selector is also read by [`runtime_env_config`](https://github.com/stackpop/ | The risk | What prevents it | | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | A host nobody configured is answered from a shared blob | A host with no blob is answered `421` and never from another key. A shared blob answers every host only where the selector reads as absent, which makes the key the logical store ID (above), or where the selector misspells the placeholder and so becomes a fixed key (see Open questions) | -| A `Host` header is written to name another key in the store, such as the logical store ID or a chunk of a blob | Only letters, digits, hyphens and dots make a host name, and nothing else is looked up. Those keys each carry an underscore. The tests write the logical store ID out as text and not as `DEFAULT_CONFIG_STORE_ID` | +| A `Host` header is written to name another key in the store, such as the logical store ID or a chunk of a blob | Only letters, digits, hyphens and dots make a host name, and nothing else is looked up. Those keys each carry an underscore, as a policy entry's key does, so under a selector of `{host}` alone no `Host` names one (Open questions, item 9). The tests write the logical store ID as text, not as `DEFAULT_CONFIG_STORE_ID` | | A sandbox that serves several requests hands the first publisher's application to the second | The sandbox keeps each application under the key of the blob it was built from, and finds one by that key alone | | Something read from one blob is kept for the life of the process | Every process-wide static in core, the modules and the Fastly adapter was read for this. Each is a constant, or is keyed by its content or by the store and key it was read from, and none holds anything read from settings. Prebid keeps its script patterns for the life of the process by leaking them (see Open questions) | | A page stored for one publisher is served for another | The template cache key already carries the request's host and scheme | -| A staged version reads production blobs | At the EdgeZero version this stack pins, a staged deploy replaces the selector, as above | +| A staged version reads production blobs | At `683202c` a staged deploy replaces the selector, as above. Under `v0.0.10` section 2.5 of the selector design says how staging works | | The store cannot say whether a host has a blob | The request is answered `500` and not served | The first table row describes the code at `b8314cb52`. Under the selector design (section 2.4), a blank selector, a store that cannot be opened and a misspelled placeholder are each answered `500`, so a service that chooses by host reads the blob at the logical store ID only where the selector entry is absent. @@ -112,12 +112,13 @@ The service has one of each of the following, and nothing divides any of them be - **The secret store.** Each publisher's blob is meant to name keys of its own in the secret store, and nothing checks that the keys differ between blobs, because each application is built from one blob and sees no other. Two blobs that name one secret read the same value. Where an Edge Cookie ID is made from a passphrase, its first 64 characters are a keyed hash (HMAC-SHA256) of the client IP address under that passphrase (`crates/trusted-server-core/src/ec/generation.rs:104-115`), so two blobs that name one passphrase give one client IP address the same prefix at both publishers, and a person can be linked across them. The harness's blobs name the same secrets (`scripts/config-by-host-local-test.sh:185-221`). Two blobs that name one Edge Cookie identity store (`ec_store`) share one identity graph. - **The request signing keys and the template cache's purge key.** The signing keys sit in two stores whose names are fixed, `jwks_store` and `signing_keys` (`crates/trusted-server-core/src/request_signing/mod.rs:40-54`). Every template in the template cache carries the cache tag (surrogate key) `ts-template` (`crates/trusted-server-core/src/platform/template_cache.rs:46`), which one purge call flushes (`crates/trusted-server-adapter-fastly/src/template_cache.rs:288-291`). Three routes act on them, `/_ts/admin/keys/rotate`, `/_ts/admin/keys/deactivate` and `/_ts/admin/cache/purge` with a scope of `all` (`app.rs:1415-1429`). Each is checked against the administration user and password in the blob of the host the request names (`app.rs:1624`), so one publisher's user and password can rotate the keys every publisher signs with and flush the cached templates of every publisher. Row 13 (`split/13-removals`) removes the administration user and password, and these three routes go with them (`2026-10-12-administration-off-the-publisher-domain-design.md`). +- **Each policy file.** Row 4 of the stack (IABTechLab/trusted-server#1345) selects the permission policy when a service is deployed (`2026-07-30-permission-model-design.md` §3.1), and this row builds on it. Each host's blob names its policy file in `[permissions] policy`, with the file's SHA-256, and carries no copy of the file. The keyed push keeps each file `--permissions` supplies once, as a policy entry under `permissions_` in the config store, which is a pointer and chunks when the entry is longer than the 8,000 characters one entry holds. It writes each entry the store does not already hold before the host's blob, and keeps one that holds the same file (`run` and `holds_policy` in `crates/trusted-server-cli/src/commands/config/keyed_push.rs`). Core forms the key with `policy_key` (`crates/trusted-server-core/src/platform/policy_source.rs`). For every blob but the one at the logical store ID, the adapter reads the policy from that entry (`blob_carries_its_policy` in `app.rs`, and `FastlyPolicyEntries` in `platform.rs`). The blob at the logical store ID carries its file, as on a service with one blob. The tests `two_hosts_naming_one_policy_file_keep_it_once` (`crates/trusted-server-cli/tests/config_push_by_key.rs`) and `a_host_blob_whose_policy_entry_is_missing_is_refused_naming_the_entry` (`platform.rs`) show one entry kept for two hosts and a missing entry refused. Hosts whose blobs pin one SHA-256 share that entry and run the same policy, and a file edited between two pushes is two digests and two entries. A host fails to load, as a blob that cannot be loaded does (the answers above), and answers `500`, when its blob names no policy, or its policy entry is absent, cannot be read, does not match its digest or fails the checks of §3.3 of the permission model design. A config store that cannot be read answers `500`, as the selection does when it cannot look a host up (§3.1 of the permission model design, item 7). The other hosts are answered as before. This row's keyed writer writes only the local server's store, and row 13 gives it the live store. `ts config diff` cannot reach a host's blob, because EdgeZero `v0.0.10` refuses every Fastly key but the logical store ID for a diff as for a push (`resolve_config_store_and_key` in its `config.rs`), so a host's pinned policy is checked through `/_ts/config` on that host. - **The rate counter,** `counter_store` (`rate_limiter.rs:9`), which limits Edge Cookie sync requests. - **The sandbox's memory bound and counters.** Where a sandbox serves several requests, every application it keeps shares the bound, and the counters it reports count the requests and builds of every publisher it serves. ### Moving a service onto a blob for each host -Sections 2.6 and 2.8 of the selector design replace steps 1 and 2 below. Section 2.6 changes how a blob is pushed, because EdgeZero `v0.0.9` refuses any Fastly config key other than the logical store ID, and section 2.8 sets the order of the steps and the entry that holds the selector. Step 3 still applies, and step 2 of section 2.8 gives the same advice. +Sections 2.6 and 2.8 of the selector design replace steps 1 and 2 below. Section 2.6 changes how a blob is pushed, because EdgeZero `v0.0.10` refuses any Fastly config key other than the logical store ID, and section 2.8 sets the order of the steps and the entry that holds the selector. Step 3 still applies, and step 2 of section 2.8 gives the same advice. 1. Push a blob for each host the service answers for, in lower case. 2. Write the selector. The entry point reads the selector on every request, so the change takes effect on the next request without a new version, which is why the blobs come first. @@ -162,7 +163,7 @@ These cases are not tested. ## 5. Open questions -The behaviours below are stated in the sections above. A change to the code would settle each one, and this section decides none of them. +The behaviors below are stated in the sections above. A change to the code would settle each one, and this section decides none of them. 1. **A `Host` that is not UTF-8 text.** The selection reads `Host` with `get_header_str` (`lib.rs:106`), which panics on a value whose bytes are not valid text (fastly 0.12.1, `src/http/request.rs:1644-1654`). The read runs for every request except `GET /health`, whatever the selector holds, so it runs for a service of one blob too. `entry_request` (`lib.rs:1098`) reads `Host` the same way once settings have loaded, so a request with such a `Host` already panicked there before this row, whenever its settings loaded. Should the adapter read `Host` as text in one place only, and treat a value that is not text as no `Host`, so that it is answered `421` where the selector carries `{host}`? `2026-10-15-fastly-per-host-selector-design.md` (sections 2.2 and 2.4) proposes that. 2. **A key over the limit a config store accepts.** A host may be 253 characters (`config_selection.rs:17`), so a selector with text around `{host}` can make a key over 255 characters, the limit EdgeZero takes (`crates/edgezero-adapter-fastly/src/chunked_config.rs:36` in EdgeZero). Fastly refuses that lookup with `KeyTooLong` (fastly 0.12.1, `src/config_store/handle.rs:113`), `blob_exists` returns the refusal as a message (`config_selection.rs:96-105`), and `resolve_key` makes it `StoreUnavailable` (`config_selection.rs:82`), so the answer is `500` and not `421`. Should a key that long count as a host with no blob? `2026-10-15-fastly-per-host-selector-design.md` (section 2.3) proposes `421` with no lookup. @@ -170,16 +171,18 @@ The behaviours below are stated in the sections above. A change to the code woul 4. **Administration that reaches every publisher.** The same section gives the effect. Row 13 removes the administration user and password and the three routes. Does row 11 need a change of its own before row 13 arrives? 5. **Cases no test covers.** The cases are listed in section 4. Which of them should the pull request cover before it merges? 6. **Memory kept, and comments that no longer hold.** Prebid registers a route for each of its `script_patterns` and leaks the pattern's text, which keeps the text for the life of the process (`crates/auction/prebid/src/lib.rs:1171-1175`). The comment there says the registration runs once at startup, but the registry registers routes for every application build, so a sandbox that builds again leaks again. The comment on `AppState` (`app.rs:195-200`) says a build is retained until the sandbox retires, where a sandbox now keeps up to eight and drops the least recently served. `Hooks::routes()` (`app.rs:1664-1668`) builds a router from the selector as written and with no request, so under `{host}` the call would look for a blob named `{host}`. The entry point does not call `routes()` and builds through `build_app_with_state` (`lib.rs:508`), and `2026-10-15-fastly-per-host-selector-design.md` (section 2.2) has `routes()` read the logical key. Should the patterns be held without a leak, and should the comments and `routes()` change? -7. **The host of a request made over HTTP/2.** HTTP/2 carries the host in an `:authority` field and not in a `Host` header. The selection reads the `Host` header only, and nothing in this row shows that Fastly fills `Host` from the authority, because the harness sends plain HTTP/1.1 requests. Where is that behaviour shown, or does it need a test of its own? +7. **The host of a request made over HTTP/2.** HTTP/2 carries the host in an `:authority` field and not in a `Host` header. The selection reads the `Host` header only, and nothing in this row shows that Fastly fills `Host` from the authority, because the harness sends plain HTTP/1.1 requests. Where is that behavior shown, or does it need a test of its own? 8. **A selector that misspells the placeholder.** A selector such as `{Host}`, `{hostname}` or `{host` is read as a fixed key for every host, and nothing refuses such a selector. Should a selector like that be refused when it is read? `2026-10-15-fastly-per-host-selector-design.md` (section 2.4) refuses a selector like that as malformed and answers `500`. +9. **A selector that forms a policy entry's key.** A policy entry's key carries an underscore, which no host name holds, so under a selector of `{host}` alone no `Host` names one. `malformed` (`config_selection.rs:68-88` at IABTechLab/trusted-server#1352) still accepts a selector with text around the placeholder, such as `permissions_{host}`, and `normalized_host` sets no limit on the length of a label, so under that selector a `Host` of 64 hexadecimal characters names a policy entry. The service would then read that entry as the host's blob, fail to load it and answer `500`. Should a selector that begins `permissions_` be refused, as a misspelled placeholder is in the selector design? ## Revision record -| Date | Change | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | -| 2026-10-16 | Round 1 fixes. The header points at the open draft pull request, the risk table is repaired, the fallback to the logical store ID is stated, the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests are added, and the code findings sit under Open questions. The selector store points at the Fastly per-host selector design. | -| 2026-10-16 | Round 2 fixes. The pointer to the Fastly per-host selector design names section 2.4, whose `500` answers replace the fallback to the logical store ID, and sections 2.6 and 2.8, which replace the push and selector-writing steps. The same sections are named where this spec describes the fallback and the steps for moving a service. | +| Date | Change | +| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-16 | Round 1 fixes. The header points at the open draft pull request, the risk table is repaired, the fallback to the logical store ID is stated, the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests are added, and the code findings sit under Open questions. The selector store points at the Fastly per-host selector design. | +| 2026-10-16 | Round 2 fixes. The pointer to the Fastly per-host selector design names section 2.4, whose `500` answers replace the fallback to the logical store ID, and sections 2.6 and 2.8, which replace the push and selector-writing steps. The same sections are named where this spec describes the fallback and the steps for moving a service. | +| 2026-10-26 | Each host's blob names the policy file row 4 selects, and one policy entry for each SHA-256 serves every host that pins it, as this row's keyed push and Fastly adapter do. Chunks and pointers are defined, the EdgeZero versions are anchored, `ts config diff` is said not to reach a host's blob, and Open questions, item 9, asks about a selector that forms a policy entry's key. | ## Review log diff --git a/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md b/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md index ab870e6a34..67354d2f03 100644 --- a/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md +++ b/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md @@ -1,11 +1,11 @@ # Design Spec: Inspection endpoints -**Status:** Proposed, 2026-10-12. Raised as draft pull request IABTechLab/trusted-server#1350, row 9 of the stack in IABTechLab/trusted-server#777, for Tech Lab review. The two earlier pull requests for the same commit, jwrosewell/trusted-server#4 and IABTechLab/trusted-server#1294, are closed. +**Status:** Proposed, 2026-10-12. Raised as pull request IABTechLab/trusted-server#1350, row 9 of the stack in IABTechLab/trusted-server#777, for Tech Lab review. The two earlier pull requests for the same commit, jwrosewell/trusted-server#4 and IABTechLab/trusted-server#1294, are closed. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** IABTechLab/trusted-server#1350, IABTechLab/trusted-server#1348, jwrosewell/trusted-server#4 (closed), IABTechLab/trusted-server#1294 (closed) **Issues:** IABTechLab/trusted-server#1268 -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-26 > **Why this spec exists.** A reader, a regulator or an auditor has only the operator's word for what a deployment runs, decides and keeps. Three endpoints in this change show those three things, and this spec says where each endpoint answers, what each shows and to whom. @@ -43,12 +43,16 @@ A deployment decides what a request is permitted, holds the settings that decide 2. `modules` says which signal modules the publisher named (`configured`, which is `null` when the publisher named none and the adapter's own list runs) and which of them produced a signal used in the answer (`contributed`). 3. `version` is the version of the core crate that answered. That is the workspace package version, `0.1.0` at `497d553c6`, and not the build identifier the `X-TS-Version` header carries. +Row 4 of the stack (IABTechLab/trusted-server#1345) selects the permission policy when a service is deployed (`2026-07-30-permission-model-design.md` §3.1). This row builds on row 4 and adds a fourth, `policy`, to `permissions_payload` (`inspect/permissions.rs`). The field names the policy file whose rules the answer resolved, as the settings give it in `[permissions] policy`. The field is `null` only for settings that name no policy, which no loaded domain has, because the load refuses them. A name is a file's stem, which two versions of a file share, so the SHA-256 the configuration endpoint shows is what matches the answer to one exact copy of the file. + One function spells the page's shape, and a test checks that the answer holds every key the page is given. The handler resolves the request's state with the call a page request uses, from the request's signals (its cookies and headers) and the location of its client address, and writes nothing for the reader. The answer shows the asker the signals the asking request carried and nothing about any other request. The answer is never stored, and any origin may read it, so a page on another site can render it. A request from another origin carries no cookies, so no stored signal reaches the resolution. The `Sec-GPC` header the browser sends and the location of the client address still do. **The configuration endpoint.** Anyone can read the scripts a page runs, and the configuration endpoint lets anyone read the configuration that runs at the edge too. The answer holds `version` (the core crate's version, as above), `settings` and `masked`. `settings` is the settings the deployment holds after defaults are applied, with each masked value replaced by `XXXX` and every object's keys sorted, so the same settings always give the same bytes. `masked` lists every masked path. The view is built from the settings alone, so every request gets the same one, and the view is built again on every request, with no rate limit. +`settings.permissions` holds `policy`, the policy file's name as the settings give it, and `sha256`, the SHA-256 of that file. The push also writes `text`, the file itself, into the blob (the settings as `ts config push` stores them) where the policy travels inside it, and row 4's runtime loader takes that text out of the settings when it loads the policy (`load_domain_policy` in `config_payload.rs`). So the view, which is the masked settings, shows `permissions` as `policy` and `sha256` only and never `text`, with no code of its own for that (`build_view` in `inspect/config.rs`). Neither `policy` nor `sha256` is masked by default, so a publisher can check which exact file a domain runs. `[inspect] config = false`, or a `hide` that covers `permissions.sha256`, withholds the SHA-256 from every address, and `/_ts/permissions` then shows only the name. + Three kinds of value are masked. 1. Every secret, whatever the publisher's document says. The settings loader records each value it fills from the secret store, and those paths are masked, along with every field declared a secret. Then any string that still contains one of those values is masked wherever it sits, and a key that contains one is renamed `XXXX` (or `XXXX-2` and on where that name is taken). @@ -68,6 +72,8 @@ A pattern that would not be honored as written refuses the configuration, both w 5. A pattern that changes nothing the view shows, being a `show` of values that are not masked by default or a `hide` of values that are masked or empty already. 6. Any `show` or `hide` beside `config = false`. +Every place that checks patterns goes through `validate_patterns` (`inspect/config.rs`), being `ts config validate`, `ts config push`, `ts config diff`, `ts audit generate` and the load, and from row 51 `ts config migrate`. `permissions.text` is never shown, so `validate_patterns` refuses a pattern under `permissions.text`, and judges every other pattern against the settings as the view shows a domain the push wrote and the loader loaded, with `permissions.text` gone and `permissions.sha256` present, because the push always writes it (`as_the_view_sees_a_pushed_domain`). So a pattern that names `permissions.sha256` is judged alike before the push and at the load. `ts config validate` takes no `--adapter`, so it cannot know whether the push adds the text (`2026-07-30-permission-model-design.md` §3.1, items 2 and 10). + At deploy time the secret fields still hold the names of keys in the secret store. The deploy check therefore knows which leaves are secrets, including the passphrase in an Edge Cookie module table the operator gave a label of their own and the secrets a module declares, and cannot scan for their values. A stored document is unchanged. The marking that lets the view find sensitive values is a mode of one serialization call, every other serialization writes what it wrote before, and the `[inspect]` section is left out of a document unless the publisher wrote one. Tests check both. @@ -113,7 +119,7 @@ When nothing is held, the page answers `200` with `held` false and a `why` that **When something goes wrong.** -1. A refusal when the settings load, an `[inspect]` pattern among the causes, stops the deployment from building its state. Every address on the deployment then answers the startup error as plain text, these five included. The status is `500` for a configuration error on Fastly, Axum and Cloudflare, and `503` on Spin. +1. A refusal when the settings load, an `[inspect]` pattern among the causes, stops the deployment from building its state. Every address on the deployment then answers as plain text, these five included. Fastly, Axum and Cloudflare answer with the status of the error's kind (`status_code` in `error.rs`), which is `500` for a configuration error such as a refused pattern, with the generic message "An internal error occurred" (`user_message` in the same file), and log the reason, which Cloudflare's own logger does only from row 54. Spin answers with the status of the error's kind too, with that status's reason phrase, such as "Internal Server Error", as the body (`startup_error_router` in the Spin adapter's `app.rs`). It answered a fixed `503` with "Service Unavailable" at `497d553c6`, and row 4 of the stack (IABTechLab/trusted-server#1345) moved it to the rule the other adapters follow. Spin writes the reason to its log as well, which records nothing until row 54 gives Spin a logger. `GET /health` still answers `200` on Fastly and Spin. 2. The configuration view fails only if the settings cannot be written as JSON. The address then answers `500` with the message `The configuration could not be shown.`, as a page or as JSON by its address, and logs the failure. 3. The permissions and data handlers answer an error from reading the request's state as any route answers an error, in plain text. A `Cookie` header that is not valid UTF-8 is `400`, and a selected Edge Cookie module that cannot be built is `500`. A failed location lookup is not an error. The request's permissions then resolve at the strictest level, where a permission is set only by a signal, and the failure is logged. 4. The data handler never answers that nothing is held when it could not read the store. A store that cannot be opened or read, or a stored record that does not parse or validate, is `503` as plain text. @@ -141,7 +147,7 @@ Nothing selects this change. All five addresses are registered on all four adapt - Operator data. It stays in the platform's own stores behind the platform's own authentication, and never comes through the publisher's domain. - An identity graph on Axum, Cloudflare or Spin, which answer that nothing is held. - A `.json` form of the data page, a rate limit on any of the five addresses, or a cached copy of the configuration view. -- A change to what a configuration push writes. A stored document is the same as before. +- A change to what a configuration push writes. A stored document is the same as before. Row 4 adds `sha256`, and `text` where the policy travels inside the blob, to `[permissions]`, and this row shows `policy` and `sha256`, which are all the settings keep once row 4's loader has read the policy. - Removing the administration endpoints from the publisher's domain, which a later change does. **Open questions.** Each of these is a question about the code. This spec records what the code does and leaves the questions open. @@ -154,8 +160,8 @@ Nothing selects this change. All five addresses are registered on all four adapt ## 4. Tests -- The permissions tests check that the answer holds every key a page is given, names each signal and the module that read it, reports the configured and contributing modules apart (with `null` when none is named), shows a withdrawal of storage, still answers with no signals, escapes a hostile signal value, and resolves the permissions of the request that asks. -- The configuration tests check the pattern grammar and each refusal with its reason, the deploy check, that every secret core declares is masked and never served (and that the test exercises each declared secret), that every value sensitive by default is masked, that a secret a module declares is masked, that a secret copied into an undeclared field is caught by the value scan, `show` and `hide`, `404` when the configuration is not published, sorted keys, and that an ordinary serialization and a stored document are unchanged. +- The permissions tests check that the answer holds every key a page is given, names each signal and the module that read it, reports the configured and contributing modules apart (with `null` when none is named), shows a withdrawal of storage, still answers with no signals, escapes a hostile signal value, and resolves the permissions of the request that asks. They also check that the answer names the policy (`the_payload_names_the_policy_whose_rules_it_resolved`). +- The configuration tests check the pattern grammar and each refusal with its reason, the deploy check, that every secret core declares is masked and never served (and that the test exercises each declared secret), that every value sensitive by default is masked, that a secret a module declares is masked, that a secret copied into an undeclared field is caught by the value scan, `show` and `hide`, `404` when the configuration is not published, sorted keys, and that an ordinary serialization and a stored document are unchanged. They also check that `permissions.policy` and `permissions.sha256` show unmasked, that `permissions.text` never shows and neither address serves the file (`the_view_shows_the_policy_by_name_and_digest_and_never_its_file`), that the deploy check and the load both refuse a pattern under `permissions.text` (`a_pattern_under_the_policy_file_is_refused`), and that a pattern naming `permissions.sha256` is judged alike before the push and at the load (`a_pattern_naming_the_policy_digest_is_judged_alike_before_and_after_the_push`). - The data tests check that each of seven kinds of request that is not a browser opening the page is refused, that the record is shown with every identifier masked, that every field of the record is listed, that a known record gives the documented answer, each reason for nothing held, that a store that cannot be read is an error, and the headers on both the page and the refusal. - Core checks that a required header outlasts an operator's header of the same name, and the Fastly finalizer test runs the data page through the finalizer under an operator's `Content-Security-Policy` and `X-Frame-Options`. - Each adapter's route tests request the addresses and check the status and the content type. They check that the configuration masks a secret and that the data page answers `403` and `200` as it should, and Fastly's also check that no Edge Cookie finalize state is attached. @@ -172,17 +178,18 @@ Nothing selects this change. All five addresses are registered on all four adapt ## 5. Acceptance -The fork's CI passed on the series head, `497d553c6`, all nineteen checks. That commit is the head of IABTechLab/trusted-server#1350 and of the two closed pull requests. +The fork's CI passed on the series head, `497d553c6`, all nineteen checks. That commit was the head of IABTechLab/trusted-server#1350 when it was reviewed on 2026-10-15, and of the two closed pull requests. On `33282f0c9`, the last of the five original commits, the whole check also passed locally on Windows: `cargo fmt --all -- --check`, six Clippy runs with warnings denied, `cargo test -p trusted-server-core` (2,615 passed), the Fastly tests under Viceroy (3,626), the Axum alias (900), Cloudflare (59), Spin (93), the seam probe (11), the parity tests (20, three of them new here), the configuration generator's tests (8), the integration tests that need no container (29), the permissions inspector's wasm build, and the docs format, lint and build. The two core-only commits, `5fb5f0793` and `16771b18f`, were each linted with the adapter changes set aside, so each commit builds on its own. Two commits came after `33282f0c9` (`1c02104bd` and `a085a3ce7`), and these counts are not re-measured here. ## Revision record -| Date | Change | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | -| 2026-10-15 | Fixes from review round 1. The addresses with their methods, forms, statuses and headers, the defaults and the one switch, what runs around each handler, the data page's full header set and its protection from an operator's headers, the failure paths, the rollout, the tests and the untested paths, the open questions, the header and the acceptance lines brought to the series head, and the review log. | -| 2026-10-16 | Fixes from review round 2. The statement that the preparers and filters never run for these addresses on Fastly and Spin, where a request with a method other than `GET` goes on Cloudflare, the matching wording of the second open question, and the review log. | +| Date | Change | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-15 | Fixes from review round 1. The addresses with their methods, forms, statuses and headers, the defaults and the one switch, what runs around each handler, the data page's full header set and its protection from an operator's headers, the failure paths, the rollout, the tests and the untested paths, the open questions, the header and the acceptance lines brought to the series head, and the review log. | +| 2026-10-16 | Fixes from review round 2. The statement that the preparers and filters never run for these addresses on Fastly and Spin, where a request with a method other than `GET` goes on Cloudflare, the matching wording of the second open question, and the review log. | +| 2026-10-26 | The permissions answer names the policy file row 4 selects, the configuration view shows its name and SHA-256 but never its text, a pattern under `permissions.text` is refused, and checks count `permissions.sha256` as present, as this row's code does. A failed load answers its error's status on every adapter, Spin included, because row 4 moved Spin off a fixed `503`. The status names the pull request but not as a draft. | ## Review log diff --git a/docs/superpowers/specs/2026-10-17-config-migrate-design.md b/docs/superpowers/specs/2026-10-17-config-migrate-design.md index bdacb993c1..abf232a8a5 100644 --- a/docs/superpowers/specs/2026-10-17-config-migrate-design.md +++ b/docs/superpowers/specs/2026-10-17-config-migrate-design.md @@ -1,17 +1,17 @@ # Design Spec: `ts config migrate` rewrites a settings document written for main into the shape the stack reads -**Status:** Proposed, 2026-10-17, revised 2026-10-18 after review rounds 1 and 2. A pull request of the stack, not yet raised, for Tech Lab review. +**Status:** Proposed, 2026-10-17, revised 2026-10-18 after review rounds 1 and 2, and revised 2026-10-26 for the permission policy, which is selected when a service is deployed. Raised as IABTechLab/trusted-server#1409, one of the stack's pull requests, row 51 of the stack in IABTechLab/trusted-server#777, opened on 2026-10-17, for Tech Lab review. References to the stack's code are to the code of IABTechLab/trusted-server#1409, by file and function, with a line number where one helps. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-14-source-modules-design.md`, `2026-10-13-middleware-settings-design.md`, `2026-10-14-rsc-flight-middleware-design.md`, `2026-10-15-geo-module-request-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 -**Related PRs:** none yet +**Related PRs:** IABTechLab/trusted-server#1409 **Issues:** none -**Last updated:** 2026-10-18 +**Last updated:** 2026-10-26 -> **Why this spec exists.** The changelog of the stack lists 39 breaking entries under Unreleased, where main lists 10, and a settings document written for Tech Lab main does not load under the stack. Every operator would rewrite their document by hand from the changelog. `ts config migrate` makes the rewrite automatic where the document holds everything the rewrite needs, and says what is left where it does not. +> **Why this spec exists.** The changelog of the stack lists 41 breaking entries under Unreleased, where main lists 10, and a settings document written for Tech Lab main does not load under the stack. Every operator would rewrite their document by hand from the changelog. `ts config migrate` makes the rewrite automatic where the document holds everything the rewrite needs, and says what is left where it does not. ## 1. Problem -A document written for Tech Lab main (`upstream/main` at `5bc388625`) is refused at startup by the stack. `[integrations]` is gone, the origin settings moved to `[source]`, the auction providers moved to `[demand]`, `[[handlers]]` is refused, and a module changes a page only where a `[[fetch]]` or `[[serve]]` entry names its middleware. The changelog lists each change under Unreleased, with 39 entries marked breaking, and the settings code holds a few more that no changelog line names. An operator who upgrades has to read all of them, work out which apply to their document, and rewrite the document by hand without losing the comments they wrote. +A document written for Tech Lab main (`upstream/main` at `5bc388625`) is refused at startup by the stack. `[integrations]` is gone, the origin settings moved to `[source]`, the auction providers moved to `[demand]`, `[[handlers]]` is refused, and a module changes a page only where a `[[fetch]]` or `[[serve]]` entry names its middleware. The changelog lists each change under Unreleased, with 41 entries marked breaking, and the settings code holds a few more that no changelog line names. An operator who upgrades has to read all of them, work out which apply to their document, and rewrite the document by hand without losing the comments they wrote. Most of the rewrite is mechanical, because the old shape says everything the new one needs. A few changes need a decision only the operator can make, or a value the document does not hold, a few change behavior in a way no setting can express, and the location settings depend on the adapter the deployment runs on. @@ -26,7 +26,7 @@ Most of the rewrite is mechanical, because the old shape says everything the new - `--check` changes nothing, prints nothing on stdout and does not validate. It exits 1 when the document needs migrating, being a rewrite to make or an item left to do, so CI can hold a document to the current shape. - `--write` and `--check` cannot be used together. -The command works on the file alone. It reads no `edgezero.toml`, applies no `TRUSTED_SERVER__` overlay and asks no platform anything, because the rewrite is a property of the document and the adapter. It validates the result as `ts audit generate` validates a document it is about to write, by parsing the text into `TrustedServerAppConfig` with no manifest and running `validate_excluding_secrets` with the modules the tool links (`crates/trusted-server-cli/src/commands/audit/generate/validate.rs` lines 55 to 66). That is the push-time validation `ts config validate` runs, and `ts config validate` also applies the overlay, checks secrets and runs EdgeZero's adapter checks (EdgeZero v0.0.9, `crates/edgezero-cli/src/config.rs` lines 245 to 277). Neither runs the checks made only when a deployment starts, being the checks core makes once the settings load that push-time validation does not repeat, of the old Edge Cookie layout, the `[ec]`, `[device]` and `[geo]` module selections, the permission policy and the jurisdiction acknowledgment (`Settings::finalize_deserialized`, core `settings.rs` lines 4355, 4356 and 4360 to 4365, where lines 4357 to 4359 run at push time too), and each adapter's checks as it builds its state. The command reports the location settings among those as items left to do, because every document written for main gains an Edge Cookie module from its passphrase, and whether that document then starts depends on its geo settings and its adapter (§3). +The command works on the file alone. It reads no `edgezero.toml`, applies no `TRUSTED_SERVER__` overlay and asks no platform anything, because the rewrite is a property of the document and the adapter. It validates the result as `ts audit generate` validates a document it is about to write, by parsing the text into `TrustedServerAppConfig` with no manifest and running `validate_excluding_secrets` with the modules the tool links (`crates/trusted-server-cli/src/commands/audit/generate/validate.rs` lines 55 to 66). That is the push-time validation `ts config validate` runs, and `ts config validate` also applies the overlay, checks secrets and runs EdgeZero's adapter checks (EdgeZero v0.0.10, `crates/edgezero-cli/src/config.rs` lines 245 to 277). Neither runs the checks made only when a deployment starts, being the checks core makes once the settings load that push-time validation does not repeat. Those are the checks of the old Edge Cookie layout, the `[ec]`, `[device]` and `[geo]` module selections and the jurisdiction acknowledgment (`Settings::finalize_deserialized` in core `settings.rs`, where `validate_resolve_allowed_origins`, `validate_module_sections` and `validate_phase_entries` run at push time too), the runtime loader's check of the permission policy file against the digest the push wrote (`load_domain_policy` in core `config_payload.rs`), and each adapter's checks as it builds its state. The command reports the location settings among those as items left to do, because every document written for main gains an Edge Cookie module from its passphrase, and whether that document then starts depends on its geo settings and its adapter (§3). The command's validation leaves the policy check out, because the command takes no `--permissions` (`ConfigMigrateArgs` in `crates/trusted-server-cli/src/commands/config/migrate/mod.rs`) and a policy file is the publisher's to supply, so a missing policy is a `to do:` line (§2.2) and a document can still reach exit status 0 once the operator has named a policy. `ts audit generate`, which shares the validation (`check_candidate` in the same `validate.rs`), leaves it out as well. `ts config validate` checks the policy against the files `--permissions` supplies, and without `--permissions` says the policy was not checked (`run_validate_with` in `crates/trusted-server-cli/src/commands/config/policy.rs`), as row 4 of the stack (IABTechLab/trusted-server#1345) builds §3.1 of `2026-07-30-permission-model-design.md`. The command's validation, like that of `ts audit generate`, counts `permissions.sha256` as present and refuses an `[inspect]` pattern under `permissions.text`, as every pattern check does (`validate_patterns` in core `inspect/config.rs`, and `2026-10-12-inspection-endpoints-design.md`). Each line on stderr starts with one of three words. @@ -40,50 +40,54 @@ The exit status is 0 when nothing is left to do and the migrated document passes The table lists every breaking entry in `CHANGELOG.md` under Unreleased that concerns the document or how a deployment upgrades, and the changes in the settings code that no changelog line names, marked "settings code". Entries for crates outside core, for the `ts` tool and for request handling with no setting are grouped in the last rows. -| Change | Handling | -| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `[integrations]` is gone, and each integration is a module selected in the section of its type | Rewritten. An enabled `[integrations.]` becomes the module in its section, `module` or `modules`, with its settings in `[
.]` and `enabled` dropped. One that was not enabled is commented out under its new name. `enabled` follows main's default for each id. | -| `[auction.providers.]` moved to `[demand] modules` and `[demand.]` | Rewritten. The name takes `_` for `-`, `profile` becomes `implementation` by module path, `profile_config` moves into the table, and `protocol`, which main only took as `openrtb-2.6`, is dropped. | -| `[auction.bidders.] provider` is `module`, naming the demand source | Rewritten, with the demand source's new name. | -| `[auction] mediator = "adserver_mock"` and `[integrations.adserver_mock]` moved to `[ad-server]` | Rewritten to `[ad-server] module = "iabtl"` and `[ad-server.iabtl]`. A mock ad server no mediator names is commented out. | -| `[integrations.aps] rendering_mode` moved to the `auction.aps` demand table | Rewritten into each `auction.aps` demand table. Main read it only from an enabled table. | -| `include_mediator_response` is `include_adserver_response` | Rewritten. | -| The origin settings moved from `[publisher]` to `[source]` | Rewritten to `[source] module = "origin"` and `[source.origin]`, with `origin_url` as `url`, `origin_host_header_override` as `host_header`, `origin_auth` as `auth` and `directory_index` unchanged. Check, below, where `host_header` is set. | -| An asset route's `origin_url` and `auth` moved to its `source` table | Rewritten to `[proxy.asset_routes.source] module = "origin"` and `[proxy.asset_routes.source.origin]`. | -| The signer spelling `s3_sig_v4` is no longer read | Rewritten to `s3_sigv4` in every `auth` table. | -| `[[handlers]]` and HTTP Basic authentication are removed | Rewritten by removing an entry where every path its pattern can match is one the stack closes, where nothing is served now, as the paragraph below the table defines. Main matched a pattern anywhere in the path (main core `settings.rs` line 734), so `^/admin/keys` alone also guarded `/admin/keystore`, which the stack forwards to the origin, and so did `^/admin/keys/?`, whose `?` lets the `/` match nothing, and such an entry is to do, below. | -| `[request_signing]` does not take `config_store_id` or `secret_store_id` | Rewritten by removing both. Check, below, for where the ids go. | -| `[ec] passphrase` is read as deprecated (settings code) | Rewritten to `[ec] module = "hmac"` and `[ec.hmac] passphrase`, where the stack documents it. The deprecated key still loads, with a warning at startup, for one release. | -| `[ec.providers.]` is `[ec.]`, and the module names `host-signals` and `client-fixed` are `host_signals` and `client_fixed` | Rewritten, in the tables, the selection and an `implementation` line. | -| `[permission_signal] sources` is `[permission-signal] modules`, with `gpp-sale-opt-out` and `gpp_sale_opt_out` as `gpp` and `us_privacy` as `us-privacy` | Rewritten. | -| `[device] module = "fastly"` is `"platform"` | Rewritten. | -| The host's location lookup runs only where `[geo]` selects it (settings code) | Rewritten for a document written for main on Fastly and Cloudflare by selecting `[geo] module = "platform"`, the lookup main ran on every request there. Check, on Cloudflare, that the lookup now reads the reader's region, which main's did not (§4). On Axum and Spin main's lookup located no request, and those adapters refuse the selection at startup, so the command selects no geo module there, and the Edge Cookie module that follows is to do, below. | -| No response carries `x-geo-*` unless `headers.geo` is selected and placed | Check, for a document written for main, saying what main's responses carried on the adapter and, on Fastly, how to keep the headers. The command never selects `headers.geo`, because the headers carry the reader's location (§4). | -| `[tinybird]` is `[analytics.tinybird]`, selected by `[analytics] module` | Rewritten, with `enabled` dropped. A table that was not enabled is commented out. Check, below, for the consent gate. | -| A module's page change runs only where a `[[fetch]]` or `[[serve]]` entry names its middleware | Rewritten for a document written for main by adding one entry for each phase and media type the selected modules supply middleware for, naming that middleware in the order the linked builders supply it, which is the order `trusted-server.example.toml` shows. A document whose modules supply no fetch middleware gets no `[[fetch]]` entry. A document in the stack's shape is left as written, because it loads with no entry, with a warning naming each middleware no entry names (CHANGELOG line 51), and whether its modules change pages is the operator's choice. | -| The Flight rewriter is the middleware `framework.nextjs.flight` | Rewritten by adding a `[[fetch]]` entry for `text/x-component`, among the page entries above, where the Next.js module is selected. Check, below, where it is not. | -| The `/robots.txt` route is the `robots_txt` middleware | Rewritten by adding a `[[fetch]]` entry for `text/plain` on `/robots.txt` to a document with `[robots-txt]` and no entry naming `robots_txt`, which the stack refuses. | -| `ad-server.mock` is `ad-server.iabtl`, `[adserver]` is `[ad-server]` (settings code for the table), and `ads-txt.sellers` is `ads-txt.iabtl` | Rewritten in the selection, the table, an `implementation` line and a `[[fetch]]` middleware name. A selected `[ads-txt]` with no entry naming its middleware gets a `[[fetch]]` entry for `text/plain` on `/ads.txt`, because the module refuses that document (`crates/ads-txt/iabtl/src/lib.rs` lines 154 to 175). | -| A `[[handlers]]` entry whose pattern can match a path the stack does not close | To do. Nothing asks for a password now, so the operator puts one on those paths at the origin or in the hosting platform and then removes the entry. The command never removes it, because the stack refuses the list so that no deployment believes a path is protected. | -| An Edge Cookie module with no geo module, and `[geo] module = "platform"` on Axum or Spin (settings code and the adapters) | To do, for any document. Startup on every adapter refuses the first unless `[geo] assume_single_jurisdiction = true` accepts that every request takes the top of the `permissions.yaml` rules (core `settings.rs` lines 1736 to 1759), and the Axum and Spin adapters refuse the second, because neither host has a location lookup. `ts config validate` checks neither. The operator chooses between a geo module and the acknowledgment, because the stack asks for that choice to be made rather than assumed. | -| `[consent.gdpr] applies_in` and `[consent.us_states] privacy_states` are not read (settings code) | To do, as a setting the stack ignores whose value the operator has to carry elsewhere. The stack loads both tables and reads neither (core `consent_config.rs` lines 15 to 28). The places each covers are the `jurisdiction` values in the `permissions.yaml` compiled into the build, so the operator carries any change of theirs there and removes the table. The command keeps the table, because the list is the value the operator has to carry, so `--check` fails on a document that still carries one, which is intended, as a table nothing reads invites edits that change nothing. | -| A `template_cache_key_cookies` entry under the public prefix is refused | To do, naming the setting. | -| A setting the command cannot place | To do, leaving it where it is. These are an `[integrations.]` naming no module, an `enabled` that is not a boolean, `[integration]`, `rendering_mode` with no `auction.aps` source, a mediator with no mock ad server table, an unknown `profile`, a `profile_config` setting named as one of its provider's own, and both forms of a setting. | -| A key in a module table that main ignored and the module now refuses, in the tables of `gpt`, `didomi`, `lockr`, `permutive`, `sourcepoint`, `testlight`, `nextjs`, `js_asset_proxy`, `google_tag_manager`, `adserver_mock` and `[tinybird]` (settings code) | Reported through the validation result, and `ts config validate` names the key. | -| No response carries `x-ts-version` or the platform's version unless `service_version` is named | Check. The command does not name the middleware, because the changelog lists the version's removal under Security. | -| A path in `origin_url` is kept, and the request path is added after it | Check, where `[source.origin] url` has a path, which main dropped. | -| Where `host_header` is set, the page's own addresses under that host are rewritten, where those under the `url`'s host were (CHANGELOG line 26) | Check, where `origin_host_header_override` moved to `[source.origin] host_header`. | -| Core no longer moves the origin's address in a Flight response | Check, where the Next.js module is not selected. Selecting it changes every HTML page, so the command does not. | -| A telemetry row is sent only for a reader whose permissions include `analytics.ad_reporting.measure_ad_performance` | Check, where the table was enabled. | -| The signing store ids go to `ts keys rotate` and `ts keys deactivate` | Check, where the command removed them. | -| The Cloudflare and Spin adapters require their key-value store | Out of scope. The binding lives in `wrangler.toml` and the Spin manifest, outside the document. Main's `wrangler.toml` declares `TRUSTED_SERVER_KV` and its `spin.toml` grants `default`, and §6 says to keep both. | -| Breaking entries for crates outside core: platform headers, staging, config store chunks, backend naming, the auction protocol, consent decoders, identifier modules, geo modules, cookie constants, the page traits, the modules crate and origin signers | Out of scope. They change code, not the document. | -| The seam probe is `testing.iabtl`, where it was `testing.seam-probe` | Out of scope. It is a test fixture that no production build links (`crates/testing/iabtl/src/lib.rs` lines 15 to 20), so no deployment's document selects it. | -| Breaking entries for the `ts` tool: `ts cache purge` and `ts prebid client` | Out of scope. | -| Breaking entries for requests with no setting: the admin path check and the removed admin, purge and key rotation routes | Out of scope. | -| Environment variable names, such as `TRUSTED_SERVER__SOURCE__ORIGIN__URL` | Out of scope. They live outside the document, and §6 says to rename them. | -| Main's own breaking entries the stack keeps: the auction maps, APS OpenRTB, `site.page`, `bid_param_zone_overrides`, string scalars, Sourcepoint and creative sanitization | Nothing to do. A document written for main already meets them, and the maps move again under the second row. | -| Earlier shapes of the stack that neither the changelog nor the settings code names, such as `[ec] provider` | Out of scope. They never reached a release, and nothing tells an operator about them. | +| Change | Handling | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `[integrations]` is gone, and each integration is a module selected in the section of its type | Rewritten. An enabled `[integrations.]` becomes the module in its section, `module` or `modules`, with its settings in `[
.]` and `enabled` dropped. One that was not enabled is commented out under its new name. `enabled` follows main's default for each id. | +| `[auction.providers.]` moved to `[demand] modules` and `[demand.]` | Rewritten. The name takes `_` for `-`, `profile` becomes `implementation` by module path, `profile_config` moves into the table, and `protocol`, which main only took as `openrtb-2.6`, is dropped. | +| `[auction.bidders.] provider` is `module`, naming the demand source | Rewritten, with the demand source's new name. | +| `[auction] mediator = "adserver_mock"` and `[integrations.adserver_mock]` moved to `[ad-server]` | Rewritten to `[ad-server] module = "iabtl"` and `[ad-server.iabtl]`. A mock ad server no mediator names is commented out. | +| `[integrations.aps] rendering_mode` moved to the `auction.aps` demand table | Rewritten into each `auction.aps` demand table. Main read it only from an enabled table. | +| `include_mediator_response` is `include_adserver_response` | Rewritten. | +| The origin settings moved from `[publisher]` to `[source]` | Rewritten to `[source] module = "origin"` and `[source.origin]`, with `origin_url` as `url`, `origin_host_header_override` as `host_header`, `origin_auth` as `auth` and `directory_index` unchanged. Check, below, where `host_header` is set. | +| An asset route's `origin_url` and `auth` moved to its `source` table | Rewritten to `[proxy.asset_routes.source] module = "origin"` and `[proxy.asset_routes.source.origin]`. | +| The signer spelling `s3_sig_v4` is no longer read | Rewritten to `s3_sigv4` in every `auth` table. | +| `[[handlers]]` and HTTP Basic authentication are removed | Rewritten by removing an entry where every path its pattern can match is one the stack closes, where nothing is served now, as the paragraph below the table defines. Main matched a pattern anywhere in the path (main core `settings.rs` line 734), so `^/admin/keys` alone also guarded `/admin/keystore`, which the stack forwards to the origin, and so did `^/admin/keys/?`, whose `?` lets the `/` match nothing, and such an entry is to do, below. | +| `[request_signing]` does not take `config_store_id` or `secret_store_id` | Rewritten by removing both. Check, below, for where the ids go. | +| `[ec] passphrase` is read as deprecated (settings code) | Rewritten to `[ec] module = "hmac"` and `[ec.hmac] passphrase`, where the stack documents it. The deprecated key still loads, with a warning at startup, for one release. | +| `[ec.providers.]` is `[ec.]`, and the module names `host-signals` and `client-fixed` are `host_signals` and `client_fixed` | Rewritten, in the tables, the selection and an `implementation` line. | +| `[permission_signal] sources` is `[permission-signal] modules`, with `gpp-sale-opt-out` and `gpp_sale_opt_out` as `gpp` and `us_privacy` as `us-privacy` | Rewritten. | +| `[device] module = "fastly"` is `"platform"` | Rewritten. | +| The host's location lookup runs only where `[geo]` selects it (settings code) | Rewritten for a document written for main on Fastly and Cloudflare by selecting `[geo] module = "platform"`, the lookup main ran on every request there. Check, on Cloudflare, that the lookup now reads the reader's region, which main's did not (§4). On Axum and Spin main's lookup located no request, and those adapters refuse the selection at startup, so the command selects no geo module there, and the Edge Cookie module that follows is to do, below. | +| No response carries `x-geo-*` unless `headers.geo` is selected and placed | Check, for a document written for main, saying what main's responses carried on the adapter and, on Fastly, how to keep the headers. The command never selects `headers.geo`, because the headers carry the reader's location (§4). | +| `[tinybird]` is `[analytics.tinybird]`, selected by `[analytics] module` | Rewritten, with `enabled` dropped. A table that was not enabled is commented out. Check, below, for the consent gate. | +| A module's page change runs only where a `[[fetch]]` or `[[serve]]` entry names its middleware | Rewritten for a document written for main by adding one entry for each phase and media type the selected modules supply middleware for, naming that middleware in the order the linked builders supply it, which is the order `trusted-server.example.toml` shows. A document whose modules supply no fetch middleware gets no `[[fetch]]` entry. A document in the stack's shape is left as written, because it loads with no entry, with a warning naming each middleware no entry names (CHANGELOG line 51), and whether its modules change pages is the operator's choice. | +| The Flight rewriter is the middleware `framework.nextjs.flight` | Rewritten by adding a `[[fetch]]` entry for `text/x-component`, among the page entries above, where the Next.js module is selected. Check, below, where it is not. | +| The `/robots.txt` route is the `robots_txt` middleware | Rewritten by adding a `[[fetch]]` entry for `text/plain` on `/robots.txt` to a document with `[robots-txt]` and no entry naming `robots_txt`, which the stack refuses. | +| `ad-server.mock` is `ad-server.iabtl`, `[adserver]` is `[ad-server]` (settings code for the table), and `ads-txt.sellers` is `ads-txt.iabtl` | Rewritten in the selection, the table, an `implementation` line and a `[[fetch]]` middleware name. A selected `[ads-txt]` with no entry naming its middleware gets a `[[fetch]]` entry for `text/plain` on `/ads.txt`, because the module refuses that document (`crates/ads-txt/iabtl/src/lib.rs` lines 154 to 175). | +| A `[[handlers]]` entry whose pattern can match a path the stack does not close | To do. Nothing asks for a password now, so the operator puts one on those paths at the origin or in the hosting platform and then removes the entry. The command never removes it, because the stack refuses the list so that no deployment believes a path is protected. | +| An Edge Cookie module with no geo module, and `[geo] module = "platform"` on Axum or Spin (settings code and the adapters) | To do, for any document. Startup on every adapter refuses the first unless `[geo] assume_single_jurisdiction = true` accepts that every request takes the top of the `permissions.yaml` rules (`GeoConfig::validate_jurisdiction_acknowledgment` in core `settings.rs`), and the Axum and Spin adapters refuse the second, because neither host has a location lookup. `ts config validate` checks neither. The operator chooses between a geo module and the acknowledgment, because the stack asks for that choice to be made rather than assumed. | +| The permission policy is a file the settings name in `[permissions] policy` and the push supplies (`2026-07-30-permission-model-design.md` §3.1, built by row 4, IABTechLab/trusted-server#1345) | To do, for any document whose `[permissions] policy` is missing or empty (`permission_policy` in `rules.rs`). No policy ships with Trusted Server, so the command never chooses one, adds nothing to `[permissions]` and writes no policy file. The line is the same on every adapter (`POLICY_TO_DO`). It says that a domain whose settings name no policy does not start, to name the publisher's file there by its stem, as `policy = "standard"` names `standard.yaml`, and to supply the file with `ts config push --permissions `. It does not say that on Axum the file goes in the folder `TRUSTED_SERVER_PERMISSIONS_DIR` names (`2026-07-30-permission-model-design.md` §3.1, item 4). So a run on a document written for main that names no policy exits 1, though the migrated document passes validation. The operator may name the policy before or after migrating, because a document written for main that names its policy is still taken as written for main (below the table). | +| `[consent.gdpr] applies_in` and `[consent.us_states] privacy_states` are not read (settings code) | To do, as a setting the stack ignores whose value the operator has to carry elsewhere. The stack loads both tables and reads neither (core `consent_config.rs` lines 15 to 28). The places each covers are the `jurisdiction` values in the policy file the document names, so the operator carries any change of their own into that file and removes the table. The command keeps the table, because the list is the value the operator has to carry, so `--check` fails on a document that still carries one, which is intended, as a table nothing reads invites edits that change nothing. | +| A `template_cache_key_cookies` entry under the public prefix is refused | To do, naming the setting. | +| A setting the command cannot place | To do, leaving it where it is. These are an `[integrations.]` naming no module, an `enabled` that is not a boolean, `[integration]`, `rendering_mode` with no `auction.aps` source, a mediator with no mock ad server table, an unknown `profile`, a `profile_config` setting named as one of its provider's own, and both forms of a setting. | +| A key in a module table that main ignored and the module now refuses, in the tables of `gpt`, `didomi`, `lockr`, `permutive`, `sourcepoint`, `testlight`, `nextjs`, `js_asset_proxy`, `google_tag_manager`, `adserver_mock` and `[tinybird]` (settings code) | Reported through the validation result, and `ts config validate` names the key. | +| No response carries `x-ts-version` or the platform's version unless `service_version` is named | Check. The command does not name the middleware, because the changelog lists the version's removal under Security. | +| A path in `origin_url` is kept, and the request path is added after it | Check, where `[source.origin] url` has a path, which main dropped. | +| Where `host_header` is set, the page's own addresses under that host are rewritten, where those under the `url`'s host were (CHANGELOG line 26) | Check, where `origin_host_header_override` moved to `[source.origin] host_header`. | +| Core no longer moves the origin's address in a Flight response | Check, where the Next.js module is not selected. Selecting it changes every HTML page, so the command does not. | +| A telemetry row is sent only for a reader whose permissions include `analytics.ad_reporting.measure_ad_performance` | Check, where the table was enabled. | +| The signing store ids go to `ts keys rotate` and `ts keys deactivate` | Check, where the command removed them. | +| The Cloudflare and Spin adapters require their key-value store | Out of scope. The binding lives in `wrangler.toml` and the Spin manifest, outside the document. Main's `wrangler.toml` declares `TRUSTED_SERVER_KV` and its `spin.toml` grants `default`, and §6 says to keep both. | +| The Cloudflare adapter reads its settings from its config store, the Workers KV namespace `ts config push` writes, and no longer reads the `TRUSTED_SERVER_CONFIG` variable | Out of scope. The binding lives in `wrangler.toml`, outside the document, and §6 says to bind the namespace and push into it. | +| Breaking entries for crates outside core: platform headers, staging, config store chunks, backend naming, the auction protocol, consent decoders, identifier modules, geo modules, cookie constants, the page traits, the modules crate and origin signers | Out of scope. They change code, not the document. | +| The seam probe is `testing.iabtl`, where it was `testing.seam-probe` | Out of scope. It is a test fixture that no production build links (the crate comment of `crates/testing/iabtl/src/lib.rs`), so no deployment's document selects it. | +| Breaking entries for the `ts` tool: `ts cache purge` and `ts prebid client` | Out of scope. | +| Breaking entries for requests with no setting: the admin path check and the removed admin, purge and key rotation routes | Out of scope. | +| Environment variable names, such as `TRUSTED_SERVER__SOURCE__ORIGIN__URL` | Out of scope. They live outside the document, and §6 says to rename them. | +| Main's own breaking entries the stack keeps: the auction maps, APS OpenRTB, `site.page`, `bid_param_zone_overrides`, string scalars, Sourcepoint and creative sanitization | Nothing to do. A document written for main already meets them, and the maps move again under the second row. | +| Earlier shapes of the stack that neither the changelog nor the settings code names, such as `[ec] provider` | Out of scope. They never reached a release, and nothing tells an operator about them. | + +Where the table and §4 say the places are the `jurisdiction` values in the policy file the document names, they describe §3.1 of `2026-07-30-permission-model-design.md`, and the command's line for each list says the same (`consent_lists` in `rules.rs`). A `[[handlers]]` pattern guards only paths the stack closes when it starts `^/_ts/admin`, because every path that begins `/_ts/admin` is closed, or starts `^/admin/keys` followed by `/`, `$`, `(?:/|$)` or `(/|$)`, because of `/admin/keys` only the address itself and the paths below `/admin/keys/` are closed (core `closed_paths.rs` lines 18, 25 and 75 to 81), and when nothing after that prefix or bound widens what it matches, which rules out the following. @@ -93,7 +97,7 @@ A `[[handlers]]` pattern guards only paths the stack closes when it starts `^/_t The command reads the pattern's text and not what the pattern means, so where the text alone cannot show that every path is closed it leaves the entry to do, as it does for `^/admin/keys/?$` and `^/_ts/admin/[a-z]+`, although every path either matches is closed. -A document is taken as written for main when it has `[publisher]` and every table at its top level is one main read, which are `publisher`, `tester_cookie`, `trusted_client_ip`, `ec`, `integrations`, `handlers`, `response_headers`, `request_signing`, `rewrite`, `auction`, `consent`, `cache`, `proxy`, `creative_opportunities`, `image_optimizer`, `tinybird` and `debug`. Main refused any other table, and every migrated document has `[source]`, so a second run never takes it as written for main. The geo row, the page entry rows, and the checks about `x-ts-version`, the location headers and Flight responses apply to such a document alone, so a document already in the stack's shape keeps its own geo settings and page entries, or the lack of them. +A document is taken as written for main when it has `[publisher]` and every table at its top level is one main read, which are `publisher`, `tester_cookie`, `trusted_client_ip`, `ec`, `integrations`, `handlers`, `response_headers`, `request_signing`, `rewrite`, `auction`, `consent`, `cache`, `proxy`, `creative_opportunities`, `image_optimizer`, `tinybird` and `debug`, or is `permissions`, which main did not read and an operator may add before migrating to name the policy file (`written_for_main` and `MAIN_TABLES` in `rules.rs`). Main refused any other table, and every migrated document has `[source]`, so a second run never takes it as written for main. The geo row, the page entry rows, and the checks about `x-ts-version`, the location headers and Flight responses apply to such a document alone, so a document already in the stack's shape keeps its own geo settings and page entries, or the lack of them. A document written for main that already names its policy migrates as the same document without that table does, with the same rewrites, the same items apart from the one that names the policy, and the same check lines, and the result keeps its `[permissions]` table (`a_document_written_for_main_that_names_its_policy_migrates_as_main`). The rules for `[permission_signal]`, `[ec.providers]`, `[adserver]`, `ad-server.mock`, `ads-txt.sellers`, `[device] module = "fastly"` and `[robots-txt]` with no entry rewrite shapes no document written for main holds. Each was the shape of an earlier pull request of this stack and never of a release, and each is named by a changelog entry or refused with directions by the settings code, so a deployment that ran a build of one of those pull requests holds a document in that shape. The rules save its operator the rewrite by hand, and an earlier shape that nothing names stays out of scope. @@ -120,6 +124,7 @@ Migrating a migrated document changes nothing, because each rule matches only a - It never sets `[geo] assume_single_jurisdiction`, because the stack asks the operator to accept that every request takes the top of the `permissions.yaml` rules rather than have that assumed. - It never names `service_version` or selects `headers.geo`, and it never selects the Next.js module for a document that did not enable `[integrations.nextjs]`. It changes a value the document holds only where the stack renamed what the value names, which is a module, a middleware, a signer, an implementation or a demand source. - It never writes the file without `--write`, and never writes it when nothing changes. +- It never chooses a permission policy and never writes a policy file. No policy ships with Trusted Server, and the tool carries none. ## 3. The adapters @@ -132,7 +137,7 @@ The command runs in the tool and does the same on every machine, but the documen | Axum | None, as the lookup answered nothing (main `crates/trusted-server-adapter-axum/src/platform.rs` lines 183 to 187) | `x-geo-info-available: false` (main `crates/trusted-server-adapter-axum/src/middleware.rs` lines 133 to 137) | No geo module, an item left to do for the Edge Cookie module, and a check line on the header | | Spin | None, as the lookup answered nothing (main `crates/trusted-server-adapter-spin/src/platform.rs` lines 757 to 761) | `x-geo-info-available: false` (main `crates/trusted-server-adapter-spin/src/middleware.rs` lines 167 to 175) | As on Axum | -The stack's Axum and Spin adapters refuse `[geo] module = "platform"` at startup naming the setting, because neither host has a lookup (`crates/trusted-server-adapter-axum/src/app.rs` lines 277 and 326, `crates/trusted-server-adapter-spin/src/app.rs` lines 252 and 286, CHANGELOG line 42). Every document main ran with has an Edge Cookie passphrase (main core `settings.rs` lines 545 and 625 to 633), which becomes `[ec] module = "hmac"`, and startup on every adapter refuses an Edge Cookie module with no geo module unless `[geo] assume_single_jurisdiction = true` accepts that every request takes the top of the `permissions.yaml` rules. On Axum and Spin the operator chooses between that acceptance and a geo module a linked crate offers, so the command leaves the choice to do. Writing `platform` there would report a document as passing that the adapter refuses, and writing the acknowledgment would make the choice the stack asks the operator to make. +The stack's Axum and Spin adapters refuse `[geo] module = "platform"` at startup naming the setting, because neither host has a lookup (`refuse_platform_geo` in `crates/trusted-server-adapter-axum/src/app.rs` and in `crates/trusted-server-adapter-spin/src/app.rs`, CHANGELOG line 42). Every document main ran with has an Edge Cookie passphrase (main core `settings.rs` lines 545 and 625 to 633), which becomes `[ec] module = "hmac"`, and startup on every adapter refuses an Edge Cookie module with no geo module unless `[geo] assume_single_jurisdiction = true` accepts that every request takes the top of the `permissions.yaml` rules. On Axum and Spin the operator chooses between that acceptance and a geo module a linked crate offers, so the command leaves the choice to do. Writing `platform` there would report a document as passing that the adapter refuses, and writing the acknowledgment would make the choice the stack asks the operator to make. The rest of the document is the same on every adapter. Only the Fastly adapter sends the telemetry an `[analytics]` selection names, as only Fastly's did on main (CHANGELOG lines 20 and 48), and the Cloudflare and Spin adapters need their key-value store (§6). @@ -142,32 +147,33 @@ A deployment that ran one document on two adapters, as main's quick start pushes - **No value is printed.** Every line on stderr names a setting and holds no value from the document, because a document can hold secrets, and the validation result gives only the byte offset or the phase that refused. Lines do carry table names the operator chose, such as an integration id or a bidder code (the test `no_line_carries_a_value_from_the_document`). - **No location header is added.** The command never selects `headers.geo`, which writes the reader's location in the `x-geo-*` headers on the responses a `[[serve]]` entry covers. Main wrote those headers on Fastly alone, and on Cloudflare wrote only `x-geo-info-available`, so selecting the middleware for every document written for main would send the reader's location on Cloudflare, where main sent none. Whether pages carry the reader's location is the operator's decision, as the stack made it one (CHANGELOG line 41), so the check line for each adapter says what main's responses carried and, on Fastly, how to keep the headers. -- **The location the consent rules see.** On Fastly and Cloudflare the document keeps the lookup main ran, so the `permissions.yaml` rules see the location main's jurisdiction check saw, where without a geo module every request would take the top of those rules. Main had no `permissions.yaml`. Its check compared the country with `[consent.gdpr] applies_in` and the region with `[consent.us_states] privacy_states` (main core `consent/jurisdiction.rs` lines 41 to 79), and the jurisdiction it found fed the Edge Cookie consent check and the OpenRTB `regs.gdpr` value (main core `ec/consent.rs` lines 11 to 22, `auction/openrtb.rs` lines 652 to 665). On Cloudflare the stack's lookup also reads the region, from `cf-region-code` where Cloudflare sends it, where main's lookup read none (`crates/trusted-server-adapter-cloudflare/src/platform.rs` lines 709 to 721 and 782 to 787), so the region rules of `permissions.yaml` can apply there now, such as the `us-state` jurisdiction it gives each US state it lists where the country gives `non-regulated` (`config/permissions/sample.yaml` lines 421 to 427), and a check line printed with the geo rewrite says so. On Axum and Spin main located no request and the document locates none, and an Edge Cookie module runs there only once the operator accepts that with `[geo] assume_single_jurisdiction = true` or selects a geo module. +- **The location the consent rules see.** On Fastly and Cloudflare the document keeps the lookup main ran, so the `permissions.yaml` rules see the location main's jurisdiction check saw, where without a geo module every request would take the top of those rules. Main had no `permissions.yaml`. Its check compared the country with `[consent.gdpr] applies_in` and the region with `[consent.us_states] privacy_states` (main core `consent/jurisdiction.rs` lines 41 to 79), and the jurisdiction it found fed the Edge Cookie consent check and the OpenRTB `regs.gdpr` value (main core `ec/consent.rs` lines 11 to 22, `auction/openrtb.rs` lines 652 to 665). On Cloudflare the stack's lookup also reads the region, from `cf-region-code` where Cloudflare sends it, where main's lookup read none (`CloudflareGeo` and `build_geo` in `crates/trusted-server-adapter-cloudflare/src/platform.rs`), so the region rules of `permissions.yaml` can apply there now, such as the `us-state` jurisdiction the repository's sample gives each US state it lists where the country gives `non-regulated` (`config/permissions/sample.yaml` lines 421 to 427), and a check line printed with the geo rewrite says so. On Axum and Spin main located no request and the document locates none, and an Edge Cookie module runs there only once the operator accepts that with `[geo] assume_single_jurisdiction = true` or selects a geo module. - **Passwords.** The command removes a `[[handlers]]` entry only where every path its pattern can match is one the stack closes, read as the paragraph below the §2.2 table says, so a publisher path such as `/admin/keystore` never loses its password without an item left to do, whether the pattern is `^/admin/keys`, `^/admin/keys/?` or one whose character class hides another alternative. The tests check each pattern they name against main's matching and the stack's closed paths (§8). - **Telemetry.** A telemetry row is sent only for a reader whose permissions include `analytics.ad_reporting.measure_ad_performance`, and a check line says so where `[tinybird]` was enabled. -- **The consent lists.** The places the GDPR and US state rules apply are the `jurisdiction` values in the `permissions.yaml` compiled into the build, so a change the operator made to `applies_in` or `privacy_states` is an item left to do until they carry it there and remove the table. +- **The consent lists.** The places the GDPR and US state rules apply are the `jurisdiction` values in the policy file the document names, so a change the operator made to `applies_in` or `privacy_states` is an item left to do until they carry it into that file and remove the table. ## 5. Failure - **A file that cannot be read, written or parsed.** The command exits 2. A file that is not TOML is refused naming the line reading stopped at and nothing from it (`migrate` in `rules.rs`, and the test `no_line_carries_a_value_from_the_document`). `--write` rewrites the file in place with one write, so a write that fails part of the way can leave the file damaged, which is one reason §6 asks the operator to keep the original. - **An item left to do, or a result that does not pass validation.** The command exits 1, and `--write` writes the result anyway, because the document did not load before either and the items say what is left. -- **The checks made only at startup.** The command's validation, like `ts config validate`, leaves out the checks core makes only once the settings load, which §2.1 lists, and those each adapter makes as it builds its state. Of those it reports the location settings as items left to do, for any document. The others, such as a module name that no linked crate supplies, show only when the deployment starts. +- **The checks made only at startup.** The command's validation, like `ts config validate`, leaves out the checks core makes only once the settings load, which §2.1 lists, and those each adapter makes as it builds its state. Of those it reports the location settings, and a document that names no policy, as items left to do, for any document. The policy file itself is checked by `ts config validate --permissions` and by `ts config push`, which always checks it (§2.1). The others, such as a module name that no linked crate supplies, show only when the deployment starts. ## 6. Compatibility and rollout -1. **The migrated document goes live with the new build.** A build from before this change refuses it, as main refuses a table it does not know (main core `settings.rs` line 2958), and this build refuses the old one, so no one document starts both builds (CHANGELOG lines 72 and 96). -2. **On Fastly, stage the pair first**, with `ts deploy --staging` and `ts config push --staging`, which read a configuration key of their own (CHANGELOG line 96). +1. **The migrated document goes live with the new build.** A build from before this change refuses it, as main refuses a table it does not know (main core `settings.rs` line 2958), and this build refuses the old one, so no one document starts both builds (CHANGELOG line 72, and the changelog entry that removes HTTP Basic authentication). +2. **On Fastly, stage the pair first**, with `ts deploy --staging` and `ts config push --staging --permissions `. Under EdgeZero `v0.0.10` a staged push writes the key its target's `__KEY` names, which on Fastly must be the logical store ID (`resolve_config_store_and_key` in its `config.rs`), so the staged pair is kept apart by the store the staging target's `__NAME` selects, which holds the staged blob with its policy file inside it. A service that chooses its blob by host pushes each host's staged blob with `--key ` and `__NAME` naming the staging store, never with `--staging`, which EdgeZero refuses beside `--key` (`2026-10-15-fastly-per-host-selector-design.md` section 2.5). That push writes the host's policy entries into the same store before the blob (`run` in `crates/trusted-server-cli/src/commands/config/keyed_push.rs`). 3. **Keep the original document before `--write`.** A rollback restores it with the old build (CHANGELOG line 72), and it holds the `[request_signing]` store ids the command removes and never prints, which `ts keys rotate` and `ts keys deactivate` take as `--config-store-id` and `--secret-store-id`. 4. **Rename the overlay variables.** `ts config push` ignores a variable for a key the document does not hold, so `TRUSTED_SERVER__PUBLISHER__ORIGIN_URL` and `TRUSTED_SERVER__INTEGRATIONS____*` set nothing once the document is migrated. They become `TRUSTED_SERVER__SOURCE__ORIGIN__URL` and `TRUSTED_SERVER__
____*` (CHANGELOG lines 26, 56 and 72). 5. **Keep the key-value store on Cloudflare and Spin.** A Worker without the `TRUSTED_SERVER_KV` binding, or a Spin component that cannot open `default`, now fails every request before any route runs (CHANGELOG line 46). Main's `wrangler.toml` declares the binding and main's `spin.toml` grants the store, so a deployment that kept those files has both. -6. **Expect the stored pages and the publisher's stored assets to be fetched again** once after the upgrade, because their keys hold a code worked out from the settings, while an asset route's stored copies keep their keys (CHANGELOG line 26). +6. **On Cloudflare, push the blob to the config namespace.** The Worker reads its blob from the KV namespace that `EDGEZERO__STORES__CONFIG__TRUSTED_SERVER_CONFIG__NAME` names, by default `trusted_server_config`, which `ts config push` writes, and no longer reads the `TRUSTED_SERVER_CONFIG` variable (`load_startup_settings` in the Cloudflare adapter's `app.rs`, as row 4, IABTechLab/trusted-server#1345, builds it). Declare that namespace in `wrangler.toml` as a namespace of its own beside `TRUSTED_SERVER_KV`, push the migrated document with `ts config push --adapter cloudflare --permissions `, and remove the variable (`2026-07-30-permission-model-design.md` §3.1, item 5). +7. **Expect the stored pages and the publisher's stored assets to be fetched again** once after the upgrade, because their keys hold a code worked out from the settings, while an asset route's stored copies keep their keys (CHANGELOG line 26). -The CLI guide's section "Upgrade a configuration written for an earlier release" gives the same steps. +The CLI guide's section "Upgrade a configuration written for an earlier release" gives every step but step 6, in the order an operator takes them. It runs the command first, and then has the operator do each `to do:` item, rename the overlay variables, run `ts config validate --permissions ` and push with the new build. The Cloudflare guide (`docs/guide/cloudflare.md`) says how to bind the config namespace and push into it. ## 7. What is not in this change - The breaking changes for crates outside core, for the `ts` tool and for request handling, as the table says. -- Pushing the result. The operator runs `ts config validate` and `ts config push` in the order §6 gives. +- Pushing the result. The operator runs `ts config validate` and then `ts config push`, each with `--permissions` naming the publisher's policy file, as the CLI guide's steps say (§6). - A write through a temporary file, which would keep the original whole when a write fails part of the way. §6 asks the operator to keep the original instead. - Rewriting comment text, which would mean guessing what a comment refers to. - Reading what a `[[handlers]]` pattern means rather than its text, which would also remove a pattern such as `^/admin/keys/?$` whose every path is closed. The command leaves such a pattern to do, and the operator removes it. @@ -176,26 +182,27 @@ The CLI guide's section "Upgrade a configuration written for an earlier release" ## 8. Tests - One test for each rewrite rule in the table, each on a small document, checking the result and the line it reports. -- One test for each item left to do, checking that the setting is left in place and that the line names it. Whether a run fails follows from whether any item is left, so one test, `an_item_left_to_do_fails_the_run`, runs the command on a document with an item left and checks that the run fails. -- A golden test that migrates main's own `trusted-server.example.toml`, copied from `upstream/main` byte for byte into `crates/trusted-server-cli/tests/fixtures/main-trusted-server.example.toml` under a header that names the commit. The test replaces its placeholder domain and origin, as an operator's document would have them. The result loads and passes validation under the stack's settings code, and migrating it again changes nothing. -- `the_location_lookup_is_the_one_each_adapter_accepts`, which migrates a document written for main for each adapter. On Fastly and Cloudflare the result selects `platform` and passes core's startup check of the jurisdiction acknowledgment. On Axum and Spin it selects no geo module, that check refuses it and an item left to do says so, and once `[geo] assume_single_jurisdiction = true` is added the check passes and a second run leaves nothing to do. +- One test for each item left to do, checking that the setting is left in place and that the line names it. Whether a run fails follows from whether any item is left, so one test, `an_item_left_to_do_fails_the_run`, runs the command on a document with an item left and checks that the run fails. Naming the policy has `a_document_that_names_no_policy_leaves_naming_one_to_do`, which also checks that the command adds no `[permissions]` and that the migrated document passes the command's validation, and `write_rewrites_the_file_and_the_default_prints_the_document` checks that a default run on a document written for main therefore fails, with that item reported and the migrated document passing validation. `a_document_written_for_main_that_names_its_policy_migrates_as_main` checks that such a document with `[permissions] policy` added makes the same rewrites, leaves the same items apart from naming the policy, prints the same check lines and gives the same document apart from the policy's table, and that the result holds `[source]` and passes validation. The other tests that check nothing is left to do leave that item aside (`left_to_do` in the tests). +- A golden test that migrates main's own `trusted-server.example.toml`, copied from `upstream/main` byte for byte into `crates/trusted-server-cli/tests/fixtures/main-trusted-server.example.toml` under a header that names the commit. The test replaces its placeholder domain and origin, as an operator's document would have them. The result loads and passes validation under the stack's settings code, and migrating it again changes nothing. The result names no policy, so the run also reports naming the policy file as an item left to do, which the test leaves aside, and the result passes the command's validation, which leaves the policy check out (§2.1). +- `the_location_lookup_is_the_one_each_adapter_accepts`, which migrates a document written for main for each adapter. On Fastly and Cloudflare the result selects `platform` and passes core's startup check of the jurisdiction acknowledgment. On Axum and Spin it selects no geo module, that check refuses it and an item left to do says so, and once `[geo] assume_single_jurisdiction = true` is added the check passes and a second run leaves nothing to do but naming the policy file. - `the_location_settings_startup_refuses_are_left_to_do_in_any_document`, for `[geo] module = "platform"` on Axum and Spin and for an Edge Cookie module with no geo module on every adapter. - `no_location_header_is_selected_for_the_operator`, on every adapter, with the check line each adapter gets, and `the_region_the_cloudflare_lookup_now_reads_is_reported_to_check`, on every adapter. - `a_handler_is_removed_only_where_every_path_it_matches_is_closed`, with the bounded and the unbounded `/admin/keys` patterns and those a quantifier after the prefix or the bound widens, and `a_handler_with_a_class_or_comment_after_its_prefix_is_left_to_do`. Both run each pattern through the `regex` crate main matched with and through core's `closed_path_response` on a set of request paths, so each pattern the command removes is shown to guard no path the stack forwards, each it keeps as unsafe is shown to guard one, and `^/admin/keys/?$` and `^/_ts/admin/[a-z]+` are shown kept although they guard none. - `a_host_header_override_is_reported_to_check`, and `a_document_in_the_stack_shape_with_no_page_entries_gets_none_added`. - A test that the comments of a document survive and its tables keep their order, and one that a document with Windows line endings keeps them. - A test that no line the command prints carries a value from the document, on every adapter, run on a document whose every value is a marker string. -- Tests of the command itself, that `--check` writes nothing and exits 1 when the document needs migrating and 0 when it does not, that `--write` writes the file, and that a file that is not TOML exits with an error that quotes nothing from it, and a test that the arguments need `--adapter` and the file and refuse an unknown adapter and `--write` with `--check`. +- Tests of the command itself, that `--check` writes nothing and exits 1 when the document needs migrating and 0 for a migrated document that names a policy, that `--write` writes the file, and that a file that is not TOML exits with an error that quotes nothing from it, and a test that the arguments need `--adapter` and the file and refuse an unknown adapter and `--write` with `--check`. - A test that the page entries name the middleware in the order `trusted-server.example.toml` shows them. No test starts the Axum or Spin adapter with a migrated document, because the tool links no adapter crate. Their refusal of `platform` is covered by the migrated document selecting no geo module on either. ## Revision record -| Date | Change | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| 2026-10-17 | First draft. | -| 2026-10-18 | Fixes from review round 1. `--adapter`, with the location lookup each adapter accepts and the Edge Cookie and geo settings startup refuses left to do. No location header is selected, a handler is removed only where every path its pattern matches is closed, a host header is reported to check, and page entries are added to a document written for main alone. The flags, the validation's reach, the consent lists, the adapters, consent and privacy, failure, compatibility and rollout, the tests, and 39 breaking entries. Fixes from review round 2. A handler pattern with a quantifier straight after its prefix or bound, or a character class or comment after them, is left to do, and a check line names the region the Cloudflare lookup now reads. The checks made only at startup, the location main's consent saw, a document run on two adapters, the stored assets, the passwords, the tests and the status line. | +| Date | Change | +| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 2026-10-17 | First draft. | +| 2026-10-18 | Fixes from review round 1. `--adapter`, with the location lookup each adapter accepts and the Edge Cookie and geo settings startup refuses left to do. No location header is selected, a handler is removed only where every path its pattern matches is closed, a host header is reported to check, and page entries are added to a document written for main alone. The flags, the validation's reach, the consent lists, the adapters, consent and privacy, failure, compatibility and rollout, the tests, and 39 breaking entries. Fixes from review round 2. A handler pattern with a quantifier straight after its prefix or bound, or a character class or comment after them, is left to do, and a check line names the region the Cloudflare lookup now reads. The checks made only at startup, the location main's consent saw, a document run on two adapters, the stored assets, the passwords, the tests and the status line. | +| 2026-10-26 | The permission policy is a file the settings name and the push supplies. A document that names none is an item left to do, whose line names the file's stem and is the same on every adapter, a document written for main may name its policy before or after it is migrated, the command never chooses a policy or writes a policy file, and its validation leaves the policy check out and follows the pattern rule for `permissions.sha256` and `permissions.text`. The consent lists are carried into the publisher's file, as the command's line says. The count of breaking entries is 41, and the Cloudflare config store has a row. The status names IABTechLab/trusted-server#1409, whose code the references cite, the EdgeZero version is `v0.0.10`, the settings lines are brought up to date, the staged push is described as `v0.0.10` does it, a host's blob is staged with `--key`, and §6 gains the step for Cloudflare's config namespace. | ## Review log From fb157989476f10e498eb6a33465656eb49fd3625 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:39 +0000 Subject: [PATCH 183/192] Four wording points from the permission designs' third review The permission model design's list of scripts carried whole now names only those of the crates a release links, and says that at row 55 the seam probe carries its js/probe.js whole too, in a crate each adapter lists as a dev-dependency alone, so the script never reaches a release. The Cloudflare item names the adapter's own test of its manifests, where the sentence had it read as the README's. The template build digest names the build-support file the build script includes, as CRATE_INPUTS lists it. The rollout design names the commit of main at which it cites the line that carries deny_unknown_fields on Settings. --- .../2026-07-30-permission-model-design.md | 34 +++++++++++-------- ...07-30-provider-migration-rollout-design.md | 7 ++-- 2 files changed, 24 insertions(+), 17 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 4487032d80..2ae220e9e1 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -349,14 +349,14 @@ that names each chunk with its length and SHA-256. `wrangler.ci.toml` bind the config store as a namespace of its own, `trusted_server_config`, beside `TRUSTED_SERVER_KV`, and `cloudflare.toml` names the same binding. The variable bridge is gone from the adapter's - code, manifests and `README.md`, whose test - `no_wrangler_manifest_carries_settings_in_a_variable` checks that no - manifest carries settings in a variable (`tests/config_defaults.rs`), from - the integration tests' Cloudflare environment, which stores its blob in the - local namespace with `wrangler kv key put`, from - `scripts/smoke-cloudflare.sh`, which pushes into the namespace with - `ts config push --adapter cloudflare --local`, and from - `docs/guide/cloudflare.md`, with its hand copy and double encoding. + code, manifests and `README.md`, from the integration tests' Cloudflare + environment, which stores its blob in the local namespace with + `wrangler kv key put`, from `scripts/smoke-cloudflare.sh`, which pushes + into the namespace with `ts config push --adapter cloudflare --local`, and + from `docs/guide/cloudflare.md`, with its hand copy and double encoding. + The adapter's test `no_wrangler_manifest_carries_settings_in_a_variable` + (`tests/config_defaults.rs`) checks that no manifest carries settings in a + variable. 6. **One way in.** Core declares `PlatformPolicySource` (`platform/policy_source.rs` lines 13 to 23), a trait with one method, `policy_text`, that returns the text of the file a reference names, the @@ -893,11 +893,15 @@ are the same for every publisher. missing, when the build stops. That check compares modification times, not contents. In place of a build it copies the bundles from the folder `TSJS_PREBUILT_DIR` names, which must hold the same set. The scripts that - crates carry whole with `include_str!` are the Google ad tag bootstrap - (`GPT_BOOTSTRAP_JS`), in core's `integrations/gpt.rs` at row 4 and in - `crates/ad-tag/google` at row 55, and at row 55 also InMobi's + the crates a release links carry whole with `include_str!` are the Google + ad tag bootstrap (`GPT_BOOTSTRAP_JS`), in core's `integrations/gpt.rs` at + row 4 and in `crates/ad-tag/google` at row 55, and at row 55 also InMobi's `gpp_stub.js` and `tcf_stub.js` (`GPP_STUB_SCRIPT` and `TCF_STUB_SCRIPT` in - `crates/cmp/inmobi/src/lib.rs`). + `crates/cmp/inmobi/src/lib.rs`). At row 55 the seam probe carries its + `js/probe.js` whole as well (`PROBE_JS` in + `crates/testing/iabtl/src/lib.rs`), but each adapter lists that crate as a + dev-dependency alone, so only test builds link it and the script never + reaches a release. 3. **The Worker glue on Cloudflare.** Cloudflare's `build.sh` installs `worker-build` at the version of the `worker` crate that `Cargo.lock` pins and runs it (lines 21 to 39 at both rows). `worker-build` downloads the @@ -925,8 +929,10 @@ are the same for every publisher. pipeline sets or core's `build.rs` reads from local git (`emit_git_version`), compiled in as `TS_GIT_VERSION` in `constants.rs`. 6. **The template build digest.** A SHA-256 that core's `build.rs` takes over - core's sources, its build script and `Cargo.toml`, and the workspace's - `Cargo.toml`, `Cargo.lock` and `edgezero.toml` (`template_build_digest`), + core's sources, its build script with the `build_support/git_version.rs` it + includes, its `Cargo.toml`, and the workspace's `Cargo.toml`, `Cargo.lock` + and `edgezero.toml` (`CRATE_INPUTS`, `WORKSPACE_INPUTS` and + `template_build_digest`), compiled in as `TEMPLATE_BUILD_DIGEST` in `publisher.rs`. It is part of the fingerprint a stored page template is kept under, so a template stored by other code is never served. diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 4ada6c2731..20abcb6e58 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -242,9 +242,10 @@ Requirements, each marked with its implementation state: permission-model spec §3.1, which row 4 of the stack builds, starts no domain whose blob names no policy, and a build without it, as main is, refuses `[permissions]` as an unknown field (`deny_unknown_fields` on - `Settings`, main's `settings.rs` line 2964), so no one blob starts both - builds. The build and the blob that names its policy therefore go live - together, and on Fastly the pair is staged first (§6 item 1). + `Settings`, `settings.rs` line 2964 at `7aa5c993b` on main), so no one + blob starts both builds. The build and the blob that names its policy + therefore go live together, and on Fastly the pair is staged first (§6 + item 1). The draft's full N+1/N+2 interim (the negative-record read/write matrix, the `permissions_v2` model promotion, the `m00` mirror, and the From 74e6c9f45f715499b2d6811ebfdf9e7d5c862818 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:40 +0000 Subject: [PATCH 184/192] Review logs record each round of the permission designs The permission model, response header hook, inspection endpoints, config by host and config migrate designs end with an entry for each of review rounds 3, 4 and 5, and the provider migration rollout design gains a review log with rounds 1, 2 and 3, as CONTRIBUTING.md asks. Each entry says what the round found and what changed after it. The last round approved the permission model and rollout designs with none blocking, and their entries name the wording points fixed after it. It found no issues in the other four. --- ...integration-response-header-hook-design.md | 24 +++++++++++++ .../2026-07-30-permission-model-design.md | 3 ++ ...07-30-provider-migration-rollout-design.md | 36 +++++++++++++++++++ .../specs/2026-10-12-config-by-host-design.md | 3 ++ .../2026-10-12-inspection-endpoints-design.md | 3 ++ .../specs/2026-10-17-config-migrate-design.md | 3 ++ 6 files changed, 72 insertions(+) diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 2cbb12e9fa..72d863af49 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -1653,3 +1653,27 @@ changes no code and decides none of them. that one function is the only response-finalization code the adapters share, and the claim in section 7 that a pull request now closed (#1047) ships the documentation set. +3. **Round 3.** A review in a fresh session, against + `split/5-response-hook-docs`. Issues found, one of them blocking. Section + 1a, which states what the code does today, said that the pushed blob is + the policy identity, when today each build compiles the repository's + sample in, and the pushed blob becomes the policy identity only once + row 4 builds section 3.1 of the permission model design. This revision + keeps today's code in section 1a and adds, as what row 4 builds, that a + domain's blob then names its policy file by name and SHA-256 and gives + `active_policy_digest` its value. Open question 2 says that the policy's + `sha256` then sits in the settings the template fingerprint hashes, so a + policy change changes the fingerprint, where a change to the compiled + sample does not. +4. **Round 4.** A review in a fresh session, against + `split/5-response-hook-docs`, of the round 3 fixes and the new text. + Approved, none blocking. This revision names the file each build + compiles in, `config/permissions/sample.yaml`, where section 1a said + `permissions.yaml`, and says that the template fingerprint leaves the + policy's `text` out, because `sha256` already names the file. +5. **Round 5.** A review in a fresh session, against + `split/5-response-hook-docs`, with row 4 read on `split/3-permissions`. + No issues found. The round found section 1a right at that commit, where + `DEFAULT_PERMISSION_RULES` still compiles the repository's sample in, and + found that row 4's template fingerprint hashes the policy's `sha256` and + leaves its text out, as open question 2 says. diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 2ae220e9e1..7dcc02e398 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -2035,3 +2035,6 @@ include the changes that followed the first revision of this section on 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/3-permissions` at `a028c00da`. Issues found, nine of them blocking. The nine were the claim that an unreadable record revokes everything ahead of the modules, four shipped modules where five ship, registration through an integration builder, a page state described with two members where it has four, a follow-up note that gave `ec_allowed` for two paths that use `ec_sharing_allowed`, the persisted consent, the resolve endpoint and the device module check that this row does not build, unknown rule fields that the parser does not reject, the Data Uses said to have no signal, and raw strings said to be kept for forwarding. This revision states what the code does in each case, adds the fifth module `mtm`, a table of what each adapter runs, an upgrade list and the privacy effects the round found, names the row that builds each item this row lacks, and leaves the code changes the round asks for under Open questions. 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/3-permissions` at `a028c00da`. Approved, none blocking. One minor point stays, that the `split/` branches the spec names are the fork's, while the IAB Tech Lab drafts carry the same commits on `feature/` branches. +3. **Round 3.** A review in a fresh session, against `split/3-permissions`. Issues found, six of them blocking. The six were that §3.1 rested on code that only later rows of the stack build and never said which row builds that code, that a Cloudflare Worker variable holds at most 5 KB while the repository's sample alone is 24,231 bytes, that nothing said what a location answers while a push is still reaching it, that the statuses a domain answers when it cannot start were partly untrue of today's code and left out Spin's fixed `503` and the `421` that Fastly answers from row 11 for a host with no blob, that fourteen places still called the repository's sample a policy that ships, and that the push could not be built from the text on any adapter. This revision says what row 4 builds by itself and what each later row adds once it is merged forward, has Cloudflare read its blob from its config store, a Workers KV namespace, in place of the variable, and says that each startup router answers with the status of the error's kind, so a domain without a valid policy file answers `500` on every adapter once row 4 moves Spin off its fixed `503`, while a host with no blob keeps its `421`. It also has the push, from row 11, write every policy entry before the blobs that name it and then wait, says that a location which still sees a blob before its policy entry answers `500`, sets out how `ts` adds `--permissions` to EdgeZero's commands, how the SHA-256 is taken and what the push refuses, and calls the sample the repository's sample throughout. +4. **Round 4.** A review in a fresh session, against `split/3-permissions`, of the round 3 fixes and the new text. Issues found, two of them blocking. The two were that the design cited row heads by commits that no published branch held, and that `ts config validate` takes no `--adapter`, so validation could not follow a rule under which the push adds the policy's text on every adapter but Axum. This revision cites commits that the published branches hold, and says that the configuration view never shows a policy's text, so a pattern under `permissions.text` is refused wherever patterns are checked and every check counts `permissions.sha256` as present, that `ts config validate` and `ts config diff` run the policy checks only when `--permissions` is given, with validate otherwise saying that the policy was not checked, and that `ts config migrate` and `ts audit generate` never run those checks. It also says how `ts` hands the settings to EdgeZero, which binding a Worker reads its blob from, what a Worker missing that binding or its blob answers, and how the tests of the `gpc`, `gpp`, `tcf` and `us-privacy` crates read the sample once `PermissionMaps::standard` is removed. +5. **Round 5.** A review in a fresh session, against `split/3-permissions`, with what rows 9, 11, 13, 51 and 55 add read on their own branches. Approved, none blocking. The round checked every file, function, line number and test that §3.1, §3.6, §3.7, §6 and §8 cite, against the code of those rows, EdgeZero `v0.0.10`, worker-build `v0.8.5` and Tech Lab main, and found each as the design states it, the order in which the runtime loader asks its source for a policy file before it looks for a parsed copy included. This revision says that the seam probe's script, which only test builds link, is carried whole as well and never reaches a release, names the adapter's test beside the Cloudflare manifests rather than the README's, and names the build-support file that the template build digest hashes with the build script. diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index 20abcb6e58..fa47ea57d1 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -614,3 +614,39 @@ the code with the row open. | Environment override documented but `#[cfg(test)]`-only in PR #838 (§7) | Override applied as typed EdgeZero app-config overlays at `ts config push`; the existing CLI overlay test covers the mechanism, and a module-specific override test is still to write | The same compiled binary switches modules at deployment through the published configuration | | Pinned known-answer HMAC vectors committed (§3) | Stability tested per inputs, and the pinned vector is still to commit | The cross-version CI pin remains open work under §3 | | GB storage baseline change only with citation and sign-off (§2 row 4) | The repository's sample adopts `granted` storage for GB without a recorded citation | Flagged in row 4, since the task force owns the decision and its record | + +## Review log + +1. **Round 1.** A review in a fresh session, against `split/3-permissions`. + Issues found, two of them blocking. The two were that the §5 recipe said + an operator no longer carries `consent.gdpr.applies_in` or + `consent.us_states.privacy_states` because "the shipped tree" holds the + same countries and states, when no policy ships, and that the design gave + the policy no rollout order, although a build with permission-model spec + §3.1 starts no domain whose blob names no policy and a build without it + refuses `[permissions]`, so no one blob starts both. This revision says + that the publisher's own tree states its countries and states, that the + repository's sample lists the 31 countries and 20 states main's lists + held, and that the build and the blob that names its policy go live + together, staged first on Fastly. It also calls the sample the + repository's sample throughout, says that a rollback needs the previous + policy file's exact bytes, and names the folder Axum reads its policy + from. +2. **Round 2.** A review in a fresh session, against `split/3-permissions`, + of the round 1 fixes and the new text. Issues found, one of them + blocking. The one was that the design cited row 4's head by a commit + that no published branch held. This revision cites a commit that the + published branch holds, says that row 4 replaces the signal order fixed + in code with the order the deployment configures, names the overlay + variables after the `module` selectors, and has a service that chooses + its blob by host stage each host's blob with `--key`, because EdgeZero + refuses `--staging` beside `--key`. +3. **Round 3.** A review in a fresh session, against `split/3-permissions`, + of the round 2 fixes and the new text. Approved, none blocking. The round + found that the build and the blob go live together as the design says, + because row 4's loader refuses a blob that names no policy and main's + settings refuse `[permissions]` as an unknown field, that the example + settings name no policy and are refused as written, and that the Axum and + Spin adapters refuse the platform lookup from row 33. This revision names + the commit of main at which it cites the line that carries + `deny_unknown_fields` on `Settings`. diff --git a/docs/superpowers/specs/2026-10-12-config-by-host-design.md b/docs/superpowers/specs/2026-10-12-config-by-host-design.md index 5c32fb63a9..ed6c0800a5 100644 --- a/docs/superpowers/specs/2026-10-12-config-by-host-design.md +++ b/docs/superpowers/specs/2026-10-12-config-by-host-design.md @@ -188,3 +188,6 @@ The behaviors below are stated in the sections above. A change to the code would 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/11-config-by-host` at `b8314cb52`. Issues found, three of them blocking. The header pointed at closed pull requests and called the fork pull request ready for review, the risk table put three header cells over two-cell rows, and the text said there is no fallback and that a selector that cannot be read fails the request, which is false when the runtime-env store cannot be opened. This revision points the header at the open draft pull request, repairs the table, states the fallback and the removal it needs, and adds the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests, while the code behaviors the review found wait for code changes and sit under Open questions. 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/11-config-by-host` at `b8314cb52`. Approved, none blocking. This revision names, at the pointer and where this spec describes them, the sections of `2026-10-15-fastly-per-host-selector-design.md` that replace the fallback text (section 2.4) and the push and selector-writing steps (sections 2.6 and 2.8). +3. **Round 3.** A review in a fresh session, against `split/11-config-by-host`. Issues found, one of them blocking. The design called both EdgeZero `683202c` and `v0.0.10` the version the stack pins, so its account of a staged deploy, which holds only at `683202c`, read as true of the stack. This revision ties `683202c` to the version this row's head pins, says that the stack now pins `v0.0.10`, and points staging under `v0.0.10` at section 2.5 of the Fastly per-host selector design. It also says that the service holds one policy entry for each SHA-256 that the hosts' blobs pin, which hosts share only when they pin the same SHA-256, that `ts config diff` cannot reach a host's blob, so a host's pinned policy is checked through `/_ts/config` on that host, and asks under Open questions, item 9, whether a selector that forms a policy entry's key should be refused. +4. **Round 4.** A review in a fresh session, against `split/11-config-by-host`, of the round 3 fixes and the new text. Issues found, one of them blocking. The one was that the design cited row 11's head by a commit that no published branch held. This revision cites a commit that the published branch holds, and says that a config store that cannot be read answers `500`, as the selection does when it cannot look a host up, where it said that the selection's failure is a configuration error. +5. **Round 5.** A review in a fresh session, against `split/11-config-by-host`. No issues found. The round found the keyed writer keeping each policy file once under its digest and writing it before the host's blob, the Fastly adapter reading a host's policy from that entry, the two tests the design names, the key function in core, and the selector and host rules that Open questions, item 9, describes, each as the design states them. diff --git a/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md b/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md index 67354d2f03..07e4a98af4 100644 --- a/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md +++ b/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md @@ -195,3 +195,6 @@ On `33282f0c9`, the last of the five original commits, the whole check also pass 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/9-visibility-endpoints` at `497d553c6`. Issues found, three of them blocking. The design said a deployment whose settings do not select the change behaves as before when all five addresses answer on all four adapters with no setting, named no address, method, `.json` form, status or header for the permissions and configuration endpoints, and described the data page's protection as `Content-Security-Policy: sandbox` when the code sends a longer policy and five more headers and keeps them under an operator's headers. This revision states what the code does at that head for the addresses, the defaults and the one switch, the failure paths, the rollout, the tests and their gaps, and lists five questions that wait for code changes. 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/9-visibility-endpoints` at `497d553c6`. Approved, none blocking. This revision says that the modules' request preparers and the request filters never run for these addresses on Fastly and Spin, in place of saying that the answer comes before them, and says where a request with a method other than `GET` goes on Cloudflare, which is the shared catch-all route and then the publisher. +3. **Round 3.** A review in a fresh session, against `split/9-visibility-endpoints`. Issues found, one of them blocking. The design called IABTechLab/trusted-server#1345 row 3 of the stack, where IABTechLab/trusted-server#777 numbers it row 4. This revision numbers it row 4, says that a failed load answers with the status of the error's kind and a generic message on Fastly, Axum and Cloudflare, while Spin answers a fixed `503` until row 4 moves Spin to the same rule, and says that the merge of row 4 adds `policy` to the permissions answer, with tests for the new fields. It also says that a policy's name is a file's stem, which two versions of a file share, so the SHA-256 that the configuration view shows is what matches a domain to one exact copy of the file, and that `[inspect] config = false`, or a `hide` that covers `permissions.sha256`, withholds that SHA-256 from every address. +4. **Round 4.** A review in a fresh session, against `split/9-visibility-endpoints`, of the round 3 fixes and the new text. Issues found, one of them blocking. The design promised that a pattern naming a policy field is judged alike at validation and at load, which could not hold for `permissions.text`, because `ts config validate` takes no `--adapter` and so cannot know whether the push adds the text. This revision says that the configuration view shows `policy` and `sha256` and never `text`, that a pattern under `permissions.text` is therefore refused wherever patterns are checked, and that every check counts `permissions.sha256` as present because the push always writes it, with tests for each. +5. **Round 5.** A review in a fresh session, against `split/9-visibility-endpoints`, with row 4 read on `split/3-permissions`. No issues found. The round found the `policy` field and its test in the permissions answer, found that the configuration view shows the policy's name and SHA-256 and never its text because row 4's loader takes the text out of the settings, and found that every place that checks `[inspect]` patterns, the deploy check included, goes through the one function that refuses a pattern under `permissions.text` and counts `permissions.sha256` as present. diff --git a/docs/superpowers/specs/2026-10-17-config-migrate-design.md b/docs/superpowers/specs/2026-10-17-config-migrate-design.md index abf232a8a5..2f85c9f682 100644 --- a/docs/superpowers/specs/2026-10-17-config-migrate-design.md +++ b/docs/superpowers/specs/2026-10-17-config-migrate-design.md @@ -208,3 +208,6 @@ No test starts the Axum or Spin adapter with a migrated document, because the to 1. **Round 1, 2026-10-18.** A review in a fresh session, against `split/51-config-migrate` at `50c68be1b`. Issues found, four of them blocking. The four were that the Axum and Spin adapters refuse the `[geo] module = "platform"` the command wrote into every document written for main, that the `[headers]` rewrite started sending the reader's location on Cloudflare, where main sent none, that the `[[handlers]]` rule removed a handler on `^/admin/keys`, which also guarded publisher paths such as `/admin/keystore` that the stack forwards to the origin, and that the spec had no compatibility and rollout section. Fixed in `18b7c7d16`, where the command selects `platform` only where the host has a lookup and leaves to do, for any document, an Edge Cookie module with no geo module and `platform` on an adapter with none, in `4465e2ad0`, where it takes `--adapter`, never selects `headers.geo` and prints a check line saying what main's responses carried, in `65bb09b14`, where it removes a handler only where every path its pattern can match is closed, and in this revision, which adds the adapters, consent and privacy, failure, and compatibility and rollout. The non-blocking findings are fixed too, with a host header reported to check in `65814d561` and page entries added to a document written for main alone in `f8dffebdc`. In this revision the consent lists stay items left to do with the definition of `to do:` widened to say so, the seam probe and the key-value store have rows of their own, and the count of breaking entries is 39. The review's statement that a selected `[ads-txt]` with no entry loads with a warning is not borne out, because the module refuses that document (`crates/ads-txt/iabtl/src/lib.rs` lines 154 to 175), so the `/ads.txt` entry is still added to any document. 2. **Round 2, 2026-10-18.** A review in a fresh session, against `split/51-config-migrate` at `6e3c09200`, of the round 1 fixes and the new text. Issues found, one of them blocking. The one was that a quantifier after the `^/admin/keys` bound or the `^/_ts/admin` prefix still let the command remove a handler that guarded publisher paths, with a `changed:` line and no `to do:`, as `^/admin/keys/?`, `^/admin/keys/*` and `^/admin/keys(?:/|$)?` matched `/admin/keystore` and `^/_ts/admin?` matched `/_ts/admit` under main's search. Fixed in `83dd4f1db`, where the command leaves such a pattern to do when `?`, `*` or `{` follows the prefix or the bound, and in `074d454e3`, where it also leaves to do a pattern with a `[` or a `#` after the prefix or the bound, because a character class or a comment the `(?x)` flag allows could hide an alternative such as `|^/members` from its scan, a fault the review did not name, found while checking which other text widens the match. The tests run each pattern through the `regex` crate and core's closed paths. The non-blocking findings are fixed too, with a check line on Cloudflare for the region its lookup now reads in `eb25e6d16`, and in this revision, which names the checks made only at startup, says the consent rules see the location main's jurisdiction check saw through `applies_in` and `privacy_states`, says a document run on two adapters is migrated once for each, adds the step on stored pages to the CLI guide, gives the comment on items left to do the spec's definition, and drops "the top pull request" from the status line. The round 1 commit hashes stay as they are, because `split/51-config-migrate` is merged forward and never rebased, so they resolve once it is pushed. +3. **Round 3.** A review in a fresh session, against `split/51-config-migrate`. Issues found, two of them blocking. The two were that the design had the command's validation check the policy against the files `--permissions` supplies, when the command takes no `--permissions`, so that every document, and every run of `ts audit generate`, which shares that validation, would fail, and that the header said the pull request was not yet raised, when IABTechLab/trusted-server#1409 was open. This revision keeps the policy checks out of the command's validation, has the command write a `to do:` line for a document that names no policy, naming the policy by the file's stem and, on Axum, the folder the file goes in, and never choose a policy or write a policy file, and names IABTechLab/trusted-server#1409 in the header. It also marks the new wording of the consent lists' `to do:` line as the design that row 4 builds, describes the staged push as EdgeZero `v0.0.10` makes it, and brings the line references up to date. +4. **Round 4.** A review in a fresh session, against `split/51-config-migrate`, of the round 3 fixes and the new text. Approved, none blocking. This revision has the command's validation, like that of `ts audit generate`, count `permissions.sha256` as present and refuse a pattern under `permissions.text` once row 4 builds the policy design, as every pattern check then does, drops a citation of a changelog entry that says a staged deploy reads a configuration key of its own, which EdgeZero `v0.0.10` does not do, has a service that chooses its blob by host stage each host's blob with `--key`, and adds the step in section 6 that pushes Cloudflare's blob to its config namespace. +5. **Round 5.** A review in a fresh session, against `split/51-config-migrate`, with main read at `5bc388625`. No issues found. The round found every citation by file and function at the row's head, the forty-one entries marked breaking in the changelog and each line it cites, the lines it cites on main, and that the command's validation reaches the one pattern check through the deploy validation it shares with `ts audit generate`, while the policy check stays out of it because the command takes no `--permissions`. From 9f9f88638a0d48e6e18251f333a1cceba88eadc1 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:41 +0000 Subject: [PATCH 185/192] Three review log entries say what each round read The round 5 entries of the permission model and config migrate designs say how the text was brought into line with the code before the round, so their round 3 entries read as records of that round. The response header hook's round 5 entry names the commit its section 1a states. --- ...2026-07-30-integration-response-header-hook-design.md | 9 +++++---- .../specs/2026-07-30-permission-model-design.md | 2 +- .../specs/2026-10-17-config-migrate-design.md | 2 +- 3 files changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index 72d863af49..bc8dfd3e97 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -1673,7 +1673,8 @@ changes no code and decides none of them. policy's `text` out, because `sha256` already names the file. 5. **Round 5.** A review in a fresh session, against `split/5-response-hook-docs`, with row 4 read on `split/3-permissions`. - No issues found. The round found section 1a right at that commit, where - `DEFAULT_PERMISSION_RULES` still compiles the repository's sample in, and - found that row 4's template fingerprint hashes the policy's `sha256` and - leaves its text out, as open question 2 says. + No issues found. The round found section 1a right at `61b1399c6`, the + commit it states, where `DEFAULT_PERMISSION_RULES` still compiles the + repository's sample in, and found that row 4's template fingerprint + hashes the policy's `sha256` and leaves its text out, as open question 2 + says. diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 7dcc02e398..c5a4174a61 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -2037,4 +2037,4 @@ include the changes that followed the first revision of this section on 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/3-permissions` at `a028c00da`. Approved, none blocking. One minor point stays, that the `split/` branches the spec names are the fork's, while the IAB Tech Lab drafts carry the same commits on `feature/` branches. 3. **Round 3.** A review in a fresh session, against `split/3-permissions`. Issues found, six of them blocking. The six were that §3.1 rested on code that only later rows of the stack build and never said which row builds that code, that a Cloudflare Worker variable holds at most 5 KB while the repository's sample alone is 24,231 bytes, that nothing said what a location answers while a push is still reaching it, that the statuses a domain answers when it cannot start were partly untrue of today's code and left out Spin's fixed `503` and the `421` that Fastly answers from row 11 for a host with no blob, that fourteen places still called the repository's sample a policy that ships, and that the push could not be built from the text on any adapter. This revision says what row 4 builds by itself and what each later row adds once it is merged forward, has Cloudflare read its blob from its config store, a Workers KV namespace, in place of the variable, and says that each startup router answers with the status of the error's kind, so a domain without a valid policy file answers `500` on every adapter once row 4 moves Spin off its fixed `503`, while a host with no blob keeps its `421`. It also has the push, from row 11, write every policy entry before the blobs that name it and then wait, says that a location which still sees a blob before its policy entry answers `500`, sets out how `ts` adds `--permissions` to EdgeZero's commands, how the SHA-256 is taken and what the push refuses, and calls the sample the repository's sample throughout. 4. **Round 4.** A review in a fresh session, against `split/3-permissions`, of the round 3 fixes and the new text. Issues found, two of them blocking. The two were that the design cited row heads by commits that no published branch held, and that `ts config validate` takes no `--adapter`, so validation could not follow a rule under which the push adds the policy's text on every adapter but Axum. This revision cites commits that the published branches hold, and says that the configuration view never shows a policy's text, so a pattern under `permissions.text` is refused wherever patterns are checked and every check counts `permissions.sha256` as present, that `ts config validate` and `ts config diff` run the policy checks only when `--permissions` is given, with validate otherwise saying that the policy was not checked, and that `ts config migrate` and `ts audit generate` never run those checks. It also says how `ts` hands the settings to EdgeZero, which binding a Worker reads its blob from, what a Worker missing that binding or its blob answers, and how the tests of the `gpc`, `gpp`, `tcf` and `us-privacy` crates read the sample once `PermissionMaps::standard` is removed. -5. **Round 5.** A review in a fresh session, against `split/3-permissions`, with what rows 9, 11, 13, 51 and 55 add read on their own branches. Approved, none blocking. The round checked every file, function, line number and test that §3.1, §3.6, §3.7, §6 and §8 cite, against the code of those rows, EdgeZero `v0.0.10`, worker-build `v0.8.5` and Tech Lab main, and found each as the design states it, the order in which the runtime loader asks its source for a policy file before it looks for a parsed copy included. This revision says that the seam probe's script, which only test builds link, is carried whole as well and never reaches a release, names the adapter's test beside the Cloudflare manifests rather than the README's, and names the build-support file that the template build digest hashes with the build script. +5. **Round 5.** A review in a fresh session, against `split/3-permissions`, with what rows 9, 11, 13, 51 and 55 add read on their own branches. Approved, none blocking. The round checked every file, function, line number and test that §3.1, §3.6, §3.7, §6 and §8 cite, against the code of those rows, EdgeZero `v0.0.10`, worker-build `v0.8.5` and Tech Lab main, and found each as the design states it, the order in which the runtime loader asks its source for a policy file before it looks for a parsed copy included. This revision says that the seam probe's script, which only test builds link, is carried whole as well and never reaches a release, names the adapter's test beside the Cloudflare manifests rather than the README's, and names the build-support file that the template build digest hashes with the build script. Before this round the text was brought into line with the code, so the wait between a new policy entry and its blob comes with row 13's live push, where rounds 3 and 4 read it at row 11. diff --git a/docs/superpowers/specs/2026-10-17-config-migrate-design.md b/docs/superpowers/specs/2026-10-17-config-migrate-design.md index 2f85c9f682..0bfdaeaff0 100644 --- a/docs/superpowers/specs/2026-10-17-config-migrate-design.md +++ b/docs/superpowers/specs/2026-10-17-config-migrate-design.md @@ -210,4 +210,4 @@ No test starts the Axum or Spin adapter with a migrated document, because the to 2. **Round 2, 2026-10-18.** A review in a fresh session, against `split/51-config-migrate` at `6e3c09200`, of the round 1 fixes and the new text. Issues found, one of them blocking. The one was that a quantifier after the `^/admin/keys` bound or the `^/_ts/admin` prefix still let the command remove a handler that guarded publisher paths, with a `changed:` line and no `to do:`, as `^/admin/keys/?`, `^/admin/keys/*` and `^/admin/keys(?:/|$)?` matched `/admin/keystore` and `^/_ts/admin?` matched `/_ts/admit` under main's search. Fixed in `83dd4f1db`, where the command leaves such a pattern to do when `?`, `*` or `{` follows the prefix or the bound, and in `074d454e3`, where it also leaves to do a pattern with a `[` or a `#` after the prefix or the bound, because a character class or a comment the `(?x)` flag allows could hide an alternative such as `|^/members` from its scan, a fault the review did not name, found while checking which other text widens the match. The tests run each pattern through the `regex` crate and core's closed paths. The non-blocking findings are fixed too, with a check line on Cloudflare for the region its lookup now reads in `eb25e6d16`, and in this revision, which names the checks made only at startup, says the consent rules see the location main's jurisdiction check saw through `applies_in` and `privacy_states`, says a document run on two adapters is migrated once for each, adds the step on stored pages to the CLI guide, gives the comment on items left to do the spec's definition, and drops "the top pull request" from the status line. The round 1 commit hashes stay as they are, because `split/51-config-migrate` is merged forward and never rebased, so they resolve once it is pushed. 3. **Round 3.** A review in a fresh session, against `split/51-config-migrate`. Issues found, two of them blocking. The two were that the design had the command's validation check the policy against the files `--permissions` supplies, when the command takes no `--permissions`, so that every document, and every run of `ts audit generate`, which shares that validation, would fail, and that the header said the pull request was not yet raised, when IABTechLab/trusted-server#1409 was open. This revision keeps the policy checks out of the command's validation, has the command write a `to do:` line for a document that names no policy, naming the policy by the file's stem and, on Axum, the folder the file goes in, and never choose a policy or write a policy file, and names IABTechLab/trusted-server#1409 in the header. It also marks the new wording of the consent lists' `to do:` line as the design that row 4 builds, describes the staged push as EdgeZero `v0.0.10` makes it, and brings the line references up to date. 4. **Round 4.** A review in a fresh session, against `split/51-config-migrate`, of the round 3 fixes and the new text. Approved, none blocking. This revision has the command's validation, like that of `ts audit generate`, count `permissions.sha256` as present and refuse a pattern under `permissions.text` once row 4 builds the policy design, as every pattern check then does, drops a citation of a changelog entry that says a staged deploy reads a configuration key of its own, which EdgeZero `v0.0.10` does not do, has a service that chooses its blob by host stage each host's blob with `--key`, and adds the step in section 6 that pushes Cloudflare's blob to its config namespace. -5. **Round 5.** A review in a fresh session, against `split/51-config-migrate`, with main read at `5bc388625`. No issues found. The round found every citation by file and function at the row's head, the forty-one entries marked breaking in the changelog and each line it cites, the lines it cites on main, and that the command's validation reaches the one pattern check through the deploy validation it shares with `ts audit generate`, while the policy check stays out of it because the command takes no `--permissions`. +5. **Round 5.** A review in a fresh session, against `split/51-config-migrate`, with main read at `5bc388625`. No issues found. The round found every citation by file and function at the row's head, the forty-one entries marked breaking in the changelog and each line it cites, the lines it cites on main, and that the command's validation reaches the one pattern check through the deploy validation it shares with `ts audit generate`, while the policy check stays out of it because the command takes no `--permissions`. Before this round the text was brought into line with the row's code, so the line left to do for the policy file is the same on every adapter and names no folder on Axum, as round 3 read it, and a document written for main may name its policy before or after it is migrated. From 616a51b0aa7e8970b94e307f1b979ab805946f33 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:42 +0000 Subject: [PATCH 186/192] The config migrate design says only stored pages are fetched again Step 7 of the upgrade steps said the publisher's stored assets are also fetched again because their keys hold a code worked out from the settings. Core holds no copy of a file once the asset cache is a module, and the asset cache module's keys hold no such code, so the step names the stored pages alone. --- .../specs/2026-10-17-config-migrate-design.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/superpowers/specs/2026-10-17-config-migrate-design.md b/docs/superpowers/specs/2026-10-17-config-migrate-design.md index 0bfdaeaff0..1bc11b006c 100644 --- a/docs/superpowers/specs/2026-10-17-config-migrate-design.md +++ b/docs/superpowers/specs/2026-10-17-config-migrate-design.md @@ -166,7 +166,7 @@ A deployment that ran one document on two adapters, as main's quick start pushes 4. **Rename the overlay variables.** `ts config push` ignores a variable for a key the document does not hold, so `TRUSTED_SERVER__PUBLISHER__ORIGIN_URL` and `TRUSTED_SERVER__INTEGRATIONS____*` set nothing once the document is migrated. They become `TRUSTED_SERVER__SOURCE__ORIGIN__URL` and `TRUSTED_SERVER__
____*` (CHANGELOG lines 26, 56 and 72). 5. **Keep the key-value store on Cloudflare and Spin.** A Worker without the `TRUSTED_SERVER_KV` binding, or a Spin component that cannot open `default`, now fails every request before any route runs (CHANGELOG line 46). Main's `wrangler.toml` declares the binding and main's `spin.toml` grants the store, so a deployment that kept those files has both. 6. **On Cloudflare, push the blob to the config namespace.** The Worker reads its blob from the KV namespace that `EDGEZERO__STORES__CONFIG__TRUSTED_SERVER_CONFIG__NAME` names, by default `trusted_server_config`, which `ts config push` writes, and no longer reads the `TRUSTED_SERVER_CONFIG` variable (`load_startup_settings` in the Cloudflare adapter's `app.rs`, as row 4, IABTechLab/trusted-server#1345, builds it). Declare that namespace in `wrangler.toml` as a namespace of its own beside `TRUSTED_SERVER_KV`, push the migrated document with `ts config push --adapter cloudflare --permissions `, and remove the variable (`2026-07-30-permission-model-design.md` §3.1, item 5). -7. **Expect the stored pages and the publisher's stored assets to be fetched again** once after the upgrade, because their keys hold a code worked out from the settings, while an asset route's stored copies keep their keys (CHANGELOG line 26). +7. **Expect the stored pages to be fetched again** once after the upgrade, because their keys hold a code worked out from the settings (CHANGELOG line 26). Core holds no copy of a file, and the asset cache module's keys hold no such code. The CLI guide's section "Upgrade a configuration written for an earlier release" gives every step but step 6, in the order an operator takes them. It runs the command first, and then has the operator do each `to do:` item, rename the overlay variables, run `ts config validate --permissions ` and push with the new build. The Cloudflare guide (`docs/guide/cloudflare.md`) says how to bind the config namespace and push into it. @@ -198,11 +198,11 @@ No test starts the Axum or Spin adapter with a migrated document, because the to ## Revision record -| Date | Change | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| 2026-10-17 | First draft. | -| 2026-10-18 | Fixes from review round 1. `--adapter`, with the location lookup each adapter accepts and the Edge Cookie and geo settings startup refuses left to do. No location header is selected, a handler is removed only where every path its pattern matches is closed, a host header is reported to check, and page entries are added to a document written for main alone. The flags, the validation's reach, the consent lists, the adapters, consent and privacy, failure, compatibility and rollout, the tests, and 39 breaking entries. Fixes from review round 2. A handler pattern with a quantifier straight after its prefix or bound, or a character class or comment after them, is left to do, and a check line names the region the Cloudflare lookup now reads. The checks made only at startup, the location main's consent saw, a document run on two adapters, the stored assets, the passwords, the tests and the status line. | -| 2026-10-26 | The permission policy is a file the settings name and the push supplies. A document that names none is an item left to do, whose line names the file's stem and is the same on every adapter, a document written for main may name its policy before or after it is migrated, the command never chooses a policy or writes a policy file, and its validation leaves the policy check out and follows the pattern rule for `permissions.sha256` and `permissions.text`. The consent lists are carried into the publisher's file, as the command's line says. The count of breaking entries is 41, and the Cloudflare config store has a row. The status names IABTechLab/trusted-server#1409, whose code the references cite, the EdgeZero version is `v0.0.10`, the settings lines are brought up to date, the staged push is described as `v0.0.10` does it, a host's blob is staged with `--key`, and §6 gains the step for Cloudflare's config namespace. | +| Date | Change | +| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-17 | First draft. | +| 2026-10-18 | Fixes from review round 1. `--adapter`, with the location lookup each adapter accepts and the Edge Cookie and geo settings startup refuses left to do. No location header is selected, a handler is removed only where every path its pattern matches is closed, a host header is reported to check, and page entries are added to a document written for main alone. The flags, the validation's reach, the consent lists, the adapters, consent and privacy, failure, compatibility and rollout, the tests, and 39 breaking entries. Fixes from review round 2. A handler pattern with a quantifier straight after its prefix or bound, or a character class or comment after them, is left to do, and a check line names the region the Cloudflare lookup now reads. The checks made only at startup, the location main's consent saw, a document run on two adapters, the stored assets, the passwords, the tests and the status line. | +| 2026-10-26 | The permission policy is a file the settings name and the push supplies. A document that names none is an item left to do, whose line names the file's stem and is the same on every adapter, a document written for main may name its policy before or after it is migrated, the command never chooses a policy or writes a policy file, and its validation leaves the policy check out and follows the pattern rule for `permissions.sha256` and `permissions.text`. The consent lists are carried into the publisher's file, as the command's line says. The count of breaking entries is 41, and the Cloudflare config store has a row. The status names IABTechLab/trusted-server#1409, whose code the references cite, the EdgeZero version is `v0.0.10`, the settings lines are brought up to date, the staged push is described as `v0.0.10` does it, a host's blob is staged with `--key`, and §6 gains the step for Cloudflare's config namespace. Step 7 of §6 says only the stored pages are fetched again, because core holds no copy of a file once the asset cache is a module. | ## Review log From 69a8a4612dfb1759d688c004c5138aff4716fc00 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:43 +0000 Subject: [PATCH 187/192] Four designs say what their rows hold The client hints design says it describes the stack from row 52, where every adapter runs request filters, because below that row only the Fastly adapter runs them. The no modules crate design names the order test the ts list has. The Fastly names design names the commit its count of 173 lines is taken at. The module scripts design's first review names the row's branch rather than a local one. --- .../specs/2026-10-14-51degrees-client-hints-design.md | 2 +- docs/superpowers/specs/2026-10-14-no-modules-crate-design.md | 2 +- .../specs/2026-10-15-fastly-names-in-core-design.md | 4 ++-- .../specs/2026-10-19-module-scripts-in-crates-design.md | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md index b68fd0838a..7270765dfb 100644 --- a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md +++ b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md @@ -1,6 +1,6 @@ # Design Spec: A deployment using 51Degrees gets client hints from one middleware -**Status:** Proposed, 2026-10-14. Raised as pull request IABTechLab/trusted-server#1386 (from jwrosewell/trusted-server#39), for Tech Lab review. +**Status:** Proposed, 2026-10-14. Raised as pull request IABTechLab/trusted-server#1386 (from jwrosewell/trusted-server#39), for Tech Lab review. This design describes the stack from row 52 (IABTechLab/trusted-server#1410) up, where every adapter runs request filters. Below row 52 only the Fastly adapter runs them, as this row's changelog says. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-12-shared-vendor-tables-design.md` (the `[shared.51degrees]` table), `2026-10-12-loading-cache-design.md` (the `51degrees-answers` cache), `2026-10-12-page-middleware-design.md` (the `[[serve]]` entries), `2026-10-14-source-modules-design.md` (the source a page is fetched from) and `2026-10-17-request-filters-on-every-adapter-design.md` (the request filters every adapter runs), all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** jwrosewell/trusted-server#39, IABTechLab/trusted-server#1386 diff --git a/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md b/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md index e2a1bad947..c6a6c33a54 100644 --- a/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md +++ b/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md @@ -84,7 +84,7 @@ The seam probe is a test fixture that no deployment ships, so its renames reach - The Axum adapter's linked builders give what the stock list gave: no loading cache module is linked, so every feature loads through `NoLoadingCache` unless a deployment links one (`2026-10-12-loading-cache-design.md`), every page module refuses a setting it does not know, the diagnostics module's request preparer is attached, the template's entries and the guide's table name the middleware the linked modules supply in order, the documented tables validate when uncommented, deploy validation runs a module's own rules and lists its secret settings, the ad server keeps an APS winner's identities, and every crate outside core declares its maintainers. - `ts audit` finds a module's selection by its integration id from the builders the tool links, and writes `text/html` entries that run the selected modules' middleware in registration order. -Three things have no test. The first is the five lists agreeing. Each adapter's order test pins its own list against a copy of the same twenty names, so the four lists agree today, but nothing compares one list with another, and the `ts` list has no order test of its own. Its order is checked only through `build_draft_config_places_the_page_changes_of_the_modules_it_selects`, for the modules that test selects, and the integration tests crate validates its fixtures against the Axum adapter's list alone. Adding a module takes five edits and a missed one passes every test, so a test that gives the same ids in the same order from all five lists is missing. The second is the registry accepting a described middleware that the registration leaves out, because no test has a registration omit a middleware its builder describes. The third is `ts audit` leaving out a middleware described for another media type, because the draft test that checks the entries selects `gpt`, `datadome`, `didomi` and `google_tag_manager`, whose middleware are all described for `text/html`, and `build_page_entries` has no direct test. +Three things have no test. The first is the five lists agreeing. Each adapter's order test, and the `ts` list's (`the_linked_modules_are_offered_in_hook_order` in `crates/trusted-server-cli/src/app_config.rs`), pins its own list against a copy of the same twenty names, so the five lists agree today, but nothing compares one list with another, and the integration tests crate validates its fixtures against the Axum adapter's list alone. Adding a module takes five edits and a missed one passes every test, so a test that gives the same ids in the same order from all five lists is missing. The second is the registry accepting a described middleware that the registration leaves out, because no test has a registration omit a middleware its builder describes. The third is `ts audit` leaving out a middleware described for another media type, because the draft test that checks the entries selects `gpt`, `datadome`, `didomi` and `google_tag_manager`, whose middleware are all described for `text/html`, and `build_page_entries` has no direct test. ## Revision record diff --git a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md index b013715b6a..dbe2c78c25 100644 --- a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md +++ b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md @@ -137,7 +137,7 @@ Core is built on EdgeZero and names no hosting platform in any line, whether cod | Fermyon | 1 (1) | 1 | 0 | | Every other name or domain | 0 | 0 | 0 | -The guard found 172 lines before this change, some of them naming two platforms, and none after. All 172 were comments, documents or test code. +At `13d57b03f`, this row's last commit before its comments, documents and tests were changed, the guard finds 173 lines. One is the `_vercel` entry of the attestation prefixes, which core no longer has, and the other 172, some of them naming two platforms, are comments, documents or test code. At this row's head the guard finds none. **What changes for an operator.** Nothing that core does when it runs changes. @@ -178,7 +178,7 @@ Not tested: | 2026-10-15 | Fixes from review round 1. The readers of each declared list and the open questions on them, `x-debug-fastly-pop`, the scheme change, the effects on the other adapters, where each refusal happens, the rollout, the narrowed constants, `x-compress-hint`, the file count, the untested paths and the review log. | | 2026-10-15 | Corrections from review round 2. The Fastly sentence of 2.3 and the changelog on the `x-debug-fastly-pop` refusal, the changelog on the spoofable strip and the scheme change, when the Axum, Cloudflare and Spin adapters strip `fastly-client-ip`, and the comment in `request_headers.rs` left for a later change. | | 2026-10-16 | Core names no hosting platform or platform domain in any line. Section 2.6 with the counts, the `_vercel` prefix and the test host, the guard and the tests that moved to the Fastly adapter, and section 3 no longer leaves core's comments and tests for later. | -| 2026-10-26 | Row 12 no longer adds `ORIGIN_ASSET_PREFIXES`, so core holds no `_vercel` entry before this change. Section 2.6 counts 172 lines before this change, the count of 2026-10-16 less that entry, all of them comments, documents or test code, and no other section names the prefix. | +| 2026-10-26 | Row 12 no longer adds `ORIGIN_ASSET_PREFIXES`, so core holds no `_vercel` entry before this change. Section 2.6 names the commit it counts at, `13d57b03f`, where the guard finds 173 lines, that entry and 172 comments, documents or test code, and no other section names the prefix. | | 2026-10-26 | The header no longer calls the pull request a draft. | ## Review log diff --git a/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md b/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md index 5fe1e9e716..717f410f4b 100644 --- a/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md +++ b/docs/superpowers/specs/2026-10-19-module-scripts-in-crates-design.md @@ -115,5 +115,5 @@ The `tsjs` guide said there are 12 integration source modules and 13 bundles. Th ## Review log -1. **Round 1, 2026-10-19.** A review in a fresh session, against `fixpass11/55-scripts-in-crates` as first committed, before it was rewritten and pushed. Issues found, two of them blocking. TypeScript had no mapping for `@tsjs/module/*`, so the GPT script was no longer type-checked against the APS renderer while this design said TypeScript used the names, and the configuration guide still linked the old Tinybird README. Of the others, editors found no settings for crate scripts, a script in a crate cannot import an npm package and the design did not say so, `build.rs` did not watch `build-module-options.mjs`, the crate lint run passed an empty match, the JavaScript and Rust discovery differed on broken links, an `index.ts` folder and a module named `core`, crate `.mjs` tests were neither run nor given Node's globals, and there was no changelog entry. This revision fixes each of them. +1. **Round 1, 2026-10-19.** A review in a fresh session, against row 55's branch as first committed, before it was rewritten and pushed. Issues found, two of them blocking. TypeScript had no mapping for `@tsjs/module/*`, so the GPT script was no longer type-checked against the APS renderer while this design said TypeScript used the names, and the configuration guide still linked the old Tinybird README. Of the others, editors found no settings for crate scripts, a script in a crate cannot import an npm package and the design did not say so, `build.rs` did not watch `build-module-options.mjs`, the crate lint run passed an empty match, the JavaScript and Rust discovery differed on broken links, an `index.ts` folder and a module named `core`, crate `.mjs` tests were neither run nor given Node's globals, and there was no changelog entry. This revision fixes each of them. 2. **Round 2, 2026-10-19.** A lighter review in a fresh session, against this page at `b89557183`, of the round 1 fixes and the new text. No blocking issues. One round 1 item was only partly fixed, the claim that every import of one module by another uses `@tsjs/module/`, which the Prebid shim's relative import of the APS renderer contradicted. It also found that `crates/tsconfig.json` left out the shared `global.d.ts`, that ESLint 9 finds its config from the folder it runs in rather than from the file, that the build reads `crates/tsconfig.json` without `build.rs` watching it, that Rust discovery failed on a module root that is a file where JavaScript passed over it, that the new configuration files and `.mjs` test were outside the format check, and that a Tinybird change now starts the documentation proposal workflow. This revision fixes each of them, and the design joins the specs pull request when the row is pushed. From dfef068e80d3bcbd4c47b11345dc658cf7521173 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:44 +0000 Subject: [PATCH 188/192] The cache designs name the caching crate's release 4.5.13 Row 31 moves fiftyone-caching from 4.5.8 to 4.5.13. In the loading cache design, the paragraph on the pin, the table of how rows 24 and 31 differ and the source link say so, and the published 4.5.8-49 and 4.5.13 are compared file by file. The integrations crate design's license sentence names 4.5.13. --- docs/superpowers/specs/2026-10-12-loading-cache-design.md | 7 ++++--- .../specs/2026-10-26-integrations-crate-design.md | 3 ++- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index 33c8877334..6d7e823d6e 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -51,7 +51,7 @@ store_idle_seconds = 2592000 **A module nothing offers.** A `module` that no linked crate offers a cache from is refused by the deploy check, `ts config validate`, and again when the adapter builds its state. Both use one lookup, `named_cache` in `integrations/mod.rs`, whose message names the table and lists the loading cache modules the build has. The deploy check runs it through `check_loading_cache_modules` from `validate_settings_for_deploy_with`. A stock build has no such module, so it refuses every `module`, `51degrees` included, and its message ends `The loading cache modules this build has are: none`, because `offered_cache_names` writes `none` for an empty list. -**The implementation.** The implementation is the module crate `crates/loading-cache/51degrees` (package `trusted-server-loading-cache-51degrees`, module `loading-cache.51degrees`), which 51Degrees maintains and which its manifest marks `vendor owned`. The module crate is written with the published [`fiftyone-caching`](https://crates.io/crates/fiftyone-caching) crate ([source](https://github.com/51Degrees/rust/tree/e2fff6f66097b81d831b7e93c9c35372831e4cc4/caching), at the commit that `4.5.8` was published from). The workspace builds the published crate with its default features off, so the published crate's own Fastly, Cloudflare, Spin and tokio modules are not compiled. The loading logic comes from the published crate, and the stores are written in the module crate (`stores.rs`) over core's `PlatformValueCache` and the host's key-value store. The module crate's `builder()` offers the cache through `IntegrationBuilder::with_loading_cache`. A deployment that wants the cache depends on the module crate and hands `builder()` to its adapter, through `run_with` on Fastly or `routes_with_registrations` on Axum, Cloudflare and Spin. Each of those runs a deployment's builders after the stock ones, and each adapter calls `loading_caches_from` with all of them, so no adapter names the module crate. +**The implementation.** The implementation is the module crate `crates/loading-cache/51degrees` (package `trusted-server-loading-cache-51degrees`, module `loading-cache.51degrees`), which 51Degrees maintains and which its manifest marks `vendor owned`. The module crate is written with the published [`fiftyone-caching`](https://crates.io/crates/fiftyone-caching) crate ([source](https://github.com/51Degrees/rust/tree/bd8428c07c4df89550fff4d08f24a2d552f4e97c/caching), at the commit that `4.5.13` was published from). The workspace builds the published crate with its default features off, so the published crate's own Fastly, Cloudflare, Spin and tokio modules are not compiled. The loading logic comes from the published crate, and the stores are written in the module crate (`stores.rs`) over core's `PlatformValueCache` and the host's key-value store. The module crate's `builder()` offers the cache through `IntegrationBuilder::with_loading_cache`. A deployment that wants the cache depends on the module crate and hands `builder()` to its adapter, through `run_with` on Fastly or `routes_with_registrations` on Axum, Cloudflare and Spin. Each of those runs a deployment's builders after the stock ones, and each adapter calls `loading_caches_from` with all of them, so no adapter names the module crate. **After the fold.** Row 56 folds every module crate into one crate, `crates/trusted-server-integrations` (`2026-10-26-integrations-crate-design.md`). The module's code moves to `src/loading-cache/51degrees/` in that crate, and the crate compiles the folder's cache, the code that uses `fiftyone-caching`, only with its Cargo feature `loading-cache-51degrees`, which is also the only thing that makes `fiftyone-caching` a dependency. The folder's module name is a constant that every build compiles, so the crate's test of module names checks it whatever features are on. No stock adapter and not the `ts` tool turns the feature on, so a stock build still compiles no `fiftyone-caching`. A deployment that links the module turns the feature on in its own `Cargo.toml` and hands the module's builder to its adapter, as above. @@ -61,13 +61,13 @@ store_idle_seconds = 2592000 **Why.** The same answer is wanted by every instance and every request, and a cache that lets every miss load at once asks a module's service as many times as there are requests. Loading each key once at a time in a process, with the value shared by every waiter, is the pattern of `LruLoadingCache` in the 51Degrees [.NET](https://github.com/51Degrees/caching-dotnet) and [Java](https://github.com/51Degrees/pipeline-java) pipelines, which the published crate's documentation points to. Core keeps only the interface. Core's cache code depends on no caching package, and once the module leaves the stock lists no stock build depends on one either. The only place core writes the module name `51degrees` is `settings.rs`, in the documentation of `LoadingCacheConfig::module` (line 3102) and in the test `a_loading_cache_module_is_read_and_written_back` (lines 7074 to 7093), and the open questions ask about it. -**The pin.** Row 24 pins `fiftyone-caching` at the pre-release `4.5.8-49` and row 31 at the release `4.5.8`. Row 31 changes `Cargo.toml` and `Cargo.lock` and nothing else, and has no design of its own. The two published crates, [`4.5.8-49`](https://crates.io/crates/fiftyone-caching/4.5.8-49) and [`4.5.8`](https://crates.io/crates/fiftyone-caching/4.5.8), were downloaded from crates.io on 2026-10-16 and compared file by file. The 21 files under `src/`, the 16 files under `tests/` and `README.md` are identical in the two. The other four files differ. `Cargo.toml`, `Cargo.toml.orig` and `Cargo.lock` differ in the crate's own version and in the version of `fiftyone-pipeline-core` that the crate names as an optional dependency, and `Cargo.lock` also holds the checksum of that version of `fiftyone-pipeline-core`. `.cargo_vcs_info.json` names the commit that each crate was published from, [`762578bf6`](https://github.com/51Degrees/rust/commit/762578bf65767856f6f3a5f5945dfb6f8679633a) for `4.5.8-49` and [`e2fff6f66`](https://github.com/51Degrees/rust/commit/e2fff6f66097b81d831b7e93c9c35372831e4cc4) for `4.5.8`, and marks both as published from a working tree with uncommitted changes. Everything this document says about the published crate therefore applies to both rows. The published crate's manifest gives its license as `EUPL-1.2`, and it is a dependency, so none of its files is in the repository. The module crate takes the workspace license, Apache-2.0, as every crate under `crates/` does. +**The pin.** Row 24 pins `fiftyone-caching` at the pre-release `4.5.8-49` and row 31 at the release `4.5.13`. Row 31 changes `Cargo.toml` and `Cargo.lock` and nothing else, and has no design of its own. The two published crates, [`4.5.8-49`](https://crates.io/crates/fiftyone-caching/4.5.8-49) and [`4.5.13`](https://crates.io/crates/fiftyone-caching/4.5.13), were downloaded from crates.io and compared file by file. The 21 files under `src/`, the 16 files under `tests/` and `README.md` are identical in the two. The other four files differ. `Cargo.toml`, `Cargo.toml.orig` and `Cargo.lock` differ in the crate's own version and in the version of `fiftyone-pipeline-core` that the crate names as an optional dependency, and `Cargo.lock` also holds the checksum of that version of `fiftyone-pipeline-core` and lists `log` among its dependencies. `.cargo_vcs_info.json` names the commit that each crate was published from, [`762578bf6`](https://github.com/51Degrees/rust/commit/762578bf65767856f6f3a5f5945dfb6f8679633a) for `4.5.8-49` and [`bd8428c07`](https://github.com/51Degrees/rust/commit/bd8428c07c4df89550fff4d08f24a2d552f4e97c) for `4.5.13`, and marks both as published from a working tree with uncommitted changes. Everything this document says about the published crate therefore applies to both rows. The published crate's manifest gives its license as `EUPL-1.2`, and it is a dependency, so none of its files is in the repository. The module crate takes the workspace license, Apache-2.0, as every crate under `crates/` does. **Where row 24 and row 31 differ.** The two rows differ in four places. | What | Row 24 | Row 31 | | ---------------------------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| The pin on `fiftyone-caching` | The pre-release `4.5.8-49` | The release `4.5.8` | +| The pin on `fiftyone-caching` | The pre-release `4.5.8-49` | The release `4.5.13` | | Where the key prefix comes from | `ServerKeyPrefix::for_settings`, from the loaded settings' `publisher.domain`, always headed `ts` | The `[prefix]` module (row 27) through `RuntimeServices::server_key_prefix()`, headed `ts` by core's own module, with a trailing dot dropped from the domain | | What asks the cache for robots.txt | The robots.txt route handler in `robots_txt/mod.rs` | The `robots_txt` middleware's preparation in `robots_txt/middleware.rs` (row 25) | | What the log keeps of a module's failure | The whole report the module gives | The report with each address in it cut to its scheme and host (row 25), so a key in an address's path, query or user part never reaches the log | @@ -233,6 +233,7 @@ These are for the reviewer. This revision records what the code does, and what t | 2026-10-26 | Fixes from review round 3. The change takes in row 32, whose probe test counts the 51Degrees module as linked, and each adapter's order test, and puts a guard beside every stock list, the `ts` tool's included, with a test that pins the tool's list. What a stock build costs is spelled out, being a call to the cloud service on each page the client hints middleware is placed for, a key-value write on each robots.txt request where a deployment adds a robots.txt module, and the README promise of the 51Degrees shared code. The other text that says a stock build holds answers is listed with the change, with a changelog entry for the refused `module = "51degrees"`. Line numbers are those of the current heads, row 31 at `490a4665d` and row 24 at `e66b955a4`, and the design says what the code there does, being the host in each held answer's key, a module's failure logged with each address cut to its scheme and host on row 31, and the adapter robots.txt tests of row 24. The searches give their output from row 44, the open question on a repeated changelog entry, which no row repeats now, is gone, one on falling back to `NoLoadingCache` without a word is added, open question 1 names row 55 and core's `loading-cache.example`, the CI results are linked at their commits, and the counts of the local checks are gone. | | 2026-10-26 | Fixes from review round 4. Row 32's first case links a second loading cache module, a test builder offering core's `MemoryLoadingCache`, so that the case still shows the table is read. The text the change rewrites adds the integration guide, the 51Degrees client's comments and the client hints middleware's tests. The 51Degrees shared code's loader is named as a caller that captures a request's own services, "stock build" is defined where it is first used, the source link names the commit that `4.5.8` was published from, the test of module names checks the constant whatever features are on, and the decision is placed after the Touchbase. | | 2026-10-26 | The change to rows 24, 32 and 43 is described as built, so the line numbers paragraph no longer says this page is written ahead of its code, and its line numbers stay those of the code before the change. | +| 2026-10-27 | Row 31 pins the release `4.5.13` in place of `4.5.8`. The two published crates the rows pin, `4.5.8-49` and `4.5.13`, are compared file by file again, and the source link names the commit that `4.5.13` was published from. | ## Review log diff --git a/docs/superpowers/specs/2026-10-26-integrations-crate-design.md b/docs/superpowers/specs/2026-10-26-integrations-crate-design.md index 86b3dd360c..ea2fb9d2fa 100644 --- a/docs/superpowers/specs/2026-10-26-integrations-crate-design.md +++ b/docs/superpowers/specs/2026-10-26-integrations-crate-design.md @@ -184,7 +184,7 @@ Every file in the integrations crate takes the repository's license, Apache-2.0, git grep -n -i -E 'EUPL|European Union Public|SPDX-License-Identifier' -- crates ``` -The package `fiftyone-caching` 4.5.8, which the 51Degrees loading cache folder uses, gives its license as `EUPL-1.2` in its manifest and on crates.io. It is a dependency, so none of its files is in the repository, and only that folder uses it, behind the feature of section 2.8. +The package `fiftyone-caching` 4.5.13, which the 51Degrees loading cache folder uses, gives its license as `EUPL-1.2` in its manifest and on crates.io. It is a dependency, so none of its files is in the repository, and only that folder uses it, behind the feature of section 2.8. ### 2.8 Cargo features @@ -325,6 +325,7 @@ These are for the reviewer, and this design decides none of them. | 2026-10-26 | Fixes from review round 1. tokio's `rt-multi-thread`, which the loading cache's tests ask for, goes in a table that only native builds read, with the threaded tests on the host. The paths that climb out of a crate's folder are listed with their fixes, and `run.cjs`, `crates/fastly.toml`, the seam probe's workflow step and the core documents that name a crate's path join section 3. The browser scripts' second reason says that the APS renderer and Prebid's code stay in `trusted-server-js`, where #1194 would move them. Row 56's own measurement goes in its pull request. The manifest's lint and doctest settings, the source labels written as text, the log labels on Fastly, the places the Rust names are used, the rule for `51degrees`, the probe's one name of another type, the Fastly SDK's host imports, the files with license text of their own, the count of rows a fold would travel and where vendor names live beside #1194 are stated. Open questions 4 and 5 are reworded and 7 and 8 added. The Touchbase of 2026-10-23 says what Tech Lab asked, and that 51Degrees decided, and the seam probe and Viceroy are defined where first used. | | 2026-10-26 | Fixes from review round 2. The split design's pull request names its author, Aram Grigoryan. The account of the Touchbase gives Tech Lab's request as the record has it, and places the decisions of 51Degrees after the Touchbase. "Stock build" is defined where first used, the split design's integrations crate is the one that puts the catalog together, and a feature gates neither a folder's name constants nor its source label. `build-fastly`, `check-fastly` and `test-fastly` name `asset-cache-51degrees` and none of them names `loading-cache-51degrees`, each `maintainers.toml` comment says "module" where the manifest says "crate", and the sample permission policy joins section 3. | | 2026-10-26 | The count of the other design documents naming a path under `crates///` is 43, with the search that gives it. | +| 2026-10-27 | The license sentence names `fiftyone-caching` 4.5.13 in place of 4.5.8, because row 31 pins 4.5.13. | ## Review log From dcb72ba35ecab58b7a1ebd9b66eec53430134917 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:45 +0000 Subject: [PATCH 189/192] The designs carry the date they are proposed, 2026-10-27 The three designs added in this pass are named for 2026-10-27, and every status line, last updated line and revision row that said 2026-10-26 says 2026-10-27, with the links to the three renamed files. --- ...integration-response-header-hook-design.md | 6 ++-- .../2026-07-30-permission-model-design.md | 6 ++-- ...07-30-provider-migration-rollout-design.md | 4 +-- ...tration-off-the-publisher-domain-design.md | 10 +++---- .../specs/2026-10-12-config-by-host-design.md | 4 +-- .../2026-10-12-diagnostic-headers-design.md | 6 ++-- .../2026-10-12-inspection-endpoints-design.md | 4 +-- .../specs/2026-10-12-loading-cache-design.md | 16 +++++----- ...026-10-14-51degrees-client-hints-design.md | 4 +-- .../2026-10-14-no-modules-crate-design.md | 4 +-- .../specs/2026-10-15-asset-cache-design.md | 18 +++++------ .../2026-10-15-fastly-names-in-core-design.md | 6 ++-- ...-10-15-module-cache-declarations-design.md | 6 ++-- ...17-asset-routes-on-every-adapter-design.md | 4 +-- .../specs/2026-10-17-config-migrate-design.md | 6 ++-- ...request-filters-on-every-adapter-design.md | 6 ++-- ...2026-10-27-developer-experience-design.md} | 30 +++++++++---------- ...> 2026-10-27-integrations-crate-design.md} | 12 ++++---- ... => 2026-10-27-module-endpoints-design.md} | 10 +++---- 19 files changed, 81 insertions(+), 81 deletions(-) rename docs/superpowers/specs/{2026-10-26-developer-experience-design.md => 2026-10-27-developer-experience-design.md} (98%) rename docs/superpowers/specs/{2026-10-26-integrations-crate-design.md => 2026-10-27-integrations-crate-design.md} (99%) rename docs/superpowers/specs/{2026-10-26-module-endpoints-design.md => 2026-10-27-module-endpoints-design.md} (98%) diff --git a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md index bc8dfd3e97..c4d57d5905 100644 --- a/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md +++ b/docs/superpowers/specs/2026-07-30-integration-response-header-hook-design.md @@ -15,7 +15,7 @@ retained as the design bar for the hook when its first consumer arrives (an integration that must set response headers such as Accept-CH or detection results). Revised 2026-08-25, on 2026-10-06 for the module wording, on 2026-10-15 and 2026-10-16 after its round 1 and round 2 reviews (Review log), -and on 2026-10-26 for the permission policy, which is selected when a service +and on 2026-10-27 for the permission policy, which is selected when a service is deployed. **Author:** Engineering **Issue references:** #782 @@ -23,7 +23,7 @@ is deployed. `2026-07-30-permission-model-design.md`, `2026-07-30-provider-migration-rollout-design.md` and the baseline DataDome design `2026-06-11-datadome-server-side-protection-design.md` -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Context.** Issue #782 already specifies this feature well; its done-when > is the contract. PR #838 shipped the trait and registry wiring with **no @@ -1630,7 +1630,7 @@ changes no code and decides none of them. | 2026-10-06 | Module wording. | | 2026-10-15 | Fixes from review round 1. The status and the closed pull request, a new section 1a on what the code does today, the cookie rule, the shared finalization code and the DataDome section name stated as the code has them, the reserved CDN cache fields taken from `EDGE_CACHE_HEADER_NAMES`, the three ids the id rule refuses, the template cache and the cache revision tuple marked as unreconciled or not built, the changes to shipped DataDome behavior in section 4a.4, the order of sections 4 and 4a, the sign-off rows, `must-revalidate` precedence, the duplicate-ID test in the done-when, the existing tests, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. What the providers design now says (section 1a), the two core functions that every adapter's `apply_finalize_headers` calls (sections 1a and 2), the closed pull request that carried the documentation set (#1047, section 7), the revision dates and the review log. | -| 2026-10-26 | Section 1a keeps that each build embeds the repository's sample at its commit, and says that row 4 of the stack (#1345) builds section 3.1 of the permission model design, so that once row 4 is merged forward a domain's blob, which names its policy file by name and SHA-256, is the policy identity and gives `active_policy_digest` its value. Open question 2 says that row 4's policy `sha256` reaches the template fingerprint, which leaves the policy's text out and which the compiled sample never reached. The stack's pull requests are named without calling them drafts. | +| 2026-10-27 | Section 1a keeps that each build embeds the repository's sample at its commit, and says that row 4 of the stack (#1345) builds section 3.1 of the permission model design, so that once row 4 is merged forward a domain's blob, which names its policy file by name and SHA-256, is the policy identity and gives `active_policy_digest` its value. Open question 2 says that row 4's policy `sha256` reaches the template fingerprint, which leaves the policy's text out and which the compiled sample never reached. The stack's pull requests are named without calling them drafts. | ## Review log diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index c5a4174a61..24b9aa5b9e 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -19,7 +19,7 @@ name a module by its crate folder, so the selector reads `[permission-signal] modules`. Revised on 2026-10-15 after a review of this page against the code at `split/3-permissions` (see the Review log), so that §1 to §8 and the records in §10 to §13 state what that code does. Revised on -2026-10-26 for the policy file, which is never compiled into a build, as +2026-10-27 for the policy file, which is never compiled into a build, as agreed with IAB Tech Lab at the Trusted Server Touchbase on 2026-10-23, and which is selected and supplied when a service is deployed, as decided by 51Degrees after the Touchbase (§3.1, §3.6 and §3.7). Row 4 builds that design, @@ -31,7 +31,7 @@ IABTechLab/trusted-server#1350 and IABTechLab/trusted-server#1352. **Issue references:** #779 **Related specs:** `2026-07-30-pluggable-providers-design.md`, `2026-07-30-provider-migration-rollout-design.md` -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Context.** PR #838 proposed a permission model whose review surfaced two > classes of defect this spec exists to prevent. The first was silent @@ -2029,7 +2029,7 @@ include the changes that followed the first revision of this section on | 2026-10-07 | A module is named by its crate folder, so the selector reads `[permission-signal] modules`. | | 2026-10-15 | Fixes from review round 1. An unreadable record answered by its own module with nothing ahead of the modules, the fifth module `mtm` with its own section, registration through each adapter, the four members of the page state, the sharing pair on navigation and page-bids, the persisted consent, resolve endpoint and device module check named as built by other rows or not at all, what the policy parser does not reject, the Data Uses a signal reaches, the raw strings removed after assembly, a table of what each adapter runs, the upgrade list, the open questions and the review log. | | 2026-10-16 | Review round 2, approved with one minor point recorded in the Review log. | -| 2026-10-26 | The policy file is never compiled into a build, as agreed with IAB Tech Lab at the Touchbase on 2026-10-23, and is selected and supplied when a service is deployed, as decided by 51Degrees after the Touchbase (§3.1). What row 4 builds and later rows add, Cloudflare reading its blob from the config store, the push, answers that follow the error's kind, and a blob's propagation (§3.1). What the build carries (§3.6), the cost of the fetch (§3.7), the failure modes (§6), the upgrade list (§6.1), the tests (§8), the records in §10 and §11, and the sample named as the sample. | +| 2026-10-27 | The policy file is never compiled into a build, as agreed with IAB Tech Lab at the Touchbase on 2026-10-23, and is selected and supplied when a service is deployed, as decided by 51Degrees after the Touchbase (§3.1). What row 4 builds and later rows add, Cloudflare reading its blob from the config store, the push, answers that follow the error's kind, and a blob's propagation (§3.1). What the build carries (§3.6), the cost of the fetch (§3.7), the failure modes (§6), the upgrade list (§6.1), the tests (§8), the records in §10 and §11, and the sample named as the sample. | ## Review log diff --git a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md index fa47ea57d1..4fc3e5373e 100644 --- a/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md +++ b/docs/superpowers/specs/2026-07-30-provider-migration-rollout-design.md @@ -13,7 +13,7 @@ and the `consent.gdpr.applies_in` and `consent.us_states.privacy_states` lists retire into the same tree (permission-model spec §3.2, §3.4 and §12). Updated on 2026-10-06 to say module where it said provider, with `provider` kept for the two auction selectors (pluggable spec §2.1). Updated on -2026-10-26, where the permission policy is the publisher's file, selected +2026-10-27, where the permission policy is the publisher's file, selected and supplied when a service is deployed and never compiled into a build (permission-model spec §3.1), which row 4 of the stack (#1345) builds. **Author:** Engineering @@ -22,7 +22,7 @@ and supplied when a service is deployed and never compiled into a build `2026-07-30-permission-model-design.md`, `2026-07-30-integration-response-header-hook-design.md` (not implemented in the series, see its status) -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Context.** The module/permission epic is a breaking change to a live > identity system. PR #838's review showed that the riskiest part of such a diff --git a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md index 617ef78742..d5cd4e612d 100644 --- a/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md +++ b/docs/superpowers/specs/2026-10-12-administration-off-the-publisher-domain-design.md @@ -2,10 +2,10 @@ **Status:** Proposed, 2026-10-12. Raised as pull request [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354), row 13 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), answering [IABTechLab/trusted-server#1271](https://github.com/IABTechLab/trusted-server/issues/1271), for Tech Lab review. The two earlier pull requests for the same commits, [jwrosewell/trusted-server#8](https://github.com/jwrosewell/trusted-server/pull/8) and [IABTechLab/trusted-server#1298](https://github.com/IABTechLab/trusted-server/pull/1298), are closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-inspection-endpoints-design.md` (row 9, which builds `GET /_ts/data`) and `2026-10-26-module-endpoints-design.md` (row 12, which this row stacks on), in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) +**Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-inspection-endpoints-design.md` (row 9, which builds `GET /_ts/data`) and `2026-10-27-module-endpoints-design.md` (row 12, which this row stacks on), in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) **Related PRs:** [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354) (row 13), [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353) (row 12, which row 13 stacks on), [IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350) (row 9, the inspection endpoints), [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7, the integration seam, which replaced the closed [IABTechLab/trusted-server#1094](https://github.com/IABTechLab/trusted-server/pull/1094)), [IABTechLab/trusted-server#1377](https://github.com/IABTechLab/trusted-server/pull/1377) (row 36, which later changes the closed path matcher) **Issues:** [IABTechLab/trusted-server#1271](https://github.com/IABTechLab/trusted-server/issues/1271), and [IABTechLab/trusted-server#1149](https://github.com/IABTechLab/trusted-server/issues/1149) for a store the development files name. [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 ## 1. Problem @@ -63,7 +63,7 @@ The matcher is the one that was there before. It tests the path as sent and afte Fastly, Cloudflare and Spin make the check the first step of their fallback, ahead of the modules' request preparers (each can clean up a query parameter or cookie its module reserves). Axum makes the check in `fallback_handler`, ahead of the preparers, in the same order (`23bde99bc`). -The check sits in the publisher fallback, so a route that the router answers first is not covered by the check. After this change no adapter registers a named route beneath a closed path, so every request for a closed path reaches the check. A module's route is answered in the publisher fallback after the check, so a module route beneath a closed path is never reached (`2026-10-26-module-endpoints-design.md`). +The check sits in the publisher fallback, so a route that the router answers first is not covered by the check. After this change no adapter registers a named route beneath a closed path, so every request for a closed path reaches the check. A module's route is answered in the publisher fallback after the check, so a module route beneath a closed path is never reached (`2026-10-27-module-endpoints-design.md`). **On the publisher's own site.** @@ -134,8 +134,8 @@ One question is open, about the guides, and this spec does not decide it. | 2026-10-16 | Fixes from review round 2. Open question 5 names the changelog entry on the two store id settings and the checks on row 12 that also read them, the sandbox answer is stated to exist only in a build with the `reusable-sandbox` feature, and the check counts of 25 on the row 13 pull request and 19 on the fork's were read again and stand. | | 2026-10-16 | Fixes from code review. The test fake answers with the 404 text the CLI prints and refuses to replace a secret (`11bf7b76d`), an attestation endpoint beneath `/_ts/admin` is refused (`df0fc18a0`), Axum answers a closed path before the preparers (`23bde99bc`), every adapter test sends the purge path and Fastly's the double-encoded paths (`18aba165c`), and three comments and two changelog entries are corrected (`e75e307de`). Open questions 1 to 6 are answered. | | 2026-10-17 | For IABTechLab/trusted-server#1149, the development files and the Viceroy template no longer name `opid_store` (`1666c38e3`), and a test checks that every store a development file names is declared (`e110aa025`). | -| 2026-10-26 | Row 12 adds no endpoint the settings choose any more, so section 2 no longer names one among the routes the closed path check does not cover, and says that a module route beneath a closed path is never reached. The header names `2026-10-26-module-endpoints-design.md` for row 12. | -| 2026-10-26 | What `df0fc18a0` changed to refuse an endpoint the settings chose beneath `/_ts/admin`, and what `ee544e27f` changed in the Cloudflare test of that endpoint's routes, leave this row in the merge of row 12's removal, while both commits stay in its history. Section 5 drops the test `df0fc18a0` added, and section 6 says what leaves. Section 4 names the two checks of row 12 by function, with the store id fields, and the header no longer calls the pull request a draft. | +| 2026-10-27 | Row 12 adds no endpoint the settings choose any more, so section 2 no longer names one among the routes the closed path check does not cover, and says that a module route beneath a closed path is never reached. The header names `2026-10-27-module-endpoints-design.md` for row 12. | +| 2026-10-27 | What `df0fc18a0` changed to refuse an endpoint the settings chose beneath `/_ts/admin`, and what `ee544e27f` changed in the Cloudflare test of that endpoint's routes, leave this row in the merge of row 12's removal, while both commits stay in its history. Section 5 drops the test `df0fc18a0` added, and section 6 says what leaves. Section 4 names the two checks of row 12 by function, with the store id fields, and the header no longer calls the pull request a draft. | ## Review log diff --git a/docs/superpowers/specs/2026-10-12-config-by-host-design.md b/docs/superpowers/specs/2026-10-12-config-by-host-design.md index ed6c0800a5..20b45dc225 100644 --- a/docs/superpowers/specs/2026-10-12-config-by-host-design.md +++ b/docs/superpowers/specs/2026-10-12-config-by-host-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-03-19-edgezero-migration-design.md` and `2026-10-15-fastly-per-host-selector-design.md`, which replaces where this design reads the selector from **Related PRs:** IABTechLab/trusted-server#1348, IABTechLab/trusted-server#1352 **Issues:** IABTechLab/trusted-server#1270 -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** An operator who runs several publishers needs a Fastly service for each. This row lets one service read the settings of the publisher whose host a request names. @@ -182,7 +182,7 @@ The behaviors below are stated in the sections above. A change to the code would | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-16 | Round 1 fixes. The header points at the open draft pull request, the risk table is repaired, the fallback to the logical store ID is stated, the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests are added, and the code findings sit under Open questions. The selector store points at the Fastly per-host selector design. | | 2026-10-16 | Round 2 fixes. The pointer to the Fastly per-host selector design names section 2.4, whose `500` answers replace the fallback to the logical store ID, and sections 2.6 and 2.8, which replace the push and selector-writing steps. The same sections are named where this spec describes the fallback and the steps for moving a service. | -| 2026-10-26 | Each host's blob names the policy file row 4 selects, and one policy entry for each SHA-256 serves every host that pins it, as this row's keyed push and Fastly adapter do. Chunks and pointers are defined, the EdgeZero versions are anchored, `ts config diff` is said not to reach a host's blob, and Open questions, item 9, asks about a selector that forms a policy entry's key. | +| 2026-10-27 | Each host's blob names the policy file row 4 selects, and one policy entry for each SHA-256 serves every host that pins it, as this row's keyed push and Fastly adapter do. Chunks and pointers are defined, the EdgeZero versions are anchored, `ts config diff` is said not to reach a host's blob, and Open questions, item 9, asks about a selector that forms a policy entry's key. | ## Review log diff --git a/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md b/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md index 9855326a7b..36a17211ee 100644 --- a/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md +++ b/docs/superpowers/specs/2026-10-12-diagnostic-headers-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-page-middleware-design.md`, `2026-10-12-inspection-endpoints-design.md` and `2026-10-13-middleware-settings-design.md` (row 38, built after this row), in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342), which holds every design document of the stack, and `2026-09-25-git-version-header-design.md`, which main already holds **Related PRs:** [IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356), [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355) (row 14, robots.txt from modules, which this row stacks on), [IABTechLab/trusted-server#1351](https://github.com/IABTechLab/trusted-server/pull/1351) (row 10, page changes as middleware), [IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350) (row 9, inspection endpoints), [IABTechLab/trusted-server#1379](https://github.com/IABTechLab/trusted-server/pull/1379) (row 38, middleware settings), [IABTechLab/trusted-server#1213](https://github.com/IABTechLab/trusted-server/pull/1213) (the git version header, merged to main), [jwrosewell/trusted-server#10](https://github.com/jwrosewell/trusted-server/pull/10) (closed), [IABTechLab/trusted-server#1302](https://github.com/IABTechLab/trusted-server/pull/1302) (closed) **Issues:** [IABTechLab/trusted-server#1273](https://github.com/IABTechLab/trusted-server/issues/1273) -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** The server passed a reader of any page three headers from the publisher's origin and wrote version headers of its own, so anyone could learn which node answered, whether a cache did and which release ran, without anyone having decided to tell them. This spec says which headers now stay off a response, where the removal runs, what a deployment writes to put the versions back on its pages, and what the change leaves as it was. @@ -181,8 +181,8 @@ The fork's CI passed on the series head, `a43a05a33`, all nineteen checks, and t | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The five commits since 9 October, being the fallback removal, the response headers a serve middleware may write, the platform in the middleware context, the `service_version` middleware and the end of the version headers on every adapter, the function names, the paths beneath `/_ts` and what the exemption keeps, the adapters and the responses that skip the finalize step, the defaults and the rollout, what breaks, the failure and privacy answers, the tests, the open questions, the header and the acceptance lines brought to the series head, and the review log. | | 2026-10-16 | Fixes from review round 2. The acceptance says what the description of the pull request for this row records, being the fork's run of all nineteen checks on the head, the ticked check commands and the manual test left unticked, in place of saying that no description records a run of the whole check. Three clauses that were only partly true are qualified, being the headers a stored page replays from the origin, the headers each proxy takes off and the one header the prefix test sends. A new open question asks whether the headers that core or an adapter writes directly should move into a serve middleware. | -| 2026-10-26 | Row 12 no longer adds the signed document that reported the build and the service's version, so the versions are readable only where `service_version` writes them, and this change adds `PlatformIdentity` to core's platform module. The header, the pages beneath `/_ts`, the namespace note, the versions, the platform a middleware is told, what breaks, the rollout, `X-TS-ENV` and open questions 3 and 7 say so. | -| 2026-10-26 | The rule says where the git version and the service's version appear, the inspection pages give the core crate's version, the commits table names the series head `a43a05a33` and gains the commit that keeps `PlatformIdentity` in the platform module, the platform bullet names `fastly_platform_identity` and when each adapter reports its platform, and the header no longer calls the pull request a draft. | +| 2026-10-27 | Row 12 no longer adds the signed document that reported the build and the service's version, so the versions are readable only where `service_version` writes them, and this change adds `PlatformIdentity` to core's platform module. The header, the pages beneath `/_ts`, the namespace note, the versions, the platform a middleware is told, what breaks, the rollout, `X-TS-ENV` and open questions 3 and 7 say so. | +| 2026-10-27 | The rule says where the git version and the service's version appear, the inspection pages give the core crate's version, the commits table names the series head `a43a05a33` and gains the commit that keeps `PlatformIdentity` in the platform module, the platform bullet names `fastly_platform_identity` and when each adapter reports its platform, and the header no longer calls the pull request a draft. | ## Review log diff --git a/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md b/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md index 07e4a98af4..f274e30f06 100644 --- a/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md +++ b/docs/superpowers/specs/2026-10-12-inspection-endpoints-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-08-27-integration-provider-seam-design.md`, in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** IABTechLab/trusted-server#1350, IABTechLab/trusted-server#1348, jwrosewell/trusted-server#4 (closed), IABTechLab/trusted-server#1294 (closed) **Issues:** IABTechLab/trusted-server#1268 -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** A reader, a regulator or an auditor has only the operator's word for what a deployment runs, decides and keeps. Three endpoints in this change show those three things, and this spec says where each endpoint answers, what each shows and to whom. @@ -189,7 +189,7 @@ On `33282f0c9`, the last of the five original commits, the whole check also pass | 2026-10-12 | First draft, from the pull request and its issue as they stand. | | 2026-10-15 | Fixes from review round 1. The addresses with their methods, forms, statuses and headers, the defaults and the one switch, what runs around each handler, the data page's full header set and its protection from an operator's headers, the failure paths, the rollout, the tests and the untested paths, the open questions, the header and the acceptance lines brought to the series head, and the review log. | | 2026-10-16 | Fixes from review round 2. The statement that the preparers and filters never run for these addresses on Fastly and Spin, where a request with a method other than `GET` goes on Cloudflare, the matching wording of the second open question, and the review log. | -| 2026-10-26 | The permissions answer names the policy file row 4 selects, the configuration view shows its name and SHA-256 but never its text, a pattern under `permissions.text` is refused, and checks count `permissions.sha256` as present, as this row's code does. A failed load answers its error's status on every adapter, Spin included, because row 4 moved Spin off a fixed `503`. The status names the pull request but not as a draft. | +| 2026-10-27 | The permissions answer names the policy file row 4 selects, the configuration view shows its name and SHA-256 but never its text, a pattern under `permissions.text` is refused, and checks count `permissions.sha256` as present, as this row's code does. A failed load answers its error's status on every adapter, Spin included, because row 4 moved Spin off a fixed `503`. The status names the pull request but not as a draft. | ## Review log diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index 6d7e823d6e..ddcbbde080 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -1,11 +1,11 @@ # Design Spec: The loading cache core declares, and the module that supplies it -**Status:** Proposed, 2026-10-12. Revised 2026-10-26, so that no stock build (an adapter or the `ts` tool built from the repository as it ships, with the builders that its own list, its stock list, names) links a loading cache module (section 2, "No stock build links a cache"). Implemented as the pull request [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365), which is row 24 of the stack, answering [IABTechLab/trusted-server#1282](https://github.com/IABTechLab/trusted-server/issues/1282). The pull request [IABTechLab/trusted-server#1372](https://github.com/IABTechLab/trusted-server/pull/1372), row 31, completes the change by moving the pin on the published caching crate. The revision of 2026-10-26 describes a change to row 24, to row 32 ([IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373)) and to row 43 ([IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384)), where the stock list moved into each adapter. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-12. Revised 2026-10-27, so that no stock build (an adapter or the `ts` tool built from the repository as it ships, with the builders that its own list, its stock list, names) links a loading cache module (section 2, "No stock build links a cache"). Implemented as the pull request [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365), which is row 24 of the stack, answering [IABTechLab/trusted-server#1282](https://github.com/IABTechLab/trusted-server/issues/1282). The pull request [IABTechLab/trusted-server#1372](https://github.com/IABTechLab/trusted-server/pull/1372), row 31, completes the change by moving the pin on the published caching crate. The revision of 2026-10-27 describes a change to row 24, to row 32 ([IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373)) and to row 43 ([IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384)), where the stock list moved into each adapter. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-robots-txt-from-modules-design.md`, `2026-10-12-robots-txt-middleware-design.md`, `2026-10-12-prefix-module-design.md`, `2026-10-14-no-modules-crate-design.md`, `2026-10-15-module-cache-declarations-design.md`, `2026-10-15-asset-cache-design.md`, `2026-10-26-integrations-crate-design.md` +**Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-robots-txt-from-modules-design.md`, `2026-10-12-robots-txt-middleware-design.md`, `2026-10-12-prefix-module-design.md`, `2026-10-14-no-modules-crate-design.md`, `2026-10-15-module-cache-declarations-design.md`, `2026-10-15-asset-cache-design.md`, `2026-10-27-integrations-crate-design.md` **Related PRs:** [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24) was reviewed on [jwrosewell/trusted-server#19](https://github.com/jwrosewell/trusted-server/pull/19) (closed) and [IABTechLab/trusted-server#1372](https://github.com/IABTechLab/trusted-server/pull/1372) (row 31) on [jwrosewell/trusted-server#26](https://github.com/jwrosewell/trusted-server/pull/26) (closed). The integration seam is in [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348), robots.txt rules from modules in [IABTechLab/trusted-server#1355](https://github.com/IABTechLab/trusted-server/pull/1355), robots.txt as a middleware in [IABTechLab/trusted-server#1366](https://github.com/IABTechLab/trusted-server/pull/1366), the prefix module in [IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), module-declared caches in [IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373), the asset cache in [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375) and each adapter's own list of the modules it links in [IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384) **Issues:** [IABTechLab/trusted-server#1282](https://github.com/IABTechLab/trusted-server/issues/1282). [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** Each robots.txt module's answer is read from the key-value store by every request and asked of the module's service by every request that finds no fresh copy, so a burst of requests for one file asks the service once for each request, on every instance. Core has one cache, for page templates, and no interface through which a module can supply a cache. @@ -53,7 +53,7 @@ store_idle_seconds = 2592000 **The implementation.** The implementation is the module crate `crates/loading-cache/51degrees` (package `trusted-server-loading-cache-51degrees`, module `loading-cache.51degrees`), which 51Degrees maintains and which its manifest marks `vendor owned`. The module crate is written with the published [`fiftyone-caching`](https://crates.io/crates/fiftyone-caching) crate ([source](https://github.com/51Degrees/rust/tree/bd8428c07c4df89550fff4d08f24a2d552f4e97c/caching), at the commit that `4.5.13` was published from). The workspace builds the published crate with its default features off, so the published crate's own Fastly, Cloudflare, Spin and tokio modules are not compiled. The loading logic comes from the published crate, and the stores are written in the module crate (`stores.rs`) over core's `PlatformValueCache` and the host's key-value store. The module crate's `builder()` offers the cache through `IntegrationBuilder::with_loading_cache`. A deployment that wants the cache depends on the module crate and hands `builder()` to its adapter, through `run_with` on Fastly or `routes_with_registrations` on Axum, Cloudflare and Spin. Each of those runs a deployment's builders after the stock ones, and each adapter calls `loading_caches_from` with all of them, so no adapter names the module crate. -**After the fold.** Row 56 folds every module crate into one crate, `crates/trusted-server-integrations` (`2026-10-26-integrations-crate-design.md`). The module's code moves to `src/loading-cache/51degrees/` in that crate, and the crate compiles the folder's cache, the code that uses `fiftyone-caching`, only with its Cargo feature `loading-cache-51degrees`, which is also the only thing that makes `fiftyone-caching` a dependency. The folder's module name is a constant that every build compiles, so the crate's test of module names checks it whatever features are on. No stock adapter and not the `ts` tool turns the feature on, so a stock build still compiles no `fiftyone-caching`. A deployment that links the module turns the feature on in its own `Cargo.toml` and hands the module's builder to its adapter, as above. +**After the fold.** Row 56 folds every module crate into one crate, `crates/trusted-server-integrations` (`2026-10-27-integrations-crate-design.md`). The module's code moves to `src/loading-cache/51degrees/` in that crate, and the crate compiles the folder's cache, the code that uses `fiftyone-caching`, only with its Cargo feature `loading-cache-51degrees`, which is also the only thing that makes `fiftyone-caching` a dependency. The folder's module name is a constant that every build compiles, so the crate's test of module names checks it whatever features are on. No stock adapter and not the `ts` tool turns the feature on, so a stock build still compiles no `fiftyone-caching`. A deployment that links the module turns the feature on in its own `Cargo.toml` and hands the module's builder to its adapter, as above. **The two layers.** A call goes through two layers. The memory layer is one least recently used store for each cache, shared by every request an adapter serves with one application state, holding up to `memory_entries` values (64 for robots.txt) for `memory_seconds` each. Below the memory layer, a shared store layer is made for each call over the host's shared store, which is the host's value cache where the host gives one, being Fastly's cache, and the host's key-value store otherwise. A read looks in memory, then in the shared store, and only then loads. A copy in the shared store lasts at most `store_seconds`, leaves after `store_idle_seconds` unused, and ends sooner when the loaded value carries a lifetime of its own, which robots.txt sets to the module's refresh. The shared store holds each entry in the `fiftyone-caching` entry format, which carries the time the entry must be dropped, so a store that deletes late, or keeps no lifetimes, never gives back an entry past that time. A write to a key-value store carries a lifetime of at least 60 seconds because Workers KV refuses less, and the entry's own drop time holds the exact lifetime. @@ -229,10 +229,10 @@ These are for the reviewer. This revision records what the code does, and what t | 2026-10-16 | Points from review round 2. How the two published crates differ, from a comparison of the crates themselves, and the date on which the check counts were read. | | 2026-10-16 | Code fixes on row 24. A load is handed services with nothing of the reader in them, each application state has a cache of its own on Cloudflare as elsewhere, `CacheSpec::detachable` and the text about loads that run apart from their caller are gone, the comments and guides say what a table naming no module takes, the deploy check refuses a `module` nothing offers, and tests cover the stock default, a zero memory lifetime, a dropped first caller and a failing store. Open questions 1, 2, 4, 5 and 7, and 3 apart from the repeated changelog entry, move into the text. | | 2026-10-17 | Code fix on row 24. The Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests and a Worker or component without its store fails every request. Who waits for a load, the adapters, a module that fails, what changes for a deployment and the tests say so. | -| 2026-10-26 | No stock build links a loading cache module. In a stock build every feature with no cache selected loads through `NoLoadingCache`, the 51Degrees module is offered only to a deployment that links it, and after the fold of row 56 its `fiftyone-caching` dependency sits behind a Cargo feature that no stock adapter turns on. The header, the design, the adapters, the configuration, what changes for a deployment, what is not in this change, the checks, the tests, what no test covers and the open questions say so. The services a load is given are named as the code names them, `without_connection`, where this page said `without_reader`, the name an earlier commit on row 24 gave them. The pull requests are no longer drafts, and the header says so. | -| 2026-10-26 | Fixes from review round 3. The change takes in row 32, whose probe test counts the 51Degrees module as linked, and each adapter's order test, and puts a guard beside every stock list, the `ts` tool's included, with a test that pins the tool's list. What a stock build costs is spelled out, being a call to the cloud service on each page the client hints middleware is placed for, a key-value write on each robots.txt request where a deployment adds a robots.txt module, and the README promise of the 51Degrees shared code. The other text that says a stock build holds answers is listed with the change, with a changelog entry for the refused `module = "51degrees"`. Line numbers are those of the current heads, row 31 at `490a4665d` and row 24 at `e66b955a4`, and the design says what the code there does, being the host in each held answer's key, a module's failure logged with each address cut to its scheme and host on row 31, and the adapter robots.txt tests of row 24. The searches give their output from row 44, the open question on a repeated changelog entry, which no row repeats now, is gone, one on falling back to `NoLoadingCache` without a word is added, open question 1 names row 55 and core's `loading-cache.example`, the CI results are linked at their commits, and the counts of the local checks are gone. | -| 2026-10-26 | Fixes from review round 4. Row 32's first case links a second loading cache module, a test builder offering core's `MemoryLoadingCache`, so that the case still shows the table is read. The text the change rewrites adds the integration guide, the 51Degrees client's comments and the client hints middleware's tests. The 51Degrees shared code's loader is named as a caller that captures a request's own services, "stock build" is defined where it is first used, the source link names the commit that `4.5.8` was published from, the test of module names checks the constant whatever features are on, and the decision is placed after the Touchbase. | -| 2026-10-26 | The change to rows 24, 32 and 43 is described as built, so the line numbers paragraph no longer says this page is written ahead of its code, and its line numbers stay those of the code before the change. | +| 2026-10-27 | No stock build links a loading cache module. In a stock build every feature with no cache selected loads through `NoLoadingCache`, the 51Degrees module is offered only to a deployment that links it, and after the fold of row 56 its `fiftyone-caching` dependency sits behind a Cargo feature that no stock adapter turns on. The header, the design, the adapters, the configuration, what changes for a deployment, what is not in this change, the checks, the tests, what no test covers and the open questions say so. The services a load is given are named as the code names them, `without_connection`, where this page said `without_reader`, the name an earlier commit on row 24 gave them. The pull requests are no longer drafts, and the header says so. | +| 2026-10-27 | Fixes from review round 3. The change takes in row 32, whose probe test counts the 51Degrees module as linked, and each adapter's order test, and puts a guard beside every stock list, the `ts` tool's included, with a test that pins the tool's list. What a stock build costs is spelled out, being a call to the cloud service on each page the client hints middleware is placed for, a key-value write on each robots.txt request where a deployment adds a robots.txt module, and the README promise of the 51Degrees shared code. The other text that says a stock build holds answers is listed with the change, with a changelog entry for the refused `module = "51degrees"`. Line numbers are those of the current heads, row 31 at `490a4665d` and row 24 at `e66b955a4`, and the design says what the code there does, being the host in each held answer's key, a module's failure logged with each address cut to its scheme and host on row 31, and the adapter robots.txt tests of row 24. The searches give their output from row 44, the open question on a repeated changelog entry, which no row repeats now, is gone, one on falling back to `NoLoadingCache` without a word is added, open question 1 names row 55 and core's `loading-cache.example`, the CI results are linked at their commits, and the counts of the local checks are gone. | +| 2026-10-27 | Fixes from review round 4. Row 32's first case links a second loading cache module, a test builder offering core's `MemoryLoadingCache`, so that the case still shows the table is read. The text the change rewrites adds the integration guide, the 51Degrees client's comments and the client hints middleware's tests. The 51Degrees shared code's loader is named as a caller that captures a request's own services, "stock build" is defined where it is first used, the source link names the commit that `4.5.8` was published from, the test of module names checks the constant whatever features are on, and the decision is placed after the Touchbase. | +| 2026-10-27 | The change to rows 24, 32 and 43 is described as built, so the line numbers paragraph no longer says this page is written ahead of its code, and its line numbers stay those of the code before the change. | | 2026-10-27 | Row 31 pins the release `4.5.13` in place of `4.5.8`. The two published crates the rows pin, `4.5.8-49` and `4.5.13`, are compared file by file again, and the source link names the commit that `4.5.13` was published from. | ## Review log diff --git a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md index 7270765dfb..8a065c1fc0 100644 --- a/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md +++ b/docs/superpowers/specs/2026-10-14-51degrees-client-hints-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-12-shared-vendor-tables-design.md` (the `[shared.51degrees]` table), `2026-10-12-loading-cache-design.md` (the `51degrees-answers` cache), `2026-10-12-page-middleware-design.md` (the `[[serve]]` entries), `2026-10-14-source-modules-design.md` (the source a page is fetched from) and `2026-10-17-request-filters-on-every-adapter-design.md` (the request filters every adapter runs), all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** jwrosewell/trusted-server#39, IABTechLab/trusted-server#1386 **Issues:** IABTechLab/trusted-server#1333 -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** A browser sends its high entropy client hints only when the site asks for them and delegates them, and nothing in the stack writes the headers and the tag that do so for a deployment that uses the 51Degrees service, so every first page view of a session is priced on the User-Agent alone. @@ -321,7 +321,7 @@ Six things have no test. | 2026-10-15 | Fixes from review round 2. The pull request that holds the design documents, the wait for the first call in the guide, and the code comments left for a later change. | | 2026-10-15 | Code fixes for the call's scope, what it sends and header merging. The permission question stays open, with what was found, and the answer lifetime is a new open question. | | 2026-10-18 | Every adapter runs the filter, from `2026-10-17-request-filters-on-every-adapter-design.md` and its first review round. The table of parts, the builder, the `timeout_ms` setting, what bounds the call on each adapter, consent, failure, rollout, section 3 and the tests say so. Fixes from review round 3, the wait on Axum for a service that cannot be reached and test item 6. | -| 2026-10-26 | The service's answer is held only where a deployment links a loading cache module, and a stock build, which links none, calls the service on every page navigation an entry covers (`2026-10-12-loading-cache-design.md`, section 7). The table of parts and the declared cache say so. | +| 2026-10-27 | The service's answer is held only where a deployment links a loading cache module, and a stock build, which links none, calls the service on every page navigation an entry covers (`2026-10-12-loading-cache-design.md`, section 7). The table of parts and the declared cache say so. | ## Review log diff --git a/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md b/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md index c6a6c33a54..6fcdcaf1a4 100644 --- a/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md +++ b/docs/superpowers/specs/2026-10-14-no-modules-crate-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-14-source-modules-design.md`, in IABTechLab/trusted-server#1342, which replaced IABTechLab/trusted-server#1084 as the one place for every design document of the stack of pull requests listed in IABTechLab/trusted-server#777 **Related PRs:** IABTechLab/trusted-server#1384, jwrosewell/trusted-server#38 **Issues:** IABTechLab/trusted-server#1334 -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** One crate held the list of the modules a stock build ships on every adapter's behalf, with a table saying which middleware each supplies, so a deployment that wanted a different set edited a crate that was not its own, and a module's description of itself lived in a crate that was not the module's. The same change renames the three modules the project owns itself, and those renames are the only part that changes what an operator writes in settings. @@ -93,7 +93,7 @@ Three things have no test. The first is the five lists agreeing. Each adapter's | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The other users of the old crate, the test that kept its table in step, the renamed packages and the seam probe's names, the adapters and the list functions, what does not change, the startup refusal, the rollout of the renamed selectors, what `ts audit` reads and writes, the missing tests, `iabtl` and the stack defined, the renames in the title, and the pull requests raised. | | 2026-10-15 | Fixes from review round 2. The claimed test for the media types `ts audit` writes, the three changes left for a later change, and the changelog entries that still named the removed crate. | -| 2026-10-26 | The Axum adapter's tests in section 4 say that its linked builders include no loading cache module, so every feature loads through `NoLoadingCache` unless a deployment links one (`2026-10-12-loading-cache-design.md`), where they said a feature selects the stock loading cache by name. | +| 2026-10-27 | The Axum adapter's tests in section 4 say that its linked builders include no loading cache module, so every feature loads through `NoLoadingCache` unless a deployment links one (`2026-10-12-loading-cache-design.md`), where they said a feature selects the stock loading cache by name. | ## Review log diff --git a/docs/superpowers/specs/2026-10-15-asset-cache-design.md b/docs/superpowers/specs/2026-10-15-asset-cache-design.md index 184bb8b9b1..b07a2a771c 100644 --- a/docs/superpowers/specs/2026-10-15-asset-cache-design.md +++ b/docs/superpowers/specs/2026-10-15-asset-cache-design.md @@ -1,11 +1,11 @@ # Design Spec: The publisher's assets held by a module, behind one interface in core -**Status:** Proposed, 2026-10-15. Written after the code, for pull request [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375), row 34 of the stack. Rewritten 2026-10-26 for the asset cache as a module, which row 34 now holds. +**Status:** Proposed, 2026-10-15. Written after the code, for pull request [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375), row 34 of the stack. Rewritten 2026-10-27 for the asset cache as a module, which row 34 now holds. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-10-12-loading-cache-design.md`, `2026-10-15-module-cache-declarations-design.md`, `2026-10-14-source-modules-design.md`, `2026-10-14-no-modules-crate-design.md`, `2026-10-26-integrations-crate-design.md` (new, added to [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) with this rewrite), `2026-09-15-852-template-and-origin-caching-design.md`, `2026-04-28-multi-backend-asset-proxy-design.md`, `2026-05-19-asset-s3-auth-fastly-io-design.md`, `2026-09-08-1138-per-cookie-template-cache-policy-design.md` +**Related specs:** `2026-10-12-loading-cache-design.md`, `2026-10-15-module-cache-declarations-design.md`, `2026-10-14-source-modules-design.md`, `2026-10-14-no-modules-crate-design.md`, `2026-10-27-integrations-crate-design.md` (new, added to [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) with this rewrite), `2026-09-15-852-template-and-origin-caching-design.md`, `2026-04-28-multi-backend-asset-proxy-design.md`, `2026-05-19-asset-s3-auth-fastly-io-design.md`, `2026-09-08-1138-per-cookie-template-cache-policy-design.md` **Related PRs:** IABTechLab/trusted-server#1375, reviewed on [jwrosewell/trusted-server#29](https://github.com/jwrosewell/trusted-server/pull/29) (closed). [IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373) (row 32) lets a module declare the caches it loads through. [IABTechLab/trusted-server#894](https://github.com/IABTechLab/trusted-server/pull/894) is IAB Tech Lab's own draft design for response caching in core **Issues:** [IABTechLab/trusted-server#1324](https://github.com/IABTechLab/trusted-server/issues/1324) -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** A file that is not a page, whether the publisher's origin sends it or an asset route fetches it, is asked of its origin on every request and by every instance, where no platform cache holds it, so a burst of readers for one file is a burst of fetches. Holding those files is a choice one deployment makes, so the code that holds them belongs in a module, and core offers only the place where a module may answer. @@ -204,7 +204,7 @@ The module's load captures the services without the connection (`RuntimeServices **No stock build links the module.** No adapter's stock list and no list of the `ts` tool names the module, as none names the 51Degrees loading cache module (`2026-10-12-loading-cache-design.md`), and the module's caches would keep nothing in a stock build in any case, because a stock build links no loading cache module. A deployment that wants assets held links a loading cache module and this module, and hands both builders to its adapter, through `run_with` on Fastly or `routes_with_registrations` on Axum, Cloudflare and Spin. Any loading cache module serves, because the module loads through core's interface. The open questions ask whether the stock lists should offer the module. -After row 56 moves every module crate into one crate (`2026-10-26-integrations-crate-design.md`), the module's code is `src/asset-cache/51degrees/` in `crates/trusted-server-integrations`, behind the Cargo feature `asset-cache-51degrees`, which no stock adapter turns on, as the 51Degrees loading cache sits behind `loading-cache-51degrees`. +After row 56 moves every module crate into one crate (`2026-10-27-integrations-crate-design.md`), the module's code is `src/asset-cache/51degrees/` in `crates/trusted-server-integrations`, behind the Cargo feature `asset-cache-51degrees`, which no stock adapter turns on, as the 51Degrees loading cache sits behind `loading-cache-51degrees`. ### 2.4 Selecting the module @@ -446,11 +446,11 @@ Row 34's `assets_turned_off_in_the_settings_are_fetched_for_every_request` and ` | 2026-10-17 | The code fix on row 24, where the Cloudflare and Spin adapters declare their key-value stores, `TRUSTED_SERVER_KV` and `default`, so copies are kept there between requests, and where each application state has its own memory layer, Cloudflare included. How far one fetch reaches, the adapters' shared stores, how long the memory layer lives, and the reads and writes of the shared store say so. | | 2026-10-18 | A fix to the code. A load that fails asking the origin is the answer to every request that ran or waited on it, so the origin is asked once, with a test on each path. The design, failure handling, the cost of a slow origin and the tests say so. | | 2026-10-18 | The timeouts of an origin fetch, which core asks for on every adapter and only the Fastly adapter applies. | -| 2026-10-26 | Rewritten for the asset cache as a module. Core keeps one interface, `AssetCacheModule`, asked where row 34 called `asset_from_cache` on the publisher path and on an asset route, with an origin request the module sends itself, and the module `crates/asset-cache/51degrees` holds everything row 34 put in core. `[asset-cache]` selects it, it declares its caches through row 32, a deployment that selects no module has every file asked of its origin, no stock build links it, and it sits beside IABTechLab/trusted-server#894 as section 2.10 says. The gate also needs the record cache to keep values, `none` goes, each cache table selects its own loading cache module, and the open questions are new. | -| 2026-10-26 | Fixes from review round 3. `origin_response_may_be_kept` stays in core and becomes public beside `origin_shared_ttl`, both returning the new `KeepRefusal`, and row 46's change to it is named beside rows 39 and 54. Core offers only a `GET`, takes the module only from the builder `[asset-cache]` selects, and gains a recording test module for what it reports. `AssetFile`, `AssetVia` and `OriginRequest` are set out with their visibility and test constructors, `path` is `via`, and `fingerprint()` goes, with the key's digest of the build and the settings, until a rewritten body is kept. The caches are `51degrees-assets` and `51degrees-assets-vary`, the variant digest's tag is `ts-asset-variant-v1`, and after row 56 the module sits behind `asset-cache-51degrees`. The helpers the module copies, the crate's workspace and alias entries, what `asks_for_origin` describes, the fourth change to a row 34 document, the changelog, the CI record, #894's keys, its costs of Fastly's Core Cache and its cookie stripping, and smaller corrections. | -| 2026-10-26 | The account of the Touchbase of 2026-10-23 says what 51Degrees said there and what 51Degrees decided after it, and the integrations crate design is named by its file, `2026-10-26-integrations-crate-design.md`. | -| 2026-10-26 | Fixes from review round 4. The pull requests and the issue are linked where first named, and the CI record is linked at its commit. "Stock build", Viceroy and the seam probe are defined where first used. Fastly's Core Cache API is named as #894 names it, the cache names follow the form row 32 asks for, and on Spin the entries that a new key or a lifetime leaves behind stay in the store. The decisions of 51Degrees are placed after the Touchbase. | -| 2026-10-26 | The header says that row 34 now holds the module, where it said the rewrite was ahead of the code. | +| 2026-10-27 | Rewritten for the asset cache as a module. Core keeps one interface, `AssetCacheModule`, asked where row 34 called `asset_from_cache` on the publisher path and on an asset route, with an origin request the module sends itself, and the module `crates/asset-cache/51degrees` holds everything row 34 put in core. `[asset-cache]` selects it, it declares its caches through row 32, a deployment that selects no module has every file asked of its origin, no stock build links it, and it sits beside IABTechLab/trusted-server#894 as section 2.10 says. The gate also needs the record cache to keep values, `none` goes, each cache table selects its own loading cache module, and the open questions are new. | +| 2026-10-27 | Fixes from review round 3. `origin_response_may_be_kept` stays in core and becomes public beside `origin_shared_ttl`, both returning the new `KeepRefusal`, and row 46's change to it is named beside rows 39 and 54. Core offers only a `GET`, takes the module only from the builder `[asset-cache]` selects, and gains a recording test module for what it reports. `AssetFile`, `AssetVia` and `OriginRequest` are set out with their visibility and test constructors, `path` is `via`, and `fingerprint()` goes, with the key's digest of the build and the settings, until a rewritten body is kept. The caches are `51degrees-assets` and `51degrees-assets-vary`, the variant digest's tag is `ts-asset-variant-v1`, and after row 56 the module sits behind `asset-cache-51degrees`. The helpers the module copies, the crate's workspace and alias entries, what `asks_for_origin` describes, the fourth change to a row 34 document, the changelog, the CI record, #894's keys, its costs of Fastly's Core Cache and its cookie stripping, and smaller corrections. | +| 2026-10-27 | The account of the Touchbase of 2026-10-23 says what 51Degrees said there and what 51Degrees decided after it, and the integrations crate design is named by its file, `2026-10-27-integrations-crate-design.md`. | +| 2026-10-27 | Fixes from review round 4. The pull requests and the issue are linked where first named, and the CI record is linked at its commit. "Stock build", Viceroy and the seam probe are defined where first used. Fastly's Core Cache API is named as #894 names it, the cache names follow the form row 32 asks for, and on Spin the entries that a new key or a lifetime leaves behind stay in the store. The decisions of 51Degrees are placed after the Touchbase. | +| 2026-10-27 | The header says that row 34 now holds the module, where it said the rewrite was ahead of the code. | ## Review log diff --git a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md index dbe2c78c25..25d4e8e6b3 100644 --- a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md +++ b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-14-last-names-in-core-design.md`, `2026-10-14-backend-naming-design.md` **Related PRs:** IABTechLab/trusted-server#1399, jwrosewell/trusted-server#44 **Issues:** IABTechLab/trusted-server#777 (its Fastly row) -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** After the series before this one, core's shipped code still names one hosting platform, Fastly, in small pieces. They are the format of a long configuration document, a staging variable, three request headers, a device module selector and the names of two functions. Each is a place where a change by one platform is a change in core. Core's comments, tests and README files name Fastly, Cloudflare, Spin, Fermyon, Viceroy and Axum as well, and three tests use a host under Fastly's `edgecompute.app` domain. After this change no line of core names a hosting platform, and a guard outside core keeps it so (section 2.6). One Fastly header, `x-compress-hint`, whose name does not name the platform, is still set by core after this change (section 3). @@ -178,8 +178,8 @@ Not tested: | 2026-10-15 | Fixes from review round 1. The readers of each declared list and the open questions on them, `x-debug-fastly-pop`, the scheme change, the effects on the other adapters, where each refusal happens, the rollout, the narrowed constants, `x-compress-hint`, the file count, the untested paths and the review log. | | 2026-10-15 | Corrections from review round 2. The Fastly sentence of 2.3 and the changelog on the `x-debug-fastly-pop` refusal, the changelog on the spoofable strip and the scheme change, when the Axum, Cloudflare and Spin adapters strip `fastly-client-ip`, and the comment in `request_headers.rs` left for a later change. | | 2026-10-16 | Core names no hosting platform or platform domain in any line. Section 2.6 with the counts, the `_vercel` prefix and the test host, the guard and the tests that moved to the Fastly adapter, and section 3 no longer leaves core's comments and tests for later. | -| 2026-10-26 | Row 12 no longer adds `ORIGIN_ASSET_PREFIXES`, so core holds no `_vercel` entry before this change. Section 2.6 names the commit it counts at, `13d57b03f`, where the guard finds 173 lines, that entry and 172 comments, documents or test code, and no other section names the prefix. | -| 2026-10-26 | The header no longer calls the pull request a draft. | +| 2026-10-27 | Row 12 no longer adds `ORIGIN_ASSET_PREFIXES`, so core holds no `_vercel` entry before this change. Section 2.6 names the commit it counts at, `13d57b03f`, where the guard finds 173 lines, that entry and 172 comments, documents or test code, and no other section names the prefix. | +| 2026-10-27 | The header no longer calls the pull request a draft. | ## Review log diff --git a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md index 5d04a5bf28..48f43b885f 100644 --- a/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md +++ b/docs/superpowers/specs/2026-10-15-module-cache-declarations-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-12-loading-cache-design.md`, `2026-10-12-robots-txt-from-modules-design.md`, `2026-08-27-integration-provider-seam-design.md` **Related PRs:** IABTechLab/trusted-server#1373, reviewed on jwrosewell/trusted-server#27 (closed) **Issues:** IABTechLab/trusted-server#1320 -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** A module that holds its own answers asks core for the cache of the name it chose, finds no `[cache.]` table for that name, and gets the cache that holds nothing, so every request loads the answer again. @@ -246,8 +246,8 @@ These joins have no test. | 2026-10-15 | Corrections from review round 2. What a module that borrows another module's cache name shares, the section the guide's push-order warning covers, the builders the deploy check is given, the round 1 log sentence on the code changes, and the follow-ups for the configuration guide, the integration guide, the rustdoc and the visibility of two functions. | | 2026-10-17 | Who can read a copy names the shared store on each adapter, being Fastly's cache, the key-value store the Cloudflare and Spin adapters declare on row 24, `TRUSTED_SERVER_KV` and `default`, and none on Axum. | | 2026-10-18 | Items 5 to 8 leave 2.10. The configuration guide says core applies no lifetime of a declared table and when to push one, the integration guide lists `.with_cache_features(...)`, and `declared_caches` and `check_cache_tables` are `pub(crate)`. The rustdoc of `loading_caches_from` and `LoadingCacheConfig::module`, and the integration guide on a table naming no module, already gave rule 2 in 2.2. Section 3 and 2.8 match the guides. 2.7 and section 3 say the deploy check also refuses a `module` that offers no loading cache. | -| 2026-10-26 | 2.2 says that a build linking one loading cache module holds a declared cache's answers without a line in the document, where it said a stock build does, because no stock build links one once the 51Degrees module leaves the stock lists (`2026-10-12-loading-cache-design.md`). | -| 2026-10-26 | Section 3, the Axum test in section 4 and the fifth join with no test follow the stock lists linking no loading cache module (`2026-10-12-loading-cache-design.md`). The `ts` command's list carries none, so `ts config validate` refuses a table naming any loading cache module unless a tool registers it, the Axum test's first two cases link core's `MemoryLoadingCache` beside the probe's module and a fourth case links none, and no test reaches the 51Degrees module for a declared cache. | +| 2026-10-27 | 2.2 says that a build linking one loading cache module holds a declared cache's answers without a line in the document, where it said a stock build does, because no stock build links one once the 51Degrees module leaves the stock lists (`2026-10-12-loading-cache-design.md`). | +| 2026-10-27 | Section 3, the Axum test in section 4 and the fifth join with no test follow the stock lists linking no loading cache module (`2026-10-12-loading-cache-design.md`). The `ts` command's list carries none, so `ts config validate` refuses a table naming any loading cache module unless a tool registers it, the Axum test's first two cases link core's `MemoryLoadingCache` beside the probe's module and a fourth case links none, and no test reaches the 51Degrees module for a declared cache. | ## Review log diff --git a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md index 75bf1e8c5b..1592ed04cd 100644 --- a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-17-request-filters-on-every-adapter-design.md`, `2026-10-17-edge-cookie-on-every-adapter-design.md`, `2026-10-14-source-modules-design.md`, `2026-10-14-origin-signing-design.md`, `2026-10-15-asset-cache-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 **Related PRs:** [IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412). The request filters run on every adapter in [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410) and the Edge Cookie in [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411) **Issues:** none -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** Only the Fastly adapter serves `[[proxy.asset_routes]]`. On Axum, Cloudflare and Spin a request under a route's prefix goes to the publisher's origin as an ordinary page request, so a route's own source, its signing and its held answers are never used there. This change dispatches asset routes on those three adapters as the Fastly adapter does. Two parts of it reach beyond asset routes. On Spin a source whose `host_header` names another host is now refused, the publisher's source as well as a route's, so a Spin deployment whose publisher source sets one answers every page with `502`, which is a breaking change. And the Cloudflare and Spin adapters now install a logger, because neither had one and every line they logged was dropped, the warnings this change adds included. @@ -114,7 +114,7 @@ The tests above pass on each of the three adapters, and the existing tests of ea | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-17 | First draft. | | 2026-10-18 | Fixes from review round 1. The pull request this spec belongs to, Spin's refusal of a source's `host_header` and what Cloudflare does with one, the startup warning for a route that enables the Image Optimizer and why it is not a refusal, the `HEAD` a signed route sends and the SVG and debug requests that pass, the Axum client's refusal on `send_async`, the startup refusal of a source that cannot be built, the per-request failures, body limits and timeouts, where answers are held, the Edge Cookie an asset response neither writes nor expires, the tests of the cache rule, the error and the Edge Cookie, the open questions and the review log. Fixes from review round 2. The loggers the Cloudflare and Spin adapters now install and where their lines appear, how often each adapter logs the Image Optimizer warning, why such a route gets a warning where those adapters refuse an auction of more than one provider, the Spin `host_header` refusal as a breaking change in the summary, the problem and the rollout, the logged failures, the reader headers a route's origin receives, the rise in key-value traffic, the signed source whose `host_header` names the address's own host in other letter case, what the warning test reads, which Spin client code the native tests reach, the logger tests and what is not in this change. | -| 2026-10-26 | Held answers follow the asset cache as a module (`2026-10-15-asset-cache-design.md`). `handle_asset_proxy_request` offers a `GET` to the module `[asset-cache]` selects and keeps no copy itself, the 51Degrees module holds its copies through `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]`, and no stock build links that module or a loading cache module. | +| 2026-10-27 | Held answers follow the asset cache as a module (`2026-10-15-asset-cache-design.md`). `handle_asset_proxy_request` offers a `GET` to the module `[asset-cache]` selects and keeps no copy itself, the 51Degrees module holds its copies through `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]`, and no stock build links that module or a loading cache module. | ## Review log diff --git a/docs/superpowers/specs/2026-10-17-config-migrate-design.md b/docs/superpowers/specs/2026-10-17-config-migrate-design.md index 1bc11b006c..6e832125e6 100644 --- a/docs/superpowers/specs/2026-10-17-config-migrate-design.md +++ b/docs/superpowers/specs/2026-10-17-config-migrate-design.md @@ -1,11 +1,11 @@ # Design Spec: `ts config migrate` rewrites a settings document written for main into the shape the stack reads -**Status:** Proposed, 2026-10-17, revised 2026-10-18 after review rounds 1 and 2, and revised 2026-10-26 for the permission policy, which is selected when a service is deployed. Raised as IABTechLab/trusted-server#1409, one of the stack's pull requests, row 51 of the stack in IABTechLab/trusted-server#777, opened on 2026-10-17, for Tech Lab review. References to the stack's code are to the code of IABTechLab/trusted-server#1409, by file and function, with a line number where one helps. +**Status:** Proposed, 2026-10-17, revised 2026-10-18 after review rounds 1 and 2, and revised 2026-10-27 for the permission policy, which is selected when a service is deployed. Raised as IABTechLab/trusted-server#1409, one of the stack's pull requests, row 51 of the stack in IABTechLab/trusted-server#777, opened on 2026-10-17, for Tech Lab review. References to the stack's code are to the code of IABTechLab/trusted-server#1409, by file and function, with a line number where one helps. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-14-source-modules-design.md`, `2026-10-13-middleware-settings-design.md`, `2026-10-14-rsc-flight-middleware-design.md`, `2026-10-15-geo-module-request-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 **Related PRs:** IABTechLab/trusted-server#1409 **Issues:** none -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** The changelog of the stack lists 41 breaking entries under Unreleased, where main lists 10, and a settings document written for Tech Lab main does not load under the stack. Every operator would rewrite their document by hand from the changelog. `ts config migrate` makes the rewrite automatic where the document holds everything the rewrite needs, and says what is left where it does not. @@ -202,7 +202,7 @@ No test starts the Axum or Spin adapter with a migrated document, because the to | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-17 | First draft. | | 2026-10-18 | Fixes from review round 1. `--adapter`, with the location lookup each adapter accepts and the Edge Cookie and geo settings startup refuses left to do. No location header is selected, a handler is removed only where every path its pattern matches is closed, a host header is reported to check, and page entries are added to a document written for main alone. The flags, the validation's reach, the consent lists, the adapters, consent and privacy, failure, compatibility and rollout, the tests, and 39 breaking entries. Fixes from review round 2. A handler pattern with a quantifier straight after its prefix or bound, or a character class or comment after them, is left to do, and a check line names the region the Cloudflare lookup now reads. The checks made only at startup, the location main's consent saw, a document run on two adapters, the stored assets, the passwords, the tests and the status line. | -| 2026-10-26 | The permission policy is a file the settings name and the push supplies. A document that names none is an item left to do, whose line names the file's stem and is the same on every adapter, a document written for main may name its policy before or after it is migrated, the command never chooses a policy or writes a policy file, and its validation leaves the policy check out and follows the pattern rule for `permissions.sha256` and `permissions.text`. The consent lists are carried into the publisher's file, as the command's line says. The count of breaking entries is 41, and the Cloudflare config store has a row. The status names IABTechLab/trusted-server#1409, whose code the references cite, the EdgeZero version is `v0.0.10`, the settings lines are brought up to date, the staged push is described as `v0.0.10` does it, a host's blob is staged with `--key`, and §6 gains the step for Cloudflare's config namespace. Step 7 of §6 says only the stored pages are fetched again, because core holds no copy of a file once the asset cache is a module. | +| 2026-10-27 | The permission policy is a file the settings name and the push supplies. A document that names none is an item left to do, whose line names the file's stem and is the same on every adapter, a document written for main may name its policy before or after it is migrated, the command never chooses a policy or writes a policy file, and its validation leaves the policy check out and follows the pattern rule for `permissions.sha256` and `permissions.text`. The consent lists are carried into the publisher's file, as the command's line says. The count of breaking entries is 41, and the Cloudflare config store has a row. The status names IABTechLab/trusted-server#1409, whose code the references cite, the EdgeZero version is `v0.0.10`, the settings lines are brought up to date, the staged push is described as `v0.0.10` does it, a host's blob is staged with `--key`, and §6 gains the step for Cloudflare's config namespace. Step 7 of §6 says only the stored pages are fetched again, because core holds no copy of a file once the asset cache is a module. | ## Review log diff --git a/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md index 7973cfac6d..3bd9f396e5 100644 --- a/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-request-filters-on-every-adapter-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-14-51degrees-client-hints-design.md` and `2026-10-15-geo-module-request-design.md`, whose statements about the adapters this change amends, `2026-06-11-datadome-server-side-protection-design.md` and `2026-08-03-datadome-ip-excluded-client-tag-design.md`, which describe DataDome's protection filter on the Fastly adapter alone, a scope this change lifts (section 1), and the design documents of the stack in IABTechLab/trusted-server#1342 **Related PRs:** [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410) **Issues:** none -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** Only the Fastly adapter runs the integration request filters and classifies each request with the device module `[device]` selects. On Axum, Cloudflare and Spin a module's request filter never runs, so the 51Degrees client hints middleware writes nothing and DataDome's protection never asks its service, and a device module is asked only for a bid request's device attributes, so no module call carries a device classification. This change runs both steps on those three adapters where the Fastly adapter runs them. @@ -122,8 +122,8 @@ The thirteen tests pass on each of the three adapters, taking out any of the thr | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2026-10-17 | First draft. | | 2026-10-18 | Fixes from review round 1. What a selected device module did on these adapters before, what DataDome's filter sends and sets, how each filter's call is bounded and why no time limit is added, the inspection endpoints the Fastly adapter answers before the preparers, the routes kept out of the cycle, which of a crawler's cookies robots.txt reads, the `Cookie` check and when it is reached, the configuration, eight new tests on each adapter with the three privacy guards among them, the open questions and the review log. Fixes from review round 2. A test on each adapter that sends every named route in the cycle through the filters, the discovery route in the context test, the tests each removal fails, the IP address and the request's address a device module is shown, the subresources it classifies, what the diagnostics preparer drops, the two DataDome designs whose scope this change lifts, what the guards do with a platform's own cache header with an open question on it, and the reason no shipped filter is tested through an adapter. | -| 2026-10-26 | Row 12 adds no route to the adapters any more, so section 2 drops row 12's pages from the routes kept out of the cycle, from the requests a device module does not classify and from the inspection test, which is renamed `the_inspection_endpoints_skip_the_request_filters`. | -| 2026-10-26 | The header names the pull request, IABTechLab/trusted-server#1410, row 52 of the stack, in place of saying that none is raised. | +| 2026-10-27 | Row 12 adds no route to the adapters any more, so section 2 drops row 12's pages from the routes kept out of the cycle, from the requests a device module does not classify and from the inspection test, which is renamed `the_inspection_endpoints_skip_the_request_filters`. | +| 2026-10-27 | The header names the pull request, IABTechLab/trusted-server#1410, row 52 of the stack, in place of saying that none is raised. | ## Review log diff --git a/docs/superpowers/specs/2026-10-26-developer-experience-design.md b/docs/superpowers/specs/2026-10-27-developer-experience-design.md similarity index 98% rename from docs/superpowers/specs/2026-10-26-developer-experience-design.md rename to docs/superpowers/specs/2026-10-27-developer-experience-design.md index 56e4ce1a2e..429479b6f3 100644 --- a/docs/superpowers/specs/2026-10-26-developer-experience-design.md +++ b/docs/superpowers/specs/2026-10-27-developer-experience-design.md @@ -1,11 +1,11 @@ # Design Spec: The developer experience, step by step, for a module author, an operator and the release manager -**Status:** Proposed, 2026-10-26. A description of the developer experience of the stack for Tech Lab review, which adds no code of its own. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-27. A description of the developer experience of the stack for Tech Lab review, which adds no code of its own. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review -**Related specs:** `2026-07-30-permission-model-design.md` (row 4), `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-inspection-endpoints-design.md` (row 9), `2026-10-12-config-by-host-design.md` and `2026-10-15-fastly-per-host-selector-design.md` (row 11), `2026-10-26-module-endpoints-design.md` (row 12), `2026-10-12-loading-cache-design.md` (row 24), `2026-10-15-module-cache-declarations-design.md` (row 32), `2026-10-15-asset-cache-design.md` (row 34), `2026-10-14-no-modules-crate-design.md` (row 43), `2026-10-17-config-migrate-design.md` (row 51), `2026-10-19-module-scripts-in-crates-design.md` (row 55) and `2026-10-26-integrations-crate-design.md` (row 56), in IABTechLab/trusted-server#1342. Tech Lab's [`2026-06-16-edgezero-based-ts-cli-design.md`](https://github.com/IABTechLab/trusted-server/blob/7aa5c993ba4311fd3b01872f2d2c0ca6e0f532cd/docs/superpowers/specs/2026-06-16-edgezero-based-ts-cli-design.md) on main, its [release attestation design](https://github.com/IABTechLab/trusted-server/blob/a501d1ba6bde2db82546d7e16c93facab8c613e6/docs/superpowers/specs/2026-10-09-1235-release-attestation-design.md) in IABTechLab/trusted-server#1435, and its [split integrations design](https://github.com/IABTechLab/trusted-server/blob/cd0f59823dfaecdc28eed521bef149a5d1064e26/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md) in IABTechLab/trusted-server#1194 +**Related specs:** `2026-07-30-permission-model-design.md` (row 4), `2026-08-27-integration-provider-seam-design.md`, `2026-10-12-inspection-endpoints-design.md` (row 9), `2026-10-12-config-by-host-design.md` and `2026-10-15-fastly-per-host-selector-design.md` (row 11), `2026-10-27-module-endpoints-design.md` (row 12), `2026-10-12-loading-cache-design.md` (row 24), `2026-10-15-module-cache-declarations-design.md` (row 32), `2026-10-15-asset-cache-design.md` (row 34), `2026-10-14-no-modules-crate-design.md` (row 43), `2026-10-17-config-migrate-design.md` (row 51), `2026-10-19-module-scripts-in-crates-design.md` (row 55) and `2026-10-27-integrations-crate-design.md` (row 56), in IABTechLab/trusted-server#1342. Tech Lab's [`2026-06-16-edgezero-based-ts-cli-design.md`](https://github.com/IABTechLab/trusted-server/blob/7aa5c993ba4311fd3b01872f2d2c0ca6e0f532cd/docs/superpowers/specs/2026-06-16-edgezero-based-ts-cli-design.md) on main, its [release attestation design](https://github.com/IABTechLab/trusted-server/blob/a501d1ba6bde2db82546d7e16c93facab8c613e6/docs/superpowers/specs/2026-10-09-1235-release-attestation-design.md) in IABTechLab/trusted-server#1435, and its [split integrations design](https://github.com/IABTechLab/trusted-server/blob/cd0f59823dfaecdc28eed521bef149a5d1064e26/docs/superpowers/specs/2026-09-17-split-integrations-crates-design.md) in IABTechLab/trusted-server#1194 **Related PRs:** the rows the journeys rest on, being [IABTechLab/trusted-server#1345](https://github.com/IABTechLab/trusted-server/pull/1345) (row 4), [IABTechLab/trusted-server#1350](https://github.com/IABTechLab/trusted-server/pull/1350) (row 9), [IABTechLab/trusted-server#1352](https://github.com/IABTechLab/trusted-server/pull/1352) (row 11), [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353) (row 12), [IABTechLab/trusted-server#1365](https://github.com/IABTechLab/trusted-server/pull/1365) (row 24), [IABTechLab/trusted-server#1373](https://github.com/IABTechLab/trusted-server/pull/1373) (row 32), [IABTechLab/trusted-server#1375](https://github.com/IABTechLab/trusted-server/pull/1375) (row 34), [IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384) (row 43), [IABTechLab/trusted-server#1409](https://github.com/IABTechLab/trusted-server/pull/1409) (row 51), [IABTechLab/trusted-server#1423](https://github.com/IABTechLab/trusted-server/pull/1423) (row 55) and row 56, which this update adds. Tech Lab's [IABTechLab/trusted-server#1262](https://github.com/IABTechLab/trusted-server/pull/1262) and [IABTechLab/trusted-server#1435](https://github.com/IABTechLab/trusted-server/pull/1435), the signed release build, and [IABTechLab/trusted-server#1194](https://github.com/IABTechLab/trusted-server/pull/1194), Tech Lab's draft design for one integrations crate **Issues:** [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), which lists the stack's pull requests and numbers its rows, [IABTechLab/trusted-server#1055](https://github.com/IABTechLab/trusted-server/issues/1055), Tech Lab's developer experience tracker, which covers the CLI's audits, its human-readable output, its guidance on adapters and tools ([IABTechLab/trusted-server#1060](https://github.com/IABTechLab/trusted-server/issues/1060) to IABTechLab/trusted-server#1062) and the development proxy, [IABTechLab/trusted-server#1062](https://github.com/IABTechLab/trusted-server/issues/1062), which asks for clearer errors when a provider's CLI is missing and for the prerequisites to be written down, and [IABTechLab/trusted-server#1235](https://github.com/IABTechLab/trusted-server/issues/1235), the attested release binaries that the third journey follows -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** At the Trusted Server Touchbase meeting of 23 October 2026, IAB Tech Lab asked to review the developer experience of the stack once the permission designs are reviewed. The guides each cover part of it, and Tech Lab's tracker IABTechLab/trusted-server#1055 covers the CLI's audits, its human-readable output, its guidance on adapters and tools and the development proxy, but nothing says what a module author, an operator or the release manager does from the first step to the last. This design follows each of the three through every step, with the command or file the step takes, what success looks like and how each likely failure reports itself. It also says which figures a fresh run still has to measure, and lists the questions that only Tech Lab can answer. @@ -32,19 +32,19 @@ The steps were read from the code and the guides of row 55 at `5be114773`, the h ## 2. Journey 1. A module author adds a module -The author is a vendor, or a contributor working for one, adding a module to Tech Lab's tree. The integration guide's worked example is the seam probe, `crates/testing/iabtl` at row 55, a fixture crate through which tests drive core's extension points from outside core. The Osano module, `crates/cmp/osano`, whose Rust side only enables its browser script, shows the shape of a module that ships. The steps add a module named `cmp.example`, the name the integration guide's own examples use. They describe row 56, where every module is a folder of one crate, as `2026-10-26-integrations-crate-design.md` defines it, and step 4 says what a crate of its own costs at row 55. +The author is a vendor, or a contributor working for one, adding a module to Tech Lab's tree. The integration guide's worked example is the seam probe, `crates/testing/iabtl` at row 55, a fixture crate through which tests drive core's extension points from outside core. The Osano module, `crates/cmp/osano`, whose Rust side only enables its browser script, shows the shape of a module that ships. The steps add a module named `cmp.example`, the name the integration guide's own examples use. They describe row 56, where every module is a folder of one crate, as `2026-10-27-integrations-crate-design.md` defines it, and step 4 says what a crate of its own costs at row 55. 1. **Open an issue, then write the design.** Tech Lab's [`CONTRIBUTING.md`](https://github.com/IABTechLab/trusted-server/blob/7aa5c993ba4311fd3b01872f2d2c0ca6e0f532cd/CONTRIBUTING.md) on main, as the fix of the contributor documents (IABTechLab/trusted-server#1387) left it, asks for an issue before any feature work, one that describes the problem, the intended behavior, the adapters affected and the constraints on compatibility, opened before an implementation is started. A new integration is a feature, so it then needs a design in `docs/superpowers/specs/YYYY-MM-DD--design.md` that ends in a `## Review log` of at least two review rounds and is linked from the pull request. Success is an issue and a design the maintainers accept. A maintainer moves a feature pull request without a linked design back to draft, and its code is not read until the design is linked. 2. **Make the module's folder and name it.** At row 56 the module is the folder `crates/trusted-server-integrations/src/cmp/example/`, whose `mod.rs` holds what a crate's `src/lib.rs` held. The type folder's `mod.rs` declares the folder with `#[path]`, as `#[path = "example/mod.rs"] pub mod example;`, where the Rust name is the folder's name with `-` written as `_`, so `us-privacy` is `us_privacy`, and the one folder name that starts with a digit, `51degrees`, is `fiftyone_degrees`. A new type folder is declared the same way in the crate's `lib.rs`. The module's name is a constant in `mod.rs`, `pub const MODULE: &str = "cmp.example";`, which a deployment selects with `[cmp] module = "example"` and configures in `[cmp.example]`, and each part of a name is lower case letters, digits, `_` or `-`. The builder's source label is `trusted-server-integrations/cmp/example`, so the registry's messages name the folder. The test `every_folder_names_its_modules_after_itself` compares the folders on disk with a list in the test of each folder and the names it supplies, so the new folder joins that list. The test fails, naming the folder, for a folder the list leaves out, a listed folder that is gone, a name that does not follow its folder, a wrong source label, or any call of `trusted_server_core::module_name!()` in the crate, which row 56 refuses there, because the macro reads Cargo's manifest folder, and after the fold that is the one crate's folder for every module. 3. **Say who maintains it.** The folder's `maintainers.toml` names an `owner` and a `status`, one of `vendor owned`, `seeking vendor owner` and `project owned`, as `owner = "Trusted Server maintainers"` and `status = "seeking vendor owner"` do for a module no vendor has adopted yet. The test `every_folder_declares_its_maintainers` fails, naming the folder, for a folder without the file, an empty `owner` or any other `status`. The file and the test replace row 55's `[package.metadata.maintainers]` table and its test `every_crate_outside_core_declares_its_maintainers`. -4. **Add its dependencies.** A dependency the module needs goes once into `crates/trusted-server-integrations/Cargo.toml`, as an ordinary dependency. A folder gets a Cargo feature, named after the folder as `-`, only for a dependency that not every build can or should carry, or for a module that no stock build links (section 2.8 of `2026-10-26-integrations-crate-design.md`). Five folders have one. The Fastly SDK sits behind `device-fastly` and `geo-fastly`, which the Fastly adapter turns on. `fiftyone-caching` sits behind `loading-cache-51degrees` and the asset cache module behind `asset-cache-51degrees`. No stock build turns either on. A deployment that links the 51Degrees loading cache or the asset cache module turns that module's feature on, and so do the aliases of step 13. The seam probe sits behind `testing-iabtl`, which only the Axum and Fastly adapters' dev-dependencies turn on, so no adapter build compiles the probe. A feature gates the code in its folder and never the folder's name constants or its source label, so the tests of steps 2 and 3 check every folder whatever features are on. Success is the aliases of step 13 building the crate. At row 55 a module is a crate of its own, which joins `members` and `[workspace.dependencies]` in the root `Cargo.toml`, the manifests of the four adapters and of `crates/trusted-server-cli`, and the six aliases in `.cargo/config.toml` that name module crates. Nothing compares those aliases with the workspace, so an alias that leaves a crate out does not lint or test it. That is why the seam probe, which is in no alias at row 55, is tested by a step of its own in `.github/workflows/test.yml`, "Run host-target seam probe integration tests". Row 56 replaces all of those entries with the one crate, the aliases name `-p trusted-server-integrations`, and the native aliases turn on `testing-iabtl`, so that step goes. +4. **Add its dependencies.** A dependency the module needs goes once into `crates/trusted-server-integrations/Cargo.toml`, as an ordinary dependency. A folder gets a Cargo feature, named after the folder as `-`, only for a dependency that not every build can or should carry, or for a module that no stock build links (section 2.8 of `2026-10-27-integrations-crate-design.md`). Five folders have one. The Fastly SDK sits behind `device-fastly` and `geo-fastly`, which the Fastly adapter turns on. `fiftyone-caching` sits behind `loading-cache-51degrees` and the asset cache module behind `asset-cache-51degrees`. No stock build turns either on. A deployment that links the 51Degrees loading cache or the asset cache module turns that module's feature on, and so do the aliases of step 13. The seam probe sits behind `testing-iabtl`, which only the Axum and Fastly adapters' dev-dependencies turn on, so no adapter build compiles the probe. A feature gates the code in its folder and never the folder's name constants or its source label, so the tests of steps 2 and 3 check every folder whatever features are on. Success is the aliases of step 13 building the crate. At row 55 a module is a crate of its own, which joins `members` and `[workspace.dependencies]` in the root `Cargo.toml`, the manifests of the four adapters and of `crates/trusted-server-cli`, and the six aliases in `.cargo/config.toml` that name module crates. Nothing compares those aliases with the workspace, so an alias that leaves a crate out does not lint or test it. That is why the seam probe, which is in no alias at row 55, is tested by a step of its own in `.github/workflows/test.yml`, "Run host-target seam probe integration tests". Row 56 replaces all of those entries with the one crate, the aliases name `-p trusted-server-integrations`, and the native aliases turn on `testing-iabtl`, so that step goes. 5. **Write the builder and its settings.** `mod.rs` gives `builder()`, which returns `IntegrationBuilder::new(ID, "trusted-server-integrations/cmp/example", register, validate).with_module_name(MODULE)`. The settings type takes `#[serde(deny_unknown_fields)]` and implements `IntegrationConfig`. The `validate` function reads `settings.module_config::(MODULE)` and runs for every builder, selected or not, and the `register` function answers `Ok(None)` when no section selects the module. Both should return a refusal as `TrustedServerError::Configuration`, as `settings.module_config` does, because each adapter's startup error router answers with the status of the error's kind, `500` for a configuration or settings error and `503` for a key-value store error, so a refusal of another kind, such as an integration error, would make the service answer `502`. The other builder calls are listed under "What the crate provides" in the [integration guide](https://github.com/IABTechLab/trusted-server/blob/5be114773345a03e207c1da2d3b163a23510985f/docs/guide/integration-guide.md). Success is the module's tests passing (step 13). The refusals a module's settings meet are these, as row 55 words them, each a configuration error that names the table. - A setting the type does not know, "[cmp.example] could not be read: …", which names the setting and never its value. - A table its section does not select, "[cmp.example] is configured, but [cmp] module does not select `example`. Add it to module, or remove the table". - A section that selects nothing, "[cmp] selects no module. A module type's section names what runs with `module` or `modules`, so select one or remove the section". - At startup, a name that no linked builder supplies, "[cmp] selects `example`, which no module this deployment supplies. The cmp modules it supplies are […]". - At startup, two builders with one id, "integration id `example` is registered twice, by `` and by ``". -6. **Serve its routes, if it has any.** The registration carries an `IntegrationProxy` through `.with_proxy(...)`, and the proxy's `routes()` lists its endpoints. The proxy's own `self.get("/country")` puts the proxy prefix in front, `/integrations/`, the integration id, so the route is `/integrations/example/country`, and `IntegrationEndpoint::get(path)` keeps a path of the module's own choosing. Row 12 adds tests on all four adapters that a module outside core serves its own endpoint this way, and the integration guide's section "Serving an endpoint from a module". `2026-10-26-module-endpoints-design.md` sets out what such a route meets. +6. **Serve its routes, if it has any.** The registration carries an `IntegrationProxy` through `.with_proxy(...)`, and the proxy's `routes()` lists its endpoints. The proxy's own `self.get("/country")` puts the proxy prefix in front, `/integrations/`, the integration id, so the route is `/integrations/example/country`, and `IntegrationEndpoint::get(path)` keeps a path of the module's own choosing. Row 12 adds tests on all four adapters that a module outside core serves its own endpoint this way, and the integration guide's section "Serving an endpoint from a module". `2026-10-27-module-endpoints-design.md` sets out what such a route meets. - A `GET` route answers neither `HEAD` nor `OPTIONS`, so a module that answers either declares a route of its own for that method, as Sourcepoint declares `HEAD` and `OPTIONS` beside its `GET` and `POST`. A `HEAD` route sends its empty body itself, because nothing in core, the adapters or EdgeZero strips a body from a `HEAD` answer. - A module route never sees a request that a request preparer refuses, whose Edge Cookie step fails or that a request filter answers. A request preparer is a function that a module hands its builder with `.with_request_preparer(...)`, and the function runs once for each request before routing and may change the request or refuse it. - Nor does a module route see six kinds of request that something ahead of the route answers, being a route of the adapter's own for the same method and path, such as `GET /_ts/config`, a request the Fastly entry point answers before its router, such as `GET /health`, a closed path beneath `/_ts/admin` or `/admin/keys`, a `GET` or `HEAD` of `/robots.txt` when `[robots-txt]` is present, a `GET` beneath `/static/tsjs=`, and, from row 35, the path a reader sent where the deployment's request addressing rewrites the path before routing. The registry does not know the adapter's routes, so the registry accepts a route on such a path, and nothing reports that the route never runs. @@ -58,7 +58,7 @@ The author is a vendor, or a contributor working for one, adding a module to Tec - The Vitest test "lists every folder of module sources for @tsjs/module, in discovery order" fails when `tsconfig.json` leaves the folder out. - The build stops with `tsjs: failed to discover modules: module is in both and `, or with `module name core, used by , is the shared core's bundle`. - A script the module carries itself, through `.with_js_module(CarriedJsModule { source, sha256 })`, is refused at startup when its stated digest is wrong, with "Integration `` carries a browser module whose declared SHA-256 does not match its source (declared …, actual …)", most often because a checkout rewrote the script's line endings. -11. **Link it into the stock build.** At row 56 the builder is `trusted_server_integrations::cmp::example::builder()`, and it joins `shipped_module_builders()` in `src/app.rs` of each of the four adapters and `linked_builders()` in `crates/trusted-server-cli/src/app_config.rs`, in the place its request hooks should run. A permission signal module joins `shipped_signal_modules()` in each adapter instead, and the CLI links no permission signal module. At row 55 the five lists of builders hold the same builders in the same order, and no test compares one list with another, as `2026-10-14-no-modules-crate-design.md` records. A module that is not on the lists is not in the gold WASM, because the stock Fastly binary calls `trusted_server_adapter_fastly::run()`, which links the stock list and nothing else. A module kept in a repository of its own runs only in a deployment that builds its own binary, handing the module's builder to `run_with` on Fastly or to `routes_with_registrations` on Axum, Cloudflare and Spin, and that binary is not the gold WASM. An adapter whose list misses the module refuses it at startup once a section selects it, with "[cmp] selects `example`, which no module this deployment supplies. …", and when the CLI's list misses it, `ts config validate` and `ts config push` skip the module's own rules. Tech Lab's IABTechLab/trusted-server#1194 agrees with the stack on one crate named `trusted-server-integrations`, and section 2.11 of `2026-10-26-integrations-crate-design.md` sets out where the two differ. +11. **Link it into the stock build.** At row 56 the builder is `trusted_server_integrations::cmp::example::builder()`, and it joins `shipped_module_builders()` in `src/app.rs` of each of the four adapters and `linked_builders()` in `crates/trusted-server-cli/src/app_config.rs`, in the place its request hooks should run. A permission signal module joins `shipped_signal_modules()` in each adapter instead, and the CLI links no permission signal module. At row 55 the five lists of builders hold the same builders in the same order, and no test compares one list with another, as `2026-10-14-no-modules-crate-design.md` records. A module that is not on the lists is not in the gold WASM, because the stock Fastly binary calls `trusted_server_adapter_fastly::run()`, which links the stock list and nothing else. A module kept in a repository of its own runs only in a deployment that builds its own binary, handing the module's builder to `run_with` on Fastly or to `routes_with_registrations` on Axum, Cloudflare and Spin, and that binary is not the gold WASM. An adapter whose list misses the module refuses it at startup once a section selects it, with "[cmp] selects `example`, which no module this deployment supplies. …", and when the CLI's list misses it, `ts config validate` and `ts config push` skip the module's own rules. Tech Lab's IABTechLab/trusted-server#1194 agrees with the stack on one crate named `trusted-server-integrations`, and section 2.11 of `2026-10-27-integrations-crate-design.md` sets out where the two differ. - IABTechLab/trusted-server#1194 finds every integration through "one compile-checked catalog" in that crate and makes "External vendor crate injection or adapter-supplied registration" a non-goal, where the stack keeps each adapter's own list and lets a deployment add a crate of its own through `run_with`. - Rule 6 of IABTechLab/trusted-server#1194 reads "No adapter reconstructs a concrete catalog or imports `aps`, `prebid`, or another integration module directly", so there the integrations are named only in the integrations crate, which puts the catalog together, where the stack also names them in the five lists written by hand. - IABTechLab/trusted-server#1194 gives each integration one flat folder, `src/integrations//`, where the stack keeps `src///`. @@ -165,7 +165,7 @@ The results and times in the table below are not measured yet. A fresh run of jo | 2.9 Check the deployment | `curl` of `/_ts/config.json`, then `ts healthcheck` | Viceroy, then a Fastly service | To be measured | To be measured | | 2.10 Change a policy, or roll back | `ts config push`, then `ts rollback` | Viceroy, then a Fastly service | To be measured | To be measured | -Row 56 itself is measured beside row 55, as section 2.10 of `2026-10-26-integrations-crate-design.md` says, with the same tests, the hashes of the 14 bundles, every module name and the time per request, and the results go in the description of row 56's pull request. That design's open question 5 asks whether rebuilding the whole integrations crate after a change to one folder's file is an acceptable cost for a module author, and no figure in this design settles it. +Row 56 itself is measured beside row 55, as section 2.10 of `2026-10-27-integrations-crate-design.md` says, with the same tests, the hashes of the 14 bundles, every module name and the time per request, and the results go in the description of row 56's pull request. That design's open question 5 asks whether rebuilding the whole integrations crate after a change to one folder's file is an acceptable cost for a module author, and no figure in this design settles it. ## 6. Open points for Tech Lab @@ -174,19 +174,19 @@ The journeys raise seven questions that only Tech Lab can answer. 1. **How an operator gets the gold WASM.** No signed release with binaries exists yet. The two pre-releases of 21 October 2026 carry no assets, IABTechLab/trusted-server#1435 is open, its item D7, immutable releases and protected `v*` tags, waits on a maintainer, and whether a tag on an rc branch may make a release is still open. The release verification guide asks an operator to obtain the approved tag and its source commit "through an independent source-review process". Where will Tech Lab publish each approved tag and its commit? 2. **When AC-8 ships.** Today the gold WASM reaches Fastly by the manual route that IABTechLab/trusted-server#1262 records as AC-9, which creates no store links and stages nothing. The AC-8 plan in IABTechLab/trusted-server#1435 adds `ts deploy --release` in task E7, for production first, and turns on `ts deploy --release --staging` only after task E8 has qualified production and then staging on disposable services. When does AC-8 ship, and will its staging cover the first deployment of this update, which changes the settings and the build together (journey 2, step 10)? 3. **Where each release lists what it carries.** Each binary of IABTechLab/trusted-server#1435 ships with an SPDX document, `.spdx.json`, and the first phase defers the planned `release-index.json`, whose schema is reserved for a later phase. The list of compile-time inputs is the audit in the design of IABTechLab/trusted-server#1435, made against main at `f9281917a`, and journey 3, step 5, shows how the stack changes it. Will each release publish the list for its own commit, in `release-index.json` or beside it, and who keeps the list current as the stack lands? Should the scan for planted configuration values also look for the text of the files in `config/permissions`, now that row 4 compiles no policy in? -4. **Which modules the gold build links.** A module reaches publishers on the gold WASM only through the stock list in Tech Lab's tree. IABTechLab/trusted-server#1194 and the stack agree on one crate, `trusted-server-integrations`. They differ in how a build finds its modules, being one compile-checked catalog with no registration from outside it in IABTechLab/trusted-server#1194, against each adapter's list and registration through `run_with` in the stack, and in the folders and the browser code (journey 1, step 11). Open question 7 of `2026-10-26-integrations-crate-design.md` asks whether the integrations crate should supply one stock list that the adapters and the tool take, with a deployment that links a different set still writing its own. Which of these will the gold build follow, what does a module need to join its list, for example a maintainers status of `vendor owned`, and will each release publish the list of modules it links? +4. **Which modules the gold build links.** A module reaches publishers on the gold WASM only through the stock list in Tech Lab's tree. IABTechLab/trusted-server#1194 and the stack agree on one crate, `trusted-server-integrations`. They differ in how a build finds its modules, being one compile-checked catalog with no registration from outside it in IABTechLab/trusted-server#1194, against each adapter's list and registration through `run_with` in the stack, and in the folders and the browser code (journey 1, step 11). Open question 7 of `2026-10-27-integrations-crate-design.md` asks whether the integrations crate should supply one stock list that the adapters and the tool take, with a deployment that links a different set still writing its own. Which of these will the gold build follow, what does a module need to join its list, for example a maintainers status of `vendor owned`, and will each release publish the list of modules it links? 5. **The caches in the gold build.** With no stock build linking a loading cache (rows 24 and 43), every load through `load_cached` on the gold WASM runs its loader, each robots.txt module's answer included. With the asset cache a module that no stock build links (row 34), the gold WASM fetches every file from its origin, as before row 34, and refuses an `[asset-cache]` section that selects the module. On Fastly the platform's own cache still answers the origin fetch for a file the origin lets that cache hold. Should the gold build link a loading cache module, or the asset cache module, and if so which? 6. **Cloudflare, Spin and the other platforms.** IABTechLab/trusted-server#1435 says that "Cloudflare, Spin, Windows and Linux ARM64 CLI releases follow separately", so operators of Cloudflare and Spin build from source today. When will those have a signed release? 7. **What the release workflow watches.** `.github/workflows/release.yml` in IABTechLab/trusted-server#1435 runs on a pull request only when a path in its filter changes. The filter names no module crate under `crates///`, no `crates/trusted-server-integrations/**`, no `crates/tsconfig.json` and no `trusted-server.example.toml`, which its design lists as embedded in the CLI, so a pull request that changes only one of those runs none of the release checks. Should the filter name them? ## Revision record -| Date | Change | -| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-26 | First draft. | -| 2026-10-26 | Fixes from review round 1. The two pre-releases of 21 October and the rc branches, AC-8 and AC-9, staging and the first deployment through versioned physical stores, the store links the manual route needs, the module folders of row 56, IABTechLab/trusted-server#1194, the build and test costs measured on row 56 itself, the issue and the pull request template, a local run under Viceroy, the route traps, the permission signal list, the statuses of startup errors, the pins of main and EdgeZero `v0.0.10`, the files the stack and IABTechLab/trusted-server#1435 both change, the release checks and the prerequisites, the policy entries on every adapter, and links pinned to commits. | -| 2026-10-26 | Fixes from review round 2. Rows are numbered as IABTechLab/trusted-server#777 numbers them, so the permission model is row 4. A folder gets a Cargo feature only for a dependency that not every build can or should carry, or for a module no stock build links. The local page check sets the origin and the hosts in the file, because the overlay only overrides keys that exist, and leaves out `--rewrite-host`. The table of changes says which were agreed at the Touchbase and which 51Degrees decided after the Touchbase. Each startup status is the status of the error's kind, `500` for a configuration or settings error and `503` for a key-value store error. The permission model, integrations crate, asset cache and module endpoints designs are followed where they changed, the file is named for 26 October, and the service and its domains, the secrets, the Edge Cookie store, the operator's one build, the versioned store commands, the clone caution, the two signer checks, main's list of builders and the asset cache in the gold build are stated. | -| 2026-10-26 | Fixes from review round 3. The configuration view never shows a policy's text. The local page check says plainly what it does and keeps the overlay out with `--no-env` or a shell with no `TRUSTED_SERVER__` variable. No stock build turns on the loading cache or asset cache feature, and a feature never gates a folder's source label. Row 32 joins the loading cache change. The operator journey adds the first release it can use, the proxy secret and the other keys, the template's placeholders, where the reason is logged, the guide for writing a policy, the operator's own build workflow, the versioned store route for an operator coming from main or from nothing, the folder every `ts` command runs from, and DNS and certificates. Blob, request preparer, SBOM, provenance, chunks, SPDX, SLSA and stock build are defined, and IABTechLab/trusted-server#1055 is said to cover the CLI's output and its guidance on adapters and tools. | +| Date | Change | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-27 | First draft. | +| 2026-10-27 | Fixes from review round 1. The two pre-releases of 21 October and the rc branches, AC-8 and AC-9, staging and the first deployment through versioned physical stores, the store links the manual route needs, the module folders of row 56, IABTechLab/trusted-server#1194, the build and test costs measured on row 56 itself, the issue and the pull request template, a local run under Viceroy, the route traps, the permission signal list, the statuses of startup errors, the pins of main and EdgeZero `v0.0.10`, the files the stack and IABTechLab/trusted-server#1435 both change, the release checks and the prerequisites, the policy entries on every adapter, and links pinned to commits. | +| 2026-10-27 | Fixes from review round 2. Rows are numbered as IABTechLab/trusted-server#777 numbers them, so the permission model is row 4. A folder gets a Cargo feature only for a dependency that not every build can or should carry, or for a module no stock build links. The local page check sets the origin and the hosts in the file, because the overlay only overrides keys that exist, and leaves out `--rewrite-host`. The table of changes says which were agreed at the Touchbase and which 51Degrees decided after the Touchbase. Each startup status is the status of the error's kind, `500` for a configuration or settings error and `503` for a key-value store error. The permission model, integrations crate, asset cache and module endpoints designs are followed where they changed, and the service and its domains, the secrets, the Edge Cookie store, the operator's one build, the versioned store commands, the clone caution, the two signer checks, main's list of builders and the asset cache in the gold build are stated. | +| 2026-10-27 | Fixes from review round 3. The configuration view never shows a policy's text. The local page check says plainly what it does and keeps the overlay out with `--no-env` or a shell with no `TRUSTED_SERVER__` variable. No stock build turns on the loading cache or asset cache feature, and a feature never gates a folder's source label. Row 32 joins the loading cache change. The operator journey adds the first release it can use, the proxy secret and the other keys, the template's placeholders, where the reason is logged, the guide for writing a policy, the operator's own build workflow, the versioned store route for an operator coming from main or from nothing, the folder every `ts` command runs from, and DNS and certificates. Blob, request preparer, SBOM, provenance, chunks, SPDX, SLSA and stock build are defined, and IABTechLab/trusted-server#1055 is said to cover the CLI's output and its guidance on adapters and tools. | ## Review log diff --git a/docs/superpowers/specs/2026-10-26-integrations-crate-design.md b/docs/superpowers/specs/2026-10-27-integrations-crate-design.md similarity index 99% rename from docs/superpowers/specs/2026-10-26-integrations-crate-design.md rename to docs/superpowers/specs/2026-10-27-integrations-crate-design.md index ea2fb9d2fa..5017850286 100644 --- a/docs/superpowers/specs/2026-10-26-integrations-crate-design.md +++ b/docs/superpowers/specs/2026-10-27-integrations-crate-design.md @@ -1,11 +1,11 @@ # Design Spec: One integrations crate holds every module, in a folder for its type and vendor -**Status:** Proposed, 2026-10-26. For a new row at the top of the stack, row 56, which folds every module crate into one crate, as IAB Tech Lab asked at the Touchbase of 2026-10-23 and as decided by 51Degrees after the Touchbase. No pull request exists for the row yet. +**Status:** Proposed, 2026-10-27. For a new row at the top of the stack, row 56, which folds every module crate into one crate, as IAB Tech Lab asked at the Touchbase of 2026-10-23 and as decided by 51Degrees after the Touchbase. No pull request exists for the row yet. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-14-no-modules-crate-design.md`, `2026-10-19-module-scripts-in-crates-design.md`, `2026-10-12-loading-cache-design.md`, `2026-10-15-asset-cache-design.md`, `2026-08-27-integration-provider-seam-design.md`, and IAB Tech Lab's `2026-09-17-split-integrations-crates-design.md` on [IABTechLab/trusted-server#1194](https://github.com/IABTechLab/trusted-server/pull/1194) **Related PRs:** [IABTechLab/trusted-server#1194](https://github.com/IABTechLab/trusted-server/pull/1194) (Aram Grigoryan, draft), [IABTechLab/trusted-server#1423](https://github.com/IABTechLab/trusted-server/pull/1423) (row 55, the top of the stack below this row). [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack **Issues:** [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** The stack gives each module a crate of its own, 32 crates under `crates///`. At the Touchbase of 2026-10-23, IAB Tech Lab asked for those folders to be kept inside one crate, as Tech Lab's own design puts every integration in one crate, and 51Degrees agreed. @@ -321,10 +321,10 @@ These are for the reviewer, and this design decides none of them. | Date | Change | | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-26 | First draft. | -| 2026-10-26 | Fixes from review round 1. tokio's `rt-multi-thread`, which the loading cache's tests ask for, goes in a table that only native builds read, with the threaded tests on the host. The paths that climb out of a crate's folder are listed with their fixes, and `run.cjs`, `crates/fastly.toml`, the seam probe's workflow step and the core documents that name a crate's path join section 3. The browser scripts' second reason says that the APS renderer and Prebid's code stay in `trusted-server-js`, where #1194 would move them. Row 56's own measurement goes in its pull request. The manifest's lint and doctest settings, the source labels written as text, the log labels on Fastly, the places the Rust names are used, the rule for `51degrees`, the probe's one name of another type, the Fastly SDK's host imports, the files with license text of their own, the count of rows a fold would travel and where vendor names live beside #1194 are stated. Open questions 4 and 5 are reworded and 7 and 8 added. The Touchbase of 2026-10-23 says what Tech Lab asked, and that 51Degrees decided, and the seam probe and Viceroy are defined where first used. | -| 2026-10-26 | Fixes from review round 2. The split design's pull request names its author, Aram Grigoryan. The account of the Touchbase gives Tech Lab's request as the record has it, and places the decisions of 51Degrees after the Touchbase. "Stock build" is defined where first used, the split design's integrations crate is the one that puts the catalog together, and a feature gates neither a folder's name constants nor its source label. `build-fastly`, `check-fastly` and `test-fastly` name `asset-cache-51degrees` and none of them names `loading-cache-51degrees`, each `maintainers.toml` comment says "module" where the manifest says "crate", and the sample permission policy joins section 3. | -| 2026-10-26 | The count of the other design documents naming a path under `crates///` is 43, with the search that gives it. | +| 2026-10-27 | First draft. | +| 2026-10-27 | Fixes from review round 1. tokio's `rt-multi-thread`, which the loading cache's tests ask for, goes in a table that only native builds read, with the threaded tests on the host. The paths that climb out of a crate's folder are listed with their fixes, and `run.cjs`, `crates/fastly.toml`, the seam probe's workflow step and the core documents that name a crate's path join section 3. The browser scripts' second reason says that the APS renderer and Prebid's code stay in `trusted-server-js`, where #1194 would move them. Row 56's own measurement goes in its pull request. The manifest's lint and doctest settings, the source labels written as text, the log labels on Fastly, the places the Rust names are used, the rule for `51degrees`, the probe's one name of another type, the Fastly SDK's host imports, the files with license text of their own, the count of rows a fold would travel and where vendor names live beside #1194 are stated. Open questions 4 and 5 are reworded and 7 and 8 added. The Touchbase of 2026-10-23 says what Tech Lab asked, and that 51Degrees decided, and the seam probe and Viceroy are defined where first used. | +| 2026-10-27 | Fixes from review round 2. The split design's pull request names its author, Aram Grigoryan. The account of the Touchbase gives Tech Lab's request as the record has it, and places the decisions of 51Degrees after the Touchbase. "Stock build" is defined where first used, the split design's integrations crate is the one that puts the catalog together, and a feature gates neither a folder's name constants nor its source label. `build-fastly`, `check-fastly` and `test-fastly` name `asset-cache-51degrees` and none of them names `loading-cache-51degrees`, each `maintainers.toml` comment says "module" where the manifest says "crate", and the sample permission policy joins section 3. | +| 2026-10-27 | The count of the other design documents naming a path under `crates///` is 43, with the search that gives it. | | 2026-10-27 | The license sentence names `fiftyone-caching` 4.5.13 in place of 4.5.8, because row 31 pins 4.5.13. | ## Review log diff --git a/docs/superpowers/specs/2026-10-26-module-endpoints-design.md b/docs/superpowers/specs/2026-10-27-module-endpoints-design.md similarity index 98% rename from docs/superpowers/specs/2026-10-26-module-endpoints-design.md rename to docs/superpowers/specs/2026-10-27-module-endpoints-design.md index ca0db7d8d1..895a508d47 100644 --- a/docs/superpowers/specs/2026-10-26-module-endpoints-design.md +++ b/docs/superpowers/specs/2026-10-27-module-endpoints-design.md @@ -1,11 +1,11 @@ # Design Spec: A module outside core serves an endpoint of its own -**Status:** Proposed, 2026-10-26. For the pull request [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353), "Prove a module outside core can serve its own endpoint", row 12 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), for Tech Lab review. One commit removes what the pull request's attestation commits changed, returning the row's files to those of row 11, and the tests and the guide section below come after that commit. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Status:** Proposed, 2026-10-27. For the pull request [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353), "Prove a module outside core can serve its own endpoint", row 12 of the stack in [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777), for Tech Lab review. One commit removes what the pull request's attestation commits changed, returning the row's files to those of row 11, and the tests and the guide section below come after that commit. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md` (row 7, the integration seam), `2026-10-12-diagnostic-headers-design.md` (row 15, which keeps `PlatformIdentity` in core), `2026-10-17-request-filters-on-every-adapter-design.md` (row 52) and `2026-10-17-edge-cookie-on-every-adapter-design.md` (row 53), in [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) **Related PRs:** [IABTechLab/trusted-server#1353](https://github.com/IABTechLab/trusted-server/pull/1353) (row 12), [IABTechLab/trusted-server#1348](https://github.com/IABTechLab/trusted-server/pull/1348) (row 7, which opens the integration seam so that a vendor module can live outside core), [IABTechLab/trusted-server#1354](https://github.com/IABTechLab/trusted-server/pull/1354) (row 13, which closes the administration paths), [IABTechLab/trusted-server#1356](https://github.com/IABTechLab/trusted-server/pull/1356) (row 15), [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410) (row 52, the request filters on every adapter) and [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411) (row 53, the Edge Cookie on every adapter) **Issues:** none. [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-26 +**Last updated:** 2026-10-27 > **Why this spec exists.** A module outside core can already answer a path of its own on every adapter, so a deployment that wants an endpoint core does not have, attestation among them, serves that endpoint from a module and core needs no interface for it. No test shows a module answering a path outside `/integrations/` on all four adapters, and no guide says how a module does so, what runs before its route or which paths it can never take. This change adds those tests and that guide section, and no production code in core or in any adapter changes. @@ -87,9 +87,9 @@ Another method on a path of item 1 reaches the fallback as any other request doe | Date | Change | | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-26 | First draft. | -| 2026-10-26 | Fixes from review round 1. What each adapter does at startup when two routes clash, and the other routes matchit refuses, the shipped modules that declare paths outside their prefix, what a module call carries at this row and from row 39 and what a gated value needs, where each adapter classifies the device and when the Edge Cookie step answers, the requests a module route never sees, the request filters and DataDome, the guide section and the two places it amends, the probe's handler and counter with five tests, the platform identity and the header. | -| 2026-10-26 | Fixes from review round 2. What a refusal does on the stock entry points and on a deployment's own, with a clash from settings alone, two routes on one method and path in place of two modules on one path, every function that takes outside builders, when each adapter builds its state, the plan registration, the shipped modules' paths outside `/integrations/`, the steps that keep a request from a module route and the rows that change them, and `HEAD` bodies. | +| 2026-10-27 | First draft. | +| 2026-10-27 | Fixes from review round 1. What each adapter does at startup when two routes clash, and the other routes matchit refuses, the shipped modules that declare paths outside their prefix, what a module call carries at this row and from row 39 and what a gated value needs, where each adapter classifies the device and when the Edge Cookie step answers, the requests a module route never sees, the request filters and DataDome, the guide section and the two places it amends, the probe's handler and counter with five tests, the platform identity and the header. | +| 2026-10-27 | Fixes from review round 2. What a refusal does on the stock entry points and on a deployment's own, with a clash from settings alone, two routes on one method and path in place of two modules on one path, every function that takes outside builders, when each adapter builds its state, the plan registration, the shipped modules' paths outside `/integrations/`, the steps that keep a request from a module route and the rows that change them, and `HEAD` bodies. | ## Review log From 85e6d81c3a31485ca4cc91a09e2f6fbddbcbfb28 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:46 +0000 Subject: [PATCH 190/192] The permission designs say a held policy is not read again A load asks for the rules held for a pinned digest before it asks a source for the text, so an instance that already holds a host's digest reads and hashes nothing for it. Section 3.7 and item 6 of section 3.1 of the permission model design say so, and the line numbers cited in permissions.rs and config_payload.rs follow the fix in #1345. The config by host design says that such an instance reads no policy entry for the host, in its account of each policy file and of the entry that makes a host fail to load. --- .../2026-07-30-permission-model-design.md | 47 ++++++++++--------- .../specs/2026-10-12-config-by-host-design.md | 14 +++--- 2 files changed, 32 insertions(+), 29 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 24b9aa5b9e..6d2f6c2a29 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -219,7 +219,7 @@ build its state from settings that carry none, through request is decided without a policy and nothing stands in for a missing one. The tests read the sample through `PermissionPolicy::sample()` and `PermissionMaps::sample()`, which only test builds and core's `test-utils` -feature compile (`permissions.rs` lines 870 to 918), and the `gpc`, `gpp`, +feature compile (`permissions.rs` lines 895 to 943), and the `gpc`, `gpp`, `tcf` and `us-privacy` crates list core with `test-utils` under their `[dev-dependencies]` for them. @@ -305,7 +305,7 @@ that names each chunk with its length and SHA-256. and no reader of its own, and hands the runtime loader `PolicyOrigin::Blob` (the Fastly adapter's `app.rs` lines 232 to 238, Spin's `app.rs` line 137 and Cloudflare's `app.rs` line 144). The loader, - `settings_from_config_blob` (`config_payload.rs` lines 51 to 83), takes + `settings_from_config_blob` (`config_payload.rs` lines 52 to 84), takes `text` out of the settings, checks its SHA-256 against `sha256`, parses it, and keeps the policy with the settings, which keep only its name and digest (`load_domain_policy`, lines 89 to 129), so a blob there without `text` @@ -367,8 +367,9 @@ that names each chunk with its length and SHA-256. text arrives inside the blob. `load_policy` (`permissions.rs` line 821) checks the SHA-256 of the text against `sha256`, parses it, and keeps one parsed copy for each digest for the life of the instance, under whatever - name it is loaded (`parsed_policy`, lines 847 to 866), so two domains that - pin one digest share one copy. + name it is loaded (`parsed_policy`, lines 862 to 875), so two domains that + pin one digest share one copy. The runtime loader asks for that copy + (`held_policy`, lines 851 to 858) before it asks a source for the text. 7. **Never started without a valid file.** A domain fails to load when its settings name no policy, or name one with no `sha256`, when the file is absent, which on Fastly, Cloudflare and Spin means a blob without `text`, @@ -626,7 +627,7 @@ The draft's specific complaints are answered: No build compiles the sample in, so core's library code reads no policy file above the crate root, which settles the packaging concern the draft raised about the `include_str!` path. Only test builds and core's `test-utils` -feature read the sample (`permissions.rs` lines 870 and 871, and §3.5), and +feature read the sample (`permissions.rs` lines 895 and 896, and §3.5), and core's build script still reads the workspace's `Cargo.toml`, `Cargo.lock` and `edgezero.toml` above the crate root (`WORKSPACE_INPUTS` in `build.rs`). @@ -1000,19 +1001,20 @@ instance is differs by adapter. | Spin | Inside the blob, in the `default` key-value store the settings come from, read with it | A component instance, which builds its state for each request and serves one request or several as Spin decides | | Axum | `/.yaml` | The process, which loads its settings once when it starts | -An instance takes the SHA-256 of the text on every load, and parses the text -the first time it meets the digest. An instance that already holds the digest -a blob names parses nothing, but it still reads the file and hashes it. The -runtime loader asks its source for the text before anything looks for a -parsed copy (`load_domain_policy`, `config_payload.rs` lines 131 to 171 at -IABTechLab/trusted-server#1352), and `load_policy` takes the SHA-256 of the -text it is handed before `parsed_policy` finds the copy the instance holds -(`permissions.rs`). So from row 11 a host's blob costs an instance that -already holds its digest the reads of its policy entry, through -`FastlyPolicyEntries::policy_text` (the Fastly adapter's `platform.rs`), -being the pointer and its chunks, five for the sample, and a hash of the -text, and no parse. Checking the per-digest cache before asking the source is -a change left for the measurement on Fastly staging that this section names. +An instance reads, hashes and parses a policy file the first time it meets +the file's digest. An instance that already holds the digest a blob names +reads nothing, hashes nothing and parses nothing, because the runtime loader +asks for the rules held for that digest before it asks its source for the +text (`load_domain_policy` in `config_payload.rs`, through `held_policy` in +`permissions.rs`). Every file that is read is still checked against the +digest, and a file that is refused leaves nothing held, so the next load +reads and checks that file again. Text that travels inside the blob is +always checked, because it is already in hand, and a blob that should carry +its policy's text and carries none is refused whatever the instance holds. So from row 11 a host's blob +costs an instance that already holds its digest no read of its policy entry, +and the reads of that entry, through `FastlyPolicyEntries::policy_text` (the +Fastly adapter's `platform.rs`), being the pointer and its chunks, five for +the sample, are paid once for each digest an instance meets. The parsed copies stay until the instance ends, with no lifetime of their own, because a digest never changes what it names. An instance keeps one copy for each digest its domains' blobs pin, so the hundred hosts that name @@ -1024,10 +1026,10 @@ instance meets its digest (`parsed_policy` in `permissions.rs`), so on Fastly, where an instance serves one request unless the opt-in feature is built, a request parses its domain's file once, where before row 4 it parsed the compiled sample twice, once to check it and once on first use. What is new on -Fastly is the digest, taken on every load, and the three to five reads that a -blob carrying the sample adds, and from row 11, for a host's blob, five reads -of its policy entry on every load, whether or not the instance already holds -the digest. +Fastly is the digest, taken on every load of a blob that carries its policy, +and the three to five reads that such a blob adds, and from row 11, for a +host's blob, five reads of its policy entry the first time an instance meets +its digest. On Cloudflare, at row 4 (IABTechLab/trusted-server#1345), the Worker's fetch entry point `main` (the Cloudflare adapter's `lib.rs`) calls `build_app` for @@ -2030,6 +2032,7 @@ include the changes that followed the first revision of this section on | 2026-10-15 | Fixes from review round 1. An unreadable record answered by its own module with nothing ahead of the modules, the fifth module `mtm` with its own section, registration through each adapter, the four members of the page state, the sharing pair on navigation and page-bids, the persisted consent, resolve endpoint and device module check named as built by other rows or not at all, what the policy parser does not reject, the Data Uses a signal reaches, the raw strings removed after assembly, a table of what each adapter runs, the upgrade list, the open questions and the review log. | | 2026-10-16 | Review round 2, approved with one minor point recorded in the Review log. | | 2026-10-27 | The policy file is never compiled into a build, as agreed with IAB Tech Lab at the Touchbase on 2026-10-23, and is selected and supplied when a service is deployed, as decided by 51Degrees after the Touchbase (§3.1). What row 4 builds and later rows add, Cloudflare reading its blob from the config store, the push, answers that follow the error's kind, and a blob's propagation (§3.1). What the build carries (§3.6), the cost of the fetch (§3.7), the failure modes (§6), the upgrade list (§6.1), the tests (§8), the records in §10 and §11, and the sample named as the sample. | +| 2026-10-27 | A load asks for the rules held for a pinned digest before it reads a source, so an instance that already holds a host's digest reads and hashes nothing for it (§3.7, item 6 of §3.1), a fix found in review. The line numbers of `permissions.rs` and `config_payload.rs` follow the fix. | ## Review log diff --git a/docs/superpowers/specs/2026-10-12-config-by-host-design.md b/docs/superpowers/specs/2026-10-12-config-by-host-design.md index 20b45dc225..c7900ec785 100644 --- a/docs/superpowers/specs/2026-10-12-config-by-host-design.md +++ b/docs/superpowers/specs/2026-10-12-config-by-host-design.md @@ -112,7 +112,7 @@ The service has one of each of the following, and nothing divides any of them be - **The secret store.** Each publisher's blob is meant to name keys of its own in the secret store, and nothing checks that the keys differ between blobs, because each application is built from one blob and sees no other. Two blobs that name one secret read the same value. Where an Edge Cookie ID is made from a passphrase, its first 64 characters are a keyed hash (HMAC-SHA256) of the client IP address under that passphrase (`crates/trusted-server-core/src/ec/generation.rs:104-115`), so two blobs that name one passphrase give one client IP address the same prefix at both publishers, and a person can be linked across them. The harness's blobs name the same secrets (`scripts/config-by-host-local-test.sh:185-221`). Two blobs that name one Edge Cookie identity store (`ec_store`) share one identity graph. - **The request signing keys and the template cache's purge key.** The signing keys sit in two stores whose names are fixed, `jwks_store` and `signing_keys` (`crates/trusted-server-core/src/request_signing/mod.rs:40-54`). Every template in the template cache carries the cache tag (surrogate key) `ts-template` (`crates/trusted-server-core/src/platform/template_cache.rs:46`), which one purge call flushes (`crates/trusted-server-adapter-fastly/src/template_cache.rs:288-291`). Three routes act on them, `/_ts/admin/keys/rotate`, `/_ts/admin/keys/deactivate` and `/_ts/admin/cache/purge` with a scope of `all` (`app.rs:1415-1429`). Each is checked against the administration user and password in the blob of the host the request names (`app.rs:1624`), so one publisher's user and password can rotate the keys every publisher signs with and flush the cached templates of every publisher. Row 13 (`split/13-removals`) removes the administration user and password, and these three routes go with them (`2026-10-12-administration-off-the-publisher-domain-design.md`). -- **Each policy file.** Row 4 of the stack (IABTechLab/trusted-server#1345) selects the permission policy when a service is deployed (`2026-07-30-permission-model-design.md` §3.1), and this row builds on it. Each host's blob names its policy file in `[permissions] policy`, with the file's SHA-256, and carries no copy of the file. The keyed push keeps each file `--permissions` supplies once, as a policy entry under `permissions_` in the config store, which is a pointer and chunks when the entry is longer than the 8,000 characters one entry holds. It writes each entry the store does not already hold before the host's blob, and keeps one that holds the same file (`run` and `holds_policy` in `crates/trusted-server-cli/src/commands/config/keyed_push.rs`). Core forms the key with `policy_key` (`crates/trusted-server-core/src/platform/policy_source.rs`). For every blob but the one at the logical store ID, the adapter reads the policy from that entry (`blob_carries_its_policy` in `app.rs`, and `FastlyPolicyEntries` in `platform.rs`). The blob at the logical store ID carries its file, as on a service with one blob. The tests `two_hosts_naming_one_policy_file_keep_it_once` (`crates/trusted-server-cli/tests/config_push_by_key.rs`) and `a_host_blob_whose_policy_entry_is_missing_is_refused_naming_the_entry` (`platform.rs`) show one entry kept for two hosts and a missing entry refused. Hosts whose blobs pin one SHA-256 share that entry and run the same policy, and a file edited between two pushes is two digests and two entries. A host fails to load, as a blob that cannot be loaded does (the answers above), and answers `500`, when its blob names no policy, or its policy entry is absent, cannot be read, does not match its digest or fails the checks of §3.3 of the permission model design. A config store that cannot be read answers `500`, as the selection does when it cannot look a host up (§3.1 of the permission model design, item 7). The other hosts are answered as before. This row's keyed writer writes only the local server's store, and row 13 gives it the live store. `ts config diff` cannot reach a host's blob, because EdgeZero `v0.0.10` refuses every Fastly key but the logical store ID for a diff as for a push (`resolve_config_store_and_key` in its `config.rs`), so a host's pinned policy is checked through `/_ts/config` on that host. +- **Each policy file.** Row 4 of the stack (IABTechLab/trusted-server#1345) selects the permission policy when a service is deployed (`2026-07-30-permission-model-design.md` §3.1), and this row builds on it. Each host's blob names its policy file in `[permissions] policy`, with the file's SHA-256, and carries no copy of the file. The keyed push keeps each file `--permissions` supplies once, as a policy entry under `permissions_` in the config store, which is a pointer and chunks when the entry is longer than the 8,000 characters one entry holds. It writes each entry the store does not already hold before the host's blob, and keeps one that holds the same file (`run` and `holds_policy` in `crates/trusted-server-cli/src/commands/config/keyed_push.rs`). Core forms the key with `policy_key` (`crates/trusted-server-core/src/platform/policy_source.rs`). For every blob but the one at the logical store ID, the adapter reads the policy from that entry (`blob_carries_its_policy` in `app.rs`, and `FastlyPolicyEntries` in `platform.rs`), unless the instance already holds the rules of that SHA-256, when it reads nothing (§3.7 of the permission model design). The blob at the logical store ID carries its file, as on a service with one blob. The tests `two_hosts_naming_one_policy_file_keep_it_once` (`crates/trusted-server-cli/tests/config_push_by_key.rs`) and `a_host_blob_whose_policy_entry_is_missing_is_refused_naming_the_entry` (`platform.rs`) show one entry kept for two hosts and a missing entry refused. Hosts whose blobs pin one SHA-256 share that entry and run the same policy, and a file edited between two pushes is two digests and two entries. A host fails to load, as a blob that cannot be loaded does (the answers above), and answers `500`, when its blob names no policy, or when the instance does not already hold the rules of that SHA-256 and its policy entry is absent, cannot be read, does not match its digest or fails the checks of §3.3 of the permission model design. A config store that cannot be read answers `500`, as the selection does when it cannot look a host up (§3.1 of the permission model design, item 7). The other hosts are answered as before. This row's keyed writer writes only the local server's store, and row 13 gives it the live store. `ts config diff` cannot reach a host's blob, because EdgeZero `v0.0.10` refuses every Fastly key but the logical store ID for a diff as for a push (`resolve_config_store_and_key` in its `config.rs`), so a host's pinned policy is checked through `/_ts/config` on that host. - **The rate counter,** `counter_store` (`rate_limiter.rs:9`), which limits Edge Cookie sync requests. - **The sandbox's memory bound and counters.** Where a sandbox serves several requests, every application it keeps shares the bound, and the counters it reports count the requests and builds of every publisher it serves. @@ -177,12 +177,12 @@ The behaviors below are stated in the sections above. A change to the code would ## Revision record -| Date | Change | -| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| 2026-10-12 | First draft, from the pull request and its issue as they stand. | -| 2026-10-16 | Round 1 fixes. The header points at the open draft pull request, the risk table is repaired, the fallback to the logical store ID is stated, the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests are added, and the code findings sit under Open questions. The selector store points at the Fastly per-host selector design. | -| 2026-10-16 | Round 2 fixes. The pointer to the Fastly per-host selector design names section 2.4, whose `500` answers replace the fallback to the logical store ID, and sections 2.6 and 2.8, which replace the push and selector-writing steps. The same sections are named where this spec describes the fallback and the steps for moving a service. | -| 2026-10-27 | Each host's blob names the policy file row 4 selects, and one policy entry for each SHA-256 serves every host that pins it, as this row's keyed push and Fastly adapter do. Chunks and pointers are defined, the EdgeZero versions are anchored, `ts config diff` is said not to reach a host's blob, and Open questions, item 9, asks about a selector that forms a policy entry's key. | +| Date | Change | +| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-12 | First draft, from the pull request and its issue as they stand. | +| 2026-10-16 | Round 1 fixes. The header points at the open draft pull request, the risk table is repaired, the fallback to the logical store ID is stated, the host rules, the answers, the steps for moving a service and acceptance criteria that name their tests are added, and the code findings sit under Open questions. The selector store points at the Fastly per-host selector design. | +| 2026-10-16 | Round 2 fixes. The pointer to the Fastly per-host selector design names section 2.4, whose `500` answers replace the fallback to the logical store ID, and sections 2.6 and 2.8, which replace the push and selector-writing steps. The same sections are named where this spec describes the fallback and the steps for moving a service. | +| 2026-10-27 | Each host's blob names the policy file row 4 selects, and one policy entry for each SHA-256 serves every host that pins it, as this row's keyed push and Fastly adapter do. Chunks and pointers are defined, the EdgeZero versions are anchored, `ts config diff` is said not to reach a host's blob, and Open questions, item 9, asks about a selector that forms a policy entry's key. An instance that already holds the rules of a SHA-256 reads no policy entry for it, as row 4 now does, a fix found in review. | ## Review log From 6d15cf1aaaa538af9d16122f6b7926934b294ddf Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:47 +0000 Subject: [PATCH 191/192] The asset cache design says the vary table refuses its lifetimes A lifetime set in [cache.51degrees-assets-vary] is refused when the settings load and by the deploy check, a fix found in review in #1375. The caches the module declares, failure handling, the rollout of a document written for row 34, the module's tests and what no test covers say so. --- .../specs/2026-10-15-asset-cache-design.md | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/docs/superpowers/specs/2026-10-15-asset-cache-design.md b/docs/superpowers/specs/2026-10-15-asset-cache-design.md index b07a2a771c..2f56020295 100644 --- a/docs/superpowers/specs/2026-10-15-asset-cache-design.md +++ b/docs/superpowers/specs/2026-10-15-asset-cache-design.md @@ -222,11 +222,11 @@ store_idle_seconds = 86400 ### 2.5 The two caches it declares -`builder()` declares `51degrees-assets` and `51degrees-assets-vary` with `with_cache_features` (row 32, `2026-10-15-module-cache-declarations-design.md`), following the form row 32 asks for, the module's short name and then what the cache holds, as row 44's `51degrees-answers` does. A build that links the module accepts `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]` tables, and a build without it refuses each as a table naming a cache no linked module declares, by the deploy check and when the state is built. A `[cache.assets]` table written for row 34 now names a cache that no module declares, so every build refuses it, and section 2.14 says how such a document changes. +`builder()` declares `51degrees-assets` and `51degrees-assets-vary` with `with_cache_features` (row 32, `2026-10-15-module-cache-declarations-design.md`), following the form row 32 asks for, the module's short name and then what the cache holds, as row 44's `51degrees-answers` does. A build that links the module accepts `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]` tables, the second only without lifetimes, and a build without it refuses each as a table naming a cache no linked module declares, by the deploy check and when the state is built. A `[cache.assets]` table written for row 34 now names a cache that no module declares, so every build refuses it, and section 2.14 says how such a document changes. Each table selects the loading cache module of its own cache, under row 32's rule. A table that names a module takes it. A table that names no module, and a declared cache with no table, take the one loading cache module the build links, and hold nothing when the build links none or several. A table that names a module no linked crate offers is refused. Before this rewrite `[cache.assets]` selected the loading cache module of both caches, and `[cache.assets-vary]` could replace it for the record cache. Now each table selects its own, so a build that links two or more loading cache modules, and names one in `[cache.51degrees-assets]`, names it in `[cache.51degrees-assets-vary]` as well. A build that links one loading cache module needs neither table. -The module sizes both caches from `[cache.51degrees-assets]`, reading `settings.cache.declared` with `LoadingCacheConfig::lifetimes` and `ASSET_DEFAULT_LIFETIMES` for what the table leaves unset, as row 44's 51Degrees engine cache sizes `51degrees-answers`. Core's `prepare_runtime` checks the ranges of both tables, as it does for every declared table. The lifetimes of `[cache.51degrees-assets-vary]` are checked and not used, as those of `[cache.assets-vary]` were on row 34. The memory layer holds at most 256 records and 32 copies, literals in the module's `asset_cache`. +The module sizes both caches from `[cache.51degrees-assets]`, reading `settings.cache.declared` with `LoadingCacheConfig::lifetimes` and `ASSET_DEFAULT_LIFETIMES` for what the table leaves unset, as row 44's 51Degrees engine cache sizes `51degrees-answers`. Core's `prepare_runtime` checks the ranges of both tables, as it does for every declared table. A lifetime set in `[cache.51degrees-assets-vary]` is refused when the settings load and by the deploy check, naming the table and each setting it sets, because the vary records take the lifetimes of `[cache.51degrees-assets]` (`refuse_record_lifetimes` in the module's `lib.rs`, called from its `validate`). The memory layer holds at most 256 records and 32 copies, literals in the module's `asset_cache`. `none` is no longer read. A table that names `none` is refused, as one naming any module no linked crate offers is. @@ -353,6 +353,7 @@ A load that fails asking the origin ends with every request that ran or waited o - **An error status.** Only a `200` is stored, so the body of an origin error or of a `404` is never held. The verdict is held, as "What is stored" says. For 60 seconds the requests that find the marker ask the origin themselves and none of them stores a copy, so a file the origin begins to answer with a `200` is stored only once the marker ends. - **A body that cannot be read whole.** The reader is served the whole body from the one fetch, as section 2.7 describes. - **A selection the build cannot meet.** `[asset-cache]` naming a module the build does not link, an `[asset-cache]` that selects more than one module, and an asset cache module supplied by a builder other than the one `[asset-cache]` selects are refused when the adapter builds its registry, so every request answers with the adapter's startup error, as for any module selection the registry refuses. A `[cache.51degrees-assets]` or `[cache.51degrees-assets-vary]` table in a build that does not link the module, and a `[cache.assets]` or `[cache.assets-vary]` table in any build, are refused by the deploy check and when the state is built. +- **A lifetime in the vary table.** `[cache.51degrees-assets-vary]` setting `memory_seconds`, `store_seconds` or `store_idle_seconds` is refused when the settings load and by the deploy check, so the service answers 500 and logs the settings it names, because nothing reads those lifetimes. - **Caches that keep nothing.** A module selected in a build whose loading caches keep nothing answers `AskOrigin` for every file, and nothing logs that it holds nothing (see the open questions). ### 2.13 Cost and limits @@ -368,7 +369,7 @@ A load that fails asking the origin ends with every request that ran or waited o 1. **A deployment that does not choose the module sees no change.** Every file is asked of its origin, as on row 33, whether or not a loading cache module is linked. 2. **A deployment that wants assets held** links a loading cache module and this module, hands both builders to its adapter, and writes `[asset-cache] module = "51degrees"`. It then holds shareable assets with no `[cache.51degrees-assets]` table when it links one loading cache module. A change at the origin shows after the lifetime the origin gave the file, and never later than `store_seconds`, which defaults to one hour. The checks in "What is stored" and the cookie rule apply to its asset traffic from the first deploy. -3. **A document written for row 34 before this rewrite** changes in four ways. It adds `[asset-cache]`, without which nothing is held. Its `[cache.assets]` and `[cache.assets-vary]` tables become `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]`, because no module declares the old names, so every build refuses them. A `module = "none"` line is refused, so it is removed. Where its build links two or more loading cache modules, it writes `[cache.51degrees-assets-vary]` naming the module its `[cache.51degrees-assets]` names, because each table now selects its own cache. +3. **A document written for row 34 before this rewrite** changes in five ways. It adds `[asset-cache]`, without which nothing is held. Its `[cache.assets]` and `[cache.assets-vary]` tables become `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]`, because no module declares the old names, so every build refuses them. A `module = "none"` line is refused, so it is removed. Where its build links two or more loading cache modules, it writes `[cache.51degrees-assets-vary]` naming the module its `[cache.51degrees-assets]` names, because each table now selects its own cache. A lifetime its `[cache.assets-vary]` table set is dropped, because the renamed table refuses it. 4. **The order of a push.** A build without the module refuses a document that carries `[cache.51degrees-assets]` or `[asset-cache] module = "51degrees"`, so a deployment deploys the build that links the module before it pushes such a document, and pushes a document without them before it rolls back to a build without the module. 5. **The `ts` tool.** The stock tool links neither the module nor a loading cache module, so `ts config validate` refuses `[cache.51degrees-assets]` as a table no linked module declares. A deployment that uses the module validates with a list that holds both builders, registered first through `register_deploy_integrations` in core, as `2026-10-14-no-modules-crate-design.md` describes for any set of modules other than the stock one. 6. **A copy outlives a new build and a change to the settings.** The key holds no digest of the build or the settings (section 2.7), so on both paths a copy made before a build or a settings change is found after it, and ends by its own lifetime, never later than `store_seconds`. On Spin, whose `default` store gives an entry no lifetime, the entry itself stays in the store after that, until a read past its drop time or a removal takes it out (see "Keys and variants without a limit"). A change to the publisher's origin URL or to the `Host` sent to it, or to a route's origin or prefix, changes `origin_identity` and so the key. A change to a route's credentials does not. @@ -420,7 +421,7 @@ The module's tests of the cookie rule build their `AssetFile` with `carries_cook | The source cache | Record and copy keys follow the publisher. `Vary` names are lowercase, sorted and named once. `*`, `Cookie` and an unreadable `Vary` allow no copies. A record reads back, and a foreign one does not. A variant tells absent, empty and reordered values apart. The first response records its `Vary` and each value gets its own copy. A response that allows no copies has every request fetch its own. A source that changes its `Vary` has its record dropped. A declared token separates copies | | The publisher path | An image is kept as sent and one fetch serves three requests. A stylesheet is served from one fetch with its age. `Vary: *`, `Vary: Cookie`, `private`, `no-store` and `public, no-store` are never kept. `s-maxage` then `max-age` set the lifetime. A header named in `Vary` separates copies. A body over 1 MiB is served whole from one fetch, with the stub streaming and not. A response whose `Vary` changed is served from its one fetch. A load that fails asking the origin is the answer, `OriginFailed` after one origin request, and the next request loads again. A request carrying `Authorization` is fetched for itself. A request carrying a cookie is fetched for itself and its answer kept for no one. An answer that sets a cookie is served only to the reader it was made for. One fetch serves two requests when the settings name no loading cache module and one is linked. A document is never answered from a copy | | An asset route | A route's answer is held and serves the next request, with its type, an `Age`, its bytes and the origin's lifetime. An answer with `Set-Cookie` is fetched each time. A reader's cookie does not keep a route's answer from a copy. A range request is fetched. `no-cache`, `no-store` and `min-fresh=10` are fetched. Nothing is held while the caches keep no values. A load that fails asking the origin is the answer, `OriginFailed` after one origin request | -| The caches and settings | The builder declares `51degrees-assets` and `51degrees-assets-vary`. `[cache.51degrees-assets]` sizes both caches, with the defaults for what it leaves unset. A lifetime out of range in `[cache.51degrees-assets]` is refused, naming the table and the field. A record cache that keeps nothing beside a copy cache that keeps values asks the origin directly. A `[cache.51degrees-assets] module = "none"` table is refused. An `[asset-cache.51degrees]` table is refused | +| The caches and settings | The builder declares `51degrees-assets` and `51degrees-assets-vary`. `[cache.51degrees-assets]` sizes both caches, with the defaults for what it leaves unset. A lifetime out of range in `[cache.51degrees-assets]` is refused, naming the table and the field. A record cache that keeps nothing beside a copy cache that keeps values asks the origin directly. A `[cache.51degrees-assets] module = "none"` table is refused. An `[asset-cache.51degrees]` table is refused. A lifetime set in `[cache.51degrees-assets-vary]` is refused, alone or with the others, naming the table and each setting it sets, and the table without lifetimes, or with only `module`, loads. | Row 34's `assets_turned_off_in_the_settings_are_fetched_for_every_request` and `assets_naming_none_turns_both_asset_caches_off_and_nothing_else` go, with `none`, and core's no-module test takes their place. The seam probe is a fixture crate through which tests drive core's extension points from outside core, and row 34's change to the probe's expected list of caches (`crates/trusted-server-adapter-axum/tests/seam_probe.rs`) goes too, because a build that does not link the module declares only `robots-txt` and the probe's cache. @@ -429,12 +430,11 @@ Row 34's `assets_turned_off_in_the_settings_are_fetched_for_every_request` and ` 1. **Concurrent requests.** Every test of the asset paths sends requests one after another. The loading cache module has tests for many callers loading a missing key once (`many_callers_on_many_threads_load_a_missing_key_once_in_the_caller`) and for a failed load reaching every waiter (`a_failed_load_reaches_every_waiter_and_is_not_kept`), and those load a generic value and not an asset. 2. **A real shared store.** No test runs the module against Fastly's cache, a key-value store, or two instances. The Axum, Cloudflare and Spin adapters have no test of asset holding. 3. **A deployment's build.** No test links the module the way a deployment does, by handing its builder to an adapter's `run_with` or `routes_with_registrations` with a loading cache module beside it. -4. **A `[cache.51degrees-assets-vary]` table.** No test writes one. -5. **Publisher path cases.** No test covers an HTML response to a request that is not for a document, a `Content-Length` over the limit, a lifetime from `Expires` or `Surrogate-Control`, a file offered to the module with `asks_for_origin` true, a request with no host, or a header that is not visible text. Core's recording test reports `asks_for_origin` for a range, module state and a personalized marker, and not for a conditional header, a `Cache-Control` or a `Pragma`. -6. **Asset route cases.** No test covers a route with `auth`, a method other than `GET`, or a request with no host. -7. **Streaming with a stored copy.** The route tests turn streaming off (`held_asset_stub`), and the publisher tests leave it off except in the oversized body test, which stores nothing. A stored copy served on an adapter that streams has no test. -8. **Two publishers in one cache.** The key test of the source cache and the loading cache module's `two_publishers_never_share_an_entry` cover the prefix, and no test serves two publishers through either asset path. -9. **Reordered page checks.** `origin_response_may_be_kept` moves the status check ahead of the cookie and shared cache header checks in `template_cache_ttl`. A page response with a status other than `200` and a fault found by a later check is now reported as a status other than `200`, and no test was added for the changed order. +4. **Publisher path cases.** No test covers an HTML response to a request that is not for a document, a `Content-Length` over the limit, a lifetime from `Expires` or `Surrogate-Control`, a file offered to the module with `asks_for_origin` true, a request with no host, or a header that is not visible text. Core's recording test reports `asks_for_origin` for a range, module state and a personalized marker, and not for a conditional header, a `Cache-Control` or a `Pragma`. +5. **Asset route cases.** No test covers a route with `auth`, a method other than `GET`, or a request with no host. +6. **Streaming with a stored copy.** The route tests turn streaming off (`held_asset_stub`), and the publisher tests leave it off except in the oversized body test, which stores nothing. A stored copy served on an adapter that streams has no test. +7. **Two publishers in one cache.** The key test of the source cache and the loading cache module's `two_publishers_never_share_an_entry` cover the prefix, and no test serves two publishers through either asset path. +8. **Reordered page checks.** `origin_response_may_be_kept` moves the status check ahead of the cookie and shared cache header checks in `template_cache_ttl`. A page response with a status other than `200` and a fault found by a later check is now reported as a status other than `200`, and no test was added for the changed order. ## Revision record @@ -451,6 +451,7 @@ Row 34's `assets_turned_off_in_the_settings_are_fetched_for_every_request` and ` | 2026-10-27 | The account of the Touchbase of 2026-10-23 says what 51Degrees said there and what 51Degrees decided after it, and the integrations crate design is named by its file, `2026-10-27-integrations-crate-design.md`. | | 2026-10-27 | Fixes from review round 4. The pull requests and the issue are linked where first named, and the CI record is linked at its commit. "Stock build", Viceroy and the seam probe are defined where first used. Fastly's Core Cache API is named as #894 names it, the cache names follow the form row 32 asks for, and on Spin the entries that a new key or a lifetime leaves behind stay in the store. The decisions of 51Degrees are placed after the Touchbase. | | 2026-10-27 | The header says that row 34 now holds the module, where it said the rewrite was ahead of the code. | +| 2026-10-27 | A lifetime set in `[cache.51degrees-assets-vary]` is refused when the settings load and by the deploy check, a fix found in review. The caches it declares, failure handling, the rollout of a document written for row 34, the module's tests and what no test covers say so. | ## Review log From c0750810f0c1d80229efee5953862e82020616ab Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Wed, 28 Oct 2026 11:33:48 +0000 Subject: [PATCH 192/192] The designs take the review rounds of 27 and 28 October A review of this update's changes on 27 October found nothing blocking. Its round is logged in the permission model, config by host, asset cache, loading cache and integrations crate designs. The config by host design says that an instance already holding a SHA-256's rules reads no policy entry for it, where it said such an instance reads nothing. A round on 28 October read again the four designs whose last logged round had found a blocking issue, being the consent decoders, device signals, publisher object storage origin and asset routes designs, against the code of their rows, and two lighter rounds then read the fixes, the last finding nothing blocking. The issues they found in these designs are fixed here as fixes found in review, with revision rows and the three rounds' log entries in each design. The consent decoders design's gate paragraph no longer says what an EU signal that only GPP carries does. The device signals design says its line numbers are those of the commit James Rogers approved, and names row 52 among the later rows. The publisher object storage origin design says what its row does with a URL in a response header and what the next row adds. The asset routes design says a route keeps copies only where a deployment links and selects the 51Degrees asset cache module and the 51Degrees loading cache module, and names EdgeZero v0.0.10. --- .../specs/2026-07-30-permission-model-design.md | 1 + .../specs/2026-10-12-config-by-host-design.md | 3 ++- .../specs/2026-10-12-device-signals-design.md | 16 +++++++++++----- .../specs/2026-10-12-loading-cache-design.md | 1 + ...13-publisher-object-storage-origin-design.md | 16 ++++++++++++---- .../specs/2026-10-14-consent-decoders-design.md | 13 +++++++++---- .../specs/2026-10-15-asset-cache-design.md | 1 + ...0-17-asset-routes-on-every-adapter-design.md | 17 ++++++++++++----- .../2026-10-27-integrations-crate-design.md | 1 + 9 files changed, 50 insertions(+), 19 deletions(-) diff --git a/docs/superpowers/specs/2026-07-30-permission-model-design.md b/docs/superpowers/specs/2026-07-30-permission-model-design.md index 6d2f6c2a29..b8d3bc0b24 100644 --- a/docs/superpowers/specs/2026-07-30-permission-model-design.md +++ b/docs/superpowers/specs/2026-07-30-permission-model-design.md @@ -2041,3 +2041,4 @@ include the changes that followed the first revision of this section on 3. **Round 3.** A review in a fresh session, against `split/3-permissions`. Issues found, six of them blocking. The six were that §3.1 rested on code that only later rows of the stack build and never said which row builds that code, that a Cloudflare Worker variable holds at most 5 KB while the repository's sample alone is 24,231 bytes, that nothing said what a location answers while a push is still reaching it, that the statuses a domain answers when it cannot start were partly untrue of today's code and left out Spin's fixed `503` and the `421` that Fastly answers from row 11 for a host with no blob, that fourteen places still called the repository's sample a policy that ships, and that the push could not be built from the text on any adapter. This revision says what row 4 builds by itself and what each later row adds once it is merged forward, has Cloudflare read its blob from its config store, a Workers KV namespace, in place of the variable, and says that each startup router answers with the status of the error's kind, so a domain without a valid policy file answers `500` on every adapter once row 4 moves Spin off its fixed `503`, while a host with no blob keeps its `421`. It also has the push, from row 11, write every policy entry before the blobs that name it and then wait, says that a location which still sees a blob before its policy entry answers `500`, sets out how `ts` adds `--permissions` to EdgeZero's commands, how the SHA-256 is taken and what the push refuses, and calls the sample the repository's sample throughout. 4. **Round 4.** A review in a fresh session, against `split/3-permissions`, of the round 3 fixes and the new text. Issues found, two of them blocking. The two were that the design cited row heads by commits that no published branch held, and that `ts config validate` takes no `--adapter`, so validation could not follow a rule under which the push adds the policy's text on every adapter but Axum. This revision cites commits that the published branches hold, and says that the configuration view never shows a policy's text, so a pattern under `permissions.text` is refused wherever patterns are checked and every check counts `permissions.sha256` as present, that `ts config validate` and `ts config diff` run the policy checks only when `--permissions` is given, with validate otherwise saying that the policy was not checked, and that `ts config migrate` and `ts audit generate` never run those checks. It also says how `ts` hands the settings to EdgeZero, which binding a Worker reads its blob from, what a Worker missing that binding or its blob answers, and how the tests of the `gpc`, `gpp`, `tcf` and `us-privacy` crates read the sample once `PermissionMaps::standard` is removed. 5. **Round 5.** A review in a fresh session, against `split/3-permissions`, with what rows 9, 11, 13, 51 and 55 add read on their own branches. Approved, none blocking. The round checked every file, function, line number and test that §3.1, §3.6, §3.7, §6 and §8 cite, against the code of those rows, EdgeZero `v0.0.10`, worker-build `v0.8.5` and Tech Lab main, and found each as the design states it, the order in which the runtime loader asks its source for a policy file before it looks for a parsed copy included. This revision says that the seam probe's script, which only test builds link, is carried whole as well and never reaches a release, names the adapter's test beside the Cloudflare manifests rather than the README's, and names the build-support file that the template build digest hashes with the build script. Before this round the text was brought into line with the code, so the wait between a new policy entry and its blob comes with row 13's live push, where rounds 3 and 4 read it at row 11. +6. **Round 6, 2026-10-27.** A review in a fresh session, against `split/3-permissions`, of the fix found in review that has a load ask for the rules held for a pinned digest before it reads a source, with the keyed push read on `split/11-config-by-host`. None blocking. The round checked §3.7 and item 6 of §3.1 against `load_domain_policy` and `held_policy`, every exit path of the loader, the line numbers in `permissions.rs` and `config_payload.rs`, and the tests that count a source's reads, and found each as the design states it. It noted that on Fastly a second load in one instance happens only when the opt-in reusable sandbox serves more than one request from it. diff --git a/docs/superpowers/specs/2026-10-12-config-by-host-design.md b/docs/superpowers/specs/2026-10-12-config-by-host-design.md index c7900ec785..3a2146dbe2 100644 --- a/docs/superpowers/specs/2026-10-12-config-by-host-design.md +++ b/docs/superpowers/specs/2026-10-12-config-by-host-design.md @@ -112,7 +112,7 @@ The service has one of each of the following, and nothing divides any of them be - **The secret store.** Each publisher's blob is meant to name keys of its own in the secret store, and nothing checks that the keys differ between blobs, because each application is built from one blob and sees no other. Two blobs that name one secret read the same value. Where an Edge Cookie ID is made from a passphrase, its first 64 characters are a keyed hash (HMAC-SHA256) of the client IP address under that passphrase (`crates/trusted-server-core/src/ec/generation.rs:104-115`), so two blobs that name one passphrase give one client IP address the same prefix at both publishers, and a person can be linked across them. The harness's blobs name the same secrets (`scripts/config-by-host-local-test.sh:185-221`). Two blobs that name one Edge Cookie identity store (`ec_store`) share one identity graph. - **The request signing keys and the template cache's purge key.** The signing keys sit in two stores whose names are fixed, `jwks_store` and `signing_keys` (`crates/trusted-server-core/src/request_signing/mod.rs:40-54`). Every template in the template cache carries the cache tag (surrogate key) `ts-template` (`crates/trusted-server-core/src/platform/template_cache.rs:46`), which one purge call flushes (`crates/trusted-server-adapter-fastly/src/template_cache.rs:288-291`). Three routes act on them, `/_ts/admin/keys/rotate`, `/_ts/admin/keys/deactivate` and `/_ts/admin/cache/purge` with a scope of `all` (`app.rs:1415-1429`). Each is checked against the administration user and password in the blob of the host the request names (`app.rs:1624`), so one publisher's user and password can rotate the keys every publisher signs with and flush the cached templates of every publisher. Row 13 (`split/13-removals`) removes the administration user and password, and these three routes go with them (`2026-10-12-administration-off-the-publisher-domain-design.md`). -- **Each policy file.** Row 4 of the stack (IABTechLab/trusted-server#1345) selects the permission policy when a service is deployed (`2026-07-30-permission-model-design.md` §3.1), and this row builds on it. Each host's blob names its policy file in `[permissions] policy`, with the file's SHA-256, and carries no copy of the file. The keyed push keeps each file `--permissions` supplies once, as a policy entry under `permissions_` in the config store, which is a pointer and chunks when the entry is longer than the 8,000 characters one entry holds. It writes each entry the store does not already hold before the host's blob, and keeps one that holds the same file (`run` and `holds_policy` in `crates/trusted-server-cli/src/commands/config/keyed_push.rs`). Core forms the key with `policy_key` (`crates/trusted-server-core/src/platform/policy_source.rs`). For every blob but the one at the logical store ID, the adapter reads the policy from that entry (`blob_carries_its_policy` in `app.rs`, and `FastlyPolicyEntries` in `platform.rs`), unless the instance already holds the rules of that SHA-256, when it reads nothing (§3.7 of the permission model design). The blob at the logical store ID carries its file, as on a service with one blob. The tests `two_hosts_naming_one_policy_file_keep_it_once` (`crates/trusted-server-cli/tests/config_push_by_key.rs`) and `a_host_blob_whose_policy_entry_is_missing_is_refused_naming_the_entry` (`platform.rs`) show one entry kept for two hosts and a missing entry refused. Hosts whose blobs pin one SHA-256 share that entry and run the same policy, and a file edited between two pushes is two digests and two entries. A host fails to load, as a blob that cannot be loaded does (the answers above), and answers `500`, when its blob names no policy, or when the instance does not already hold the rules of that SHA-256 and its policy entry is absent, cannot be read, does not match its digest or fails the checks of §3.3 of the permission model design. A config store that cannot be read answers `500`, as the selection does when it cannot look a host up (§3.1 of the permission model design, item 7). The other hosts are answered as before. This row's keyed writer writes only the local server's store, and row 13 gives it the live store. `ts config diff` cannot reach a host's blob, because EdgeZero `v0.0.10` refuses every Fastly key but the logical store ID for a diff as for a push (`resolve_config_store_and_key` in its `config.rs`), so a host's pinned policy is checked through `/_ts/config` on that host. +- **Each policy file.** Row 4 of the stack (IABTechLab/trusted-server#1345) selects the permission policy when a service is deployed (`2026-07-30-permission-model-design.md` §3.1), and this row builds on it. Each host's blob names its policy file in `[permissions] policy`, with the file's SHA-256, and carries no copy of the file. The keyed push keeps each file `--permissions` supplies once, as a policy entry under `permissions_` in the config store, which is a pointer and chunks when the entry is longer than the 8,000 characters one entry holds. It writes each entry the store does not already hold before the host's blob, and keeps one that holds the same file (`run` and `holds_policy` in `crates/trusted-server-cli/src/commands/config/keyed_push.rs`). Core forms the key with `policy_key` (`crates/trusted-server-core/src/platform/policy_source.rs`). For every blob but the one at the logical store ID, the adapter reads the policy from that entry (`blob_carries_its_policy` in `app.rs`, and `FastlyPolicyEntries` in `platform.rs`), unless the instance already holds the rules of that SHA-256, when it reads no policy entry (§3.7 of the permission model design). The blob at the logical store ID carries its file, as on a service with one blob. The tests `two_hosts_naming_one_policy_file_keep_it_once` (`crates/trusted-server-cli/tests/config_push_by_key.rs`) and `a_host_blob_whose_policy_entry_is_missing_is_refused_naming_the_entry` (`platform.rs`) show one entry kept for two hosts and a missing entry refused. Hosts whose blobs pin one SHA-256 share that entry and run the same policy, and a file edited between two pushes is two digests and two entries. A host fails to load, as a blob that cannot be loaded does (the answers above), and answers `500`, when its blob names no policy, or when the instance does not already hold the rules of that SHA-256 and its policy entry is absent, cannot be read, does not match its digest or fails the checks of §3.3 of the permission model design. A config store that cannot be read answers `500`, as the selection does when it cannot look a host up (§3.1 of the permission model design, item 7). The other hosts are answered as before. This row's keyed writer writes only the local server's store, and row 13 gives it the live store. `ts config diff` cannot reach a host's blob, because EdgeZero `v0.0.10` refuses every Fastly key but the logical store ID for a diff as for a push (`resolve_config_store_and_key` in its `config.rs`), so a host's pinned policy is checked through `/_ts/config` on that host. - **The rate counter,** `counter_store` (`rate_limiter.rs:9`), which limits Edge Cookie sync requests. - **The sandbox's memory bound and counters.** Where a sandbox serves several requests, every application it keeps shares the bound, and the counters it reports count the requests and builds of every publisher it serves. @@ -191,3 +191,4 @@ The behaviors below are stated in the sections above. A change to the code would 3. **Round 3.** A review in a fresh session, against `split/11-config-by-host`. Issues found, one of them blocking. The design called both EdgeZero `683202c` and `v0.0.10` the version the stack pins, so its account of a staged deploy, which holds only at `683202c`, read as true of the stack. This revision ties `683202c` to the version this row's head pins, says that the stack now pins `v0.0.10`, and points staging under `v0.0.10` at section 2.5 of the Fastly per-host selector design. It also says that the service holds one policy entry for each SHA-256 that the hosts' blobs pin, which hosts share only when they pin the same SHA-256, that `ts config diff` cannot reach a host's blob, so a host's pinned policy is checked through `/_ts/config` on that host, and asks under Open questions, item 9, whether a selector that forms a policy entry's key should be refused. 4. **Round 4.** A review in a fresh session, against `split/11-config-by-host`, of the round 3 fixes and the new text. Issues found, one of them blocking. The one was that the design cited row 11's head by a commit that no published branch held. This revision cites a commit that the published branch holds, and says that a config store that cannot be read answers `500`, as the selection does when it cannot look a host up, where it said that the selection's failure is a configuration error. 5. **Round 5.** A review in a fresh session, against `split/11-config-by-host`. No issues found. The round found the keyed writer keeping each policy file once under its digest and writing it before the host's blob, the Fastly adapter reading a host's policy from that entry, the two tests the design names, the key function in core, and the selector and host rules that Open questions, item 9, describes, each as the design states them. +6. **Round 6, 2026-10-27.** A review in a fresh session, against `split/11-config-by-host`, of the dates of this revision and of the keyed push tests that now push a policy only each test loads. Issues found, none blocking. Since IABTechLab/trusted-server#1345 an instance that already holds the rules of a digest reads no policy entry for it, so the account of each policy file, and of the entry that makes a host fail to load, holds for an instance that meets the digest for the first time. This revision says so in that account. diff --git a/docs/superpowers/specs/2026-10-12-device-signals-design.md b/docs/superpowers/specs/2026-10-12-device-signals-design.md index 52f146ec05..68ac8d4a09 100644 --- a/docs/superpowers/specs/2026-10-12-device-signals-design.md +++ b/docs/superpowers/specs/2026-10-12-device-signals-design.md @@ -3,9 +3,9 @@ **Status:** Approved on 2026-10-14, having been proposed on 2026-10-12. James Rogers approved the change at `b680d0780` on the fork's pull request [jwrosewell/trusted-server#23](https://github.com/jwrosewell/trusted-server/pull/23), which is now closed. The pull request [IABTechLab/trusted-server#1369](https://github.com/IABTechLab/trusted-server/pull/1369), row 28 of the stack, carries `b680d0780` with the row before it merged in, is open for Tech Lab review and answers [IABTechLab/trusted-server#1286](https://github.com/IABTechLab/trusted-server/issues/1286). The earlier pull request for the same commits, [IABTechLab/trusted-server#1316](https://github.com/IABTechLab/trusted-server/pull/1316), is closed. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-08-27-integration-provider-seam-design.md`, `2026-07-30-pluggable-providers-design.md`, `2026-10-12-module-context-design.md` and `2026-07-30-permission-model-design.md` -**Related PRs:** [IABTechLab/trusted-server#1369](https://github.com/IABTechLab/trusted-server/pull/1369) builds on the prefix module in [IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), and the next row is [IABTechLab/trusted-server#1370](https://github.com/IABTechLab/trusted-server/pull/1370). The module context that the change relies on is in [IABTechLab/trusted-server#1349](https://github.com/IABTechLab/trusted-server/pull/1349), and the permission model is in [IABTechLab/trusted-server#1345](https://github.com/IABTechLab/trusted-server/pull/1345). Three later rows change what this document describes, being row 43 in [IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384), which renames the seam probe crate to `iabtl`, row 45 in [IABTechLab/trusted-server#1388](https://github.com/IABTechLab/trusted-server/pull/1388), which moves the OpenRTB driver out of core, and row 49 in [IABTechLab/trusted-server#1399](https://github.com/IABTechLab/trusted-server/pull/1399), which renames the `fastly` value of `[device] module` to `platform` +**Related PRs:** [IABTechLab/trusted-server#1369](https://github.com/IABTechLab/trusted-server/pull/1369) builds on the prefix module in [IABTechLab/trusted-server#1368](https://github.com/IABTechLab/trusted-server/pull/1368), and the next row is [IABTechLab/trusted-server#1370](https://github.com/IABTechLab/trusted-server/pull/1370). The module context that the change relies on is in [IABTechLab/trusted-server#1349](https://github.com/IABTechLab/trusted-server/pull/1349), and the permission model is in [IABTechLab/trusted-server#1345](https://github.com/IABTechLab/trusted-server/pull/1345). Four later rows change what this document describes, being row 43 in [IABTechLab/trusted-server#1384](https://github.com/IABTechLab/trusted-server/pull/1384), which renames the seam probe crate to `iabtl`, row 45 in [IABTechLab/trusted-server#1388](https://github.com/IABTechLab/trusted-server/pull/1388), which moves the OpenRTB driver out of core, row 49 in [IABTechLab/trusted-server#1399](https://github.com/IABTechLab/trusted-server/pull/1399), which renames the `fastly` value of `[device] module` to `platform`, and row 52 in [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410), where the Axum, Cloudflare and Spin adapters call `detect` too **Issues:** [IABTechLab/trusted-server#1286](https://github.com/IABTechLab/trusted-server/issues/1286). [IABTechLab/trusted-server#777](https://github.com/IABTechLab/trusted-server/issues/777) lists the stack -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-28 > **Why this spec exists.** A device module may now give two answers beyond the coarse browser-or-bot signals, being how confident it is that a person is looking at the page and what the device is. This document records what core does with each answer at the head of IABTechLab/trusted-server#1369, what the change leaves out, and the questions the code leaves open. @@ -13,7 +13,7 @@ A device module answers one question, whether the request looks like a browser, and the Edge Cookie gate reads the answer. A module that reaches a detection service knows far more, including how confident it is that a person is looking at the page and what the device is, and a bidder prices inventory on the device it is told about. Core has no place for either answer, so a module's knowledge stops at the gate and a bid request leaves without the device. -Paths and line numbers in this document are those of `b680d0780`, the head of IABTechLab/trusted-server#1369, unless a row or `4badbdaf8` is named. `4badbdaf8` ends the five commits after `b680d0780` that fix what the reviews found in the code (see the revision record). +Paths and line numbers in this document are those of `b680d0780`, the commit James Rogers approved on jwrosewell/trusted-server#23, unless a row or `4badbdaf8` is named. `4badbdaf8` ends the five commits after `b680d0780` that fix what the reviews found in the code (see the revision record). Those five commits and the merges of the rows below since then moved the code, so at the pull request's head most of the same code sits at other lines. ## 2. Design @@ -21,7 +21,7 @@ Two answers a device module may give beyond the coarse browser-or-bot signals th **The score.** `DeviceSignals::human_confidence` says how confident the module is that a person is looking at the page (`crates/trusted-server-core/src/ec/device.rs`, lines 115 and 123). `High` means high confidence that a person is viewing the page, `Medium` means the evidence is mixed and the request cannot be placed either way, `Low` means almost certainly no person is directly involved in viewing the content, and `Unknown` means not established. A module that scores the request writes the field in the signals that `detect` returns, and every other module leaves it `Unknown`, which is the default. `Unknown` is kept apart from `Low` because a request alone cannot establish a person, so a first-time visitor is never called a machine. The built-in module, the Fastly module and `DeviceSignals::unknown` all leave the score `Unknown` (lines 169 and 203). -Only the Fastly adapter calls `detect` in production, once for each request at its entry point (`crates/trusted-server-adapter-fastly/src/lib.rs`, lines 590 and 1089), so the score can only be written on Fastly. The Axum, Cloudflare and Spin adapters create no device signals at all. No production code reads the score, and no row of the stack through row 50 adds a production reader. Two tests read it, being `a_request_alone_establishes_no_person`, which reads it at line 912 of `device.rs`, and the Axum test of the seam probe, which reads it at line 971 of `crates/trusted-server-adapter-axum/tests/seam_probe.rs`. The module context, which holds what core hands a module when core calls the module, passes `&DeviceSignals` to a module that names the type, so a module could read the score, and no module in this repository does. The score is not stored in the identity graph row, the record kept for an Edge Cookie, because `to_kv_device` (line 209) copies five signals into the row, being `is_mobile`, `ja4_class`, `platform_class`, `h2_fp_hash` and `known_browser`. `looks_like_browser` is not copied either. On Fastly, `looks_like_browser` gates the Edge Cookie writes as it did (`crates/trusted-server-adapter-fastly/src/app.rs`, lines 643 to 688). The redirect middlewares that would treat `Low`, `Medium` and `High` differently are kept out of this change so that the set of changes does not grow. +Only the Fastly adapter calls `detect` in production, once for each request at its entry point (`crates/trusted-server-adapter-fastly/src/lib.rs`, lines 590 and 1089), so the score can only be written on Fastly. The Axum, Cloudflare and Spin adapters create no device signals at all. No production code reads the score, and no later row of the stack adds a production reader. Two tests read it, being `a_request_alone_establishes_no_person`, which reads it at line 912 of `device.rs`, and the Axum test of the seam probe, which reads it at line 971 of `crates/trusted-server-adapter-axum/tests/seam_probe.rs`. The module context, which holds what core hands a module when core calls the module, passes `&DeviceSignals` to a module that names the type, so a module could read the score, and no module in this repository does. The score is not stored in the identity graph row, the record kept for an Edge Cookie, because `to_kv_device` (line 209) copies five signals into the row, being `is_mobile`, `ja4_class`, `platform_class`, `h2_fp_hash` and `known_browser`. `looks_like_browser` is not copied either. On Fastly, `looks_like_browser` gates the Edge Cookie writes as it did (`crates/trusted-server-adapter-fastly/src/app.rs`, lines 643 to 688). The redirect middlewares that would treat `Low`, `Medium` and `High` differently are kept out of this change so that the set of changes does not grow. **The attributes.** `DeviceAttributes` is what a bidder is told about the device, being the OpenRTB device type, make, model, operating system and version, and screen width and height (`device.rs`, lines 41 to 65). Every field is optional because every source is partial. The device type and the two screen sizes are `i32`, a 32-bit whole number, and the other four fields are strings. `DeviceModule::advertising_attributes` gives the module's answer, and the default answers nothing, so no existing module changes (line 273). What core does with the answer, in the order it happens: @@ -33,7 +33,7 @@ Only the Fastly adapter calls `detect` in production, once for each request at i 6. **When the module cannot answer.** `advertising_attributes` returns an `Option` and has no other way to report a failure, so a module that cannot reach its service returns `None`, and core cannot tell that answer from the answer of a module with nothing to say. Core logs neither case. Each of the three handlers waits for the call, so the time a module takes is added to the time before the auction starts. At `4badbdaf8` the services in the call register backends through `CappedBackend`, which caps the first-byte and between-bytes timers of every backend the module registers during the call at the auction's own timeout (`formats.rs`, lines 588 to 591 and 621 to 659, and `RuntimeServices::with_backend`, `crates/trusted-server-core/src/platform/types.rs`, line 431). The two timers bound how long a service may take to start answering and to send each further part, not the whole call, so connecting, a slow trickle of bytes, a backend registered before the call and work done without a backend are not bounded, as they are not for an auction provider. Core has no clock that could stop a call, and no adapter enforces a total deadline on a request. The cap binds where the adapter applies a backend's timers, which today is Fastly only. The Axum adapter sends to the address and ignores backend names, and the Cloudflare and Spin adapters register no backend, so the cap changes nothing on those three. 7. **Two calls for two answers.** Core calls `detect` and `advertising_attributes` separately, each with a module call of its own, being the module context together with the permissions the module declares, and the second call carries no device signals. A module that reaches an outside service to give both answers makes a call for each request on Fastly and another for each auction request, unless the module keeps the first answer itself. -**Signals and attributes stay apart.** `DeviceAttributes` is a type of its own and not a field of `DeviceSignals`, on purpose (`device.rs`, lines 29 to 39). Five of the signals reach the identity graph row through `to_kv_device`. The attributes answer what a bidder prices, and core stores them nowhere. At `4badbdaf8` the comment on `DeviceAttributes` no longer says that the identity path serialises the signals, which derive no `Serialize`, and the comment on `DeviceInfo::attributes` no longer says that a stored request from before the field still reads, since no code reads a stored `AuctionRequest` (`device.rs`, lines 29 to 38, and `types.rs`, lines 116 to 121). +**Signals and attributes stay apart.** `DeviceAttributes` is a type of its own and not a field of `DeviceSignals`, on purpose (`device.rs`, lines 29 to 39). Five of the signals reach the identity graph row through `to_kv_device`. The attributes answer what a bidder prices, and core stores them nowhere. At `4badbdaf8` the comment on `DeviceAttributes` no longer says that the identity path serializes the signals, which derive no `Serialize`, and the comment on `DeviceInfo::attributes` no longer says that a stored request from before the field still reads, since no code reads a stored `AuctionRequest` (`device.rs`, lines 29 to 38, and `types.rs`, lines 116 to 121). ## 3. Consent and privacy @@ -110,8 +110,14 @@ These are for the reviewer. This revision records what the code does and decides | 2026-10-16 | Fixes from review round 1. The header and the pull requests the header names, the removed type name mapping, the three places core asks, what the call carries, the consent and privacy section, what changes for a deployment and for code, the adapters, the acceptance without the process claims, the untested paths, the open questions and the review log. | | 2026-10-16 | Fixes from review round 2. What the integration guide describes, what the changelog entry names and leaves out, and the statements that no code reads the score, which now say no production code does. The pull request facts in the header and the acceptance were read against the pull requests and stand. | | 2026-10-16 | Code fixes in the five commits ending at `4badbdaf8`, and open questions 1 to 5 and the guide half of question 7 moved into the body. The call carries the permissions resolved for the request, a backend the module registers in the call has its timers capped at the auction's timeout, core drops device values a bidder cannot read, tests run the three handlers with consent allowed and denied, and the two comments say only what the code does. | +| 2026-10-28 | Fixes from review round 3. The commit the line numbers are read at, the later rows with row 52 among them, and the spelling of serializes. | +| 2026-10-28 | Fixes from review round 4. The commits that moved the lines this page cites. | +| 2026-10-28 | Fixes from review round 5. Most of the cited code, not all of it, sits at other lines at the head. | ## Review log 1. **Round 1, 2026-10-16.** A review in a fresh session, against `split/28-device-signals` at `b680d0780`. Issues found, five of them blocking. The blocking issues were that the spec described a device type name mapping that the code no longer has, that the spec named two places where core asks the module when the code has three, that the spec's acceptance section claimed a proof by the Axum test and a per-commit record that neither pull request holds, that the spec said the module is handed the call that `detect` is handed and left out that the module sees every header and that the attributes go under no permission check, and that the spec's header pointed at closed pull requests and said Proposed. This revision states what the code does for each, adds the consent and privacy, deployment, adapter and untested-path sections, and leaves seven questions in section 7 for code changes, being the missing test of the three points, the unchecked values, the missing time limit, what the call carries, two comments, where the score belongs, and the guide and the changelog. 2. **Round 2, 2026-10-16.** A lighter review in a fresh session, checking the round 1 fixes against `split/28-device-signals` at `b680d0780`. Issues found, two of them blocking. This revision says that the integration guide describes only the attributes, that the changelog names the score and `DeviceModule::advertising_attributes` but not `DeviceInfo::attributes` and gives no breaking-change note, and that no production code reads the score although two tests do, and it confirms the pull request facts in the header and the acceptance. +3. **Round 3, 2026-10-28.** A review in a fresh session, against `split/28-device-signals`, of the round 2 fixes, the five commits ending at `4badbdaf8` and the whole design against the code. One blocking issue and two smaller ones. The page called `b680d0780` the head of IABTechLab/trusted-server#1369 and gave its line numbers there, when the head is now a later merge, so outside `device.rs` and `formats.rs` most of the lines it names are elsewhere at the head. The smaller ones were the spelling of serializes, and the later rows, which left out row 52 in IABTechLab/trusted-server#1410, where the Axum, Cloudflare and Spin adapters call `detect`, and counted the rows without a reader of the score only to row 50. This revision says that the line numbers are those of `b680d0780`, the commit James Rogers approved, or of `4badbdaf8` where it is named, adds row 52 to the later rows, says that no later row adds a production reader of the score, and spells serializes. +4. **Round 4, 2026-10-28.** A lighter review in a fresh session, against `split/28-device-signals`, of the round 3 fixes. No blocking issue and one smaller one. The commit James Rogers approved, row 52 in IABTechLab/trusted-server#1410, where the Axum, Cloudflare and Spin adapters call `detect`, the absence of a production reader of the score in every later row, and the spelling were found correct. The smaller one was that the page put the move of the lines it cites down to the merges alone, when in `device.rs` and `formats.rs` the five commits that fix what the reviews found moved them. This revision says that those five commits and the merges moved the code. +5. **Round 5, 2026-10-28.** A lighter review in a fresh session, against `split/28-device-signals`, of the round 4 fixes. No blocking issue and one smaller one. The page now says that the five commits and the merges of the rows below moved the code it cites, which was found correct against the approved commit, the end of the five commits and the head of IABTechLab/trusted-server#1369, the five commits having moved the lines it gives in `device.rs` and `formats.rs` and the merges most of the rest. The smaller one was that the page says all of that code sits at other lines at the head of the pull request, when `endpoints.rs` is unchanged before the auction endpoint's call to `attach_device_attributes`, so the two early returns and that call start at the lines the page gives. This revision says that most of the code sits at other lines. diff --git a/docs/superpowers/specs/2026-10-12-loading-cache-design.md b/docs/superpowers/specs/2026-10-12-loading-cache-design.md index ddcbbde080..9bdd0a27ff 100644 --- a/docs/superpowers/specs/2026-10-12-loading-cache-design.md +++ b/docs/superpowers/specs/2026-10-12-loading-cache-design.md @@ -242,3 +242,4 @@ These are for the reviewer. This revision records what the code does, and what t 3. **Round 3.** A review in a fresh session, against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. Issues found, one of them blocking. The change that takes the 51Degrees module out of the stock lists named only rows 24 and 43, when row 32's probe test counts the module as linked and each adapter's order test names it, and the change's guard test read only Axum's list, when five stock lists, the `ts` tool's among them, could each link the module again. This revision takes row 32 into the change with those tests, puts a guard test beside every stock list, with a test that pins the tool's list, and spells out what a stock build then costs, being a call to the cloud service on each page the client hints middleware is placed for and a key-value write on each robots.txt request where a deployment adds a robots.txt module. It also lists the other text that says a stock build holds answers, asks under Open questions whether loading through `NoLoadingCache` with nothing logged should be logged or refused, and gives the design's line numbers at the current heads of rows 24 and 31. 4. **Round 4.** A review in a fresh session, checking the round 3 fixes against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. Issues found, one of them blocking. The design said that once the 51Degrees module leaves the stock lists, the first case of row 32's probe test still shows the table selecting the module it names, when that case's build would then link only the probe's cache, which a table naming no module takes anyway. This revision has that case link a second loading cache module, a test builder offering core's `MemoryLoadingCache`, so that the case still shows that the table is read, adds the integration guide, the 51Degrees client's comments and the client hints middleware's tests to the text the change rewrites, defines stock build where it is first used, and links the source of `fiftyone-caching` at the commit that `4.5.8` was published from. 5. **Round 5.** A review in a fresh session, checking the round 4 fixes against `split/31-caching-release`, with the design's code read on `split/24-loading-cache` and the rows above it. None blocking. Row 32's probe test now links a second loading cache module in its first case, a test builder offering core's `MemoryLoadingCache`, so the case shows that the table is read, and the guard test sits beside every stock list, the `ts` tool's included, with the tool's list pinned in order. The search of section 9 prints nothing on rows 24, 31, 32 and 43 and six lines at the top of the stack, the manifests, the comment in core, the guides, the template, the changelog and the 51Degrees crates say that a stock build links no loading cache module, and the line numbers and tests the design names were read at the commits it cites. +6. **Round 6, 2026-10-27.** A review in a fresh session, against `split/31-caching-release`, with `split/24-loading-cache` and the rows above it, of the move to the release `4.5.13`. No issues found. The round compared the published `4.5.8`, `4.5.8-49` and `4.5.13` file by file and found the 21 files under `src/`, the 16 under `tests/` and `README.md` identical, matched the checksum in `Cargo.lock` with crates.io's, found `4.5.9` to `4.5.12` yanked and the crate's files identical to those of the source commit this design links, and found that from row 31 up `Cargo.toml` and `Cargo.lock` change only in the crate's line, version and checksum. diff --git a/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md b/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md index 0f4e8c09c3..6bf30e2a4a 100644 --- a/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md +++ b/docs/superpowers/specs/2026-10-13-publisher-object-storage-origin-design.md @@ -1,11 +1,11 @@ # Design Spec: A publisher's site served from a folder of object storage -**Status:** Proposed, 2026-10-13. Raised as pull request IABTechLab/trusted-server#1377 (from jwrosewell/trusted-server#31, which is closed), for Tech Lab review. +**Status:** Approved on 2026-10-14, having been proposed on 2026-10-13. James Rogers approved the change at `f5f8ac2d0` on the fork's pull request jwrosewell/trusted-server#31, which is closed. Raised as pull request IABTechLab/trusted-server#1377, for Tech Lab review. **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-12-page-middleware-design.md`, `2026-10-12-middleware-creates-a-missing-file-design.md`, `2026-04-28-multi-backend-asset-proxy-design.md` and `2026-10-14-origin-signing-design.md`, with every design document of the stack held in IABTechLab/trusted-server#1342 **Related PRs:** IABTechLab/trusted-server#1377, jwrosewell/trusted-server#31 (closed) **Issues:** IABTechLab/trusted-server#1329 -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-28 ## 1. Problem @@ -41,9 +41,9 @@ secret_access_key = "publisher_origin_secret_access_key" The page that comes back is a publisher page like any other, so it is fetched, changed by the middleware named for it, stored and served as one from a web server is. -**The origin path.** A path in `origin_url` is kept, and the request path is added after it with exactly one `/` between them, whichever side carries one. The request's query is kept unless `origin_auth` sets `origin_query = "strip"` (see "The query"). An `origin_url` with no path sends the request path as it arrived, apart from the rebuilding of a signed address (see "Signing"). A query in `origin_url` is ignored, as it was. The path is no setting, because `origin_url` is required, and before this change the path was discarded. A deployment whose existing document has a path in `origin_url` therefore sends different requests to its origin, whether or not the document sets the two settings, because each request now goes to the folder the path names and a request path with a `.` or `..` segment is answered `400`. The path is a breaking change for such a deployment. +**The origin path.** A path in `origin_url` is kept, and the request path is added after it with exactly one `/` between them, whichever side carries one. The request's query is kept unless `origin_auth` sets `origin_query = "strip"` (see "The query"). An `origin_url` with no path sends the request path as it arrived, apart from `directory_index` and the rebuilding of a signed address (see "Signing"). A query in `origin_url` is ignored, as it was. The path is no setting, because `origin_url` is required, and before this change the path was discarded. A deployment whose existing document has a path in `origin_url` therefore sends different requests to its origin, whether or not the document sets the two settings, because each request now goes to the folder the path names and a request path with a `.` or `..` segment is answered `400`. The path is a breaking change for such a deployment. -The publisher proxy builds the origin address in one place, `handle_publisher_request` in `publisher.rs`, and that address now comes from `publisher_origin_path_and_query`. The streamed page, the buffered page, the template cache key, a `HEAD` and every other method take the address built there. The publisher path follows no redirect. A URL in the folder in a response header, a redirect's `Location` among them, reaches the reader without the folder, and one outside the folder is left as the origin wrote it, as `2026-10-13-origin-redirect-design.md` sets out. +The publisher proxy builds the origin address in one place, `handle_publisher_request` in `publisher.rs`, and that address now comes from `publisher_origin_path_and_query`. The streamed page, the buffered page, the template cache key, a `HEAD` and every other method take the address built there. The publisher path follows no redirect. The response header rewrite maps a URL on the origin's host in `Location`, `Content-Location`, `Refresh` and `Link` to the reader's host with its path unchanged, so at this row a URL in the folder reaches the reader with the folder still in its path, and a URL outside the folder is mapped as well. The next row, IABTechLab/trusted-server#1378, maps only a URL in the folder and takes the folder off it, as `2026-10-13-origin-redirect-design.md` sets out. **Signing.** `origin_auth` is the table an asset route's `auth` takes, read as the same type, with the same fields, defaults and secret store key references, and it signs through the same function, `apply_asset_origin_auth`. There is one signing path. The request is signed last, once its address and `Host` are final. Signing rebuilds the address through the URL parser, `url::Url`, signs the rebuilt address and sends the request to it, so the two cannot differ. The parser writes the address in its own form. It percent-encodes an apostrophe or a non-ASCII character in the query, and a double quote, a curly brace or a non-ASCII character in the path, all of which `http::Uri` accepts as they are, and it resolves dot segments in the path, which are refused with `400` first when `origin_url` has a path. The table still accepts the deprecated `secret_store` field that an asset route's `auth` accepts. The field is ignored with a logged warning and is never written back. @@ -100,6 +100,7 @@ The build is deployed before a document that sets `origin_auth` or `directory_in - **Validators and ranges on a signed origin.** A browser revalidating a file through a signed publisher origin is sent the file again, and a range request is answered with the whole file. An asset route forwards both, and stays the place for large media. - **A missing object read as a missing file.** A file the origin does not have is made when the origin answers `404`. Some object storage answers a missing object with `403`, and with `404` only where the credentials may also list the bucket. No storage service was called in testing this change, so a deployment that wants a file made checks what its storage answers for an object it does not have. +- **Addresses in a response header.** At this row a redirect within the folder whose `Location` names the storage host or is a path from the root sends the reader to an address that still carries the folder. The next row maps most such addresses, and section 4 of `2026-10-13-origin-redirect-design.md` names the two it does not correct, an index file redirected back to its folder and a `Location` that names no host. - **Addresses in a page that name the storage host.** A site that links to itself by relative or root-relative addresses is unaffected. An absolute address naming the storage host is rewritten to the reader's host as before. In a style sheet or a script the folder is removed with it, because the whole of `origin_url` is matched there. In an HTML document only the host is matched, so the folder stays in the path, and the address then names nothing beneath the folder. - **Media types.** The page pipeline runs on what the origin says a file is, so each object is stored with its media type. - **A trailing slash on `origin_url`** is refused, as it was. The join gives one `/` whether or not the path ends in one. @@ -145,8 +146,15 @@ The tests of the publisher path are in the `object_storage_origin_tests` module | 2026-10-14 | Names its draft pull request and issue. | | 2026-10-15 | Fixes from review round 1. The path of `origin_url` and the closed path check as breaking changes with the upgrade and rollback order, `origin_query`, the vendor claim, the adapters and the `Host` that is signed, what a signed read sends and what a reader sees on a refusal, the rule for `directory_index`, the missing tests and the header. | | 2026-10-15 | Corrections from review round 2. The changelog entry, the `x-amz-` rule this series adds, the rebuilt address, the methods the `400` applies to, and the statements about S3 attributed to AWS's documentation. | +| 2026-10-19 | The sentence on a URL in a response header follows the origin redirect design. | +| 2026-10-28 | Fixes from review round 3. What this row does with a URL in a response header and what the next row adds, the approval, and `directory_index` beside an `origin_url` with no path. | +| 2026-10-28 | Fixes from review round 4. The two redirects the next row does not correct, and the revision row for 2026-10-19. | +| 2026-10-28 | Fixes from review round 5. The redirects within the folder that keep it, and the two the next row does not correct. | ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/36-object-storage` at `f5f8ac2d0`. Issues found, four of them blocking. Section 2 counted the path of `origin_url` as an optional setting that is off unless set, when it changes the requests of every deployment whose document has one, left out that the closed path check now answers `404` to a path still changing after four rounds of percent-decoding on every deployment, said the query is always kept when `origin_query = "strip"` leaves it off, and said no setting is a vendor's when AWS signing is. This revision states both changes as breaking with the upgrade and rollback order, puts `origin_query` in the design, says that this series adds the AWS signing setting where core already kept AWS signing and that a later series moves it, and adds the adapters, the `Host` that is signed, what a signed read sends, what a reader sees when a read is refused, the exact rule for `directory_index`, the tests that are missing and the pull request that holds the stack's design documents. 2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `1465a29d1`, of the round 1 fixes and the new text. One blocking issue, in the changelog, and three smaller ones. This revision moves the path in `origin_url` and the closed path change into the Changed section of the changelog as breaking entries and corrects the changelog entry's count of settings, says that this series adds the `x-amz-` header rule to core, says that signing rebuilds the address through the URL parser and that a signed origin answers a method other than `GET` or `HEAD` with `405` before the dot segment check, and attributes what the page says about S3 to AWS's documentation. +3. **Round 3, 2026-10-28.** A review in a fresh session, against `split/36-object-storage`, of the round 2 fixes and the whole design against the code. One blocking issue and four smaller ones. Section 2 said that a URL in the folder in a response header reaches the reader without the folder, and that one outside the folder is left as the origin wrote it, when at this row the response header rewrite maps every URL on the storage host to the reader's host with its path unchanged, and only the next row, IABTechLab/trusted-server#1378, maps the folder. The smaller ones were the approval on jwrosewell/trusted-server#31, which the status left out, the change to that sentence on 2026-10-19, which the revision record left out, an `origin_url` with no path, whose requests still have `directory_index` added, and the copy of this page on the branch of IABTechLab/trusted-server#1377, which called the pull request a draft and gave another sentence on redirects. This revision says what this row does with a URL in a response header and what the next row adds, records the approval and the change, and names `directory_index` beside an `origin_url` with no path. +4. **Round 4, 2026-10-28.** A lighter review in a fresh session, against `split/36-object-storage`, of the round 3 fixes. One blocking issue and one smaller one. The item added to section 3 said that the next row corrects a redirect within the folder, when `2026-10-13-origin-redirect-design.md` says the next row does not correct two such redirects, an index file redirected back to its folder and a `Location` that names no host, and the code of IABTechLab/trusted-server#1378 does the same. The smaller one was that the revision record had no row for the change of 2026-10-19. The response header sentence in section 2, the approval and `directory_index` were found correct. This revision names the two redirects the next row does not correct, and adds the revision row for 2026-10-19. +5. **Round 5, 2026-10-28.** A lighter review in a fresh session, against `split/36-object-storage`, of the round 4 fixes. No blocking issue and two smaller ones. The item in section 3 now says that the next row maps most redirects within the folder and sends the reader to section 4 of `2026-10-13-origin-redirect-design.md` for the two it does not correct, and the revision record has a row for the change of 2026-10-19, and both were found correct against the code of IABTechLab/trusted-server#1378, that design and the history of this page. The smaller ones were that the item, its revision row and the round 4 entry said the next row leaves those two redirects as they are, when that design says the next row maps an index file redirect that sets a cookie, so the reader's browser asks again each time, and that a `Location` written relative to the requested path reaches the reader without the folder at this row, because the reader's browser resolves it against the address the reader asked for. This revision says that the next row does not correct the two, and limits the item to an address written with the storage host or as a path from the root. diff --git a/docs/superpowers/specs/2026-10-14-consent-decoders-design.md b/docs/superpowers/specs/2026-10-14-consent-decoders-design.md index f83a58b352..7d3a4e4864 100644 --- a/docs/superpowers/specs/2026-10-14-consent-decoders-design.md +++ b/docs/superpowers/specs/2026-10-14-consent-decoders-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-14-source-modules-design.md`, in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** IABTechLab/trusted-server#1383, jwrosewell/trusted-server#37 **Issues:** IABTechLab/trusted-server#1335, one of the removals listed in IABTechLab/trusted-server#777 -**Last updated:** 2026-10-16 +**Last updated:** 2026-10-28 > **Why this spec exists.** Core decoded the TCF, GPP and US Privacy strings itself, in `crates/trusted-server-core/src/consent`, and the permission signal crates named for those standards read what core had decoded. A standard is a vendor's thing as much as a product is, so the decoding belongs in the crate for the standard, and core keeps only the interface every signal module fills. @@ -25,7 +25,7 @@ The permission model made how a permission becomes held the business of modules **The settings stay where they were.** `[consent]` keeps its keys. Core parses `mode`, which says whether the modules decode or leave the strings as received, and holds every other key as written for the modules to read into types of their own. The TCF, GPP and US Privacy modules act on `mode`, and the Global Privacy Control and Model Terms for Marketing modules never read it. The TCF module reads `check_expiration`, `max_consent_age_days` and `conflict_resolution`, and the US Privacy module reads `us_privacy_defaults`. A key no linked module reads is kept as written and not refused, where the typed table refused it, because core no longer knows which keys exist. The one exception is `consent_store`, which core still refuses by name, saying that the key was removed. -**The policy names schemes by the modules' names.** `signals..authoritative` in `permissions.yaml` says whether a record of that scheme answers, for any scheme, where only `signals.tcf` was parsed. `signals.us_opt_out.sources` lists the opt-out sources by the names the modules give them, where it listed three fixed names. The shipped sample keeps the same text. +**The policy names schemes by the modules' names.** `signals..authoritative` in `permissions.yaml` says whether a record of that scheme answers, for any scheme, where only `signals.tcf` was parsed. `signals.us_opt_out.sources` lists the opt-out sources by the names the modules give them, where it listed three fixed names. The repository's sample policy, `config/permissions/sample.yaml`, keeps the same text. **Defaults, and what is no longer checked.** The defaults are the ones the typed table held, now written in the module that reads each key. The TCF module defaults `check_expiration` to true, `max_consent_age_days` to 395, `conflict_resolution.mode` to `restrictive` and `freshness_threshold_days` to 30, and the US Privacy module defaults `notice_given` to true, `lspa_covered` to false and `gpc_implies_optout` to true. Four checks that were made before are gone. @@ -42,7 +42,7 @@ The running configuration page (`/_ts/config`) shows `[consent]` as `mode` and t **The identity graph stores more.** The identity graph record stores every signal a module vouched for, keyed by scheme, in the same JSON shape as before for `tcf` and `gpp`. It now also stores `us_privacy` and `gpc`, where only `tcf` and `gpp` were stored. A US Privacy string that a module constructed from a Global Privacy Control header is stored as if the visitor had sent the string, because the record does not keep where a string came from. A record written by an earlier build is read as written, and an earlier build that reads a record this build wrote ignores the added keys. -**The auction gate reads the records.** The server-side auction gate used to read core's decoded TCF record. It required TCF Purpose 1 where the place was GDPR or unknown, or where the request carried an EU signal (a TC String, or a GPP section list naming the EU section), and passed every other request. It keeps that rule, reading it from the records with nothing of any standard in core. Where the jurisdiction is GDPR or unknown, or a record says the regulation its scheme answers for applies, the auction runs only when a record that applies consents to device storage (`necessary.operations.storage`), which the TCF module records from Purpose 1 through its own map. The gate is `consent_allows_server_side_auction` in `crates/trusted-server-core/src/consent/mod.rs`, which `EcContext::server_side_auction_allowed` applies to the request, and its callers are the `/auction` handler in `crates/trusted-server-core/src/auction/endpoints.rs` and the publisher page and page bids paths in `crates/trusted-server-core/src/publisher.rs`. The gate reads neither the resolved permissions nor the policy, so a TC String consenting to Purpose 2 alone does not run the auction, an expired TC String or an EU signal that only GPP carries keeps it from running in every place, `signals.tcf.authoritative` does not change it, the `mtm` module's grant does not open it, and neither a place rule nor an opt-out closes it. It differs from the old gate in three ways. +**The auction gate reads the records.** The server-side auction gate used to read core's decoded TCF record. It required TCF Purpose 1 where the place was GDPR or unknown, or where the request carried an EU signal (a TC String, or a GPP section list naming the EU section), and passed every other request. It keeps that rule, reading it from the records with nothing of any standard in core. Where the jurisdiction is GDPR or unknown, or a record says the regulation its scheme answers for applies, the auction runs only when a record that applies consents to device storage (`necessary.operations.storage`), which the TCF module records from Purpose 1 through its own map. The gate is `consent_allows_server_side_auction` in `crates/trusted-server-core/src/consent/mod.rs`, which `EcContext::server_side_auction_allowed` applies to the request, and its callers are the `/auction` handler in `crates/trusted-server-core/src/auction/endpoints.rs` and the publisher page and page bids paths in `crates/trusted-server-core/src/publisher.rs`. The gate reads neither the resolved permissions nor the policy, so a TC String consenting to Purpose 2 alone does not run the auction, an expired TC String keeps it from running in every place, `signals.tcf.authoritative` does not change it, the `mtm` module's grant does not open it, and neither a place rule nor an opt-out closes it. It differs from the old gate in three ways. 1. **A TC String that cannot be read.** A standalone TC String, or the EU section of a GPP string, that cannot be read is no consent whatever the other string says, where the old gate used whichever of the two decoded. 2. **The module list.** With `tcf` left out of `[permission-signal] modules`, no module reads a TC String, so a GDPR or unknown place never runs the auction and a place outside GDPR runs it whatever the TC String says, where the old gate read the string whatever the list said. @@ -101,9 +101,14 @@ The running configuration page (`/_ts/config`) shows `[consent]` as `mode` and t | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The auction gate described as a change with its rollout effect and an open question, the adapters and tools, the defaults and what is no longer checked, the failure paths, the cost, the public items that go, the untested paths and the review log. | | 2026-10-15 | Fixes from review round 2. The sixth gate difference with its rollout effect and the open question, the failure paths for an EU section in a GPP string, the specs pull request cited, the expired string item, the stale core comments listed and the CHANGELOG entry corrected. | -| 2026-10-15 | Fixes to the auction gate and to unreadable strings. The gate reads the records again, which carry the permissions a signal consents to, so it asks for Purpose 1, and an expired TC String, an EU signal that only GPP carries, the policy, the Model Terms grant, place rules and opt-outs leave it as the gate it replaced. An EU section of a GPP string that cannot be read refuses every purpose TCF maps. | +| 2026-10-16 | Fixes to the auction gate and to unreadable strings. The gate reads the records again, which carry the permissions a signal consents to, so it asks for Purpose 1, and an expired TC String, an EU signal that only GPP carries, the policy, the Model Terms grant, place rules and opt-outs leave it as the gate it replaced. An EU section of a GPP string that cannot be read refuses every purpose TCF maps. | +| 2026-10-28 | Fixes from review round 3. The EU signals that only GPP carries and that keep the auction from running, the repository's sample policy, and the date of the last revision. | +| 2026-10-28 | Fixes from review round 4. The gate paragraph no longer says what an EU signal that only GPP carries does. | ## Review log 1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/42-consent-decoders` at `564788e6e`. Issues found, three of them blocking. The design said the Cloudflare adapter read core's decoded fields when no adapter's source did, wrote the move of the auction gate to the resolved permissions as a move when it changes the TCF purpose checked and which requests pass, and said core reads `mode` when core only parses it. This revision corrects the three and the claims about unknown keys and Additional Consent, adds the adapters, defaults, failure paths, rollout, cost and public items, lists the tests the change lacks, and corrects the guide sentences that said every module reads `mode` and that core decodes the InMobi cookie. 2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `f8ffb2170`, of the round 1 fixes and the new text. One blocking issue, a sixth change to the auction gate, and four smaller ones. This revision adds the sixth difference, for an EU signal that only GPP carries, to the gate list, the rollout and the open question, corrects the failure paragraph for an EU section inside a GPP string, cites the open specs pull request, completes the expired string item, lists the stale core comments as left for a later change, and corrects the CHANGELOG entry. +3. **Round 3, 2026-10-28.** A review in a fresh session, against `split/42-consent-decoders`, of the round 2 fixes, the revision that has the gate read the records again, and the whole design against the code. One blocking issue and three smaller ones. The gate paragraph said that an EU signal that only GPP carries keeps the auction from running in every place, when a GPP string whose EU section decodes and consents to Purpose 1 runs it in every place, as it ran under the old gate. The smaller ones were the sample policy called shipped, when no build carries a policy, the date of the last revision, and the copy of this page on the branch of IABTechLab/trusted-server#1383, which called the pull request a draft and put this log before the revision record. This revision names the EU signals that only GPP carries and that keep the auction from running, calls the sample the repository's sample policy and dates the last revision 2026-10-16. +4. **Round 4, 2026-10-28.** A lighter review in a fresh session, against `split/42-consent-decoders`, of the round 3 fixes. One blocking issue. The gate paragraph said that a GPP string whose EU section consents to Purpose 1 runs the auction, as it did before, which holds outside proxy mode, when in proxy mode the TCF module reads no section and reports the regulation as applying, so such a string keeps the auction from running in every place, as the failure paragraph says. The sample policy and the date of the last revision were found correct. This revision takes that clause out of the gate paragraph, which no longer says what an EU signal that only GPP carries does, and leaves that to the gate's rule, the failure paragraph and the three differences from the old gate. +5. **Round 5, 2026-10-28.** A lighter review in a fresh session, against `split/42-consent-decoders`, of the round 4 fixes. No blocking issue and one smaller one. The gate paragraph no longer says what an EU signal that only GPP carries does, so nothing in it depends on the mode, and what it still says of a TC String consenting to Purpose 2 alone and of an expired TC String holds in interpreter and proxy mode alike. The failure paragraph and the third difference from the old gate say what proxy mode does with such a signal, and were found to agree with the code. The smaller one was that the revision record and the round 4 entry said the gate paragraph names only the expired TC String among the signals that keep the auction from running, when it also names a TC String consenting to Purpose 2 alone, which keeps the auction from running as well. This revision says that the gate paragraph no longer says what an EU signal that only GPP carries does. diff --git a/docs/superpowers/specs/2026-10-15-asset-cache-design.md b/docs/superpowers/specs/2026-10-15-asset-cache-design.md index 2f56020295..b147f9a06f 100644 --- a/docs/superpowers/specs/2026-10-15-asset-cache-design.md +++ b/docs/superpowers/specs/2026-10-15-asset-cache-design.md @@ -459,3 +459,4 @@ Row 34's `assets_turned_off_in_the_settings_are_fetched_for_every_request` and ` 2. **Round 2, 2026-10-15.** A lighter review in a fresh session, checking the round 1 fixes against `split/34-asset-cache` at `e9832a39e`. Issues found, none blocking. This revision states the cookie rule, the off switch and the range check of `[cache.assets]` as the code now has them, says which entries each load writes, removes the claim that no test sends a `Cookie` header through the publisher path, which navigation tests and now the asset gate's tests do, and gives the one linked module as the default in the guide and the code comments. 3. **Round 3.** A review in a fresh session, against `split/34-asset-cache`. Issues found, three of them blocking. The three were that row 46 changes the keep check that this rewrite moves out of core, and the design never named row 46, that core's planned tests no longer covered what core reports for the cookie rule once that rule's end-to-end test moves into the module, and that the tag the variant digest hashes needed a new name in the move. This revision keeps `origin_response_may_be_kept` in core as a public function, so that row 46 changes it once for core and the module, adds core tests with a recording test module that show what core reports, and gives the variant digest the tag `ts-asset-variant-v1`, in the form of the key's `ts-asset-cache` tag, which changes every copy key, so a copy kept before simply expires. It also has core offer the module only `GET` requests and take the module only from the builder that `[asset-cache]` selects, sets out `AssetFile`, `AssetVia` and `OriginRequest`, names the caches `51degrees-assets` and `51degrees-assets-vary`, and puts the module behind the `asset-cache-51degrees` feature after row 56. 4. **Round 4.** A review in a fresh session, checking the round 3 fixes against `split/34-asset-cache`. Issues found, none blocking. This revision links each pull request and the issue where first named, and the CI record at its commit, defines stock build, Viceroy and the seam probe where first used, names Fastly's Core Cache API as IABTechLab/trusted-server#894 names it, calls the form of the cache names the form row 32 asks for, where it called that form a rule, and says that on Spin the entries a new key or a lifetime leaves behind stay in the store. +5. **Round 5, 2026-10-27.** A review in a fresh session, against `split/34-asset-cache`, of the fix found in review that refuses a lifetime in `[cache.51degrees-assets-vary]`. Issues found, none blocking. The round found `refuse_record_lifetimes` called from the module's `validate` before the selection check, so loading the settings and the deploy check refuse each lifetime whether or not `[asset-cache]` selects the module, found that nothing but the assets table's lifetimes sizes either cache, and found the three new tests passing at `split/55-scripts-in-crates`. One sentence of the table in section 4 ran into the next without a full stop. This revision adds it. diff --git a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md index 1592ed04cd..112d7158d8 100644 --- a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md +++ b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md @@ -4,20 +4,20 @@ **Author:** 51Degrees (contributed), for Tech Lab review **Related specs:** `2026-10-17-request-filters-on-every-adapter-design.md`, `2026-10-17-edge-cookie-on-every-adapter-design.md`, `2026-10-14-source-modules-design.md`, `2026-10-14-origin-signing-design.md`, `2026-10-15-asset-cache-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 **Related PRs:** [IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412). The request filters run on every adapter in [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410) and the Edge Cookie in [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411) -**Issues:** none -**Last updated:** 2026-10-27 +**Issues:** part of IABTechLab/trusted-server#777 +**Last updated:** 2026-10-28 > **Why this spec exists.** Only the Fastly adapter serves `[[proxy.asset_routes]]`. On Axum, Cloudflare and Spin a request under a route's prefix goes to the publisher's origin as an ordinary page request, so a route's own source, its signing and its held answers are never used there. This change dispatches asset routes on those three adapters as the Fastly adapter does. Two parts of it reach beyond asset routes. On Spin a source whose `host_header` names another host is now refused, the publisher's source as well as a route's, so a Spin deployment whose publisher source sets one answers every page with `502`, which is a breaking change. And the Cloudflare and Spin adapters now install a logger, because neither had one and every line they logged was dropped, the warnings this change adds included. ## 1. Problem -`dispatch_fallback` in `crates/trusted-server-adapter-fastly/src/app.rs` checks a `GET` or `HEAD` against `Settings::asset_route_index_for_path` once robots.txt, the Edge Cookie context, the request filters, tsjs and the module routes have had their turn, and before the publisher's origin. `dispatch_asset_fallback` then asks the registry for the route's source (`IntegrationRegistry::route_source`) and answers with `proxy::handle_asset_proxy_request`, which locates the asset through the source module, signs the request where the source's `auth` table names a signer, looks for the answer among the copies its caches hold before it asks the origin, and keeps the origin's answer the way the publisher path keeps an asset. The response carries its `AssetProxyCachePolicy`, which the entry point applies after the finalize headers, and the request filters' effects, and no Edge Cookie state, because an asset response skips the Edge Cookie steps. +`dispatch_fallback` in `crates/trusted-server-adapter-fastly/src/app.rs` checks a `GET` or `HEAD` against `Settings::asset_route_index_for_path` once robots.txt, the Edge Cookie context, the request filters, tsjs and the module routes have had their turn, and before the publisher's origin. `dispatch_asset_fallback` then asks the registry for the route's source (`IntegrationRegistry::route_source`) and answers with `proxy::handle_asset_proxy_request`, which locates the asset through the source module, signs the request where the source's `auth` table names a signer, offers each `GET` the Image Optimizer does not handle to the asset cache module the deployment selects before it asks the origin, and keeps no copy itself. The response carries its `AssetProxyCachePolicy`, which the entry point applies after the finalize headers, and the request filters' effects, and no Edge Cookie state, because an asset response skips the Edge Cookie steps. No code in the Axum, Cloudflare or Spin adapter calls `handle_asset_proxy_request`. `docs/guide/api-reference.md` lists the route as Fastly's alone, and the Cloudflare HTTP client's note says these requests are not routed to it. The Spin client sends the address's own host as the `Host` of every request, so a source whose `host_header` names another host reaches its origin with the address's host in place of the one it names. That holds for the publisher's `[source.origin]` as much as for a route's source, so the refusal this change adds on Spin (section 2) changes the publisher path as well as the asset routes. -Neither the Cloudflare adapter nor the Spin adapter installs a logger. Each hands every request to EdgeZero's `run_app`, whose `init_logger` does nothing on Workers or on Spin at the locked v0.0.9 (`edgezero-adapter-cloudflare/src/lib.rs` and `edgezero-adapter-spin/src/lib.rs`), so the `log` crate drops every line the two adapters write, including the warnings they log as they build their state, a startup error and the report behind each error answer. +Neither the Cloudflare adapter nor the Spin adapter installs a logger. Each hands every request to EdgeZero's `run_app`, whose `init_logger` does nothing on Workers or on Spin at the locked v0.0.10 (`edgezero-adapter-cloudflare/src/lib.rs` and `edgezero-adapter-spin/src/lib.rs`), so the `log` crate drops every line the two adapters write, including the warnings they log as they build their state, a startup error and the report behind each error answer. ## 2. Design @@ -40,6 +40,8 @@ Neither the Cloudflare adapter nor the Spin adapter installs a logger. Each hand | Cloudflare | Yes | Not sent outside the Worker's zone | Loading caches, copies in `TRUSTED_SERVER_KV` | Held whole, up to 10 MiB | Refused by the client, `502` | | Spin | Yes | Refused by the backend, `502` | Loading caches, copies in `default` | Held whole, up to 8 MiB | Refused by the client, `502` | +The held answers column is for a deployment that links and selects the 51Degrees asset cache module and the 51Degrees loading cache module. A stock build links neither and keeps no copy of its own on any adapter. + **Logging.** Each of the Cloudflare and Spin adapters now installs a small logger of its own at its entry point, before `run_app` (`init_logger` in `src/logging.rs` of each adapter), and says so through `Hooks::owns_logging`, so `run_app` does not try to install one. The Worker writes each line through the `worker` crate's console macros by level, `console.error`, `console.warn` and, for `info`, `console.log`, with `console.debug` for the two levels below, which it does not write, so `wrangler dev` prints a line, and for a deployed Worker `wrangler tail` and the dashboard's live logs show it, as Workers Logs does where the Wrangler file enables `[observability]`. The Spin component writes each line to standard error, which `spin up` prints in its console and keeps in the `.spin/logs` folder beside a local manifest, and a host that runs the component another way decides where standard error goes. A line gives the level and the module before the message. Both loggers write `info` and above, the level the Axum adapter writes by default. Each is installed once for an instance, because the entry point runs for every request and a call that finds a logger installed changes nothing. With them every adapter logs the full error of a state that cannot be built, which `2026-10-14-origin-signing-design.md` says of a startup failure, and the report behind each error answer, which on Spin names the `Host` refused. **Configuration.** No key is added or changed, and `[[proxy.asset_routes]]` now takes effect on every adapter. As it builds its state, each of the Axum, Cloudflare and Spin adapters logs a warning naming the prefix of every route that enables the Image Optimizer (`warn_once_when_an_asset_route_enables_the_image_optimizer`), and then serves the route. A flag keeps the warning to one for each instance of the adapter. Axum builds its state once for the process, so it warns once, at startup. Cloudflare and Spin build their state for every request, so there the flag is what keeps the warning from every request. A Workers instance can serve many requests ([How Workers works](https://developers.cloudflare.com/workers/reference/how-workers-works/)), and Spin can serve several requests from one instance of the component, one after another and at the same time, to a degree it does not guarantee ([HTTP trigger](https://spinframework.dev/v4/http-trigger)), so on Spin the warning appears each time Spin starts an instance, which is every request when an instance serves only one (`spin up --max-instance-reuse-count`). @@ -65,7 +67,7 @@ They warn rather than refuse the settings, for three reasons. Such a route still **Rollout.** 1. A deployment with no asset routes serves its requests as before, because `asset_route_index_for_path` answers `None` when there are none, with two exceptions. On Spin a publisher source whose `host_header` names another host is now refused (item 4), and on Cloudflare and Spin the lines the adapter logs are now written (item 6). -2. One with asset routes now has their `GET` and `HEAD` requests answered by each route's origin on Axum, Cloudflare and Spin, where the publisher's origin answered them, with the route's signing and cache rule, and with no Edge Cookie written or expired. A route reads and keeps copies for a request that carries a cookie, which the publisher path fetches for that request alone, so after the upgrade more requests read and write the key-value store, `TRUSTED_SERVER_KV` on Cloudflare and `default` on Spin. +2. One with asset routes now has their `GET` and `HEAD` requests answered by each route's origin on Axum, Cloudflare and Spin, where the publisher's origin answered them, with the route's signing and cache rule, and with no Edge Cookie written or expired. Where a deployment links and selects the 51Degrees asset cache module and the 51Degrees loading cache module, a route reads and keeps copies for a request that carries a cookie, which the publisher path fetches for that request alone, so after the upgrade more requests read and write the key-value store, `TRUSTED_SERVER_KV` on Cloudflare and `default` on Spin. A stock build keeps no copy and asks each route's origin for every answer. 3. On those adapters a route that enables the Image Optimizer answers each request it would optimize with `502` and `no-store, private`, where the publisher's origin answered it before, after a signed route has sent its origin the `HEAD`. Its SVG paths and debug requests are served unoptimized, and each adapter logs a warning naming each such route, once for each instance (Configuration). 4. On Spin a request to a source whose `host_header` names another host answers `502` and its origin is not asked, a route's source and the publisher's alike, where before the request reached the origin with the address's own host as the `Host`, and the component logs the refusal as an error naming both hosts. A Spin deployment whose publisher source sets such a `host_header` therefore answers every page with `502`, robots.txt included. The changelog gives this as a breaking change, with the check to make before upgrading. 5. A body over 10 MiB on Cloudflare or 8 MiB on Spin answers `502`. @@ -115,8 +117,13 @@ The tests above pass on each of the three adapters, and the existing tests of ea | 2026-10-17 | First draft. | | 2026-10-18 | Fixes from review round 1. The pull request this spec belongs to, Spin's refusal of a source's `host_header` and what Cloudflare does with one, the startup warning for a route that enables the Image Optimizer and why it is not a refusal, the `HEAD` a signed route sends and the SVG and debug requests that pass, the Axum client's refusal on `send_async`, the startup refusal of a source that cannot be built, the per-request failures, body limits and timeouts, where answers are held, the Edge Cookie an asset response neither writes nor expires, the tests of the cache rule, the error and the Edge Cookie, the open questions and the review log. Fixes from review round 2. The loggers the Cloudflare and Spin adapters now install and where their lines appear, how often each adapter logs the Image Optimizer warning, why such a route gets a warning where those adapters refuse an auction of more than one provider, the Spin `host_header` refusal as a breaking change in the summary, the problem and the rollout, the logged failures, the reader headers a route's origin receives, the rise in key-value traffic, the signed source whose `host_header` names the address's own host in other letter case, what the warning test reads, which Spin client code the native tests reach, the logger tests and what is not in this change. | | 2026-10-27 | Held answers follow the asset cache as a module (`2026-10-15-asset-cache-design.md`). `handle_asset_proxy_request` offers a `GET` to the module `[asset-cache]` selects and keeps no copy itself, the 51Degrees module holds its copies through `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]`, and no stock build links that module or a loading cache module. | +| 2026-10-28 | Fixes from review round 3. A route keeps copies only where a deployment links and selects the asset cache and loading cache modules, the EdgeZero version, and the issue line. | +| 2026-10-28 | Fixes from review round 4. The Image Optimizer in the problem, the 51Degrees loading cache module, and a stock build keeping no copy of its own. | ## Review log 1. **Round 1, 2026-10-18.** A review in a fresh session, against `split/54-asset-routes` at `692ce7b8b`. Issues found, four of them blocking. The four were that the spec gave no configuration decision for a route that enables the Image Optimizer, which those adapters accepted and then answered with `502` request by request with no word at startup, that Spin drops a source's `host_header`, so a route reaches its origin with the wrong `Host` and a signature made for another, that the spec said a source that cannot be built answers per request, when it stops the adapter when the state is built, and left out the body limits, the timeouts and the per-request errors, and that no test proved the finalize step, the `no-store, private` error or that no Edge Cookie is written. Fixed in `2369fc42e`, where the three adapters warn at startup about a route that enables the Image Optimizer, `f1bba5d78`, where Spin refuses a `Host` other than the address's own, `76986f0b0` for the failure handling, and `ada05a9bf`, which tests the cache rule, the error and the Edge Cookie on each adapter. The non-blocking findings fixed are the Axum client's refusal on `send_async` in `dad986582`, the Cloudflare `Content-Length` comment in `efadc7b7e`, the API reference, the Cloudflare reason and the two designs that contradicted this one in `914800f98` and `ff7ae409a`, and the consent and privacy text and this review log in `76986f0b0`, which also states what the code does for each issue, corrects the partly true claims on where answers are held, the Image Optimizer and the Axum client, names the pull request this spec belongs to, and leaves the Cloudflare `host_header` and `HEAD` length as open questions, with a parity case, the timeouts and streaming as not in this change. 2. **Round 2, 2026-10-18.** A review in a fresh session, against `split/54-asset-routes` at `76986f0b0`, of the round 1 fixes and the new text. Issues found, two of them blocking. The two were that the Cloudflare and Spin adapters install no logger, so there the Image Optimizer warning, a startup error and the reason Spin refused a request were never written, and that Spin's refusal of a `host_header` naming another host also breaks a Spin deployment whose publisher source sets one, which the summary and Rollout 1 did not say and the changelog did not mark as breaking. Fixed in `e621e75e4`, where each of the two adapters installs a logger of its own at its entry point, which also makes the origin signing design's statement that a failed startup logs the full error true on every adapter, `9801e2279`, where the changelog gives the Spin refusal a breaking entry of its own and the guides say where each adapter's lines appear, and this revision, which states the publisher path effect in the summary, the problem and Rollout 1, how often each adapter logs the warning, and why a route that enables the Image Optimizer gets a warning where Cloudflare and Spin refuse an auction of more than one provider. The non-blocking findings fixed are the Spin `502` in the API reference's publisher fallback contract in `9801e2279`, the order of the review log in the origin signing and asset cache designs in `08122e33f`, and the reader headers a route's origin receives, the rise in key-value traffic, the signed source whose `host_header` names the address's own host in other letter case or with its default port, what the warning test reads and which Spin client code the native tests reach in this revision, which leaves a test that reads the warning's text, a startup check of `host_header` on Spin and a log level setting as not in this change. +3. **Round 3, 2026-10-28.** A review in a fresh session, against `split/54-asset-routes`, of the round 2 fixes, the held answers paragraph that follows the asset cache as a module, and the whole design against the code. Issues found, two of them blocking. The two were that the problem, the table of adapters and Rollout 2 still said that `handle_asset_proxy_request` keeps copies and that every route keeps them, when it keeps none, offers each `GET` to the asset cache module a deployment selects, and in a stock build asks the origin for every answer and adds no key-value traffic, and that the page named EdgeZero v0.0.9 as the locked version, when the main branch moved the lock to v0.0.10, whose `init_logger` still does nothing on Workers or on Spin. Fixed in this revision, which says in the problem, the table and Rollout 2 that a route keeps copies only where a deployment links and selects the 51Degrees asset cache module and a loading cache module, and names v0.0.10. The non-blocking finding fixed is the issue line, which said none where IABTechLab/trusted-server#777 lists the row as part of that issue. +4. **Round 4, 2026-10-28.** A lighter review in a fresh session, against `split/54-asset-routes`, of the round 3 fixes. Issues found, none of them blocking. The problem, the table and Rollout 2 now say that a route keeps copies only where a deployment links and selects the asset cache and loading cache modules, the page names EdgeZero v0.0.10, whose `init_logger` does nothing on Workers or on Spin, and the issue line names IABTechLab/trusted-server#777, and each was found correct against the code and the asset cache design. The non-blocking findings fixed in this revision are the problem, which said the handler offers each `GET` to the asset cache module where it leaves out a request the Image Optimizer handles, the note under the table and Rollout 2, which named a loading cache module where the stores they give are those of the 51Degrees loading cache module, and the note's statement that a stock build keeps no copy on any adapter, which now says it keeps no copy of its own. +5. **Round 5, 2026-10-28.** A lighter review in a fresh session, against `split/54-asset-routes`, of the round 4 fixes. No blocking issue and one smaller one. The problem now leaves a request the Image Optimizer handles out of the offer to the asset cache module, the note under the table and Rollout 2 name the 51Degrees loading cache module, whose stores the table and Rollout 2 give, and the note says a stock build keeps no copy of its own, and each was found correct against the code and `2026-10-15-asset-cache-design.md`. The smaller one was that the round 4 entry said the problem, the table and Rollout 2 already named the 51Degrees modules, when they named the 51Degrees asset cache module and a loading cache module, as the same entry goes on to say. This revision says that they named the asset cache and loading cache modules. diff --git a/docs/superpowers/specs/2026-10-27-integrations-crate-design.md b/docs/superpowers/specs/2026-10-27-integrations-crate-design.md index 5017850286..0fb636046d 100644 --- a/docs/superpowers/specs/2026-10-27-integrations-crate-design.md +++ b/docs/superpowers/specs/2026-10-27-integrations-crate-design.md @@ -331,3 +331,4 @@ These are for the reviewer, and this design decides none of them. 1. **Round 1.** A review in a fresh session, against `split/55-scripts-in-crates`. Issues found, three of them blocking. The three were that one manifest would carry the loading cache's test dependency on tokio's `rt-multi-thread`, which tokio refuses to build for WebAssembly, so `test-fastly` and `clippy-fastly` would stop, that three paths climb out of a crate's folder and break after the move, and that the design said a separate browser crate would move no more vendor code out, when the APS renderer and Prebid's code stay in `trusted-server-js`, where IABTechLab/trusted-server#1194 would move them. This revision puts that dependency in a table that only native builds read, with the threaded tests on the host, lists every path that climbs out of a folder with its fix, and says where the APS renderer and Prebid's code stay. It also states the manifest's lint and doctest settings, the source labels written as text, the log labels on Fastly and where vendor names live beside IABTechLab/trusted-server#1194, and leaves row 56's own measurement to that row's pull request. 2. **Round 2.** A review in a fresh session, checking the round 1 fixes against `split/55-scripts-in-crates`. Issues found, none blocking. This revision names the author of Tech Lab's split design, Aram Grigoryan, gives Tech Lab's request at the Touchbase as the record has it and places the decisions of 51Degrees after the Touchbase, defines stock build where first used, and calls the split design's integrations crate the crate that puts the catalog together. It also says that a feature gates neither a folder's name constants nor its source label, that `build-fastly`, `check-fastly` and `test-fastly` name `asset-cache-51degrees` and none of them names `loading-cache-51degrees`, and adds the sample permission policy to section 3. +3. **Round 3, 2026-10-27.** A review in a fresh session, against `split/55-scripts-in-crates`, of the license sentence and the dates of this revision. No issues found. `fiftyone-caching` 4.5.13 gives its license as `EUPL-1.2` in its manifest, the page carries 2026-10-27 in its name, status and revision rows, and no link names it under an earlier date.