Skip to content

Let a consent prompt run in the phase the publisher chooses - #1308

Closed
Jamesr51d wants to merge 557 commits into
IABTechLab:mainfrom
jwrosewell:split/21-cmp-phases
Closed

Jamesr51d wants to merge 557 commits into
IABTechLab:mainfrom
jwrosewell:split/21-cmp-phases

Conversation

@Jamesr51d

Copy link
Copy Markdown

Stacks on the sellers module (#1307). The additions that follow the seven pull requests open here are listed in #777.
Compare split/20-ads-txt with split/21-cmp-phases
to see only this pull request's change.

What this pull request does

cmp.inmobi, cmp.didomi and cmp.sourcepoint may be named in a [[fetch]] or a [[serve]] entry. Each ran in the fetch phase alone, and an entry of the other phase naming one was refused when the settings loaded. Osano is not changed, because it changes no page and so has no middleware.

[[serve]]
media_type = "text/html"
middleware = ["cmp.inmobi"]

A serve middleware's head markup is written straight before the script bundle, after every fetch middleware's. So a prompt placed in [[serve]] comes after a vendor's tag a [[fetch]] entry writes, and [[serve]] middleware = ["cmp.inmobi"] with [[fetch]] middleware = ["tag.google-tag-manager"] puts Tag Manager ahead of the prompt. The InMobi guide now says to place the prompt in [[fetch]] wherever a fetch middleware writes a vendor's tag.

Why

What a consent prompt writes is the same for every reader, so the fetch phase stores it in the page every reader is served from, and that is the phase ts audit generate places it in. A publisher who wants the stored page without it, so that one stored copy can serve readers with and without the prompt, places it in the serve phase instead, where it is written on each reader's copy. Which phase is the publisher's choice, and the modules no longer make it for them.

What is checked

  • Each of the three modules declares both phases.
  • The InMobi module selected with no entry of either phase naming it is refused, as before, and an entry of the serve phase alone is accepted and registers the prompt for that phase.
  • The stock list keeps each prompt in the fetch phase, and its test allows a middleware that runs in several phases to be listed in one of them, checking that what is listed for a phase is registered for it and that everything registered is listed somewhere. The test asserted per-phase equality before and now asserts containment, so it no longer holds the order of the stock list to the registry's, nor that a middleware registered for a phase is listed for that phase.

How it was verified

CI passed on 77e4f137c on the fork. Not exercised: no page has been served through an adapter with a prompt placed in the serve phase. Each module's registration is checked for both phases, and the stock list's registry for its listing.

References #1279.

- The context query parameter example names `[ad-server.mock]`, the table
  the mock ad server's settings are read from.
- A deploy validation test, the jurisdiction acknowledgment, two selector
  tests, the registry's geo test, the settings fixture and the integration
  fixture say what the code does without saying what it did before.

No code changes.
The Fastly adapter was a binary only, and the function that builds its
state with a deployment's integration builders was private to the crate. A
deployment that shipped a vendor crate had to edit the adapter, and the
round trip that proves the integration seam ran on the Axum dev server
alone, where the seam design asks for it on the primary deployment target.

The adapter is now a library with a thin binary over it. `run` serves the
way the binary does. `run_with` records the integration builders a
deployment offers, before any request is served, and they are composed with
the built-in ones when the application state is built. A deployment's binary
is one line, and offering a builder does not switch it on, because a builder
runs only when the settings select its module.

    fn main() {
        trusted_server_adapter_fastly::run_with(vec![a_vendor_crate::builder()]);
    }

The round trip runs under Viceroy on wasm32-wasip1, in
`src/app/seam_probe_tests.rs`, driven by the seam probe, a crate core does
not know about. Through this adapter's own router:

- the probe's carried browser module is served in the unified bundle under
  the hash composed from its content,
- its proxy route reports the country its own geo module resolved and that
  its request preparer ran exactly once, on a named route and on the
  fallback,
- a geo selector naming a module that declares no geo module fails at
  startup, naming the module and the capability,
- two builders claiming one id are refused, naming both sources,
- the Edge Cookie and device selectors resolve the modules the probe
  declares, and its Edge Cookie module creates an identifier the built-in
  module cannot make,
- the builders registered through `run_with` reach the state the application
  hooks build.

The seam probe is a test-only dependency of the adapter, as it is of the
Axum adapter.
The guide described an integration inside core only. A section on a vendor
crate was written for this branch in August and was lost when the guide was
rewritten on `main`, so the seam had no developer documentation beyond its
tests. The section is back, written for the selection model the branch has
now.

- What the crate provides: an `IntegrationBuilder` with an id, a source, a
  build function, a validate function, a module name taken from the crate's
  folder, and optionally a request preparer and auction implementations.
- What a registration can declare, including a carried browser script and the
  Edge Cookie, geo and device modules the three selectors can choose.
- How an adapter composes it in, with `routes_with_registrations` on Axum,
  Cloudflare and Spin, and what a Fastly deployment has to do instead.
- Two traps: the carried script's hash literal against line endings, and a
  vendor's deploy rules not running through the CLI.

The integrations overview points at the section, and the registration
checklist no longer asks for a disabled default, because an integration runs
when a section selects its module.
… crate

The integration guide's section on integrations outside core said the Fastly adapter could not take a vendor crate. It now shows the one-line binary a deployment writes around `run_with`, and points at the round-trip tests that run on the Fastly adapter. The changelog records that the adapter is a library with a thin binary over it and that the Wasm artifact keeps its name.
…point

The Fastly adapter's entry point moved from `main.rs` to `lib.rs`, so the comments in `app.rs` and `middleware.rs` that point at it, and the two onboarding tables, name `lib.rs`. No code changes.
`[ec] module`, `[geo] module` and `[device] module` picked a module a
registration declared by that registration's integration id, a hand-written
name where every other selection reads a module name.

A registration now declares each of the three under a name, the path under
`crates/` of the crate the module lives in, and the selectors read a written
name the way every section does, as written or with the type folder
(`edgecookie`, `geo` or `device`) in front. A geo module from
`crates/geo/example` is `[geo] module = "example"`, and one registration can
supply a module of each type, each under the name of its own crate.

- `with_ec_module`, `with_geo_module` and `with_device_module` take the name.
  An Edge Cookie module is declared under the name its `id` returns, and the
  registry refuses a registration where the two differ, because core checks
  the selected module by its id. Two registrations that supply a module of one
  type under one name are refused, naming both.
- `[ec] module` resolves the implementation its block names, as core does, so
  a label of the operator's own reaches a registered module. It looked for a
  registration under the label itself before, so a labelled block never
  reached one.
- A `[geo] module` or `[device] module` that names nothing running lists the
  modules of that type the deployment runs, and says when the name is a
  module no section selects or one that supplies no module of that type.
- The seam probe declares its three modules under `testing.seam-probe` and
  reports that name from its route. Its integration id stays the name of its
  routes and its browser module.

The module that supplies one of these still has to be selected in its
section, because the registry builds only what a section selects.

The configuration reference loses a paragraph and a heading that appeared
twice above `[ec.hmac]` and says that `[device]` holds no module settings
table where it implied one, and the auction README names the `modules` and
`module` keys where it still said `provider`.
…ition branch

The merged-in registry reads a registration's geo, Edge Cookie and device
modules by module name, so the Fastly round trip selects the probe's three as
`testing.seam-probe` and `seam_probe_tests.rs` changes with the merge.
…ings load

The settings are validated as they load, and that validation compiled the
auction plan with the built-in implementations alone. A deployment whose
`[demand]` or `[ad-server]` named an implementation one of its own builders
supplies was therefore refused while its settings loaded, before the adapter
built the state that compiles the plan with those builders. Deploy validation
had the same split, taking the builders for the block checks and compiling
the plan without them.

- `validate_settings_for_runtime_with`, `settings_from_config_blob_with` and
  `get_settings_from_config_store_with` carry a deployment's builders through
  the load, and the functions without the suffix call them with none. Each
  builder's validate function runs as part of that load, so a vendor's own
  rules hold at startup for a deployment that composes it.
- `validate_settings_for_deploy_with` compiles the plan with the builders it
  was given, so it and the running service agree on which names are valid.
- Four items an auction crate outside core needs are public:
  `UPSTREAM_RTB_MAX_RESPONSE_BYTES`, `collect_response_bounded`,
  `ensure_integration_backend_with_timeout` and
  `predict_integration_backend_name`. A crate rolling its own would bound a
  response at another size and derive another backend name for the same URL.
- The seam probe supplies an ad server, so that seam is reached from a crate
  core does not know. Two Axum tests start the adapter and run deploy
  validation with the probe's ad server selected, and both refuse the same
  settings without the probe's builder.

Four tests in core assert both halves, through the plan, runtime validation,
deploy validation and the real settings load, so each fails if the built-in
path stops refusing the name or the path with the builder stops accepting it.
The deploy validation test failed before this change. A fifth shows a
builder's own rejection surfacing from runtime validation.

The integration guide shows the load with builders, and a doc comment names
`[demand] modules` where it said `provider`.
…ition branch

The merged-in settings load takes a deployment's builders, so the Fastly
adapter hands it the ones `run_with` registered, as it hands them to the
state build. A round trip test builds this adapter's state with the probe's
ad server selected and shows the same settings refused without the probe's
builder. The guide keeps both accounts, the load with builders and the
Fastly library.
The Fastly entry point built each request's device signals with
`build_device_module`, which knows only `builtin` and `fastly`. With
`[device] module` naming a module a registration supplies, the registry
resolved that module and the adapter put it in the request's services, and
nothing ever asked it to classify a request, so the built-in User-Agent
module answered in its place.

The entry point now hands `derive_device_signals` the module the registry
resolved. A registered module is shown the User-Agent and the cookies, which
carry what a page gathers in the browser. With no registered module the
built-in or the Fastly module answers as before, shown the User-Agent alone.

Two tests in the adapter show a registered module classifying the request
and seeing its cookies, and the built-in module answering when nothing is
registered. `build_device_module`'s doc comment said a name it does not know
is refused by the settings, which is the registry's job, and says so.

Fastly is the one adapter that classifies a request. The Axum, Cloudflare
and Spin adapters resolve the selected device module and set no device
signals, on `main` as here.
…ition branch

The merged-in entry point hands the device module the registry resolved to
`derive_device_signals`. That function lives in `lib.rs` on this branch,
so the change is made there and `main.rs` stays the thin binary. A ninth
round trip test selects the probe's device module and shows it classifying a
request through the function the entry point calls, with the built-in module
answering when the selector is unset.
The probe supplied an ad server, so one half of the auction seam was reached
from a crate core does not know and the other was held by tests inside core
alone. Its builder now registers a demand source as well, one that sends the
standard request and reads every answer as no bid, which is enough to show
that everything a demand source needs is reachable from outside core.

An Axum test starts the adapter and runs deploy validation with the probe's
demand source selected, and both refuse the same settings without the
probe's builder. The probe's own tests pin both implementation names to its
module name and show the demand source refusing a setting it does not read.
…ition branch

The merged-in probe supplies a demand source, so a tenth round trip test
builds this adapter's state with it selected and shows the same settings
refused without the probe's builder, as the ad server's test does.
The Fastly adapter becomes a library with a thin binary, so a deployment
that ships an integration crate writes its own binary and passes that
crate's builder to `run_with`. The seam's round trip then runs on Fastly
under Viceroy from a crate core does not know, as it already does on Axum.

The branch was built and checked apart from this one so the seam could be
read without it. It is merged here because the seam's design asks for a
crate outside core to register on every adapter, and Fastly was the one
adapter where that could not be shown.
…ords

`KvNetwork` said a low cluster count "indicates an individual or
household". A count of connections is a fact about a network, and reading
a low one as an individual describes a device record as a record about a
person. It now says a small network, such as a home connection.

`KvEntry::ids` now says what it holds, which is each partner's identifier
for the same browser on the same device, never an identifier for a person.

Comments only, with no change in behavior.
The crate's description and the workspace manifest said only the Axum
adapter's tests depend on the probe. The Fastly adapter's tests drive the
same round trip, so both say so, and the description points at both test
files.
…e crate

The Osano consent mirror is the module `cmp.osano`, and it now lives in the
crate that name describes, `crates/cmp/osano`. Core no longer declares or
lists it. The settings that select and configure it do not change.

`crates/trusted-server-modules` lists the modules a stock build ships, in
the order their hooks run. Each adapter builds its state with that list
followed by the builders a deployment added, and loads its settings against
the same list. The `ts` tool registers the list with
`register_deploy_integrations`, because `TrustedServerAppConfig` validates
through a trait that takes no arguments, so `ts config validate` and
`ts config push` run a stock module's own rules.

Core's tests of how a selected module's table is read used the
configuration types of Osano and Testlight as examples, and now use two
example modules of their own. The two tests that needed Osano's builder
inside core run in Osano's crate with that builder. Core's shared test
settings are offered under the `test-utils` feature, so a module crate's
tests start from the same settings core's do.
An integration that leaves core still bounds a request body and an upstream
response, registers a backend, strips the headers an upstream must not set
on a first-party response, matches a host against the allowed domains, marks
a synthesized response private, reads a list setting and rewrites a bare
host. Those were private to the crate because every caller lived in it.
They are public, each with the reason on it.

The platform test doubles are offered under the `test-utils` feature, where
the adapters already take core's other fixtures, so a moved integration's
tests keep the stub HTTP client and stores they were written against.
The feature turns on the platform library's feature of the same name,
because the doubles are built on that library's own.
The Lockr identity module is `identity.lockr`, and it now lives in
`crates/identity/lockr`. The stock list offers it in the place its hooks
ran before, and the settings that select and configure it do not change.

Three of core's tests used Lockr as their example of a built-in module.
The duplicate id test and the standalone script test use core's own
modules, the JavaScript asset proxy and the creative browser module. The
test that the documented settings template's module tables validate moves
to the stock list crate, where each module's settings type is in reach,
and the two template helpers it shares with core's own tests are offered
from core's test support.
The Permutive audience module is `audience.permutive`, and it now lives in
`crates/audience/permutive`. The stock list offers it in the place its
hooks ran before, and the settings that select and configure it do not
change.
The Sourcepoint consent management module is `cmp.sourcepoint`, and it now
lives in `crates/cmp/sourcepoint`. The stock list offers it in the place
its hooks ran before, and the settings that select and configure it do not
change.

Its tests build a registry with the crate's own builder, through
`IntegrationRegistry::with_registrations`, which is offered under the
`test-utils` feature for that. The test that a selected consent mirror's
browser script loads with the page moves here from core, as Osano's half
of it did.
The Didomi consent management module is `cmp.didomi`, and it now lives in
`crates/cmp/didomi`. The stock list offers it in the place its hooks ran
before, and the settings that select and configure it do not change.

Core's test that a module's table survives the configuration blob round
trip used Didomi's settings as its example. It uses an example module of
its own, with a flag set away from its default.
The Google Tag Manager module is `tag.google-tag-manager`, and it now
lives in `crates/tag/google-tag-manager`. The stock list offers it in the
place its hooks ran before, and the settings that select and configure it
do not change.

One of its tests runs the HTML pipeline over the publisher page fixture
core's own HTML tests use. Core's test support offers that page, so the
half megabyte file stays in one place.
Jamesr51d and others added 24 commits October 13, 2026 14:38
…d run the Cloudflare auction test with no demand modules

The build script's TRUSTED_SERVER__GIT_VERSION reaches test processes and the strict loader refused it as an unknown field. Cloudflare runs one demand source at a time, so the fixture's two demand modules stopped worker startup.
The wasm clock lint reaches core's build script, which runs on the host.
Core's version_header module is removed, with its call from the finalize
step of the Axum, Cloudflare and Spin adapters and from the Fastly adapter's
health probe and its settings fallback, so no response carries x-ts-version
unless a [[serve]] entry names the service_version middleware. That
middleware now writes the git version the build compiled in from
TRUSTED_SERVER__GIT_VERSION or local git as X-TS-Version, when the build
knows one, and the version the adapter reports for the service as
X-TS-Service-Version, when it reports one. HEADER_X_TS_FASTLY_VERSION, which
nothing wrote, goes with the module.

The health tests of the three adapters prove the probe carries no version,
the middleware's own tests prove each header is written when its version is
known and nothing when neither is, and the publisher tests prove both
headers reach a buffered page and neither reaches a page no entry names the
middleware for. The configuration guide, the getting started guide, the
EdgeZero guide and the changelog say the same.
# Conflicts:
#	docs/guide/configuration.md
@jwrosewell

Copy link
Copy Markdown
Collaborator

Moved into this repository as #1362, one branch of the stack, so the review continues there. This pull request is closed in its favor and its comments stay here for the record.

@jwrosewell jwrosewell closed this Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants