Skip to content

Preserve Prebid EID fidelity from browser capture to bidder request - #1415

Merged
dhruv8sh merged 15 commits into
spec/1246-prebid-eid-fidelityfrom
feat/1246-prebid-eid-fidelity
Oct 19, 2026
Merged

dhruv8sh merged 15 commits into
spec/1246-prebid-eid-fidelityfrom
feat/1246-prebid-eid-fidelity

Conversation

@dhruv8sh

@dhruv8sh dhruv8sh commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • The trustedServer Prebid adapter now forwards exactly the EIDs Prebid placed in its own bidder request (ortb2.user.ext.eids and user.eids), with their OpenRTB 2.6 inserter, matcher, mm and ext. It no longer reads pbjs.getUserIdsAsEids(), which leaked IDs restricted to other bidders or redacted for trustedServer (D1), and missed every ortb2 EID such as Lockr's ID5 (D2, Write Lockr IDs to KV store #1246).
  • The ts-eids writer moved into buildRequests, so the cookie copies each request's set (without EID-level ext) before the request is sent, and a tcf2Enforcement listener clears it when Prebid's TCF rules block trustedServer. The server validates body and cookie with one parser that drops only the bad entry or field, forwards browser entries as sent with KV filling only absent registered sources, and ingests into KV on /auction only the client set the auction used.
  • TS stops overwriting the publisher's Prebid userSync settings (the forced auctionDelay: 0 that sent first auctions before async ID modules resolved, D4).

ts-eids and x-ts-eids both stay. Stacked on #1416, which carries the spec and plan; this PR stays in draft until #1416 merges.

Changes

File Change
crates/trusted-server-core/src/openrtb.rs Eid gains inserter, matcher, mm, ext (skipped when absent, byte-identical otherwise) and Eid::new
crates/trusted-server-core/src/ec/client_eids.rs (new) One parser for browser EIDs: per-element parse isolation, field rules and caps, dedupe before the 64-entry cap, value-free debug logs
crates/trusted-server-core/src/auction/formats.rs, auction/endpoints.rs AdRequest.eids captured as raw JSON; body and cookie go through the shared parser; merge_auction_eids forwards browser entries as sent and adds KV only for absent registered sources
crates/trusted-server-core/src/ec/prebid_eids.rs, ec/admin.rs Structured ts-eids cookie decoded through the shared parser; admin diagnostics keep raw sources and UIDs; non-array cookie JSON reported as a parse error
crates/trusted-server-core/src/ec/finalize.rs KV ingestion input chosen by sync source: /auction reads its own client set, other routes the cookie
crates/trusted-server-core/src/{publisher.rs, ec/eids.rs, ec/identify.rs, integrations/*, auction/*} Eid::new call sites; outbound serialization, merge and ingestion tests
crates/trusted-server-adapter-fastly/src/app.rs Adapter-layer test for the raw ts-eids handed to finalization
crates/trusted-server-js/lib/src/integrations/prebid/eids.ts (new) Pure collector (64 KiB UTF-8 budget) and ts-eids value builder (no EID-level ext, 3,072 bytes, UTF-8 base64)
crates/trusted-server-js/lib/src/integrations/prebid/index.ts, src/core/auction.ts buildRequests forwards the bidder request's EIDs and writes or clears the cookie; TCF listener; old collector, cookie sync and installUserIdModules removed; AuctionEid is opaque
crates/trusted-server-js/lib/src/integrations/prebid/user_id_modules.{ts,json} Unused EID-to-module resolver and eidSources removed
crates/trusted-server-js/lib/test/** eids.test.ts; adapter tests with a getUserIdsAsEids canary; real-Prebid JSDOM suite prebid-eid-fidelity.test.mjs; cross-language fixtures
docs/guide/** Prebid, Lockr, edge cookies, EC setup, API reference and configuration guides, including the full trustedServer TCF exception rule
docs/superpowers/implementation-notes/2026-10-18-prebid-eid-fidelity.md Red log: every new test seen failing first, or proven by reverting the line it guards

Closes

Closes #1246

Spec

Spec: docs/superpowers/specs/2026-10-18-prebid-eid-fidelity-design.md (in #1416; five agent review rounds in its review log)

What breaks (spec 8.2)

  • Bidders: browser entries come first, KV-only sources after. KV's value and atype are no longer sent for a source the browser sends, and repeated sources are no longer folded.
  • Publishers: TS stops overwriting syncEnabled, filterSettings, syncsPerBidder, syncDelay and auctionDelay, and no longer force-enables iframe syncs. Auctions wait up to auctionDelay (500 ms by default) while IDs are pending. A late managed seed still leaves autoRefresh: true.
  • Server-side bidders folded into trustedServer: IDs restricted to them are no longer forwarded.
  • GDPR traffic without a trustedServer exception: the cookie is cleared instead of written from getUserIdsAsEids(). Body-less routes and KV ingestion lose browser EIDs for that traffic, including the IdentityLink flow.
  • Cookie contents: ortb2 EIDs and provenance fields, never EID-level ext, written at request time without a Purpose 1 check of its own.
  • KV ingestion on /auction no longer reads the cookie next to the body.
  • Providers: additive fields and possibly repeated sources in user.ext.eids and x-ts-eids.
  • /_ts/admin/eids consumers: new fields in the cookie entries; a mistyped field is dropped instead of failing the whole cookie.
  • Old TSJS, baked-in shims, rolled-back servers: old TSJS's body and cookie are parsed by the new parser; a rolled-back server reads only source and uids from new bodies and cookies.

Operator actions (spec 8.3)

  • Before production, send one request with a repeated source and the four new fields (inserter, matcher, mm and EID-level ext) to each configured provider in staging and confirm it is accepted.
  • Regenerate external bundles that carry a baked-in shim (ts prebid client).
  • Tell publishers about the userSync change: set auctionDelay and filterSettings in their own Prebid config if Prebid's defaults do not suit them.
  • For sites with GDPR traffic, decide with the publisher whether to grant trustedServer the full TCF exception rule (and the full personalizedAds rule when eidsRequireP4Consent is set). A partial rule switches off TCF checks for every bidder.

Follow-ups (spec 8.5)

Prebid.js identityLink fix for a malformed _lr_env (D7); sharedId cookie ingestion ignoring Prebid restrictions; TS's own TCF identity; per-bidder EID forwarding through Prebid Server eidpermissions; refresh rules for browser-sourced KV IDs; restoring autoRefresh after a late managed seed; gating the ts-eids write on Purpose 1; a server-written cookie; a lenient parse of the rest of the /auction body; no client-side trustedServer demand under GDPR without a publisher exception (predates this work).

Test plan

  • cargo test-fastly && cargo test-axum (core 2985 passed), plus cargo test-fastly-reuse, cargo test-cloudflare, cargo test-spin
  • cargo clippy-fastly && cargo clippy-axum, plus clippy-cloudflare, clippy-cloudflare-wasm, clippy-spin-native, clippy-spin-wasm, clippy-cli, clippy-codegen
  • cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run (1252 passed, 52 files, Node 24.12.0), including the real-Prebid suite whose arms fail against main
  • JS format and lint: cd crates/trusted-server-js/lib && npm run format && npm run lint
  • Docs format: cd docs && npm run format
  • WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
  • Manual testing via fastly compute serve
  • Other: parity test (17 passed); template_build_digest test and clippy; scripts/lint-wasm-clock.sh; scripts/test-cli.sh; Markdown Prettier outside docs/
  • Other: staging provider check (repeated source and the four new fields per provider)
  • Other: live check through ts dev proxy on a page with Lockr and the default User ID preset (spec 7.6)

Checklist

  • Feature PRs: spec linked above, ending in a review log of two or more agent review rounds
  • Changes follow AGENTS.md conventions
  • No unwrap() in production code — use expect("should ...")
  • Uses log macros per AGENTS.md (not println!)
  • New code has tests
  • No secrets or credentials committed

Eid gains inserter, matcher, mm and ext, each skipped when absent, so serialization without them is byte-identical. Eid::new replaces the struct literals that never carry these fields.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
ec::client_eids parses each array element on its own, so one malformed entry or field costs only itself. It validates sources, UIDs, atype, EID-level provenance fields and ext sizes, drops duplicates before the 64-entry cap, and logs field names and sources but never values.
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
AdRequest captures eids as raw JSON so a lone surrogate in one entry no longer rejects the whole /auction body. The ts-eids structured branch uses the same parser, so a mistyped field or bad entry costs only itself and the body's caps apply to the cookie. Admin diagnostics still report raw sources and UIDs.

The end-to-end fixture tests land with the merge rewrite, which they also depend on.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Browser entries reach providers unchanged, in wire order, with repeated sources kept apart. KV adds entries only for registered sources the browser did not send, so its stale UID and operator atype no longer override the browser's.

Includes the end-to-end TSJS fixture tests deferred from the shared-parser change, since they depend on this merge keeping EID-level fields.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Finalization chooses its browser input by sync source. /auction reads the client set the auction used, so a cookie from an earlier page is not ingested next to a scoped body. Other routes keep reading ts-eids, and the sharedId cookie is still applied last.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
collectBidderEids copies ortb2.user.ext.eids and user.eids as plain data within a 64 KiB UTF-8 budget. fitEidCookie builds the ts-eids value without EID-level ext, trimmed to 3,072 bytes and encoded as UTF-8 base64.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
…buildRequests

buildRequests reads bidderRequest.ortb2 instead of getUserIdsAsEids(), so IDs restricted to other bidders or redacted for trustedServer are no longer forwarded, and ortb2 EIDs such as Lockr's ID5 now arrive with their EID-level fields. The cookie copies each request's set before it is sent and is cleared when Prebid's TCF rules block trustedServer. bidsBackHandler no longer touches it.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
The served shim measured 42,144 characters after it gained the bidder-request EID collector, ts-eids cookie fitting and the tcf2Enforcement listener, so the bound moves from 41,000 to 42,500.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
installUserIdModules forced auctionDelay 0, syncDelay 3000, syncsPerBidder 5 and iframe syncs for every bidder on window.load. Prebid accumulates userSync settings, so every later auction ran without waiting for pending IDs. TS now writes only managed userIds and, on a late seed, autoRefresh.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
A JSDOM suite evaluates the generated bundle and the served shim and captures /auction bodies. It covers a bidder-restricted ID, same-source entries with their own fields, the first auction after window.load, and TCF with and without the full trustedServer exception. Every arm except the timing control fails against main.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
D1, D3 and both TCF arms now assert that document.readyState is still loading after install and after the auction, so each proves it ran before window.load as spec 7.5 requires.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Only tests read resolvePrebidUserIdModulesFromEids and the registry's eidSources. The data could not be made correct, since several modules share sources. The CLI and bundle builder read only module and config names.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Guides now describe forwarding the trustedServer bidder request's EIDs, the browser-first merge, per-entry validation, KV ingestion inputs by route, TS no longer setting userSync, and the full trustedServer TCF rule with its warning.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Guard the cookie clear so a blocked document.cookie costs only the cookie, and cover the nothing-fits clear path and both failure paths with adapter tests.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Admin diagnostics lost the parse error for cookies that decode to a non-array JSON value. Routing is unchanged. Record the final review fixes in the red log.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
@dhruv8sh
dhruv8sh marked this pull request as draft October 19, 2026 11:28
@dhruv8sh dhruv8sh self-assigned this Oct 19, 2026
@dhruv8sh dhruv8sh linked an issue Oct 19, 2026 that may be closed by this pull request
@dhruv8sh
dhruv8sh merged commit 0ea795a into spec/1246-prebid-eid-fidelity Oct 19, 2026
15 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant