Repository navigation
Conversation
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Every Cookie field is now read and checked, and a field that is not valid UTF-8 is dropped instead of forwarded. Two transport tests that pinned raw forwarding of malformed cookie bytes now expect the field to be dropped. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
The origin keeps ts-ec and the pull-complete marker. ts-eids, __Host-ts-eids-s and the __Host-ts-eids-r- cookies no longer reach it. An origin that read ts-eids must read x-ts-eids instead. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Documents which identity cookies reach the publisher origin and Prebid Server. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Adds aes-gcm, which the sealed identity cookies use. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
New fields are optional and omitted when absent, so legacy rows read and write unchanged and the schema version stays 1. KvPartnerId and KvEntry derive Default, and literals move to constructors. The admin lookup redacts provider state. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Reuses TcfConsent::has_vendor_consent and the effective TCF lookup the consent module already uses. Adds shared consent fixtures for identity tests. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Expired IDs, IDs from disabled or opted-out writers, and IDs whose writer or owner policy fails are no longer sent to bidders, the admin view or /identify. /identify uses the request's consent, never the row's stored snapshot. Stored atype and UID-level ext are emitted. The provider catalog is empty until generic pull is wired in. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Removals apply first and win equal-second ties, a provider opt-out removes every record that provider wrote, stateful results apply atomically, refresh anchors survive while their owner holds state, and an invalid row falls back to removals only. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Each removal-producing response gets its own sealed cookie. A batch is never coarsened: an oversized one is planned as an overflow write, or split across cookies when that write fails. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Validated at deploy and at startup. ec.pull_sync_concurrency is deprecated in favour of [ec.identity].max_concurrency and is still honoured when changed from its default. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Adds the merged eligibility view of KV, the identity cookies and removal batches, the eligibility steps, and the mapping from each call outcome to KV changes and backoff. Also adds the consent fingerprint bits a provider opt-out records. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Each call is checked for https, a declared host and no Cookie header, capped per visitor, sent with the provider's timeout, and decoded under max_response_bytes before the provider sees it. An undeclared input read fails the call and nothing is sent. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Each pull-enabled partner becomes a provider with an unowned source, its allowed domains as hosts, and the parity consent policy. Its legacy hourly rate limit maps to the per-minute cap the limiter already enforces. Adds the basic provider harness and a KV test store that counts writes by mode. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Only an organic real-browser top-level document navigation that is not a prefetch is the writer. EcContext now captures the user agent, DNT and the raw identity cookies when the request is read, because after-send code no longer has the request. Integration-proxy EC generation now requires a real browser explicitly, and is_prefetch_request moves to http_util. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
/identify uses a stored cluster size, or computes one without writing it. The writer navigation stores it in its single flush. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Returning-visitor finalize no longer writes KV. The writer navigation adds browser ingest, flush-checked staged entries and removal batches to its single mutation. /auction saves the KV gap of its body EIDs in __Host-ts-eids-s, and /_ts/page-bids no longer ingests. A new visitor's create now carries its browser EIDs, so it stays one write. Orphan rotation carries removals into the new row and never positive staged IDs. Consent loss and tombstone reads expire __Host-ts-eids-s. The provider catalog now indexes every registered partner source, so a KV-gap entry can be staged for any of them. The old ingest-sync path in kv.rs is removed with its tests. The queued mutation is written by the after-send flush added later in this series. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
When a removal batch does not fit one cookie, or four removal cookies are already present, it is applied to KV with the withdrawal path's CAS retries, which retry only on a generation mismatch. If that write fails it is split across new cookies, never coarsened. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
The server set is KV plus the __Host-ts-eids-s entries, staged ones included, with removed, expired and unservable records left out. A removal cookie hides its record before the removal reaches KV. Server values keep their stored atype and UID-level ext and gain the writing provider's EID-level provenance. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Phase 4 replaces the pull sync after-send hook. It reuses the phase 1 snapshot when no provider is marked, otherwise re-reads the row, dispatches the marked providers in an hourly rotation, and applies the whole mutation in one IfGenerationMatch write that is never retried. Pull partners now run only on a returning visitor's writer navigation, are marked pending in phase 3 after a read shows their source missing, and back off through ps. A read that shows a pull partner's sources present records a cookie-only next-due time, so the unowned read trigger does not fire on every navigation. The pull sync dispatcher, its bulk upsert and their tests are removed; the adapter builds a KV graph after send only when there is something to flush. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Drives the create before routing, finalize before send and the after-send flush, as the adapter runs them, against a write-counting store: one write per page load, none on stagers or non-2xx/304 navigations, no retry on a CAS conflict, zero reads for a recent cookie with nothing due, and an /auction body-only source reaching KV on the next navigation even after a refused flush. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Test-only helpers are gated on cfg(test). The capture, hold-mode and breaker hooks the providers change uses carry a targeted dead_code expectation, which fails once that change starts using them. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
One measurement line per identity request reports the inbound Cookie header size, the number and total size of removal cookies, and whether the writer read KV and queued changes. The adapter logs each flush outcome. No ID or cookie value is logged. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Collaborator
Author
|
Closing in favor of #1428 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ts-ecbut no longer receivests-eidsor the new identity cookies, Prebid Server receives no TS identity cookie in any mode, and the sealed cookies can't be template-cache keys./auction,/_ts/page-bids,/identifyand proxy calls no longer write. They carry their changes forward in a sealed__Host-ts-eids-scookie and per-batch__Host-ts-eids-r-*removal cookies. This keeps a busy page well under Fastly's per-key write limit, and pull sync now runs as the first provider on the newIdentityProvidertrait.Changes
crates/trusted-server-core/src/constants.rs,cookies.rscrates/trusted-server-core/src/auction/routing.rs,integrations/prebid.rsCookieheader instead of forwarding it rawcrates/trusted-server-core/src/publisher.rscrates/trusted-server-core/src/cookies/template_cache_policy.rscrates/trusted-server-core/src/ec/identity/(new)crates/trusted-server-core/src/ec/kv_types.rs,kv.rs,kv_backend.rswrite_identitypath; write-counting backend for tests; remove the old sync and bulk-upsert writescrates/trusted-server-core/src/ec/finalize.rs,ec/mod.rscrates/trusted-server-core/src/ec/eids.rs,identify.rs,admin.rs/identifyand the admin view;/identifyno longer writes; admin redacts provider statecrates/trusted-server-core/src/ec/pull_sync.rs,pull_sync_marker.rscrates/trusted-server-core/src/auction/endpoints.rs/auctionstages a KV gap instead of writingcrates/trusted-server-core/src/http_util.rs,integrations/registry.rsis_prefetch_requestmoved here, plusis_document_navigation; writer decision in proxy dispatchcrates/trusted-server-core/src/settings.rs,config.rs,migration_guards.rs,trusted-server.example.toml[ec.identity]settings and validation; warning for the deprecated keycrates/trusted-server-adapter-fastly/src/app.rs,main.rscrates/trusted-server-adapter-{axum,cloudflare,spin}/src/app.rsis_real_browser: falseCargo.toml,crates/trusted-server-core/Cargo.toml,Cargo.lockaes-gcm = "0.10"(the spec's choice for the sealed cookie)docs/guide/configuration.md,edge-cookies.md,api-reference.md,CHANGELOG.mdChanges from the plan
__Host-ts-eids-ssource index covers every registered[[ec.partners]]source, not only provider-claimed ones. Otherwise a browser entry for a partner that no provider claims could not be staged.test-utilsfeature andTcfConsent::has_vendor_consentare reused instead of new equivalents.#[expect(dead_code)], so the build fails once Part 2 starts using them and the attribute can't be forgotten.Closes
Part of #1246. It doesn't close the issue; #1416 does.
Spec
Spec:
docs/superpowers/specs/2026-10-20-server-side-identity-providers-design.md(#1428)Test plan
cargo test-fastly && cargo test-axumcargo clippy-fastly && cargo clippy-axumcargo fmt --all -- --checkcd crates/trusted-server-js/lib && npx vitest runcd crates/trusted-server-js/lib && npm run formatcd docs && npm run formatcargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1fastly compute servecargo test-fastly-reuse,test-cloudflareandtest-spin;node --testandtest.shsuites.shellcheckwas not run because it isn't installed locally, and this PR doesn't touchscripts/.Checklist
unwrap()in production code — useexpect("should ...")logmacros (notprintln!)