Add identity provider core with one KV write per page load

CHANGELOG.md

@@ -9,6 +9,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+- A page load now writes the EC identity row at most once. Only the writer request, a real browser's top-level document navigation, writes, after the page has been sent; a failed write is not retried. `/auction`, `/_ts/page-bids` and `/_ts/api/v1/identify` no longer write KV. `/auction` stages the body EIDs that KV lacks in the new sealed `__Host-ts-eids-s` cookie, and the next navigation writes them; `/_ts/page-bids` EIDs reach KV through `ts-eids` at the next navigation. Removals and opt-outs not yet in KV travel in per-batch `__Host-ts-eids-r-*` cookies. A new visitor's row creation now carries its browser EIDs, so it stays one write.
+- Pull sync partners now run as server-side identity providers. They are called after send on a returning visitor's writer navigation, from the visitor's second page load, back off per visitor on errors instead of retrying on every request, and are rate-limited per visitor instead of per client IP. `ec.pull_sync_concurrency` is deprecated in favour of `[ec.identity].max_concurrency`. Rollback: once providers write metadata, do not roll back past this release; an older reader drops the new KV fields on its next write.
+- KV IDs sent to bidders, the admin view and `/_ts/api/v1/identify` now pass one servability check: expired IDs, and IDs whose writing provider is disabled, opted out or blocked by its consent policy, are never served. `/_ts/api/v1/identify` uses the request's consent, never the row's stored consent snapshot. Stored provider `atype` and UID-level `ext` are emitted.
+- **Breaking:** The publisher origin no longer receives the `ts-eids` cookie, and Prebid Server no longer receives any Trusted Server identity cookie (`ts-ec`, `ts-eids`, `ts-ec-pull-complete`, and the server-owned `__Host-ts-eids-s` and `__Host-ts-eids-r-*` cookies that a later release introduces). The origin still receives `ts-ec`. Origin code that read `ts-eids` should read the `x-ts-eids` response header instead. A Prebid Server `Cookie` header that is not valid UTF-8 is now dropped instead of forwarded. This release is the rollback floor for the server-side identity provider work.
 - **Breaking:** `x-ts-version` now reports the deployed git version — the release tag, else the branch, else the first 6 characters of the commit — compiled in from the build-time `TRUSTED_SERVER__GIT_VERSION` (set by the deploy pipeline) or local git, and is sent by every adapter, including on the Fastly `GET /health` probe. The Fastly service version it previously carried is now `x-ts-fastly-version`; update dashboards, monitors, and scripts that read `x-ts-version` as the Fastly version number. Builds with neither the override nor git omit the header.
 - **Breaking:** `ts prebid bundle` is now `ts prebid client`, alongside the new `ts prebid server` namespace. Update scripts and runbooks to use `ts prebid client` with the same arguments. The old `bundle` spelling is no longer accepted and has no compatibility alias.
 - The S2S `/_ts/api/v1/batch-sync` endpoint now validates the full batch and calls the CAS-protected update path once per distinct normalized EC ID. The last valid UID wins within a group, and infrastructure failures reject the failing and each unprocessed group, so accepted and `kv_unavailable` input indexes may interleave.

Cargo.toml

@@ -36,6 +36,7 @@ publish = false
 version = "0.1.0"
 
 [workspace.dependencies]
+aes-gcm = "0.10"
 anyhow = "1"
 assert_cmd = "2"
 async-stream = "0.3"

crates/trusted-server-adapter-axum/src/app.rs

@@ -233,6 +233,7 @@ async fn dispatch_fallback(
                 ec_context: &mut ec_context,
                 services,
                 req,
+                is_real_browser: false,
             })
             .await
             .unwrap_or_else(|| {

crates/trusted-server-adapter-cloudflare/src/app.rs

@@ -545,6 +545,7 @@ fn build_router(state: &Arc<AppState>) -> RouterService {
                         ec_context: &mut ec_context,
                         services: &services,
                         req,
+                        is_real_browser: false,
                     })
                     .await
                     .unwrap_or_else(|| {

crates/trusted-server-adapter-fastly/src/app.rs

@@ -325,9 +325,8 @@ pub(crate) struct EcFinalizeState {
     pub(crate) use_finalize_kv: bool,
     pub(crate) eids_cookie: Option<String>,
     pub(crate) sharedid_cookie: Option<String>,
-    pub(crate) is_real_browser: bool,
-    /// Per-request services carried to the entry point so the pull-sync
-    /// dispatcher can reuse the same platform HTTP client.
+    /// Per-request services carried to the entry point so the after-send
+    /// identity flush can reuse the same platform HTTP client.
     pub(crate) services: RuntimeServices,
 }
 
@@ -361,7 +360,6 @@ impl EcRequestState {
             use_finalize_kv: self.finalize_kv_graph.is_some(),
             eids_cookie: self.eids_cookie,
             sharedid_cookie: self.sharedid_cookie,
-            is_real_browser: self.is_real_browser,
             services: self.services,
         }
     }
@@ -735,8 +733,6 @@ async fn run_named_route(
 /// graph + partner registry + rate limiter, with a default EC context for
 /// response finalization.
 fn run_batch_sync(state: &AppState, services: &RuntimeServices, req: Request) -> Response {
-    let device_signals = device_signals_for(&req);
-    let is_real_browser = device_signals.looks_like_browser();
     let eids_cookie = crate::extract_cookie_value(&req, COOKIE_TS_EIDS);
     let sharedid_cookie = crate::extract_cookie_value(&req, COOKIE_SHAREDID);
 
@@ -754,7 +750,6 @@ fn run_batch_sync(state: &AppState, services: &RuntimeServices, req: Request) ->
         use_finalize_kv: false,
         eids_cookie,
         sharedid_cookie,
-        is_real_browser,
         services: services.clone(),
     });
     response
@@ -820,6 +815,7 @@ async fn dispatch_fallback(
                 ec_context: &mut ec.ec_context,
                 services,
                 req,
+                is_real_browser: ec.is_real_browser,
             })
             .await
             .unwrap_or_else(|| {
@@ -847,13 +843,25 @@ async fn dispatch_fallback(
         if is_navigation {
             ec.ec_context.set_eid_sync_source(EidSyncSource::Navigation);
         }
+        ec.ec_context.decide_writer(&req, true, ec.is_real_browser);
         let is_publisher_navigation = ec.is_real_browser && is_navigation;
-        if is_publisher_navigation
-            && let Err(err) = ec
+        if is_publisher_navigation {
+            // A new visitor's create carries the browser EIDs, so it stays
+            // the page load's one KV write.
+            if let Ok(partner_registry) = PartnerRegistry::from_config(&state.settings.ec.partners)
+            {
+                ec.ec_context.seed_generation_from_cookies(
+                    &partner_registry,
+                    ec.eids_cookie.as_deref(),
+                    ec.sharedid_cookie.as_deref(),
+                );
+            }
+            if let Err(err) = ec
                 .ec_context
                 .generate_if_needed(&state.settings, ec.kv_graph.as_ref())
-        {
-            log::warn!("EC generation failed for publisher proxy: {err:?}");
+            {
+                log::warn!("EC generation failed for publisher proxy: {err:?}");
+            }
         }
 
         // Run the server-side auction with the configured creative-
@@ -2417,7 +2425,10 @@ mod tests {
             .get::<super::EcFinalizeState>()
             .expect("fallback response should carry EcFinalizeState");
         assert!(
-            finalize.is_real_browser,
+            finalize
+                .ec_context
+                .device_signals()
+                .is_some_and(DeviceSignals::looks_like_browser),
             "browser-shaped JA4/H2 signals from the request extension must mark the request as a real browser"
         );
     }
@@ -2436,7 +2447,10 @@ mod tests {
             .get::<super::EcFinalizeState>()
             .expect("fallback response should carry EcFinalizeState");
         assert!(
-            !finalize.is_real_browser,
+            !finalize
+                .ec_context
+                .device_signals()
+                .is_some_and(DeviceSignals::looks_like_browser),
             "a request without captured device signals should not be classified as a real browser"
         );
     }