diff --git a/CHANGELOG.md b/CHANGELOG.md index c00487769..7078e1154 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - **Breaking** — Replaced the legacy APS contextual integration with APS OpenRTB at `/e/pb/bid`. APS configuration now uses canonical `account_id` (`pub_id` remains a compatibility alias), no longer requires APS-specific slot IDs, and defaults script creative eligibility off. Operators must update the endpoint, disable native APS demand for Trusted Server cohorts, and prepare GAM/Universal Creative targeting for `hb_bidder=aps` before rollout. `aps` entries in Prebid bidder lists are logged and stripped. APS renderer winners now preserve the upstream bid `id`, omit `crid` when APS omits it, and carry `ext.trusted_server.renderer` instead of `adm`; external `/auction` consumers must support this response shape. +- Publisher HTML now uses `Cache-Control: max-age=60` when server-side ad templates are inactive, while preserving origin `private`/`no-store` policies and CDN-specific cache headers. Set `[creative_opportunities].enabled = false` to disable publisher HTML and SPA template delivery without disabling direct `POST /auction` callers. - **Breaking** — All auction paths now forward only a validated publisher-owned page URL as `site.page`, removing query and fragment data. APS OpenRTB omits `site.ref`; the existing Prebid Server path continues to forward the browser `Referer` as `site.ref`. Query-driven sites may lose contextual targeting and per-page reporting signals that previously came from query parameters. - **Breaking** — `bid_param_zone_overrides` inner values must now be JSON objects; previously non-object or empty values (`"header" = "x"`, `"header" = {}`) were accepted and silently produced a dead rule at runtime. They now fail at startup with a configuration error. Operators upgrading should audit their `bid_param_zone_overrides` config for non-object zone entries. - **Breaking** — Integration configuration strings are no longer globally reinterpreted as JSON scalars. Operators upgrading should audit `[integrations.*]` settings and use native TOML/typed-config booleans and numbers (for example, `enabled = true`, not `enabled = "true"`); quoted numeric and boolean scalars now fail validation instead of silently converting. @@ -34,6 +35,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - `creative_opportunities.slot.gam_unit_path` is now a template supporting `{network_id}`, `{slot_id}`, and `{section}`, so a publisher whose ad unit varies by site section expresses it in one slot rule instead of one per (slot × section). `{section}` derives from the request path: `[creative_opportunities].section_segment` selects which path segment names the section (0-based, default `0`; set `1` for locale-prefixed URLs), and `section_root` supplies the value for paths with no such segment. `section_root` is required when a template uses `{section}`. Existing static and absent `gam_unit_path` configs are unchanged. Startup rejects a blank `gam_network_id` only when an absent/default path or `{network_id}` template consumes it. Trusted Server conservatively caps whole rendered dynamic paths at 100 UTF-8 bytes, informed by Google's 100-character per-ad-unit-code limit; an over-limit request-specific path omits that slot without failing the response. During typed/startup finalization, every placeholder-bearing template that omits `section_segment` materializes `section_segment = 0`, so an older binary rejects the blob loudly. Static and absent paths remain legacy-schema compatible only when both `section_root` and `section_segment` are omitted. Before rolling back below this feature, replace or remove dynamic paths, remove both keys, re-push and finalize the config, then roll back the binary. - Added opt-in APS HTTP debug metadata for controlled test sites, exposing the direct request and response under `/auction` provider metadata using the Prebid Server `debug.httpcalls` shape. - Added typed APS renderer transport for direct auctions and GAM/Prebid Universal Creative, using a minimized one-bid envelope, a fragment-bound nonce, and an opaque sandboxed renderer endpoint. +- Added the `[auction].rewrite_creatives` (default `true`) and `[auction].sanitize_creatives` (default `false`) options. `rewrite_creatives` rewrites winning-bid adm to first-party endpoints across `POST /auction` and publisher SSAT/page-bids delivery (proxy/click URL conversion, bidder `` removal; creative TSJS injection on `POST /auction` only). Enabling `sanitize_creatives` strips executable markup from winning-bid adm before delivery. +- `creative_opportunities.slot.gam_unit_path` is now a template supporting `{network_id}`, `{slot_id}`, and `{section}`, so a publisher whose ad unit varies by site section expresses it in one slot rule instead of one per (slot × section). `{section}` derives from the request path: `[creative_opportunities].section_segment` selects which path segment names the section (0-based, default `0`; set `1` for locale-prefixed URLs), and `section_root` supplies the value for paths with no such segment. `section_root` is required when a template uses `{section}`. Existing static and absent `gam_unit_path` configs are unchanged. Startup rejects a blank `gam_network_id` only when an absent/default path or `{network_id}` template consumes it. Trusted Server conservatively caps whole rendered dynamic paths at 100 UTF-8 bytes, informed by Google's 100-character per-ad-unit-code limit; an over-limit request-specific path omits that slot without failing the response. During typed/startup finalization, every placeholder-bearing template that omits `section_segment` materializes `section_segment = 0`, so an older binary rejects the blob loudly. Static and absent paths remain legacy-schema compatible only when both `section_root` and `section_segment` are omitted. Before rolling back below this feature, replace or remove dynamic paths, remove both keys, re-push and finalize the config, then roll back the binary. - Added Osano consent mirror integration docs and public enablement guidance. - Implemented basic authentication for configurable endpoint paths (#73) - Added integrations guide with example `testlight` integration diff --git a/crates/trusted-server-adapter-axum/src/app.rs b/crates/trusted-server-adapter-axum/src/app.rs index 1bed830ac..85bd2a211 100644 --- a/crates/trusted-server-adapter-axum/src/app.rs +++ b/crates/trusted-server-adapter-axum/src/app.rs @@ -29,6 +29,7 @@ use trusted_server_core::settings::Settings; use trusted_server_core::settings_data::{ default_config_key, default_config_store_name, get_settings_from_config_store, }; +use trusted_server_core::trace_cookie::handle_trace_mode; use trusted_server_core::platform::RuntimeServices; @@ -262,6 +263,7 @@ enum NamedRouteHandler { /// Legacy `/admin/keys/*` aliases — denied locally with 404 so they never /// reach the publisher fallback (which would leak admin credentials). LegacyAdminDenied, + TraceMode, Auction, PageBids, FirstPartyProxy, @@ -286,7 +288,7 @@ const LEGACY_ADMIN_DENY_METHODS: &[Method] = &[ Method::DELETE, ]; -fn named_routes() -> [NamedRoute; 13] { +fn named_routes() -> [NamedRoute; 14] { [ NamedRoute { path: "/.well-known/trusted-server.json", @@ -327,6 +329,11 @@ fn named_routes() -> [NamedRoute; 13] { primary_methods: LEGACY_ADMIN_DENY_METHODS, handler: NamedRouteHandler::LegacyAdminDenied, }, + NamedRoute { + path: "/_ts/trace", + primary_methods: &[Method::GET], + handler: NamedRouteHandler::TraceMode, + }, NamedRoute { path: "/auction", primary_methods: &[Method::POST], @@ -408,6 +415,9 @@ fn named_route_handler( Ok(resp) } NamedRouteHandler::LegacyAdminDenied => Ok(legacy_admin_alias_denied()), + NamedRouteHandler::TraceMode => { + handle_trace_mode(&state.settings, req.uri().query()) + } NamedRouteHandler::Auction => { // Build the geo-aware EC context so the auction consent // gate sees the caller's jurisdiction — `EcContext::default()` diff --git a/crates/trusted-server-adapter-cloudflare/src/app.rs b/crates/trusted-server-adapter-cloudflare/src/app.rs index 644676fc5..dc7a7e91f 100644 --- a/crates/trusted-server-adapter-cloudflare/src/app.rs +++ b/crates/trusted-server-adapter-cloudflare/src/app.rs @@ -29,6 +29,7 @@ use trusted_server_core::request_signing::{ handle_trusted_server_discovery, handle_verify_signature, }; use trusted_server_core::settings::Settings; +use trusted_server_core::trace_cookie::handle_trace_mode; use crate::middleware::{AuthMiddleware, FinalizeResponseMiddleware}; use crate::platform::build_runtime_services; @@ -474,6 +475,15 @@ fn build_router(state: &Arc) -> RouterService { .post("/_ts/admin/keys/deactivate", |_ctx: RequestContext| async { Ok::(admin_key_management_not_supported()) }) + // Render-trace toggle: arms/disarms the ts-trace cookie and + // redirects to `/`. Gated by [debug] trace_route_enabled (404 when + // off). + .get( + "/_ts/trace", + make_handler(Arc::clone(&state), |s, _services, req| async move { + handle_trace_mode(&s.settings, req.uri().query()) + }), + ) .post( "/auction", make_handler(Arc::clone(&state), |s, services, req| async move { diff --git a/crates/trusted-server-adapter-fastly/src/app.rs b/crates/trusted-server-adapter-fastly/src/app.rs index d6090c983..8e56916b2 100644 --- a/crates/trusted-server-adapter-fastly/src/app.rs +++ b/crates/trusted-server-adapter-fastly/src/app.rs @@ -26,6 +26,7 @@ //! | GET | `/_ts/api/v1/identify` | [`handle_identify`] | //! | GET | `/_ts/set-tester` | [`handle_set_tester`] | //! | GET | `/_ts/clear-tester` | [`handle_clear_tester`] | +//! | GET | `/_ts/trace` | [`handle_trace_mode`] | //! | OPTIONS | `/_ts/api/v1/identify` | [`cors_preflight_identify`] | //! | POST | `/auction` | [`handle_auction`] | //! | GET | `/first-party/proxy` | [`handle_first_party_proxy`] | @@ -130,6 +131,7 @@ use trusted_server_core::settings_data::{ default_config_key, default_config_store_name, get_settings_from_config_store, }; use trusted_server_core::tester_cookie::{handle_clear_tester, handle_set_tester}; +use trusted_server_core::trace_cookie::handle_trace_mode; use crate::middleware::{AuthMiddleware, FinalizeResponseMiddleware}; use crate::platform::{ @@ -596,6 +598,7 @@ async fn run_named_route( } NamedRouteHandler::SetTester => handle_set_tester(&state.settings), NamedRouteHandler::ClearTester => handle_clear_tester(&state.settings), + NamedRouteHandler::TraceMode => handle_trace_mode(&state.settings, req.uri().query()), NamedRouteHandler::Auction => { // The auction reads consent data, so the consent KV store must be // available — fail closed with 503 when it is configured but @@ -1008,6 +1011,7 @@ enum NamedRouteHandler { Identify, SetTester, ClearTester, + TraceMode, Auction, PageBids, FirstPartyProxy, @@ -1089,6 +1093,11 @@ const NAMED_ROUTES: &[NamedRoute] = &[ primary_methods: &[Method::GET], handler: NamedRouteHandler::ClearTester, }, + NamedRoute { + path: "/_ts/trace", + primary_methods: &[Method::GET], + handler: NamedRouteHandler::TraceMode, + }, NamedRoute { path: "/auction", primary_methods: &[Method::POST], @@ -1815,6 +1824,55 @@ mod tests { ); } + #[test] + fn dispatch_trace_route_is_disabled_by_default() { + let router = test_router(); + let response = route(&router, empty_request(Method::GET, "/_ts/trace")); + + assert_eq!( + response.status(), + StatusCode::NOT_FOUND, + "disabled trace route should return 404" + ); + assert!( + response.headers().get(header::SET_COOKIE).is_none(), + "disabled trace route should not set a cookie" + ); + } + + #[test] + fn dispatch_trace_route_arms_cookie_and_redirects() { + let mut settings = test_settings(); + settings.debug.trace_route_enabled = true; + let state = app_state_for_settings(settings); + let router = TrustedServerApp::routes_for_state(&state); + let response = route(&router, empty_request(Method::GET, "/_ts/trace")); + + assert_eq!( + response.status(), + StatusCode::FOUND, + "enabled trace route should redirect to root" + ); + assert_eq!( + response + .headers() + .get(header::LOCATION) + .and_then(|v| v.to_str().ok()), + Some("/"), + "trace route should redirect to /" + ); + let set_cookie = response + .headers() + .get(header::SET_COOKIE) + .expect("should set trace cookie") + .to_str() + .expect("should render set-cookie as utf-8"); + assert!( + set_cookie.starts_with("ts-trace=1;"), + "trace route should arm the ts-trace cookie" + ); + } + #[test] fn dispatch_set_tester_sets_cookie_on_configured_domain() { let mut settings = test_settings(); diff --git a/crates/trusted-server-adapter-fastly/src/backend.rs b/crates/trusted-server-adapter-fastly/src/backend.rs index f2ff5d9e5..db55aa07b 100644 --- a/crates/trusted-server-adapter-fastly/src/backend.rs +++ b/crates/trusted-server-adapter-fastly/src/backend.rs @@ -328,10 +328,11 @@ impl<'a> BackendConfig<'a> { /// Ensure a dynamic backend exists for this configuration and return its name. /// - /// The name is a collision-resistant function of the complete backend spec - /// (see `Self::compute_name`), so different specs — for example, different - /// timeout values — always produce different backend registrations and a - /// tight deadline cannot be silently widened by an earlier registration. + /// The backend name is derived from the scheme, host, port, certificate + /// setting, `first_byte_timeout`, and `between_bytes_timeout` to avoid + /// collisions. Different timeout values produce different backend + /// registrations so that a tight deadline cannot be silently widened by an + /// earlier registration. /// /// # Errors /// diff --git a/crates/trusted-server-adapter-spin/src/app.rs b/crates/trusted-server-adapter-spin/src/app.rs index 960bafc41..4f4ba5133 100644 --- a/crates/trusted-server-adapter-spin/src/app.rs +++ b/crates/trusted-server-adapter-spin/src/app.rs @@ -28,6 +28,7 @@ use trusted_server_core::request_signing::{ handle_trusted_server_discovery, handle_verify_signature, }; use trusted_server_core::settings::Settings; +use trusted_server_core::trace_cookie::handle_trace_mode; use crate::middleware::{AuthMiddleware, FinalizeResponseMiddleware, NormalizeMiddleware}; use crate::platform::build_runtime_services; @@ -142,7 +143,7 @@ const LEGACY_ADMIN_DENY_METHODS: &[Method] = &[ Method::DELETE, ]; -fn named_fallback_paths() -> [(&'static str, &'static [Method]); 13] { +fn named_fallback_paths() -> [(&'static str, &'static [Method]); 14] { [ ("/.well-known/trusted-server.json", &[Method::GET]), ("/verify-signature", &[Method::POST]), @@ -150,6 +151,7 @@ fn named_fallback_paths() -> [(&'static str, &'static [Method]); 13] { ("/_ts/admin/keys/deactivate", &[Method::POST]), ("/admin/keys/rotate", LEGACY_ADMIN_DENY_METHODS), ("/admin/keys/deactivate", LEGACY_ADMIN_DENY_METHODS), + ("/_ts/trace", &[Method::GET]), ("/auction", &[Method::POST]), (PAGE_BIDS_PATH, &[Method::GET, Method::OPTIONS]), (PAGE_BIDS_LEGACY_PATH, &[Method::GET, Method::OPTIONS]), @@ -551,6 +553,21 @@ fn build_router(state: &Arc) -> RouterService { } }; + // GET /_ts/trace — render-trace toggle: arms/disarms the ts-trace + // cookie and redirects to `/`. Gated by [debug] trace_route_enabled + // (404 when off). + let s = Arc::clone(&state); + let trace_mode_handler = move |ctx: RequestContext| { + let s = Arc::clone(&s); + async move { + let req = ctx.into_request(); + Ok::( + handle_trace_mode(&s.settings, req.uri().query()) + .unwrap_or_else(|e| http_error(&e)), + ) + } + }; + // GET /_ts/page-bids — SPA re-auction endpoint. let s = Arc::clone(&state); let page_bids_handler = move |ctx: RequestContext| { @@ -758,6 +775,7 @@ fn build_router(state: &Arc) -> RouterService { // credentials and key-management payloads to the origin. .post("/_ts/admin/keys/rotate", admin_not_supported_handler) .post("/_ts/admin/keys/deactivate", admin_not_supported_handler) + .get("/_ts/trace", trace_mode_handler) .post("/auction", auction_handler) .get(PAGE_BIDS_PATH, page_bids_handler.clone()) .route(PAGE_BIDS_PATH, Method::OPTIONS, page_bids_options_handler) diff --git a/crates/trusted-server-core/benches/html_processor_bench.rs b/crates/trusted-server-core/benches/html_processor_bench.rs index 96eec2f1f..7c1303dd4 100644 --- a/crates/trusted-server-core/benches/html_processor_bench.rs +++ b/crates/trusted-server-core/benches/html_processor_bench.rs @@ -13,6 +13,7 @@ fn make_config() -> HtmlProcessorConfig { ad_bids_state: std::sync::Arc::new(std::sync::Mutex::new(None)), max_buffered_body_bytes: 16 * 1024 * 1024, gpt_diagnostics: None, + suppress_datadome_client_side_tag: false, } } diff --git a/crates/trusted-server-core/src/auction/endpoints.rs b/crates/trusted-server-core/src/auction/endpoints.rs index c4af6fd3d..5f3323dbe 100644 --- a/crates/trusted-server-core/src/auction/endpoints.rs +++ b/crates/trusted-server-core/src/auction/endpoints.rs @@ -587,10 +587,11 @@ mod tests { use crate::consent::types::ConsentContext; use crate::openrtb::Uid; use crate::platform::test_support::{ - NoopBackend, NoopConfigStore, NoopGeo, NoopHttpClient, NoopSecretStore, noop_services, + NoopBackend, NoopConfigStore, NoopGeo, NoopHttpClient, NoopSecretStore, StubHttpClient, + noop_services, }; - use crate::platform::{ClientInfo, PlatformResponse}; - use crate::test_support::tests::create_test_settings; + use crate::platform::{ClientInfo, PlatformHttpClient, PlatformHttpRequest, PlatformResponse}; + use crate::test_support::tests::{crate_test_settings_str, create_test_settings}; use base64::Engine as _; use base64::engine::general_purpose::STANDARD as BASE64; use serde_json::json; @@ -675,6 +676,124 @@ mod tests { } } + /// Provider used to prove that direct `/auction` remains available when + /// publisher server-side ad templates are disabled. + struct TemplateSwitchProbeProvider { + calls: Arc>, + } + + #[async_trait::async_trait(?Send)] + impl AuctionProvider for TemplateSwitchProbeProvider { + fn provider_name(&self) -> &'static str { + "template_switch_probe" + } + + async fn request_bids( + &self, + _request: &AuctionRequest, + context: &AuctionContext<'_>, + ) -> Result> { + *self.calls.lock().expect("should lock provider call count") += 1; + let request = Request::builder() + .method("POST") + .uri("https://bidder.example/auction") + .body(EdgeBody::empty()) + .expect("should build probe provider request"); + let pending = context + .services + .http_client() + .send_async(PlatformHttpRequest::new( + request, + "template-switch-probe-backend", + )) + .await + .change_context(TrustedServerError::Auction { + message: "probe provider launch failed".to_string(), + })?; + Ok(ProviderRequestOutcome::pending(pending)) + } + + async fn parse_response( + &self, + _response: PlatformResponse, + _response_time_ms: u64, + ) -> Result> { + Ok(AuctionResponse::success( + self.provider_name(), + Vec::new(), + 0, + )) + } + + fn timeout_ms(&self) -> u32 { + 100 + } + + fn backend_name(&self, _services: &RuntimeServices, _timeout_ms: u32) -> Option { + Some("template-switch-probe-backend".to_string()) + } + } + + #[tokio::test] + async fn direct_auction_remains_available_when_templates_are_disabled() { + let settings_toml = format!( + "{}\n[auction]\nenabled = true\nproviders = [\"template_switch_probe\"]\n\n[creative_opportunities]\nenabled = false\ngam_network_id = \"12345\"\n", + crate_test_settings_str() + ); + let settings = Settings::from_toml(&settings_toml) + .expect("should parse settings with disabled templates"); + let calls = Arc::new(Mutex::new(0)); + let mut orchestrator = AuctionOrchestrator::new(settings.auction.clone()); + orchestrator.register_provider(Arc::new(TemplateSwitchProbeProvider { + calls: Arc::clone(&calls), + })); + + let stub = Arc::new(StubHttpClient::new()); + stub.push_response(200, b"probe response".to_vec()); + let services = RuntimeServices::builder() + .config_store(Arc::new(NoopConfigStore)) + .secret_store(Arc::new(NoopSecretStore)) + .kv_store(Arc::new(edgezero_core::key_value_store::NoopKvStore)) + .backend(Arc::new(NoopBackend)) + .http_client(Arc::clone(&stub) as Arc) + .geo(Arc::new(NoopGeo)) + .client_info(ClientInfo::default()) + .build(); + let ec_context = make_ec_context(Jurisdiction::NonRegulated, None); + let body = json!({ + "adUnits": [{ + "code": "div-gpt-ad-1", + "mediaTypes": { "banner": { "sizes": [[300, 250]] } } + }] + }); + let req = Request::builder() + .method("POST") + .uri("https://test-publisher.com/auction") + .body(EdgeBody::from( + serde_json::to_vec(&body).expect("should serialize body"), + )) + .expect("should build auction request"); + + let response = handle_auction( + &settings, + &orchestrator, + None, + None, + &ec_context, + &services, + req, + ) + .await + .expect("direct auction should remain available"); + + assert_eq!( + *calls.lock().expect("should lock provider call count"), + 1, + "disabling publisher templates must not disable direct /auction" + ); + assert_eq!(response.status(), StatusCode::OK); + } + #[tokio::test] async fn auction_endpoint_consent_gate_returns_no_bid_without_contacting_providers() { // GDPR/unknown jurisdiction lacking effective TCF Purpose 1 must not run diff --git a/crates/trusted-server-core/src/auction/orchestrator.rs b/crates/trusted-server-core/src/auction/orchestrator.rs index 1552ee4fd..28cb866a9 100644 --- a/crates/trusted-server-core/src/auction/orchestrator.rs +++ b/crates/trusted-server-core/src/auction/orchestrator.rs @@ -3406,4 +3406,150 @@ mod tests { "Price should be preserved" ); } + + #[test] + fn parallel_duplicate_backend_name_fails_second_provider_attributably() { + futures::executor::block_on(async { + // Two providers that canonicalize to the SAME backend name (e.g. two + // auction providers behind one gateway origin). The correlation map + // keys on backend name, so the second must not silently overwrite + // the first — it must fail attributably so no bid is misparsed or + // lost. + let stub = Arc::new(StubHttpClient::new()); + stub.push_response(200, b"{}".to_vec()); // provider-a send_async + stub.push_response(200, b"{}".to_vec()); // provider-b send_async (dropped after guard) + let services = build_services_with_http_client(stub); + // SAFETY: `Box::leak` creates a `'static` reference for test use only. + // The leaked allocation is bounded to the test process lifetime. + let services: &'static RuntimeServices = Box::leak(Box::new(services)); + + let config = AuctionConfig { + enabled: true, + providers: vec!["provider-a".to_string(), "provider-b".to_string()], + timeout_ms: 2000, + mediator: None, + ..Default::default() + }; + let mut orchestrator = AuctionOrchestrator::new(config); + orchestrator.register_provider(Arc::new(StubAuctionProvider { + name: "provider-a", + backend: "shared-backend", + })); + orchestrator.register_provider(Arc::new(StubAuctionProvider { + name: "provider-b", + backend: "shared-backend", + })); + + let request = create_test_auction_request(); + let settings = create_test_settings(); + let req = http::Request::builder() + .method(http::Method::GET) + .uri("https://example.com/test") + .body(edgezero_core::body::Body::empty()) + .expect("should build request"); + let context = AuctionContext { + settings: &settings, + request: &req, + timeout_ms: 2000, + provider_responses: None, + services, + }; + + let result = orchestrator + .run_auction(&request, &context) + .await + .expect("should complete auction despite the name collision"); + + assert_eq!( + result.provider_responses.len(), + 2, + "should account for both providers" + ); + let provider_a = result + .provider_responses + .iter() + .find(|r| r.provider == "provider-a") + .expect("should have provider-a response"); + let provider_b = result + .provider_responses + .iter() + .find(|r| r.provider == "provider-b") + .expect("should have provider-b response"); + assert_eq!( + provider_a.status, + BidStatus::Success, + "the first provider on the shared name should launch and succeed" + ); + assert_eq!( + provider_b.status, + BidStatus::Error, + "the second provider on the shared name should fail attributably, not be dropped" + ); + }); + } + + #[test] + fn dispatched_duplicate_backend_name_fails_second_provider_attributably() { + futures::executor::block_on(async { + // Same collision defense on the dispatch/collect path. + let stub = Arc::new(StubHttpClient::new()); + stub.push_response(200, b"{}".to_vec()); // provider-a send_async + stub.push_response(200, b"{}".to_vec()); // provider-b send_async (dropped after guard) + let services = build_services_with_http_client(stub); + // SAFETY: `Box::leak` creates a `'static` reference for test use only. + // The leaked allocation is bounded to the test process lifetime. + let services: &'static RuntimeServices = Box::leak(Box::new(services)); + + let config = AuctionConfig { + enabled: true, + providers: vec!["provider-a".to_string(), "provider-b".to_string()], + timeout_ms: 2000, + mediator: None, + ..Default::default() + }; + let mut orchestrator = AuctionOrchestrator::new(config); + orchestrator.register_provider(Arc::new(StubAuctionProvider { + name: "provider-a", + backend: "shared-backend", + })); + orchestrator.register_provider(Arc::new(StubAuctionProvider { + name: "provider-b", + backend: "shared-backend", + })); + + let request = create_test_auction_request(); + let settings = create_test_settings(); + let req = http::Request::builder() + .method(http::Method::GET) + .uri("https://example.com/test") + .body(edgezero_core::body::Body::empty()) + .expect("should build request"); + let context = AuctionContext { + settings: &settings, + request: &req, + timeout_ms: 2000, + provider_responses: None, + services, + }; + + let dispatched = match orchestrator.dispatch_auction(&request, &context).await { + DispatchAuctionOutcome::Dispatched(dispatched) => dispatched, + _ => panic!("should dispatch the first provider despite the name collision"), + }; + let result = orchestrator + .collect_dispatched_auction(dispatched, services, &context) + .await; + + let provider_b = result + .provider_responses + .iter() + .find(|r| r.provider == "provider-b") + .expect("should have provider-b response"); + assert_eq!( + provider_b.status, + BidStatus::Error, + "the second provider on the shared name should fail attributably, not be dropped" + ); + }); + } } diff --git a/crates/trusted-server-core/src/auction/types.rs b/crates/trusted-server-core/src/auction/types.rs index f61334787..f1630c346 100644 --- a/crates/trusted-server-core/src/auction/types.rs +++ b/crates/trusted-server-core/src/auction/types.rs @@ -288,6 +288,28 @@ pub struct Bid { pub metadata: HashMap, } +/// Length of the hex-encoded creative trace hash. +const ADM_TRACE_HASH_LEN: usize = 16; + +/// Compute the trace hash for delivered creative markup. +#[must_use] +pub fn adm_trace_hash(adm: &str) -> String { + use sha2::{Digest as _, Sha256}; + + let digest = Sha256::digest(adm.as_bytes()); + let mut hex = hex::encode(digest); + hex.truncate(ADM_TRACE_HASH_LEN); + hex +} + +impl Bid { + /// Trace hash of this bid's creative markup, when present. + #[must_use] + pub fn creative_trace_hash(&self) -> Option { + self.creative.as_deref().map(adm_trace_hash) + } +} + /// Per-provider summary included in the auction response. #[derive(Debug, Clone, Serialize, Deserialize)] pub struct ProviderSummary { diff --git a/crates/trusted-server-core/src/config.rs b/crates/trusted-server-core/src/config.rs index cdee3b222..da82e7581 100644 --- a/crates/trusted-server-core/src/config.rs +++ b/crates/trusted-server-core/src/config.rs @@ -155,7 +155,9 @@ fn validate_enabled_integrations( validate_integration::(settings, "sourcepoint")?; validate_integration::(settings, "osano")?; validate_integration::(settings, "google_tag_manager")?; - validate_integration::(settings, "datadome")?; + if let Some(config) = settings.integration_config::("datadome")? { + crate::integrations::datadome::DataDomeIntegration::validate_config_for_startup(config)?; + } validate_integration::(settings, "gpt")?; validate_integration::(settings, "gpt_diagnostics")?; @@ -321,10 +323,37 @@ formats = [{ width = 300, height = 250 }] fn absent_gam_unit_template_is_accepted_by_legacy_schema() { let creative_opportunities = serialized_creative_opportunities(None); + assert!( + creative_opportunities.get("enabled").is_none(), + "default template switch should be omitted for legacy binaries" + ); serde_json::from_value::(creative_opportunities) .expect("should accept absent GAM unit template"); } + #[test] + fn disabled_creative_opportunities_flag_is_visible_to_legacy_schema() { + let mut toml = crate_test_settings_str(); + toml.push_str( + r#" + +[creative_opportunities] +enabled = false +gam_network_id = "99999" +"#, + ); + let app_config: TrustedServerAppConfig = + toml::from_str(&toml).expect("should deserialize app config wrapper"); + let creative_opportunities = serde_json::to_value(app_config) + .expect("should serialize app config wrapper") + .get("creative_opportunities") + .cloned() + .expect("should contain creative opportunities"); + + serde_json::from_value::(creative_opportunities) + .expect_err("legacy binaries should reject an explicit disabled switch"); + } + #[test] fn deploy_validation_rejects_placeholders() { let settings = Settings::from_toml( @@ -404,6 +433,44 @@ password = "production-admin-password-32-bytes" ); } + #[test] + fn deploy_validation_rejects_invalid_datadome_test_bypass() { + for (enable_protection, store, name, expected_message) in [ + ( + false, + "ts_secrets", + "datadome_test_bypass", + "requires enable_protection", + ), + (true, "", "datadome_test_bypass", "credential_secret_store"), + (true, "ts_secrets", "", "credential_secret_name"), + ] { + let mut settings = valid_settings(); + settings + .integrations + .insert_config( + "datadome", + &serde_json::json!({ + "enabled": true, + "enable_protection": enable_protection, + "protection_test_bypass": { + "enabled": true, + "credential_secret_store": store, + "credential_secret_name": name, + }, + }), + ) + .expect("should insert DataDome config"); + + let err = validate_settings_for_deploy(&settings) + .expect_err("should reject invalid DataDome test bypass"); + assert!( + format!("{err:?}").contains(expected_message), + "error should mention the invalid bypass setting: {err:?}" + ); + } + } + #[test] fn validate_trait_reports_deploy_errors() { let mut settings = valid_settings(); diff --git a/crates/trusted-server-core/src/constants.rs b/crates/trusted-server-core/src/constants.rs index e1152b1e7..828311f12 100644 --- a/crates/trusted-server-core/src/constants.rs +++ b/crates/trusted-server-core/src/constants.rs @@ -5,6 +5,7 @@ pub const COOKIE_TS_EC: &str = "ts-ec"; /// JSON array of Extended User IDs (`[{ source, uids }]`) from identity providers. pub const COOKIE_TS_EIDS: &str = "ts-eids"; pub const COOKIE_TS_TESTER: &str = "ts-tester"; +pub const COOKIE_TS_TRACE: &str = "ts-trace"; pub const COOKIE_SHAREDID: &str = "sharedId"; pub const HEADER_X_PUB_USER_ID: HeaderName = HeaderName::from_static("x-pub-user-id"); diff --git a/crates/trusted-server-core/src/creative_opportunities.rs b/crates/trusted-server-core/src/creative_opportunities.rs index e44b0cbcf..b6950a6eb 100644 --- a/crates/trusted-server-core/src/creative_opportunities.rs +++ b/crates/trusted-server-core/src/creative_opportunities.rs @@ -183,10 +183,27 @@ fn derive_section(path: &str, section_root: &str, section_segment: usize) -> Str } } +const fn default_enabled() -> bool { + true +} + +const fn is_default_enabled(value: &bool) -> bool { + *value == default_enabled() +} + /// Top-level configuration for the creative opportunities system. #[derive(Debug, Clone, Deserialize, Serialize)] #[serde(deny_unknown_fields)] pub struct CreativeOpportunitiesConfig { + /// Enables server-side ad template delivery on publisher HTML and page-bids requests. + /// + /// This does not disable the direct `POST /auction` endpoint. The default is + /// `true` so existing creative-opportunity configurations retain their behavior. + #[serde( + default = "default_enabled", + skip_serializing_if = "is_default_enabled" + )] + pub enabled: bool, /// GAM network ID used to build default unit paths. pub gam_network_id: String, /// Maximum time in milliseconds to wait for the server-side auction before @@ -244,7 +261,7 @@ pub struct CreativeOpportunitiesConfig { /// [`section_root`](Self::section_root) are omitted. #[serde(default, skip_serializing_if = "Option::is_none")] pub section_segment: Option, - /// Slot templates. Empty vec = feature disabled (no auction fired, no globals injected). + /// Slot templates. An empty vec or `enabled = false` disables template delivery. #[serde(default, deserialize_with = "vec_from_seq_or_map")] pub slot: Vec, } @@ -333,6 +350,12 @@ impl CreativeOpportunitiesConfig { for slot in &self.slot { slot.validate_runtime()?; + if slot.providers.aps.is_some() { + log::warn!( + "creative opportunity slot '{}': providers.aps is retained only for configuration compatibility and is ignored by APS OpenRTB", + slot.id + ); + } } if self @@ -1143,12 +1166,39 @@ mod tests { assert_eq!(derive_section("/%%%/x", "home", 0), "_"); } + #[test] + fn enabled_defaults_true_and_is_omitted_from_serialized_config() { + let config = make_config_with_section_template(None); + assert!( + config.enabled, + "template delivery should default to enabled" + ); + let value = serde_json::to_value(&config).expect("should serialize config"); + assert!( + value.get("enabled").is_none(), + "default enabled value should be omitted for rollback compatibility" + ); + } + + #[test] + fn disabled_template_switch_is_serialized() { + let mut config = make_config_with_section_template(None); + config.enabled = false; + let value = serde_json::to_value(&config).expect("should serialize config"); + assert_eq!( + value.get("enabled"), + Some(&serde_json::Value::Bool(false)), + "explicitly disabled template delivery must remain in config blobs" + ); + } + fn make_config_with_section_template( section_root: Option<&str>, ) -> CreativeOpportunitiesConfig { let mut slot = make_slot("ad-header-0", vec!["/news/*"]); slot.gam_unit_path = Some("/{network_id}/example/{section}".to_string()); CreativeOpportunitiesConfig { + enabled: true, gam_network_id: "99999".to_string(), auction_timeout_ms: None, price_granularity: PriceGranularity::default(), @@ -1546,6 +1596,7 @@ mod tests { // Older binaries deserialize this struct with `deny_unknown_fields`, so // a pushed config blob must not carry `"section_root": null`. let config = CreativeOpportunitiesConfig { + enabled: true, gam_network_id: "99999".to_string(), auction_timeout_ms: None, price_granularity: PriceGranularity::default(), diff --git a/crates/trusted-server-core/src/ec/prebid_eids.rs b/crates/trusted-server-core/src/ec/prebid_eids.rs index 2f99c8182..160be5d64 100644 --- a/crates/trusted-server-core/src/ec/prebid_eids.rs +++ b/crates/trusted-server-core/src/ec/prebid_eids.rs @@ -29,6 +29,10 @@ const MAX_EIDS_COOKIE_BYTES: usize = 8 * 1024; struct LegacyCookieEid { source: String, id: String, + #[allow( + dead_code, + reason = "legacy cookie field is deserialized for compatibility but not emitted" + )] atype: i32, } diff --git a/crates/trusted-server-core/src/html_processor.rs b/crates/trusted-server-core/src/html_processor.rs index 889234b56..4c827ace0 100644 --- a/crates/trusted-server-core/src/html_processor.rs +++ b/crates/trusted-server-core/src/html_processor.rs @@ -13,6 +13,7 @@ use lol_html::{ text, }; +use crate::integrations::datadome::{DATADOME_INTEGRATION_ID, DataDomeClientTagSuppressed}; use crate::integrations::gpt_diagnostics::GptDiagnosticsRequestDecision; use crate::integrations::{ AttributeRewriteOutcome, IntegrationAttributeContext, IntegrationDocumentState, @@ -175,6 +176,8 @@ pub struct HtmlProcessorConfig { pub max_buffered_body_bytes: usize, /// Request-scoped conditional diagnostics delivery decision. pub gpt_diagnostics: Option, + /// Whether to omit Trusted Server's automatic `DataDome` client-side tag. + pub suppress_datadome_client_side_tag: bool, } impl HtmlProcessorConfig { @@ -196,6 +199,7 @@ impl HtmlProcessorConfig { ad_bids_state: std::sync::Arc::new(std::sync::Mutex::new(None)), max_buffered_body_bytes: settings.publisher.max_buffered_body_bytes, gpt_diagnostics: None, + suppress_datadome_client_side_tag: false, } } @@ -223,6 +227,13 @@ impl HtmlProcessorConfig { self.gpt_diagnostics = decision; self } + + /// Attach the request-scoped `DataDome` client-tag suppression decision. + #[must_use] + pub fn with_datadome_client_tag_suppression(mut self, suppress: bool) -> Self { + self.suppress_datadome_client_side_tag = suppress; + self + } } /// Create an HTML processor with URL replacement and integration hooks. @@ -235,6 +246,9 @@ impl HtmlProcessorConfig { pub fn create_html_processor(config: HtmlProcessorConfig) -> impl StreamProcessor { let post_processors = config.integrations.html_post_processors(); let document_state = IntegrationDocumentState::default(); + if config.suppress_datadome_client_side_tag { + document_state.get_or_insert_with(DATADOME_INTEGRATION_ID, || DataDomeClientTagSuppressed); + } // Simplified URL patterns structure - stores only core data and generates variants on-demand struct UrlPatterns { @@ -692,6 +706,7 @@ mod tests { ad_bids_state: std::sync::Arc::new(std::sync::Mutex::new(None)), max_buffered_body_bytes: 16 * 1024 * 1024, gpt_diagnostics: None, + suppress_datadome_client_side_tag: false, } } @@ -950,6 +965,46 @@ mod tests { assert_eq!(config.request_scheme, "https"); } + #[test] + fn suppressed_datadome_tag_is_not_injected_into_processed_html() { + let mut settings = create_test_settings(); + settings + .integrations + .insert_config( + "datadome", + &json!({ + "enabled": true, + "client_side_key": "test-client-key", + }), + ) + .expect("should configure DataDome integration"); + let registry = IntegrationRegistry::new(&settings) + .expect("should create integration registry with DataDome"); + let config = HtmlProcessorConfig::from_settings( + &settings, + ®istry, + "origin.example.com", + "test.example.com", + "https", + ) + .with_datadome_client_tag_suppression(true); + let mut processor = create_html_processor(config); + + let output = processor + .process_chunk(b"content", true) + .expect("should process HTML"); + let html = String::from_utf8(output).expect("should produce UTF-8 HTML"); + + assert!( + !html.contains("window.ddjskey"), + "should omit the DataDome client configuration" + ); + assert!( + !html.contains("/integrations/datadome/tags.js"), + "should omit the DataDome client tag URL" + ); + } + #[test] fn test_real_publisher_html() { // Test with publisher HTML from test_publisher.html @@ -1539,6 +1594,7 @@ mod tests { ad_bids_state: std::sync::Arc::new(std::sync::Mutex::new(None)), max_buffered_body_bytes: 16 * 1024 * 1024, gpt_diagnostics: None, + suppress_datadome_client_side_tag: false, }; let mut processor = create_html_processor(config); let output = processor @@ -1613,6 +1669,7 @@ mod tests { ad_bids_state: state, max_buffered_body_bytes: 16 * 1024 * 1024, gpt_diagnostics: None, + suppress_datadome_client_side_tag: false, }; let mut processor = create_html_processor(config); let output = processor @@ -1649,6 +1706,7 @@ mod tests { ad_bids_state: state, max_buffered_body_bytes: 16 * 1024 * 1024, gpt_diagnostics: None, + suppress_datadome_client_side_tag: false, }; let mut processor = create_html_processor(config); // Malformed HTML with two elements (common in CMS template pages) @@ -1684,6 +1742,7 @@ mod tests { ad_bids_state: std::sync::Arc::new(std::sync::Mutex::new(None)), max_buffered_body_bytes: 16 * 1024 * 1024, gpt_diagnostics: None, + suppress_datadome_client_side_tag: false, }; let mut processor = create_html_processor(config); let output = processor @@ -1737,6 +1796,7 @@ mod tests { ad_bids_state: state, max_buffered_body_bytes: 16 * 1024 * 1024, gpt_diagnostics: None, + suppress_datadome_client_side_tag: false, }; let mut processor = create_html_processor(config); let output = processor @@ -1764,6 +1824,7 @@ mod tests { ad_bids_state: state, max_buffered_body_bytes: 16 * 1024 * 1024, gpt_diagnostics: None, + suppress_datadome_client_side_tag: false, }; let mut processor = create_html_processor(config); let output = processor diff --git a/crates/trusted-server-core/src/integrations/adserver_mock.rs b/crates/trusted-server-core/src/integrations/adserver_mock.rs index ba3f82776..c1f36b337 100644 --- a/crates/trusted-server-core/src/integrations/adserver_mock.rs +++ b/crates/trusted-server-core/src/integrations/adserver_mock.rs @@ -319,17 +319,19 @@ impl AdServerMockProvider { }), nurl: original.and_then(|b| b.nurl.clone()), burl: original.and_then(|b| b.burl.clone()), - // The mediation response is itself `OpenRTB`, so the mediated - // bid's own `id` is this bid's identifier. Fall back to the - // original SSP bid's id when the mediator omits one. Without - // either, a mediated bid whose only `hb_adid` source is the bid - // id would lose it and never render — including APS bids, which - // carry no `ad_id` or `cache_id` for the restore to recover. - bid_id: bid["id"] - .as_str() - .filter(|id| !id.is_empty()) - .map(String::from) - .or_else(|| original.and_then(|bid| bid.bid_id.clone())), + // The original SSP bid's id wins: a typed `renderer` envelope is + // minted against it, and `build_bid_map` derives `hb_adid` from + // that pairing, so substituting the mediator's own id would key + // targeting to an id the renderer does not know. The mediated + // `OpenRTB` bid's `id` is the fallback for a mediator whose + // upstream bid carried none — without either, a bid whose only + // `hb_adid` source is the bid id loses it and never renders. + bid_id: original.and_then(|b| b.bid_id.clone()).or_else(|| { + bid["id"] + .as_str() + .filter(|id| !id.is_empty()) + .map(String::from) + }), ad_id: original.and_then(|bid| bid.ad_id.clone()), creative_id: original.and_then(|bid| bid.creative_id.clone()), renderer: original.and_then(|bid| bid.renderer.clone()), @@ -880,8 +882,8 @@ mod tests { ); assert_eq!( bid.bid_id.as_deref(), - Some("mediated-bid-001"), - "should carry the mediated OpenRTB bid id so hb_adid always has a source" + Some("source-bid-id"), + "should keep the original bid id the renderer envelope is keyed on, not the mediator's" ); assert_eq!( bid.ad_id.as_deref(), @@ -908,23 +910,31 @@ mod tests { } #[test] - fn parse_mediation_response_falls_back_to_original_bid_id() { - // A mediator that omits the per-bid `id` must not strand a pass-through - // bid whose only hb_adid source is its OpenRTB bid id. + fn parse_mediation_response_falls_back_to_mediated_bid_id() { + // The original bid carries no id of its own, so there is nothing to + // restore and no renderer envelope to stay consistent with. The mediation + // response is itself OpenRTB, so its bid `id` is the remaining hb_adid + // source — without it the bid reaches the page with no hb_adid and the + // render bridge never receives a matching request. let provider = AdServerMockProvider::new(AdServerMockConfig::default()); let mediation_response = json!({ "id": "test-auction-123", - "seatbid": [{ - "seat": "prebid", - "bid": [{ - "impid": "header-banner", - "price": 0.20, - "adm": "
Mediated Ad
", - "w": 728, - "h": 90, - "crid": "example-bidder-creative" - }] - }], + "seatbid": [ + { + "seat": "prebid", + "bid": [ + { + "id": "mediated-bid-002", + "impid": "header-banner", + "price": 0.20, + "adm": "
Mediated Ad
", + "w": 728, + "h": 90, + "crid": "example-bidder-creative", + } + ] + } + ], "cur": "USD" }); let mut bid_index = BidIndex::new(); @@ -945,7 +955,7 @@ mod tests { height: 90, nurl: None, burl: None, - bid_id: Some("019f7e2a-b45b-70b0-a2d1-b651c430700b".to_string()), + bid_id: None, ad_id: None, creative_id: None, renderer: None, @@ -959,10 +969,11 @@ mod tests { let auction_response = provider.parse_mediation_response(&mediation_response, 42, &bid_index); + let bid = &auction_response.bids[0]; assert_eq!( - auction_response.bids[0].bid_id.as_deref(), - Some("019f7e2a-b45b-70b0-a2d1-b651c430700b"), - "should restore the original SSP bid id when the mediator omits one" + bid.bid_id.as_deref(), + Some("mediated-bid-002"), + "should fall back to the mediated OpenRTB bid id when the original has none" ); } diff --git a/crates/trusted-server-core/src/integrations/aps.rs b/crates/trusted-server-core/src/integrations/aps.rs index 4fed9278d..421f6a2d7 100644 --- a/crates/trusted-server-core/src/integrations/aps.rs +++ b/crates/trusted-server-core/src/integrations/aps.rs @@ -50,6 +50,7 @@ const APS_RENDERER_CSP: &str = "default-src 'none'; sandbox allow-forms allow-po const APS_RENDERER_DOCUMENT: &str = r#" + ".to_string(), + width: 300, + height: 250, + })); + bid.nurl = None; + bid.burl = None; + let winning_bids = HashMap::from([("atf_sidebar_ad".to_string(), bid)]); + + let map = build_bid_map( + &winning_bids, + PriceGranularity::Dense, + &test_settings(), + "", + false, + ); + let obj = map["atf_sidebar_ad"] + .as_object() + .expect("should include APS bid"); + + assert_eq!(obj["hb_bidder"], "aps"); + assert_eq!(obj["hb_adid"], "selected-bid"); + assert_eq!(obj["renderer"]["type"], "aps"); + assert_eq!(obj["renderer"]["bidId"], "selected-bid"); + assert!(obj.get("adm").is_none()); + assert!(obj.get("nurl").is_none()); + assert!(obj.get("burl").is_none()); + assert!(obj.get("metadata").is_none()); + + let script = build_bids_script(&map); + assert!(!script.contains("")); + assert!(script.contains("\\u003C/script\\u003E")); + } + + #[test] + fn bid_map_falls_back_to_bid_id_when_cache_id_and_ad_id_absent() { + // Real shape for bidders that return neither a Prebid Cache UUID nor + // `adid` in the OpenRTB response, but always carry `id` (the bid's own + // identifier) per spec. Without this fallback the bid reaches the page + // with no hb_adid, so no targeting key is set and the render bridge + // never receives a matching `Prebid Request`. + let mut winning_bids = HashMap::new(); + winning_bids.insert( + "atf_sidebar_ad".to_string(), + Bid { + slot_id: "atf_sidebar_ad".to_string(), + price: Some(1.00), + currency: "USD".to_string(), + creative: None, + adomain: None, + bidder: "example-bidder".to_string(), + width: 300, + height: 250, + nurl: None, + burl: None, + bid_id: Some("019f7e2a-b45b-70b0-a2d1-b651c430700b".to_string()), + ad_id: None, + creative_id: None, + renderer: None, + cache_id: None, + cache_host: None, + cache_path: None, + metadata: Default::default(), + }, + ); + let map = build_bid_map( + &winning_bids, + PriceGranularity::Dense, + &test_settings(), + "", + false, + ); + let obj = map + .get("atf_sidebar_ad") + .expect("should have bid entry") + .as_object() + .expect("should be object"); + assert_eq!( + obj.get("hb_adid").and_then(|v| v.as_str()), + Some("019f7e2a-b45b-70b0-a2d1-b651c430700b"), + "should fall back to bid_id when cache_id and ad_id are both absent" + ); + } + + #[test] + fn bid_map_omits_cache_coordinates_for_a_blank_cache_id() { + // A blank `cacheId` loses the hb_adid precedence to `adid`/the bid + // id, so the cache gate must treat it as absent too. Otherwise the + // coordinates ship alongside a non-cache hb_adid and the Universal + // Creative fetches `?uuid=` — a guaranteed miss — rather than + // falling through to the inline adm. + let mut winning_bids = HashMap::new(); + winning_bids.insert( + "atf_sidebar_ad".to_string(), + Bid { + slot_id: "atf_sidebar_ad".to_string(), + price: Some(1.00), + currency: "USD".to_string(), + creative: None, + adomain: None, + bidder: "example-bidder".to_string(), + width: 300, + height: 250, + nurl: None, + burl: None, + bid_id: Some("019f7e2a-b45b-70b0-a2d1-b651c430700b".to_string()), + ad_id: Some("creative-123".to_string()), + creative_id: None, + renderer: None, + cache_id: Some(String::new()), + cache_host: Some("cache.example.com".to_string()), + cache_path: Some("/cache".to_string()), + metadata: Default::default(), + }, + ); + let map = build_bid_map( + &winning_bids, + PriceGranularity::Dense, + &test_settings(), + "", + false, + ); + let obj = map + .get("atf_sidebar_ad") + .expect("should have bid entry") + .as_object() + .expect("should be object"); + assert_eq!( + obj.get("hb_adid").and_then(|v| v.as_str()), + Some("creative-123"), + "should fall back to ad_id when cache_id is blank" + ); + assert!( + obj.get("hb_cache_host").is_none(), + "should omit hb_cache_host when the cache UUID is blank" + ); + assert!( + obj.get("hb_cache_path").is_none(), + "should omit hb_cache_path when the cache UUID is blank" + ); + } + + #[test] + fn bid_map_omits_cache_coordinates_without_a_cache_id() { + // PBS reports the cache `url` and `cacheId` independently. With + // coordinates but no UUID, hb_adid holds a non-cache identifier, so + // emitting them would send the Universal Creative to + // `?uuid=` — a guaranteed miss — instead of the inline adm. + let mut winning_bids = HashMap::new(); + winning_bids.insert( + "atf_sidebar_ad".to_string(), + Bid { + slot_id: "atf_sidebar_ad".to_string(), + price: Some(1.00), + currency: "USD".to_string(), + creative: None, + adomain: None, + bidder: "example-bidder".to_string(), + width: 300, + height: 250, + nurl: None, + burl: None, + bid_id: Some("019f7e2a-b45b-70b0-a2d1-b651c430700b".to_string()), + ad_id: None, + creative_id: None, + renderer: None, + cache_id: None, + cache_host: Some("cache.example.com".to_string()), + cache_path: Some("/cache".to_string()), + metadata: Default::default(), + }, + ); + let map = build_bid_map( + &winning_bids, + PriceGranularity::Dense, + &test_settings(), + "", + false, + ); + let obj = map + .get("atf_sidebar_ad") + .expect("should have bid entry") + .as_object() + .expect("should be object"); + assert!( + obj.get("hb_cache_host").is_none(), + "should omit hb_cache_host when there is no cache UUID to look up" + ); + assert!( + obj.get("hb_cache_path").is_none(), + "should omit hb_cache_path when there is no cache UUID to look up" + ); + } + + #[test] + fn bid_map_skips_blank_cache_id_and_ad_id_for_hb_adid() { + // A bidder that emits `cacheId`/`adid` as empty strings must not win + // the precedence: an empty hb_adid is falsey on the page, so GPT skips + // the targeting key and the render bridge has nothing to match — the + // same failure as omitting hb_adid entirely. + let mut winning_bids = HashMap::new(); + winning_bids.insert( + "atf_sidebar_ad".to_string(), + Bid { + slot_id: "atf_sidebar_ad".to_string(), + price: Some(1.00), + currency: "USD".to_string(), + creative: None, + adomain: None, + bidder: "example-bidder".to_string(), + width: 300, + height: 250, + nurl: None, + burl: None, + bid_id: Some("019f7e2a-b45b-70b0-a2d1-b651c430700b".to_string()), + ad_id: Some(String::new()), + creative_id: None, + renderer: None, + cache_id: Some(String::new()), + cache_host: None, + cache_path: None, + metadata: Default::default(), + }, + ); + let map = build_bid_map( + &winning_bids, + PriceGranularity::Dense, + &test_settings(), + "", + false, + ); + let obj = map + .get("atf_sidebar_ad") + .expect("should have bid entry") + .as_object() + .expect("should be object"); + assert_eq!( + obj.get("hb_adid").and_then(|v| v.as_str()), + Some("019f7e2a-b45b-70b0-a2d1-b651c430700b"), + "should treat blank cache_id and ad_id as absent and use bid_id" + ); + } + #[test] fn initial_document_bids_script_includes_auction_id_only_for_winning_bids() { let slot = make_slot(); let slots = [slot]; let slots_ctx = MatchedSlotsContext { matched_slots: &slots, - request_path: "/2024/01/my-article/", + request_path_and_query: "/2024/01/my-article/", }; let request_info = RequestInfo { host: "publisher.example.com".to_string(), @@ -8976,6 +9665,7 @@ mod tests { let mut settings = test_settings(); settings.auction.rewrite_creatives = true; settings.publisher.domain = "example.com".to_string(); + settings.auction.rewrite_creatives = true; let mut winning_bids = HashMap::new(); let mut bid = make_bid( @@ -9026,6 +9716,7 @@ mod tests { let mut settings = test_settings(); settings.auction.rewrite_creatives = true; settings.publisher.domain = "example.com".to_string(); + settings.auction.rewrite_creatives = true; let mut winning_bids = HashMap::new(); let mut bid = make_bid( @@ -9075,6 +9766,7 @@ mod tests { // signature locks the wrong value. let mut settings = test_settings(); settings.publisher.domain = "example.com".to_string(); + settings.auction.rewrite_creatives = true; let mut winning_bids = HashMap::new(); let mut bid = make_bid( @@ -9181,6 +9873,10 @@ mod tests { false, ); let script = build_bids_script(&map); + assert!( + !script.contains(" + +``` + +Do not alter: + +- publisher-originated DataDome script tags; +- `rewrite_sdk` behavior; +- the DataDome SDK proxy route; +- the signal collection API proxy; +- DataDome configuration serialization for non-suppressed requests; or +- injection behavior for requests without the marker. + +## Testing plan + +### Protection-filter tests + +Add or extend tests in +`crates/trusted-server-core/src/integrations/datadome/protection.rs` to verify +that the marker is attached for: + +- a matching inline IPv4 CIDR; +- a matching Config Store-backed CIDR source; +- a matching structured `ip_cidr` rule; and +- a matching structured `ip_cidr_source` rule. + +Verify that the marker is absent for: + +- a non-matching IP; +- an ASN exclusion; +- a path exclusion; +- a query-parameter exclusion; +- an excluded method; and +- an internal or integration route. + +Verify the existing protection behavior remains unchanged: IP-matched requests +continue without a Protection API call. + +### Head-injector tests + +Add tests in +`crates/trusted-server-core/src/integrations/datadome.rs` verifying that: + +- a configured client tag is omitted when suppression is active; +- a configured client tag is emitted when suppression is inactive; +- a blank client-side key remains a no-op; and +- `inject_client_side_tag = false` remains a no-op. + +### HTML pipeline tests + +Add coverage for the request-scoped value flowing through the HTML processor, +including the streaming path. Confirm that a suppressed processed HTML response +contains neither the injected `window.ddjskey` configuration nor the configured +DataDome `tags.js` script. For a suppressed HTML stream, assert the response is +private and has no surrogate cache headers. Confirm a non-suppressed HTML stream +retains its origin cache behavior. + +Confirm that publisher-originated DataDome tags remain in the output and are +still rewritten according to the existing `rewrite_sdk` behavior. + +### Fastly dispatch tests + +Add a Fastly adapter dispatch test with: + +- DataDome protection enabled; +- a client IP matching an inline exclusion; +- a configured client-side key; and +- an HTML publisher response. + +The test should verify that the request continues without a Protection API +call, the response includes the `client_tag=omitted` decision log through the +existing test logging seam where available, and the generated tag is absent. + +Also cover a non-excluded request to confirm the generated tag remains present. + +## Documentation changes + +Update `docs/guide/integrations/datadome.md` to state that IP-excluded Fastly +requests skip both: + +- server-side Protection API validation; and +- Trusted Server's automatic client-side tag injection. + +Document that this does not remove or disable publisher-originated DataDome +tags, and that non-IP exclusions do not automatically suppress the client-side +tag. + +No configuration template changes are required because this behavior has no +new setting. + +## Files expected to change + +- `crates/trusted-server-core/src/integrations/registry.rs` + - Support the internal request-scoped annotation mechanism. +- `crates/trusted-server-core/src/integrations/datadome.rs` + - Define the marker and conditionally suppress head injection. +- `crates/trusted-server-core/src/integrations/datadome/protection.rs` + - Attach the marker for IP-based scope skips and enrich the skip log. +- `crates/trusted-server-core/src/integrations/registry.rs` or the relevant + HTML context definition + - Carry the suppression decision into head injection. +- `crates/trusted-server-core/src/html_processor.rs` + - Carry the request-scoped value into HTML integration context. +- `crates/trusted-server-core/src/publisher.rs` + - Snapshot and propagate the request marker through response processing. +- `docs/guide/integrations/datadome.md` + - Document the behavior. +- Relevant unit and Fastly adapter test modules. + +The exact split between registry request annotations and HTML context plumbing +should remain minimal and should not introduce a new public configuration API. + +## Verification + +Implementation verification should use the repository's target-matched +commands: + +```bash +cargo fmt --all -- --check +cargo test-fastly +cargo test-axum +cargo test-cloudflare +cargo clippy-fastly +cargo clippy-axum +cargo clippy-cloudflare +``` + +No live production validation is required for this implementation task. Live +browser verification will be performed later through the deployment/testing +workflow. diff --git a/trusted-server.example.toml b/trusted-server.example.toml index 19ecda4a5..9bfcb9e4a 100644 --- a/trusted-server.example.toml +++ b/trusted-server.example.toml @@ -180,7 +180,19 @@ ja4_endpoint_enabled = false # in production. auction_html_comment = false +# Expose GET /_ts/trace, which toggles the `ts-trace` cookie and redirects to /. +# While the cookie is set, the TSJS overlay draws a floating panel summarising +# every traced ad slot (render path, bidder, and GAM/injected/visible state) +# plus a confirmation badge on each genuinely-rendered creative. It only +# surfaces data already present on window.tsjs, so it leaks nothing new — but +# it is off by default so the toggle route is not live on deployments that +# never asked for it. Enable only for render-verification debugging. +# trace_route_enabled = false + [creative_opportunities] +# Set to false to disable server-side ad templates while retaining slot definitions +# and direct POST /auction callers. +enabled = true gam_network_id = "123456789" # FCP is not affected by this value — body content above has already # streamed and painted before the hold begins. What this caps is the slip on