Add a host allowlist for creative asset rewriting

CHANGELOG.md

@@ -36,7 +36,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
-- `rewrite.exclude_domains` matching is now case-insensitive and ignores surrounding whitespace. Entries written with uppercase letters previously never matched and now take effect, so those hosts stop being proxied and click-wrapped, and `/first-party/sign` now rejects them with `502`; it used to sign them, or return `403` when the host was also outside `proxy.allowed_domains`. Empty and bare `"*"` `exclude_domains` entries, which never matched a host, are dropped at load with a warning. Absolute `http(s)` creative URLs that cannot be parsed (for example, a host containing a space) are now left untouched; previously the trimmed value was written back re-quoted (with `https:` prepended to `//` input, so `src=" //exa mple.example/x "` became `https://exa mple.example/x`), an unparseable quoted URL in `<style>` or `style=` was re-quoted through `cssparser::serialize_string`, and a link also gained `data-tsclick`. `/first-party/sign` now answers unparseable `http(s)` input with `unsupported url` instead of `invalid url`; both map to `502`. The first `ts config push` after upgrading shows a one-time `exclude_domains` diff for configs with mixed-case, padded, empty or bare `"*"` entries. Audit `exclude_domains` for mixed-case entries before upgrading.
+- `rewrite.exclude_domains` matching is now case-insensitive and ignores surrounding whitespace. Entries written with uppercase letters previously never matched and now take effect, so those hosts stop being proxied and click-wrapped, and `/first-party/sign` now rejects them with `502`; it used to sign them, or return `403` when the host was also outside `proxy.allowed_domains`. Empty and bare `"*"` `exclude_domains` entries, which never matched a host, are dropped at load with a warning, and other entries that can never match a host are logged with a warning. Absolute `http(s)` creative URLs that cannot be parsed (for example, a host containing a space) are now left untouched; previously the trimmed value was written back re-quoted (with `https:` prepended to `//` input, so `src=" //exa mple.example/x "` became `https://exa mple.example/x`), an unparseable quoted URL in `<style>` or `style=` was re-quoted through `cssparser::serialize_string`, and a link also gained `data-tsclick`. `/first-party/sign` now answers unparseable `http(s)` input with `unsupported url` instead of `invalid url`; both map to `502`. The first `ts config push` after upgrading shows a one-time `exclude_domains` diff for configs with mixed-case, padded, empty or bare `"*"` entries. Audit `exclude_domains` for mixed-case entries before upgrading.
 - Next.js and Google Tag Manager script rewrites now compose in streamed HTML instead of overwriting each other, preserving Pages Router JSON and App Router Flight payloads with recomputed decoded `T`-record lengths. GTM/GA Flight rewrites require both integrations to be enabled. Buffering is bounded and per document; an uncaptured data head (including inert text quoting a Flight push), an unsupported or binary push, or a size limit keeps the remaining Flight stream raw and restores queued originals, but cannot undo already-streamed rewrites. Complete serialized form state does not disable later rewriting, foreign property references no longer hide a later qualified Flight push, and placeholder-stage document fallback emits a reason-bearing warning without source data (#1206, #1207, #1208).
 - Publisher HTML is now processed when the request carries its host only in the URI (absolute-form HTTP/1.1, or HTTP/2 without a usable `Host`), instead of being returned unmodified with an "Empty request host" warning (#1118). URI userinfo is excluded from rewritten URLs, while ports and IPv6 brackets are preserved.
 - Publisher responses now map origin URLs in `Location`, `Content-Location`, `Refresh`, and `Link` headers to the serving host, and each CSP source naming the origin gains a serving-host source beside it (#1128). A redirect to the current URL under a different scheme, or a `GET`/`HEAD` redirect to the current URL that sets no cookie (or any such redirect under a `Host` override), is left unchanged to avoid a loop. `<meta>` refresh and CSP tags in the page body are not rewritten. See "Origin URLs in proxied response headers" in the configuration guide.
@@ -46,9 +46,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - TSJS-generated envelopes now send `trustedServer.params.storedRequest: false`, preventing accidental PBS stored lookups without suppressing eligible non-PBS demand. PBS filters unusable impressions after overrides; explicit `true` and omission in valid envelopes retain inline-first stored fallback. A malformed envelope disables stored fallback for the entire slot, including independent direct demand left unusable after overrides. Publisher intent survives repeated and refresh auctions. Deploy compatible server admission everywhere before serving the new JS, and retain it during rollback while cached clients remain. See the Prebid deployment guide.
 - Protocol-relative creative URLs now honor `rewrite.exclude_domains`, so excluded creative assets stay direct and excluded absolute or protocol-relative URLs submitted to `/first-party/sign` are rejected.
 - Server-side ad template bids now always carry `hb_adid` in `window.tsjs.bids`. Bidders that return neither a Prebid Cache UUID nor an `adid` previously produced no `hb_adid` at all, so no `hb_adid` GPT targeting key was set and the Universal Creative render bridge had nothing to match — the winning creative never rendered. The OpenRTB bid `id`, which is mandatory per spec, is now the last-resort source; `cache_id` and `adid` still take priority where present. Blank `cacheId`/`adid` values no longer win that precedence and emit an unusable empty `hb_adid`, and `hb_cache_host`/`hb_cache_path` are now emitted only alongside a real Prebid Cache UUID — without one they pointed the Universal Creative at a guaranteed cache miss instead of letting it fall through to the inline creative.
+- Inline SSAT/page-bids creatives no longer proxy `<link rel="preload" imagesrcset>` candidates twice. The `imagesrcset` attribute is now rewritten once, by the same handler on every creative path; previously the inline path's absolute proxy URLs were wrapped in a second `/first-party/proxy` URL.
 
 ### Added
 
+- `[rewrite] include_domains` limits creative asset rewriting to listed hosts. When non-empty, only asset URLs on matching hosts are proxied through `/first-party/proxy`, so only those hosts receive the EC ID on newly rewritten or signed asset URLs. The list is not enforced when a proxy URL is fetched, so previously issued proxy URLs can still forward the EC ID to hosts later removed from it. Other assets keep their original URL, and `/first-party/sign` returns `502` for them so the creative runtime loads them directly. Click-through links are not affected, and `exclude_domains` still wins. Entries must be host names, optionally prefixed with `*.`, dotted-quad IPv4 addresses, or bracketed IPv6 addresses; entries with a scheme, port, path, spaces or non-ASCII characters are rejected at load. Upgrading: deploy the binary first, then push a config that sets `include_domains`. Rolling back: remove `include_domains`, push the resulting config, then roll back the binary; older binaries reject the key because `[rewrite]` denies unknown fields.
 - Added `[auction].rewrite_clicks` to control creative click-through wrapping (`<a href>`/`<area href>` → signed `/first-party/click`) independently of asset rewriting. Unset (the default) follows `rewrite_creatives` for `POST /auction` and SSAT/page-bids and keeps wrapping links in HTML fetched through `/first-party/proxy`, so existing configs keep their asset and link behavior; an explicit value applies to every path. `rewrite_creatives` now governs asset URLs only; bidder `<base>` handling and, on `POST /auction`, creative TSJS injection run when either setting is on. Click wrapping is what forwards the visitor's EC ID (`ts-ec`) to landing pages, so `rewrite_clicks = false` stops that on every path and `rewrite_clicks = true` starts it for configs with `rewrite_creatives = false`. Also changed for every config with rewriting on, including the default, and for HTML fetched through `/first-party/proxy` under every config: a bidder `<base target>` is now kept with its `href` dropped, instead of removing the whole element, so raw links keep opening where the bidder intended. Upgrading: deploy the binary first, then push a config that sets `rewrite_clicks`. Rolling back: remove any explicit `rewrite_clicks` (and its environment override), push the resulting config, then roll back the binary. Older binaries tie click wrapping to `rewrite_creatives` in both directions, so rolling back turns clicks back on for `rewrite_creatives = true` with `rewrite_clicks = false`, and off for `rewrite_creatives = false` with `rewrite_clicks = true`; rolling back from any `rewrite_clicks = false` also turns click wrapping back on in HTML fetched through `/first-party/proxy`.
 - Added the `[auction].rewrite_creatives` (default `true`) and `[auction].sanitize_creatives` (default `false`) options. `rewrite_creatives` rewrites winning-bid adm to first-party endpoints across `POST /auction` and publisher SSAT/page-bids delivery (asset URL conversion to `/first-party/proxy`; bidder `<base>` handling, which drops the `href` and keeps a `target`, and creative TSJS injection on `POST /auction` only, both of which also run when `rewrite_clicks` is on). Click-through wrapping is controlled by `[auction].rewrite_clicks`, which follows `rewrite_creatives` when unset. Enabling `sanitize_creatives` strips executable markup from winning-bid adm before delivery.
 - `creative_opportunities.slot.gam_unit_path` is now a template supporting `{network_id}`, `{slot_id}`, and `{section}`, so a publisher whose ad unit varies by site section expresses it in one slot rule instead of one per (slot × section). `{section}` derives from the request path: `[creative_opportunities].section_segment` selects which path segment names the section (0-based, default `0`; set `1` for locale-prefixed URLs), and `section_root` supplies the value for paths with no such segment. `section_root` is required when a template uses `{section}`. Existing static and absent `gam_unit_path` configs are unchanged. Startup rejects a blank `gam_network_id` only when an absent/default path or `{network_id}` template consumes it. Trusted Server conservatively caps whole rendered dynamic paths at 100 UTF-8 bytes, informed by Google's 100-character per-ad-unit-code limit; an over-limit request-specific path omits that slot without failing the response. During typed/startup finalization, every placeholder-bearing template that omits `section_segment` materializes `section_segment = 0`, so an older binary rejects the blob loudly. Static and absent paths remain legacy-schema compatible only when both `section_root` and `section_segment` are omitted. Before rolling back below this feature, replace or remove dynamic paths, remove both keys, re-push and finalize the config, then roll back the binary.

crates/trusted-server-core/src/config_payload.rs

@@ -735,6 +735,43 @@ mod tests {
         );
     }
 
+    #[test]
+    fn legacy_blob_without_rewrite_include_domains_loads_with_empty_list() {
+        let data =
+            serde_json::to_value(test_settings()).expect("should serialize settings to JSON");
+        let rewrite = data
+            .get("rewrite")
+            .and_then(serde_json::Value::as_object)
+            .expect("should serialize rewrite settings as an object");
+        assert!(
+            !rewrite.contains_key("include_domains"),
+            "should omit the default include list from the payload"
+        );
+
+        let reconstructed = load_settings(&envelope_json(&test_settings()))
+            .expect("should reconstruct settings without include_domains");
+
+        assert!(
+            reconstructed.rewrite.include_domains.is_empty(),
+            "should load an empty include list from a blob without the key"
+        );
+    }
+
+    #[test]
+    fn rewrite_include_domains_survive_blob_round_trip() {
+        let mut original = test_settings();
+        original.rewrite.include_domains = vec!["*.cdn.example.com".to_owned()];
+
+        let reconstructed = load_settings(&envelope_json(&original))
+            .expect("should reconstruct settings with include_domains");
+
+        assert_eq!(
+            reconstructed.rewrite.include_domains,
+            vec!["*.cdn.example.com".to_owned()],
+            "should preserve the configured include list"
+        );
+    }
+
     #[test]
     fn disabled_rewrite_creatives_survives_blob_round_trip() {
         let mut original = test_settings();