diff --git a/CHANGELOG.md b/CHANGELOG.md index 6046612e8d..d7dcb6004a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- A consent management platform's middleware runs in the phase the publisher chooses. `cmp.inmobi`, `cmp.didomi` and `cmp.sourcepoint` may be named in a `[[fetch]]` or a `[[serve]]` entry, where each ran in the fetch phase alone. A generated configuration still places them in `[[fetch]]`. A `[[serve]]` entry naming `cmp.inmobi` for a page whose `[[fetch]]` entry names another middleware is refused, because the serve phase writes its head markup after what the fetch phase writes ahead of the script bundle, so a vendor's tag there would come ahead of the prompt. Naming `cmp.inmobi` for one page in both phases is refused, because the prompt would be written twice. - **Breaking:** auction telemetry is selected the way every other module is. `[tinybird]` is now `[analytics.tinybird]`, and `[analytics] module = "tinybird"` selects it, which replaces the `enabled` line. A configuration that still carries `[tinybird]` is refused with directions. A stored configuration written by an earlier version carries the table even where telemetry was off, so push the configuration again with this version. The secret reference is `analytics.tinybird.auction_token_secret`, the table refuses a key it does not know, and a section that selects nothing, or a module this build does not have, is refused. A row is sent only for a reader whose permissions include `analytics.ad_reporting.measure_ad_performance`, so none is sent for a reader the consent gate refuses unless they permit it. The Axum, Cloudflare and Spin adapters send no telemetry, and log a warning at startup when the section selects a module. - The DataDome module asks its Protection API about a publisher's pages beneath `/admin/` like any other page. It skipped every path beneath `/admin/` and `/_ts/admin/`, which left a publisher's own `/admin/` pages out of bot protection. - **Breaking:** `ts cache purge` asks the hosting platform to purge instead of calling the service. It takes `--service-id ` where it took `--service `, reads a Fastly API token from `FASTLY_API_TOKEN` where it read `TRUSTED_SERVER_ADMIN_PASSWORD`, and no longer takes `--username`. It runs `fastly service purge --key`, which needs Fastly CLI 14.0.0 or later, with the surrogate key it derives by the function the service tags its cached objects with, so `--all` and `--page ` purge exactly what they purged before. diff --git a/crates/cmp/didomi/src/lib.rs b/crates/cmp/didomi/src/lib.rs index 1ff33e2dd6..08483c93ba 100644 --- a/crates/cmp/didomi/src/lib.rs +++ b/crates/cmp/didomi/src/lib.rs @@ -594,7 +594,8 @@ impl DidomiIntegration { } /// Writes the browser module's settings into the head, on the pages a -/// `[[fetch]]` entry names [`MODULE`] for. +/// `[[fetch]]` or `[[serve]]` entry names [`MODULE`] for, in the phase the +/// publisher chooses. struct ClientConfig { /// The script, made once when the module registers. script: String, @@ -614,7 +615,7 @@ impl Middleware for ClientConfig { } fn phases(&self) -> &[MiddlewarePhase] { - &[MiddlewarePhase::Fetch] + &MiddlewarePhase::ALL } fn create(&self, _context: &MiddlewareContext<'_>) -> MiddlewareAction { @@ -1388,6 +1389,16 @@ mod tests { assert!(validate_proxy_path("/leading-slash-ok").is_ok()); } + #[test] + fn the_page_change_may_run_in_either_phase() { + let integration = DidomiIntegration::new(Arc::new(config())); + assert_eq!( + ClientConfig::new(&integration).phases(), + MiddlewarePhase::ALL, + "the publisher chooses the phase by the entry" + ); + } + #[test] fn head_markup_emits_proxy_path() { let custom_config = DidomiIntegrationConfig { @@ -1555,4 +1566,33 @@ mod tests { ), ); } + + /// Named in `[[serve]]` in place of `[[fetch]]`, the page change passes + /// the registry's check and gives a reader the recorded page, because a + /// page with no other middleware gets it in the same place from either + /// phase. + #[test] + fn placed_in_serve_the_page_change_gives_a_reader_the_recorded_page() { + use trusted_server_core::html_processor::test_support::{ + page_settings, place_on_every_page, processed_page, + }; + let mut settings = page_settings(include_str!("fixtures/page-change.settings.toml")); + settings.fetch = trusted_server_core::middleware::PhaseEntries::default(); + place_on_every_page(&mut settings, MiddlewarePhase::Serve, &[MODULE]); + let registry = IntegrationRegistry::with_registrations(&settings, &[super::builder()]) + .expect("the registry should accept the page change in a serve entry"); + + let page = processed_page( + &settings, + ®istry, + include_str!("fixtures/page-change.input.html"), + 8192, + ); + + assert_eq!( + page, + include_str!("fixtures/page-change.recorded.html"), + "a serve entry should give the reader the page a fetch entry does" + ); + } } diff --git a/crates/cmp/inmobi/src/lib.rs b/crates/cmp/inmobi/src/lib.rs index 96cfe0a536..cf19f02dfa 100644 --- a/crates/cmp/inmobi/src/lib.rs +++ b/crates/cmp/inmobi/src/lib.rs @@ -3,7 +3,8 @@ //! //! The prompt is the first thing a reader sees and no advertising may run //! before they answer it. Its tag is this module's middleware, written where -//! a `[[fetch]]` entry names [`MODULE`], before any vendor's tag. The +//! a `[[fetch]]` or `[[serve]]` entry names [`MODULE`], before any vendor's +//! tag, in the phase the publisher chooses. The //! middleware writes the IAB TCF stub and `InMobi`'s GPP stub ahead of //! the deferred Choice loader, so a call to `__tcfapi` or `__gpp` made before //! the prompt arrives is answered or queued rather than thrown, and each stub @@ -27,8 +28,8 @@ use trusted_server_core::settings::{IntegrationConfig, Settings}; /// The id the prompt's registration carries. const INMOBI_INTEGRATION_ID: &str = "inmobi"; -/// The name `[cmp]` selects the prompt by and a `[[fetch]]` entry names its -/// middleware by, from this crate's folder. Its settings are the table `[cmp.inmobi]`. +/// The name `[cmp]` selects the prompt by and an entry names its middleware +/// by, from this crate's folder. Its settings are the table `[cmp.inmobi]`. pub const MODULE: &str = "cmp.inmobi"; /// Hosts a Choice tag may be served from. @@ -158,10 +159,12 @@ fn validate_script_url(url: &str) -> Result<(), Report> { /// Refuse a document that selects the prompt but leaves a page served as /// `text/html` without it. /// -/// A response takes the first entry that covers it, so every `text/html` -/// entry up to the first that covers every path has to name the prompt, and -/// such an entry has to exist. A path covers every page when the home page, -/// `/`, begins with it. +/// A response takes the first entry of each phase that covers it. With the +/// prompt in `[[fetch]]` alone, every `text/html` entry up to the first that +/// covers every path has to name the prompt, and such an entry has to exist. +/// A path covers every page when the home page, `/`, begins with it. With the +/// prompt in a `[[serve]]` entry, each page has to take an entry of one phase +/// or the other that names it. fn require_an_entry(settings: &Settings) -> Result<(), Report> { let refusal = |gap: String| { Report::new(TrustedServerError::Settings { @@ -169,6 +172,32 @@ fn require_an_entry(settings: &Settings) -> Result<(), Report Ok(()), + Some(path) => Err(refusal(format!( + "a page at `{path}` takes no [[fetch]] or [[serve]] entry that names \ + `{MODULE}`, so it would be served with no consent prompt. Name it in one \ + phase's entry for that page" + ))), + }; + } let entries = settings .fetch .entries() @@ -177,9 +206,9 @@ fn require_an_entry(settings: &Settings) -> Result<(), Report = Vec::new(); @@ -211,19 +240,95 @@ fn require_an_entry(settings: &Settings) -> Result<(), Report Vec<&str> { + let mut paths = vec!["/"]; + for entry in settings + .fetch + .entries() + .iter() + .chain(settings.serve.entries()) + { + if entry.media_type != HTML_MEDIA_TYPE { + continue; + } + if let Some(path) = entry.path.as_deref() + && !paths.contains(&path) + { + paths.push(path); + } + } + paths +} + +/// Refuse a document that would write the prompt on a page after the fetch +/// phase's head markup, or in both phases. +/// +/// A serve middleware's head markup is written straight before the script +/// bundle, after what the fetch phase wrote ahead of it, so a prompt that a +/// `[[serve]]` entry names follows whatever the `[[fetch]]` entry for the +/// same page writes there. Which of those middleware write a vendor's tag is +/// not known here, so any is refused. +fn require_the_prompt_once_and_first( + settings: &Settings, +) -> Result<(), Report> { + for path in page_paths(settings) { + let serve = settings.serve.for_response(HTML_MEDIA_TYPE, path); + if !serve.iter().any(|name| name == MODULE) { + continue; + } + let fetch = settings.fetch.for_response(HTML_MEDIA_TYPE, path); + let ahead: Vec<&str> = fetch + .iter() + .map(String::as_str) + .filter(|name| *name != MODULE) + .collect(); + if !ahead.is_empty() { + return Err(Report::new(TrustedServerError::Settings { + message: format!( + "[[serve]] names `{MODULE}` for a page at `{path}`, where [[fetch]] names \ + [{}]. A serve middleware's head markup is written after what each fetch \ + middleware writes ahead of the script bundle, so a vendor's tag any of \ + those write there would come ahead of the consent prompt. Name \ + `{MODULE}` first in that [[fetch]] entry instead", + ahead.join(", ") + ), + })); + } + if fetch.iter().any(|name| name == MODULE) { + return Err(Report::new(TrustedServerError::Settings { + message: format!( + "[[fetch]] and [[serve]] both name `{MODULE}` for a page at `{path}`, so \ + the consent prompt would be written on it twice. Name it in one phase's \ + entry for that page" + ), + })); + } + } + Ok(()) +} + /// Validates the `InMobi` Choice configuration and reports whether a section /// selects the module. /// /// # Errors /// /// Returns an error when the configuration cannot be parsed, fails -/// validation, carries a script URL that should not be served, or names the -/// prompt in no entry. +/// validation, carries a script URL that should not be served, would write +/// the prompt twice or after the fetch phase's head markup, or leaves a page +/// without it. pub(crate) fn validate(settings: &Settings) -> Result> { let Some(config) = settings.module_config::(MODULE)? else { return Ok(false); }; validate_script_url(&config.script_url)?; + require_the_prompt_once_and_first(settings)?; require_an_entry(settings)?; Ok(true) } @@ -233,7 +338,8 @@ pub(crate) fn validate(settings: &Settings) -> Result Result, Report> { @@ -241,6 +347,7 @@ pub fn register( return Ok(None); }; validate_script_url(&config.script_url)?; + require_the_prompt_once_and_first(settings)?; require_an_entry(settings)?; Ok(Some( IntegrationRegistration::builder(INMOBI_INTEGRATION_ID) @@ -294,7 +401,7 @@ impl Middleware for Choice { } fn phases(&self) -> &[MiddlewarePhase] { - &[MiddlewarePhase::Fetch] + &MiddlewarePhase::ALL } fn create(&self, _context: &MiddlewareContext<'_>) -> MiddlewareAction { @@ -328,7 +435,7 @@ fn escape_attribute(value: &str) -> String { mod tests { use super::*; use trusted_server_core::html_processor::test_support::{ - assert_page_is_recorded, page_settings, + assert_page_is_recorded, page_settings, place_on_every_page, processed_page, }; use trusted_server_core::integrations::{IntegrationDocumentState, IntegrationRegistry}; use trusted_server_core::middleware::test_support::context; @@ -446,6 +553,271 @@ mod tests { } } + /// The name an entry gives [`VendorTag`], and the module that supplies + /// it, selected by `[vendor] module = "tag"`. + const VENDOR_TAG: &str = "vendor.tag"; + + /// The module that supplies [`VendorTag`]. + fn vendor_builder() -> IntegrationBuilder { + IntegrationBuilder::new( + "vendor_tag", + env!("CARGO_PKG_NAME"), + |_| { + Ok(Some( + IntegrationRegistration::builder("vendor_tag") + .without_js() + .with_middleware(Arc::new(VendorTag)) + .build(), + )) + }, + |_| Ok(true), + ) + .with_module_name(VENDOR_TAG) + } + + /// An entry of `phase` for `text/html`, under `path` where one is given, + /// naming `middleware` in order. + fn entry(phase: MiddlewarePhase, path: Option<&str>, middleware: &[&str]) -> String { + let path = path.map_or_else(String::new, |path| format!("path = \"{path}\"\n")); + let names: Vec = middleware + .iter() + .map(|name| format!("\"{name}\"")) + .collect(); + format!( + "[[{phase}]]\nmedia_type = \"text/html\"\n{path}middleware = [{}]\n\n", + names.join(", ") + ) + } + + /// A document selecting the prompt and the vendor's tag, with `entries` + /// placing them, every `[[fetch]]` entry before any `[[serve]]` one. + fn placed(entries: &[String]) -> Settings { + page_settings(&format!( + "[cmp]\nmodule = \"inmobi\"\n\n[cmp.inmobi]\nscript_url = \"{REAL}\"\n\n\ + [vendor]\nmodule = \"tag\"\n\n{}", + entries.concat() + )) + } + + /// The page a reader receives from `settings`, run through both phases, + /// or why the service refuses to start. + fn served(settings: &Settings) -> Result { + let registry = IntegrationRegistry::with_registrations( + settings, + &[super::builder(), vendor_builder()], + ) + .map_err(|refused| refused.current_context().to_string())?; + Ok(processed_page( + settings, + ®istry, + "A page

Text

", + 8192, + )) + } + + /// The prompt's loader, the last of its three parts. + fn loader() -> String { + format!("") + } + + /// Wherever the service starts with the prompt and a vendor's tag placed, + /// a reader's page carries the whole prompt ahead of the tag, and the + /// placement that would write the tag first is refused. + #[test] + fn no_placement_the_service_starts_with_puts_a_vendor_tag_ahead_of_the_prompt() { + use MiddlewarePhase::{Fetch, Serve}; + // (case, entries, whether the service starts) + let cases = [ + ( + "both in [[fetch]], the prompt first", + vec![entry(Fetch, None, &[MODULE, VENDOR_TAG])], + true, + ), + ( + "both in [[serve]], the prompt first", + vec![entry(Serve, None, &[MODULE, VENDOR_TAG])], + true, + ), + ( + "the prompt in [[fetch]], the tag in [[serve]]", + vec![ + entry(Fetch, None, &[MODULE]), + entry(Serve, None, &[VENDOR_TAG]), + ], + true, + ), + ( + "the prompt in [[serve]], the tag in [[fetch]]", + vec![ + entry(Fetch, None, &[VENDOR_TAG]), + entry(Serve, None, &[MODULE]), + ], + false, + ), + ]; + for (case, entries, starts) in cases { + let settings = placed(&entries); + match served(&settings) { + Ok(page) => { + let position = |needle: &str| { + page.find(needle) + .unwrap_or_else(|| panic!("{case}: {needle} should be written: {page}")) + }; + assert!( + position(&loader()) < position("vendor.example"), + "{case}: the reader's page carries the vendor's tag ahead of the \ + prompt: {page}" + ); + assert!(starts, "{case}: the service should refuse to start: {page}"); + } + Err(refused) => { + assert!(!starts, "{case}: the service should start: {refused}"); + assert!( + refused.contains( + "[[serve]] names `cmp.inmobi` for a page at `/`, where [[fetch]] \ + names [vendor.tag]" + ), + "{case}: {refused}" + ); + let validated = validate(&settings) + .expect_err("`ts config validate` should refuse it too") + .current_context() + .to_string(); + assert_eq!(validated, refused, "{case}"); + } + } + } + } + + /// The prompt is refused after the fetch phase's head markup on the pages + /// both phases reach and on no others, whatever paths the entries cover. + #[test] + fn a_prompt_after_the_fetch_phase_is_refused_on_the_pages_both_phases_reach() { + use MiddlewarePhase::{Fetch, Serve}; + // (case, entries, whether it is refused) + let cases = [ + ( + "the tag under /news/, the prompt in [[serve]] everywhere", + vec![ + entry(Fetch, Some("/news/"), &[VENDOR_TAG]), + entry(Serve, None, &[MODULE]), + ], + true, + ), + ( + "the tag everywhere, the prompt in [[serve]] under /sport/", + vec![ + entry(Fetch, None, &[VENDOR_TAG]), + entry(Serve, Some("/sport/"), &[MODULE]), + ], + true, + ), + ( + "the tag under /news/, the prompt in [[serve]] under /news/sport/", + vec![ + entry(Fetch, Some("/news/"), &[VENDOR_TAG]), + entry(Serve, Some("/news/sport/"), &[MODULE]), + ], + true, + ), + ( + "the tag under /news/sport/, the prompt in [[serve]] under /news/", + vec![ + entry(Fetch, Some("/news/sport/"), &[VENDOR_TAG]), + entry(Serve, Some("/news/"), &[MODULE]), + ], + true, + ), + ( + "the tag under /news/ after the prompt, the prompt in [[serve]] elsewhere", + vec![ + entry(Fetch, Some("/news/"), &[MODULE, VENDOR_TAG]), + // Keeps the serve prompt off the pages under /news/. + entry(Serve, Some("/news/"), &[VENDOR_TAG]), + entry(Serve, None, &[MODULE]), + ], + false, + ), + ]; + for (case, entries, refused) in cases { + let settings = placed(&entries); + assert_eq!( + validate(&settings).is_err(), + refused, + "{case}: {:?}", + validate(&settings) + ); + assert_eq!(register(&settings).is_err(), refused, "{case}"); + } + } + + /// Named for one page in both phases, the prompt would be written on it + /// twice, loader and all, so the service refuses to start. Named in each + /// phase for pages that never meet, it is written once on each. + #[test] + fn a_prompt_named_for_one_page_in_both_phases_is_refused() { + use MiddlewarePhase::{Fetch, Serve}; + let both = placed(&[entry(Fetch, None, &[MODULE]), entry(Serve, None, &[MODULE])]); + match served(&both) { + Ok(page) => panic!( + "the service should refuse to start, and wrote the prompt's loader {} times: \ + {page}", + page.matches(&loader()).count() + ), + Err(refused) => assert!( + refused.contains( + "[[fetch]] and [[serve]] both name `cmp.inmobi` for a page at `/`, so the \ + consent prompt would be written on it twice" + ), + "{refused}" + ), + } + assert!( + validate(&both).is_err(), + "`ts config validate` should refuse it too" + ); + + // (case, entries, whether it is refused) + let cases = [ + ( + "[[fetch]] everywhere, [[serve]] under /sport/", + vec![ + entry(Fetch, None, &[MODULE]), + entry(Serve, Some("/sport/"), &[MODULE]), + ], + true, + ), + ( + "[[fetch]] under /news/, [[serve]] everywhere", + vec![ + entry(Fetch, Some("/news/"), &[MODULE]), + entry(Serve, None, &[MODULE]), + ], + true, + ), + ( + "[[fetch]] under /news/, [[serve]] elsewhere", + vec![ + entry(Fetch, Some("/news/"), &[MODULE]), + // Keeps the serve prompt off the pages under /news/. + entry(Serve, Some("/news/"), &[VENDOR_TAG]), + entry(Serve, None, &[MODULE]), + ], + false, + ), + ]; + for (case, entries, refused) in cases { + let settings = placed(&entries); + assert_eq!( + validate(&settings).is_err(), + refused, + "{case}: {:?}", + validate(&settings) + ); + assert_eq!(register(&settings).is_err(), refused, "{case}"); + } + } + #[test] fn module_constant_is_the_crate_folder() { assert_eq!( @@ -467,11 +839,30 @@ mod tests { } #[test] - fn the_prompt_runs_in_the_fetch_phase_alone() { + fn the_prompt_may_run_in_either_phase() { assert_eq!( Choice::new(REAL, CMP_ID).phases(), - [MiddlewarePhase::Fetch], - "the tag is the same for every reader, so it belongs in the stored page" + MiddlewarePhase::ALL, + "the publisher chooses the phase by the entry" + ); + } + + #[test] + fn a_prompt_named_in_a_serve_entry_alone_is_accepted() { + let settings = page_settings(&format!( + "[cmp]\nmodule = \"inmobi\"\n\n[cmp.inmobi]\nscript_url = \"{REAL}\"\n\n\ + [[serve]]\nmedia_type = \"text/html\"\nmiddleware = [\"cmp.inmobi\"]\n" + )); + assert!( + validate(&settings).expect("should read"), + "a serve entry places the prompt" + ); + let registration = register(&settings) + .expect("should parse") + .expect("should register"); + assert_eq!( + head_markup(MiddlewarePhase::Serve, registration.middleware).len(), + 1 ); } @@ -506,7 +897,7 @@ mod tests { refused .current_context() .to_string() - .contains("no [[fetch]] entry for \"text/html\" names `cmp.inmobi`"), + .contains("no [[fetch]] or [[serve]] entry for \"text/html\" names `cmp.inmobi`"), "{refused:?}" ); assert!(register(&settings).is_err()); @@ -528,7 +919,7 @@ mod tests { refused .current_context() .to_string() - .contains("no [[fetch]] entry for \"text/html\" names `cmp.inmobi`"), + .contains("no [[fetch]] or [[serve]] entry for \"text/html\" names `cmp.inmobi`"), "{refused:?}" ); assert!(register(&settings).is_err()); @@ -612,6 +1003,48 @@ mod tests { } } + /// With the prompt in a `[[serve]]` entry, each page has to take an entry + /// of one phase or the other that names it. A serve entry limited to a + /// path, with nothing for the rest of the site, is refused naming a page + /// left out, and entries that give each page the prompt from one phase are + /// accepted. + #[test] + fn a_prompt_in_serve_has_to_reach_every_page_from_one_phase() { + let serve_entry = |path: Option<&str>, middleware: &str| { + let path = path + .map(|path| format!("path = \"{path}\"\n")) + .unwrap_or_default(); + format!( + "\n[[serve]]\nmedia_type = \"text/html\"\n{path}middleware = [\"{middleware}\"]\n" + ) + }; + + let limited = settings_with_rest(&serve_entry(Some("/news/"), MODULE)); + assert_refused( + &limited, + &[ + "a page at `/` takes no [[fetch]] or [[serve]] entry that names `cmp.inmobi`", + "with no consent prompt", + ], + ); + + let split = settings_with_rest(&format!( + "{}{}{}", + html_entry(Some("/news/"), &[MODULE]), + serve_entry(Some("/news/"), "vendor.tag"), + serve_entry(None, MODULE) + )); + assert!( + validate(&split).expect("should accept the entries"), + "the pages under /news/ get the prompt from [[fetch]] and the rest from [[serve]]" + ); + assert!( + register(&split) + .expect("should accept the entries") + .is_some() + ); + } + /// A script URL that should never reach a reader's browser is refused at /// load and at registration, with a message naming the section and the /// reason. @@ -890,4 +1323,30 @@ mod tests { ), ); } + + /// Named in `[[serve]]` in place of `[[fetch]]`, the prompt passes the + /// registry's check and gives a reader the recorded page, because a page + /// with no other middleware gets it in the same place from either phase. + #[test] + fn placed_in_serve_the_prompt_gives_a_reader_the_recorded_page() { + let mut settings = page_settings(include_str!("fixtures/page-change.settings.toml")); + settings.fetch = PhaseEntries::default(); + place_on_every_page(&mut settings, MiddlewarePhase::Serve, &[MODULE]); + let registry = IntegrationRegistry::with_registrations(&settings, &[super::builder()]) + .expect("the registry should accept the prompt in a serve entry"); + + let page = processed_page( + &settings, + ®istry, + include_str!("fixtures/page-change.input.html"), + 8192, + ); + + assert_eq!(page.matches(&loader()).count(), 1, "{page}"); + assert_eq!( + page, + include_str!("fixtures/page-change.recorded.html"), + "a serve entry should give the reader the page a fetch entry does" + ); + } } diff --git a/crates/cmp/sourcepoint/src/lib.rs b/crates/cmp/sourcepoint/src/lib.rs index 2f32250e76..4773da65f4 100644 --- a/crates/cmp/sourcepoint/src/lib.rs +++ b/crates/cmp/sourcepoint/src/lib.rs @@ -1091,8 +1091,8 @@ impl IntegrationProxy for SourcepointIntegration { } } -/// Sourcepoint's change to a page, on the pages a `[[fetch]]` entry names -/// [`MODULE`] for. It writes the browser module's settings into the head and, +/// Sourcepoint's change to a page, on the pages a `[[fetch]]` or `[[serve]]` +/// entry names [`MODULE`] for, in the phase the publisher chooses. It writes the browser module's settings into the head and, /// when `rewrite_sdk` is set, the trap on `window._sp_`, and points the /// CDN's addresses at the first-party path. struct PageChange { @@ -1117,7 +1117,7 @@ impl Middleware for PageChange { } fn phases(&self) -> &[MiddlewarePhase] { - &[MiddlewarePhase::Fetch] + &MiddlewarePhase::ALL } fn create(&self, _context: &MiddlewareContext<'_>) -> MiddlewareAction { @@ -2382,6 +2382,16 @@ mod tests { )); } + #[test] + fn the_page_change_may_run_in_either_phase() { + let integration = SourcepointIntegration::new(Arc::new(config())); + assert_eq!( + PageChange::new(integration).phases(), + MiddlewarePhase::ALL, + "the publisher chooses the phase by the entry" + ); + } + #[test] fn head_markup_emits_config_script_plus_trap_when_enabled() { let integration = SourcepointIntegration::new(Arc::new(config())); @@ -3159,4 +3169,33 @@ mod tests { ), ); } + + /// Named in `[[serve]]` in place of `[[fetch]]`, the page change passes + /// the registry's check and gives a reader the recorded page, because a + /// page with no other middleware gets it in the same place from either + /// phase. + #[test] + fn placed_in_serve_the_page_change_gives_a_reader_the_recorded_page() { + use trusted_server_core::html_processor::test_support::{ + page_settings, place_on_every_page, processed_page, + }; + let mut settings = page_settings(include_str!("fixtures/page-change.settings.toml")); + settings.fetch = trusted_server_core::middleware::PhaseEntries::default(); + place_on_every_page(&mut settings, MiddlewarePhase::Serve, &[MODULE]); + let registry = IntegrationRegistry::with_registrations(&settings, &[super::builder()]) + .expect("the registry should accept the page change in a serve entry"); + + let page = processed_page( + &settings, + ®istry, + include_str!("fixtures/page-change.input.html"), + 8192, + ); + + assert_eq!( + page, + include_str!("fixtures/page-change.recorded.html"), + "a serve entry should give the reader the page a fetch entry does" + ); + } } diff --git a/crates/trusted-server-modules/src/lib.rs b/crates/trusted-server-modules/src/lib.rs index 3baed593ee..d4bc77289c 100644 --- a/crates/trusted-server-modules/src/lib.rs +++ b/crates/trusted-server-modules/src/lib.rs @@ -219,17 +219,34 @@ mod tests { .map(|integration| integration.id) .collect(); + // A middleware that runs in several phases is listed in the one a + // tool writes it in, so what is listed for a phase is registered + // for it, and everything registered is listed somewhere. + let listed_anywhere: Vec<&str> = listed + .iter() + .filter(|row| running.contains(&row.integration)) + .map(|row| row.name) + .collect(); for phase in MiddlewarePhase::ALL { - let expected: Vec<&str> = listed + let registered = registry.middleware_in(phase); + for row in listed .iter() .filter(|row| row.phase == phase && running.contains(&row.integration)) - .map(|row| row.name) - .collect(); - assert_eq!( - registry.middleware_in(phase), - expected, - "should list for {phase} what the modules {running:?} register for it" - ); + { + assert!( + registered.contains(&row.name), + "{} is listed for {phase} and the modules {running:?} do not register it \ + for that phase: {registered:?}", + row.name + ); + } + for name in ®istered { + assert!( + listed_anywhere.contains(name), + "{name} is registered for {phase} by the modules {running:?} and listed \ + in no phase" + ); + } } checked.extend(running); } diff --git a/docs/guide/configuration.md b/docs/guide/configuration.md index 7a40821ebc..d6b9c61ad0 100644 --- a/docs/guide/configuration.md +++ b/docs/guide/configuration.md @@ -1926,7 +1926,10 @@ The modules this repository ships supply the middleware below. A deployment that runs several of them names them all in one entry's list, because a page takes one entry, and the order written is the order they run in. The table is in the order the modules register, which is the order `ts audit` writes -the names in and the order `trusted-server.example.toml` shows them in. +the names in and the order `trusted-server.example.toml` shows them in. A +consent management platform's middleware runs in either phase, as the +publisher chooses, and is listed under `[[fetch]]`, which is the phase a +generated configuration places it in. | Middleware | Phase | What it changes | | ----------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -2028,9 +2031,13 @@ See [Didomi](/guide/integrations/didomi) for the routed endpoint shapes. | `cmp_id` | Number | `10` | The IAB registered CMP identifier, from 1 to 4095, that the GPP stub answers `ping` with while the prompt loads | The table refuses a field it does not read. Selecting the module is refused -when the `[[fetch]]` entries leave any page without `cmp.inmobi`, because the -prompt would not be written there, and each entry names it before any -middleware that writes a vendor's tag. See [InMobi Choice](/guide/integrations/inmobi) +when the `[[fetch]]` and `[[serve]]` entries leave any page without +`cmp.inmobi`, because the prompt would not be written there, and each entry +names it before any middleware that writes a vendor's tag. A `[[serve]]` entry +naming it for a page whose `[[fetch]]` entry names another middleware is +refused, because a serve middleware's head markup is written after what each +fetch middleware writes ahead of the script bundle, and so is naming it for one +page in both phases, which would write it twice. See [InMobi Choice](/guide/integrations/inmobi) for the tag's parts and what the module keeps to. ### Google Tag Manager Integration diff --git a/docs/guide/integrations/inmobi.md b/docs/guide/integrations/inmobi.md index c72e54c0f6..941d76bab8 100644 --- a/docs/guide/integrations/inmobi.md +++ b/docs/guide/integrations/inmobi.md @@ -10,8 +10,8 @@ InMobi Choice is a TCF consent prompt loaded from the publisher's own InMobi account. The module writes the prompt's tag into the head of each page a -`[[fetch]]` entry for `text/html` names it for, in three parts and in this -order: +`[[fetch]]` or `[[serve]]` entry for `text/html` names it for, in three parts +and in this order: 1. The IAB TCF v2 stub, which creates the `__tcfapiLocator` frame, queues every `__tcfapi` call made before the prompt arrives, answers `ping` as a @@ -57,19 +57,29 @@ The table refuses a field it does not read. ## Placing the prompt -The prompt changes a page only where a `[[fetch]]` entry names `cmp.inmobi`, -in the fetch phase alone, because the tag is the same for every reader and -belongs in the page every reader is served from. Core writes each +The prompt changes a page only where a `[[fetch]]` or `[[serve]]` entry +names `cmp.inmobi`, in the phase the publisher chooses. The tag is the same +for every reader, so the fetch phase stores it in the page every reader is +served from, which is what `ts audit generate` writes, and the serve phase +writes it on each reader's copy instead. Core writes each middleware's head markup in the order the entry names them, and nothing advertising related may run before the reader answers the prompt, so name `cmp.inmobi` before any middleware that writes a vendor's tag, as the example -above does ahead of Google Tag Manager. +above does ahead of Google Tag Manager. A serve middleware's head markup is +written after what each fetch middleware writes ahead of the script bundle, +and the module cannot tell which middleware write a vendor's tag there, so a +`[[serve]]` entry may name the prompt only for pages whose `[[fetch]]` entry +names no other middleware. Selecting the module is refused when the settings load and by `ts config validate` if the entries leave any page served as `text/html` without the -prompt, and the refusal names those pages. A page takes the first entry that -covers it, so every entry names `cmp.inmobi`, and one with no `path` comes -after any that has one. +prompt, and the refusal names those pages. A page takes the first entry of +each phase that covers it, so each page has to take an entry of one phase or +the other that names `cmp.inmobi`. With the prompt in `[[fetch]]` alone, +every entry names it, and one with no `path` comes after any that has one. A +`[[serve]]` entry naming it for a page whose `[[fetch]]` entry names another +middleware is refused too, and so is naming it for one page in both phases, +which would write it twice. The tag is written at the start of the head, ahead of the page's own ``, and is about 3.6 KB. A browser looks for `` @@ -95,7 +105,7 @@ reader is asked before anything else runs. This module keeps to the following, and they are what every `[cmp]` module is held to. 1. The prompt is written before any vendor's tag, and the entry's order is - the head's order. + the head's order. Which phase it is written in is the publisher's choice. 2. The IAB interfaces a page may call before the prompt arrives, `__tcfapi` and `__gpp`, are stubbed ahead of the loader and queue or answer every call, and a stub steps aside where the page already has the interface.