diff --git a/CHANGELOG.md b/CHANGELOG.md index 506ec7ee13..7e2086a6a1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -96,6 +96,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Protocol-relative creative URLs now honor `rewrite.exclude_domains`, so excluded creative assets stay direct and excluded absolute or protocol-relative URLs submitted to `/first-party/sign` are rejected. - Server-side ad template bids now always carry `hb_adid` in `window.tsjs.bids`. Bidders that return neither a Prebid Cache UUID nor an `adid` previously produced no `hb_adid` at all, so no `hb_adid` GPT targeting key was set and the Universal Creative render bridge had nothing to match — the winning creative never rendered. The OpenRTB bid `id`, which is mandatory per spec, is now the last-resort source; `cache_id` and `adid` still take priority where present. Blank `cacheId`/`adid` values no longer win that precedence and emit an unusable empty `hb_adid`, and `hb_cache_host`/`hb_cache_path` are now emitted only alongside a real Prebid Cache UUID — without one they pointed the Universal Creative at a guaranteed cache miss instead of letting it fall through to the inline creative. - The origin's address in a CSS, JavaScript, JSON or other text response that core rewrites without parsing it is now replaced in one pass. A request host that contains the origin host was matched again inside its own replacement, so `https://example.com/` served on `www.example.com` came out as `https://www.www.example.com/`, and `https://publisher.example/` served on `publisher.example.localhost` over `http` as `http://publisher.example.localhost.localhost.localhost.localhost/`. +- A request that names its host only in its URI's authority, as an HTTP/2 request with no `Host` header may, is now processed. The request host was read from `Forwarded`, `X-Forwarded-Host` and `Host` alone, so such a request had an empty host and its page was returned unmodified, with no URL rewriting, middleware, injected script or auction. The authority is read last, so each of those headers still takes precedence ([#1118](https://github.com/IABTechLab/trusted-server/issues/1118)). ### Added @@ -122,6 +123,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - The stock modules' fetch middleware give as their `template_key` the text compiled into their crates that they write into a page, so a change to it stores pages again. The Google ad tag gives its head scripts, the `tsjs.adInit` bootstrap among them, and the GPT script's first-party path, Prebid its configuration script and bundle tag, Didomi its settings script, and Sourcepoint its head scripts and its CDN's first-party path. Permutive, Lockr, DataDome and Google Tag Manager give the first-party path they point a vendor's address at. - `[robots-txt]` serves a publisher's `/robots.txt` from the settings. `modules` names what makes the file, in order. `refuse_all` and `allow_all` are Trusted Server's own, and any other name is a module that contributes rules through the new `RobotsTxtContributor` extension point, declared with `with_robots_txt_contributor`. Trusted Server writes the file, so a contributor cannot produce a malformed one, a refusal cannot be loosened, and a path in `always_allow` stays open to every crawler refused the whole site. With `refuse_all` selected every response an adapter finalizes carries `X-Robots-Tag: noindex, nofollow`. A module is asked for the host the request was addressed to, never one a forwarded header names, and is given nothing of the crawler's connection. Its answer is held for that host by the loading cache until its refresh is due. A module that cannot answer has its last held answer served, and with none held, or with no rule in the whole file, the answer is `503` and never an empty file. On Fastly the file is answered before the Edge Cookie steps and the request filters. Without the section `/robots.txt` reaches the publisher's origin as before. - `ts keys rotate` and `ts keys deactivate` rotate and retire the request signing keys from an operator's machine. They write the two stores the service links as `jwks_store` and `signing_keys` through the `fastly` CLI, with the token from `FASTLY_API_TOKEN`. A private key reaches `fastly` on its standard input and never in its arguments, and a key id no platform can store is refused before anything is written. A command that cannot read the config store writes nothing, and a delete that failed part way can be run again. +- A deployment may address a request by its path before its configuration is chosen, so one host serves several publishers, each under a path of its own and from its own app-config blob. It supplies a `RequestAddressing` from `trusted_server_core::addressing`, through `trusted_server_adapter_fastly::run_with_addressing` or `TrustedServerApp::routes_with_addressing` on the other adapters. The hook answers either that a request is addressed by its host, as before, or the selector values that take the place of `{host}`, the path the request continues with and the prefix it removed. The first value whose blob exists is read and a miss on every value is answered `421`, never from a key the hook did not name. The request is routed at the continuing path with its query kept, and carries the prefix as a `RemovedPathPrefix` extension, which a deployment's own middleware puts back into its pages. Adapters that read one blob ignore the values. With no hook every request is served as before. - One Fastly service can serve several publishers. A `TS__CONFIG__KEY` entry in the `trusted_server_config` store that contains `{host}` names an app-config blob for each host, pushed with `ts config push --key `, and each request reads the blob under its own host. Each host's blob names its permission policy file and pins its SHA-256, and the push keeps each file `--permissions` supplies once, in a shared entry under `permissions_`, written before the blob, which a live push writes a minute after it writes an entry. A host with no blob is answered `421 Misdirected Request`, never from the logical store ID or from another host's blob, a malformed selector is answered `500`, and a sandbox that serves several requests keeps an application for each blob, found only by that blob's key. Without the entry, or with a selector that has no placeholder, the service reads one blob. `scripts/config-by-host-local-test.sh` runs it under Viceroy. - `ts audit generate` writes the `[[fetch]]` and `[[serve]]` entries that run the page changes of the modules it selects on every HTML page, and `trusted-server.example.toml` documents the entries with every middleware the stock modules supply. `trusted-server-modules` lists those middleware with `middleware()` and `middleware_for(...)`. - Page changes as middleware. A module registers a change it makes to a page with `.with_middleware(...)`, and the ordered `[[fetch]]` and `[[serve]]` entries of the settings say which pages each one runs on and in what order. A fetch middleware runs on the page as the origin sent it and what it writes is stored with a shared template. A serve middleware runs on each reader's copy, whether the page came from the store or from the origin, and nothing it writes is stored. Where text a middleware writes into a script would end the script, or begin `, + ) -> Result> { + let router = Self::routes_with_registrations(settings, integrations)?; + Ok(addressed_router(router, addressing)) + } + /// Build the full router with explicit settings and runtime services. /// /// Each request receives a clone of the supplied services, allowing callers diff --git a/crates/trusted-server-adapter-axum/tests/routes.rs b/crates/trusted-server-adapter-axum/tests/routes.rs index f44ce10ccd..c256d63a16 100644 --- a/crates/trusted-server-adapter-axum/tests/routes.rs +++ b/crates/trusted-server-adapter-axum/tests/routes.rs @@ -4,11 +4,15 @@ //! and self-contained. Each test builds the full `TrustedServerApp` router and //! drives it through the Tower `Service` interface. +use std::sync::Arc; + use axum::body::Body as AxumBody; use axum::http::Request; use edgezero_adapter_axum::service::EdgeZeroAxumService; use tower::{Service as _, ServiceExt as _}; use trusted_server_adapter_axum::app::TrustedServerApp; +use trusted_server_core::addressing::ByHostAddressing; +use trusted_server_core::test_support::addressing::{FirstSegment, status_and_type}; const LEGACY_ADMIN_DENY_METHODS: &[&str] = &["GET", "POST", "HEAD", "OPTIONS", "PUT", "PATCH", "DELETE"]; @@ -1088,3 +1092,29 @@ async fn robots_txt_is_served_through_the_publisher_path_with_its_tag() { "each request reaches the origin" ); } + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_addressing_continues_the_request_at_its_path_before_routing() { + let router = + TrustedServerApp::routes_with_addressing(test_settings(), &[], Arc::new(FirstSegment)) + .expect("should build router with a request addressing"); + + assert_eq!( + status_and_type(router, "/publisher.example/_ts/config.json?view=1").await, + (200, "application/json".to_owned()), + "the configuration endpoint should answer at the path the addressing continues with" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_addressed_by_host_is_routed_as_it_is() { + let router = + TrustedServerApp::routes_with_addressing(test_settings(), &[], Arc::new(ByHostAddressing)) + .expect("should build router with a request addressing"); + + assert_eq!( + status_and_type(router, "/_ts/config.json").await, + status_and_type(test_router(), "/_ts/config.json").await, + "should answer as the router with no addressing answers" + ); +} diff --git a/crates/trusted-server-adapter-cloudflare/src/app.rs b/crates/trusted-server-adapter-cloudflare/src/app.rs index 90673e1827..710969f834 100644 --- a/crates/trusted-server-adapter-cloudflare/src/app.rs +++ b/crates/trusted-server-adapter-cloudflare/src/app.rs @@ -9,6 +9,7 @@ use edgezero_core::error::EdgeError; use edgezero_core::http::{HeaderValue, Method, Request, Response, header}; use edgezero_core::router::RouterService; use error_stack::Report; +use trusted_server_core::addressing::{RequestAddressing, addressed_router}; use trusted_server_core::auction::endpoints::handle_auction; use trusted_server_core::auction::telemetry::warn_once_when_analytics_is_unsent; use trusted_server_core::auction::{ @@ -623,6 +624,29 @@ impl TrustedServerApp { Ok(build_router(&state)) } + /// Build the full application router as + /// [`TrustedServerApp::routes_with_registrations`] does, asking + /// `addressing` about every request before it is routed. + /// + /// This adapter reads one app-config blob, so the selector values the + /// addressing names are ignored. A request it addresses continues at the + /// path it gives, with its query kept, and carries the prefix it removed + /// as a [`RemovedPathPrefix`](trusted_server_core::addressing::RemovedPathPrefix) + /// extension. A request it addresses by host is routed as it is. + /// + /// # Errors + /// + /// Returns the errors [`TrustedServerApp::routes_with_registrations`] + /// returns. + pub fn routes_with_addressing( + settings: Settings, + integrations: &[IntegrationBuilder], + addressing: Arc, + ) -> Result> { + let router = Self::routes_with_registrations(settings, integrations)?; + Ok(addressed_router(router, addressing)) + } + /// Build the full router with explicit settings and runtime services. /// /// Each request receives a clone of the supplied services, allowing callers diff --git a/crates/trusted-server-adapter-cloudflare/tests/routes.rs b/crates/trusted-server-adapter-cloudflare/tests/routes.rs index 6b705551d0..811740c28e 100644 --- a/crates/trusted-server-adapter-cloudflare/tests/routes.rs +++ b/crates/trusted-server-adapter-cloudflare/tests/routes.rs @@ -4,11 +4,15 @@ //! `TrustedServerApp::routes()` builds without panicking. Does not exercise //! the platform layer or outbound network calls. +use std::sync::Arc; + use edgezero_core::app::Hooks as _; use edgezero_core::http::{Request, Response, request_builder}; use edgezero_core::router::RouterService; use trusted_server_adapter_cloudflare::app::TrustedServerApp; +use trusted_server_core::addressing::ByHostAddressing; use trusted_server_core::settings::Settings; +use trusted_server_core::test_support::addressing::{FirstSegment, status_and_type}; const LEGACY_ADMIN_DENY_METHODS: &[&str] = &["GET", "POST", "HEAD", "OPTIONS", "PUT", "PATCH", "DELETE"]; @@ -24,7 +28,7 @@ fn test_router() -> RouterService { .expect("should build router from test settings") } -/// The settings [`test_router`] is built from. +/// Explicit settings for the route tests. fn test_settings() -> Settings { Settings::from_toml( r#" @@ -1084,3 +1088,29 @@ async fn a_response_carries_no_blanket_rule_without_the_section() { assert_eq!(robots_tag(&response), None); } + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_addressing_continues_the_request_at_its_path_before_routing() { + let router = + TrustedServerApp::routes_with_addressing(test_settings(), &[], Arc::new(FirstSegment)) + .expect("should build router with a request addressing"); + + assert_eq!( + status_and_type(router, "/publisher.example/_ts/config.json?view=1").await, + (200, "application/json".to_owned()), + "the configuration endpoint should answer at the path the addressing continues with" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_addressed_by_host_is_routed_as_it_is() { + let router = + TrustedServerApp::routes_with_addressing(test_settings(), &[], Arc::new(ByHostAddressing)) + .expect("should build router with a request addressing"); + + assert_eq!( + status_and_type(router, "/_ts/config.json").await, + status_and_type(test_router(), "/_ts/config.json").await, + "should answer as the router with no addressing answers" + ); +} diff --git a/crates/trusted-server-adapter-fastly/src/app.rs b/crates/trusted-server-adapter-fastly/src/app.rs index 7409bd5e05..578e64335b 100644 --- a/crates/trusted-server-adapter-fastly/src/app.rs +++ b/crates/trusted-server-adapter-fastly/src/app.rs @@ -161,7 +161,7 @@ use crate::platform::{ /// [`Self::logical`] gives the logical stores and key. The entry point puts /// the selector the config store holds in place of the key and resolves it /// for the request before anything is built, with -/// `stores_with_stored_selector` and `stores_for_request`. +/// `stores_with_stored_selector` and `address_request`. #[derive(Debug, Clone, PartialEq, Eq)] pub(crate) struct RuntimeStoreConfig { pub(crate) config_store_name: StoreName, diff --git a/crates/trusted-server-adapter-fastly/src/config_selection.rs b/crates/trusted-server-adapter-fastly/src/config_selection.rs index 16ac668de4..d15269317c 100644 --- a/crates/trusted-server-adapter-fastly/src/config_selection.rs +++ b/crates/trusted-server-adapter-fastly/src/config_selection.rs @@ -8,6 +8,9 @@ //! placeholder is replaced by the host a request names, so one service serves //! several publishers, each from a blob of its own. //! +//! A deployment's request addressing may name selector values in place of +//! the host, and the first value whose blob the store holds is read. +//! //! A host that has no blob is refused. It is never answered from another key. use core::fmt::Display; @@ -27,7 +30,9 @@ const MAX_KEY_LEN: usize = 255; /// Why a request is not served. #[derive(Debug, Clone, PartialEq, Eq)] pub(crate) enum Refusal { - /// The request names no host, or a host the store holds no blob for. + /// The request names no host, or a host the store holds no blob for, or + /// the store holds a blob for none of the selector values a request + /// addressing named. NoConfigForHost, /// The selector, or whether the store holds the host's blob, could not /// be read. @@ -130,21 +135,59 @@ pub(crate) fn normalized_host(host: &str) -> Option { pub(crate) fn resolve_key( selector: &str, host: Option<&str>, - exists: impl FnOnce(&str) -> Result, + exists: impl FnMut(&str) -> Result, +) -> Result { + resolve_key_from(selector, host, exists) +} + +/// Resolves the key of the blob that serves a request a deployment's request +/// addressing named, from the selector values it answered. +/// +/// A selector without the placeholder is the key itself and the values are +/// not read, because a service that serves one publisher reads its one blob +/// whatever a request names. Otherwise each value takes the placeholder's +/// place in turn, as a host does, and the first whose blob the store holds +/// is the key. A value that is not a host name, or whose key would be +/// longer than the store takes, is passed over without a lookup, so no value +/// can name the logical store ID or a part of a blob. +/// +/// # Errors +/// +/// Returns [`Refusal::NoConfigForHost`] when the store holds a blob for none +/// of the values, so a request is never served from a key the hook did not +/// name. Returns [`Refusal::StoreUnavailable`] at the first lookup that +/// fails, because a later value must not serve while it is unknown whether +/// an earlier one has a blob. +pub(crate) fn resolve_addressed_key( + selector: &str, + values: &[String], + exists: impl FnMut(&str) -> Result, +) -> Result { + resolve_key_from(selector, values.iter().map(String::as_str), exists) +} + +/// The key of the first of `names` whose blob `exists`, or the selector +/// itself when it carries no placeholder. +fn resolve_key_from<'a, E: Display>( + selector: &str, + names: impl IntoIterator, + mut exists: impl FnMut(&str) -> Result, ) -> Result { if !selector.contains(HOST_PLACEHOLDER) { return Ok(selector.to_owned()); } - let key = host - .and_then(normalized_host) - .map(|host| selector.replace(HOST_PLACEHOLDER, &host)) - .filter(|key| key.chars().count() <= MAX_KEY_LEN) - .ok_or(Refusal::NoConfigForHost)?; - match exists(&key) { - Ok(true) => Ok(key), - Ok(false) => Err(Refusal::NoConfigForHost), - Err(error) => Err(Refusal::StoreUnavailable(error.to_string())), + for name in names.into_iter().filter_map(normalized_host) { + let key = selector.replace(HOST_PLACEHOLDER, &name); + if key.chars().count() > MAX_KEY_LEN { + continue; + } + match exists(&key) { + Ok(true) => return Ok(key), + Ok(false) => {} + Err(error) => return Err(Refusal::StoreUnavailable(error.to_string())), + } } + Err(Refusal::NoConfigForHost) } /// The selector in the config store linked as `store_name`, as @@ -177,8 +220,9 @@ pub(crate) fn selector_in_store(store_name: &str, logical_key: &str) -> Result Result { let store = fastly::ConfigStore::try_open(store_name) .map_err(|error| format!("failed to open config store `{store_name}`: {error}"))?; @@ -186,12 +230,14 @@ pub(crate) fn blob_exists(store_name: &str, key: &str) -> Result { .try_get(key) .map(|value| value.is_some()) .map_err(|error| { - format!("failed to look up `{key}` in config store `{store_name}`: {error}") + format!("failed to look up an entry in config store `{store_name}`: {error}") }) } #[cfg(test)] mod tests { + use std::cell::RefCell; + use super::*; /// Stands in for a store lookup that must not happen. @@ -200,10 +246,26 @@ mod tests { } /// A store holding exactly `keys`. - fn holding<'a>(keys: &'a [&'a str]) -> impl FnOnce(&str) -> Result + 'a { + fn holding<'a>(keys: &'a [&'a str]) -> impl FnMut(&str) -> Result + 'a { move |key| Ok(keys.contains(&key)) } + /// A store holding exactly `keys` that records each key it is asked + /// about. + fn recording<'a>( + keys: &'a [&'a str], + asked: &'a RefCell>, + ) -> impl FnMut(&str) -> Result + 'a { + move |key| { + asked.borrow_mut().push(key.to_owned()); + Ok(keys.contains(&key)) + } + } + + fn values(names: &[&str]) -> Vec { + names.iter().map(|name| (*name).to_owned()).collect() + } + /// A store whose selector entry holds `selector`, or nothing. fn selector_entry( selector: Option<&str>, @@ -393,6 +455,30 @@ mod tests { ); } + #[test] + fn a_selector_value_takes_the_placeholder_s_place() { + for (selector, key) in [ + ("{host}", "publisher.example"), + ("{host}_staging", "publisher.example_staging"), + ("sites.{host}", "sites.publisher.example"), + ] { + assert_eq!( + resolve_addressed_key(selector, &values(&["publisher.example"]), holding(&[key])), + Ok(key.to_owned()), + "should key the blob by the selector value under `{selector}`" + ); + } + assert_eq!( + resolve_addressed_key( + "{host}", + &values(&["Publisher.EXAMPLE."]), + holding(&["publisher.example"]) + ), + Ok("publisher.example".to_owned()), + "a selector value should be keyed as a host is" + ); + } + #[test] fn a_host_key_over_255_characters_is_not_looked_up() { let label = "a".repeat(63); @@ -412,6 +498,81 @@ mod tests { ); } + #[test] + fn the_first_selector_value_whose_blob_exists_is_read() { + let asked = RefCell::new(Vec::new()); + + let resolved = resolve_addressed_key( + "{host}", + &values(&["absent.example", "b.example", "a.example"]), + recording(&["a.example", "b.example"], &asked), + ); + + assert_eq!( + resolved, + Ok("b.example".to_owned()), + "should read the first value the store holds a blob for" + ); + assert_eq!( + *asked.borrow(), + ["absent.example", "b.example"], + "should ask in order and stop at the first blob found" + ); + } + + #[test] + fn every_selector_value_missing_is_refused_and_not_served_from_another_key() { + // The store holds a blob at the logical store ID, one for the host + // the request arrived on and one for another publisher. None of them + // may answer for values that have none. + let stored = ["trusted_server_config", "front.example", "a.example"]; + + for names in [&[][..], &["unknown.example"], &["x.example", "y.example"]] { + assert_eq!( + resolve_addressed_key("{host}", &values(names), holding(&stored)), + Err(Refusal::NoConfigForHost), + "values {names:?} with no blob should be refused" + ); + } + } + + #[test] + fn a_selector_value_that_is_not_a_host_name_is_passed_over_without_a_lookup() { + let asked = RefCell::new(Vec::new()); + + let resolved = resolve_addressed_key( + "{host}", + &values(&[ + "trusted_server_config", + "publisher.example.__edgezero_chunks.abc.0", + "", + "a.example/path", + "a.example", + ]), + recording(&["trusted_server_config", "a.example"], &asked), + ); + + assert_eq!( + resolved, + Ok("a.example".to_owned()), + "should read the one value that is a host name" + ); + assert_eq!( + *asked.borrow(), + ["a.example"], + "should look up nothing a host could not name" + ); + } + + #[test] + fn a_selector_without_the_placeholder_ignores_the_selector_values() { + assert_eq!( + resolve_addressed_key("trusted_server_config", &values(&["a.example"]), no_lookup), + Ok("trusted_server_config".to_owned()), + "a service that serves one publisher should read its one key" + ); + } + #[test] fn an_absent_entry_selects_the_logical_key() { assert_eq!( @@ -421,6 +582,27 @@ mod tests { ); } + #[test] + fn a_failed_lookup_of_a_selector_value_refuses_and_tries_no_later_value() { + let asked = RefCell::new(Vec::new()); + + let refused = resolve_addressed_key( + "{host}", + &values(&["a.example", "b.example"]), + |key: &str| { + asked.borrow_mut().push(key.to_owned()); + Err::("lookup failed") + }, + ); + + assert_eq!( + refused, + Err(Refusal::StoreUnavailable("lookup failed".to_owned())), + "should not serve while an earlier value's blob is unknown" + ); + assert_eq!(*asked.borrow(), ["a.example"], "should try no later value"); + } + #[test] fn a_well_formed_entry_is_the_selector_as_written() { for selector in ["{host}", "{host}_v2", "site_a"] { @@ -467,4 +649,28 @@ mod tests { "should refuse rather than fall back to the logical key" ); } + + #[test] + fn an_addressed_value_whose_key_is_too_long_is_passed_over() { + let label = "a".repeat(63); + let longest_host = format!("{label}.{label}.{label}.{}", "a".repeat(61)); + let asked = RefCell::new(Vec::new()); + + let resolved = resolve_addressed_key( + "{host}_v2", + &values(&[longest_host.as_str(), "a.example"]), + recording(&["a.example_v2"], &asked), + ); + + assert_eq!( + resolved, + Ok("a.example_v2".to_owned()), + "should go on to the next value" + ); + assert_eq!( + *asked.borrow(), + ["a.example_v2"], + "should never look up a key longer than the store takes" + ); + } } diff --git a/crates/trusted-server-adapter-fastly/src/lib.rs b/crates/trusted-server-adapter-fastly/src/lib.rs index 3d31f34fe8..1384ade302 100644 --- a/crates/trusted-server-adapter-fastly/src/lib.rs +++ b/crates/trusted-server-adapter-fastly/src/lib.rs @@ -3,9 +3,10 @@ //! A library with a thin binary over it, so a deployment that ships a vendor //! crate builds on this adapter rather than editing it. [`run`] serves the way //! the binary does, and [`run_with`] serves with the integration builders the -//! deployment offers composed with the built-in ones. +//! deployment offers composed with the built-in ones. [`run_with_addressing`] +//! also takes the deployment's request addressing. -use std::sync::Arc; +use std::sync::{Arc, OnceLock}; use edgezero_adapter_fastly::config_store::FastlyConfigStore as EdgeZeroFastlyConfigStore; use edgezero_adapter_fastly::request::into_core_request; @@ -20,6 +21,7 @@ use error_stack::Report; use fastly::http::Method as FastlyMethod; use fastly::{Request as FastlyRequest, Response as FastlyResponse}; +use trusted_server_core::addressing::{ByHostAddressing, RemovedPathPrefix}; use trusted_server_core::cache_policy::{EdgeCacheHeader, cache_control_headers_have_directive}; use trusted_server_core::ec::device::{DeviceModule, DeviceSignals, build_device_module}; use trusted_server_core::ec::finalize::ec_finalize_response; @@ -63,6 +65,9 @@ use crate::middleware::{HEADER_X_TS_FINALIZED, apply_finalize_headers}; use crate::platform::client_info_from_request; use crate::rate_limiter::{FastlyRateLimiter, RATE_COUNTER_NAME}; use crate::sandbox::{RetainedApp, Sandbox, SandboxCounters, ServeMode, StartupDiagnostics}; +/// Re-exported so a deployment names the request addressing it hands over +/// without depending on core directly. +pub use trusted_server_core::addressing::{Addressing, RequestAddress, RequestAddressing}; /// Re-exported so a crate registering an integration names the type it hands /// over without depending on core directly. pub use trusted_server_core::integrations::IntegrationBuilder; @@ -137,23 +142,63 @@ fn stores_for_request( let resolved = config_selection::resolve_key(&stores.config_key, host, |key| { config_selection::blob_exists(stores.config_store_name.as_ref(), key) }); + stores_with_key(stores, resolved, || { + format!("host {:?}", host.unwrap_or_default()) + }) +} + +/// Chooses the app-config blob from the selector values a request addressing +/// named, the first whose blob exists. +/// +/// The values came from the request, so a refusal logs how many there were +/// and not what they were. +/// +/// # Errors +/// +/// Returns why the request is not served, which [`refusal_response`] turns +/// into the answer. +fn stores_for_address( + stores: RuntimeStoreConfig, + selector_values: &[String], +) -> Result { + let resolved = + config_selection::resolve_addressed_key(&stores.config_key, selector_values, |key| { + config_selection::blob_exists(stores.config_store_name.as_ref(), key) + }); + stores_with_key(stores, resolved, || { + format!( + "any of the {} selector value(s) the request addressing named", + selector_values.len() + ) + }) +} + +/// The stores with the resolved key, or the refusal, logged with what +/// `subject` says the request was addressed by. +/// +/// # Errors +/// +/// Returns the refusal `resolved` carries. +fn stores_with_key( + stores: RuntimeStoreConfig, + resolved: Result, + subject: impl FnOnce() -> String, +) -> Result { match resolved { Ok(config_key) => Ok(RuntimeStoreConfig { config_key, ..stores }), Err(refusal) => { - let named = req - .get_header_str_lossy(header::HOST.as_str()) - .unwrap_or_default(); + let subject = subject(); match &refusal { config_selection::Refusal::NoConfigForHost => log::warn!( - "no app-config blob for host {named:?} under selector `{}`", + "no app-config blob for {subject} under selector `{}`", stores.config_key ), config_selection::Refusal::StoreUnavailable(reason) | config_selection::Refusal::MalformedSelector(reason) => { - log::error!("cannot choose the app-config blob for host {named:?}: {reason}"); + log::error!("cannot tell whether {subject} has an app-config blob: {reason}"); } } Err(refusal) @@ -161,6 +206,73 @@ fn stores_for_request( } } +/// Asks `addressing` about the request, chooses the app-config blob that +/// serves it and sets the path it continues with. +/// +/// A request addressed by host is chosen by [`stores_for_request`] and keeps +/// its path. Otherwise the blob is the first of the hook's selector values +/// whose blob exists, the path becomes the hook's continuing path with the +/// query kept, and the prefix the hook removed is returned for the core +/// request. The hook is handed the request's `Host`, or none where it is +/// absent or not UTF-8. +/// +/// # Errors +/// +/// Returns why the request is not served, which [`refusal_response`] turns +/// into the answer. The request's path is then left as it was. +fn address_request( + addressing: &dyn RequestAddressing, + stores: RuntimeStoreConfig, + req: &mut FastlyRequest, +) -> Result<(RuntimeStoreConfig, Option), config_selection::Refusal> { + let host = request_host(req); + match addressing.address(host, req.get_path()) { + Addressing::ByHost => Ok((stores_for_request(stores, req)?, None)), + Addressing::Addressed(address) => { + let stores = stores_for_address(stores, address.selector_values())?; + req.set_path(address.path()); + Ok((stores, Some(address.into_removed_prefix()))) + } + } +} + +/// The request addressing a deployment supplied through +/// [`run_with_addressing`], set once before any request is served. +/// +/// Held as the integration builders are, because [`run`] takes no arguments. +/// The first registration wins, and [`run_with_addressing`] is the only +/// caller, so nothing registers after serving has begun. +static REGISTERED_ADDRESSING: OnceLock> = OnceLock::new(); + +/// The request addressing a deployment registered, or the one that +/// addresses every request by its host. +fn registered_addressing() -> &'static dyn RequestAddressing { + addressing_in(®ISTERED_ADDRESSING) +} + +/// The addressing `slot` holds, or [`ByHostAddressing`] when it holds none. +fn addressing_in(slot: &OnceLock>) -> &dyn RequestAddressing { + slot.get() + .map_or(&ByHostAddressing, |addressing| addressing.as_ref()) +} + +/// Converts the client request for the router, carrying the prefix the +/// request addressing removed, when it removed one. +/// +/// # Errors +/// +/// Returns the conversion's error. +fn into_addressed_core_request( + req: FastlyRequest, + removed_prefix: Option, +) -> Result { + let mut core_req = into_core_request(req)?; + if let Some(prefix) = removed_prefix { + core_req.extensions_mut().insert(prefix); + } + Ok(core_req) +} + /// The request's `Host` header as text, or `None` when it is absent or its /// bytes are not valid UTF-8. /// @@ -200,6 +312,30 @@ fn health_response(req: &FastlyRequest) -> Option { None } +/// A debug probe answered before the application is built. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum DebugProbe { + /// The sandbox counters snapshot. + #[cfg(feature = "reusable-sandbox")] + SandboxMetrics, + /// The TLS and JA4 signals the request arrived with. + Ja4, +} + +/// The debug probe `req` asks for, read from the path it continues with, so +/// a probe is found at the path the request addressing gives. +fn debug_probe(req: &FastlyRequest) -> Option { + if req.get_method() != FastlyMethod::GET { + return None; + } + match req.get_path() { + #[cfg(feature = "reusable-sandbox")] + sandbox::SANDBOX_METRICS_PATH => Some(DebugProbe::SandboxMetrics), + "/_ts/debug/ja4" => Some(DebugProbe::Ja4), + _ => None, + } +} + /// Runs the Fastly Compute program with the integrations a deployment offers. /// /// The builders are composed with the ones this adapter ships rather than @@ -222,6 +358,38 @@ pub fn run_with(integrations: Vec) { run(); } +/// Runs the Fastly Compute program with the integrations a deployment offers +/// and the request addressing it supplies. +/// +/// The addressing is asked about each request after `/health` and before its +/// app-config blob is chosen. A request it addresses by host is served as +/// [`run_with`] serves it. +/// +/// ```no_run +/// use std::sync::Arc; +/// +/// use trusted_server_adapter_fastly::{Addressing, RequestAddressing}; +/// +/// struct ByHostOnly; +/// +/// impl RequestAddressing for ByHostOnly { +/// fn address(&self, _host: Option<&str>, _path: &str) -> Addressing { +/// Addressing::ByHost +/// } +/// } +/// +/// fn main() { +/// trusted_server_adapter_fastly::run_with_addressing(Vec::new(), Arc::new(ByHostOnly)); +/// } +/// ``` +pub fn run_with_addressing( + integrations: Vec, + addressing: Arc, +) { + let _ = REGISTERED_ADDRESSING.set(addressing); + run_with(integrations); +} + /// Runs the Fastly Compute program, which the binary's `main()` calls. /// /// Uses an undecorated entry with `FastlyRequest::from_client()` instead of @@ -465,11 +633,12 @@ fn heap_mib() -> String { /// Handles a request through the `EdgeZero` router path. fn edgezero_main(mut req: FastlyRequest, sandbox: &mut Sandbox, ordinal: u64, request_id: &str) { // Settled before anything reads settings, so every path below reads the - // blob of the publisher this request is for. - let runtime_stores = match stores_with_stored_selector(RuntimeStoreConfig::logical()) - .and_then(|stores| stores_for_request(stores, &req)) - { - Ok(stores) => stores, + // blob of the publisher this request is for, at the path it continues + // with. + let addressed = stores_with_stored_selector(RuntimeStoreConfig::logical()) + .and_then(|stores| address_request(registered_addressing(), stores, &mut req)); + let (runtime_stores, removed_prefix) = match addressed { + Ok(addressed) => addressed, Err(refusal) => { refusal_response(&refusal).send_to_client(); return; @@ -479,8 +648,9 @@ fn edgezero_main(mut req: FastlyRequest, sandbox: &mut Sandbox, ordinal: u64, re // Short-circuit the sandbox counters probe before app construction. It must // not build the application: polling it would otherwise increment the very // build counter it reports. + let probe = debug_probe(&req); #[cfg(feature = "reusable-sandbox")] - if req.get_method() == FastlyMethod::GET && req.get_path() == sandbox::SANDBOX_METRICS_PATH { + if probe == Some(DebugProbe::SandboxMetrics) { match load_settings_from_config_store(&runtime_stores) { Ok(settings) if sandbox::metrics_enabled(&settings) => { sandbox_metrics_response(sandbox, ordinal).send_to_client(); @@ -501,7 +671,7 @@ fn edgezero_main(mut req: FastlyRequest, sandbox: &mut Sandbox, ordinal: u64, re // Short-circuit the JA4 debug probe before app construction. Must run here // because TLS/JA4 accessors are only available on FastlyRequest before // conversion to edgezero types. - if req.get_method() == FastlyMethod::GET && req.get_path() == "/_ts/debug/ja4" { + if probe == Some(DebugProbe::Ja4) { match load_settings_from_config_store(&runtime_stores) { Ok(settings) if settings.debug.ja4_endpoint_enabled => { build_ja4_debug_response(&req).send_to_client(); @@ -647,7 +817,7 @@ fn edgezero_main(mut req: FastlyRequest, sandbox: &mut Sandbox, ordinal: u64, re // Dispatch directly through the EdgeZero router without an intermediate // fastly::Response conversion. That preserves duplicate header values such // as multiple Set-Cookie headers. - let mut response = match into_core_request(req) { + let mut response = match into_addressed_core_request(req, removed_prefix) { Ok(mut core_req) => { core_req.extensions_mut().insert(config_store); core_req.extensions_mut().insert(device_signals); @@ -1129,6 +1299,7 @@ mod tests { use edgezero_core::http::response_builder; use fastly::mime; use trusted_server_core::integrations::HeaderMutation; + use trusted_server_core::test_support::addressing::NamesSelectorValues; fn test_settings() -> Settings { Settings::from_toml( @@ -1397,6 +1568,247 @@ mod tests { ); } + /// A request for a page under a publisher's path on `host`. + fn addressed_page(host: &str) -> FastlyRequest { + let mut req = FastlyRequest::get(format!( + "https://{host}/publisher.example/news/today?page=2" + )); + req.set_header(header::HOST.as_str(), host); + req + } + + /// Asserts that `addressing` chooses every request's blob as + /// [`stores_for_request`] does and leaves its path as it is. + fn assert_addressed_as_by_host(addressing: &dyn RequestAddressing) { + for selector in ["trusted_server_config", "{host}"] { + let stores = stores_with_selector(selector); + for host in [ + None, + Some("publisher.example"), + Some("not a host"), + Some("[::1]:7676"), + ] { + let mut req = request_for_host(host); + let today = stores_for_request(stores.clone(), &req); + + let addressed = address_request(addressing, stores.clone(), &mut req); + + assert_eq!( + addressed, + today.map(|stores| (stores, None)), + "should choose as today for `{selector}` and host {host:?}" + ); + assert_eq!( + req.get_path(), + "/article", + "should leave the path as it is for host {host:?}" + ); + } + } + } + + #[test] + fn with_no_addressing_supplied_a_request_is_chosen_and_routed_as_today() { + let nothing_registered = OnceLock::new(); + + assert_addressed_as_by_host(addressing_in(¬hing_registered)); + } + + #[test] + fn an_addressing_answering_by_host_leaves_selection_and_path_as_today() { + assert_addressed_as_by_host(&ByHostAddressing); + } + + #[test] + fn the_addressing_a_deployment_registers_is_the_one_asked() { + let registered: OnceLock> = OnceLock::new(); + assert!( + registered + .set(Arc::new(NamesSelectorValues(&["a.example"]))) + .is_ok(), + "should register the addressing" + ); + + assert!( + matches!( + addressing_in(®istered).address(None, "/publisher.example/news"), + Addressing::Addressed(_) + ), + "should ask the registered addressing, not the host" + ); + } + + #[test] + fn a_rewriting_addressing_continues_at_its_path_keeps_the_query_and_carries_the_prefix() { + // A selector without the placeholder reads its one blob, so the + // selector values are ignored and nothing is looked up. + let stores = stores_with_selector("trusted_server_config"); + let mut req = addressed_page("front.example"); + + let (resolved, removed_prefix) = address_request( + &NamesSelectorValues(&["publisher.example"]), + stores.clone(), + &mut req, + ) + .expect("should serve the addressed request"); + + assert_eq!(resolved, stores, "should read the one blob"); + assert_eq!(req.get_path(), "/news/today", "should continue at the path"); + assert_eq!(req.get_query_str(), Some("page=2"), "should keep the query"); + + let core_req = into_addressed_core_request(req, removed_prefix) + .expect("should convert the addressed request"); + assert_eq!( + core_req.uri().path(), + "/news/today", + "the router should see the continuing path" + ); + assert_eq!( + core_req.uri().query(), + Some("page=2"), + "the router should see the query" + ); + assert_eq!( + core_req + .extensions() + .get::() + .map(RemovedPathPrefix::as_str), + Some("/publisher.example"), + "the core request should carry the prefix the addressing removed" + ); + } + + #[test] + fn a_request_addressed_by_host_carries_no_removed_prefix() { + let core_req = into_addressed_core_request(addressed_page("front.example"), None) + .expect("should convert the request"); + + assert!( + core_req.extensions().get::().is_none(), + "a request whose path nothing removed should carry no prefix" + ); + } + + #[test] + fn the_first_selector_value_whose_blob_exists_takes_the_host_s_place() { + // The local Viceroy configuration fills `jwks_store`, and + // `current-kid` is a key in it that is also a host name, so the + // lookups here are real ones. + let stores = RuntimeStoreConfig { + config_store_name: trusted_server_core::platform::StoreName::from("jwks_store"), + ..stores_with_selector("{host}") + }; + let mut req = addressed_page("front.example"); + + let (resolved, _) = address_request( + &NamesSelectorValues(&["absent-key", "current-kid", "front.example"]), + stores.clone(), + &mut req, + ) + .expect("should serve from the first value whose blob exists"); + + assert_eq!( + resolved, + RuntimeStoreConfig { + config_key: "current-kid".to_owned(), + ..stores + }, + "should read the blob of the first value the store holds" + ); + assert_eq!(req.get_path(), "/news/today", "should continue at the path"); + } + + #[test] + fn every_selector_value_missing_is_refused_421_and_the_request_is_left_as_it_was() { + // The local store holds no blob for any of these values, and the + // request's own host is never tried in their place. + let stores = stores_with_selector("{host}"); + let mut req = addressed_page("front.example"); + + let refusal = address_request( + &NamesSelectorValues(&["a.example", "b.example", "trusted_server_config"]), + stores, + &mut req, + ) + .expect_err("should refuse values with no blob"); + + assert_eq!( + refusal, + config_selection::Refusal::NoConfigForHost, + "should refuse as a host with no blob is refused" + ); + assert_eq!( + refusal_response(&refusal).get_status(), + fastly::http::StatusCode::MISDIRECTED_REQUEST, + "should be answered 421" + ); + assert_eq!( + req.get_path(), + "/publisher.example/news/today", + "a refused request should keep its path" + ); + } + + /// `path` on `front.example` after the addressing that removes its first + /// segment, with the blob the addressing chose. + fn probe_addressed_at(path: &str) -> (FastlyRequest, RuntimeStoreConfig) { + let mut req = FastlyRequest::get(format!("https://front.example{path}")); + req.set_header(header::HOST.as_str(), "front.example"); + let (stores, _) = address_request( + &NamesSelectorValues(&["publisher.example"]), + stores_with_selector("trusted_server_config"), + &mut req, + ) + .expect("should serve the addressed request"); + (req, stores) + } + + #[test] + fn the_ja4_probe_is_found_at_the_path_the_addressing_gives() { + let (req, stores) = probe_addressed_at("/publisher.example/_ts/debug/ja4"); + + assert_eq!( + debug_probe(&req), + Some(DebugProbe::Ja4), + "the JA4 probe should answer under the publisher's path" + ); + assert_eq!( + stores, + stores_with_selector("trusted_server_config"), + "the probe should read the blob the addressing chose" + ); + + let (unaddressed, _) = probe_addressed_at("/_ts/debug/ja4"); + assert_eq!( + debug_probe(&unaddressed), + None, + "the probe should read the path the request continues with, not the one it arrived at" + ); + } + + #[cfg(feature = "reusable-sandbox")] + #[test] + fn the_counters_probe_is_found_at_the_path_the_addressing_gives() { + let (req, _) = probe_addressed_at(&format!( + "/publisher.example{}", + sandbox::SANDBOX_METRICS_PATH + )); + + assert_eq!( + debug_probe(&req), + Some(DebugProbe::SandboxMetrics), + "the counters probe should answer under the publisher's path" + ); + } + + #[test] + fn a_debug_probe_is_answered_only_to_a_get() { + let mut req = FastlyRequest::post("https://front.example/_ts/debug/ja4"); + req.set_header(header::HOST.as_str(), "front.example"); + + assert_eq!(debug_probe(&req), None, "a POST should not reach the probe"); + } + /// A `Host` header whose bytes are not valid UTF-8. const HOST_NOT_UTF8: &[u8] = b"\xffa.example"; @@ -1489,6 +1901,67 @@ mod tests { ); } + /// A request addressing that records the host it is handed and addresses + /// every request by its host. + struct RecordsHost(std::sync::Mutex>>); + + impl RequestAddressing for RecordsHost { + fn address(&self, host: Option<&str>, _path: &str) -> Addressing { + self.0 + .lock() + .expect("should lock") + .push(host.map(str::to_owned)); + Addressing::ByHost + } + } + + #[test] + fn a_hook_is_handed_no_host_for_a_value_that_is_not_utf8() { + let hook = RecordsHost(std::sync::Mutex::default()); + let mut req = request_with_host_not_utf8(); + + let refusal = address_request(&hook, stores_with_selector("{host}"), &mut req) + .expect_err("should refuse a Host that is not UTF-8 when addressed by host"); + + assert_eq!( + *hook.0.lock().expect("should lock"), + [None], + "the hook should be handed no host for a Host that is not UTF-8" + ); + assert_eq!( + refusal, + config_selection::Refusal::NoConfigForHost, + "a request addressed by its host should then be refused as one with no host" + ); + } + + #[test] + fn a_request_addressed_by_its_path_with_a_host_that_is_not_utf8_is_served_without_a_panic() { + // The local Viceroy configuration fills `jwks_store`, and + // `current-kid` is a key in it that is also a host name, so the + // lookup here is a real one. + let stores = RuntimeStoreConfig { + config_store_name: trusted_server_core::platform::StoreName::from("jwks_store"), + ..stores_with_selector("{host}") + }; + let mut req = addressed_page("front.example"); + req.set_header(header::HOST.as_str(), HOST_NOT_UTF8); + + let (resolved, _) = + address_request(&NamesSelectorValues(&["current-kid"]), stores, &mut req) + .expect("should serve the request from the value the hook names"); + + assert_eq!( + resolved.config_key, "current-kid", + "should read the blob the hook's value names, whatever the Host" + ); + assert_eq!( + entry_request(&req).host(), + "", + "the module request should carry no host where Host is not UTF-8" + ); + } + #[test] fn take_finalize_sentinel_strips_sentinel() { let mut response = HttpResponse::new(EdgeBody::empty()); diff --git a/crates/trusted-server-adapter-spin/src/app.rs b/crates/trusted-server-adapter-spin/src/app.rs index 2a16036275..4c32ecf45e 100644 --- a/crates/trusted-server-adapter-spin/src/app.rs +++ b/crates/trusted-server-adapter-spin/src/app.rs @@ -13,6 +13,7 @@ use edgezero_core::error::EdgeError; use edgezero_core::http::{HeaderValue, Method, Request, Response, StatusCode, header}; use edgezero_core::router::RouterService; use error_stack::Report; +use trusted_server_core::addressing::{RequestAddressing, addressed_router}; use trusted_server_core::auction::endpoints::handle_auction; use trusted_server_core::auction::telemetry::warn_once_when_analytics_is_unsent; use trusted_server_core::auction::{ @@ -809,6 +810,29 @@ impl TrustedServerApp { Ok(build_router(&state)) } + /// Build the full application router as + /// [`TrustedServerApp::routes_with_registrations`] does, asking + /// `addressing` about every request before it is routed. + /// + /// This adapter reads one app-config blob, so the selector values the + /// addressing names are ignored. A request it addresses continues at the + /// path it gives, with its query kept, and carries the prefix it removed + /// as a [`RemovedPathPrefix`](trusted_server_core::addressing::RemovedPathPrefix) + /// extension. A request it addresses by host is routed as it is. + /// + /// # Errors + /// + /// Returns the errors [`TrustedServerApp::routes_with_registrations`] + /// returns. + pub fn routes_with_addressing( + settings: Settings, + integrations: &[IntegrationBuilder], + addressing: Arc, + ) -> Result> { + let router = Self::routes_with_registrations(settings, integrations)?; + Ok(addressed_router(router, addressing)) + } + /// Build the full router with explicit settings and runtime services. /// /// Each request receives a clone of the supplied services, allowing callers diff --git a/crates/trusted-server-adapter-spin/tests/routes.rs b/crates/trusted-server-adapter-spin/tests/routes.rs index f40392a6f8..fc9fd8ed94 100644 --- a/crates/trusted-server-adapter-spin/tests/routes.rs +++ b/crates/trusted-server-adapter-spin/tests/routes.rs @@ -6,11 +6,15 @@ //! that depends on external stores assert routing only; deterministic auth and //! method gates assert exact status codes. +use std::sync::Arc; + use edgezero_core::app::Hooks as _; use edgezero_core::http::{Request, Response, request_builder}; use edgezero_core::router::RouterService; use trusted_server_adapter_spin::app::TrustedServerApp; +use trusted_server_core::addressing::ByHostAddressing; use trusted_server_core::settings::Settings; +use trusted_server_core::test_support::addressing::{FirstSegment, status_and_type}; /// Build the full application router from explicit test settings. /// @@ -23,7 +27,7 @@ fn test_router() -> RouterService { .expect("should build router from test settings") } -/// The settings [`test_router`] is built from. +/// Explicit settings for the route tests. fn test_settings() -> Settings { Settings::from_toml( r#" @@ -1410,3 +1414,29 @@ async fn a_response_carries_no_blanket_rule_without_the_section() { assert_eq!(robots_tag(&response), None); } + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_addressing_continues_the_request_at_its_path_before_routing() { + let router = + TrustedServerApp::routes_with_addressing(test_settings(), &[], Arc::new(FirstSegment)) + .expect("should build router with a request addressing"); + + assert_eq!( + status_and_type(router, "/publisher.example/_ts/config.json?view=1").await, + (200, "application/json".to_owned()), + "the configuration endpoint should answer at the path the addressing continues with" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_addressed_by_host_is_routed_as_it_is() { + let router = + TrustedServerApp::routes_with_addressing(test_settings(), &[], Arc::new(ByHostAddressing)) + .expect("should build router with a request addressing"); + + assert_eq!( + status_and_type(router, "/_ts/config.json").await, + status_and_type(test_router(), "/_ts/config.json").await, + "should answer as the router with no addressing answers" + ); +} diff --git a/crates/trusted-server-core/src/addressing.rs b/crates/trusted-server-core/src/addressing.rs new file mode 100644 index 0000000000..3b69686a5d --- /dev/null +++ b/crates/trusted-server-core/src/addressing.rs @@ -0,0 +1,594 @@ +//! How a request is addressed before its configuration is chosen. +//! +//! An adapter chooses the app-config blob that serves a request before any +//! settings exist, and by default chooses it by the request's host. A +//! deployment that serves several publishers on one host, each under a path +//! of its own, supplies a [`RequestAddressing`]. It reads the request's host +//! and path and answers three things, being the selector values that name +//! the request's blob, the path the request continues with, and the path +//! prefix it removed. +//! +//! The removed prefix travels on the request as a [`RemovedPathPrefix`] +//! extension, where a module's request preparer or a handler reads it. Core +//! never writes it into a response. A deployment whose pages must carry the +//! prefix puts it back with a middleware of its own. + +use std::sync::Arc; + +use edgezero_core::body::Body as EdgeBody; +use edgezero_core::context::RequestContext; +use edgezero_core::error::EdgeError; +use edgezero_core::http::{HandlerFuture, Method}; +use edgezero_core::router::RouterService; +use error_stack::Report; +use http::uri::PathAndQuery; +use http::{Request, Uri, header}; + +/// Answers how a request is addressed, before its configuration is chosen. +/// +/// Asked once for each request, before the adapter chooses the app-config +/// blob and before routing. It is given the request's `Host`, or `None` when +/// the request names none, and the path without its query. +/// +/// An adapter that chooses a blob for each request puts the selector values +/// in the `{host}` place of its key selector and reads the first whose blob +/// exists. An adapter that reads one app-config blob has no blob to choose, +/// so it ignores the selector values, and the path and the removed prefix +/// apply there too. +/// +/// # Examples +/// +/// ``` +/// use trusted_server_core::addressing::{Addressing, RequestAddress, RequestAddressing}; +/// +/// /// Serves `//` from the publisher's own blob. +/// struct FirstSegment; +/// +/// impl RequestAddressing for FirstSegment { +/// fn address(&self, _host: Option<&str>, path: &str) -> Addressing { +/// let rest = path.strip_prefix('/').unwrap_or(path); +/// let (segment, tail) = rest.split_once('/').unwrap_or((rest, "")); +/// RequestAddress::new( +/// vec![segment.to_owned()], +/// format!("/{tail}"), +/// format!("/{segment}"), +/// ) +/// .map_or(Addressing::ByHost, Addressing::Addressed) +/// } +/// } +/// +/// let addressing = FirstSegment.address(None, "/publisher.example/news"); +/// let Addressing::Addressed(address) = addressing else { +/// panic!("should address the request by its first segment"); +/// }; +/// assert_eq!(address.selector_values(), ["publisher.example"]); +/// assert_eq!(address.path(), "/news"); +/// assert_eq!(address.removed_prefix(), "/publisher.example"); +/// ``` +pub trait RequestAddressing: Send + Sync { + /// How the request for `host` at `path` is addressed. + fn address(&self, host: Option<&str>, path: &str) -> Addressing; +} + +/// What a [`RequestAddressing`] answers for one request. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Addressing { + /// Chosen by the request's host, with its path unchanged, which is how + /// every request is addressed when no hook is supplied. + ByHost, + /// Chosen by the selector values the hook names, with the path it gives. + Addressed(RequestAddress), +} + +/// The selector values, the continuing path and the removed prefix a +/// [`RequestAddressing`] names for a request. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct RequestAddress { + selector_values: Vec, + path: String, + removed_prefix: String, +} + +impl RequestAddress { + /// An address that names `selector_values` in order, continues the + /// request at `path` and records `removed_prefix` as the prefix removed. + /// + /// The request's query is kept, so `path` carries none. + /// + /// # Errors + /// + /// Returns [`AddressingError::InvalidPath`] when `path` does not start + /// with `/`, is not percent-encoded ASCII, carries a `?` or a `#`, or is + /// not a valid URI path. + pub fn new( + selector_values: Vec, + path: String, + removed_prefix: String, + ) -> Result> { + if !is_continuing_path(&path) { + return Err(Report::new(AddressingError::InvalidPath)); + } + Ok(Self { + selector_values, + path, + removed_prefix, + }) + } + + /// The values to try in the selector's `{host}` place, in order. + #[must_use] + pub fn selector_values(&self) -> &[String] { + &self.selector_values + } + + /// The path the request continues with, without a query. + #[must_use] + pub fn path(&self) -> &str { + &self.path + } + + /// The path prefix the hook removed. + #[must_use] + pub fn removed_prefix(&self) -> &str { + &self.removed_prefix + } + + /// The request extension carrying the removed prefix. + #[must_use] + pub fn into_removed_prefix(self) -> RemovedPathPrefix { + RemovedPathPrefix::new(self.removed_prefix) + } +} + +/// Why a request could not be addressed as a hook named. +/// +/// The messages carry nothing from the request, so they can be logged. +#[derive(Debug, Clone, Copy, PartialEq, Eq, derive_more::Display)] +pub enum AddressingError { + /// The continuing path is not one a request can carry. + #[display( + "a continuing path is percent-encoded ASCII that starts with `/` and carries no query or \ + fragment" + )] + InvalidPath, + /// The request's address could not be rebuilt with the continuing path. + #[display("the request's address could not be rebuilt with the continuing path")] + Rewrite, +} + +impl core::error::Error for AddressingError {} + +/// The addressing an adapter given no hook uses, which answers +/// [`Addressing::ByHost`] for every request. +#[derive(Debug, Clone, Copy, Default)] +pub struct ByHostAddressing; + +impl RequestAddressing for ByHostAddressing { + fn address(&self, _host: Option<&str>, _path: &str) -> Addressing { + Addressing::ByHost + } +} + +/// The path prefix a [`RequestAddressing`] removed from a request, carried +/// as a request extension. +/// +/// Present only on a request a hook addressed. Putting the prefix back into +/// a response, such as into the links of a page, is the deployment's job, +/// done by a middleware of its own. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct RemovedPathPrefix(String); + +impl RemovedPathPrefix { + /// The extension carrying `prefix`. + #[must_use] + pub fn new(prefix: String) -> Self { + Self(prefix) + } + + /// The prefix that was removed. + #[must_use] + pub fn as_str(&self) -> &str { + &self.0 + } +} + +/// Asks `addressing` about `request` and applies its answer. +/// +/// A request addressed by host is left as it is. Otherwise its path becomes +/// the continuing path, its query is kept, and the removed prefix is +/// inserted as a [`RemovedPathPrefix`] extension. The selector values are +/// not read here, because choosing a blob is the adapter's step and comes +/// first. +/// +/// The hook is given the `Host` header, or the request's authority where +/// there is no `Host`. +/// +/// # Errors +/// +/// Returns [`AddressingError::Rewrite`] when the request's address cannot be +/// rebuilt with the continuing path. The request is then left as it was. +pub fn address_request( + addressing: &dyn RequestAddressing, + request: &mut Request, +) -> Result<(), Report> { + let host = request + .headers() + .get(header::HOST) + .and_then(|value| value.to_str().ok()) + .or_else(|| request.uri().authority().map(http::uri::Authority::as_str)); + let Addressing::Addressed(address) = addressing.address(host, request.uri().path()) else { + return Ok(()); + }; + *request.uri_mut() = with_path(request.uri(), address.path())?; + request + .extensions_mut() + .insert(address.into_removed_prefix()); + Ok(()) +} + +/// A router that asks `addressing` about every request, applies its answer +/// with [`address_request`], and then hands the request to `router`. +/// +/// An adapter that reads one app-config blob wraps its router with this, so +/// the selector values a hook names are ignored there. It routes every path +/// for each method `router` registers and passes every request on, so a +/// path or method `router` refuses is refused by `router` with its own +/// answer. The route table it reports is its own catch-all routes, not those +/// of `router`. +#[must_use] +pub fn addressed_router( + router: RouterService, + addressing: Arc, +) -> RouterService { + let mut methods: Vec = router + .routes() + .iter() + .map(|route| route.method().clone()) + .collect(); + methods.sort_by(|left, right| left.as_str().cmp(right.as_str())); + methods.dedup(); + + let handler = addressed_handler(router, addressing); + let mut builder = RouterService::builder(); + for method in methods { + builder = builder.route("/", method.clone(), handler.clone()); + builder = builder.route("/{*rest}", method, handler.clone()); + } + builder.build() +} + +/// Addresses the request, then routes it through `router`. +fn addressed_handler( + router: RouterService, + addressing: Arc, +) -> impl Fn(RequestContext) -> HandlerFuture + Clone + Send + Sync + 'static { + move |ctx: RequestContext| { + let router = router.clone(); + let addressing = Arc::clone(&addressing); + Box::pin(async move { + let mut request = ctx.into_request(); + if let Err(report) = address_request(addressing.as_ref(), &mut request) { + log::error!("request addressing failed: {report:?}"); + return Err(EdgeError::internal(*report.current_context())); + } + router.oneshot(request).await + }) + } +} + +/// Whether `path` is one a request can continue with, which is a +/// percent-encoded ASCII path as a request carries it, so every adapter +/// routes the same path. +fn is_continuing_path(path: &str) -> bool { + path.starts_with('/') + && path.is_ascii() + && !path.contains(['?', '#']) + && PathAndQuery::try_from(path).is_ok_and(|parsed| parsed.as_str() == path) +} + +/// `uri` with its path replaced by `path` and its query kept. +fn with_path(uri: &Uri, path: &str) -> Result> { + let path_and_query = match uri.query() { + Some(query) => format!("{path}?{query}"), + None => path.to_owned(), + }; + let mut parts = uri.clone().into_parts(); + parts.path_and_query = Some( + PathAndQuery::try_from(path_and_query) + .map_err(|_| Report::new(AddressingError::Rewrite))?, + ); + Uri::from_parts(parts).map_err(|_| Report::new(AddressingError::Rewrite)) +} + +#[cfg(test)] +mod tests { + use std::sync::Mutex; + + use edgezero_core::http::{Response, StatusCode, request_builder}; + + use super::*; + + /// A hook that serves `//` from the blob `segment` names + /// and records what it was asked. + #[derive(Default)] + struct FirstSegment { + asked: Mutex, String)>>, + } + + impl RequestAddressing for FirstSegment { + fn address(&self, host: Option<&str>, path: &str) -> Addressing { + self.asked + .lock() + .expect("should lock what the hook was asked") + .push((host.map(str::to_owned), path.to_owned())); + let rest = path.strip_prefix('/').unwrap_or(path); + let (segment, tail) = rest.split_once('/').unwrap_or((rest, "")); + let address = RequestAddress::new( + vec![segment.to_owned()], + format!("/{tail}"), + format!("/{segment}"), + ) + .expect("should name a valid continuing path"); + Addressing::Addressed(address) + } + } + + fn request(uri: &str, host: Option<&str>) -> Request { + let mut builder = request_builder().method("GET").uri(uri); + if let Some(host) = host { + builder = builder.header(header::HOST, host); + } + builder + .body(EdgeBody::empty()) + .expect("should build the request") + } + + /// A router that answers with the path, the query and the removed + /// prefix the request reached it with. + fn echo_router() -> RouterService { + let echo = |ctx: RequestContext| async move { + let request = ctx.request(); + let prefix = request + .extensions() + .get::() + .map_or("none", RemovedPathPrefix::as_str); + let body = format!( + "{} {} {prefix}", + request.uri().path(), + request.uri().query().unwrap_or("-") + ); + Ok::(Response::new(EdgeBody::from(body))) + }; + RouterService::builder() + .route("/", Method::GET, echo) + .route("/{*rest}", Method::GET, echo) + .build() + } + + fn body_of(response: Response) -> String { + String::from_utf8( + response + .into_body() + .into_bytes() + .unwrap_or_default() + .to_vec(), + ) + .expect("should answer in UTF-8") + } + + fn route(router: &RouterService, request: Request) -> Response { + futures::executor::block_on(router.oneshot(request)).expect("should route the request") + } + + #[test] + fn the_default_addressing_answers_by_host() { + for (host, path) in [ + (None, "/"), + (Some("publisher.example"), "/news"), + (Some("publisher.example"), "/other.example/news"), + ] { + assert_eq!( + ByHostAddressing.address(host, path), + Addressing::ByHost, + "should address {host:?} {path} by its host" + ); + } + } + + #[test] + fn a_continuing_path_starts_with_a_slash_and_carries_no_query() { + for path in ["/", "/news", "/news/today.html", "/a%20b", "//double"] { + assert!( + RequestAddress::new(Vec::new(), path.to_owned(), String::new()).is_ok(), + "should accept `{path}`" + ); + } + for path in ["", "news", "/news?page=2", "/news#top", "/a b", "/\u{e9}"] { + let error = RequestAddress::new(Vec::new(), path.to_owned(), String::new()) + .expect_err("should refuse a path a request cannot continue with"); + assert_eq!( + *error.current_context(), + AddressingError::InvalidPath, + "should refuse `{path}` as an invalid path" + ); + } + } + + #[test] + fn the_error_carries_nothing_from_the_request() { + let error = RequestAddress::new(Vec::new(), "/private?id=reader".to_owned(), String::new()) + .expect_err("should refuse a path with a query"); + + assert!( + !error.to_string().contains("reader"), + "a logged refusal should not carry what the request named" + ); + } + + #[test] + fn a_request_addressed_by_host_is_left_as_it_is() { + let mut addressed = request( + "https://publisher.example/other.example/news?page=2", + Some("publisher.example"), + ); + let before = addressed.uri().clone(); + + address_request(&ByHostAddressing, &mut addressed) + .expect("should address the request by its host"); + + assert_eq!( + addressed.uri(), + &before, + "should leave the address unchanged" + ); + assert!( + addressed.extensions().get::().is_none(), + "should carry no removed prefix when nothing was removed" + ); + } + + #[test] + fn a_rewriting_hook_continues_the_request_at_its_path_and_keeps_the_query() { + let hook = FirstSegment::default(); + let mut addressed = request( + "https://front.example/publisher.example/news/today?page=2&sort=new", + Some("front.example"), + ); + + address_request(&hook, &mut addressed).expect("should address the request"); + + assert_eq!( + addressed.uri().path(), + "/news/today", + "should continue at the path" + ); + assert_eq!( + addressed.uri().query(), + Some("page=2&sort=new"), + "should keep the query" + ); + assert_eq!( + addressed.uri().host(), + Some("front.example"), + "should keep the authority" + ); + assert_eq!( + addressed + .extensions() + .get::() + .map(RemovedPathPrefix::as_str), + Some("/publisher.example"), + "should carry the prefix the hook removed" + ); + assert_eq!( + *hook + .asked + .lock() + .expect("should lock what the hook was asked"), + [( + Some("front.example".to_owned()), + "/publisher.example/news/today".to_owned() + )], + "the hook should be given the host and the path without its query" + ); + } + + #[test] + fn a_request_with_no_host_header_offers_its_authority() { + let hook = FirstSegment::default(); + let mut addressed = request("https://front.example:8443/publisher.example/", None); + + address_request(&hook, &mut addressed).expect("should address the request"); + + assert_eq!( + hook.asked + .lock() + .expect("should lock what the hook was asked")[0] + .0, + Some("front.example:8443".to_owned()), + "should offer the authority where no Host header is sent" + ); + assert_eq!(addressed.uri().path(), "/", "should continue at the root"); + + let hook = FirstSegment::default(); + let mut addressed = request("/publisher.example/news", None); + address_request(&hook, &mut addressed).expect("should address the request"); + assert_eq!( + hook.asked + .lock() + .expect("should lock what the hook was asked")[0] + .0, + None, + "should offer no host when the request names none" + ); + } + + #[test] + fn an_addressed_router_routes_the_continuing_path_with_the_prefix() { + let router = addressed_router(echo_router(), Arc::new(FirstSegment::default())); + + let response = route( + &router, + request( + "https://front.example/publisher.example/news?page=2", + Some("front.example"), + ), + ); + + assert_eq!( + response.status(), + StatusCode::OK, + "should route the request" + ); + assert_eq!( + body_of(response), + "/news page=2 /publisher.example", + "the router should see the continuing path, the query and the prefix" + ); + } + + #[test] + fn a_by_host_hook_leaves_routing_unchanged() { + let by_host = addressed_router(echo_router(), Arc::new(ByHostAddressing)); + + for uri in ["/", "/news?page=2", "/publisher.example/news"] { + let plain = body_of(route(&echo_router(), request(uri, Some("front.example")))); + assert_eq!( + body_of(route(&by_host, request(uri, Some("front.example")))), + plain, + "`{uri}` should route as it does with no hook" + ); + } + } + + #[test] + fn a_method_the_router_refuses_is_still_refused() { + let plain = route( + &echo_router(), + request_builder() + .method("POST") + .uri("/news") + .body(EdgeBody::empty()) + .expect("should build the request"), + ); + let addressed = route( + &addressed_router(echo_router(), Arc::new(FirstSegment::default())), + request_builder() + .method("POST") + .uri("/publisher.example/news") + .body(EdgeBody::empty()) + .expect("should build the request"), + ); + + assert_eq!( + plain.status(), + StatusCode::METHOD_NOT_ALLOWED, + "the plain router should refuse a method it does not route" + ); + assert_eq!( + addressed.status(), + plain.status(), + "the addressed router should refuse it the same way" + ); + } +} diff --git a/crates/trusted-server-core/src/http_util.rs b/crates/trusted-server-core/src/http_util.rs index 9d121252e2..62bb825c11 100644 --- a/crates/trusted-server-core/src/http_util.rs +++ b/crates/trusted-server-core/src/http_util.rs @@ -140,6 +140,8 @@ impl RequestInfo { /// 1. `Forwarded` header (`host=...`) /// 2. `X-Forwarded-Host` /// 3. `Host` header + /// 4. The request URI's authority, which an HTTP/2 request may carry + /// in place of a `Host` header /// /// Scheme fallback order: /// 1. Fastly SDK TLS detection @@ -212,6 +214,10 @@ fn extract_request_host(req: &Request) -> &str { .get(header::HOST) .and_then(|h| h.to_str().ok()) }) + // An HTTP/2 client may send its `:authority` and no `Host` header, and + // an empty host leaves a page unprocessed, so the URI's authority comes + // last, as `Host` would carry it. + .or_else(|| req.uri().authority().map(http::uri::Authority::as_str)) .unwrap_or_default() } @@ -662,6 +668,55 @@ mod tests { ); } + #[test] + fn request_info_falls_back_to_the_uri_authority_without_a_host_header() { + // An HTTP/2 client may send its `:authority` and no `Host` header. + let req = build_request(Method::GET, "https://publisher.example:8443/page"); + assert!( + req.headers().get(header::HOST).is_none(), + "the request must carry no Host header, or it proves nothing" + ); + + let info = RequestInfo::from_request(&req, &default_client_info()); + assert_eq!( + info.host, "publisher.example:8443", + "should read the host from the URI's authority, with its port as Host carries it" + ); + + let path_only = build_request(Method::GET, "/page"); + assert_eq!( + RequestInfo::from_request(&path_only, &default_client_info()).host, + "", + "should name no host when neither a header nor the URI names one" + ); + } + + #[test] + fn request_info_prefers_each_host_header_to_the_uri_authority() { + for (name, value, expected) in [ + ("host", "from-host.example", "from-host.example"), + ( + "x-forwarded-host", + "from-x-forwarded-host.example", + "from-x-forwarded-host.example", + ), + ( + "forwarded", + "host=from-forwarded.example", + "from-forwarded.example", + ), + ] { + let mut req = build_request(Method::GET, "https://from-authority.example/page"); + set_header(&mut req, name, value); + + let info = RequestInfo::from_request(&req, &default_client_info()); + assert_eq!( + info.host, expected, + "should prefer {name} to the URI's authority" + ); + } + } + #[test] fn test_request_info_x_forwarded_host_precedence() { let mut req = build_request(Method::GET, "https://test.example.com/page"); diff --git a/crates/trusted-server-core/src/lib.rs b/crates/trusted-server-core/src/lib.rs index b6de7efd7e..45962e873e 100644 --- a/crates/trusted-server-core/src/lib.rs +++ b/crates/trusted-server-core/src/lib.rs @@ -28,6 +28,7 @@ ) )] +pub mod addressing; pub mod asset_cache; pub(crate) mod asset_image_optimizer; pub mod auction; diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index e81bd64f6c..45c483b976 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -24273,6 +24273,47 @@ mod tests { ); } + /// A request that names its host only in its URI, as an HTTP/2 request + /// with no `Host` header does, reaches the entry's middleware, which is + /// told that host. + #[tokio::test] + async fn a_request_naming_its_host_only_in_its_uri_reaches_the_middleware() { + use crate::integrations::registry_test_support::middleware_fixture as fixture; + use crate::middleware::HTML_MEDIA_TYPE; + + let request = HttpRequest::builder() + .method(Method::GET) + .uri("https://publisher.example/article") + .body(EdgeBody::empty()) + .expect("should build request"); + assert!( + request.headers().get(header::HOST).is_none(), + "the request must carry no Host header, or it proves nothing" + ); + let proxied = proxy_answered( + settings_running_the_middleware_fixture(vec![PhaseEntry { + media_type: HTML_MEDIA_TYPE.to_owned(), + path: None, + middleware: vec![fixture::PREPARING.to_owned()], + }]), + request, + 200, + b"the origin's page", + vec![("content-type", "text/html; charset=utf-8")], + ) + .await; + + let (_, page) = streamed(proxied).await; + + assert!( + page.contains(&fixture::prepared_marker( + fixture::PREPARING, + "publisher.example" + )), + "should run the entry's middleware for the host the URI names: {page}" + ); + } + #[tokio::test] async fn a_serve_middleware_prepares_for_each_reader() { use crate::integrations::registry_test_support::middleware_fixture as fixture; diff --git a/crates/trusted-server-core/src/test_support.rs b/crates/trusted-server-core/src/test_support.rs index 495f42a683..fa26b07853 100644 --- a/crates/trusted-server-core/src/test_support.rs +++ b/crates/trusted-server-core/src/test_support.rs @@ -220,6 +220,74 @@ pub mod fixtures { pub const PUBLISHER_PAGE_HTML: &str = include_str!("html_processor.test.html"); } +/// Request addressings and a probe that each adapter's route tests share. +#[cfg(any(test, feature = "test-utils"))] +pub mod addressing { + use edgezero_core::body::Body as EdgeBody; + use edgezero_core::http::request_builder; + use edgezero_core::router::RouterService; + + use crate::addressing::{Addressing, RequestAddress, RequestAddressing}; + + /// A request addressing that continues `//` at `/` + /// and names `segment` as the one selector value, as a deployment serving + /// publishers under a path of their own would. + pub struct FirstSegment; + + impl RequestAddressing for FirstSegment { + fn address(&self, _host: Option<&str>, path: &str) -> Addressing { + first_segment_removed(path, |segment| vec![segment.to_owned()]) + } + } + + /// A request addressing that continues `//` at `/` + /// as [`FirstSegment`] does and names these selector values for every + /// request. + pub struct NamesSelectorValues(pub &'static [&'static str]); + + impl RequestAddressing for NamesSelectorValues { + fn address(&self, _host: Option<&str>, path: &str) -> Addressing { + first_segment_removed(path, |_| { + self.0.iter().map(|value| (*value).to_owned()).collect() + }) + } + } + + /// `path` addressed with its first segment removed and the selector + /// values `values` gives for that segment. + fn first_segment_removed(path: &str, values: impl FnOnce(&str) -> Vec) -> Addressing { + let rest = path.strip_prefix('/').unwrap_or(path); + let (segment, tail) = rest.split_once('/').unwrap_or((rest, "")); + let address = + RequestAddress::new(values(segment), format!("/{tail}"), format!("/{segment}")) + .expect("should name a valid continuing path"); + Addressing::Addressed(address) + } + + /// The status and content type `router` answers a GET for `uri` on + /// `front.example` with. + /// + /// # Panics + /// + /// Panics when the request cannot be built or routed. + pub async fn status_and_type(router: RouterService, uri: &str) -> (u16, String) { + let request = request_builder() + .method("GET") + .uri(uri) + .header("host", "front.example") + .body(EdgeBody::empty()) + .expect("should build request"); + let response = router.oneshot(request).await.expect("should route request"); + let content_type = response + .headers() + .get("content-type") + .and_then(|value| value.to_str().ok()) + .unwrap_or_default() + .to_owned(); + (response.status().as_u16(), content_type) + } +} + /// What a test logs on its own thread, for a test of a log line. /// /// The first use installs a logger for the whole test binary that keeps the diff --git a/docs/guide/configuration.md b/docs/guide/configuration.md index 66c21d4a35..5fef48beba 100644 --- a/docs/guide/configuration.md +++ b/docs/guide/configuration.md @@ -3722,6 +3722,43 @@ serves no host from it. Viceroy instance. It checks that each host is answered from its own blob and that a host with none is refused. +### One blob for each path + +A deployment that serves several publishers from one host, each under a path +of its own such as `https://front.example/publisher-a.example/news`, supplies a +request addressing hook. The hook implements `RequestAddressing` from +`trusted_server_core::addressing`, and a Fastly deployment hands it to +`trusted_server_adapter_fastly::run_with_addressing` with its integration +builders. The hook is asked about each request after `/health` is answered and +before any settings are read. It answers either that the request is addressed +by its host, as in the section above, or three things, being the selector +values, the path the request continues with, and the path prefix it removed. +Which part of a path names a publisher is the deployment's decision. + +- Each selector value takes the place of `{host}` in the selector in turn and + is keyed as a host is, and the first whose blob exists is read. A value whose + key would be over 255 characters is passed over without a lookup. Push a + blob, in lower case, under each value the hook can name. +- A miss on every value is answered `421`, as a host with no blob is. The + request is never served from the logical store ID, from the blob of the host + it arrived on, or from any other key the hook did not name. +- The request continues at the path the hook gives, with its query kept, so the + routes and every module see that path. +- The removed prefix travels on the request as a `RemovedPathPrefix` extension. + Nothing in core writes it into a response. A deployment whose pages must + carry it, in their links for example, puts it back with a serve phase + middleware of its own. +- A page stored for an addressed request is keyed by the host and the + continuing path, so `ts cache purge --page` takes that address. Purging it + also purges the page at the same continuing path of every other publisher on + the host. +- Publishers on one host share the host's cookies unless the deployment's + prefix module names them apart. +- A selector without `{host}` reads its one blob and the values are ignored. + The Axum, Cloudflare and Spin adapters take the hook through + `TrustedServerApp::routes_with_addressing` and ignore the values in the same + way, while the path and the prefix apply. + ### Local development The repository's Viceroy configuration uses the logical app-config store ID and diff --git a/docs/superpowers/specs/2026-10-13-request-addressing-design.md b/docs/superpowers/specs/2026-10-13-request-addressing-design.md new file mode 100644 index 0000000000..443b833296 --- /dev/null +++ b/docs/superpowers/specs/2026-10-13-request-addressing-design.md @@ -0,0 +1,129 @@ +# Design Spec: A deployment may address a request by its path before its configuration is chosen + +**Status:** Proposed, 2026-10-13. Raised as draft pull request IABTechLab/trusted-server#1376 (from jwrosewell/trusted-server#30, which is closed), for Tech Lab review. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-10-12-config-by-host-design.md`, `2026-10-12-middleware-prepare-design.md`, `2026-10-12-prefix-module-design.md`, `2026-08-27-integration-provider-seam-design.md` and `2026-10-15-cookies-by-path-design.md`, all in IABTechLab/trusted-server#1342, which holds every design document of the stack +**Related PRs:** IABTechLab/trusted-server#1376, jwrosewell/trusted-server#30 (closed) +**Issues:** IABTechLab/trusted-server#1328, IABTechLab/trusted-server#1118 +**Last updated:** 2026-10-17 + +> **Why this spec exists.** An adapter chooses the app-config blob that serves a request by the request's host alone, before any settings exist, so one host cannot serve several publishers under paths of their own. + +## 1. Problem + +A Fastly service reads the app-config blob its `__KEY` selector names, and a selector that carries `{host}` names a blob for each host. Every publisher therefore needs a host of its own. A deployment that serves several publishers from one host, as `https://front.example//`, each from its own settings, has no way to say which blob a request is for. The blob is chosen from the host before any settings exist, and the routes, the modules and the origin then see the publisher's part at the start of every path. + +A middleware cannot do this. A middleware is named by settings, so it runs after a blob has been chosen and after the request has been routed, which is too late for both. + +## 2. Design + +Core offers one extension point, `RequestAddressing`, in `crates/trusted-server-core/src/addressing.rs`. A deployment supplies a hook, meaning its own implementation of `RequestAddressing`, and the adapter asks the hook about each request before it chooses the blob and before it routes. The hook is given the request's host and path and answers either "by host", which is how every request is addressed today, or an address made of three things and nothing else. + +```rust +pub trait RequestAddressing: Send + Sync { + fn address(&self, host: Option<&str>, path: &str) -> Addressing; +} + +pub enum Addressing { + ByHost, + Addressed(RequestAddress), +} + +impl RequestAddress { + pub fn new( + selector_values: Vec, + path: String, + removed_prefix: String, + ) -> Result>; +} +``` + +1. **The selector values**, in order. Each takes the place of `{host}` in the `__KEY` selector in turn, and the first whose blob exists is read. +2. **The path the request continues with**, which is set on the request before routing. The query is kept. +3. **The removed path prefix**, as a string. It travels on the core request as a `RemovedPathPrefix` extension, where a module's request preparer or a handler reads it. + +`ByHostAddressing` answers "by host" for every request. The Fastly adapter uses it when a deployment registers no hook, and the other three adapters ask a hook only when a deployment builds its router with `routes_with_addressing`, so a deployment that supplies none is served exactly as before. + +A deployment's policy, such as which part of the path names a publisher, lives in the deployment and not in this repository. On a Fastly service whose `__KEY` selector is `{host}`, the hook below gives the results in the table. + +```rust +impl RequestAddressing for FirstSegment { + fn address(&self, _host: Option<&str>, path: &str) -> Addressing { + let rest = path.strip_prefix('/').unwrap_or(path); + let (segment, tail) = rest.split_once('/').unwrap_or((rest, "")); + RequestAddress::new(vec![segment.to_owned()], format!("/{tail}"), format!("/{segment}")) + .map_or(Addressing::ByHost, Addressing::Addressed) + } +} +``` + +| A request for | Reads the blob at | Is routed as | Carries the removed path prefix | +| ---------------------------------------------------- | -------------------------- | ------------------------------------------- | ------------------------------- | +| `https://front.example/publisher-a.example/news?p=2` | `publisher-a.example` | `/news?p=2` | `/publisher-a.example` | +| `https://front.example/publisher-b.example/` | `publisher-b.example` | `/` | `/publisher-b.example` | +| `https://front.example/unknown.example/news` | None | `421 Misdirected Request`, not to be stored | None | +| `https://front.example/not_a_host/news` | None, nothing is looked up | `421 Misdirected Request` | None | +| `https://front.example/health` on Fastly | None | `200`, before the hook is asked | None | + +A selector that carries no `{host}` ignores the selector values, so given a hook it serves every addressed request from its one blob, as a staged version does. + +**The rules** are those the guide gives under "One blob for each path" in `docs/guide/configuration.md`. How a value is keyed and why a failed lookup stops the search are in the rustdoc of `resolve_addressed_key`, and what a continuing path may be is in that of `RequestAddress::new`. The values and the path come from the request, so no log line prints either. Each value costs one store lookup until one is found, so a hook names as few values as it can, usually one. `RequestAddress::new` refuses a continuing path that does not start with `/`, carries a query or a fragment, or is not a valid URI path, but it does not refuse `.` or `..` segments, so a hook returns a path without them. + +**When it fails.** Every value missing is answered `421 Misdirected Request` with `Cache-Control: private, no-store`, as the table shows. A store that cannot be read is answered `500` on Fastly, and on the other three adapters `addressed_router` answers `500` when the continuing path cannot be set on the request. Neither is served from another publisher's blob. + +**Rollout.** A deployment that supplies no hook changes nothing. One that adopts a hook on a Fastly service whose selector carries `{host}` writes each publisher's blob before it deploys the build that carries the hook, because a request addressed to a value with no blob is answered `421`. The other three adapters ignore the selector values, so a hook there needs no blob written first. The Fastly log line for a failed lookup now names the config store but not the key, because the key is built from the request. + +**Why it is in core and not a middleware.** No settings exist when the blob is chosen, and the router has already matched a path by the time a middleware runs. Only a step the adapter takes before both can choose the blob and change the path the router sees, so the extension point is in core, and the policy that uses it is the deployment's. + +**Why the values take the place of `{host}`.** The selector already carries the placeholder, and a staged deploy already replaces the selector with one that carries none. A staged version of a service that addresses requests by path therefore reads its one staging blob for every request and never a production one, as a service that chooses by host does. The values also pass the same host name check, so a value reaches no key a host could not. A sandbox that serves several requests still keeps each application under its blob's key. Two publishers on one host do not share a stored page, because the stored template's key carries the origin and a digest of the blob's settings as well as the host and the path, and the asset cache's key carries the origin as well as the host and the URL (`crates/trusted-server-core/src/asset_cache.rs`). + +**How each adapter takes the hook.** Each adapter takes it beside the integration builders a deployment offers, in the same call. + +| Adapter | Takes the hook through | Gives the hook the host from | Chooses a blob for each request | +| ---------- | ------------------------------------------------------------------------------- | ------------------------------------------ | ----------------------------------- | +| Fastly | `trusted_server_adapter_fastly::run_with_addressing(integrations, addressing)` | The `Host` header | Yes, by the selector values | +| Axum | `TrustedServerApp::routes_with_addressing(settings, &integrations, addressing)` | The `Host` header, or else the authority | No, the selector values are ignored | +| Cloudflare | `TrustedServerApp::routes_with_addressing(settings, &integrations, addressing)` | The `Host` header, or else the authority | No, the selector values are ignored | +| Spin | `TrustedServerApp::routes_with_addressing(settings, &integrations, addressing)` | Usually nothing, as the next lines explain | No, the selector values are ignored | + +On Spin the hook is usually given no host, because Spin rarely passes the `Host` header and its request URI carries only the path. A `Host` header that is present is the client's own, because the hook is asked before `NormalizeMiddleware` sets the trusted host, so a hook on Spin reads the path alone. The shipped entry points of the Axum, Cloudflare and Spin adapters build their router from `TrustedServerApp::routes()`, so a deployment that supplies a hook there builds its own entry point that calls `routes_with_addressing`. + +`run_with` and `routes_with_registrations` are unchanged, so no existing caller changes. The Fastly adapter records the hook before any request is served, as it records the builders. It answers `/health` first, then asks the hook, then serves the debug probes, so the probes read the blob and the path the request is addressed to. The other three adapters read one app-config blob, so `addressed_router` in core wraps the router they build. It routes every path for each method the inner router registers, applies the hook, and hands the request on, so a path or method the inner router refuses is refused with its own answer. There the hook is asked about every path, `/health` included where the adapter routes it, so a hook that wants the probe answered at the root answers "by host" for `/health`. The wrapped router reports its own catch-all routes as its route table, not the inner router's. + +**Putting the removed path prefix back.** Core never writes the removed path prefix into a response. A deployment whose pages must carry it, in their links for example, does that with a middleware of its own, whose module's request preparer reads `RemovedPathPrefix` and leaves it for the middleware in `IntegrationRequestState`. That middleware runs in the serve phase. This has a cost. A request that carries any value in `IntegrationRequestState` keeps to the origin path, so its page is never read from a stored template or stored as one, and its HTML is sent `private, no-store` (see `IntegrationRequestState` in `crates/trusted-server-core/src/integrations/registry.rs`). Every page whose removed path prefix is put back this way is therefore made for its reader alone. The request to the origin for that page bypasses the platform's cache only when origin readthrough is on or the ad stack runs (`apply_origin_cache_intent` in `crates/trusted-server-core/src/publisher.rs`). Origin readthrough is off by default (`origin_readthrough_enabled`, see "Origin readthrough caching" in `docs/guide/configuration.md`), so with no ad stack the platform's cache can still answer under the origin's own `Cache-Control`. The stored template is keyed by the request's host and continuing path and by its blob's settings, not by the removed path prefix, so a page changed in the fetch phase could be stored with one removed path prefix and served under another that names the same blob. + +**The host core processes a request for.** Once a request is routed, core reads the host it rewrites a page for and tells each middleware from `Forwarded`, then `X-Forwarded-Host`, then `Host`, and last from the URI's authority, as `address_request` gives the hook the authority where no `Host` is sent (`RequestInfo::from_request` in `crates/trusted-server-core/src/http_util.rs`). An HTTP/2 client may send its `:authority` and no `Host` header, and before the authority was read such a request had no host, so its page was returned unprocessed (IABTechLab/trusted-server#1118). + +## 3. What is not in this change + +- **The deployment's policy.** Which part of a path names a publisher, and how a value is spelled, belong to the deployment that supplies the hook. +- **Writing the removed path prefix into pages**, which is a middleware outside core. +- **Choosing a blob for each request on Axum, Cloudflare and Spin.** Those adapters read one app-config blob, as `2026-10-12-config-by-host-design.md` records. +- **Purging one page by its address.** `ts cache purge --page` derives its key from the address it is given, and a page stored for an addressed request is keyed by the front host and its continuing path. Such a page is purged by that address, which also purges the page at the same continuing path of every other publisher on the host. +- **Keeping publishers on one host apart in the browser.** Publishers addressed by path on one host are one browser origin, so they share its cookies and first-party storage. A deployment's prefix module names apart only the cookies core names through `cookie_name`, such as the Edge Cookie family and the `eids` cookie (`2026-10-12-prefix-module-design.md`). The consent cookies and the `sharedId` cookie have fixed names that core reads, so consent given on one publisher's pages is read on another's. The Cookie header also reaches the chosen origin unchanged, so each publisher's origin receives the cookies the browser holds for the others. `2026-10-15-cookies-by-path-design.md` keeps those cookies with the publisher. +- **The auction under a removed path prefix.** The client script posts to the root-relative `/auction` (`crates/trusted-server-js/lib/src/core/request.ts`), which carries no publisher's part, so on a host addressed by path the hook cannot tell which publisher an auction is for. Taking the script's requests under the publisher's path is taken up in `2026-10-15-cookies-by-path-design.md`. +- **`/health` under a removed path prefix on Fastly.** The Fastly adapter answers `/health` before the hook is asked, so the probe stays free of settings and of the hook, and `//health` is addressed like any other path. + +## 4. Acceptance + +Tests in `crates/trusted-server-core/src/addressing.rs` cover the default answer, the path rules, a request left as it was when addressed by host, a rewritten path with its query and authority kept and its removed path prefix carried, the host offered to the hook, and `addressed_router` routing the continuing path, routing as before with no hook or a "by host" hook, and refusing a method the inner router refuses. + +Tests in `crates/trusted-server-adapter-fastly/src/config_selection.rs` cover a value taking the placeholder's place, the first existing value winning with no later lookup, every value missing refused and not served from the logical store ID or another host's blob, a value that is not a host name passed over without a lookup, a selector without the placeholder ignoring the values, and a failed lookup refusing and trying no later value. Tests in `crates/trusted-server-adapter-fastly/src/lib.rs` drive the entry point's selection under Viceroy, with no hook and with a "by host" hook choosing as before, the registered hook being the one asked, a rewritten request converted with its path, query and removed path prefix, the first existing value read from a real local store, and every value missing answered `421` with the request left as it was. Each of the Axum, Cloudflare and Spin route tests serves `//_ts/config.json` as the configuration endpoint through `routes_with_addressing`, and routes a "by host" request as the router with no hook does. + +Tests in `crates/trusted-server-core/src/http_util.rs` cover the host read from the URI's authority, with its port, where no header names one, no host where nothing names one, and each of `Host`, `X-Forwarded-Host` and `Forwarded` taking precedence over the authority. `a_request_naming_its_host_only_in_its_uri_reaches_the_middleware`, in `crates/trusted-server-core/src/publisher.rs`, sends a request with no `Host` header through the publisher path and finds that the entry's middleware ran for the host the URI names. + +Two paths have no test. No test drives `edgezero_main` with a hook registered through `run_with_addressing`, which appears only in a doctest that is not run, and no test covers the `500` that `addressed_router` answers when the continuing path cannot be set on the request. + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/35-request-addressing` at `0af81b181`. Issues found, two of them blocking. "Putting the removed path prefix back" left out that a page whose removed path prefix is put back is never stored and is sent `private, no-store`, and section 3 claimed publishers on one host could be kept apart in the browser when the consent cookies, the `sharedId` cookie and the forwarded Cookie header are shared. Both are fixed in this revision, with two of the three partly true claims, the coverage gaps and the advice the round gave. The third partly true claim concerned the copy of this page on `split/6-integration-seam`, which still lists the rules in full where this page points to the guide and the rustdoc, and that copy is brought up to date in the specs pull request IABTechLab/trusted-server#1342. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `d58b3f404`, of the round 1 fixes and the new text. No blocking issues. This revision corrects the claim that every page whose removed path prefix is put back is fetched from the origin for each reader, limits the rollout order to a Fastly service whose selector carries `{host}`, says in the round 1 entry that the copy of this page on `split/6-integration-seam` is brought up to date in the specs pull request, and writes "removed path prefix" wherever the text means it. + +## Revision record + +| Date | Change | +| ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-13 | First draft. | +| 2026-10-15 | Fixes from review round 1. The cost of putting the removed path prefix back, the shared browser origin as a limit, the host each adapter gives the hook, failure handling, rollout, the untested paths and the review log. | +| 2026-10-15 | Corrections from review round 2. The origin fetch of a page whose removed path prefix is put back, the rollout order on Fastly, where the copy on `split/6-integration-seam` is brought up to date, and the removed path prefix named in full. | +| 2026-10-17 | The host core processes a request for falls back to the URI's authority where no header names one (IABTechLab/trusted-server#1118), with its tests. |