Address requests by path before choosing settings

CHANGELOG.md

@@ -96,6 +96,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Protocol-relative creative URLs now honor `rewrite.exclude_domains`, so excluded creative assets stay direct and excluded absolute or protocol-relative URLs submitted to `/first-party/sign` are rejected.
 - Server-side ad template bids now always carry `hb_adid` in `window.tsjs.bids`. Bidders that return neither a Prebid Cache UUID nor an `adid` previously produced no `hb_adid` at all, so no `hb_adid` GPT targeting key was set and the Universal Creative render bridge had nothing to match — the winning creative never rendered. The OpenRTB bid `id`, which is mandatory per spec, is now the last-resort source; `cache_id` and `adid` still take priority where present. Blank `cacheId`/`adid` values no longer win that precedence and emit an unusable empty `hb_adid`, and `hb_cache_host`/`hb_cache_path` are now emitted only alongside a real Prebid Cache UUID — without one they pointed the Universal Creative at a guaranteed cache miss instead of letting it fall through to the inline creative.
 - The origin's address in a CSS, JavaScript, JSON or other text response that core rewrites without parsing it is now replaced in one pass. A request host that contains the origin host was matched again inside its own replacement, so `https://example.com/` served on `www.example.com` came out as `https://www.www.example.com/`, and `https://publisher.example/` served on `publisher.example.localhost` over `http` as `http://publisher.example.localhost.localhost.localhost.localhost/`.
+- A request that names its host only in its URI's authority, as an HTTP/2 request with no `Host` header may, is now processed. The request host was read from `Forwarded`, `X-Forwarded-Host` and `Host` alone, so such a request had an empty host and its page was returned unmodified, with no URL rewriting, middleware, injected script or auction. The authority is read last, so each of those headers still takes precedence ([#1118](https://github.com/IABTechLab/trusted-server/issues/1118)).
 
 ### Added
 
@@ -122,6 +123,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - The stock modules' fetch middleware give as their `template_key` the text compiled into their crates that they write into a page, so a change to it stores pages again. The Google ad tag gives its head scripts, the `tsjs.adInit` bootstrap among them, and the GPT script's first-party path, Prebid its configuration script and bundle tag, Didomi its settings script, and Sourcepoint its head scripts and its CDN's first-party path. Permutive, Lockr, DataDome and Google Tag Manager give the first-party path they point a vendor's address at.
 - `[robots-txt]` serves a publisher's `/robots.txt` from the settings. `modules` names what makes the file, in order. `refuse_all` and `allow_all` are Trusted Server's own, and any other name is a module that contributes rules through the new `RobotsTxtContributor` extension point, declared with `with_robots_txt_contributor`. Trusted Server writes the file, so a contributor cannot produce a malformed one, a refusal cannot be loosened, and a path in `always_allow` stays open to every crawler refused the whole site. With `refuse_all` selected every response an adapter finalizes carries `X-Robots-Tag: noindex, nofollow`. A module is asked for the host the request was addressed to, never one a forwarded header names, and is given nothing of the crawler's connection. Its answer is held for that host by the loading cache until its refresh is due. A module that cannot answer has its last held answer served, and with none held, or with no rule in the whole file, the answer is `503` and never an empty file. On Fastly the file is answered before the Edge Cookie steps and the request filters. Without the section `/robots.txt` reaches the publisher's origin as before.
 - `ts keys rotate` and `ts keys deactivate` rotate and retire the request signing keys from an operator's machine. They write the two stores the service links as `jwks_store` and `signing_keys` through the `fastly` CLI, with the token from `FASTLY_API_TOKEN`. A private key reaches `fastly` on its standard input and never in its arguments, and a key id no platform can store is refused before anything is written. A command that cannot read the config store writes nothing, and a delete that failed part way can be run again.
+- A deployment may address a request by its path before its configuration is chosen, so one host serves several publishers, each under a path of its own and from its own app-config blob. It supplies a `RequestAddressing` from `trusted_server_core::addressing`, through `trusted_server_adapter_fastly::run_with_addressing` or `TrustedServerApp::routes_with_addressing` on the other adapters. The hook answers either that a request is addressed by its host, as before, or the selector values that take the place of `{host}`, the path the request continues with and the prefix it removed. The first value whose blob exists is read and a miss on every value is answered `421`, never from a key the hook did not name. The request is routed at the continuing path with its query kept, and carries the prefix as a `RemovedPathPrefix` extension, which a deployment's own middleware puts back into its pages. Adapters that read one blob ignore the values. With no hook every request is served as before.
 - One Fastly service can serve several publishers. A `TS__CONFIG__KEY` entry in the `trusted_server_config` store that contains `{host}` names an app-config blob for each host, pushed with `ts config push --key <host>`, and each request reads the blob under its own host. Each host's blob names its permission policy file and pins its SHA-256, and the push keeps each file `--permissions` supplies once, in a shared entry under `permissions_<sha256>`, written before the blob, which a live push writes a minute after it writes an entry. A host with no blob is answered `421 Misdirected Request`, never from the logical store ID or from another host's blob, a malformed selector is answered `500`, and a sandbox that serves several requests keeps an application for each blob, found only by that blob's key. Without the entry, or with a selector that has no placeholder, the service reads one blob. `scripts/config-by-host-local-test.sh` runs it under Viceroy.
 - `ts audit generate` writes the `[[fetch]]` and `[[serve]]` entries that run the page changes of the modules it selects on every HTML page, and `trusted-server.example.toml` documents the entries with every middleware the stock modules supply. `trusted-server-modules` lists those middleware with `middleware()` and `middleware_for(...)`.
 - Page changes as middleware. A module registers a change it makes to a page with `.with_middleware(...)`, and the ordered `[[fetch]]` and `[[serve]]` entries of the settings say which pages each one runs on and in what order. A fetch middleware runs on the page as the origin sent it and what it writes is stored with a shared template. A serve middleware runs on each reader's copy, whether the page came from the store or from the origin, and nothing it writes is stored. Where text a middleware writes into a script would end the script, or begin `<script` after `<!--`, its `<` is written as `\u003c`. An entry with no path after one with `path = "/"` is refused, because both cover every page.

crates/trusted-server-adapter-axum/src/app.rs

@@ -10,6 +10,7 @@ use edgezero_core::http::{
 };
 use edgezero_core::router::RouterService;
 use error_stack::Report;
+use trusted_server_core::addressing::{RequestAddressing, addressed_router};
 use trusted_server_core::auction::endpoints::handle_auction;
 use trusted_server_core::auction::telemetry::warn_once_when_analytics_is_unsent;
 use trusted_server_core::auction::{
@@ -807,6 +808,29 @@ impl TrustedServerApp {
         Ok(build_router(&state))
     }
 
+    /// Build the full application router as
+    /// [`TrustedServerApp::routes_with_registrations`] does, asking
+    /// `addressing` about every request before it is routed.
+    ///
+    /// This adapter reads one app-config blob, so the selector values the
+    /// addressing names are ignored. A request it addresses continues at the
+    /// path it gives, with its query kept, and carries the prefix it removed
+    /// as a [`RemovedPathPrefix`](trusted_server_core::addressing::RemovedPathPrefix)
+    /// extension. A request it addresses by host is routed as it is.
+    ///
+    /// # Errors
+    ///
+    /// Returns the errors [`TrustedServerApp::routes_with_registrations`]
+    /// returns.
+    pub fn routes_with_addressing(
+        settings: Settings,
+        integrations: &[IntegrationBuilder],
+        addressing: Arc<dyn RequestAddressing>,
+    ) -> Result<RouterService, Report<TrustedServerError>> {
+        let router = Self::routes_with_registrations(settings, integrations)?;
+        Ok(addressed_router(router, addressing))
+    }
+
     /// Build the full router with explicit settings and runtime services.
     ///
     /// Each request receives a clone of the supplied services, allowing callers

crates/trusted-server-adapter-axum/tests/routes.rs

@@ -4,11 +4,15 @@
 //! and self-contained. Each test builds the full `TrustedServerApp` router and
 //! drives it through the Tower `Service` interface.
 
+use std::sync::Arc;
+
 use axum::body::Body as AxumBody;
 use axum::http::Request;
 use edgezero_adapter_axum::service::EdgeZeroAxumService;
 use tower::{Service as _, ServiceExt as _};
 use trusted_server_adapter_axum::app::TrustedServerApp;
+use trusted_server_core::addressing::ByHostAddressing;
+use trusted_server_core::test_support::addressing::{FirstSegment, status_and_type};
 
 const LEGACY_ADMIN_DENY_METHODS: &[&str] =
     &["GET", "POST", "HEAD", "OPTIONS", "PUT", "PATCH", "DELETE"];
@@ -1088,3 +1092,29 @@ async fn robots_txt_is_served_through_the_publisher_path_with_its_tag() {
         "each request reaches the origin"
     );
 }
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn a_request_addressing_continues_the_request_at_its_path_before_routing() {
+    let router =
+        TrustedServerApp::routes_with_addressing(test_settings(), &[], Arc::new(FirstSegment))
+            .expect("should build router with a request addressing");
+
+    assert_eq!(
+        status_and_type(router, "/publisher.example/_ts/config.json?view=1").await,
+        (200, "application/json".to_owned()),
+        "the configuration endpoint should answer at the path the addressing continues with"
+    );
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn a_request_addressed_by_host_is_routed_as_it_is() {
+    let router =
+        TrustedServerApp::routes_with_addressing(test_settings(), &[], Arc::new(ByHostAddressing))
+            .expect("should build router with a request addressing");
+
+    assert_eq!(
+        status_and_type(router, "/_ts/config.json").await,
+        status_and_type(test_router(), "/_ts/config.json").await,
+        "should answer as the router with no addressing answers"
+    );
+}

crates/trusted-server-adapter-cloudflare/src/app.rs

@@ -9,6 +9,7 @@ use edgezero_core::error::EdgeError;
 use edgezero_core::http::{HeaderValue, Method, Request, Response, header};
 use edgezero_core::router::RouterService;
 use error_stack::Report;
+use trusted_server_core::addressing::{RequestAddressing, addressed_router};
 use trusted_server_core::auction::endpoints::handle_auction;
 use trusted_server_core::auction::telemetry::warn_once_when_analytics_is_unsent;
 use trusted_server_core::auction::{
@@ -623,6 +624,29 @@ impl TrustedServerApp {
         Ok(build_router(&state))
     }
 
+    /// Build the full application router as
+    /// [`TrustedServerApp::routes_with_registrations`] does, asking
+    /// `addressing` about every request before it is routed.
+    ///
+    /// This adapter reads one app-config blob, so the selector values the
+    /// addressing names are ignored. A request it addresses continues at the
+    /// path it gives, with its query kept, and carries the prefix it removed
+    /// as a [`RemovedPathPrefix`](trusted_server_core::addressing::RemovedPathPrefix)
+    /// extension. A request it addresses by host is routed as it is.
+    ///
+    /// # Errors
+    ///
+    /// Returns the errors [`TrustedServerApp::routes_with_registrations`]
+    /// returns.
+    pub fn routes_with_addressing(
+        settings: Settings,
+        integrations: &[IntegrationBuilder],
+        addressing: Arc<dyn RequestAddressing>,
+    ) -> Result<RouterService, Report<TrustedServerError>> {
+        let router = Self::routes_with_registrations(settings, integrations)?;
+        Ok(addressed_router(router, addressing))
+    }
+
     /// Build the full router with explicit settings and runtime services.
     ///
     /// Each request receives a clone of the supplied services, allowing callers

crates/trusted-server-adapter-cloudflare/tests/routes.rs

@@ -4,11 +4,15 @@
 //! `TrustedServerApp::routes()` builds without panicking. Does not exercise
 //! the platform layer or outbound network calls.
 
+use std::sync::Arc;
+
 use edgezero_core::app::Hooks as _;
 use edgezero_core::http::{Request, Response, request_builder};
 use edgezero_core::router::RouterService;
 use trusted_server_adapter_cloudflare::app::TrustedServerApp;
+use trusted_server_core::addressing::ByHostAddressing;
 use trusted_server_core::settings::Settings;
+use trusted_server_core::test_support::addressing::{FirstSegment, status_and_type};
 
 const LEGACY_ADMIN_DENY_METHODS: &[&str] =
     &["GET", "POST", "HEAD", "OPTIONS", "PUT", "PATCH", "DELETE"];
@@ -24,7 +28,7 @@ fn test_router() -> RouterService {
         .expect("should build router from test settings")
 }
 
-/// The settings [`test_router`] is built from.
+/// Explicit settings for the route tests.
 fn test_settings() -> Settings {
     Settings::from_toml(
         r#"
@@ -1084,3 +1088,29 @@ async fn a_response_carries_no_blanket_rule_without_the_section() {
 
     assert_eq!(robots_tag(&response), None);
 }
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn a_request_addressing_continues_the_request_at_its_path_before_routing() {
+    let router =
+        TrustedServerApp::routes_with_addressing(test_settings(), &[], Arc::new(FirstSegment))
+            .expect("should build router with a request addressing");
+
+    assert_eq!(
+        status_and_type(router, "/publisher.example/_ts/config.json?view=1").await,
+        (200, "application/json".to_owned()),
+        "the configuration endpoint should answer at the path the addressing continues with"
+    );
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn a_request_addressed_by_host_is_routed_as_it_is() {
+    let router =
+        TrustedServerApp::routes_with_addressing(test_settings(), &[], Arc::new(ByHostAddressing))
+            .expect("should build router with a request addressing");
+
+    assert_eq!(
+        status_and_type(router, "/_ts/config.json").await,
+        status_and_type(test_router(), "/_ts/config.json").await,
+        "should answer as the router with no addressing answers"
+    );
+}

crates/trusted-server-adapter-fastly/src/app.rs

@@ -161,7 +161,7 @@ use crate::platform::{
 /// [`Self::logical`] gives the logical stores and key. The entry point puts
 /// the selector the config store holds in place of the key and resolves it
 /// for the request before anything is built, with
-/// `stores_with_stored_selector` and `stores_for_request`.
+/// `stores_with_stored_selector` and `address_request`.
 #[derive(Debug, Clone, PartialEq, Eq)]
 pub(crate) struct RuntimeStoreConfig {
     pub(crate) config_store_name: StoreName,