diff --git a/CHANGELOG.md b/CHANGELOG.md index 8f63e52030..4cae2f48d5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -40,7 +40,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Breaking:** Settings must name the publisher's permission policy in `[permissions] policy`, and its file is supplied with `ts config push --permissions`. A domain without a valid policy no longer starts. No adapter build carries a policy anymore, and the policy a domain loads decides the permissions of each request it serves. The push writes the SHA-256 of the file's exact bytes into the blob and, on Fastly, Cloudflare and Spin, the file itself, and Axum reads `.yaml` from the folder `TRUSTED_SERVER_PERMISSIONS_DIR` names. A domain whose settings name no policy, or whose file is missing, is not the file that was pushed or does not parse, answers `500`. `ts config validate` checks the policy when given `--permissions`, and `ts config diff` needs the file whenever the settings name a policy, because the stored settings carry its SHA-256. - **Breaking:** The Cloudflare adapter reads its settings from its config store, the Workers KV namespace `ts config push` writes, bound as `trusted_server_config` by default, and no longer reads the `TRUSTED_SERVER_CONFIG` Worker variable, which holds at most 5 KB. Bind the namespace in `wrangler.toml` and push into it. - A Spin service whose settings fail to load answers with the status of the startup error's kind, `500` for a configuration or settings error and `503` for a key-value store error, as the other adapters do, where it answered `503` for every kind. -- **Breaking:** `x-ts-version` now reports the deployed git version — the release tag, else the branch, else the first 6 characters of the commit — compiled in from the build-time `TRUSTED_SERVER__GIT_VERSION` (set by the deploy pipeline) or local git, and is sent by every adapter, including on the Fastly `GET /health` probe. The Fastly service version it previously carried is now `x-ts-fastly-version`; update dashboards, monitors, and scripts that read `x-ts-version` as the Fastly version number. Builds with neither the override nor git omit the header. +- **Breaking:** No adapter writes `x-ts-version` on a response, and the Fastly adapter no longer writes the service version. The build compiles the deployed git version in from the build-time `TRUSTED_SERVER__GIT_VERSION` (set by the deploy pipeline) or local git, being the release tag, else the branch, else the first 6 characters of the commit, and core's `service_version` middleware writes it as `x-ts-version`, with the platform's service version as `x-ts-service-version`, on the pages a `[[serve]]` entry names it for. Update dashboards, monitors and scripts that read either from a page, or name the middleware. - **Breaking:** `ts prebid bundle` is now `ts prebid client`, alongside the new `ts prebid server` namespace. Update scripts and runbooks to use `ts prebid client` with the same arguments. The old `bundle` spelling is no longer accepted and has no compatibility alias. - The S2S `/_ts/api/v1/batch-sync` endpoint now validates the full batch and calls the CAS-protected update path once per distinct normalized EC ID. The last valid UID wins within a group, and infrastructure failures reject the failing and each unprocessed group, so accepted and `kv_unavailable` input indexes may interleave. - **Breaking:** Auction providers and bidder routes now use the configuration-first `[auction.providers.]` and `[auction.bidders.]` maps. The removed `[auction].providers = [...]` list and removed server fields under `[integrations.prebid]` and `[integrations.aps]` are rejected even when those integrations are disabled, and `ts config push` rejects the old shape before publication. Move PBS `server_url` to provider `endpoint`, server timeout to provider `timeout_ms`, request controls and bidder-parameter overrides to the `prebid-server` `profile_config`, notification suppression to `notifications`, and each former server bidder to an `[auction.bidders.]` route. Move APS endpoint, timeout, account, inventory, debug, and creative controls to an `aps` provider and its `profile_config`. Browser Prebid settings remain under `[integrations.prebid]`; values such as timeout and debug that previously affected both browser and server behavior must now be configured for each owner. Provider endpoints must be absolute HTTPS URLs. Only bidder codes present in `[auction.bidders]` are folded into Trusted Server requests; unlisted publisher bids remain native browser demand. Provider response names now use the configured provider ID, such as `pbs-main`, instead of the legacy literal `prebid`; audit consumers that match `AuctionResponse.provider`. This schema has no mixed-version-safe deployment order: old binaries reject the maps and new binaries reject the retired fields, so activate the new binary and config blob together. Rollbacks must restore an old-schema blob together with the old binary. @@ -67,6 +67,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Security +- A response no longer carries `X-Served-By`, `X-Cache` or `X-Cache-Hits`, unless this server answered a request for a path beneath `/_ts`. The three name the node that answered and say how a cache treated the request, and arrive from the publisher's origin or a cache on the way to it. Every adapter removes them before it finalizes a response, so one set in `[response_headers]` is still sent. +- No adapter writes a version on a response. A deployment that wants the versions on its pages names core's `service_version` middleware in a `[[serve]]` entry, which writes `X-TS-Version` from the git version the build compiled in and `X-TS-Service-Version` from the version the adapter reports. - A module's table that cannot be read or fails its validation is refused naming the setting at fault and never its value, because at startup the table holds the module's secrets resolved from the secret store and each adapter logs a refusal whole. - `/first-party/sign` now rejects valid targets outside `proxy.allowed_domains` before minting a proxy token. The creative runtime keeps image and iframe assignments blocked after this `403` policy response instead of loading the rejected URL directly; fetch-time checks still cover the initial target and every redirect. - The `/_ts/admin` prefix and the `/admin/keys` aliases are closed at the publisher-fallback boundary on every adapter. A request to either, including trailing, descendant and encoded-separator forms, is answered `404` by the server itself, so a caller's `Authorization` header and body are never forwarded to the publisher origin. The path is evaluated as sent and on each of its bounded percent-decodings, so multi-encoded separators such as `/admin%252Fkeys/rotate` cannot survive to fallback for a proxy or origin to decode again. diff --git a/crates/trusted-server-adapter-axum/src/app.rs b/crates/trusted-server-adapter-axum/src/app.rs index f418265fef..165f6250bd 100644 --- a/crates/trusted-server-adapter-axum/src/app.rs +++ b/crates/trusted-server-adapter-axum/src/app.rs @@ -44,7 +44,7 @@ use trusted_server_core::settings_data::{ default_config_key, default_config_store_name, get_settings_from_config_store_with, }; -use trusted_server_core::platform::{PolicyOrigin, RuntimeServices}; +use trusted_server_core::platform::{PlatformIdentity, PolicyOrigin, RuntimeServices}; use crate::middleware::{FinalizeResponseMiddleware, SanitizeRequestMiddleware}; use crate::platform::{ @@ -183,7 +183,8 @@ fn build_state_with_registrations_and_services( let plan = Arc::new(compile_auction_plan_with(&settings, integrations)?); plan.validate_for_target(trusted_server_core::platform::AuctionTargetId::Axum)?; let orchestrator = build_orchestrator_with_plan(Arc::clone(&plan))?; - let registry = IntegrationRegistry::with_plan_and_registrations(&settings, plan, integrations)?; + let registry = IntegrationRegistry::with_plan_and_registrations(&settings, plan, integrations)? + .with_platform(PlatformIdentity::named("axum")); // Composition root: reject a module selection this adapter can never // supply, once, before any request is served. The registry is built first diff --git a/crates/trusted-server-adapter-axum/src/middleware.rs b/crates/trusted-server-adapter-axum/src/middleware.rs index 41c226abeb..afeb310356 100644 --- a/crates/trusted-server-adapter-axum/src/middleware.rs +++ b/crates/trusted-server-adapter-axum/src/middleware.rs @@ -6,7 +6,9 @@ use edgezero_core::error::EdgeError; use edgezero_core::http::{HeaderValue, Response}; use edgezero_core::middleware::{Middleware, Next}; use trusted_server_core::constants::HEADER_X_GEO_INFO_AVAILABLE; -use trusted_server_core::http_util::sanitize_trusted_client_ip_headers; +use trusted_server_core::http_util::{ + ROUTE_NAMESPACE, sanitize_trusted_client_ip_headers, strip_deployment_headers, +}; use trusted_server_core::settings::Settings; // --------------------------------------------------------------------------- @@ -52,9 +54,8 @@ impl Middleware for SanitizeRequestMiddleware { /// Response-finalization middleware: injects all standard TS response headers. /// /// Geo lookup is unavailable in the Axum dev server — `X-Geo-Info-Available: false` -/// is always emitted. `X-TS-Version` carries the compiled-in git version. The -/// Fastly-specific headers (`X-TS-Fastly-Version`, `X-TS-ENV`) are skipped -/// because the corresponding env vars are not set in a local dev context. +/// is always emitted. The Fastly adapter's `X-TS-ENV` is skipped because its +/// env var is not set in a local dev context. /// /// Registered directly inside [`SanitizeRequestMiddleware`], so that every /// outgoing response carries a consistent set of headers. @@ -73,7 +74,10 @@ impl FinalizeResponseMiddleware { #[async_trait(?Send)] impl Middleware for FinalizeResponseMiddleware { async fn handle(&self, ctx: RequestContext, next: Next<'_>) -> Result { + let path = ctx.request().uri().path().to_owned(); let mut response = next.run(ctx).await?; + // Before finalizing, so a header finalizing writes is kept. + strip_deployment_headers(&path, ROUTE_NAMESPACE, &mut response); apply_finalize_headers(&self.settings, &mut response); Ok(response) } @@ -86,8 +90,7 @@ impl Middleware for FinalizeResponseMiddleware { /// Applies standard Trusted Server response headers to the given response. /// /// Unlike the Fastly variant, geo is always unavailable so `X-Geo-Info-Available: false` -/// is unconditionally emitted, followed by the compiled-in `X-TS-Version`. -/// Fastly-specific headers are omitted. +/// is unconditionally emitted. Fastly-specific headers are omitted. /// Operator-configured `settings.response_headers` are applied last (with the /// shared cookie cache-privacy hardening) and can override any managed header. pub(crate) fn apply_finalize_headers(settings: &Settings, response: &mut Response) { @@ -96,8 +99,6 @@ pub(crate) fn apply_finalize_headers(settings: &Settings, response: &mut Respons HeaderValue::from_static("false"), ); - trusted_server_core::version_header::apply_git_version_header(response); - // Cookie-bearing responses stay private to shared caches and operator // headers cannot re-enable caching for uncacheable per-user payloads. trusted_server_core::response_privacy::apply_response_headers_with_cache_privacy( @@ -257,17 +258,58 @@ mod tests { ); } + fn ctx_for(path: &str) -> RequestContext { + let req = request_builder() + .method(Method::GET) + .uri(path) + .body(Body::empty()) + .expect("should build test request"); + RequestContext::new(req, PathParams::new(HashMap::new())) + } + + /// A response as an origin behind a cache sends it, naming the node that + /// answered and how the cache treated the request. + fn response_naming_its_node() -> Response { + response_builder() + .header("x-served-by", "cache-node-1") + .header("x-cache", "HIT") + .header("x-cache-hits", "3") + .body(Body::empty()) + .expect("should build test response") + } + + /// What the finalize middleware makes of that response for a request to + /// `path`. + fn finalized(path: &str, settings: Settings) -> Response { + let middleware = FinalizeResponseMiddleware::new(Arc::new(settings)); + let handler = Arc::new(|_ctx: RequestContext| async move { + Ok::(response_naming_its_node()) + }); + + block_on(middleware.handle(ctx_for(path), Next::new(&[], &*handler))) + .expect("should run middleware") + } + #[test] - fn emits_git_version_header() { - let mut response = empty_response(); - apply_finalize_headers(&settings_with_response_headers(vec![]), &mut response); + fn finalize_middleware_strips_deployment_headers_before_it_writes_the_operators() { + let response = finalized( + "/articles/an-article", + settings_with_response_headers(vec![("X-Cache", "configured")]), + ); + + for name in ["x-served-by", "x-cache-hits"] { + assert!( + !response.headers().contains_key(name), + "should strip {name} from a publisher's page" + ); + } assert_eq!( response .headers() - .get("x-ts-version") + .get("x-cache") .and_then(|v| v.to_str().ok()), - trusted_server_core::constants::TS_GIT_VERSION, - "should report the compiled-in git version as x-ts-version" + Some("configured"), + "should carry the operator's header and not the origin's" ); } } diff --git a/crates/trusted-server-adapter-axum/tests/routes.rs b/crates/trusted-server-adapter-axum/tests/routes.rs index e33a5d43d6..085a4c0466 100644 --- a/crates/trusted-server-adapter-axum/tests/routes.rs +++ b/crates/trusted-server-adapter-axum/tests/routes.rs @@ -806,7 +806,7 @@ async fn nextjs_auction_output_holds_until_the_structural_body_close() { } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] -async fn health_reports_git_version() { +async fn health_carries_no_version() { let mut service = make_service(); let request = Request::builder() .method("GET") @@ -827,12 +827,8 @@ async fn health_reports_git_version() { 200, "should return 200 on /health" ); - assert_eq!( - response - .headers() - .get("x-ts-version") - .and_then(|v| v.to_str().ok()), - trusted_server_core::constants::TS_GIT_VERSION, - "should report the compiled-in git version on /health" + assert!( + response.headers().get("x-ts-version").is_none(), + "should write no version on /health, which the service_version middleware writes on a page" ); } diff --git a/crates/trusted-server-adapter-cloudflare/src/app.rs b/crates/trusted-server-adapter-cloudflare/src/app.rs index f0b6942e62..81fa864e87 100644 --- a/crates/trusted-server-adapter-cloudflare/src/app.rs +++ b/crates/trusted-server-adapter-cloudflare/src/app.rs @@ -27,9 +27,9 @@ use trusted_server_core::inspect::permissions::{PERMISSIONS_PATHS, handle_permis use trusted_server_core::integrations::{ IntegrationBuilder, IntegrationRegistry, ProxyDispatchInput, }; -use trusted_server_core::platform::RuntimeServices; #[cfg(target_arch = "wasm32")] use trusted_server_core::platform::StoreName; +use trusted_server_core::platform::{PlatformIdentity, RuntimeServices}; use trusted_server_core::proxy::{ handle_first_party_click, handle_first_party_proxy, handle_first_party_proxy_rebuild, handle_first_party_proxy_sign, @@ -231,7 +231,8 @@ fn build_state_with_registrations_and_services( let plan = Arc::new(compile_auction_plan_with(&settings, integrations)?); plan.validate_for_target(trusted_server_core::platform::AuctionTargetId::Cloudflare)?; let orchestrator = build_orchestrator_with_plan(Arc::clone(&plan))?; - let registry = IntegrationRegistry::with_plan_and_registrations(&settings, plan, integrations)?; + let registry = IntegrationRegistry::with_plan_and_registrations(&settings, plan, integrations)? + .with_platform(PlatformIdentity::named("cloudflare")); // Composition root: resolve the module selection once, before any request // is served, so a selection this adapter can never supply fails here rather diff --git a/crates/trusted-server-adapter-cloudflare/src/middleware.rs b/crates/trusted-server-adapter-cloudflare/src/middleware.rs index e8cc4e414e..93584b19a7 100644 --- a/crates/trusted-server-adapter-cloudflare/src/middleware.rs +++ b/crates/trusted-server-adapter-cloudflare/src/middleware.rs @@ -6,7 +6,9 @@ use edgezero_core::error::EdgeError; use edgezero_core::http::{HeaderValue, Response}; use edgezero_core::middleware::{Middleware, Next}; use trusted_server_core::constants::HEADER_X_GEO_INFO_AVAILABLE; -use trusted_server_core::http_util::sanitize_trusted_client_ip_headers; +use trusted_server_core::http_util::{ + ROUTE_NAMESPACE, sanitize_trusted_client_ip_headers, strip_deployment_headers, +}; use trusted_server_core::settings::Settings; // --------------------------------------------------------------------------- @@ -80,7 +82,10 @@ impl Middleware for FinalizeResponseMiddleware { .filter(|s| !s.is_empty() && *s != "XX") .is_some(); + let path = ctx.request().uri().path().to_owned(); let mut response = next.run(ctx).await?; + // Before finalizing, so a header finalizing writes is kept. + strip_deployment_headers(&path, ROUTE_NAMESPACE, &mut response); apply_finalize_headers(&self.settings, geo_available, &mut response); Ok(response) } @@ -106,8 +111,6 @@ pub(crate) fn apply_finalize_headers( HeaderValue::from_static(if geo_available { "true" } else { "false" }), ); - trusted_server_core::version_header::apply_git_version_header(response); - // Cloudflare is a real shared cache: cookie-bearing responses must stay // private and operator headers must not re-enable caching for uncacheable // per-user payloads. @@ -287,21 +290,58 @@ mod tests { ); } + fn ctx_for(path: &str) -> RequestContext { + let req = request_builder() + .method(Method::GET) + .uri(path) + .body(Body::empty()) + .expect("should build test request"); + RequestContext::new(req, PathParams::new(HashMap::new())) + } + + /// A response as an origin behind a cache sends it, naming the node that + /// answered and how the cache treated the request. + fn response_naming_its_node() -> Response { + response_builder() + .header("x-served-by", "cache-node-1") + .header("x-cache", "HIT") + .header("x-cache-hits", "3") + .body(Body::empty()) + .expect("should build test response") + } + + /// What the finalize middleware makes of that response for a request to + /// `path`. + fn finalized(path: &str, settings: Settings) -> Response { + let middleware = FinalizeResponseMiddleware::new(Arc::new(settings)); + let handler = Arc::new(|_ctx: RequestContext| async move { + Ok::(response_naming_its_node()) + }); + + block_on(middleware.handle(ctx_for(path), Next::new(&[], &*handler))) + .expect("should run middleware") + } + #[test] - fn emits_git_version_header() { - let mut response = empty_response(); - apply_finalize_headers( - &settings_with_response_headers(vec![]), - false, - &mut response, + fn finalize_middleware_strips_deployment_headers_before_it_writes_the_operators() { + let response = finalized( + "/articles/an-article", + settings_with_response_headers(vec![("X-Cache", "configured")]), ); + + for name in ["x-served-by", "x-cache-hits"] { + assert!( + !response.headers().contains_key(name), + "should strip {name} from a publisher's page" + ); + } assert_eq!( response .headers() - .get("x-ts-version") + .get("x-cache") .and_then(|v| v.to_str().ok()), - trusted_server_core::constants::TS_GIT_VERSION, - "should report the compiled-in git version as x-ts-version" + Some("configured"), + "should carry the operator's header and not the origin's" ); } } diff --git a/crates/trusted-server-adapter-fastly/src/app.rs b/crates/trusted-server-adapter-fastly/src/app.rs index 62711b20fb..98f8b48b50 100644 --- a/crates/trusted-server-adapter-fastly/src/app.rs +++ b/crates/trusted-server-adapter-fastly/src/app.rs @@ -123,8 +123,8 @@ use trusted_server_core::integrations::{ }; use trusted_server_core::permissions::PermissionState; use trusted_server_core::platform::{ - ClientInfo, GeoInfo, PlatformKvStore, PolicyOrigin, RuntimeServices, StoreName, - build_geo_module, + ClientInfo, GeoInfo, PlatformIdentity, PlatformKvStore, PolicyOrigin, RuntimeServices, + StoreName, UNKNOWN, build_geo_module, }; use trusted_server_core::proxy::{ AssetProxyCachePolicy, handle_asset_proxy_request, handle_first_party_click, @@ -331,7 +331,8 @@ pub(crate) fn build_state_with_registrations( let plan = Arc::new(compile_auction_plan_with(&settings, integrations)?); plan.validate_for_target(trusted_server_core::platform::AuctionTargetId::Fastly)?; let orchestrator = build_orchestrator_with_plan(Arc::clone(&plan))?; - let registry = IntegrationRegistry::with_plan_and_registrations(&settings, plan, integrations)?; + let registry = IntegrationRegistry::with_plan_and_registrations(&settings, plan, integrations)? + .with_platform(fastly_platform_identity()); // Composition root: resolve the module selection once, before any request // is served, so a selection this adapter can never supply fails here rather @@ -374,6 +375,24 @@ pub(crate) fn build_state_with_registrations( })) } +/// What Fastly reports about the service answering, from the environment +/// variables Fastly gives every Compute service. +fn fastly_platform_identity() -> PlatformIdentity { + let read = |name: &str| { + std::env::var(name) + .ok() + .filter(|value| !value.is_empty()) + .unwrap_or_else(|| UNKNOWN.to_owned()) + }; + PlatformIdentity { + name: "fastly".to_owned(), + service_id: read("FASTLY_SERVICE_ID"), + service_version: read("FASTLY_SERVICE_VERSION"), + staging: std::env::var("FASTLY_IS_STAGING").is_ok_and(|value| value == "1"), + pop: read("FASTLY_POP"), + } +} + /// The permission signal modules this adapter links, in the order they run /// when configuration names none. Global Privacy Control is first because it /// is a browser setting with no interface of its own, and the three that diff --git a/crates/trusted-server-adapter-fastly/src/lib.rs b/crates/trusted-server-adapter-fastly/src/lib.rs index 93d6bb6a45..b978535dfe 100644 --- a/crates/trusted-server-adapter-fastly/src/lib.rs +++ b/crates/trusted-server-adapter-fastly/src/lib.rs @@ -21,7 +21,6 @@ use fastly::http::Method as FastlyMethod; use fastly::{Request as FastlyRequest, Response as FastlyResponse}; use trusted_server_core::cache_policy::{EdgeCacheHeader, cache_control_headers_have_directive}; -use trusted_server_core::constants::HEADER_X_TS_VERSION; use trusted_server_core::ec::device::{DeviceModule, DeviceSignals, build_device_module}; use trusted_server_core::ec::finalize::ec_finalize_response; use trusted_server_core::ec::kv::KvIdentityGraph; @@ -40,7 +39,6 @@ use trusted_server_core::platform::{ use trusted_server_core::proxy::{AssetProxyCachePolicy, stream_asset_body}; use trusted_server_core::response_privacy::TerminalPrivateResponse; use trusted_server_core::settings::Settings; -use trusted_server_core::version_header::{apply_git_version_header, git_version_header_value}; use trusted_server_device_fastly::{FastlyDeviceModule, FastlyHostSignals}; mod app; @@ -197,12 +195,7 @@ fn refusal_response(refusal: &config_selection::Refusal) -> FastlyResponse { fn health_response(req: &FastlyRequest) -> Option { if req.get_method() == FastlyMethod::GET && req.get_path() == "/health" { - let mut response = FastlyResponse::from_status(200).with_body_text_plain("ok"); - // Compiled-in constant: keeps the probe free of settings and app construction. - if let Some(version) = git_version_header_value() { - response.set_header(HEADER_X_TS_VERSION, version); - } - return Some(response); + return Some(FastlyResponse::from_status(200).with_body_text_plain("ok")); } None @@ -761,9 +754,6 @@ fn edgezero_main(mut req: FastlyRequest, sandbox: &mut Sandbox, ordinal: u64, re } Err(e) => { log::warn!("EdgeZero EC finalize skipped: failed to reload settings: {e:?}"); - // The compiled-in version needs no settings, so a deploy that cannot - // load its config still reports which build is serving. - apply_git_version_header(&mut response); } } } @@ -1308,19 +1298,18 @@ mod tests { } #[test] - fn health_response_reports_git_version_only() { + fn health_response_carries_no_version() { let req = FastlyRequest::get("https://example.com/health"); let response = health_response(&req).expect("should build health response"); - assert_eq!( - response.get_header_str("x-ts-version"), - trusted_server_core::constants::TS_GIT_VERSION, - "should report the compiled-in git version on /health" + assert!( + response.get_header("x-ts-version").is_none(), + "should write no version on /health, which the service_version middleware writes on a page" ); assert!( - response.get_header("x-ts-fastly-version").is_none(), - "should keep x-ts-fastly-version off the /health fast path" + response.get_header("x-ts-service-version").is_none(), + "should write no service version on /health" ); } diff --git a/crates/trusted-server-adapter-fastly/src/middleware.rs b/crates/trusted-server-adapter-fastly/src/middleware.rs index b2d38884ec..ae57eda168 100644 --- a/crates/trusted-server-adapter-fastly/src/middleware.rs +++ b/crates/trusted-server-adapter-fastly/src/middleware.rs @@ -14,13 +14,12 @@ use edgezero_core::http::{HeaderValue, Response, StatusCode}; use edgezero_core::middleware::{Middleware, Next}; use edgezero_core::response::IntoResponse; use trusted_server_core::constants::{ - ENV_FASTLY_IS_STAGING, ENV_FASTLY_SERVICE_VERSION, HEADER_X_GEO_INFO_AVAILABLE, - HEADER_X_TS_ENV, HEADER_X_TS_FASTLY_VERSION, + ENV_FASTLY_IS_STAGING, HEADER_X_GEO_INFO_AVAILABLE, HEADER_X_TS_ENV, }; use trusted_server_core::geo::GeoInfo; +use trusted_server_core::http_util::{ROUTE_NAMESPACE, strip_deployment_headers}; use trusted_server_core::platform::{ClientInfo, PlatformGeo, RuntimeServices}; use trusted_server_core::settings::Settings; -use trusted_server_core::version_header::apply_git_version_header; pub(crate) const HEADER_X_TS_FINALIZED: &str = "x-ts-finalized"; @@ -42,11 +41,8 @@ pub(crate) const HEADER_X_TS_FINALIZED: &str = "x-ts-finalized"; /// /// Headers are written in this order (last write wins): /// 1. Geo headers (or `X-Geo-Info-Available: false` when geo is unavailable) -/// 2. `X-TS-Version` from the compiled-in git version -/// (`TRUSTED_SERVER__GIT_VERSION`), and `X-TS-Fastly-Version` from the -/// `FASTLY_SERVICE_VERSION` env var -/// 3. `X-TS-ENV: staging` when `FASTLY_IS_STAGING == "1"` -/// 4. Operator-configured `settings.response_headers` (can override any managed header) +/// 2. `X-TS-ENV: staging` when `FASTLY_IS_STAGING == "1"` +/// 3. Operator-configured `settings.response_headers` (can override any managed header) pub struct FinalizeResponseMiddleware { settings: Arc, geo: Arc, @@ -79,6 +75,7 @@ impl Middleware for FinalizeResponseMiddleware { || FastlyRequestContext::get(ctx.request()).and_then(|c| c.client_ip), |info| info.client_ip, ); + let path = ctx.request().uri().path().to_owned(); let mut response = match next.run(ctx).await { Ok(r) => r, @@ -104,6 +101,8 @@ impl Middleware for FinalizeResponseMiddleware { None }; + // Before finalizing, so a header finalizing writes is kept. + strip_deployment_headers(&path, ROUTE_NAMESPACE, &mut response); apply_finalize_headers(&self.settings, geo_info.as_ref(), &mut response); response .headers_mut() @@ -152,16 +151,13 @@ pub(crate) fn geo_allowed_for_response(response: &Response) -> bool { /// /// Header write order (last write wins): /// 1. Geo headers (`x-geo-*`) — or `X-Geo-Info-Available: false` when absent -/// 2. `X-TS-Version` from the compiled-in git version -/// (`TRUSTED_SERVER__GIT_VERSION`), and `X-TS-Fastly-Version` from the -/// `FASTLY_SERVICE_VERSION` env var -/// 3. `X-TS-ENV: staging` when `FASTLY_IS_STAGING == "1"` -/// 4. Set-Cookie cache privacy — strip surrogate cache headers and downgrade +/// 2. `X-TS-ENV: staging` when `FASTLY_IS_STAGING == "1"` +/// 3. Set-Cookie cache privacy — strip surrogate cache headers and downgrade /// `Cache-Control` to `private, max-age=0` on cookie-bearing responses -/// 5. `settings.response_headers` — operator-configured overrides, except the +/// 4. `settings.response_headers` — operator-configured overrides, except the /// cache-controlling headers are skipped on uncacheable (`private`/`no-store`) /// responses so operators cannot re-enable shared caching for per-user payloads -/// 6. `X-Robots-Tag: noindex, nofollow` when `[robots-txt] modules` selects +/// 5. `X-Robots-Tag: noindex, nofollow` when `[robots-txt] modules` selects /// `refuse_all` pub(crate) fn apply_finalize_headers( settings: &Settings, @@ -177,13 +173,6 @@ pub(crate) fn apply_finalize_headers( ); } - apply_git_version_header(response); - - apply_fastly_version_header( - std::env::var(ENV_FASTLY_SERVICE_VERSION).ok().as_deref(), - response, - ); - if std::env::var(ENV_FASTLY_IS_STAGING).as_deref() == Ok("1") { response .headers_mut() @@ -218,25 +207,6 @@ pub(crate) use trusted_server_core::response_privacy::{ // Tests // --------------------------------------------------------------------------- -/// Sets `X-TS-Fastly-Version` to `service_version`, or removes an inherited -/// value when the version is unset or invalid, so it is never reported as ours. -fn apply_fastly_version_header(service_version: Option<&str>, response: &mut Response) { - match service_version.map(HeaderValue::from_str) { - Some(Ok(value)) => { - response - .headers_mut() - .insert(HEADER_X_TS_FASTLY_VERSION, value); - } - Some(Err(_)) => { - log::warn!("Skipping invalid FASTLY_SERVICE_VERSION response header value"); - response.headers_mut().remove(HEADER_X_TS_FASTLY_VERSION); - } - None => { - response.headers_mut().remove(HEADER_X_TS_FASTLY_VERSION); - } - } -} - #[cfg(test)] mod tests { use super::*; @@ -858,45 +828,62 @@ mod tests { ); } - #[test] - fn version_headers_split_git_and_fastly_versions() { - let settings = settings_with_response_headers(vec![]); - let mut response = empty_response(); - response.headers_mut().insert( - HEADER_X_TS_FASTLY_VERSION, - HeaderValue::from_static("origin-value"), - ); + fn ctx_for(path: &str) -> RequestContext { + let req = request_builder() + .method(Method::GET) + .uri(path) + .body(Body::empty()) + .expect("should build test request"); + RequestContext::new(req, PathParams::new(HashMap::new())) + } - apply_finalize_headers(&settings, None, &mut response); + /// A response as an origin behind a cache sends it, naming the node that + /// answered and how the cache treated the request. + fn response_naming_its_node() -> Response { + response_builder() + .header("x-served-by", "cache-node-1") + .header("x-cache", "HIT") + .header("x-cache-hits", "3") + .body(Body::empty()) + .expect("should build test response") + } - let header = |name: &str| response.headers().get(name).and_then(|v| v.to_str().ok()); - assert_eq!( - header("x-ts-version"), - trusted_server_core::constants::TS_GIT_VERSION, - "should report the compiled-in git version as x-ts-version" - ); - assert_eq!( - header("x-ts-fastly-version"), - std::env::var(ENV_FASTLY_SERVICE_VERSION).ok().as_deref(), - "should report FASTLY_SERVICE_VERSION as x-ts-fastly-version" + /// What the finalize middleware makes of that response for a request to + /// `path`. + fn finalized(path: &str, settings: Settings) -> Response { + let middleware = FinalizeResponseMiddleware::new( + Arc::new(settings), + Arc::new(FixedGeo(None)), + test_finalize_services(), ); + let handler = Arc::new(|_ctx: RequestContext| async move { + Ok::(response_naming_its_node()) + }); + + block_on(middleware.handle(ctx_for(path), Next::new(&[], &*handler))) + .expect("should run middleware") } #[test] - fn fastly_version_header_removes_inherited_value_when_unknown() { - for service_version in [None, Some("bad\nvalue")] { - let mut response = empty_response(); - response.headers_mut().insert( - HEADER_X_TS_FASTLY_VERSION, - HeaderValue::from_static("origin-value"), - ); - - apply_fastly_version_header(service_version, &mut response); + fn finalize_middleware_strips_deployment_headers_before_it_writes_the_operators() { + let response = finalized( + "/articles/an-article", + settings_with_response_headers(vec![("X-Cache", "configured")]), + ); + for name in ["x-served-by", "x-cache-hits"] { assert!( - response.headers().get(HEADER_X_TS_FASTLY_VERSION).is_none(), - "should remove an inherited x-ts-fastly-version for {service_version:?}" + !response.headers().contains_key(name), + "should strip {name} from a publisher's page" ); } + assert_eq!( + response + .headers() + .get("x-cache") + .and_then(|v| v.to_str().ok()), + Some("configured"), + "should carry the operator's header and not the origin's" + ); } } diff --git a/crates/trusted-server-adapter-spin/src/app.rs b/crates/trusted-server-adapter-spin/src/app.rs index 83460f2a8a..d302976c85 100644 --- a/crates/trusted-server-adapter-spin/src/app.rs +++ b/crates/trusted-server-adapter-spin/src/app.rs @@ -32,9 +32,9 @@ use trusted_server_core::inspect::permissions::{ use trusted_server_core::integrations::{ IntegrationBuilder, IntegrationRegistry, ProxyDispatchInput, }; -use trusted_server_core::platform::RuntimeServices; #[cfg(all(feature = "spin", target_arch = "wasm32"))] use trusted_server_core::platform::{PlatformConfigStore, StoreName}; +use trusted_server_core::platform::{PlatformIdentity, RuntimeServices}; use trusted_server_core::proxy::{ handle_first_party_click, handle_first_party_proxy, handle_first_party_proxy_rebuild, handle_first_party_proxy_sign, @@ -219,7 +219,8 @@ fn build_state_with_registrations_and_services( let plan = Arc::new(compile_auction_plan_with(&settings, integrations)?); plan.validate_for_target(trusted_server_core::platform::AuctionTargetId::Spin)?; let orchestrator = build_orchestrator_with_plan(Arc::clone(&plan))?; - let registry = IntegrationRegistry::with_plan_and_registrations(&settings, plan, integrations)?; + let registry = IntegrationRegistry::with_plan_and_registrations(&settings, plan, integrations)? + .with_platform(PlatformIdentity::named("spin")); // Composition root: resolve the module selection once, before any request // is served, so a selection this adapter can never supply fails here rather @@ -1713,12 +1714,9 @@ mod tests { 200, "GET /health must return 200 from the startup fallback" ); - assert_eq!( - resp.headers() - .get("x-ts-version") - .and_then(|v| v.to_str().ok()), - trusted_server_core::constants::TS_GIT_VERSION, - "startup-fallback /health should report the compiled-in git version" + assert!( + resp.headers().get("x-ts-version").is_none(), + "startup-fallback /health should carry no version" ); let body = resp.into_body().into_bytes().unwrap_or_default(); assert_eq!( diff --git a/crates/trusted-server-adapter-spin/src/middleware.rs b/crates/trusted-server-adapter-spin/src/middleware.rs index 9418a4dc92..547e8eb035 100644 --- a/crates/trusted-server-adapter-spin/src/middleware.rs +++ b/crates/trusted-server-adapter-spin/src/middleware.rs @@ -6,7 +6,9 @@ use edgezero_core::error::EdgeError; use edgezero_core::http::{HeaderValue, Response}; use edgezero_core::middleware::{Middleware, Next}; use trusted_server_core::constants::HEADER_X_GEO_INFO_AVAILABLE; -use trusted_server_core::http_util::sanitize_trusted_client_ip_headers; +use trusted_server_core::http_util::{ + ROUTE_NAMESPACE, sanitize_trusted_client_ip_headers, strip_deployment_headers, +}; use trusted_server_core::settings::Settings; // --------------------------------------------------------------------------- @@ -73,7 +75,10 @@ impl Middleware for FinalizeResponseMiddleware { async fn handle(&self, ctx: RequestContext, next: Next<'_>) -> Result { let geo_available = false; + let path = ctx.request().uri().path().to_owned(); let mut response = next.run(ctx).await?; + // Before finalizing, so a header finalizing writes is kept. + strip_deployment_headers(&path, ROUTE_NAMESPACE, &mut response); apply_finalize_headers(&self.settings, geo_available, &mut response); Ok(response) } @@ -131,8 +136,6 @@ pub(crate) fn apply_finalize_headers( HeaderValue::from_static(if geo_available { "true" } else { "false" }), ); - trusted_server_core::version_header::apply_git_version_header(response); - // Cookie-bearing responses stay private to shared caches and operator // headers cannot re-enable caching for uncacheable per-user payloads. trusted_server_core::response_privacy::apply_response_headers_with_cache_privacy( @@ -311,21 +314,58 @@ mod tests { ); } + fn ctx_for(path: &str) -> RequestContext { + let req = request_builder() + .method(Method::GET) + .uri(path) + .body(Body::empty()) + .expect("should build test request"); + RequestContext::new(req, PathParams::new(HashMap::new())) + } + + /// A response as an origin behind a cache sends it, naming the node that + /// answered and how the cache treated the request. + fn response_naming_its_node() -> Response { + response_builder() + .header("x-served-by", "cache-node-1") + .header("x-cache", "HIT") + .header("x-cache-hits", "3") + .body(Body::empty()) + .expect("should build test response") + } + + /// What the finalize middleware makes of that response for a request to + /// `path`. + fn finalized(path: &str, settings: Settings) -> Response { + let middleware = FinalizeResponseMiddleware::new(Arc::new(settings)); + let handler = Arc::new(|_ctx: RequestContext| async move { + Ok::(response_naming_its_node()) + }); + + block_on(middleware.handle(ctx_for(path), Next::new(&[], &*handler))) + .expect("should run middleware") + } + #[test] - fn emits_git_version_header() { - let mut response = empty_response(); - apply_finalize_headers( - &settings_with_response_headers(vec![]), - false, - &mut response, + fn finalize_middleware_strips_deployment_headers_before_it_writes_the_operators() { + let response = finalized( + "/articles/an-article", + settings_with_response_headers(vec![("X-Cache", "configured")]), ); + + for name in ["x-served-by", "x-cache-hits"] { + assert!( + !response.headers().contains_key(name), + "should strip {name} from a publisher's page" + ); + } assert_eq!( response .headers() - .get("x-ts-version") + .get("x-cache") .and_then(|v| v.to_str().ok()), - trusted_server_core::constants::TS_GIT_VERSION, - "should report the compiled-in git version as x-ts-version" + Some("configured"), + "should carry the operator's header and not the origin's" ); } } diff --git a/crates/trusted-server-core/src/constants.rs b/crates/trusted-server-core/src/constants.rs index 91d7f883a9..7d74790afa 100644 --- a/crates/trusted-server-core/src/constants.rs +++ b/crates/trusted-server-core/src/constants.rs @@ -44,11 +44,13 @@ pub const HEADER_X_REQUEST_ID: HeaderName = HeaderName::from_static("x-request-i pub const HEADER_X_COMPRESS_HINT: HeaderName = HeaderName::from_static("x-compress-hint"); pub const HEADER_X_DEBUG_FASTLY_POP: HeaderName = HeaderName::from_static("x-debug-fastly-pop"); -// Staging / version identification headers -/// Deployed git version: tag, else branch, else 6-char commit (see [`TS_GIT_VERSION`]). +// Staging / version identification headers. The two version headers are +// written by core's `service_version` middleware where a `[[serve]]` entry +// names it, and by no adapter. +/// The deployed git version the build compiled in (see [`TS_GIT_VERSION`]). pub const HEADER_X_TS_VERSION: HeaderName = HeaderName::from_static("x-ts-version"); -/// Fastly service version (`FASTLY_SERVICE_VERSION`), formerly sent as `x-ts-version`. -pub const HEADER_X_TS_FASTLY_VERSION: HeaderName = HeaderName::from_static("x-ts-fastly-version"); +/// The version the adapter reports for the service. +pub const HEADER_X_TS_SERVICE_VERSION: HeaderName = HeaderName::from_static("x-ts-service-version"); pub const HEADER_X_TS_ENV: HeaderName = HeaderName::from_static("x-ts-env"); /// Deployed git version compiled in by `build.rs`, from the deploy pipeline's @@ -59,7 +61,6 @@ pub const TS_GIT_VERSION: Option<&str> = match option_env!("TRUSTED_SERVER__GIT_ }; // Fastly environment variables -pub const ENV_FASTLY_SERVICE_VERSION: &str = "FASTLY_SERVICE_VERSION"; pub const ENV_FASTLY_IS_STAGING: &str = "FASTLY_IS_STAGING"; // Common standard header names used across modules diff --git a/crates/trusted-server-core/src/html_processor.rs b/crates/trusted-server-core/src/html_processor.rs index 19a815dc15..365f803eb6 100644 --- a/crates/trusted-server-core/src/html_processor.rs +++ b/crates/trusted-server-core/src/html_processor.rs @@ -19,8 +19,9 @@ use crate::integrations::{ }; use crate::middleware::{ ElementHandler, MatchedAttribute, MiddlewareChain, MiddlewareContext, MiddlewarePlan, - TextHandler, + ResponseHeader, TextHandler, }; +use crate::platform::PlatformIdentity; use crate::publisher::build_empty_bids_script; use crate::settings::Settings; use crate::streaming_processor::{HtmlRewriterAdapter, StreamProcessor}; @@ -255,6 +256,8 @@ pub fn create_html_processor(config: HtmlProcessorConfig) -> impl StreamProcesso /// /// `serve` is `None` for a document that may be stored, whose serve chain /// runs on each reader's copy instead, through [`create_serve_processor`]. +/// The headers returned are what the serve chain writes on the response, +/// none when `serve` is `None`. /// /// # Errors /// @@ -267,13 +270,14 @@ pub fn create_html_processor_with_middleware( config: HtmlProcessorConfig, fetch: &MiddlewareChain, serve: Option<&MiddlewareChain>, -) -> Result, String> { +) -> Result<(impl StreamProcessor + use<>, Vec), String> { let document_state = IntegrationDocumentState::default(); let plan = fetch.plan(&MiddlewareContext { phase: fetch.phase(), request_host: &config.request_host, request_scheme: &config.request_scheme, origin_host: &config.origin_host, + platform: config.integrations.platform(), document_state: &document_state, max_buffered_script_bytes: config.max_buffered_body_bytes, })?; @@ -284,15 +288,16 @@ pub fn create_html_processor_with_middleware( request_host: &config.request_host, request_scheme: &config.request_scheme, origin_host: &config.origin_host, + platform: config.integrations.platform(), request_state: &config.request_state, max_buffered_script_bytes: config.max_buffered_body_bytes, }, )?, - None => None, + None => ServePass::default(), }; let mut processor = build_html_processor(config, plan); - processor.processors.extend(reader); - Ok(processor) + processor.processors.extend(reader.processor); + Ok((processor, reader.response_headers)) } /// What a serve chain is told about the reader's copy it works on. @@ -304,15 +309,16 @@ pub struct ReaderDocument<'a> { pub request_scheme: &'a str, /// Host the document was fetched from. pub origin_host: &'a str, + /// What the adapter reports about the platform the deployment runs on. + pub platform: &'a PlatformIdentity, /// What modules left on this reader's request for their page changes. pub request_state: &'a IntegrationRequestState, /// The most a middleware may hold of one script while it decides. pub max_buffered_script_bytes: usize, } -/// A processor that carries out a serve chain alone, on one reader's copy of -/// a document core has already rewritten, or `None` when the chain has -/// nothing to do. +/// What a serve chain does to one reader's copy of a document core has +/// already rewritten, and to the reader's response. /// /// Nothing of core's own is written or rewritten a second time. Head markup /// goes straight before the script bundle's tag and markup for after the @@ -325,24 +331,41 @@ pub struct ReaderDocument<'a> { pub fn create_serve_processor( chain: &MiddlewareChain, document: &ReaderDocument<'_>, -) -> Result>, String> { +) -> Result { if chain.is_empty() { - return Ok(None); + return Ok(ServePass::default()); } let document_state = IntegrationDocumentState::default(); document.request_state.seed(&document_state); - let plan = chain.plan(&MiddlewareContext { + let mut plan = chain.plan(&MiddlewareContext { phase: chain.phase(), request_host: document.request_host, request_scheme: document.request_scheme, origin_host: document.origin_host, + platform: document.platform, document_state: &document_state, max_buffered_script_bytes: document.max_buffered_script_bytes, })?; - if plan.is_empty() { - return Ok(None); - } - Ok(Some(Box::new(build_serve_processor(plan)))) + let response_headers = std::mem::take(&mut plan.response_headers); + let processor: Option> = if plan.is_empty() { + None + } else { + Some(Box::new(build_serve_processor(plan))) + }; + Ok(ServePass { + processor, + response_headers, + }) +} + +/// What a serve chain does to one reader's response. +#[derive(Default)] +pub struct ServePass { + /// The processor over the reader's copy, or `None` when the chain + /// leaves the document as it arrived. + pub processor: Option>, + /// The headers the chain writes on the response. + pub response_headers: Vec, } /// The tag of the script bundle, which core writes once at the start of @@ -438,6 +461,8 @@ fn build_serve_processor(plan: MiddlewarePlan) -> HtmlWithStreamingProcessors { element_handlers, text_handlers, processors, + // The caller writes the plan's headers on the response. + response_headers: _, } = plan; let mut element_content_handlers: Vec = Vec::new(); if !head_inserts.is_empty() || !after_bundle_inserts.is_empty() { @@ -516,6 +541,9 @@ fn build_html_processor( element_handlers: middleware_element_handlers, text_handlers: middleware_text_handlers, processors: middleware_processors, + // A fetch plan writes no headers, which `MiddlewareChain::plan` + // refuses. + response_headers: _, } = plan; // The spellings of the origin's address in a URL attribute, each with its // replacement, built once per document. They are replaced in one pass and a @@ -920,6 +948,7 @@ pub mod test_support { ); create_html_processor_with_middleware(config, &fetch, Some(&serve)) .expect("should plan the page's middleware") + .0 } /// The page a reader of `html` receives from a deployment running diff --git a/crates/trusted-server-core/src/http_util.rs b/crates/trusted-server-core/src/http_util.rs index 163091aaa7..9d121252e2 100644 --- a/crates/trusted-server-core/src/http_util.rs +++ b/crates/trusted-server-core/src/http_util.rs @@ -521,6 +521,41 @@ pub fn enforce_max_body_size( Ok(()) } +/// The namespace the inspection endpoints answer under. +pub const ROUTE_NAMESPACE: &str = "/_ts"; + +/// Response headers that describe the deployment and not the page. +const DEPLOYMENT_HEADERS: &[&str] = &["x-cache", "x-cache-hits", "x-served-by"]; + +/// Removes the headers that describe the deployment from a response, unless +/// the request was for a path in `namespace`, which is where a deployment is +/// asked about itself. +/// +/// An adapter calls this before it finalizes a response, so a header that +/// finalizing writes is not removed, an operator's `[response_headers]` +/// among them. The publisher fallback has already removed them from what the +/// origin sent, with [`remove_deployment_headers`], so a response under +/// `namespace` keeps them only when this server answered it. +pub fn strip_deployment_headers(path: &str, namespace: &str, response: &mut Response) { + // A path that only starts with the namespace's text is not in it. `/_tsx` + // is a publisher's page. + let in_namespace = path == namespace + || path + .strip_prefix(namespace) + .is_some_and(|rest| rest.starts_with('/')); + if !in_namespace { + remove_deployment_headers(response); + } +} + +/// Removes the headers that describe the deployment from a response, whatever +/// path it answers. +pub fn remove_deployment_headers(response: &mut Response) { + for name in DEPLOYMENT_HEADERS { + response.headers_mut().remove(*name); + } +} + #[cfg(test)] mod tests { use super::*; @@ -1075,4 +1110,87 @@ mod tests { "should match text/html case-insensitively in fallback" ); } + + const STRIPPED: [&str; 3] = ["x-cache", "x-cache-hits", "x-served-by"]; + + fn response_with(names: &[&'static str]) -> Response<()> { + let mut response = Response::new(()); + for name in names { + response + .headers_mut() + .insert(*name, HeaderValue::from_static("value")); + } + response + } + + #[test] + fn a_publisher_page_loses_every_deployment_header() { + let mut response = response_with(&STRIPPED); + + strip_deployment_headers("/articles/an-article", ROUTE_NAMESPACE, &mut response); + + for name in STRIPPED { + assert!( + response.headers().get(name).is_none(), + "{name} should not reach a reader" + ); + } + } + + #[test] + fn the_servers_own_namespace_keeps_them() { + for path in ["/_ts", "/_ts/permissions", "/_ts/debug/ja4"] { + let mut response = response_with(&STRIPPED); + + strip_deployment_headers(path, ROUTE_NAMESPACE, &mut response); + + for name in STRIPPED { + assert!( + response.headers().get(name).is_some(), + "{path} should keep {name}" + ); + } + } + } + + #[test] + fn a_header_outside_the_list_is_left_alone() { + let kept = ["x-ts-version", "x-geo-info-available", "cache-control"]; + let mut response = response_with(&kept); + + strip_deployment_headers("/articles/an-article", ROUTE_NAMESPACE, &mut response); + + for name in kept { + assert!( + response.headers().get(name).is_some(), + "{name} should be left alone" + ); + } + } + + #[test] + fn a_path_that_only_starts_with_the_namespace_is_not_in_it() { + for path in ["/_tsx", "/_ts-archive", "/_tsfoo/bar"] { + let mut response = response_with(&["x-served-by"]); + + strip_deployment_headers(path, ROUTE_NAMESPACE, &mut response); + + assert!( + response.headers().get("x-served-by").is_none(), + "{path} is a publisher's page" + ); + } + } + + #[test] + fn the_boundary_is_the_namespace_passed_in() { + let mut kept = response_with(&["x-served-by"]); + let mut stripped = response_with(&["x-served-by"]); + + strip_deployment_headers("/_other/permissions", "/_other", &mut kept); + strip_deployment_headers("/_ts/permissions", "/_other", &mut stripped); + + assert!(kept.headers().get("x-served-by").is_some()); + assert!(stripped.headers().get("x-served-by").is_none()); + } } diff --git a/crates/trusted-server-core/src/integrations/js_asset_proxy.rs b/crates/trusted-server-core/src/integrations/js_asset_proxy.rs index 5d325da105..5c8a52fcd5 100644 --- a/crates/trusted-server-core/src/integrations/js_asset_proxy.rs +++ b/crates/trusted-server-core/src/integrations/js_asset_proxy.rs @@ -687,7 +687,7 @@ mod tests { request_state: crate::integrations::IntegrationRequestState::default(), permissions_script: None, }; - let processor = create_html_processor_with_middleware(config, &chain, None) + let (processor, _) = create_html_processor_with_middleware(config, &chain, None) .expect("should plan the asset proxy's middleware"); let pipeline_config = PipelineConfig { input_compression: Compression::None, diff --git a/crates/trusted-server-core/src/integrations/registry.rs b/crates/trusted-server-core/src/integrations/registry.rs index 48d4e3a013..7002197284 100644 --- a/crates/trusted-server-core/src/integrations/registry.rs +++ b/crates/trusted-server-core/src/integrations/registry.rs @@ -20,7 +20,7 @@ use crate::geo::GeoInfo; use crate::http_util::is_navigation_request; use crate::middleware::{Middleware, MiddlewareChain, MiddlewarePhase, PhaseEntries}; use crate::module_context::{ModuleCall, ModuleContext, ResolvedRequest}; -use crate::platform::{DisabledGeo, PlatformGeo, RuntimeServices}; +use crate::platform::{DisabledGeo, PlatformGeo, PlatformIdentity, RuntimeServices}; use crate::robots_txt::{ALLOW_ALL, REFUSE_ALL, RobotsTxtContributor}; use crate::settings::Settings; @@ -1411,7 +1411,9 @@ fn check_middleware( } /// The middleware no entry names in a phase it runs in, each with the -/// integration that supplies it, in registration order. +/// integration that supplies it, in registration order. Core's own +/// middleware is offered to every deployment, so one that no entry names +/// is not a module running for nothing, and is left out. fn unnamed_middleware<'a>( settings: &Settings, inner: &'a IntegrationRegistryInner, @@ -1419,6 +1421,7 @@ fn unnamed_middleware<'a>( inner .middleware .iter() + .filter(|(integration, _)| *integration != crate::service_version::MODULE) .filter(|(_, middleware)| { let id = middleware.middleware_id(); !middleware @@ -1435,6 +1438,9 @@ fn unnamed_middleware<'a>( pub struct IntegrationRegistry { inner: Arc, plan: Option>, + /// What the adapter reports about the platform the deployment runs on, + /// which a middleware is told about every document. + platform: Arc, } /// Refuses a name a section selects that no module in this deployment @@ -1489,6 +1495,20 @@ fn check_section_selections( } impl IntegrationRegistry { + /// Tells the registry what the adapter reports about the platform, which + /// a middleware is told about every document. + #[must_use] + pub fn with_platform(mut self, platform: PlatformIdentity) -> Self { + self.platform = Arc::new(platform); + self + } + + /// What the adapter reports about the platform the deployment runs on. + #[must_use] + pub fn platform(&self) -> &PlatformIdentity { + &self.platform + } + /// Build a registry and auction plan from the provided settings for tests. /// /// Runtime adapters should compile one plan and pass it to @@ -1811,12 +1831,19 @@ impl IntegrationRegistry { inner.ec_module = resolve_ec_module(settings, &inner); inner.device_module = resolve_device_module(settings, &inner)?; inner.geo_module = geo_module; + // Core's own middleware, which an entry may name whatever the + // sections select. + inner.middleware.push(( + crate::service_version::MODULE, + Arc::new(crate::service_version::ServiceVersion::default()), + )); check_middleware(settings, &inner)?; check_robots_txt_selection(settings, &inner)?; Ok(Self { inner: Arc::new(inner), plan: Some(plan), + platform: Arc::default(), }) } @@ -2341,6 +2368,7 @@ impl IntegrationRegistry { Self { inner: Arc::new(IntegrationRegistryInner::default()), plan: None, + platform: Arc::default(), } } @@ -2379,6 +2407,7 @@ impl IntegrationRegistry { device_module: None, }), plan: None, + platform: Arc::default(), } } @@ -2457,6 +2486,7 @@ impl IntegrationRegistry { device_module: None, }), plan: None, + platform: Arc::default(), } } } @@ -6093,8 +6123,9 @@ mod tests { fixture::BROKEN, fixture::READER, fixture::BROKEN_READER, + crate::service_version::MODULE, ], - "should list the middleware the selected module supplies" + "should list the middleware the selected module supplies, and core's own after them" ); assert_eq!( registry.middleware_in(MiddlewarePhase::Fetch), @@ -6103,8 +6134,12 @@ mod tests { ); assert_eq!( registry.middleware_in(MiddlewarePhase::Serve), - [fixture::READER, fixture::BROKEN_READER], - "should list the middleware that run in the serve phase" + [ + fixture::READER, + fixture::BROKEN_READER, + crate::service_version::MODULE + ], + "should list the middleware that run in the serve phase, core's own last" ); let chain_for = |media_type: &str, path: &str| { registry @@ -6151,12 +6186,13 @@ mod tests { message.contains( "[[fetch]] entry 2 names `testing.nothing`, which no module that runs supplies" ) && message.contains(&format!( - "[{}, {}, {}, {}, {}]", + "[{}, {}, {}, {}, {}, {}]", fixture::HEAD, fixture::LINKS, fixture::BROKEN, fixture::READER, - fixture::BROKEN_READER + fixture::BROKEN_READER, + crate::service_version::MODULE )), "should name the entry and list what could be named: {message}" ); diff --git a/crates/trusted-server-core/src/lib.rs b/crates/trusted-server-core/src/lib.rs index 652c69b89b..14eaf141e7 100644 --- a/crates/trusted-server-core/src/lib.rs +++ b/crates/trusted-server-core/src/lib.rs @@ -72,6 +72,7 @@ pub mod robots_txt; pub mod rsc_flight; pub(crate) mod s3_sigv4; pub mod secret_resolution; +pub mod service_version; pub mod settings; pub mod settings_data; pub mod streaming_processor; @@ -81,7 +82,6 @@ pub mod test_support; pub mod tester_cookie; pub mod tsjs; pub mod tsjs_bundle; -pub mod version_header; #[cfg(test)] mod migration_guards; diff --git a/crates/trusted-server-core/src/middleware.rs b/crates/trusted-server-core/src/middleware.rs index ed67c1a533..ccae6c1ae9 100644 --- a/crates/trusted-server-core/src/middleware.rs +++ b/crates/trusted-server-core/src/middleware.rs @@ -34,9 +34,11 @@ use std::fmt; use std::rc::Rc; use std::sync::Arc; +use http::{HeaderName, HeaderValue}; use serde::{Deserialize, Deserializer, Serialize, Serializer, de}; use crate::integrations::{AttributeRewriteAction, IntegrationDocumentState, ScriptRewriteAction}; +use crate::platform::PlatformIdentity; use crate::streaming_processor::StreamProcessor; /// The media type a document is matched as, and the only one a middleware @@ -89,6 +91,8 @@ pub struct MiddlewareContext<'a> { /// Host the document was fetched from, the same for a page served from /// the store as for one fetched for this request. pub origin_host: &'a str, + /// What the adapter reports about the platform the deployment runs on. + pub platform: &'a PlatformIdentity, /// State shared by the middleware working on this document in this /// phase. pub document_state: &'a IntegrationDocumentState, @@ -204,9 +208,19 @@ pub trait TextHandler { fn decide(&mut self, text: &str, is_last: bool) -> ScriptRewriteAction; } -/// What a middleware does to one document. A middleware may do several of -/// these at once, and an action with nothing set leaves the document as it -/// arrived. +/// A header a middleware writes on the reader's response. +pub type ResponseHeader = (HeaderName, HeaderValue); + +/// Writes `headers` on `response`, each replacing a header of the same name. +pub fn write_response_headers(response: &mut http::Response, headers: Vec) { + for (name, value) in headers { + response.headers_mut().insert(name, value); + } +} + +/// What a middleware does to one document and its response. A middleware +/// may do several of these at once, and an action with nothing set leaves +/// the document as it arrived. #[derive(Default)] pub struct MiddlewareAction { /// Markup to write in `` ahead of the script bundle, in the order @@ -222,6 +236,11 @@ pub struct MiddlewareAction { pub text_handlers: Vec>, /// A processor over the document the handlers left. pub stream: Option>, + /// Headers to write on the reader's response, each replacing one of the + /// same name. Only a serve middleware may write one, because what a + /// fetch middleware leaves is stored for every reader and a header goes + /// on one reader's response. + pub response_headers: Vec, } impl MiddlewareAction { @@ -239,6 +258,7 @@ impl MiddlewareAction { && self.element_handlers.is_empty() && self.text_handlers.is_empty() && self.stream.is_none() + && self.response_headers.is_empty() } /// Checks every handler's selector parses, before a handler is built @@ -260,6 +280,11 @@ impl MiddlewareAction { } } +/// The names of `headers`, for a debug view. +fn header_names(headers: &[ResponseHeader]) -> Vec<&str> { + headers.iter().map(|(name, _)| name.as_str()).collect() +} + impl fmt::Debug for MiddlewareAction { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { formatter @@ -272,6 +297,7 @@ impl fmt::Debug for MiddlewareAction { ) .field("text_handlers", &selectors_of_text(&self.text_handlers)) .field("stream", &self.stream.is_some()) + .field("response_headers", &header_names(&self.response_headers)) .finish() } } @@ -558,12 +584,21 @@ impl MiddlewareChain { middleware.middleware_id() ) })?; + if !action.response_headers.is_empty() && self.phase != MiddlewarePhase::Serve { + return Err(format!( + "the {} middleware `{}` cannot run, because it writes a response header, \ + which only a serve middleware does", + self.phase, + middleware.middleware_id() + )); + } plan.head_inserts.extend(action.head_inserts); plan.after_bundle_inserts .extend(action.after_bundle_inserts); plan.element_handlers.extend(action.element_handlers); plan.text_handlers.extend(action.text_handlers); plan.processors.extend(action.stream); + plan.response_headers.extend(action.response_headers); } Ok(plan) } @@ -585,6 +620,9 @@ pub struct MiddlewarePlan { pub text_handlers: Vec>, /// Stream processors, each handed what the one before it produced. pub processors: Vec>, + /// Headers for the reader's response, in the order the middleware were + /// named, a later one replacing an earlier of the same name. + pub response_headers: Vec, } impl MiddlewarePlan { @@ -596,6 +634,7 @@ impl MiddlewarePlan { && self.element_handlers.is_empty() && self.text_handlers.is_empty() && self.processors.is_empty() + && self.response_headers.is_empty() } } @@ -611,6 +650,7 @@ impl fmt::Debug for MiddlewarePlan { ) .field("text_handlers", &selectors_of_text(&self.text_handlers)) .field("processors", &self.processors.len()) + .field("response_headers", &header_names(&self.response_headers)) .finish() } } @@ -618,23 +658,45 @@ impl fmt::Debug for MiddlewarePlan { /// What a module's own tests hand a middleware they call directly. #[cfg(any(test, feature = "test-utils"))] pub mod test_support { + use std::sync::OnceLock; + use super::{MiddlewareContext, MiddlewarePhase}; use crate::html_processor::test_support::{ORIGIN_HOST, REQUEST_HOST}; use crate::integrations::IntegrationDocumentState; + use crate::platform::PlatformIdentity; + + /// The platform a test page is served from, which reports its name and + /// nothing else. + #[must_use] + pub fn platform() -> &'static PlatformIdentity { + static PLATFORM: OnceLock = OnceLock::new(); + PLATFORM.get_or_init(|| PlatformIdentity::named("test-platform")) + } /// What a middleware is told in `phase` about a test page, fetched from - /// [`ORIGIN_HOST`] for a reader of [`REQUEST_HOST`] over HTTPS, with - /// `document_state` as the document's. + /// [`ORIGIN_HOST`] for a reader of [`REQUEST_HOST`] over HTTPS on + /// [`platform`], with `document_state` as the document's. #[must_use] pub fn context( phase: MiddlewarePhase, document_state: &IntegrationDocumentState, ) -> MiddlewareContext<'_> { + context_on(phase, document_state, platform()) + } + + /// As [`context`], on `platform`. + #[must_use] + pub fn context_on<'a>( + phase: MiddlewarePhase, + document_state: &'a IntegrationDocumentState, + platform: &'a PlatformIdentity, + ) -> MiddlewareContext<'a> { MiddlewareContext { phase, request_host: REQUEST_HOST, request_scheme: "https", origin_host: ORIGIN_HOST, + platform, document_state, max_buffered_script_bytes: 16 * 1024 * 1024, } diff --git a/crates/trusted-server-core/src/middleware/tests.rs b/crates/trusted-server-core/src/middleware/tests.rs index c3a10dc485..4b47ffa08b 100644 --- a/crates/trusted-server-core/src/middleware/tests.rs +++ b/crates/trusted-server-core/src/middleware/tests.rs @@ -67,6 +67,7 @@ fn reader(request_state: &IntegrationRequestState) -> ReaderDocument<'_> { request_host: REQUEST_HOST, request_scheme: "https", origin_host: ORIGIN_HOST, + platform: test_support::platform(), request_state, max_buffered_script_bytes: 16 * 1024 * 1024, } @@ -83,6 +84,7 @@ fn served_with( ) -> String { let Some(mut processor) = create_serve_processor(chain, &reader(request_state)) .expect("should plan the reader's middleware") + .processor else { return html.to_owned(); }; @@ -106,6 +108,17 @@ fn served(chain: &MiddlewareChain, html: &str) -> String { served_with(chain, &IntegrationRequestState::default(), html, 8192) } +/// A middleware that writes one header on the reader's response. +fn header(id: &'static str, name: &'static str, value: &'static str) -> Arc { + middleware(id, move |_| MiddlewareAction { + response_headers: vec![( + HeaderName::from_static(name), + HeaderValue::from_static(value), + )], + ..MiddlewareAction::pass() + }) +} + /// A middleware that writes `markup` at the start of ``. fn head(id: &'static str, markup: &'static str) -> Arc { middleware(id, move |_| MiddlewareAction { @@ -185,7 +198,7 @@ fn page_with( html: &str, chunk_size: usize, ) -> String { - let processor = create_html_processor_with_middleware(config, chain, None) + let (processor, _) = create_html_processor_with_middleware(config, chain, None) .expect("should plan the document's middleware"); let mut pipeline = StreamingPipeline::new( PipelineConfig { @@ -1193,6 +1206,106 @@ fn a_selector_that_does_not_parse_refuses_the_document_naming_the_middleware() { } } +#[test] +fn a_fetch_middleware_that_writes_a_response_header_is_refused() { + let chain = chain_of(vec![header("example.version", "x-example", "1")]); + + let message = create_html_processor_with_middleware(config(), &chain, None) + .err() + .expect("should refuse to plan the document"); + + assert!( + message.contains("the fetch middleware `example.version` cannot run") + && message.contains("writes a response header"), + "should name the middleware and the reason: {message}" + ); +} + +#[test] +fn a_serve_middleware_s_headers_are_written_on_the_reader_s_response() { + let chain = serve_chain_of(vec![ + header("example.a", "x-example", "a"), + header("example.b", "x-example", "b"), + header("example.c", "x-other", "c"), + ]); + + let pass = create_serve_processor(&chain, &reader(&IntegrationRequestState::default())) + .expect("should plan the reader's middleware"); + let mut response = http::Response::new(()); + write_response_headers(&mut response, pass.response_headers); + + assert!( + pass.processor.is_none(), + "should leave the document as it arrived" + ); + assert_eq!( + response + .headers() + .get("x-example") + .map(HeaderValue::as_bytes), + Some(b"b".as_slice()), + "a later middleware's header should replace an earlier one's" + ); + assert_eq!( + response.headers().get("x-other").map(HeaderValue::as_bytes), + Some(b"c".as_slice()), + "should write a header no other middleware wrote" + ); +} + +#[test] +fn the_inline_path_hands_back_the_serve_chain_s_headers() { + let serve = serve_chain_of(vec![header("example.version", "x-example", "1")]); + + let (_, headers) = + create_html_processor_with_middleware(config(), &chain_of(Vec::new()), Some(&serve)) + .expect("should plan the document"); + + assert_eq!( + headers + .iter() + .map(|(name, value)| (name.as_str(), value.as_bytes())) + .collect::>(), + [("x-example", b"1".as_slice())], + "should hand back the serve chain's headers for the response" + ); +} + +#[test] +fn an_action_with_only_a_header_is_not_a_pass() { + let action = MiddlewareAction { + response_headers: vec![( + HeaderName::from_static("x-example"), + HeaderValue::from_static("1"), + )], + ..MiddlewareAction::pass() + }; + + assert!(!action.is_pass(), "a header is a change to the response"); + assert!(MiddlewareAction::pass().is_pass()); +} + +#[test] +fn a_middleware_is_told_the_platform_the_deployment_runs_on() { + let platform = PlatformIdentity { + service_version: "39".to_owned(), + ..PlatformIdentity::named("example-platform") + }; + let document_state = IntegrationDocumentState::default(); + + let context = test_support::context_on(MiddlewarePhase::Serve, &document_state, &platform); + + assert_eq!(context.platform.name, "example-platform"); + assert_eq!(context.platform.service_version, "39"); + assert_eq!( + test_support::context(MiddlewarePhase::Serve, &document_state) + .platform + .name, + "test-platform", + "a test context should name the test platform" + ); +} + #[test] fn a_chain_reports_its_phase_and_the_names_in_the_order_they_run() { let chain = chain_of(vec![ @@ -1376,7 +1489,10 @@ fn a_serve_chain_with_nothing_to_do_makes_no_processor() { let none = create_serve_processor(&serve_chain_of(Vec::new()), &reader(&state)) .expect("should plan an empty chain"); - assert!(none.is_none(), "should make no processor for no middleware"); + assert!( + none.processor.is_none(), + "should make no processor for no middleware" + ); let passes = serve_chain_of(vec![middleware("example.pass", |_| { MiddlewareAction::pass() @@ -1384,7 +1500,7 @@ fn a_serve_chain_with_nothing_to_do_makes_no_processor() { let none = create_serve_processor(&passes, &reader(&state)).expect("should plan a chain that passes"); assert!( - none.is_none(), + none.processor.is_none(), "should make no processor where every middleware leaves the page alone" ); } @@ -1426,7 +1542,7 @@ fn on_a_page_that_is_not_stored_the_serve_chain_follows_the_fetch_chain_on_the_o // Read whole, and then seven bytes at a time, so the second rewriter is // fed the first one's output in pieces. for chunk_size in [8192, 7] { - let processor = create_html_processor_with_middleware(config(), &fetch, Some(&serve)) + let (processor, _) = create_html_processor_with_middleware(config(), &fetch, Some(&serve)) .expect("should plan both chains"); let mut pipeline = StreamingPipeline::new( PipelineConfig { diff --git a/crates/trusted-server-core/src/platform/identity.rs b/crates/trusted-server-core/src/platform/identity.rs new file mode 100644 index 0000000000..815725bdf4 --- /dev/null +++ b/crates/trusted-server-core/src/platform/identity.rs @@ -0,0 +1,43 @@ +//! What an adapter reports about the platform a deployment runs on. + +use serde::Serialize; + +/// What the adapter reports about the platform the deployment runs on. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct PlatformIdentity { + /// The platform's name. + pub name: String, + /// The platform's identifier for the service running this build. + pub service_id: String, + /// The platform's version number of that service. + pub service_version: String, + /// Whether the platform's staging environment answered. + pub staging: bool, + /// The location that answered. + pub pop: String, +} + +/// What a fact the platform did not report reads as. +pub const UNKNOWN: &str = "unknown"; + +impl PlatformIdentity { + /// A platform that reports its name and nothing else. + #[must_use] + pub fn named(name: &str) -> Self { + Self { + name: name.to_owned(), + service_id: UNKNOWN.to_owned(), + service_version: UNKNOWN.to_owned(), + staging: false, + pop: UNKNOWN.to_owned(), + } + } +} + +impl Default for PlatformIdentity { + /// A platform nothing has been reported about. + fn default() -> Self { + Self::named(UNKNOWN) + } +} diff --git a/crates/trusted-server-core/src/platform/mod.rs b/crates/trusted-server-core/src/platform/mod.rs index a9e23a4536..672af1419e 100644 --- a/crates/trusted-server-core/src/platform/mod.rs +++ b/crates/trusted-server-core/src/platform/mod.rs @@ -39,6 +39,7 @@ mod backend_naming; mod error; mod geo; mod http; +mod identity; mod image_optimizer; mod kv; mod policy_source; @@ -67,6 +68,7 @@ pub use http::{ PlatformCacheIntent, PlatformHttpClient, PlatformHttpRequest, PlatformPendingRequest, PlatformResponse, PlatformSelectResult, UnavailableHttpClient, }; +pub use identity::{PlatformIdentity, UNKNOWN}; pub use image_optimizer::{ PlatformImageOptimizerCrop, PlatformImageOptimizerCropMode, PlatformImageOptimizerOptions, PlatformImageOptimizerParams, PlatformImageOptimizerRegion, diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index 32c126688d..0809c70782 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -71,8 +71,12 @@ use crate::ec::registry::PartnerRegistry; use crate::ec::{EcContext, EidSyncSource}; use crate::error::TrustedServerError; use crate::html_processor::BodyCloseInjection; -use crate::http_util::{RequestInfo, is_navigation_request, serve_static_with_etag}; +use crate::html_processor::ServePass; +use crate::http_util::{ + RequestInfo, is_navigation_request, remove_deployment_headers, serve_static_with_etag, +}; use crate::integrations::{IntegrationRegistry, IntegrationRequestState}; +use crate::middleware::{ResponseHeader, write_response_headers}; use crate::permissions::PermissionState; use crate::platform::{ GeoInfo, PlatformBackendSpec, PlatformHttpRequest, RuntimeServices, @@ -657,6 +661,9 @@ struct ProcessResponseParams<'a> { struct PublisherBodyProcessor { inner: Box, + /// What the serve middleware write on the response, taken by the + /// caller before the response leaves. + response_headers: Vec, } impl PublisherBodyProcessor { @@ -680,8 +687,9 @@ impl PublisherBodyProcessor { params.ad_slots_script.is_some(), is_html && params.dispatched_auction.is_some(), ); + let mut response_headers = Vec::new(); let inner: Box = if is_html { - Box::new(create_html_stream_processor(HtmlStreamProcessorParams { + let (processor, headers) = create_html_stream_processor(HtmlStreamProcessorParams { origin_host: ¶ms.origin_host, request_host: ¶ms.request_host, request_scheme: ¶ms.request_scheme, @@ -697,7 +705,9 @@ impl PublisherBodyProcessor { deferred_inline_marker: inline_seam_token .as_ref() .map(|token| String::from_utf8_lossy(token).into_owned()), - })?) + })?; + response_headers = headers; + Box::new(processor) } else if is_rsc_flight { Box::new(RscFlightUrlRewriter::new( ¶ms.origin_host, @@ -714,7 +724,20 @@ impl PublisherBodyProcessor { )) }; - Ok((Self { inner }, inline_seam_token)) + Ok(( + Self { + inner, + response_headers, + }, + inline_seam_token, + )) + } +} + +impl PublisherBodyProcessor { + /// The headers the serve middleware write on the response, once. + fn take_response_headers(&mut self) -> Vec { + std::mem::take(&mut self.response_headers) } } @@ -739,7 +762,7 @@ fn process_response_streaming( output: &mut W, params: &ProcessResponseParams, output_compression: Compression, -) -> Result<(), Report> { +) -> Result, Report> { let is_html = is_html_content_type(params.content_type); let is_rsc_flight = content_type_contains_ascii_case_insensitive(params.content_type, "text/x-component"); @@ -765,8 +788,9 @@ fn process_response_streaming( // gzip would reject bodies the identity, deflate and brotli paths accept. let max_pending_decoded_bytes = params.settings.publisher.max_buffered_body_bytes; + let mut response_headers = Vec::new(); if is_html { - let processor = create_html_stream_processor(HtmlStreamProcessorParams { + let (processor, headers) = create_html_stream_processor(HtmlStreamProcessorParams { origin_host: params.origin_host, request_host: params.request_host, request_scheme: params.request_scheme, @@ -781,6 +805,7 @@ fn process_response_streaming( csp_nonce_observed: params.csp_nonce_observed.cloned(), deferred_inline_marker: None, })?; + response_headers = headers; StreamingPipeline::new(config, processor) .with_max_pending_decoded_bytes(max_pending_decoded_bytes) .process(body_as_reader(body)?, output)?; @@ -808,7 +833,7 @@ fn process_response_streaming( .process(body_as_reader(body)?, output)?; } - Ok(()) + Ok(response_headers) } async fn process_response_streaming_async( @@ -817,7 +842,7 @@ async fn process_response_streaming_async( params: &OwnedProcessResponseParams, settings: &Settings, integration_registry: &IntegrationRegistry, -) -> Result<(), Report> { +) -> Result, Report> { log::debug!( "process_response_streaming_async: content_type={}, content_encoding={}", params.content_type, @@ -836,6 +861,7 @@ async fn process_response_streaming_async( // pending auction; `deferred_inline_seam_token` returns `None` for it. let (mut processor, inline_seam_token) = PublisherBodyProcessor::new(params, settings, integration_registry)?; + let response_headers = processor.take_response_headers(); if inline_seam_token.is_some() { // A `debug_assert!` would be compiled out of the release wasm builds that // actually ship, which is exactly where an unresolved token would reach a @@ -853,7 +879,8 @@ async fn process_response_streaming_async( output_compression, settings.publisher.max_buffered_body_bytes, ) - .await + .await?; + Ok(response_headers) } /// Pull, decode, process, and encode the next chunk of a no-hold pipeline. @@ -1506,7 +1533,7 @@ fn deferred_inline_seam_token( fn create_html_stream_processor( params: HtmlStreamProcessorParams<'_>, -) -> Result, Report> { +) -> Result<(impl StreamProcessor + use<>, Vec), Report> { use crate::html_processor::{HtmlProcessorConfig, create_html_processor_with_middleware}; use crate::middleware::{HTML_MEDIA_TYPE, MiddlewarePhase}; @@ -1562,9 +1589,9 @@ fn create_html_stream_processor( .map_err(|message| Report::new(TrustedServerError::Configuration { message })) } -/// The processor that carries out the `[[serve]]` entry covering this request -/// on a reader's copy that is assembled and not yet encoded, or `None` when -/// no entry covers the request or its middleware have nothing to do. +/// What the `[[serve]]` entry covering this request does to a reader's copy +/// that is assembled and not yet encoded, and to the response, which is +/// nothing when no entry covers the request. /// /// The origin host is read from the settings. A page served from the store /// keeps no record of it, and a serve middleware is told the same about a @@ -1573,12 +1600,12 @@ fn reader_serve_processor( params: &OwnedProcessResponseParams, settings: &Settings, integration_registry: &IntegrationRegistry, -) -> Result>, Report> { +) -> Result> { use crate::html_processor::{ReaderDocument, create_serve_processor}; use crate::middleware::{HTML_MEDIA_TYPE, MiddlewarePhase}; if !is_html_content_type(¶ms.content_type) { - return Ok(None); + return Ok(ServePass::default()); } let chain = integration_registry.middleware_chain( &settings.serve, @@ -1593,6 +1620,7 @@ fn reader_serve_processor( request_host: ¶ms.request_host, request_scheme: ¶ms.request_scheme, origin_host: &origin_host, + platform: integration_registry.platform(), request_state: ¶ms.request_state, max_buffered_script_bytes: settings.publisher.max_buffered_body_bytes, }, @@ -1601,15 +1629,18 @@ fn reader_serve_processor( } /// Runs the `[[serve]]` entry covering this request over a reader's whole -/// copy, assembled and not yet encoded. +/// copy, assembled and not yet encoded, and writes its headers on the +/// response. fn run_serve_pass( bytes: Vec, params: &OwnedProcessResponseParams, settings: &Settings, integration_registry: &IntegrationRegistry, + response: &mut Response, ) -> Result, Report> { - let Some(mut processor) = reader_serve_processor(params, settings, integration_registry)? - else { + let pass = reader_serve_processor(params, settings, integration_registry)?; + write_response_headers(response, pass.response_headers); + let Some(mut processor) = pass.processor else { return Ok(bytes); }; processor @@ -1932,7 +1963,7 @@ pub async fn buffer_publisher_response_async( return Ok(response); } let mut output = BoundedWriter::new(settings.publisher.max_buffered_body_bytes); - stream_publisher_body_async( + let response_headers = stream_publisher_body_async( body, &mut output, &mut params, @@ -1942,6 +1973,7 @@ pub async fn buffer_publisher_response_async( services, ) .await?; + write_response_headers(&mut response, response_headers); // Authorized template cache transforms are emitted as identity by // `process_response_streaming_async`; inline transforms retain the origin // coding. This avoids recompressing and immediately decoding a full document. @@ -2048,7 +2080,13 @@ pub async fn buffer_publisher_response_async( let bytes = if was_authorized { // The [[serve]] entry runs here on a shared template, and // inside the transform on a page that was never authorized. - let bytes = run_serve_pass(bytes, ¶ms, settings, integration_registry)?; + let bytes = run_serve_pass( + bytes, + ¶ms, + settings, + integration_registry, + &mut response, + )?; encode_complete_body(bytes, response_compression(&response))? } else { bytes @@ -2098,7 +2136,13 @@ pub async fn buffer_publisher_response_async( assembled.extend_from_slice(head); assembled.extend_from_slice(seam.as_bytes()); assembled.extend_from_slice(tail); - let assembled = run_serve_pass(assembled, ¶ms, settings, integration_registry)?; + let assembled = run_serve_pass( + assembled, + ¶ms, + settings, + integration_registry, + &mut response, + )?; let assembled = encode_complete_body(assembled, response_compression(&response))?; response.headers_mut().insert( http::header::CONTENT_LENGTH, @@ -2603,8 +2647,9 @@ pub async fn publisher_response_into_streaming_response( // Planned before the lazy body is built, so a serve entry that // cannot run fails the response while it still can be failed. - let mut serve = match reader_serve_processor(¶ms, &settings, integration_registry) { - Ok(serve) => serve, + let serve_pass = match reader_serve_processor(¶ms, &settings, integration_registry) + { + Ok(serve_pass) => serve_pass, Err(err) => { if let Some(dispatched) = params.dispatched_auction.take() { emit_abandoned_auction( @@ -2618,6 +2663,8 @@ pub async fn publisher_response_into_streaming_response( return Err(err); } }; + write_response_headers(&mut response, serve_pass.response_headers); + let mut serve = serve_pass.processor; let services = services.clone(); let settings = Arc::clone(&settings); let orchestrator = Arc::clone(&orchestrator); @@ -2752,6 +2799,7 @@ pub async fn publisher_response_into_streaming_response( return Err(err); } }; + write_response_headers(&mut response, processor.take_response_headers()); // The guard is created before the lazy stream so an auction whose // response body is dropped unpolled still logs the loss. let dispatched_auction = params.dispatched_auction.take().map(|dispatched| { @@ -3073,7 +3121,7 @@ pub fn stream_publisher_body( params: &OwnedProcessResponseParams, settings: &Settings, integration_registry: &IntegrationRegistry, -) -> Result<(), Report> { +) -> Result, Report> { let permissions_script = permissions_script_for(params, settings); let borrowed = ProcessResponseParams { content_encoding: ¶ms.content_encoding, @@ -3129,7 +3177,7 @@ pub async fn stream_publisher_body_async( integration_registry: &IntegrationRegistry, orchestrator: &AuctionOrchestrator, services: &RuntimeServices, -) -> Result<(), Report> { +) -> Result, Report> { let Some(dispatched) = params.dispatched_auction.take() else { if body.is_stream() { return process_response_streaming_async( @@ -3184,35 +3232,36 @@ pub async fn stream_publisher_body_async( params.ad_slots_script.is_some(), true, ); - let mut processor = match create_html_stream_processor(HtmlStreamProcessorParams { - origin_host: ¶ms.origin_host, - request_host: ¶ms.request_host, - request_scheme: ¶ms.request_scheme, - request_path: ¶ms.request_path, - settings, - integration_registry, - permissions_script: permissions_script_for(params, settings), - ad_slots_script: params.ad_slots_script.as_deref().map(str::to_string), - ad_bids_state: Arc::clone(params.ad_bids_state.script_cell()), - request_state: params.request_state.clone(), - shared_template_authorized: params.template_cache_key.is_some(), - csp_nonce_observed: params.csp_nonce_observed.clone(), - deferred_inline_marker: inline_seam_token - .as_ref() - .map(|token| String::from_utf8_lossy(token).into_owned()), - }) { - Ok(processor) => processor, - Err(err) => { - emit_abandoned_auction( - services, - telemetry.observation, - dispatched, - "processor_init_error", - ) - .await; - return Err(err); - } - }; + let (mut processor, response_headers) = + match create_html_stream_processor(HtmlStreamProcessorParams { + origin_host: ¶ms.origin_host, + request_host: ¶ms.request_host, + request_scheme: ¶ms.request_scheme, + request_path: ¶ms.request_path, + settings, + integration_registry, + permissions_script: permissions_script_for(params, settings), + ad_slots_script: params.ad_slots_script.as_deref().map(str::to_string), + ad_bids_state: Arc::clone(params.ad_bids_state.script_cell()), + request_state: params.request_state.clone(), + shared_template_authorized: params.template_cache_key.is_some(), + csp_nonce_observed: params.csp_nonce_observed.clone(), + deferred_inline_marker: inline_seam_token + .as_ref() + .map(|token| String::from_utf8_lossy(token).into_owned()), + }) { + Ok(built) => built, + Err(err) => { + emit_abandoned_auction( + services, + telemetry.observation, + dispatched, + "processor_init_error", + ) + .await; + return Err(err); + } + }; let input_compression = Compression::from_content_encoding(¶ms.content_encoding); let output_compression = if params.template_cache_key.is_some() { @@ -3240,7 +3289,8 @@ pub async fn stream_publisher_body_async( }, }, ) - .await + .await?; + Ok(response_headers) } /// Builds the canonical ad server placeholder [`Request`] passed to the collect @@ -5387,7 +5437,12 @@ pub async fn handle_publisher_request( services.http_client().send(platform_request).await }; let mut response = match origin_result { - Ok(platform_response) => platform_response.response, + // The origin answered, so nothing in it describes this deployment, + // beneath `/_ts` as anywhere else. + Ok(mut platform_response) => { + remove_deployment_headers(&mut platform_response.response); + platform_response.response + } Err(err) => { if let Some(dispatched) = dispatched_auction.take() { emit_abandoned_auction( @@ -9687,7 +9742,7 @@ mod tests { "/", ) }); - let mut processor = + let (mut processor, _) = create_html_processor_with_middleware(config, &fetch, serve.as_ref()) .expect("should plan the document's middleware"); let out = processor @@ -12295,6 +12350,102 @@ mod tests { } } + /// As [`run`], with the finalizer's registry on a platform reporting + /// `version` for its service, so a stored page's serve middleware are + /// told it. + async fn run_on_a_platform_reporting( + settings: &Arc, + services: &RuntimeServices, + request: Request, + version: &str, + ) -> Response { + let orchestrator = Arc::new(AuctionOrchestrator::new( + crate::auction::test_support::legacy_auction_config(settings), + )); + let registry = IntegrationRegistry::new(settings) + .expect("should create integration registry") + .with_platform(crate::platform::PlatformIdentity { + service_version: version.to_owned(), + ..crate::platform::PlatformIdentity::named("example-platform") + }); + let consent = crate::consent::ConsentContext { + jurisdiction: crate::consent::jurisdiction::Jurisdiction::NonRegulated, + ..Default::default() + }; + let mut ec_context = EcContext::new_for_test(None, consent); + let publisher_response = handle_publisher_request( + AppContext { + settings, + integration_registry: ®istry, + }, + services, + None, + &mut ec_context, + AuctionDispatch { + orchestrator: &orchestrator, + slots: &[article_slot()], + registry: None, + }, + request, + EdgeCacheHeader::SMaxageFallback, + ) + .await + .expect("should proxy publisher request"); + + finalize_test_publisher_response( + publisher_response, + settings, + services, + ®istry, + orchestrator, + Finalizer::Streaming, + ) + .await + } + + /// A stored page's serve middleware write their headers on each + /// reader's response, on the fill and on the hit, and are told the + /// platform the registry carries. + #[tokio::test] + async fn a_stored_page_s_serve_middleware_write_their_headers_on_each_reader_s_response() { + let stub = Arc::new(StubHttpClient::new()); + let cache = Arc::new(MemoryTemplateCache::default()); + let mut settings = settings_with_mode("esi"); + settings.serve = + crate::middleware::PhaseEntries::new(vec![crate::middleware::PhaseEntry { + media_type: crate::middleware::HTML_MEDIA_TYPE.to_owned(), + path: None, + middleware: vec![crate::service_version::MODULE.to_owned()], + }]); + let settings = Arc::new(settings); + let services = services(Arc::clone(&stub), Arc::clone(&cache)); + queue_shareable_html(&stub); + + for cache_status in ["miss-stored", "hit"] { + let response = + run_on_a_platform_reporting(&settings, &services, navigation_request(), "52") + .await; + + assert_eq!( + response + .headers() + .get(HEADER_X_TS_TEMPLATE_CACHE) + .and_then(|value| value.to_str().ok()), + Some(cache_status), + "the page should be stored first and then served from the store" + ); + assert_eq!( + response + .headers() + .get("x-ts-service-version") + .and_then(|value| value.to_str().ok()), + Some("52"), + "should write the version the platform reports on the {cache_status} response" + ); + let _ = body_of(response).await; + } + } + #[tokio::test] async fn a_second_request_is_served_from_the_cache_without_touching_the_origin() { let stub = Arc::new(StubHttpClient::new()); @@ -17537,6 +17688,45 @@ mod tests { ); } + #[tokio::test] + async fn the_origins_answer_beneath_the_route_namespace_loses_the_deployment_headers() { + let settings = create_test_settings(); + let stub = Arc::new(StubHttpClient::new()); + stub.push_response_with_headers( + 404, + b"not found".to_vec(), + vec![ + (header::CONTENT_TYPE.as_str(), "text/plain"), + ("x-served-by", "cache-node-1"), + ("x-cache", "MISS"), + ("x-cache-hits", "0"), + ], + ); + let services = build_services_with_http_client( + Arc::clone(&stub) as Arc + ); + let request = HttpRequest::builder() + .method(Method::GET) + .uri("https://publisher.example/_ts/anything") + .header(header::HOST, "publisher.example") + .body(EdgeBody::empty()) + .expect("should build request"); + + let response = match run_publisher_proxy(&settings, &services, request).await { + PublisherResponse::Buffered(response) + | PublisherResponse::Stream { response, .. } + | PublisherResponse::AssembleTemplate { response, .. } + | PublisherResponse::PassThrough { response, .. } => response, + }; + + for name in ["x-served-by", "x-cache", "x-cache-hits"] { + assert!( + response.headers().get(name).is_none(), + "the origin's {name} should not reach a reader" + ); + } + } + #[tokio::test] async fn publisher_asset_policy_response_with_cookie_is_private_after_finalization() { let settings = settings_from_toml(&format!( @@ -18379,6 +18569,163 @@ mod tests { } } + fn settings_naming_the_service_version() -> Settings { + let mut settings = settings_without_creative_opportunities(); + settings.serve = + crate::middleware::PhaseEntries::new(vec![crate::middleware::PhaseEntry { + media_type: crate::middleware::HTML_MEDIA_TYPE.to_owned(), + path: None, + middleware: vec![crate::service_version::MODULE.to_owned()], + }]); + settings + } + + /// The registry of `settings` on a platform reporting `version` for + /// its service. + fn registry_on_a_platform_reporting( + settings: &Settings, + version: &str, + ) -> IntegrationRegistry { + IntegrationRegistry::new(settings) + .expect("should create integration registry") + .with_platform(crate::platform::PlatformIdentity { + service_version: version.to_owned(), + ..crate::platform::PlatformIdentity::named("example-platform") + }) + } + + fn service_version_header(response: &Response) -> Option { + response + .headers() + .get("x-ts-service-version") + .map(|value| String::from_utf8_lossy(value.as_bytes()).into_owned()) + } + + fn git_version_header(response: &Response) -> Option<&str> { + response + .headers() + .get("x-ts-version") + .and_then(|value| value.to_str().ok()) + } + + /// A page from the origin, as the publisher path hands it to a + /// finalizer, with the services it was fetched through. + async fn origin_page(settings: &Settings) -> (PublisherResponse, RuntimeServices) { + let stub = Arc::new(StubHttpClient::new()); + queue_html_response_with_cache_control(&stub, "public, max-age=300"); + let services = build_services_with_http_client( + Arc::clone(&stub) as Arc + ); + let response = run_with_slots( + settings, + &services, + &[article_slot()], + conditional_navigation_request(), + ) + .await; + (response, services) + } + + async fn buffered( + settings: &Settings, + registry: &IntegrationRegistry, + ) -> Response { + let (response, services) = origin_page(settings).await; + let orchestrator = AuctionOrchestrator::new( + crate::auction::test_support::legacy_auction_config(settings), + ); + buffer_publisher_response_async( + response, + &Method::GET, + settings, + registry, + &orchestrator, + &services, + ) + .await + .expect("should buffer the page") + } + + #[tokio::test] + async fn the_service_version_middleware_writes_the_header_on_a_buffered_page() { + let settings = settings_naming_the_service_version(); + let registry = registry_on_a_platform_reporting(&settings, "52"); + + let response = buffered(&settings, ®istry).await; + + assert_eq!( + service_version_header(&response).as_deref(), + Some("52"), + "should write the version the platform reports on the buffered page" + ); + assert_eq!( + git_version_header(&response), + crate::constants::TS_GIT_VERSION, + "should write the git version the build compiled in on the buffered page" + ); + } + + #[tokio::test] + async fn the_service_version_middleware_writes_the_header_on_a_streamed_page() { + let settings = settings_naming_the_service_version(); + let registry = registry_on_a_platform_reporting(&settings, "52"); + let (response, services) = origin_page(&settings).await; + let orchestrator = Arc::new(AuctionOrchestrator::new( + crate::auction::test_support::legacy_auction_config(&settings), + )); + + let response = publisher_response_into_streaming_response( + response, + &Method::GET, + Arc::new(settings), + ®istry, + orchestrator, + services, + ) + .await + .expect("should stream the page"); + + assert_eq!( + service_version_header(&response).as_deref(), + Some("52"), + "should write the version the platform reports before the body streams" + ); + } + + #[tokio::test] + async fn a_page_no_serve_entry_names_the_middleware_on_carries_no_version() { + let settings = settings_without_creative_opportunities(); + let registry = registry_on_a_platform_reporting(&settings, "52"); + + let response = buffered(&settings, ®istry).await; + + assert_eq!( + service_version_header(&response), + None, + "should write nothing where no entry names the middleware" + ); + assert_eq!( + git_version_header(&response), + None, + "should write no git version where no entry names the middleware" + ); + } + + #[tokio::test] + async fn a_platform_reporting_no_version_writes_no_header() { + let settings = settings_naming_the_service_version(); + let registry = + IntegrationRegistry::new(&settings).expect("should create integration registry"); + + let response = buffered(&settings, ®istry).await; + + assert_eq!( + service_version_header(&response), + None, + "should write nothing for a version the platform did not report" + ); + } + #[tokio::test] async fn disabled_ad_templates_use_short_browser_cache_policy() { // Arrange diff --git a/crates/trusted-server-core/src/service_version.rs b/crates/trusted-server-core/src/service_version.rs new file mode 100644 index 0000000000..46b9d82ab1 --- /dev/null +++ b/crates/trusted-server-core/src/service_version.rs @@ -0,0 +1,168 @@ +//! The deployment's versions as response headers, a middleware of core's +//! own named `service_version` in a `[[serve]]` entry. A deployment that +//! names it writes `X-TS-Version`, the git version the build compiled in, +//! when the build knows one, and `X-TS-Service-Version`, the version the +//! adapter reports for the service, when it reports one, on every page the +//! entry covers. A deployment that does not name it sends nothing about its +//! versions on a page, because no adapter writes either header. + +use http::HeaderValue; + +use crate::constants::{HEADER_X_TS_SERVICE_VERSION, HEADER_X_TS_VERSION, TS_GIT_VERSION}; +use crate::middleware::{Middleware, MiddlewareAction, MiddlewareContext, MiddlewarePhase}; +use crate::platform::UNKNOWN; + +/// The name an entry writes, and the integration the registry lists the +/// middleware under. +pub const MODULE: &str = "service_version"; + +/// The middleware. It holds the git version the build compiled in, and reads +/// the platform's version from the context. +#[derive(Debug)] +pub struct ServiceVersion { + /// The deployed git version, `None` when the build knew none. + pub git_version: Option<&'static str>, +} + +impl Default for ServiceVersion { + fn default() -> Self { + Self { + git_version: TS_GIT_VERSION, + } + } +} + +impl Middleware for ServiceVersion { + fn middleware_id(&self) -> &'static str { + MODULE + } + + fn phases(&self) -> &[MiddlewarePhase] { + &[MiddlewarePhase::Serve] + } + + fn create(&self, context: &MiddlewareContext<'_>) -> MiddlewareAction { + let mut response_headers = Vec::new(); + if let Some(value) = header_value("git version", self.git_version) { + response_headers.push((HEADER_X_TS_VERSION, value)); + } + let service_version = + Some(context.platform.service_version.as_str()).filter(|version| *version != UNKNOWN); + if let Some(value) = header_value("service version", service_version) { + response_headers.push((HEADER_X_TS_SERVICE_VERSION, value)); + } + MiddlewareAction { + response_headers, + ..MiddlewareAction::pass() + } + } +} + +/// `version` as a header value, or `None`, with a warning, when a header +/// cannot carry it. +fn header_value(what: &str, version: Option<&str>) -> Option { + let version = version?; + match HeaderValue::from_str(version) { + Ok(value) => Some(value), + Err(_) => { + log::warn!("the {what} {version:?} is not a header value, so it is not written"); + None + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::integrations::IntegrationDocumentState; + use crate::middleware::test_support; + use crate::platform::PlatformIdentity; + + fn action_for(git_version: Option<&'static str>, service_version: &str) -> MiddlewareAction { + let platform = PlatformIdentity { + service_version: service_version.to_owned(), + ..PlatformIdentity::named("example-platform") + }; + let document_state = IntegrationDocumentState::default(); + ServiceVersion { git_version }.create(&test_support::context_on( + MiddlewarePhase::Serve, + &document_state, + &platform, + )) + } + + fn headers_of(action: &MiddlewareAction) -> Vec<(&str, &[u8])> { + action + .response_headers + .iter() + .map(|(name, value)| (name.as_str(), value.as_bytes())) + .collect() + } + + #[test] + fn the_git_version_the_build_compiled_in_is_written_as_x_ts_version() { + let action = action_for(Some("v1.3.0"), UNKNOWN); + + assert_eq!( + headers_of(&action), + [("x-ts-version", b"v1.3.0".as_slice())], + "should write the git version alone on a platform reporting no version" + ); + } + + #[test] + fn the_version_the_platform_reports_is_written_as_x_ts_service_version() { + let action = action_for(None, "52"); + + assert_eq!( + headers_of(&action), + [("x-ts-service-version", b"52".as_slice())], + "should write the platform's version alone when the build knew no git version" + ); + } + + #[test] + fn both_versions_are_written_when_both_are_known() { + let action = action_for(Some("v1.3.0"), "52"); + + assert_eq!( + headers_of(&action), + [ + ("x-ts-version", b"v1.3.0".as_slice()), + ("x-ts-service-version", b"52".as_slice()), + ], + "should write the git version and then the platform's" + ); + } + + #[test] + fn a_build_knowing_no_git_version_on_a_platform_reporting_none_writes_nothing() { + assert!( + action_for(None, UNKNOWN).is_pass(), + "should write nothing when neither version is known" + ); + } + + #[test] + fn a_version_that_is_not_a_header_value_is_not_written() { + assert!( + action_for(Some("v1\n"), "52\n").is_pass(), + "should write nothing for versions a header cannot carry" + ); + } + + #[test] + fn the_default_carries_the_git_version_the_build_compiled_in() { + assert_eq!( + ServiceVersion::default().git_version, + TS_GIT_VERSION, + "should carry the compiled-in TS_GIT_VERSION" + ); + } + + #[test] + fn it_runs_in_the_serve_phase_under_its_name() { + assert_eq!(ServiceVersion::default().phases(), [MiddlewarePhase::Serve]); + assert_eq!(ServiceVersion::default().middleware_id(), "service_version"); + } +} diff --git a/crates/trusted-server-core/src/version_header.rs b/crates/trusted-server-core/src/version_header.rs deleted file mode 100644 index 822e04b26c..0000000000 --- a/crates/trusted-server-core/src/version_header.rs +++ /dev/null @@ -1,176 +0,0 @@ -//! `x-ts-version`: the deployed git version compiled in by `build.rs`. - -use edgezero_core::http::{HeaderValue, Response}; - -use crate::constants::{HEADER_X_TS_VERSION, TS_GIT_VERSION}; - -/// Returns the compiled-in git version as a header value. -/// -/// `None` when no version is known or the value is not a valid header value. -#[must_use] -pub fn git_version_header_value() -> Option { - header_value_from(TS_GIT_VERSION) -} - -/// Converts `version` to a header value, logging and returning `None` when it -/// is not a valid header value. -#[must_use] -fn header_value_from(version: Option<&str>) -> Option { - let version = version?; - match HeaderValue::from_str(version) { - Ok(value) => Some(value), - Err(_) => { - log::warn!("Skipping invalid TRUSTED_SERVER__GIT_VERSION response header value"); - None - } - } -} - -/// Sets `x-ts-version` to the compiled-in git version, if one is known. -pub fn apply_git_version_header(response: &mut Response) { - apply_git_version_header_from(TS_GIT_VERSION, response); -} - -/// Sets `x-ts-version` to `version`, or removes it when `version` is unknown -/// or invalid, so an origin's `x-ts-version` is never reported as ours. -fn apply_git_version_header_from(version: Option<&str>, response: &mut Response) { - if let Some(value) = header_value_from(version) { - response.headers_mut().insert(HEADER_X_TS_VERSION, value); - } else { - response.headers_mut().remove(HEADER_X_TS_VERSION); - } -} - -#[cfg(test)] -mod tests { - use super::*; - use edgezero_core::body::Body; - use edgezero_core::http::response_builder; - - fn empty_response() -> Response { - response_builder() - .body(Body::empty()) - .expect("should build empty test response") - } - - fn response_with_upstream_version() -> Response { - response_builder() - .header(HEADER_X_TS_VERSION, "upstream-v1") - .body(Body::empty()) - .expect("should build test response with an upstream x-ts-version") - } - - fn version_of(response: &Response) -> Option<&str> { - response - .headers() - .get(HEADER_X_TS_VERSION) - .and_then(|v| v.to_str().ok()) - } - - #[test] - fn header_value_from_valid_version() { - assert_eq!( - header_value_from(Some("v1.2.3")), - Some(HeaderValue::from_static("v1.2.3")), - "should convert a valid version" - ); - } - - #[test] - fn header_value_from_unknown_or_invalid_version() { - assert_eq!(header_value_from(None), None, "should be None when unknown"); - assert_eq!( - header_value_from(Some("bad\nvalue")), - None, - "should be None for a non-header-safe version" - ); - } - - #[test] - fn git_version_header_value_matches_compiled_in_version() { - assert_eq!( - git_version_header_value() - .as_ref() - .and_then(|v| v.to_str().ok()), - TS_GIT_VERSION, - "should convert the compiled-in TS_GIT_VERSION" - ); - } - - #[test] - fn sets_header_from_version() { - let mut response = empty_response(); - apply_git_version_header_from(Some("v1.2.3"), &mut response); - assert_eq!( - version_of(&response), - Some("v1.2.3"), - "should set x-ts-version" - ); - } - - #[test] - fn omits_header_when_version_unknown() { - let mut response = empty_response(); - apply_git_version_header_from(None, &mut response); - assert_eq!( - version_of(&response), - None, - "should omit x-ts-version when unknown" - ); - } - - #[test] - fn skips_invalid_header_value() { - let mut response = empty_response(); - apply_git_version_header_from(Some("bad\nvalue"), &mut response); - assert_eq!( - version_of(&response), - None, - "should skip a non-header-safe version" - ); - } - - #[test] - fn replaces_upstream_header_with_version() { - let mut response = response_with_upstream_version(); - apply_git_version_header_from(Some("v2.0.0"), &mut response); - assert_eq!( - version_of(&response), - Some("v2.0.0"), - "should replace an upstream x-ts-version with ours" - ); - } - - #[test] - fn removes_upstream_header_when_version_unknown() { - let mut response = response_with_upstream_version(); - apply_git_version_header_from(None, &mut response); - assert_eq!( - version_of(&response), - None, - "should remove an upstream x-ts-version when ours is unknown" - ); - } - - #[test] - fn removes_upstream_header_when_version_invalid() { - let mut response = response_with_upstream_version(); - apply_git_version_header_from(Some("bad\nvalue"), &mut response); - assert_eq!( - version_of(&response), - None, - "should remove an upstream x-ts-version when ours is invalid" - ); - } - - #[test] - fn default_uses_compiled_in_version() { - let mut response = empty_response(); - apply_git_version_header(&mut response); - assert_eq!( - version_of(&response), - TS_GIT_VERSION, - "should report the compiled-in TS_GIT_VERSION" - ); - } -} diff --git a/crates/trusted-server-modules/src/lib.rs b/crates/trusted-server-modules/src/lib.rs index c55f0ba443..471f9ba364 100644 --- a/crates/trusted-server-modules/src/lib.rs +++ b/crates/trusted-server-modules/src/lib.rs @@ -85,8 +85,8 @@ pub struct StockMiddleware { } /// The middleware a stock build's modules supply, core's script proxy among -/// them, in the order a deployment that runs them all names them in an -/// entry. +/// them and core's service version last, in the order a deployment that +/// runs them all names them in an entry. /// /// That is the order the registry registers the modules in. Prebid registers /// from the auction plan and so comes first, core's script proxy is the first @@ -101,6 +101,7 @@ pub fn middleware() -> Vec { use trusted_server_cmp_didomi as didomi; use trusted_server_cmp_sourcepoint as sourcepoint; use trusted_server_core::integrations::js_asset_proxy; + use trusted_server_core::service_version; use trusted_server_framework_nextjs as nextjs; use trusted_server_identity_lockr as lockr; use trusted_server_tag_google_tag_manager as google_tag_manager; @@ -134,6 +135,9 @@ pub fn middleware() -> Vec { Serve, google::diagnostics::MODULE, ), + // Core writes the platform's service version where a serve entry + // names it. + row(service_version::MODULE, Serve, service_version::MODULE), ] } @@ -232,6 +236,8 @@ mod tests { order.push(builder.id()); } } + // Core's own middleware is registered after every module's. + order.push(trusted_server_core::service_version::MODULE); let place = |integration: &str| { order .iter() diff --git a/docs/guide/configuration.md b/docs/guide/configuration.md index 2e199e135a..26d266df19 100644 --- a/docs/guide/configuration.md +++ b/docs/guide/configuration.md @@ -1128,6 +1128,22 @@ on that host, including `/first-party/proxy`, which serves any target that allows those too. Set `proxy.allowed_domains` to bound what the proxy serves. ::: +### Headers that describe the deployment + +`X-Served-By`, `X-Cache` and `X-Cache-Hits` name the node that answered and say how a cache treated the request. They arrive on a response from the publisher's origin or from a cache on the way to it. On a publisher's page they tell a reader nothing, and tell something probing the site how it is built. + +Every adapter removes the three from a response before it finalizes it, unless this server answered a request for a path beneath `/_ts`, which is where a deployment is asked about itself. What the publisher's origin sends loses them beneath `/_ts` too. The removal comes before `[response_headers]` is applied, so a header of one of those names set there is still sent. + +No adapter writes a version on a response unless the deployment names the middleware that writes one. + +A deployment that wants the versions on its pages names core's `service_version` middleware in the `[[serve]]` entry that covers them, after the module middleware the entry already names. The middleware runs on every page the entry covers, and on nothing else. It writes `X-TS-Version`, the git version the build compiled in from `TRUSTED_SERVER__GIT_VERSION` or local git, when the build knows one, and `X-TS-Service-Version`, the version the adapter reports for the service, when it reports one. No section selects the middleware, and a deployment that does not name it sends nothing about its versions on a page. + +```toml +[[serve]] +media_type = "text/html" +middleware = ["service_version"] +``` + ## robots.txt Configuration What `/robots.txt` answers. With no `[robots-txt]` section the publisher keeps their own file and `/robots.txt` reaches the origin like any other path. @@ -1831,6 +1847,7 @@ the names in and the order `trusted-server.example.toml` shows them in. | `bot-protection.datadome.tag` | `[[serve]]` | Writes DataDome's client tag into the head, unless the request filter marked the request, `inject_client_side_tag` is off or no `client_side_key` is set | | `ad-tag.google` | `[[fetch]]` | Writes the `tsjs.adInit` bootstrap into the head and, when `rewrite_script` is set, points a `src` or `href` that is the GPT script's address at `/integrations/gpt/script` | | `ad-tag.google.diagnostics` | `[[serve]]` | For a request that activated diagnostics, writes the bootstrap ahead of the script bundle and the tag that loads the diagnostics module straight after it | +| `service_version` | `[[serve]]` | Writes `X-TS-Version`, the git version the build compiled in, and `X-TS-Service-Version`, the platform's version number of the service, when each is known. Core supplies it, so no section selects it | An entry that could not do what it says refuses the configuration, both when a deployment is validated and when the settings load. That is a media type diff --git a/docs/guide/edgezero.md b/docs/guide/edgezero.md index d42290fb52..9d46137684 100644 --- a/docs/guide/edgezero.md +++ b/docs/guide/edgezero.md @@ -76,8 +76,8 @@ store before deletion, especially when using `--store` or `--no-env`. 4. Run `ts config push --adapter ADAPTER --permissions `. 5. Start or deploy the adapter and verify a non-health publisher route. A healthy endpoint alone may not prove that application configuration loaded. - Confirm that its `x-ts-version` response header reports the release you - deployed. + Confirm which build is serving from the `x-ts-version` header on a page + where a `[[serve]]` entry names `service_version`. See [Configuration](/guide/configuration) for the field contract, the [CLI guide](/guide/cli) for exact command help, and diff --git a/docs/guide/getting-started.md b/docs/guide/getting-started.md index 7550363167..cb53398fdd 100644 --- a/docs/guide/getting-started.md +++ b/docs/guide/getting-started.md @@ -239,14 +239,16 @@ upgrading a service that was deployed before logical store links, and [the CLI guide](/guide/cli#lifecycle-commands) for where `` comes from. -The build compiles `TRUSTED_SERVER__GIT_VERSION` into the binary and reports it -in the `x-ts-version` response header, so the variable must be set for the -build the deploy runs, not just for the activation step. The Fastly service -version is in `x-ts-fastly-version`. Without the variable, a build reports the -checked-out tag, else the branch, else the first 6 characters of the commit, -and a build outside a git checkout omits the header. The value is public on -every response, so pass a neutral value such as the commit hash when a branch -name is sensitive. +The build compiles `TRUSTED_SERVER__GIT_VERSION` into the binary, so the +variable must be set for the build the deploy runs, not just for the +activation step. Without the variable, a build compiles in the checked-out +tag, else the branch, else the first 6 characters of the commit, and a build +outside a git checkout knows no version. No adapter writes it on a response. +A deployment that names core's `service_version` middleware in a `[[serve]]` +entry writes it as `x-ts-version` on the pages the entry covers, with the +Fastly service version as `x-ts-service-version`, so pass a neutral value such +as the commit hash when a branch name is sensitive. See +[Headers that describe the deployment](/guide/configuration#headers-that-describe-the-deployment). ## Next Steps diff --git a/trusted-server.example.toml b/trusted-server.example.toml index d49fb52e84..55b3d96090 100644 --- a/trusted-server.example.toml +++ b/trusted-server.example.toml @@ -327,10 +327,11 @@ pull_sync_concurrency = 3 # the pages where a [[fetch]] or [[serve]] entry names it. A page takes the # first entry that covers it and runs that entry's middleware in the order # written. The two entries below name every middleware the modules further -# down supply, in the order `ts audit` writes them. Keep the names of the -# modules that are selected and remove the rest, because an entry that names -# the middleware of a module no section selects refuses startup. See "Placing -# page changes" in the configuration guide. +# down supply, in the order `ts audit` writes them, and core's own +# `service_version` last. Keep the names of the modules that are selected and +# remove the rest, because an entry that names the middleware of a module no +# section selects refuses startup. See "Placing page changes" in the +# configuration guide. # [[fetch]] # media_type = "text/html" # middleware = [ @@ -349,7 +350,7 @@ pull_sync_concurrency = 3 # [[serve]] # media_type = "text/html" -# middleware = ["bot-protection.datadome.tag", "ad-tag.google.diagnostics"] +# middleware = ["bot-protection.datadome.tag", "ad-tag.google.diagnostics", "service_version"] # First-party proxying of named third-party scripts. Uncomment the # `modules` line under [proxy] below to switch it on.