diff --git a/CHANGELOG.md b/CHANGELOG.md index 4fdba05db..24a506495 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- The Axum, Cloudflare and Spin adapters serve `[[proxy.asset_routes]]`, where only the Fastly adapter did, so a `GET` or `HEAD` under a route's prefix is answered from the route's source, with its signing and with the answers a selected asset cache module holds for it, and no longer by the publisher's origin. The response skips the Edge Cookie steps, so it neither writes nor expires the Edge Cookie, and takes the route's cache policy after the finalize headers. Those adapters hold the asset body whole, up to 10 MiB on Cloudflare and 8 MiB on Spin. Their HTTP clients refuse Image Optimizer metadata, which the Axum client now does on both of its send paths, so there a route that enables the Image Optimizer answers each request it would optimize with `502` and `no-store, private`, after a signed route has asked its origin for the object with a `HEAD`, and serves its SVG paths and requests that set the debug parameter unoptimized. Each adapter logs a warning naming every such route, once for each instance of the adapter, which on Axum is the process. +- **Breaking:** on Spin, a request to a source whose `host_header` names a host other than the one in its `url` answers `502` before its origin is asked, the publisher's `[source.origin]` as well as an asset route's source, because Spin sends the host in `url` as the `Host`. Before, the request reached its origin with the host in `url` in place of `host_header`. A Spin deployment whose publisher source sets such a `host_header` answers every page with `502`, robots.txt included. Before upgrading, check that no source of a Spin deployment sets a `host_header` naming another host, or remove it once its origin answers requests that carry the host in `url` as the `Host`. - The Edge Cookie is written without an identity graph as it is with one, because the graph is one optional store of the data kept with the cookie, such as partner IDs. A Fastly service with no `[ec] ec_store` now writes the cookie a real browser's page navigation creates, and `POST /_ts/api/v1/ec/resolve` answers there as it does with a graph, where neither wrote one. So the identifier auction providers get as `user.id`, where storage and personalized-ad selection are both set, is one value for the life of the cookie, where a server-side module such as `hmac` gave a new value on each page view and a client-side module gave none. Naming an `ec_store` later gives each browser holding a cookie a server-side module wrote without one a new identifier on its first page view, because the identity graph replaces an identifier it has no row for. A client-side module creates nothing at the edge to replace its cookie with, so that cookie keeps its identifier with no row for as long as the browser keeps the cookie and its resolved marker, and each of that browser's page views pays for an extra identity-graph read and a strongly consistent existence check. **Breaking for crates outside core:** `ec_finalize_response` and `handle_ec_resolve` read the bot gate from the context's device signals (`EcContext::set_device_signals`), so a context with none gets no created cookie. - The Axum, Cloudflare and Spin adapters create and write the Edge Cookie and serve `POST /_ts/api/v1/ec/resolve`, where only the Fastly adapter did. A real browser's page navigation is given an Edge Cookie where its permissions allow one, and every route in the request cycle finalizes the Edge Cookie once the response is finalized, which writes a created cookie under the request's public prefix and path and expires one on a withdrawal. Those adapters open no identity graph, so the cookie is written with no graph row, by the resolve route too, the graph's syncs are skipped, and startup logs once that the store `[ec] ec_store` names is not opened. The bot gate there passes any client that sends a browser's `User-Agent`, and where storage and personalized-ad selection are both set a real browser's identifier reaches auction providers as `user.id`, from its first page view with a server-side module and from the requests after its resolve with a client-side one. - The Axum, Cloudflare and Spin adapters run the integration request filters and classify each request with the device module `[device]` selects, as the Fastly adapter does. Before, they ran no filter and asked a selected device module only for a bid request's device attributes. Every named route but the inspection endpoints, and the publisher fallback, reads the request's Edge Cookie context with the device classification and then runs the filters before the route, so a filter that answers, such as DataDome's challenge, stands in for the route, and the response headers a filter names are written after the finalize headers, with the Fastly adapter's privacy guards applied after them. The 51Degrees client hints middleware and DataDome's protection therefore work on every adapter, through HTTP clients that apply no module's own timeout on Axum, Cloudflare and Spin, and DataDome's protection checks `/auction` and `GET` on both page-bids paths as well as the publisher's pages. On those adapters `[device] module = "platform"` classifies from the `User-Agent`, a response core marks private loses any `ETag`, `Last-Modified`, `Expires` or `Age` that `[response_headers]` writes, robots.txt is answered before the Edge Cookie steps and the filters with an empty Edge Cookie context, and on Spin the discovery, signature verification and first-party routes and the page-bids preflight now run the modules' preparers. @@ -118,6 +120,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- The Cloudflare and Spin adapters install a logger, so the warnings and errors they log are written. EdgeZero installs none on either platform, so every line was dropped, startup failures and each error answer's report included. The Worker writes each line with the console method of its level, which `wrangler dev` prints and `wrangler tail` and the dashboard's live logs show, and the Spin component writes each line to standard error, which Spin keeps as the component's log. Both write `info` and above. - The origin's address in a page's `href`, `src`, `action`, `srcset` and `imagesrcset` is now replaced in one pass. A request host that begins with the origin host, such as `publisher.example.localhost` for `publisher.example`, was matched again inside its own replacement, so `https://publisher.example/` served over `http` came out as `http://publisher.example.localhost.localhost.localhost/`. A `srcset` served on a `www.` host in front of an apex origin came out with `www.www.`. - Publisher responses now map origin URLs in `Location`, `Content-Location`, `Refresh`, and `Link` headers to the serving host, and each CSP source naming the origin gains a serving-host source beside it (#1128). A redirect to the current URL under a different scheme, or a `GET`/`HEAD` redirect to the current URL that sets no cookie (or any such redirect under a `Host` override), is left unchanged to avoid a loop. `` refresh and CSP tags in the page body are not rewritten. See "Origin URLs in proxied response headers" in the configuration guide. - Image Optimizer configuration now rejects profile and profile-set names within the same map that collide after trimming, even when their values match, instead of silently overwriting an entry. Empty names continue to be discarded. diff --git a/crates/trusted-server-adapter-axum/src/app.rs b/crates/trusted-server-adapter-axum/src/app.rs index 97ef9f96b..1c64aa4a9 100644 --- a/crates/trusted-server-adapter-axum/src/app.rs +++ b/crates/trusted-server-adapter-axum/src/app.rs @@ -37,8 +37,8 @@ use trusted_server_core::integrations::{ }; use trusted_server_core::module_context::{ModuleContext, ModuleRequest}; use trusted_server_core::proxy::{ - handle_first_party_click, handle_first_party_proxy, handle_first_party_proxy_rebuild, - handle_first_party_proxy_sign, + AssetProxyCachePolicy, handle_asset_proxy_request, handle_first_party_click, + handle_first_party_proxy, handle_first_party_proxy_rebuild, handle_first_party_proxy_sign, }; use trusted_server_core::publisher::{ AppContext, AuctionDispatch, PAGE_BIDS_LEGACY_PATH, PAGE_BIDS_PATH, @@ -120,6 +120,38 @@ fn warn_once_when_the_identity_graph_is_unavailable(settings: &Settings, logged: ); } +/// Set once startup has logged that an asset route enables the Image +/// Optimizer, which this adapter does not have. The state can be built more +/// than once, so the flag keeps the warning to one for each instance. +static IMAGE_OPTIMIZER_UNAVAILABLE_LOGGED: AtomicBool = AtomicBool::new(false); + +/// Logs once, at warning level, the prefix of each asset route that enables +/// the Image Optimizer, which this adapter does not have. +/// +/// The HTTP client refuses a request carrying Image Optimizer metadata, so +/// such a route answers each request it would optimize with `502`. The route +/// is not refused at startup, because its SVG paths and debug requests are +/// still answered, as are the deployment's other routes and pages. +fn warn_once_when_an_asset_route_enables_the_image_optimizer(settings: &Settings) { + let prefixes: Vec = settings + .proxy + .asset_routes + .iter() + .filter(|route| route.image_optimizer.as_ref().is_some_and(|io| io.enabled)) + .map(|route| format!("`{}`", route.prefix)) + .collect(); + if prefixes.is_empty() || IMAGE_OPTIMIZER_UNAVAILABLE_LOGGED.swap(true, Ordering::Relaxed) { + return; + } + log::warn!( + "the Axum adapter has no Image Optimizer, so each request these asset \ + routes would optimize answers 502 Bad Gateway, and their SVG paths and \ + requests that set the profile set's debug parameter are served \ + unoptimized: {}", + prefixes.join(", ") + ); +} + /// The permission signal modules this adapter links, in the order they run /// when configuration names none. Global Privacy Control is first because it /// is a browser setting with no interface of its own, and the three that @@ -231,7 +263,7 @@ fn build_state_with_settings( } /// The header core writes a shared cache policy in on this adapter. -const EDGE_CACHE_HEADER: EdgeCacheHeader = EdgeCacheHeader::SMaxageFallback; +pub(crate) const EDGE_CACHE_HEADER: EdgeCacheHeader = EdgeCacheHeader::SMaxageFallback; /// Build the application state from explicit settings, composing the modules /// this adapter links with the externally supplied builders in @@ -329,6 +361,7 @@ fn build_state_with_registrations_and_services( ); warn_when_certificate_check_is_off(&settings); warn_once_when_the_identity_graph_is_unavailable(&settings, &IDENTITY_GRAPH_UNAVAILABLE_LOGGED); + warn_once_when_an_asset_route_enables_the_image_optimizer(&settings); let permission_policy = settings.required_permission_policy()?; Ok(Arc::new(AppState { @@ -849,6 +882,15 @@ async fn dispatch_fallback( })) }) } else { + // A `GET` or `HEAD` under an asset route goes to the route's own + // source in place of the publisher's origin. Its response skips the + // Edge Cookie steps, so it carries no Edge Cookie state. + let matched_asset_route = matches!(method, Method::GET | Method::HEAD) + .then(|| state.settings.asset_route_index_for_path(&path)) + .flatten(); + if let Some(index) = matched_asset_route { + return dispatch_asset_fallback(state, services, req, index, effects).await; + } // A real browser's page navigation is given an Edge Cookie when it has // none. Another request may lack consent signals such as `Sec-GPC`. if ec.is_real_browser @@ -908,6 +950,64 @@ async fn dispatch_fallback( ) } +/// Whether an asset response carries a body. +/// +/// `HEAD` responses and the bodiless statuses (204, 304) keep the origin's +/// `Content-Length` and carry no body, so a body stream is dropped for them. +fn asset_response_carries_body(method: &Method, status: StatusCode) -> bool { + *method != Method::HEAD + && status != StatusCode::NO_CONTENT + && status != StatusCode::NOT_MODIFIED +} + +/// Answers a request under the asset route at `index` from the route's own +/// source. +/// +/// The route's source module locates the asset, its signer signs the request +/// where the source's `auth` table names one, and core answers from a copy it +/// holds before it asks the origin. The response carries its cache policy, +/// which [`FinalizeResponseMiddleware`] applies after the finalize headers, +/// and the request filters' effects, and no Edge Cookie state, because an +/// asset response skips the Edge Cookie steps. +async fn dispatch_asset_fallback( + state: &AppState, + services: &RuntimeServices, + req: Request, + index: usize, + effects: RequestFilterEffects, +) -> Response { + let method = req.method().clone(); + let asset_route = &state.settings.proxy.asset_routes[index]; + let answer = match state.registry.route_source(&state.settings, index) { + Ok(source) => { + handle_asset_proxy_request(&state.settings, services, req, asset_route, source.as_ref()) + .await + } + Err(report) => Err(report), + }; + let response = match answer { + Ok(asset_response) => { + let cache_policy = asset_response.cache_policy(); + let (mut response, stream_body) = asset_response.into_response_and_body(); + if let Some(body) = stream_body + && asset_response_carries_body(&method, response.status()) + { + *response.body_mut() = body; + } + response.extensions_mut().insert(cache_policy); + response + } + Err(report) => { + let mut response = http_error(&report); + response + .extensions_mut() + .insert(AssetProxyCachePolicy::NoStorePrivate); + response + } + }; + attach_request_filter_effects(response, effects) +} + /// Whether the request is a crawler's `GET` or `HEAD` of the robots.txt that /// `[robots-txt]` makes. fn asks_for_robots_txt(settings: &Settings, method: &Method, path: &str) -> bool { diff --git a/crates/trusted-server-adapter-axum/src/app/request_cycle_tests.rs b/crates/trusted-server-adapter-axum/src/app/request_cycle_tests.rs index 5cd104ab5..bfaab9f9f 100644 --- a/crates/trusted-server-adapter-axum/src/app/request_cycle_tests.rs +++ b/crates/trusted-server-adapter-axum/src/app/request_cycle_tests.rs @@ -4,9 +4,11 @@ //! without making the response less private, the route's module call carries //! the device classification, an Edge Cookie context that cannot be read //! answers before the filters, robots.txt and the inspection endpoints are -//! answered without them, and the Edge Cookie is created and written for a -//! real browser's page view, set by the resolve route for a client-side -//! module and expired on a withdrawal. +//! answered without them, the Edge Cookie is created and written for a real +//! browser's page view, set by the resolve route for a client-side module and +//! expired on a withdrawal, and a request under an asset route is answered +//! from the route's own source, with the route's cache rule written after the +//! finalize headers, an error that no cache stores and no Edge Cookie. use std::net::{IpAddr, Ipv4Addr}; use std::sync::atomic::{AtomicBool, Ordering}; @@ -1069,6 +1071,217 @@ fn startup_warns_that_this_adapter_opens_no_identity_graph() { ); } +// --------------------------------------------------------------------------- +// Asset routes +// --------------------------------------------------------------------------- + +/// Settings with an asset route under `/assets/` whose source signs its +/// requests with the stock `s3_sigv4` signer. +fn settings_with_a_signed_asset_route() -> Settings { + settings_with( + r#" + [proxy] + + [[proxy.asset_routes]] + prefix = "/assets/" + + [proxy.asset_routes.source] + module = "origin" + + [proxy.asset_routes.source.origin] + url = "https://assets.example.com" + + [proxy.asset_routes.source.origin.auth] + type = "s3_sigv4" + region = "us-east-1" + "#, + ) +} + +/// What the origin was asked, being each request's address and whether it +/// carried an `s3_sigv4` signature. +fn asked(origin: &StubHttpClient) -> Vec<(String, bool)> { + origin + .recorded_request_uris() + .into_iter() + .zip(origin.recorded_request_headers()) + .map(|(uri, headers)| { + let signed = headers.iter().any(|(name, value)| { + name == "authorization" && value.starts_with("AWS4-HMAC-SHA256 ") + }); + (uri, signed) + }) + .collect() +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_under_an_asset_route_reaches_the_route_s_origin_signed() { + let origin = Arc::new(StubHttpClient::new()); + origin.push_response_with_headers( + 200, + b"image bytes".to_vec(), + vec![("content-type", "image/png")], + ); + let router = router_with(settings_with_a_signed_asset_route(), &[], &origin); + + let response = route(&router, get("/assets/logo.png", None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!(text(response), "image bytes"); + assert_eq!( + asked(&origin), + [( + "https://assets.example.com/assets/logo.png".to_owned(), + true + )], + "the route's origin should be asked, signed by the route's signer" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_outside_the_asset_routes_reaches_the_publisher_s_origin() { + let origin = Arc::new(StubHttpClient::new()); + origin.push_response_with_headers( + 200, + b"body { color: black; }".to_vec(), + vec![("content-type", "text/css")], + ); + let router = router_with(settings_with_a_signed_asset_route(), &[], &origin); + + let response = route(&router, get("/styles/site.css", None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + asked(&origin), + [( + "https://origin.test-publisher.example.com/styles/site.css".to_owned(), + false + )], + "the publisher's origin should be asked, unsigned" + ); +} + +/// Settings with an asset route under `/assets/` and a cache rule covering +/// it, which keeps an answer for a minute in a browser and an hour in a +/// shared cache. +fn settings_with_a_cached_asset_route() -> Settings { + settings_with( + r#" + [proxy] + + [[proxy.asset_routes]] + prefix = "/assets/" + + [proxy.asset_routes.source] + module = "origin" + + [proxy.asset_routes.source.origin] + url = "https://assets.example.com" + + [[cache.asset_rules]] + id = "assets" + enabled = true + path_prefix = "/assets/" + browser_ttl_seconds = 60 + edge_ttl_seconds = 3600 + "#, + ) +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn an_asset_route_s_cache_rule_is_written_after_the_finalize_headers() { + let origin = Arc::new(StubHttpClient::new()); + origin.push_response_with_headers( + 200, + b"image bytes".to_vec(), + vec![("content-type", "image/png")], + ); + let router = router_with(settings_with_a_cached_asset_route(), &[], &origin); + + let response = route(&router, get("/assets/logo.png", None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + header(&response, "cache-control"), + Some("public, max-age=60, s-maxage=3600"), + "the rule should be written with this adapter's `s-maxage` for shared caches" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn an_asset_route_whose_origin_fails_answers_502_and_is_not_stored() { + // No answer is queued, so the request to the route's origin fails. + let origin = Arc::new(StubHttpClient::new()); + let router = router_with(settings_with_a_cached_asset_route(), &[], &origin); + + let response = route(&router, get("/assets/logo.png", None)).await; + + assert_eq!(response.status(), StatusCode::BAD_GATEWAY); + assert_eq!( + header(&response, "cache-control"), + Some("no-store, private"), + "an asset route's error should not be stored by any cache" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_page_view_under_an_asset_route_gets_no_edge_cookie() { + let origin = origin_answering(1); + let mut settings = settings_with_a_cached_asset_route(); + settings.geo.module = Some("platform".to_owned()); + let router = router_for(settings, &[], services_for_a_reader(&origin)); + + let response = route(&router, page_view("/assets/guide.html", BROWSER, None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + edge_cookie(&response), + None, + "an asset response skips the Edge Cookie steps, so none is written" + ); + assert_eq!( + origin.recorded_request_uris(), + ["https://assets.example.com/assets/guide.html"], + "the route's origin should answer the page view" + ); +} + +#[test] +fn startup_warns_when_an_asset_route_enables_the_image_optimizer() { + let settings = settings_with( + r#" + [image_optimizer.profile_sets.photos] + default_profile = "default" + + [image_optimizer.profile_sets.photos.profiles] + default = "width=640" + + [proxy] + + [[proxy.asset_routes]] + prefix = "/images/" + + [proxy.asset_routes.source] + module = "origin" + + [proxy.asset_routes.source.origin] + url = "https://images.example.com" + + [proxy.asset_routes.image_optimizer] + region = "us_east" + profile_set = "photos" + "#, + ); + let origin = Arc::new(StubHttpClient::new()); + + let _router = router_with(settings, &[], &origin); + + assert!( + super::IMAGE_OPTIMIZER_UNAVAILABLE_LOGGED.load(std::sync::atomic::Ordering::Relaxed), + "startup should warn that this adapter cannot optimize the route's images" + ); +} + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn robots_txt_shows_the_device_module_none_of_the_crawler_s_cookies() { let origin = Arc::new(StubHttpClient::new()); diff --git a/crates/trusted-server-adapter-axum/src/middleware.rs b/crates/trusted-server-adapter-axum/src/middleware.rs index e9b5a1787..fd6828c7b 100644 --- a/crates/trusted-server-adapter-axum/src/middleware.rs +++ b/crates/trusted-server-adapter-axum/src/middleware.rs @@ -11,6 +11,7 @@ use trusted_server_core::http_util::{ ROUTE_NAMESPACE, sanitize_trusted_client_ip_headers, strip_deployment_headers, }; use trusted_server_core::integrations::RequestFilterEffects; +use trusted_server_core::proxy::AssetProxyCachePolicy; use trusted_server_core::response_privacy::{ TerminalPrivateResponse, enforce_private_no_store, enforce_set_cookie_cache_privacy, enforce_uncacheable_cache_privacy, @@ -66,8 +67,9 @@ impl Middleware for SanitizeRequestMiddleware { /// /// Registered directly inside [`SanitizeRequestMiddleware`], so that every /// outgoing response carries a consistent set of headers. Once the response -/// is finalized it finalizes the Edge Cookie and then applies the response -/// effects the request filters left. +/// is finalized it applies an asset response's cache policy, finalizes the +/// Edge Cookie and then applies the response effects the request filters +/// left. pub struct FinalizeResponseMiddleware { settings: Arc, } @@ -85,11 +87,15 @@ impl Middleware for FinalizeResponseMiddleware { async fn handle(&self, ctx: RequestContext, next: Next<'_>) -> Result { let path = ctx.request().uri().path().to_owned(); let mut response = next.run(ctx).await?; + let asset_cache_policy = response.extensions_mut().remove::(); let ec_state = response.extensions_mut().remove::(); let request_filter_effects = response.extensions_mut().remove::(); // Before finalizing, so a header finalizing writes is kept. strip_deployment_headers(&path, ROUTE_NAMESPACE, &mut response); apply_finalize_headers(&self.settings, &mut response); + if let Some(policy) = asset_cache_policy { + policy.apply_after_route_finalization(&mut response, crate::app::EDGE_CACHE_HEADER); + } if let Some(mut ec_state) = ec_state { finalize_edge_cookie(&self.settings, &mut ec_state, &mut response).await; } diff --git a/crates/trusted-server-adapter-axum/src/platform.rs b/crates/trusted-server-adapter-axum/src/platform.rs index 30dbe3997..3ee6bf7bc 100644 --- a/crates/trusted-server-adapter-axum/src/platform.rs +++ b/crates/trusted-server-adapter-axum/src/platform.rs @@ -438,10 +438,25 @@ impl AxumPlatformHttpClient { } } + /// Refuses a request carrying Image Optimizer metadata, on both send + /// paths, rather than send it untransformed, as the request contract asks + /// of a client that cannot honor it. Only an asset route that enables the + /// Image Optimizer sets it. + fn refuse_image_optimizer_metadata( + request: &PlatformHttpRequest, + ) -> Result<(), Report> { + if request.image_optimizer.is_some() { + return Err(Report::new(PlatformError::HttpClient) + .attach("the Axum dev server has no Image Optimizer")); + } + Ok(()) + } + async fn execute( &self, request: PlatformHttpRequest, ) -> Result> { + Self::refuse_image_optimizer_metadata(&request)?; let uri = request.request.uri().to_string(); let method = reqwest::Method::from_bytes(request.request.method().as_str().as_bytes()) .change_context(PlatformError::HttpClient)?; @@ -506,6 +521,7 @@ impl PlatformHttpClient for AxumPlatformHttpClient { &self, request: PlatformHttpRequest, ) -> Result> { + Self::refuse_image_optimizer_metadata(&request)?; let backend_name = request.backend_name.clone(); // Extract all Send-compatible parts before spawning. @@ -1008,6 +1024,42 @@ mod tests { ); } + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn a_request_with_image_optimizer_metadata_is_refused() { + use trusted_server_core::platform::{ + PlatformImageOptimizerOptions, PlatformImageOptimizerParams, + }; + + // A request to an origin that would answer, so only the refusal can + // fail the send. + async fn optimized_request() -> PlatformHttpRequest { + let url = serve_raw_response(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nok").await; + let request = edgezero_core::http::request_builder() + .uri(url) + .body(EdgeBody::empty()) + .expect("should build outbound request"); + PlatformHttpRequest::new(request, "test_backend").with_image_optimizer( + PlatformImageOptimizerOptions::new( + "us_west", + PlatformImageOptimizerParams::default(), + ), + ) + } + let client = AxumPlatformHttpClient::new(); + + let sent = client.send(optimized_request().await).await; + let sent_async = client.send_async(optimized_request().await).await; + + assert!( + sent.is_err(), + "send should refuse Image Optimizer metadata rather than drop it" + ); + assert!( + sent_async.is_err(), + "send_async should refuse Image Optimizer metadata rather than drop it" + ); + } + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn http_client_strips_hop_by_hop_response_headers() { let url = serve_raw_response( diff --git a/crates/trusted-server-adapter-cloudflare/src/app.rs b/crates/trusted-server-adapter-cloudflare/src/app.rs index 8290ac520..540533772 100644 --- a/crates/trusted-server-adapter-cloudflare/src/app.rs +++ b/crates/trusted-server-adapter-cloudflare/src/app.rs @@ -6,7 +6,7 @@ use std::sync::atomic::{AtomicBool, Ordering}; use edgezero_core::app::{Hooks, StoreMetadata, StoresMetadata}; use edgezero_core::context::RequestContext; use edgezero_core::error::EdgeError; -use edgezero_core::http::{HeaderMap, HeaderValue, Method, Request, Response, header}; +use edgezero_core::http::{HeaderMap, HeaderValue, Method, Request, Response, StatusCode, header}; use edgezero_core::router::RouterService; use error_stack::Report; use trusted_server_core::addressing::{RequestAddressing, addressed_router}; @@ -42,8 +42,8 @@ use trusted_server_core::module_context::{ModuleContext, ModuleRequest}; use trusted_server_core::platform::StoreName; use trusted_server_core::platform::{PlatformIdentity, RuntimeServices}; use trusted_server_core::proxy::{ - handle_first_party_click, handle_first_party_proxy, handle_first_party_proxy_rebuild, - handle_first_party_proxy_sign, + AssetProxyCachePolicy, handle_asset_proxy_request, handle_first_party_click, + handle_first_party_proxy, handle_first_party_proxy_rebuild, handle_first_party_proxy_sign, }; use trusted_server_core::publisher::{ AppContext, AuctionDispatch, PAGE_BIDS_LEGACY_PATH, PAGE_BIDS_PATH, PublisherResponse, @@ -234,7 +234,8 @@ fn build_state_with_services( pub const CDN_CACHE_CONTROL_HEADER: &str = "cloudflare-cdn-cache-control"; /// The header core writes a shared cache policy in on this adapter. -const EDGE_CACHE_HEADER: EdgeCacheHeader = EdgeCacheHeader::Platform(CDN_CACHE_CONTROL_HEADER); +pub(crate) const EDGE_CACHE_HEADER: EdgeCacheHeader = + EdgeCacheHeader::Platform(CDN_CACHE_CONTROL_HEADER); fn build_state_with_registrations_and_services( settings: Settings, @@ -291,6 +292,7 @@ fn build_state_with_registrations_and_services( &UNSENT_ANALYTICS_LOGGED, ); warn_once_when_the_identity_graph_is_unavailable(&settings, &IDENTITY_GRAPH_UNAVAILABLE_LOGGED); + warn_once_when_an_asset_route_enables_the_image_optimizer(&settings); let permission_policy = settings.required_permission_policy()?; Ok(Arc::new(AppState { @@ -336,6 +338,38 @@ fn warn_once_when_the_identity_graph_is_unavailable(settings: &Settings, logged: ); } +/// Set once startup has logged that an asset route enables the Image +/// Optimizer, which this adapter does not translate. The state is built for +/// every request, so the flag keeps the warning to one for each instance. +static IMAGE_OPTIMIZER_UNAVAILABLE_LOGGED: AtomicBool = AtomicBool::new(false); + +/// Logs once, at warning level, the prefix of each asset route that enables +/// the Image Optimizer, whose metadata this adapter does not translate. +/// +/// The HTTP client refuses a request carrying Image Optimizer metadata, so +/// such a route answers each request it would optimize with `502`. The route +/// is not refused at startup, because its SVG paths and debug requests are +/// still answered, as are the deployment's other routes and pages. +fn warn_once_when_an_asset_route_enables_the_image_optimizer(settings: &Settings) { + let prefixes: Vec = settings + .proxy + .asset_routes + .iter() + .filter(|route| route.image_optimizer.as_ref().is_some_and(|io| io.enabled)) + .map(|route| format!("`{}`", route.prefix)) + .collect(); + if prefixes.is_empty() || IMAGE_OPTIMIZER_UNAVAILABLE_LOGGED.swap(true, Ordering::Relaxed) { + return; + } + log::warn!( + "the Cloudflare adapter does not translate Image Optimizer metadata, so \ + each request these asset routes would optimize answers 502 Bad Gateway, \ + and their SVG paths and requests that set the profile set's debug \ + parameter are served unoptimized: {}", + prefixes.join(", ") + ); +} + /// The permission signal modules this adapter links, in the order they run /// when configuration names none. Global Privacy Control is first because it /// is a browser setting with no interface of its own, and the three that @@ -914,6 +948,15 @@ async fn dispatch_fallback( })) }) } else { + // A `GET` or `HEAD` under an asset route goes to the route's own + // source in place of the publisher's origin. Its response skips the + // Edge Cookie steps, so it carries no Edge Cookie state. + let matched_asset_route = matches!(method, Method::GET | Method::HEAD) + .then(|| state.settings.asset_route_index_for_path(&path)) + .flatten(); + if let Some(index) = matched_asset_route { + return dispatch_asset_fallback(state, services, req, index, effects).await; + } // A real browser's page navigation is given an Edge Cookie when it has // none. Another request may lack consent signals such as `Sec-GPC`. if ec.is_real_browser @@ -967,6 +1010,66 @@ async fn dispatch_fallback( ) } +/// Whether an asset response carries a body. +/// +/// `HEAD` responses and the bodiless statuses (204, 304) carry no body, so a +/// body stream is dropped for them. This adapter's HTTP client holds every +/// response whole and writes the length of the body it read as the +/// `Content-Length`, which for a `HEAD` is `0`. +fn asset_response_carries_body(method: &Method, status: StatusCode) -> bool { + *method != Method::HEAD + && status != StatusCode::NO_CONTENT + && status != StatusCode::NOT_MODIFIED +} + +/// Answers a request under the asset route at `index` from the route's own +/// source. +/// +/// The route's source module locates the asset, its signer signs the request +/// where the source's `auth` table names one, and core answers from a copy it +/// holds before it asks the origin. The response carries its cache policy, +/// which [`FinalizeResponseMiddleware`] applies after the finalize headers, +/// and the request filters' effects, and no Edge Cookie state, because an +/// asset response skips the Edge Cookie steps. +async fn dispatch_asset_fallback( + state: &AppState, + services: &RuntimeServices, + req: Request, + index: usize, + effects: RequestFilterEffects, +) -> Response { + let method = req.method().clone(); + let asset_route = &state.settings.proxy.asset_routes[index]; + let answer = match state.registry.route_source(&state.settings, index) { + Ok(source) => { + handle_asset_proxy_request(&state.settings, services, req, asset_route, source.as_ref()) + .await + } + Err(report) => Err(report), + }; + let response = match answer { + Ok(asset_response) => { + let cache_policy = asset_response.cache_policy(); + let (mut response, stream_body) = asset_response.into_response_and_body(); + if let Some(body) = stream_body + && asset_response_carries_body(&method, response.status()) + { + *response.body_mut() = body; + } + response.extensions_mut().insert(cache_policy); + response + } + Err(report) => { + let mut response = http_error(&report); + response + .extensions_mut() + .insert(AssetProxyCachePolicy::NoStorePrivate); + response + } + }; + attach_request_filter_effects(response, effects) +} + /// Whether the request is a crawler's `GET` or `HEAD` of the robots.txt that /// `[robots-txt]` makes. fn asks_for_robots_txt(settings: &Settings, method: &Method, path: &str) -> bool { @@ -1136,6 +1239,13 @@ impl Hooks for TrustedServerApp { "TrustedServer" } + /// The fetch entry point installs this adapter's logger itself + /// ([`crate::logging::init_logger`]) before it builds the application, so + /// no `EdgeZero` entry point installs another. + fn owns_logging() -> bool { + true + } + /// The startup error router, because this hook cannot read the settings. /// /// A Worker reads its settings from the config store with an awaited diff --git a/crates/trusted-server-adapter-cloudflare/src/app/request_cycle_tests.rs b/crates/trusted-server-adapter-cloudflare/src/app/request_cycle_tests.rs index 9c94d5787..ac78aceac 100644 --- a/crates/trusted-server-adapter-cloudflare/src/app/request_cycle_tests.rs +++ b/crates/trusted-server-adapter-cloudflare/src/app/request_cycle_tests.rs @@ -4,9 +4,11 @@ //! without making the response less private, the route's module call carries //! the device classification, an Edge Cookie context that cannot be read //! answers before the filters, robots.txt and the inspection endpoints are -//! answered without them, and the Edge Cookie is created and written for a -//! real browser's page view, set by the resolve route for a client-side -//! module and expired on a withdrawal. +//! answered without them, the Edge Cookie is created and written for a real +//! browser's page view, set by the resolve route for a client-side module and +//! expired on a withdrawal, and a request under an asset route is answered +//! from the route's own source, with the route's cache rule written after the +//! finalize headers, an error that no cache stores and no Edge Cookie. use std::net::{IpAddr, Ipv4Addr}; use std::sync::atomic::{AtomicBool, Ordering}; @@ -1070,6 +1072,221 @@ fn startup_warns_that_this_adapter_opens_no_identity_graph() { ); } +// --------------------------------------------------------------------------- +// Asset routes +// --------------------------------------------------------------------------- + +/// Settings with an asset route under `/assets/` whose source signs its +/// requests with the stock `s3_sigv4` signer. +fn settings_with_a_signed_asset_route() -> Settings { + settings_with( + r#" + [proxy] + + [[proxy.asset_routes]] + prefix = "/assets/" + + [proxy.asset_routes.source] + module = "origin" + + [proxy.asset_routes.source.origin] + url = "https://assets.example.com" + + [proxy.asset_routes.source.origin.auth] + type = "s3_sigv4" + region = "us-east-1" + "#, + ) +} + +/// What the origin was asked, being each request's address and whether it +/// carried an `s3_sigv4` signature. +fn asked(origin: &StubHttpClient) -> Vec<(String, bool)> { + origin + .recorded_request_uris() + .into_iter() + .zip(origin.recorded_request_headers()) + .map(|(uri, headers)| { + let signed = headers.iter().any(|(name, value)| { + name == "authorization" && value.starts_with("AWS4-HMAC-SHA256 ") + }); + (uri, signed) + }) + .collect() +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_under_an_asset_route_reaches_the_route_s_origin_signed() { + let origin = Arc::new(StubHttpClient::new()); + origin.push_response_with_headers( + 200, + b"image bytes".to_vec(), + vec![("content-type", "image/png")], + ); + let router = router_with(settings_with_a_signed_asset_route(), &[], &origin); + + let response = route(&router, get("/assets/logo.png", None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!(text(response), "image bytes"); + assert_eq!( + asked(&origin), + [( + "https://assets.example.com/assets/logo.png".to_owned(), + true + )], + "the route's origin should be asked, signed by the route's signer" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_outside_the_asset_routes_reaches_the_publisher_s_origin() { + let origin = Arc::new(StubHttpClient::new()); + origin.push_response_with_headers( + 200, + b"body { color: black; }".to_vec(), + vec![("content-type", "text/css")], + ); + let router = router_with(settings_with_a_signed_asset_route(), &[], &origin); + + let response = route(&router, get("/styles/site.css", None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + asked(&origin), + [( + "https://origin.test-publisher.example.com/styles/site.css".to_owned(), + false + )], + "the publisher's origin should be asked, unsigned" + ); +} + +/// Settings with an asset route under `/assets/` and a cache rule covering +/// it, which keeps an answer for a minute in a browser and an hour in a +/// shared cache. +fn settings_with_a_cached_asset_route() -> Settings { + settings_with( + r#" + [proxy] + + [[proxy.asset_routes]] + prefix = "/assets/" + + [proxy.asset_routes.source] + module = "origin" + + [proxy.asset_routes.source.origin] + url = "https://assets.example.com" + + [[cache.asset_rules]] + id = "assets" + enabled = true + path_prefix = "/assets/" + browser_ttl_seconds = 60 + edge_ttl_seconds = 3600 + "#, + ) +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn an_asset_route_s_cache_rule_is_written_after_the_finalize_headers() { + let origin = Arc::new(StubHttpClient::new()); + origin.push_response_with_headers( + 200, + b"image bytes".to_vec(), + vec![("content-type", "image/png")], + ); + let router = router_with(settings_with_a_cached_asset_route(), &[], &origin); + + let response = route(&router, get("/assets/logo.png", None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + header(&response, "cache-control"), + Some("public, max-age=60") + ); + assert_eq!( + header(&response, "cloudflare-cdn-cache-control"), + Some("max-age=3600"), + "the rule should be written in this adapter's own shared cache header" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn an_asset_route_whose_origin_fails_answers_502_and_is_not_stored() { + // No answer is queued, so the request to the route's origin fails. + let origin = Arc::new(StubHttpClient::new()); + let router = router_with(settings_with_a_cached_asset_route(), &[], &origin); + + let response = route(&router, get("/assets/logo.png", None)).await; + + assert_eq!(response.status(), StatusCode::BAD_GATEWAY); + assert_eq!( + header(&response, "cache-control"), + Some("no-store, private"), + "an asset route's error should not be stored by any cache" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_page_view_under_an_asset_route_gets_no_edge_cookie() { + let origin = origin_answering(1); + let mut settings = settings_with_a_cached_asset_route(); + settings.geo.module = Some("platform".to_owned()); + let router = router_for(settings, &[], services_for_a_reader(&origin)); + + let response = route(&router, page_view("/assets/guide.html", BROWSER, None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + edge_cookie(&response), + None, + "an asset response skips the Edge Cookie steps, so none is written" + ); + assert_eq!( + origin.recorded_request_uris(), + ["https://assets.example.com/assets/guide.html"], + "the route's origin should answer the page view" + ); +} + +#[test] +fn startup_warns_when_an_asset_route_enables_the_image_optimizer() { + let settings = settings_with( + r#" + [image_optimizer.profile_sets.photos] + default_profile = "default" + + [image_optimizer.profile_sets.photos.profiles] + default = "width=640" + + [proxy] + + [[proxy.asset_routes]] + prefix = "/images/" + + [proxy.asset_routes.source] + module = "origin" + + [proxy.asset_routes.source.origin] + url = "https://images.example.com" + + [proxy.asset_routes.image_optimizer] + region = "us_east" + profile_set = "photos" + "#, + ); + let origin = Arc::new(StubHttpClient::new()); + + let _router = router_with(settings, &[], &origin); + + assert!( + super::IMAGE_OPTIMIZER_UNAVAILABLE_LOGGED.load(std::sync::atomic::Ordering::Relaxed), + "startup should warn that this adapter cannot optimize the route's images" + ); +} + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn robots_txt_shows_the_device_module_none_of_the_crawler_s_cookies() { let origin = Arc::new(StubHttpClient::new()); diff --git a/crates/trusted-server-adapter-cloudflare/src/lib.rs b/crates/trusted-server-adapter-cloudflare/src/lib.rs index b09efcbd8..2c5b2b1ba 100644 --- a/crates/trusted-server-adapter-cloudflare/src/lib.rs +++ b/crates/trusted-server-adapter-cloudflare/src/lib.rs @@ -9,6 +9,7 @@ compile_error!( ); pub mod app; +pub mod logging; pub mod middleware; pub mod platform; @@ -17,7 +18,8 @@ use worker::{Context, Env, Request, Response, Result, event}; #[cfg(target_arch = "wasm32")] #[event(fetch)] -/// Dispatches an incoming Cloudflare Worker fetch event. +/// Installs the Worker's logger and dispatches an incoming Cloudflare Worker +/// fetch event. /// /// The application is built for the request from the settings in the /// Worker's config store, then served by `EdgeZero`. @@ -27,6 +29,7 @@ use worker::{Context, Env, Request, Response, Result, event}; /// Returns a Workers runtime error when the fallback error response cannot be /// constructed. pub async fn main(req: Request, env: Env, ctx: Context) -> Result { + logging::init_logger(); let app = app::build_app(&env).await; match edgezero_adapter_cloudflare::dispatch_app::(&app, req, env, ctx) .await diff --git a/crates/trusted-server-adapter-cloudflare/src/logging.rs b/crates/trusted-server-adapter-cloudflare/src/logging.rs new file mode 100644 index 000000000..6926d0210 --- /dev/null +++ b/crates/trusted-server-adapter-cloudflare/src/logging.rs @@ -0,0 +1,139 @@ +//! The Worker's logger. It writes each record through the console method of +//! the record's level, so the Workers runtime keeps the line at that level. + +use log::{Level, LevelFilter, Log, Metadata, Record}; + +/// The most detailed level the Worker writes. Debug and trace lines stay out, +/// because some of them carry auction payloads and consent strings. +pub const MAX_LEVEL: LevelFilter = LevelFilter::Info; + +/// The console method a record is written with. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum ConsoleMethod { + Error, + Warn, + Log, + Debug, +} + +/// `console.error` and `console.warn` for those levels, `console.log` for +/// `info`, and `console.debug` for `debug` and `trace`. +const fn console_method(level: Level) -> ConsoleMethod { + match level { + Level::Error => ConsoleMethod::Error, + Level::Warn => ConsoleMethod::Warn, + Level::Info => ConsoleMethod::Log, + Level::Debug | Level::Trace => ConsoleMethod::Debug, + } +} + +/// The line written for `record`, its level and target before its message. +fn line(record: &Record<'_>) -> String { + format!("{} [{}] {}", record.level(), record.target(), record.args()) +} + +/// Writes each record at [`MAX_LEVEL`] or above to the console. +struct ConsoleLogger; + +impl Log for ConsoleLogger { + fn enabled(&self, metadata: &Metadata<'_>) -> bool { + metadata.level() <= MAX_LEVEL + } + + fn log(&self, record: &Record<'_>) { + if self.enabled(record.metadata()) { + write(console_method(record.level()), &line(record)); + } + } + + fn flush(&self) {} +} + +#[cfg(target_arch = "wasm32")] +fn write(method: ConsoleMethod, line: &str) { + match method { + ConsoleMethod::Error => worker::console_error!("{line}"), + ConsoleMethod::Warn => worker::console_warn!("{line}"), + ConsoleMethod::Log => worker::console_log!("{line}"), + ConsoleMethod::Debug => worker::console_debug!("{line}"), + } +} + +/// Off the Workers runtime, which only the native tests are, a line goes +/// nowhere. +#[cfg(not(target_arch = "wasm32"))] +fn write(_method: ConsoleMethod, _line: &str) {} + +static LOGGER: ConsoleLogger = ConsoleLogger; + +/// Installs the Worker's logger at [`MAX_LEVEL`], unless a logger is +/// installed already. The entry point calls this for every request, so only +/// the first call in an instance installs the logger and a later call changes +/// nothing. +pub fn init_logger() { + if log::set_logger(&LOGGER).is_ok() { + log::set_max_level(MAX_LEVEL); + } +} + +#[cfg(test)] +mod tests { + use log::{Level, Log as _, Metadata, Record}; + + use super::{ConsoleLogger, ConsoleMethod, console_method, line}; + + #[test] + fn each_level_is_written_with_the_console_method_for_it() { + let methods = [ + Level::Error, + Level::Warn, + Level::Info, + Level::Debug, + Level::Trace, + ] + .map(console_method); + + assert_eq!( + methods, + [ + ConsoleMethod::Error, + ConsoleMethod::Warn, + ConsoleMethod::Log, + ConsoleMethod::Debug, + ConsoleMethod::Debug, + ], + "should write each level with the console method for it" + ); + } + + #[test] + fn a_line_gives_the_level_and_target_before_the_message() { + let written = line( + &Record::builder() + .level(Level::Warn) + .target("trusted_server_adapter_cloudflare::app") + .args(format_args!("a route enables the Image Optimizer")) + .build(), + ); + + assert_eq!( + written, + "WARN [trusted_server_adapter_cloudflare::app] a route enables the Image Optimizer", + "should give the level and target before the message" + ); + } + + #[test] + fn info_and_above_are_written_and_debug_and_trace_are_not() { + let writes = |level| ConsoleLogger.enabled(&Metadata::builder().level(level).build()); + + assert!( + writes(Level::Error) && writes(Level::Warn) && writes(Level::Info), + "should write info, warnings and errors" + ); + assert!( + !writes(Level::Debug) && !writes(Level::Trace), + "should leave debug and trace lines out" + ); + } +} diff --git a/crates/trusted-server-adapter-cloudflare/src/middleware.rs b/crates/trusted-server-adapter-cloudflare/src/middleware.rs index c254009ce..0b5152df5 100644 --- a/crates/trusted-server-adapter-cloudflare/src/middleware.rs +++ b/crates/trusted-server-adapter-cloudflare/src/middleware.rs @@ -11,6 +11,7 @@ use trusted_server_core::http_util::{ ROUTE_NAMESPACE, sanitize_trusted_client_ip_headers, strip_deployment_headers, }; use trusted_server_core::integrations::RequestFilterEffects; +use trusted_server_core::proxy::AssetProxyCachePolicy; use trusted_server_core::response_privacy::{ TerminalPrivateResponse, enforce_private_no_store, enforce_set_cookie_cache_privacy, enforce_uncacheable_cache_privacy, @@ -63,8 +64,9 @@ impl Middleware for SanitizeRequestMiddleware { /// /// Registered directly inside [`SanitizeRequestMiddleware`], so that every /// outgoing response carries a consistent set of headers. Once the response -/// is finalized it finalizes the Edge Cookie and then applies the response -/// effects the request filters left. +/// is finalized it applies an asset response's cache policy, finalizes the +/// Edge Cookie and then applies the response effects the request filters +/// left. pub struct FinalizeResponseMiddleware { settings: Arc, } @@ -82,11 +84,15 @@ impl Middleware for FinalizeResponseMiddleware { async fn handle(&self, ctx: RequestContext, next: Next<'_>) -> Result { let path = ctx.request().uri().path().to_owned(); let mut response = next.run(ctx).await?; + let asset_cache_policy = response.extensions_mut().remove::(); let ec_state = response.extensions_mut().remove::(); let request_filter_effects = response.extensions_mut().remove::(); // Before finalizing, so a header finalizing writes is kept. strip_deployment_headers(&path, ROUTE_NAMESPACE, &mut response); apply_finalize_headers(&self.settings, &mut response); + if let Some(policy) = asset_cache_policy { + policy.apply_after_route_finalization(&mut response, crate::app::EDGE_CACHE_HEADER); + } if let Some(mut ec_state) = ec_state { finalize_edge_cookie(&self.settings, &mut ec_state, &mut response).await; } diff --git a/crates/trusted-server-adapter-cloudflare/src/platform.rs b/crates/trusted-server-adapter-cloudflare/src/platform.rs index 94332e9be..f95ea01b3 100644 --- a/crates/trusted-server-adapter-cloudflare/src/platform.rs +++ b/crates/trusted-server-adapter-cloudflare/src/platform.rs @@ -330,14 +330,14 @@ impl CloudflareHttpClient { ) -> Result> { use worker::{CacheMode, Fetch, Headers, Method, Request}; - // The Cloudflare fetch path cannot honor Fastly-style Image Optimizer - // metadata, and it always buffers the response body (see below). The + // The Cloudflare fetch path cannot honor Image Optimizer metadata, and + // it always buffers the response body (see below). The // `PlatformHttpRequest` contract requires adapters that cannot honor // these to reject rather than silently drop the behavior, so surface a // typed error instead of returning an untransformed or fully-buffered - // response. These fields are only set by asset routes, which are not - // routed to the Cloudflare adapter today; the guard keeps the contract - // honest if that changes. + // response. Only an asset route sets these fields. A route that + // enables the Image Optimizer answers with this error, and none asks + // for a stream, because this client reports that it cannot stream. if request.image_optimizer.is_some() { return Err(Report::new(PlatformError::HttpClient) .attach("Image Optimizer is not supported on the Cloudflare Workers runtime")); diff --git a/crates/trusted-server-adapter-cloudflare/tests/logging.rs b/crates/trusted-server-adapter-cloudflare/tests/logging.rs new file mode 100644 index 000000000..d7f1024de --- /dev/null +++ b/crates/trusted-server-adapter-cloudflare/tests/logging.rs @@ -0,0 +1,52 @@ +//! The Worker's logger, installed once for an instance at `info` and above. +//! +//! A logger is global to a process, so these tests have a binary of their +//! own, in which nothing else installs one. + +use edgezero_core::app::Hooks as _; +use log::{Level, LevelFilter, Metadata}; +use trusted_server_adapter_cloudflare::app::TrustedServerApp; +use trusted_server_adapter_cloudflare::logging::init_logger; + +/// Whether the installed logger writes a record at `level`. +fn writes(level: Level) -> bool { + log::logger().enabled(&Metadata::builder().level(level).build()) +} + +/// The entry point installs the logger for every request, so a second call +/// keeps the first logger rather than failing. +#[test] +fn the_logger_is_installed_once_at_info_and_above() { + assert_eq!( + log::max_level(), + LevelFilter::Off, + "no logger should be installed before the first call" + ); + + init_logger(); + init_logger(); + + assert_eq!( + log::max_level(), + LevelFilter::Info, + "should write info and above" + ); + assert!( + writes(Level::Error) && writes(Level::Warn) && writes(Level::Info), + "the installed logger should write info, warnings and errors" + ); + assert!( + !writes(Level::Debug) && !writes(Level::Trace), + "the installed logger should leave debug and trace lines out" + ); +} + +/// The adapter installs its own logger, so `EdgeZero`'s `run_app` installs +/// none. +#[test] +fn the_app_owns_its_logging() { + assert!( + TrustedServerApp::owns_logging(), + "should tell EdgeZero that the adapter installs its own logger" + ); +} diff --git a/crates/trusted-server-adapter-spin/src/app.rs b/crates/trusted-server-adapter-spin/src/app.rs index 13931ae28..62e40150d 100644 --- a/crates/trusted-server-adapter-spin/src/app.rs +++ b/crates/trusted-server-adapter-spin/src/app.rs @@ -48,8 +48,8 @@ use trusted_server_core::module_context::{ModuleContext, ModuleRequest}; use trusted_server_core::platform::{PlatformConfigStore, StoreName}; use trusted_server_core::platform::{PlatformIdentity, RuntimeServices}; use trusted_server_core::proxy::{ - handle_first_party_click, handle_first_party_proxy, handle_first_party_proxy_rebuild, - handle_first_party_proxy_sign, + AssetProxyCachePolicy, handle_asset_proxy_request, handle_first_party_click, + handle_first_party_proxy, handle_first_party_proxy_rebuild, handle_first_party_proxy_sign, }; use trusted_server_core::publisher::{ AppContext, AuctionDispatch, PAGE_BIDS_LEGACY_PATH, PAGE_BIDS_PATH, PublisherResponse, @@ -186,7 +186,7 @@ fn build_state_with_settings( } /// The header core writes a shared cache policy in on this adapter. -const EDGE_CACHE_HEADER: EdgeCacheHeader = EdgeCacheHeader::SMaxageFallback; +pub(crate) const EDGE_CACHE_HEADER: EdgeCacheHeader = EdgeCacheHeader::SMaxageFallback; /// Build the application state from explicit settings, composing the modules /// this adapter links with the externally supplied builders in @@ -279,6 +279,7 @@ fn build_state_with_registrations_and_services( &UNSENT_ANALYTICS_LOGGED, ); warn_once_when_the_identity_graph_is_unavailable(&settings, &IDENTITY_GRAPH_UNAVAILABLE_LOGGED); + warn_once_when_an_asset_route_enables_the_image_optimizer(&settings); let permission_policy = settings.required_permission_policy()?; Ok(Arc::new(AppState { @@ -343,6 +344,38 @@ fn warn_once_when_the_identity_graph_is_unavailable(settings: &Settings, logged: ); } +/// Set once startup has logged that an asset route enables the Image +/// Optimizer, which this adapter does not have. The state is built for every +/// request, so the flag keeps the warning to one for each instance. +static IMAGE_OPTIMIZER_UNAVAILABLE_LOGGED: AtomicBool = AtomicBool::new(false); + +/// Logs once, at warning level, the prefix of each asset route that enables +/// the Image Optimizer, which this adapter does not have. +/// +/// The HTTP client refuses a request carrying Image Optimizer metadata, so +/// such a route answers each request it would optimize with `502`. The route +/// is not refused at startup, because its SVG paths and debug requests are +/// still answered, as are the deployment's other routes and pages. +fn warn_once_when_an_asset_route_enables_the_image_optimizer(settings: &Settings) { + let prefixes: Vec = settings + .proxy + .asset_routes + .iter() + .filter(|route| route.image_optimizer.as_ref().is_some_and(|io| io.enabled)) + .map(|route| format!("`{}`", route.prefix)) + .collect(); + if prefixes.is_empty() || IMAGE_OPTIMIZER_UNAVAILABLE_LOGGED.swap(true, Ordering::Relaxed) { + return; + } + log::warn!( + "the Spin adapter has no Image Optimizer, so each request these asset \ + routes would optimize answers 502 Bad Gateway, and their SVG paths and \ + requests that set the profile set's debug parameter are served \ + unoptimized: {}", + prefixes.join(", ") + ); +} + /// The permission signal modules this adapter links, in the order they run /// when configuration names none. Global Privacy Control is first because it /// is a browser setting with no interface of its own, and the three that @@ -1153,6 +1186,15 @@ async fn dispatch_fallback( })) }) } else { + // A `GET` or `HEAD` under an asset route goes to the route's own + // source in place of the publisher's origin. Its response skips the + // Edge Cookie steps, so it carries no Edge Cookie state. + let matched_asset_route = matches!(method, Method::GET | Method::HEAD) + .then(|| state.settings.asset_route_index_for_path(&path)) + .flatten(); + if let Some(index) = matched_asset_route { + return dispatch_asset_fallback(state, services, req, index, effects).await; + } // A real browser's page navigation is given an Edge Cookie when it has // none. Another request may lack consent signals such as `Sec-GPC`. if ec.is_real_browser @@ -1206,6 +1248,64 @@ async fn dispatch_fallback( ) } +/// Whether an asset response carries a body. +/// +/// `HEAD` responses and the bodiless statuses (204, 304) keep the origin's +/// `Content-Length` and carry no body, so a body stream is dropped for them. +fn asset_response_carries_body(method: &Method, status: StatusCode) -> bool { + *method != Method::HEAD + && status != StatusCode::NO_CONTENT + && status != StatusCode::NOT_MODIFIED +} + +/// Answers a request under the asset route at `index` from the route's own +/// source. +/// +/// The route's source module locates the asset, its signer signs the request +/// where the source's `auth` table names one, and core answers from a copy it +/// holds before it asks the origin. The response carries its cache policy, +/// which [`FinalizeResponseMiddleware`] applies after the finalize headers, +/// and the request filters' effects, and no Edge Cookie state, because an +/// asset response skips the Edge Cookie steps. +async fn dispatch_asset_fallback( + state: &AppState, + services: &RuntimeServices, + req: Request, + index: usize, + effects: RequestFilterEffects, +) -> Response { + let method = req.method().clone(); + let asset_route = &state.settings.proxy.asset_routes[index]; + let answer = match state.registry.route_source(&state.settings, index) { + Ok(source) => { + handle_asset_proxy_request(&state.settings, services, req, asset_route, source.as_ref()) + .await + } + Err(report) => Err(report), + }; + let response = match answer { + Ok(asset_response) => { + let cache_policy = asset_response.cache_policy(); + let (mut response, stream_body) = asset_response.into_response_and_body(); + if let Some(body) = stream_body + && asset_response_carries_body(&method, response.status()) + { + *response.body_mut() = body; + } + response.extensions_mut().insert(cache_policy); + response + } + Err(report) => { + let mut response = http_error(&report); + response + .extensions_mut() + .insert(AssetProxyCachePolicy::NoStorePrivate); + response + } + }; + attach_request_filter_effects(response, effects) +} + /// Whether the request is a crawler's `GET` or `HEAD` of the robots.txt that /// `[robots-txt]` makes. fn asks_for_robots_txt(settings: &Settings, method: &Method, path: &str) -> bool { @@ -1322,6 +1422,12 @@ impl Hooks for TrustedServerApp { "TrustedServer" } + /// The entry point installs this adapter's logger before `run_app` + /// ([`crate::logging::init_logger`]), so `run_app` installs none. + fn owns_logging() -> bool { + true + } + fn routes() -> RouterService { let state = match build_state() { Ok(s) => s, diff --git a/crates/trusted-server-adapter-spin/src/app/request_cycle_tests.rs b/crates/trusted-server-adapter-spin/src/app/request_cycle_tests.rs index 9c94d5787..266b4495e 100644 --- a/crates/trusted-server-adapter-spin/src/app/request_cycle_tests.rs +++ b/crates/trusted-server-adapter-spin/src/app/request_cycle_tests.rs @@ -4,9 +4,11 @@ //! without making the response less private, the route's module call carries //! the device classification, an Edge Cookie context that cannot be read //! answers before the filters, robots.txt and the inspection endpoints are -//! answered without them, and the Edge Cookie is created and written for a -//! real browser's page view, set by the resolve route for a client-side -//! module and expired on a withdrawal. +//! answered without them, the Edge Cookie is created and written for a real +//! browser's page view, set by the resolve route for a client-side module and +//! expired on a withdrawal, and a request under an asset route is answered +//! from the route's own source, with the route's cache rule written after the +//! finalize headers, an error that no cache stores and no Edge Cookie. use std::net::{IpAddr, Ipv4Addr}; use std::sync::atomic::{AtomicBool, Ordering}; @@ -38,6 +40,7 @@ use trusted_server_core::platform::{ use trusted_server_core::publisher::{PAGE_BIDS_LEGACY_PATH, PAGE_BIDS_PATH}; use trusted_server_core::response_privacy::enforce_terminal_private_cache_privacy; use trusted_server_core::settings::Settings; +use trusted_server_core::source::OriginSettings; use trusted_server_core::test_support::addressing::FirstSegment; use super::{build_router, build_state_with_registrations_and_services}; @@ -1070,6 +1073,312 @@ fn startup_warns_that_this_adapter_opens_no_identity_graph() { ); } +// --------------------------------------------------------------------------- +// Asset routes +// --------------------------------------------------------------------------- + +/// Settings with an asset route under `/assets/` whose source signs its +/// requests with the stock `s3_sigv4` signer. +fn settings_with_a_signed_asset_route() -> Settings { + settings_with( + r#" + [proxy] + + [[proxy.asset_routes]] + prefix = "/assets/" + + [proxy.asset_routes.source] + module = "origin" + + [proxy.asset_routes.source.origin] + url = "https://assets.example.com" + + [proxy.asset_routes.source.origin.auth] + type = "s3_sigv4" + region = "us-east-1" + "#, + ) +} + +/// What the origin was asked, being each request's address and whether it +/// carried an `s3_sigv4` signature. +fn asked(origin: &StubHttpClient) -> Vec<(String, bool)> { + origin + .recorded_request_uris() + .into_iter() + .zip(origin.recorded_request_headers()) + .map(|(uri, headers)| { + let signed = headers.iter().any(|(name, value)| { + name == "authorization" && value.starts_with("AWS4-HMAC-SHA256 ") + }); + (uri, signed) + }) + .collect() +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_under_an_asset_route_reaches_the_route_s_origin_signed() { + let origin = Arc::new(StubHttpClient::new()); + origin.push_response_with_headers( + 200, + b"image bytes".to_vec(), + vec![("content-type", "image/png")], + ); + let router = router_with(settings_with_a_signed_asset_route(), &[], &origin); + + let response = route(&router, get("/assets/logo.png", None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!(text(response), "image bytes"); + assert_eq!( + asked(&origin), + [( + "https://assets.example.com/assets/logo.png".to_owned(), + true + )], + "the route's origin should be asked, signed by the route's signer" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_request_outside_the_asset_routes_reaches_the_publisher_s_origin() { + let origin = Arc::new(StubHttpClient::new()); + origin.push_response_with_headers( + 200, + b"body { color: black; }".to_vec(), + vec![("content-type", "text/css")], + ); + let router = router_with(settings_with_a_signed_asset_route(), &[], &origin); + + let response = route(&router, get("/styles/site.css", None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + asked(&origin), + [( + "https://origin.test-publisher.example.com/styles/site.css".to_owned(), + false + )], + "the publisher's origin should be asked, unsigned" + ); +} + +/// Settings with an asset route under `/assets/` and a cache rule covering +/// it, which keeps an answer for a minute in a browser and an hour in a +/// shared cache. +fn settings_with_a_cached_asset_route() -> Settings { + settings_with( + r#" + [proxy] + + [[proxy.asset_routes]] + prefix = "/assets/" + + [proxy.asset_routes.source] + module = "origin" + + [proxy.asset_routes.source.origin] + url = "https://assets.example.com" + + [[cache.asset_rules]] + id = "assets" + enabled = true + path_prefix = "/assets/" + browser_ttl_seconds = 60 + edge_ttl_seconds = 3600 + "#, + ) +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn an_asset_route_s_cache_rule_is_written_after_the_finalize_headers() { + let origin = Arc::new(StubHttpClient::new()); + origin.push_response_with_headers( + 200, + b"image bytes".to_vec(), + vec![("content-type", "image/png")], + ); + let router = router_with(settings_with_a_cached_asset_route(), &[], &origin); + + let response = route(&router, get("/assets/logo.png", None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + header(&response, "cache-control"), + Some("public, max-age=60, s-maxage=3600"), + "the rule should be written with this adapter's `s-maxage` for shared caches" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn an_asset_route_whose_origin_fails_answers_502_and_is_not_stored() { + // No answer is queued, so the request to the route's origin fails. + let origin = Arc::new(StubHttpClient::new()); + let router = router_with(settings_with_a_cached_asset_route(), &[], &origin); + + let response = route(&router, get("/assets/logo.png", None)).await; + + assert_eq!(response.status(), StatusCode::BAD_GATEWAY); + assert_eq!( + header(&response, "cache-control"), + Some("no-store, private"), + "an asset route's error should not be stored by any cache" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_page_view_under_an_asset_route_gets_no_edge_cookie() { + let origin = origin_answering(1); + let mut settings = settings_with_a_cached_asset_route(); + settings.geo.module = Some("platform".to_owned()); + let router = router_for(settings, &[], services_for_a_reader(&origin)); + + let response = route(&router, page_view("/assets/guide.html", BROWSER, None)).await; + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + edge_cookie(&response), + None, + "an asset response skips the Edge Cookie steps, so none is written" + ); + assert_eq!( + origin.recorded_request_uris(), + ["https://assets.example.com/assets/guide.html"], + "the route's origin should answer the page view" + ); +} + +#[test] +fn startup_warns_when_an_asset_route_enables_the_image_optimizer() { + let settings = settings_with( + r#" + [image_optimizer.profile_sets.photos] + default_profile = "default" + + [image_optimizer.profile_sets.photos.profiles] + default = "width=640" + + [proxy] + + [[proxy.asset_routes]] + prefix = "/images/" + + [proxy.asset_routes.source] + module = "origin" + + [proxy.asset_routes.source.origin] + url = "https://images.example.com" + + [proxy.asset_routes.image_optimizer] + region = "us_east" + profile_set = "photos" + "#, + ); + let origin = Arc::new(StubHttpClient::new()); + + let _router = router_with(settings, &[], &origin); + + assert!( + super::IMAGE_OPTIMIZER_UNAVAILABLE_LOGGED.load(std::sync::atomic::Ordering::Relaxed), + "startup should warn that this adapter cannot optimize the route's images" + ); +} + +// --------------------------------------------------------------------------- +// A source's `Host` +// --------------------------------------------------------------------------- + +/// The services of a reader whose requests are registered through the +/// backend this adapter's own services use, every request sent to `origin`. +fn services_with_this_adapter_s_backend(origin: &Arc) -> RuntimeServices { + RuntimeServices::builder() + .config_store(Arc::new(NoopConfigStore)) + .secret_store(Arc::new(NoopSecretStore)) + .kv_store(Arc::new(UnavailableKvStore)) + .backend(Arc::new(crate::platform::NoopBackend)) + .http_client(Arc::clone(origin) as Arc) + .geo(Arc::new(PlatformGeoModule::new( + "platform", + Arc::new(UsGeo), + ))) + .client_info(ClientInfo::default()) + .permission_policy(trusted_server_core::permissions::PermissionPolicy::sample()) + .build() +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn an_asset_route_whose_source_names_another_host_is_refused_before_its_origin_is_asked() { + let origin = Arc::new(StubHttpClient::new()); + origin.push_response_with_headers( + 200, + b"image bytes".to_vec(), + vec![("content-type", "image/png")], + ); + let settings = settings_with( + r#" + [proxy] + + [[proxy.asset_routes]] + prefix = "/assets/" + + [proxy.asset_routes.source] + module = "origin" + + [proxy.asset_routes.source.origin] + url = "https://assets.example.com" + host_header = "media.example.com" + + [[proxy.asset_routes]] + prefix = "/media/" + + [proxy.asset_routes.source] + module = "origin" + + [proxy.asset_routes.source.origin] + url = "https://media.example.com" + "#, + ); + let router = router_for(settings, &[], services_with_this_adapter_s_backend(&origin)); + + let refused = route(&router, get("/assets/logo.png", None)).await; + let served = route(&router, get("/media/logo.png", None)).await; + + assert_eq!( + refused.status(), + StatusCode::BAD_GATEWAY, + "Spin cannot send the source's Host, so the route's request should be refused" + ); + assert_eq!(header(&refused, "cache-control"), Some("no-store, private")); + assert_eq!(served.status(), StatusCode::OK); + assert_eq!( + origin.recorded_request_uris(), + ["https://media.example.com/media/logo.png"], + "only the route that sends its address's own Host should reach its origin" + ); +} + +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn a_publisher_source_that_names_another_host_is_refused_before_its_origin_is_asked() { + let origin = origin_answering(1); + let mut settings = settings_with(""); + let mut source = OriginSettings::new("https://origin.test-publisher.example.com"); + source.host_header = Some("www.test-publisher.example.com".to_owned()); + settings.source = source.selection(); + let router = router_for(settings, &[], services_with_this_adapter_s_backend(&origin)); + + let response = route(&router, get("/article", None)).await; + + assert_eq!( + response.status(), + StatusCode::BAD_GATEWAY, + "Spin cannot send the source's Host, so the page request should be refused" + ); + assert!( + origin.recorded_request_uris().is_empty(), + "the publisher's origin should not be asked with the wrong Host" + ); +} + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn robots_txt_shows_the_device_module_none_of_the_crawler_s_cookies() { let origin = Arc::new(StubHttpClient::new()); diff --git a/crates/trusted-server-adapter-spin/src/lib.rs b/crates/trusted-server-adapter-spin/src/lib.rs index f47877ff2..502940a01 100644 --- a/crates/trusted-server-adapter-spin/src/lib.rs +++ b/crates/trusted-server-adapter-spin/src/lib.rs @@ -1,6 +1,7 @@ //! Fermyon Spin adapter for Trusted Server. pub mod app; +pub mod logging; pub mod middleware; pub mod platform; @@ -13,5 +14,6 @@ use spin_sdk::http_service; #[http_service] // FORCED: edgezero_adapter_spin::run_app returns anyhow::Result — EdgeZero SDK constraint, not a project choice. async fn handle(req: Request) -> anyhow::Result { + logging::init_logger(); edgezero_adapter_spin::run_app::(req).await } diff --git a/crates/trusted-server-adapter-spin/src/logging.rs b/crates/trusted-server-adapter-spin/src/logging.rs new file mode 100644 index 000000000..2cef41014 --- /dev/null +++ b/crates/trusted-server-adapter-spin/src/logging.rs @@ -0,0 +1,97 @@ +//! The component's logger. It writes each record as one line to standard +//! error, which Spin keeps as the component's log. + +use std::io::Write; + +use log::{LevelFilter, Log, Metadata, Record}; + +/// The most detailed level the component writes. Debug and trace lines stay +/// out, because some of them carry auction payloads and consent strings. +pub const MAX_LEVEL: LevelFilter = LevelFilter::Info; + +/// Writes `record` to `out` as one line, its level and target before its +/// message. The line is formatted first, so it reaches `out` in one write. +fn write_line(out: &mut impl Write, record: &Record<'_>) -> std::io::Result<()> { + let line = format!( + "{} [{}] {}\n", + record.level(), + record.target(), + record.args() + ); + out.write_all(line.as_bytes()) +} + +/// Writes each record at [`MAX_LEVEL`] or above to standard error. +struct StderrLogger; + +impl Log for StderrLogger { + fn enabled(&self, metadata: &Metadata<'_>) -> bool { + metadata.level() <= MAX_LEVEL + } + + fn log(&self, record: &Record<'_>) { + if self.enabled(record.metadata()) { + // A line standard error cannot take is dropped, because nothing + // is left to report that to. + let _ = write_line(&mut std::io::stderr().lock(), record); + } + } + + fn flush(&self) { + let _ = std::io::stderr().flush(); + } +} + +static LOGGER: StderrLogger = StderrLogger; + +/// Installs the component's logger at [`MAX_LEVEL`], unless a logger is +/// installed already. The entry point calls this for every request, so only +/// the first call in an instance installs the logger and a later call changes +/// nothing. +pub fn init_logger() { + if log::set_logger(&LOGGER).is_ok() { + log::set_max_level(MAX_LEVEL); + } +} + +#[cfg(test)] +mod tests { + use log::{Level, Log as _, Metadata, Record}; + + use super::{StderrLogger, write_line}; + + #[test] + fn a_record_is_written_as_one_line_with_its_level_and_target() { + let mut out = Vec::new(); + + write_line( + &mut out, + &Record::builder() + .level(Level::Warn) + .target("trusted_server_adapter_spin::app") + .args(format_args!("a source names another host")) + .build(), + ) + .expect("should write to memory"); + + assert_eq!( + String::from_utf8(out).expect("should write text"), + "WARN [trusted_server_adapter_spin::app] a source names another host\n", + "should give the level and target before the message and end the line" + ); + } + + #[test] + fn info_and_above_are_written_and_debug_and_trace_are_not() { + let writes = |level| StderrLogger.enabled(&Metadata::builder().level(level).build()); + + assert!( + writes(Level::Error) && writes(Level::Warn) && writes(Level::Info), + "should write info, warnings and errors" + ); + assert!( + !writes(Level::Debug) && !writes(Level::Trace), + "should leave debug and trace lines out" + ); + } +} diff --git a/crates/trusted-server-adapter-spin/src/middleware.rs b/crates/trusted-server-adapter-spin/src/middleware.rs index eb12d6f65..d925a7500 100644 --- a/crates/trusted-server-adapter-spin/src/middleware.rs +++ b/crates/trusted-server-adapter-spin/src/middleware.rs @@ -11,6 +11,7 @@ use trusted_server_core::http_util::{ ROUTE_NAMESPACE, sanitize_trusted_client_ip_headers, strip_deployment_headers, }; use trusted_server_core::integrations::RequestFilterEffects; +use trusted_server_core::proxy::AssetProxyCachePolicy; use trusted_server_core::response_privacy::{ TerminalPrivateResponse, enforce_private_no_store, enforce_set_cookie_cache_privacy, enforce_uncacheable_cache_privacy, @@ -63,8 +64,9 @@ impl Middleware for SanitizeRequestMiddleware { /// /// Registered directly inside [`SanitizeRequestMiddleware`], so that every /// outgoing response carries a consistent set of headers. Once the response -/// is finalized it finalizes the Edge Cookie and then applies the response -/// effects the request filters left. +/// is finalized it applies an asset response's cache policy, finalizes the +/// Edge Cookie and then applies the response effects the request filters +/// left. pub struct FinalizeResponseMiddleware { settings: Arc, } @@ -82,11 +84,15 @@ impl Middleware for FinalizeResponseMiddleware { async fn handle(&self, ctx: RequestContext, next: Next<'_>) -> Result { let path = ctx.request().uri().path().to_owned(); let mut response = next.run(ctx).await?; + let asset_cache_policy = response.extensions_mut().remove::(); let ec_state = response.extensions_mut().remove::(); let request_filter_effects = response.extensions_mut().remove::(); // Before finalizing, so a header finalizing writes is kept. strip_deployment_headers(&path, ROUTE_NAMESPACE, &mut response); apply_finalize_headers(&self.settings, &mut response); + if let Some(policy) = asset_cache_policy { + policy.apply_after_route_finalization(&mut response, crate::app::EDGE_CACHE_HEADER); + } if let Some(mut ec_state) = ec_state { finalize_edge_cookie(&self.settings, &mut ec_state, &mut response).await; } diff --git a/crates/trusted-server-adapter-spin/src/platform.rs b/crates/trusted-server-adapter-spin/src/platform.rs index 200a7a7af..2aca7a784 100644 --- a/crates/trusted-server-adapter-spin/src/platform.rs +++ b/crates/trusted-server-adapter-spin/src/platform.rs @@ -84,7 +84,25 @@ impl PlatformSecretStore for NoopSecretStore { /// Spin outbound HTTP registers no backend, so the name is a correlation /// key alone, made from every field of the specification, and the transport /// timer is the exact bounded value. -struct NoopBackend; +/// +/// Spin sends the address's own host as the `Host` of every request, so a +/// backend asked to send another `Host`, as a source's `host_header` asks, is +/// refused rather than reached with the wrong one. +pub(crate) struct NoopBackend; + +/// Whether `host_header` names the host Spin sends as `Host`, which is the +/// backend's host, with its port where that is not the scheme's default. +fn is_the_address_s_own_host(spec: &PlatformBackendSpec, host_header: &str) -> bool { + let default_port = if spec.scheme == "https" { 443 } else { 80 }; + let (name, port) = match host_header.rsplit_once(':') { + Some((name, port)) => match port.parse::() { + Ok(port) => (name, port), + Err(_) => (host_header, default_port), + }, + None => (host_header, default_port), + }; + name.eq_ignore_ascii_case(&spec.host) && port == spec.port.unwrap_or(default_port) +} /// The correlation name for a backend, carrying the first-byte timer so two /// timers to one host are two backends, as the host's own naming would make @@ -118,6 +136,15 @@ impl PlatformBackend for NoopBackend { } fn ensure(&self, spec: &PlatformBackendSpec) -> Result> { + if let Some(host_header) = spec.host_header_override.as_deref() + && !is_the_address_s_own_host(spec, host_header) + { + return Err(Report::new(PlatformError::Backend).attach(format!( + "Spin sends the address's own host as `Host`, so the `Host` \ + `{host_header}` asked for `{}` cannot be sent", + spec.host + ))); + } self.predict_name(spec) } } @@ -893,6 +920,55 @@ mod tests { assert_eq!(backend.canonicalize_transport_timeout_ms(2000, 100), 100); assert_eq!(backend.transport_timeout_bucket_count(2000), 1); } + + /// A backend for `origin.example.com` at `port`, asked to send + /// `host_header` as the `Host`. + fn spec_sending(port: Option, host_header: &str) -> PlatformBackendSpec { + PlatformBackendSpec { + scheme: "https".to_owned(), + host: "origin.example.com".to_owned(), + port, + host_header_override: Some(host_header.to_owned()), + certificate_check: true, + first_byte_timeout: std::time::Duration::from_secs(15), + between_bytes_timeout: std::time::Duration::from_secs(15), + discriminator: None, + } + } + + #[test] + fn a_backend_asked_to_send_another_host_is_refused() { + for (port, host_header) in [ + (None, "www.example.com"), + (None, "origin.example.com:8443"), + (Some(8443), "origin.example.com"), + ] { + let error = NoopBackend + .ensure(&spec_sending(port, host_header)) + .expect_err("Spin cannot send a Host other than the address's own"); + assert!( + matches!(error.current_context(), PlatformError::Backend), + "the refusal should be a backend error for {host_header} at {port:?}" + ); + } + } + + #[test] + fn a_backend_asked_to_send_the_address_s_own_host_is_named() { + for (port, host_header) in [ + (None, "origin.example.com"), + (None, "Origin.Example.com:443"), + (Some(8443), "origin.example.com:8443"), + ] { + assert_eq!( + NoopBackend + .ensure(&spec_sending(port, host_header)) + .expect("the address's own host is the Host Spin sends"), + backend_name(&spec_sending(port, host_header)), + "the backend should be named for {host_header} at {port:?}" + ); + } + } use edgezero_core::context::RequestContext; use edgezero_core::http::request_builder; use edgezero_core::params::PathParams; diff --git a/crates/trusted-server-adapter-spin/tests/logging.rs b/crates/trusted-server-adapter-spin/tests/logging.rs new file mode 100644 index 000000000..31cb1c3ec --- /dev/null +++ b/crates/trusted-server-adapter-spin/tests/logging.rs @@ -0,0 +1,53 @@ +//! The component's logger, installed once for an instance at `info` and +//! above. +//! +//! A logger is global to a process, so these tests have a binary of their +//! own, in which nothing else installs one. + +use edgezero_core::app::Hooks as _; +use log::{Level, LevelFilter, Metadata}; +use trusted_server_adapter_spin::app::TrustedServerApp; +use trusted_server_adapter_spin::logging::init_logger; + +/// Whether the installed logger writes a record at `level`. +fn writes(level: Level) -> bool { + log::logger().enabled(&Metadata::builder().level(level).build()) +} + +/// The entry point installs the logger for every request, so a second call +/// keeps the first logger rather than failing. +#[test] +fn the_logger_is_installed_once_at_info_and_above() { + assert_eq!( + log::max_level(), + LevelFilter::Off, + "no logger should be installed before the first call" + ); + + init_logger(); + init_logger(); + + assert_eq!( + log::max_level(), + LevelFilter::Info, + "should write info and above" + ); + assert!( + writes(Level::Error) && writes(Level::Warn) && writes(Level::Info), + "the installed logger should write info, warnings and errors" + ); + assert!( + !writes(Level::Debug) && !writes(Level::Trace), + "the installed logger should leave debug and trace lines out" + ); +} + +/// The adapter installs its own logger, so `EdgeZero`'s `run_app` installs +/// none. +#[test] +fn the_app_owns_its_logging() { + assert!( + TrustedServerApp::owns_logging(), + "should tell EdgeZero that the adapter installs its own logger" + ); +} diff --git a/docs/guide/api-reference.md b/docs/guide/api-reference.md index 2a37930bc..a5372f738 100644 --- a/docs/guide/api-reference.md +++ b/docs/guide/api-reference.md @@ -61,7 +61,7 @@ authority, scheme, and client-address headers in the innermost middleware. | normal | `/verify-signature` | `POST` | literal | `always` | real | real | real | real | | normal | `/{*rest}` | `DELETE`, `GET`, `HEAD`, `OPTIONS`, `PATCH`, `POST`, `PUT` | template | `publisher_fallback` | publisher fallback | publisher fallback | publisher fallback | publisher fallback | | normal | `/` | `DELETE`, `GET`, `HEAD`, `OPTIONS`, `PATCH`, `POST`, `PUT` | literal | `publisher_fallback` | publisher fallback | publisher fallback | publisher fallback | publisher fallback | -| normal | `{*rest}` | `GET`, `HEAD` | config derived | `settings.proxy.asset_routes[]` | real | — | — | — | +| normal | `{*rest}` | `GET`, `HEAD` | config derived | `settings.proxy.asset_routes[]` | real | real | real | real | | startup | `/health` | `GET` | literal | `startup_error` | real | — | — | real | | startup | `/{*rest}` | `DELETE`, `GET`, `HEAD`, `OPTIONS`, `PATCH`, `POST`, `PUT` | template | `startup_error` | startup error (`500`) | startup error (`500`) | startup error (`500`) | startup error (`500`) | | startup | `/` | `DELETE`, `GET`, `HEAD`, `OPTIONS`, `PATCH`, `POST`, `PUT` | literal | `startup_error` | startup error (`500`) | startup error (`500`) | startup error (`500`) | startup error (`500`) | @@ -1013,8 +1013,8 @@ https://edge.example.com/_ts/debug/ja4`. ### GET or HEAD `{path}` -**Contract:** Fastly-only and registered once for each -`[[proxy.asset_routes]]` entry. Auth: optional origin signing by the signer +**Contract:** Answered on every adapter for each `[[proxy.asset_routes]]` +entry. Auth: optional origin signing by the signer the route's `auth` table names, which authenticates Trusted Server to the origin rather than the caller. Request body: not applicable. The route maps the remainder of the path and the @@ -1022,8 +1022,11 @@ configured query policy to the selected origin, preserves range and conditional request semantics, and returns the origin status/body after stripping unsafe response headers. Cache behavior follows the route's normalized cache policy; credentialed or otherwise private responses are forced `private, no-store`. -No dedicated CORS grant or rate limiter is installed. Example: if the prefix is -`/assets/`, request `curl -I https://edge.example.com/assets/logo.svg`. +No dedicated CORS grant or rate limiter is installed. On Axum, Cloudflare and +Spin a request the route's Image Optimizer would optimize answers `502`, and +on Spin so does a request to a source whose `host_header` names another +host. Example: if the prefix is `/assets/`, request +`curl -I https://edge.example.com/assets/logo.svg`. ### Publisher fallback: `/` and `/{*rest}` @@ -1035,7 +1038,9 @@ streaming where supported. The origin status is normally preserved; fetch or processing failures use shared adapter errors. Cacheability is decided from the origin response plus personalization, cookie, template, and rewrite effects. CORS is therefore the sanitized publisher-origin policy, not a global -Trusted Server policy. There is no built-in endpoint rate limiter. Example: +Trusted Server policy. There is no built-in endpoint rate limiter. On Spin, +where `[source.origin] host_header` names a host other than the one in `url`, +every request answers `502` and the origin is not asked. Example: `curl -i https://edge.example.com/article`. When startup fails, the degraded router replaces normal routing. Every diff --git a/docs/guide/asset-routes.md b/docs/guide/asset-routes.md index 9d013969d..385c26ecc 100644 --- a/docs/guide/asset-routes.md +++ b/docs/guide/asset-routes.md @@ -18,6 +18,8 @@ For a matching `GET` or `HEAD` request, Trusted Server runs this sequence: The origin query policy runs before S3 signing, so the signature covers the exact URL sent to S3. Image Optimizer metadata stays separate from the origin URL and is translated by the Fastly adapter. +Every adapter serves asset routes. The Axum, Cloudflare and Spin adapters hold the asset body whole, which Cloudflare limits to 10 MiB and Spin to 8 MiB, answering a larger one with `502`. They cannot optimize images, so each logs a warning naming every route that enables the Image Optimizer, once for each instance of the adapter, which on Axum is the process (the [Cloudflare](./cloudflare) and [Spin](./spin) guides say where the lines appear), and answers each request such a route would optimize with `502 Bad Gateway` and `Cache-Control: no-store, private`. A signed route first asks its origin for the object with a `HEAD`, and where that fails it answers with the origin's own error. SVG and SVGZ paths, and a request that sets the profile set's debug parameter to `1`, carry no Image Optimizer metadata, so they are served unoptimized. + ## Basic route Each route names where its content comes from in a `source` table, in the same shape as the publisher's [`[source]`](/guide/configuration#source). `module = "origin"` selects core's HTTP origin, and `[proxy.asset_routes.source.origin]` holds its settings. Write the route's own keys, `prefix`, `path_pattern` and `target_path`, before the source tables, because a table header ends the route's own keys. @@ -44,7 +46,7 @@ https://assets.example.com/assets/logo.png?v=1 Plain asset routes preserve the incoming query string by default. `url` must be an absolute `http` or `https` address with a host, without a user name or password, a query, a fragment, or a trailing `/`. A path in `url` names a folder, and the route's target path is added under it. With `url = "https://storage.example.com/bucket/site"`, a request for `/assets/logo.png` is sent to `https://storage.example.com/bucket/site/assets/logo.png`. -`host_header` sets the outbound `Host` header where it is not the host of `url`. Trusted Server still connects, uses SNI, and checks certificates against the host in `url`. +`host_header` sets the outbound `Host` header where it is not the host of `url`. Trusted Server still connects, uses SNI, and checks certificates against the host in `url`. Spin sends the host in `url` as the `Host`, so there every request under a route whose `host_header` names another host answers `502` before its origin is asked. Cloudflare documents that a Worker cannot set a `Host` naming a host outside its zone unless the request goes to that host (see [Cloudflare](./cloudflare)). ## Path rewrite diff --git a/docs/guide/axum-dev.md b/docs/guide/axum-dev.md index 4636ac442..7cb79cb44 100644 --- a/docs/guide/axum-dev.md +++ b/docs/guide/axum-dev.md @@ -75,7 +75,11 @@ request filters and the device module `[device]` selects run as on Fastly, and `[device] module = "platform"` classifies from the `User-Agent`, because the dev server supplies no TLS or HTTP/2 signals. An Edge Cookie module creates and writes its cookie with no identity graph behind it, and -`POST /_ts/api/v1/ec/resolve` sets one the same way. +`POST /_ts/api/v1/ec/resolve` sets one the same way. `[[proxy.asset_routes]]` +are served from each route's own source with the asset body held whole, with +no limit. The dev server has no Image Optimizer, so startup warns about each +route that enables one, and each request such a route would optimize answers +`502` (see [Asset Routes](./asset-routes)). Axum sends no [auction telemetry](./telemetry#adapter-support), and warns at startup when `[analytics]` selects a module. diff --git a/docs/guide/cloudflare.md b/docs/guide/cloudflare.md index fabe89cb8..b85c962d8 100644 --- a/docs/guide/cloudflare.md +++ b/docs/guide/cloudflare.md @@ -50,9 +50,11 @@ The generated local Wrangler files define `publisher_proxy_secret` and `ec_passphrase` separately. Edge Cookie identity is off by default, so the smoke's settings select the built-in `hmac` module, whose passphrase is the `ec_passphrase` secret. The smoke launches -`wrangler dev` once for each omitted secret. Each case requires a 500 response -and verifies the exact missing binding against Wrangler's runtime binding -inventory. The Worker installs no logger, so no case waits for a log line. The +`wrangler dev` once for each omitted secret. Each case requires a 500 response, +verifies the exact missing binding against Wrangler's runtime binding +inventory, and waits for the Worker's log line naming the missing input, the +app-config key before the push and the omitted secret after it, which +`wrangler dev` prints with its own output (see [Logging](#logging)). The positive publisher request must return 200, include the stub-origin sentinel, rewrite its origin URL to the Worker listener, and omit the original URL. @@ -60,6 +62,24 @@ The exit trap stops Wrangler and the stub origin and removes the isolated KV, generated Wrangler manifests, response captures, and logs. It does not modify the checked-in manifests. +## Logging + +The Worker installs its own logger, because EdgeZero installs none on +Workers. Each line goes to the console method of its level, `console.error`, +`console.warn` or, for `info`, `console.log`, with its level and module first, +such as `WARN [trusted_server_adapter_cloudflare::app] ...`. `wrangler dev` prints the +lines in its terminal. For a deployed Worker, `wrangler tail` and the +dashboard's live logs show them, and +[Workers Logs](https://developers.cloudflare.com/workers/observability/logs/workers-logs/) +keeps them where the Wrangler file sets `[observability] enabled = true`, +which the checked-in `wrangler.toml` does not. The Worker writes `info` and +above, and no setting changes the level. + +A warning the Worker logs once, such as the one about a route that enables +the Image Optimizer, is logged once for each instance of the Worker. One +instance can serve many requests, so the warning appears again only when +Cloudflare starts another instance. + ## Configure a deployed Worker Local `[vars]` are only for the isolated smoke. For a deployed Worker, run @@ -84,7 +104,18 @@ selects run as on Fastly, and `[device] module = "platform"` classifies from the `User-Agent`, because the Worker supplies no TLS or HTTP/2 signals. An Edge Cookie module creates and writes its cookie with no identity graph behind it, and `POST /_ts/api/v1/ec/resolve` sets one the same way. -See the [API reference](./api-reference) for route-level behavior. +`[[proxy.asset_routes]]` are served from each route's own source with the +asset body held whole, up to 10 MiB, and a larger one answers `502`. The +adapter does not translate Image Optimizer metadata into Cloudflare's own +image options, so the Worker warns about each route that enables the Image +Optimizer, once for each instance (see [Logging](#logging)), and each request +such a route would optimize answers `502` (see +[Asset Routes](./asset-routes)). Cloudflare +[documents](https://developers.cloudflare.com/workers/runtime-apis/request/) +that a Worker cannot set a `Host` naming a host outside its zone unless the +request goes to that host, so a source's `host_header` naming such a host +does not reach its origin. See the [API reference](./api-reference) for +route-level behavior. The Worker sends no [auction telemetry](./telemetry#adapter-support), and warns at startup when `[analytics]` selects a module. diff --git a/docs/guide/configuration.md b/docs/guide/configuration.md index 05b4b5b40..5e46500b7 100644 --- a/docs/guide/configuration.md +++ b/docs/guide/configuration.md @@ -608,6 +608,9 @@ One blob for each path). - Connects, uses SNI, and checks certificates against the host in `url` - Sends the configured value as the outbound HTTP `Host` header - Useful when the origin endpoint expects a canonical publisher hostname +- Cannot name another host on Spin, which sends the host in `url` as the + `Host`, so there such a value makes every request for the origin answer + `502`, and the origin is not asked **Format**: Hostname with optional port, without protocol, path, query, or fragment @@ -1796,7 +1799,8 @@ beneath. Core's `origin` source takes these settings: - `directory_index` follows the same rules as the publisher's [`directory_index`](#directory-index). - With `host_header`, Trusted Server still connects, uses SNI, and checks - certificates against the host in `url`. + certificates against the host in `url`. Spin cannot send one that names + another host, so there every request under such a route answers `502`. ### `[proxy.asset_routes.source.origin.auth]` @@ -2078,7 +2082,7 @@ origin cache directives, including `no-cache`, are replaced by the configured policy. `Vary` is preserved, so do not assign a public immutable rule to paths that vary by cookies or other user-specific request state. -On a configured Fastly asset-rehost route, a matching rule is authoritative +On a configured asset route, a matching rule is authoritative over the third-party origin's cache defaults, including `no-store`, because Trusted Server owns the rehosted copy. A later Trusted Server or operator-applied `private` or `no-store` directive still vetoes public policy reapplication and diff --git a/docs/guide/spin.md b/docs/guide/spin.md index 6e69d5146..47ee4a396 100644 --- a/docs/guide/spin.md +++ b/docs/guide/spin.md @@ -62,15 +62,37 @@ variables: - `v_trusted_x5fserver_x5fsecrets_v_ec_x5fpassphrase` It first launches `spin up` before config is pushed, then relaunches it while -omitting each variable independently. Every publisher request must return 500. -The component installs no logger, so the smoke checks these cases by their -status alone. +omitting each variable independently. Every publisher request must return 500, +and the component's log, which the smoke keeps with `--log-dir`, must name the +intended missing input, the app-config blob before the push and the omitted +variable's secret after it. A generic degraded response cannot satisfy the +check. The final non-health publisher request must return 200, retain the stub-origin sentinel, rewrite its origin URL to the Spin listener, and omit the original URL. The exit trap stops Spin and the stub origin and removes the isolated manifest, SQLite store, response captures, and logs. +## Logging + +The component installs its own logger, because EdgeZero installs none on +Spin. Each line goes to standard error with its level and module first, such +as `WARN [trusted_server_adapter_spin::app] ...`. +[`spin up`](https://spinframework.dev/v4/running-apps) prints a component's +standard error in its console and, for an application run from a local +manifest, keeps a copy in the `.spin/logs` folder beside the manifest, or in +the folder `--log-dir` names. A host that runs the component another way +decides where its standard error goes. The component writes `info` and above, +and no setting changes the level. + +A warning the component logs once, such as the one about a route that +enables the Image Optimizer, is logged once for each instance of the +component. Spin can serve several requests from one instance, one after +another and at the same time, to a degree it does not guarantee +([HTTP trigger](https://spinframework.dev/v4/http-trigger), +`spin up --max-instance-reuse-count`), so the warning appears each time Spin +starts an instance, which is every request when an instance serves only one. + ## Runtime boundaries Use the same store mapping and encoded variables in the deployment provider; @@ -91,6 +113,15 @@ refused at startup. The modules' request filters and the device module classifies from the `User-Agent`, because Spin supplies no TLS or HTTP/2 signals. An Edge Cookie module creates and writes its cookie with no identity graph behind it, and `POST /_ts/api/v1/ec/resolve` sets one the same way. +`[[proxy.asset_routes]]` are served from each route's own source with the +asset body held whole, up to 8 MiB, and a larger one answers `502`. Spin has +no Image Optimizer, so the component warns about each route that enables one, +once for each instance (see [Logging](#logging)), and each request such a +route would optimize answers `502` (see [Asset Routes](./asset-routes)). Spin +sends the address's own host as the `Host`, so a request to a source whose +`host_header` names another host, a route's or the publisher's, answers `502` +before its origin is asked, and the component logs the refusal as an error +that names both hosts. Spin sends no [auction telemetry](./telemetry#adapter-support), and warns at startup when `[analytics]` selects a module. diff --git a/docs/superpowers/specs/2026-10-14-origin-signing-design.md b/docs/superpowers/specs/2026-10-14-origin-signing-design.md index a312cafb7..3324d3ffc 100644 --- a/docs/superpowers/specs/2026-10-14-origin-signing-design.md +++ b/docs/superpowers/specs/2026-10-14-origin-signing-design.md @@ -5,7 +5,7 @@ **Related specs:** `2026-10-14-source-modules-design.md`, `2026-10-13-publisher-object-storage-origin-design.md`, `2026-05-19-asset-s3-auth-fastly-io-design.md` and `2026-10-14-no-modules-crate-design.md`, all in IABTechLab/trusted-server#1342, which holds every design document of the stack **Related PRs:** jwrosewell/trusted-server#43, IABTechLab/trusted-server#1391 **Issues:** IABTechLab/trusted-server#1338 -**Last updated:** 2026-10-15 +**Last updated:** 2026-10-18 > **Why this spec exists.** Core signs reads from the publisher's origin and from each asset route's origin with one cloud vendor's signing scheme, and its settings know that scheme's credentials, so an origin that wants another scheme has to add a branch to core. @@ -17,7 +17,7 @@ So core knows one vendor's scheme. A signed read from another vendor's storage, ## 2. Design -**Core keeps an origin authentication interface.** `trusted_server_core::origin_auth` declares `OriginSigner`, which signs the headers of a request for a method on a final address, and removes every header in the scheme's own namespace so a reader's header there cannot reach the origin beside the signature. Core calls it where it signed before, in the asset route path, in the preflight that path runs ahead of the image optimizer, and in the publisher path, through the `OriginSigning` a `SourceLocation` carries. The publisher path is core's own, so it signs on all four adapters. Asset routes and the preflight run on the Fastly adapter alone, because only that adapter calls `handle_asset_proxy_request`. +**Core keeps an origin authentication interface.** `trusted_server_core::origin_auth` declares `OriginSigner`, which signs the headers of a request for a method on a final address, and removes every header in the scheme's own namespace so a reader's header there cannot reach the origin beside the signature. Core calls it where it signed before, in the asset route path, in the preflight that path runs ahead of the image optimizer, and in the publisher path, through the `OriginSigning` a `SourceLocation` carries. The publisher path and the asset route path with its preflight are core's own, and every adapter calls both, so all three sign on all four adapters. On Spin a request to a source whose `host_header` names another host is refused before it is signed, because Spin sends the address's own host as the `Host`. Three things differ from what core did before. First, the `HEAD` preflight, which asked an S3 origin for the object before the image optimizer did, now runs for any signed asset route that uses the optimizer, whatever its signer. Second, removing the headers in the signer's namespace is each signer's duty, so core calls `remove_namespace_headers` before `sign` on the publisher path and the asset route path, and does not check afterwards that none is left. Third, the asset route path's call to `remove_namespace_headers` is new, because that call was made on the publisher path alone. The asset route path starts its outbound headers from eleven named request headers (`ASSET_PROXY_FORWARD_HEADERS` in `crates/trusted-server-core/src/proxy.rs`) and none of them is in the `x-amz-` namespace, so for `s3_sigv4` the call removes nothing. The preflight signs its `HEAD` and its `GET` without calling `remove_namespace_headers`, which also makes no difference for `s3_sigv4`, because it signs copies of those same headers and `Host`. @@ -92,11 +92,6 @@ Four cases have no test that can fail: 3. No deploy validation test gives an `s3_sigv4` table an empty `session_token`. The refusal is tested only through the core fixture signer's `token_name`, a setting declared the same way. 4. No test in the `ts` tool's crate names the AWS builder or an `s3_sigv4` table, so nothing fails if the builder leaves the list in `linked_builders()`, and `ts config validate` would then skip every `s3_sigv4` table. -## Review log - -1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/48-aws-signing` at `a3c17125b`. Issues found, two of them blocking. The section on AWS signing named a modules crate that no longer exists and deploy rules the builder does not carry, and the spec left out that a `type` nothing offers passes `ts config validate` and `ts config push` and only stops the adapter at startup, while it listed the removed `s3_sig_v4` spelling under what is not in the change. This revision corrects both, states which stage refuses each mistake, what a failed signature answers, the rollout, the adapters that run each path, the settings that change and the tests that are missing or cannot fail, leaves closing the gap as an open question, and corrects the changelog, integration guide and configuration guide sentences that repeated the stale claims. -2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `5c43f6f9a`, of the round 1 fixes and the new text. No blocking issues once the pull request number was corrected. This revision names IABTechLab/trusted-server#1342 as the specs pull request, says that the preflight signs without calling `remove_namespace_headers`, leaves `GET /health` and the Fastly debug probes out of the startup error answer, limits the missing test for an empty `session_token` to deploy validation, and corrects the Tinybird entry in the changelog. - ## Revision record | Date | Change | @@ -104,3 +99,9 @@ Four cases have no test that can fail: | 2026-10-14 | First draft. | | 2026-10-15 | Fixes from review round 1. The stale modules crate and deploy rules claims, the stages that refuse a mistake, the gap and open question for a `type` nothing offers, the breaking removal of `s3_sig_v4`, failure answers, rollout, the adapters that run each path, the missing tests and the review log. | | 2026-10-15 | Corrections from review round 2. The specs pull request number, the preflight's signing without removing namespace headers, the requests an adapter answers whatever its startup error, the deploy validation test for an empty token and the Tinybird changelog entry. | +| 2026-10-18 | Where the asset route path and its preflight run, which is every adapter once asset routes are served on every adapter, and Spin's refusal of a `host_header` that names another host. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/48-aws-signing` at `a3c17125b`. Issues found, two of them blocking. The section on AWS signing named a modules crate that no longer exists and deploy rules the builder does not carry, and the spec left out that a `type` nothing offers passes `ts config validate` and `ts config push` and only stops the adapter at startup, while it listed the removed `s3_sig_v4` spelling under what is not in the change. This revision corrects both, states which stage refuses each mistake, what a failed signature answers, the rollout, the adapters that run each path, the settings that change and the tests that are missing or cannot fail, leaves closing the gap as an open question, and corrects the changelog, integration guide and configuration guide sentences that repeated the stale claims. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `5c43f6f9a`, of the round 1 fixes and the new text. No blocking issues once the pull request number was corrected. This revision names IABTechLab/trusted-server#1342 as the specs pull request, says that the preflight signs without calling `remove_namespace_headers`, leaves `GET /health` and the Fastly debug probes out of the startup error answer, limits the missing test for an empty `session_token` to deploy validation, and corrects the Tinybird entry in the changelog. diff --git a/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md new file mode 100644 index 000000000..112d7158d --- /dev/null +++ b/docs/superpowers/specs/2026-10-17-asset-routes-on-every-adapter-design.md @@ -0,0 +1,129 @@ +# Design Spec: Asset routes are served on every adapter + +**Status:** Proposed, 2026-10-17. Implemented as the pull request [IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412), row 54 of the stack, above [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411), which creates and writes the Edge Cookie on every adapter, for Tech Lab review. [IABTechLab/trusted-server#1342](https://github.com/IABTechLab/trusted-server/pull/1342) carries every design document of the stack in one place. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-10-17-request-filters-on-every-adapter-design.md`, `2026-10-17-edge-cookie-on-every-adapter-design.md`, `2026-10-14-source-modules-design.md`, `2026-10-14-origin-signing-design.md`, `2026-10-15-asset-cache-design.md`, and the design documents of the stack in IABTechLab/trusted-server#1342 +**Related PRs:** [IABTechLab/trusted-server#1412](https://github.com/IABTechLab/trusted-server/pull/1412). The request filters run on every adapter in [IABTechLab/trusted-server#1410](https://github.com/IABTechLab/trusted-server/pull/1410) and the Edge Cookie in [IABTechLab/trusted-server#1411](https://github.com/IABTechLab/trusted-server/pull/1411) +**Issues:** part of IABTechLab/trusted-server#777 +**Last updated:** 2026-10-28 + +> **Why this spec exists.** Only the Fastly adapter serves `[[proxy.asset_routes]]`. On Axum, Cloudflare and Spin a request under a route's prefix goes to the publisher's origin as an ordinary page request, so a route's own source, its signing and its held answers are never used there. This change dispatches asset routes on those three adapters as the Fastly adapter does. Two parts of it reach beyond asset routes. On Spin a source whose `host_header` names another host is now refused, the publisher's source as well as a route's, so a Spin deployment whose publisher source sets one answers every page with `502`, which is a breaking change. And the Cloudflare and Spin adapters now install a logger, because neither had one and every line they logged was dropped, the warnings this change adds included. + +## 1. Problem + +`dispatch_fallback` in `crates/trusted-server-adapter-fastly/src/app.rs` checks a `GET` or `HEAD` against `Settings::asset_route_index_for_path` once robots.txt, the Edge Cookie context, the request filters, tsjs and the module routes have had their turn, and before the publisher's origin. `dispatch_asset_fallback` then asks the registry for the route's source (`IntegrationRegistry::route_source`) and answers with `proxy::handle_asset_proxy_request`, which locates the asset through the source module, signs the request where the source's `auth` table names a signer, offers each `GET` the Image Optimizer does not handle to the asset cache module the deployment selects before it asks the origin, and keeps no copy itself. The response carries its `AssetProxyCachePolicy`, which the entry point applies after the finalize headers, and the request filters' effects, and no Edge Cookie state, because an asset response skips the Edge Cookie steps. + +No code in the Axum, Cloudflare or Spin adapter calls `handle_asset_proxy_request`. `docs/guide/api-reference.md` lists the route as Fastly's alone, and the Cloudflare HTTP client's note says these requests are not routed to it. + +The Spin client sends the address's own host as the `Host` of every request, so a source whose `host_header` names another host reaches its origin with the address's host in place of the one it names. That holds for the publisher's `[source.origin]` as much as for a route's source, so the refusal this change adds on Spin (section 2) changes the publisher path as well as the asset routes. + +Neither the Cloudflare adapter nor the Spin adapter installs a logger. Each hands every request to EdgeZero's `run_app`, whose `init_logger` does nothing on Workers or on Spin at the locked v0.0.10 (`edgezero-adapter-cloudflare/src/lib.rs` and `edgezero-adapter-spin/src/lib.rs`), so the `log` crate drops every line the two adapters write, including the warnings they log as they build their state, a startup error and the report behind each error answer. + +## 2. Design + +**Dispatch.** Each adapter's `dispatch_fallback` checks a `GET` or `HEAD` against the asset routes where the Fastly adapter does, and `dispatch_asset_fallback` follows the Fastly function of that name: the route's source, `handle_asset_proxy_request`, the response's cache policy in its extensions, and the request filters' effects. An error from the handler answers with the status the error's kind gives and the `no-store, private` policy, as on Fastly. No Edge Cookie state is attached, so the finalize middleware neither writes nor expires an Edge Cookie on an asset response. The Fastly adapter is unaffected, because this change touches no file of the Fastly adapter or of core. + +**Finalizing.** `FinalizeResponseMiddleware` takes the cache policy off the response and applies it after the finalize headers with the adapter's own shared cache header (`AssetProxyCachePolicy::apply_after_route_finalization`), before the Edge Cookie and the filters' effects, the order the Fastly entry point uses. Axum and Spin write a shared cache lifetime as `s-maxage` in `Cache-Control`, and Cloudflare writes it in `cloudflare-cdn-cache-control`. + +**Held answers.** `handle_asset_proxy_request` offers each `GET` that the platform's Image Optimizer does not answer to the asset cache module `[asset-cache]` selects before asking the route's origin, as on Fastly, and keeps no copy itself (`2026-10-15-asset-cache-design.md`), so a deployment that selects no module, a stock build among them, asks the origin for every answer. The 51Degrees asset cache module holds its copies, and its records of what each address varies on, through the loading caches `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]` select, and holds nothing in a build that links no loading cache module. The 51Degrees loading cache module keeps a memory layer for each application state and its shared copies in the host's value cache where the services carry one, which only the Fastly adapter supplies, and in the key-value store otherwise (`crates/loading-cache/51degrees/src/stores.rs`). Cloudflare and Spin build their state for every request, so there a copy outlives its request only in the key-value store each declares, `TRUSTED_SERVER_KV` on Cloudflare and `default` on Spin. Axum builds its state once for the process, but its key-value store is the unavailable one, so a copy lives only in memory, for `[cache.51degrees-assets] memory_seconds`, 5 seconds by default. An asset fetch also asks for the platform's default cache behavior (`PlatformCacheIntent::Default`), which the Cloudflare client leaves to the Workers runtime, whose subrequests are eligible for Cloudflare's cache by default, so on Cloudflare the runtime may hold an answer as well. + +**The body.** None of the three HTTP clients streams a response (`supports_streaming_responses` is `false` there), so core asks for no stream and each client holds the asset body whole, where the Fastly adapter streams it with no limit. The Cloudflare client refuses a body over 10 MiB (`MAX_PLATFORM_RESPONSE_BODY_BYTES`) and the Spin client one over 8 MiB, before or after it is decompressed (`MAX_DECOMPRESSED_SIZE`), and the Axum client holds any size. The Cloudflare client writes the length of the body it read as the `Content-Length`, so its answer to a `HEAD` says `0`, where the Fastly and Spin adapters keep the origin's length. + +**A source's `Host`.** A route's source can name the `Host` its origin is asked with (`host_header`), which core writes before the signer signs it. The Fastly adapter sends it through its backend's host override, and the Axum client hands it to its HTTP library, which adds a `Host` only to a request that has none. Spin sends the address's own host as the `Host` of every request, because WASI HTTP takes the authority from the address and forbids a `host` header, so a source whose `host_header` names another host would reach its origin with the wrong `Host`, and a signed one with a signature made for another. The Spin backend (`NoopBackend::ensure` in `crates/trusted-server-adapter-spin/src/platform.rs`) therefore refuses to name a backend asked for a `Host` other than the address's own, with its port where that is not the scheme's default, and core answers the request with `502` before anything is sent. Nothing checks the setting at startup, so a deployment learns of the refusal from the first request refused, through the error line the component logs, which names both hosts (Logging). The publisher's own source reaches the same backend, so a publisher source with such a `host_header` is refused on Spin too. The backend accepts a `host_header` that names the address's own host in other letter case or with the scheme's default port written, and Spin then sends the address's host, while `s3_sigv4` signs the `Host` as the source writes it (`canonical_headers` in `crates/shared/aws/src/sigv4.rs`), so a signed source written that way still fails its origin's signature check. That was so before this change too, and a `host_header` written in lower case without the default port, or none, avoids it. The Cloudflare client hands the header to `fetch` as given, and Cloudflare documents that a Worker cannot set a `Host` naming a host outside its zone unless the request goes to that host ([Request](https://developers.cloudflare.com/workers/runtime-apis/request/)), so there a `host_header` naming such a host does not reach the origin. + +**Image Optimizer.** A route that enables `[proxy.asset_routes.image_optimizer]` sets Image Optimizer metadata on each request whose path is not an SVG or SVGZ file and whose query does not set the profile set's debug parameter to `1` (`asset_path_skips_image_optimizer` and `options_for_asset_request` in core). The Fastly adapter translates the metadata, and the other three cannot. The Cloudflare and Spin HTTP clients already refused such a request on both send paths, as the `PlatformHttpRequest` contract asks of a client that cannot honor it, and the Axum client now refuses it on `send` and `send_async`, where it sent the request unchanged before. A refused request answers `502 Bad Gateway` with `no-store, private`. Before the refusal, a route whose source signs asks its origin for the object with a `HEAD` (`preflight_signed_origin_for_image_optimizer`), and where that `HEAD` fails the request is answered with the origin's own error, with no metadata and with `no-store, private`, as on Fastly. An SVG path and a debug request carry no metadata, so they are served unoptimized and their answers are kept as any other asset's are. + +| Adapter | Source and signing | A source's `host_header` | Held answers | Body | Image Optimizer | +| ---------- | ------------------ | ---------------------------------- | --------------------------------------------- | ------------------------ | ---------------------------- | +| Fastly | Yes | Sent | Loading caches, copies in the value cache | Streamed, no limit | Translated | +| Axum | Yes | Sent | Loading caches, copies in memory alone | Held whole, no limit | Refused by the client, `502` | +| Cloudflare | Yes | Not sent outside the Worker's zone | Loading caches, copies in `TRUSTED_SERVER_KV` | Held whole, up to 10 MiB | Refused by the client, `502` | +| Spin | Yes | Refused by the backend, `502` | Loading caches, copies in `default` | Held whole, up to 8 MiB | Refused by the client, `502` | + +The held answers column is for a deployment that links and selects the 51Degrees asset cache module and the 51Degrees loading cache module. A stock build links neither and keeps no copy of its own on any adapter. + +**Logging.** Each of the Cloudflare and Spin adapters now installs a small logger of its own at its entry point, before `run_app` (`init_logger` in `src/logging.rs` of each adapter), and says so through `Hooks::owns_logging`, so `run_app` does not try to install one. The Worker writes each line through the `worker` crate's console macros by level, `console.error`, `console.warn` and, for `info`, `console.log`, with `console.debug` for the two levels below, which it does not write, so `wrangler dev` prints a line, and for a deployed Worker `wrangler tail` and the dashboard's live logs show it, as Workers Logs does where the Wrangler file enables `[observability]`. The Spin component writes each line to standard error, which `spin up` prints in its console and keeps in the `.spin/logs` folder beside a local manifest, and a host that runs the component another way decides where standard error goes. A line gives the level and the module before the message. Both loggers write `info` and above, the level the Axum adapter writes by default. Each is installed once for an instance, because the entry point runs for every request and a call that finds a logger installed changes nothing. With them every adapter logs the full error of a state that cannot be built, which `2026-10-14-origin-signing-design.md` says of a startup failure, and the report behind each error answer, which on Spin names the `Host` refused. + +**Configuration.** No key is added or changed, and `[[proxy.asset_routes]]` now takes effect on every adapter. As it builds its state, each of the Axum, Cloudflare and Spin adapters logs a warning naming the prefix of every route that enables the Image Optimizer (`warn_once_when_an_asset_route_enables_the_image_optimizer`), and then serves the route. A flag keeps the warning to one for each instance of the adapter. Axum builds its state once for the process, so it warns once, at startup. Cloudflare and Spin build their state for every request, so there the flag is what keeps the warning from every request. A Workers instance can serve many requests ([How Workers works](https://developers.cloudflare.com/workers/reference/how-workers-works/)), and Spin can serve several requests from one instance of the component, one after another and at the same time, to a degree it does not guarantee ([HTTP trigger](https://spinframework.dev/v4/http-trigger)), so on Spin the warning appears each time Spin starts an instance, which is every request when an instance serves only one (`spin up --max-instance-reuse-count`). + +They warn rather than refuse the settings, for three reasons. Such a route still answers its SVG paths, its debug requests and, where it signs, the origin's own errors. The deployment's other routes and pages do not depend on it. And a refusal would stop a Spin or Cloudflare deployment, or an Axum dev server, that shares its settings with a Fastly service whose routes optimize images. Cloudflare and Spin do refuse at startup an enabled auction with more than one provider (`AuctionPlan::validate_for_target`), although a Fastly service runs one, because such an auction can do nothing there. Their HTTP clients send one request at a time, and the auction orchestrator refuses an auction of more than one provider before any provider is asked, so those settings could run no auction at all, where a route that enables the Image Optimizer still answers its SVG paths and debug requests. The third reason holds for such an auction too, and the refusal stands there because no auction would run either way. + +**Consent and privacy.** An asset route forwards only `ASSET_PROXY_FORWARD_HEADERS` to the route's origin, plus the `Host` and the signer's headers, as on Fastly, so the reader's cookies are not sent there. Before this change the request went to the publisher's origin with the headers the publisher path forwards. A route's origin, which need not be the publisher's, now receives the reader's `User-Agent`, `Accept`, `Accept-Encoding`, `Accept-Language` and `Referer`, with the cache validation and range headers, and not the reader's address, because the forwarding headers are stripped at the edge and not rebuilt for an asset route. An asset response skips the Edge Cookie steps (`ec_finalize_response` does not run), so it neither writes a created Edge Cookie nor expires the cookie and its markers when the reader has withdrawn permission, which matches Fastly. Before this change the publisher fallback did both for the same requests on these adapters, and a response from the publisher path still does. + +**Failure handling.** + +- A route whose source cannot be built stops the adapter when it builds its state, as the publisher's source does, because the registry builds every source from the settings (`build_sources`). That happens at startup on Axum and for every request on Cloudflare and Spin, and each request is then answered with the startup error response, `500` on Fastly, Axum and Cloudflare and `503` on Spin. `route_source` answers an error for a single request only for a registry built without settings, which no adapter's state builder makes. +- Each of these answers one request with the status its error gives, and with `no-store, private`, on every adapter: + - a source module's `locate` error, with the source's own status, where core's `origin` source never fails + - a request path the route's `path_pattern` does not match, `502`, a rewrite that does not start with `/`, `500`, and a target path with a `.` or `..` segment under a source folder, `400` + - an unknown image profile in `reject` mode, `400` + - a signer that cannot sign, with the signer's status, which for `s3_sigv4` is `400` when a header it signs or writes is not valid + - a backend the platform refuses, `502`, which on Spin is a source whose `host_header` names another host + - an origin that cannot be reached, a body over the client's limit and a request the client refuses, such as one carrying Image Optimizer metadata, `502` +- An origin's own error status is passed through as its answer, and no cache rule is applied to it. +- Every adapter logs each per-request error listed above at error level with its full report (`http_error`), which on Spin names both hosts of a refused `host_header`, and logs a state that cannot be built with its full error. On Cloudflare and Spin both kinds of error line were dropped before this change, with no logger to write them (Logging). +- Core asks the route's backend for a first byte timeout and a between bytes timeout of 15 seconds (`DEFAULT_FIRST_BYTE_TIMEOUT`), which the Fastly adapter applies. The Axum client applies limits of its own instead, 5 seconds to connect and 30 seconds for the whole request, and the Cloudflare and Spin backends only make a name, so there a slow origin is bounded by the runtime's own limits. `2026-10-15-asset-cache-design.md` says the same of the asset caches. + +**Rollout.** + +1. A deployment with no asset routes serves its requests as before, because `asset_route_index_for_path` answers `None` when there are none, with two exceptions. On Spin a publisher source whose `host_header` names another host is now refused (item 4), and on Cloudflare and Spin the lines the adapter logs are now written (item 6). +2. One with asset routes now has their `GET` and `HEAD` requests answered by each route's origin on Axum, Cloudflare and Spin, where the publisher's origin answered them, with the route's signing and cache rule, and with no Edge Cookie written or expired. Where a deployment links and selects the 51Degrees asset cache module and the 51Degrees loading cache module, a route reads and keeps copies for a request that carries a cookie, which the publisher path fetches for that request alone, so after the upgrade more requests read and write the key-value store, `TRUSTED_SERVER_KV` on Cloudflare and `default` on Spin. A stock build keeps no copy and asks each route's origin for every answer. +3. On those adapters a route that enables the Image Optimizer answers each request it would optimize with `502` and `no-store, private`, where the publisher's origin answered it before, after a signed route has sent its origin the `HEAD`. Its SVG paths and debug requests are served unoptimized, and each adapter logs a warning naming each such route, once for each instance (Configuration). +4. On Spin a request to a source whose `host_header` names another host answers `502` and its origin is not asked, a route's source and the publisher's alike, where before the request reached the origin with the address's own host as the `Host`, and the component logs the refusal as an error naming both hosts. A Spin deployment whose publisher source sets such a `host_header` therefore answers every page with `502`, robots.txt included. The changelog gives this as a breaking change, with the check to make before upgrading. +5. A body over 10 MiB on Cloudflare or 8 MiB on Spin answers `502`. +6. On Cloudflare and Spin the lines the adapter logs at `info` and above are written, where every line was dropped, so the warnings above, the identity graph and analytics warnings these adapters already logged, a startup error and the report behind each error answer reach the Worker's console and the component's standard error. + +**Tests.** In `src/app/request_cycle_tests.rs` of each of the three adapters, with a stub origin: + +- `a_request_under_an_asset_route_reaches_the_route_s_origin_signed`: with a route whose source signs with the stock `s3_sigv4` signer, the route's origin is asked, at the route's address, with an `AWS4-HMAC-SHA256` `Authorization` header, and its answer is the response. +- `a_request_outside_the_asset_routes_reaches_the_publisher_s_origin`: the publisher's origin is asked, unsigned. +- `an_asset_route_s_cache_rule_is_written_after_the_finalize_headers`: with a `[[cache.asset_rules]]` rule of a minute for browsers and an hour for shared caches, the answer carries `public, max-age=60, s-maxage=3600` on Axum and Spin, and `public, max-age=60` with `cloudflare-cdn-cache-control: max-age=3600` on Cloudflare. +- `an_asset_route_whose_origin_fails_answers_502_and_is_not_stored`: an origin that fails gives `502` with `cache-control: no-store, private`. +- `a_page_view_under_an_asset_route_gets_no_edge_cookie`: a browser's page navigation under the route, whose permissions would grant an Edge Cookie on a page, gets none, and the route's origin answers it. +- `startup_warns_when_an_asset_route_enables_the_image_optimizer`: building the router with such a route sets the flag the warning function sets as it logs the warning, and the test reads the flag rather than a log line. + +The Spin adapter's request cycle tests also send requests through the backend its own services use. In `an_asset_route_whose_source_names_another_host_is_refused_before_its_origin_is_asked` the route whose source names another host answers `502` with `no-store, private` and only the route without one reaches its origin, and in `a_publisher_source_that_names_another_host_is_refused_before_its_origin_is_asked` a page request answers `502` and the publisher's origin is not asked. Its platform tests `a_backend_asked_to_send_another_host_is_refused` and `a_backend_asked_to_send_the_address_s_own_host_is_named` cover another name, another port and the same host written with its default port or in other letter case. The Axum adapter's platform test `a_request_with_image_optimizer_metadata_is_refused` sends through `send` and `send_async`. + +The Cloudflare and Spin loggers are tested natively. In the Cloudflare adapter's `src/logging.rs`, `each_level_is_written_with_the_console_method_for_it`, `a_line_gives_the_level_and_target_before_the_message` and `info_and_above_are_written_and_debug_and_trace_are_not` cover the console method, the line and the level, and in the Spin adapter's, `a_record_is_written_as_one_line_with_its_level_and_target` and `info_and_above_are_written_and_debug_and_trace_are_not` cover the line and the level. In `tests/logging.rs` of each adapter, a test binary of its own because a logger is global to a process, `the_logger_is_installed_once_at_info_and_above` calls `init_logger` twice and finds one logger installed that writes `info` and above and leaves debug and trace out, and `the_app_owns_its_logging` checks `Hooks::owns_logging`. The entry points that call `init_logger` compile for wasm32 alone, which `cargo clippy-cloudflare-wasm` checks for the Worker and `cargo clippy-spin-wasm` for the component. + +Each test was run with the code it guards taken out, and failed. Without the finalize middleware's cache policy step the cache rule test sees `public, max-age=60` and no shared lifetime, and the error test sees no `Cache-Control`, on each adapter, and so does the Spin route refusal test, which also checks `no-store, private`. Without `NoStorePrivate` in the error arm the error test fails on each adapter, an asset response given the publisher path's Edge Cookie state writes a `ts-ec` cookie and fails the Edge Cookie test on each adapter, and a state builder without the warning call fails the warning test on each adapter. With Spin's refusal disabled, `a_backend_asked_to_send_another_host_is_refused` fails and both Spin request cycle tests answer `200`, while `a_backend_asked_to_send_the_address_s_own_host_is_named` still passes, and with the `send_async` check taken out the Axum client test fails at its `send_async` assertion. With `init_logger` installing nothing, as EdgeZero's does, `the_logger_is_installed_once_at_info_and_above` fails on each adapter with the level still `Off`, and with a second call that fails rather than being ignored it fails at that call with `SetLoggerError`. With `owns_logging` left at its default, `the_app_owns_its_logging` fails on each adapter. With the level raised to `debug`, the level test fails on each adapter, with `info` written through `console.debug` the Cloudflare console method test fails, and without its line ending the Spin line test fails. + +No test sends an asset request through the Cloudflare HTTP client, whose refusals, body limit and `Content-Length` handling are wasm32 code that the native tests do not run. In the Spin client only the send itself is wasm32 code, and its refusals (`reject_unsupported_request_contracts`), its body limits and its `HEAD` length (`apply_spin_response_policy`) have native tests of their own. The cross-adapter parity tests have no asset route case. + +**Open questions.** + +1. Should the Cloudflare adapter refuse a source's `host_header` the way Spin now does? Cloudflare documents that a Worker cannot set a `Host` naming a host outside its zone unless the request goes to that host, but the repository has no test of the Workers runtime that shows what it sends for a `host_header` inside the zone, so this change leaves the Cloudflare client handing the header to `fetch` as before. +2. Should the Cloudflare client keep the origin's `Content-Length` on a `HEAD`, as the Fastly and Spin adapters do? The repository does not show whether the Workers runtime corrects the header on the way out, and the origin's length describes the body as it was encoded, which the runtime decodes. + +## 3. What is not in this change + +- An Image Optimizer on Axum, Cloudflare or Spin. Cloudflare offers image transformations to a Worker through the `cf.image` option of `fetch` ([Transform via Workers](https://developers.cloudflare.com/images/optimization/transformations/transform-via-workers/)), and the adapter does not translate the metadata into it. +- Streaming an asset body on those adapters, whose HTTP clients hold every response whole. +- Applying the 15-second timeouts core asks for on Axum, Cloudflare and Spin. +- Sending a source's `host_header` on Spin, whose outbound HTTP sends the address's own host. +- A startup check on Spin of each source's `host_header`, which would refuse the settings before the first request rather than each request as it comes. +- A setting for the log level on Cloudflare and Spin, whose loggers write `info` and above. +- A test that reads the Image Optimizer warning's text. The warning tests read the flag the warning function sets as it logs, and of the three adapters only Axum has a log capture in its tests that could read the line. +- A parity test of asset routes. The parity tests build each router with its own services, where the Cloudflare and Spin HTTP clients are unavailable on a native host and the Axum client is a real one, so an asset request reaches no origin there, and the request cycle tests send the same requests through each adapter's composition root with a stub origin instead. + +## 4. Acceptance + +The tests above pass on each of the three adapters, and the existing tests of each adapter pass. The cross-adapter parity tests pass too, though none of them sends an asset request. The Worker's entry point passes `cargo clippy-cloudflare-wasm` and the component's passes `cargo clippy-spin-wasm`. + +## Revision record + +| Date | Change | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-17 | First draft. | +| 2026-10-18 | Fixes from review round 1. The pull request this spec belongs to, Spin's refusal of a source's `host_header` and what Cloudflare does with one, the startup warning for a route that enables the Image Optimizer and why it is not a refusal, the `HEAD` a signed route sends and the SVG and debug requests that pass, the Axum client's refusal on `send_async`, the startup refusal of a source that cannot be built, the per-request failures, body limits and timeouts, where answers are held, the Edge Cookie an asset response neither writes nor expires, the tests of the cache rule, the error and the Edge Cookie, the open questions and the review log. Fixes from review round 2. The loggers the Cloudflare and Spin adapters now install and where their lines appear, how often each adapter logs the Image Optimizer warning, why such a route gets a warning where those adapters refuse an auction of more than one provider, the Spin `host_header` refusal as a breaking change in the summary, the problem and the rollout, the logged failures, the reader headers a route's origin receives, the rise in key-value traffic, the signed source whose `host_header` names the address's own host in other letter case, what the warning test reads, which Spin client code the native tests reach, the logger tests and what is not in this change. | +| 2026-10-27 | Held answers follow the asset cache as a module (`2026-10-15-asset-cache-design.md`). `handle_asset_proxy_request` offers a `GET` to the module `[asset-cache]` selects and keeps no copy itself, the 51Degrees module holds its copies through `[cache.51degrees-assets]` and `[cache.51degrees-assets-vary]`, and no stock build links that module or a loading cache module. | +| 2026-10-28 | Fixes from review round 3. A route keeps copies only where a deployment links and selects the asset cache and loading cache modules, the EdgeZero version, and the issue line. | +| 2026-10-28 | Fixes from review round 4. The Image Optimizer in the problem, the 51Degrees loading cache module, and a stock build keeping no copy of its own. | + +## Review log + +1. **Round 1, 2026-10-18.** A review in a fresh session, against `split/54-asset-routes` at `692ce7b8b`. Issues found, four of them blocking. The four were that the spec gave no configuration decision for a route that enables the Image Optimizer, which those adapters accepted and then answered with `502` request by request with no word at startup, that Spin drops a source's `host_header`, so a route reaches its origin with the wrong `Host` and a signature made for another, that the spec said a source that cannot be built answers per request, when it stops the adapter when the state is built, and left out the body limits, the timeouts and the per-request errors, and that no test proved the finalize step, the `no-store, private` error or that no Edge Cookie is written. Fixed in `2369fc42e`, where the three adapters warn at startup about a route that enables the Image Optimizer, `f1bba5d78`, where Spin refuses a `Host` other than the address's own, `76986f0b0` for the failure handling, and `ada05a9bf`, which tests the cache rule, the error and the Edge Cookie on each adapter. The non-blocking findings fixed are the Axum client's refusal on `send_async` in `dad986582`, the Cloudflare `Content-Length` comment in `efadc7b7e`, the API reference, the Cloudflare reason and the two designs that contradicted this one in `914800f98` and `ff7ae409a`, and the consent and privacy text and this review log in `76986f0b0`, which also states what the code does for each issue, corrects the partly true claims on where answers are held, the Image Optimizer and the Axum client, names the pull request this spec belongs to, and leaves the Cloudflare `host_header` and `HEAD` length as open questions, with a parity case, the timeouts and streaming as not in this change. +2. **Round 2, 2026-10-18.** A review in a fresh session, against `split/54-asset-routes` at `76986f0b0`, of the round 1 fixes and the new text. Issues found, two of them blocking. The two were that the Cloudflare and Spin adapters install no logger, so there the Image Optimizer warning, a startup error and the reason Spin refused a request were never written, and that Spin's refusal of a `host_header` naming another host also breaks a Spin deployment whose publisher source sets one, which the summary and Rollout 1 did not say and the changelog did not mark as breaking. Fixed in `e621e75e4`, where each of the two adapters installs a logger of its own at its entry point, which also makes the origin signing design's statement that a failed startup logs the full error true on every adapter, `9801e2279`, where the changelog gives the Spin refusal a breaking entry of its own and the guides say where each adapter's lines appear, and this revision, which states the publisher path effect in the summary, the problem and Rollout 1, how often each adapter logs the warning, and why a route that enables the Image Optimizer gets a warning where Cloudflare and Spin refuse an auction of more than one provider. The non-blocking findings fixed are the Spin `502` in the API reference's publisher fallback contract in `9801e2279`, the order of the review log in the origin signing and asset cache designs in `08122e33f`, and the reader headers a route's origin receives, the rise in key-value traffic, the signed source whose `host_header` names the address's own host in other letter case or with its default port, what the warning test reads and which Spin client code the native tests reach in this revision, which leaves a test that reads the warning's text, a startup check of `host_header` on Spin and a log level setting as not in this change. +3. **Round 3, 2026-10-28.** A review in a fresh session, against `split/54-asset-routes`, of the round 2 fixes, the held answers paragraph that follows the asset cache as a module, and the whole design against the code. Issues found, two of them blocking. The two were that the problem, the table of adapters and Rollout 2 still said that `handle_asset_proxy_request` keeps copies and that every route keeps them, when it keeps none, offers each `GET` to the asset cache module a deployment selects, and in a stock build asks the origin for every answer and adds no key-value traffic, and that the page named EdgeZero v0.0.9 as the locked version, when the main branch moved the lock to v0.0.10, whose `init_logger` still does nothing on Workers or on Spin. Fixed in this revision, which says in the problem, the table and Rollout 2 that a route keeps copies only where a deployment links and selects the 51Degrees asset cache module and a loading cache module, and names v0.0.10. The non-blocking finding fixed is the issue line, which said none where IABTechLab/trusted-server#777 lists the row as part of that issue. +4. **Round 4, 2026-10-28.** A lighter review in a fresh session, against `split/54-asset-routes`, of the round 3 fixes. Issues found, none of them blocking. The problem, the table and Rollout 2 now say that a route keeps copies only where a deployment links and selects the asset cache and loading cache modules, the page names EdgeZero v0.0.10, whose `init_logger` does nothing on Workers or on Spin, and the issue line names IABTechLab/trusted-server#777, and each was found correct against the code and the asset cache design. The non-blocking findings fixed in this revision are the problem, which said the handler offers each `GET` to the asset cache module where it leaves out a request the Image Optimizer handles, the note under the table and Rollout 2, which named a loading cache module where the stores they give are those of the 51Degrees loading cache module, and the note's statement that a stock build keeps no copy on any adapter, which now says it keeps no copy of its own. +5. **Round 5, 2026-10-28.** A lighter review in a fresh session, against `split/54-asset-routes`, of the round 4 fixes. No blocking issue and one smaller one. The problem now leaves a request the Image Optimizer handles out of the offer to the asset cache module, the note under the table and Rollout 2 name the 51Degrees loading cache module, whose stores the table and Rollout 2 give, and the note says a stock build keeps no copy of its own, and each was found correct against the code and `2026-10-15-asset-cache-design.md`. The smaller one was that the round 4 entry said the problem, the table and Rollout 2 already named the 51Degrees modules, when they named the 51Degrees asset cache module and a loading cache module, as the same entry goes on to say. This revision says that they named the asset cache and loading cache modules. diff --git a/scripts/smoke-cloudflare.sh b/scripts/smoke-cloudflare.sh index 413b49103..4570fe405 100755 --- a/scripts/smoke-cloudflare.sh +++ b/scripts/smoke-cloudflare.sh @@ -89,6 +89,7 @@ run_case() { local missing_key="$3" local expected_status="$4" local missing_binding="${5:-}" + local expected_diagnostic="${6:-}" local config_path="$CLOUDFLARE_WORK/wrangler.$case_name.toml" local log_path="$WORKSPACE/$case_name.log" local body_path="$WORKSPACE/$case_name.body" @@ -105,8 +106,6 @@ run_case() { --ip 127.0.0.1 ) >"$log_path" 2>&1 & APP_PID=$! - # The Worker installs no logger, so a negative case is checked by its - # status alone, which smoke_wait_http waits for. smoke_wait_http "http://127.0.0.1:$port/" "$expected_status" \ "$body_path" "$headers_path" smoke_assert_process_alive "$APP_PID" "Wrangler" "$log_path" @@ -115,6 +114,8 @@ run_case() { APP_PID="" if [ "$expected_status" != 200 ]; then + [ -n "$expected_diagnostic" ] || + smoke_die "negative case $case_name names no log line to wait for" grep --fixed-strings --quiet 'Your Worker has access to the following bindings:' "$log_path" || smoke_die "Wrangler did not report the runtime binding inventory" grep --fixed-strings --quiet "env.$CONFIG_BINDING" "$log_path" || @@ -122,13 +123,18 @@ run_case() { if [ -n "$missing_binding" ] && grep --fixed-strings --quiet "$missing_binding" "$log_path"; then smoke_die "negative case unexpectedly exposed $missing_binding" fi + # The Worker's log must name the missing input, so a failure of another + # kind cannot pass the case. Wrangler prints the Worker's console lines + # with its own. + smoke_assert_failure "$expected_status" "$expected_diagnostic" "$log_path" else smoke_assert_success "$body_path" "$ORIGIN_PORT" "$port" fi } # Before the push the config store holds no blob, so the Worker cannot load. -run_case missing-config "$BASE_PORT" "" 500 "" +run_case missing-config "$BASE_PORT" "" 500 "" \ + "failed to read Trusted Server app config key \`trusted_server_config\` from config store \`$CONFIG_BINDING\`" ( cd "$WORKSPACE" @@ -151,8 +157,10 @@ ENVELOPE=$( [ -n "$ENVELOPE" ] || smoke_die "Wrangler KV read-back returned an empty envelope" run_case missing-proxy "$((BASE_PORT + 1))" publisher_proxy_secret 500 \ - 'env.publisher_proxy_secret' -run_case missing-ec "$((BASE_PORT + 2))" ec_passphrase 500 'env.ec_passphrase' + 'env.publisher_proxy_secret' \ + "failed to resolve secret reference at \`publisher.proxy_secret\`" +run_case missing-ec "$((BASE_PORT + 2))" ec_passphrase 500 'env.ec_passphrase' \ + "failed to resolve secret reference at \`ec.hmac.passphrase\`" run_case positive "$((BASE_PORT + 3))" "" 200 "" echo "Cloudflare first-success smoke passed" diff --git a/scripts/smoke-spin.sh b/scripts/smoke-spin.sh index 6711c5ee7..872c95a53 100755 --- a/scripts/smoke-spin.sh +++ b/scripts/smoke-spin.sh @@ -63,6 +63,7 @@ run_case() { local port="$2" local missing_variable="$3" local expected_status="$4" + local expected_diagnostic="${5:-}" local log_path="$WORKSPACE/$case_name.log" local body_path="$WORKSPACE/$case_name.body" local headers_path="$WORKSPACE/$case_name.headers" @@ -84,8 +85,6 @@ run_case() { "${spin_args[@]}" >"$log_path" 2>&1 & APP_PID=$! - # The component installs no logger, so a negative case is checked by - # its status alone, which smoke_wait_http waits for. smoke_wait_http "http://127.0.0.1:$port/" "$expected_status" \ "$body_path" "$headers_path" smoke_assert_process_alive "$APP_PID" "Spin" "$log_path" @@ -94,6 +93,8 @@ run_case() { APP_PID="" if [ "$expected_status" != 200 ]; then + [ -n "$expected_diagnostic" ] || + smoke_die "negative case $case_name names no log line to wait for" grep --fixed-strings --quiet "Serving http://127.0.0.1:$port" "$log_path" || smoke_die "Spin launcher did not reach the requested listener" if [ "$case_name" = "missing-config" ]; then @@ -103,12 +104,20 @@ run_case() { [ "$CONFIG_PUSHED" = true ] || smoke_die "missing-secret case ran before the config push" fi + # The component's log must name the missing input, so a failure of + # another kind cannot pass the case. Spin keeps the component's + # standard error under --log-dir. + smoke_assert_failure \ + "$expected_status" \ + "$expected_diagnostic" \ + "$component_log_dir/trusted-server_stderr.txt" else smoke_assert_success "$body_path" "$ORIGIN_PORT" "$port" fi } -run_case missing-config "$BASE_PORT" "" 500 +run_case missing-config "$BASE_PORT" "" 500 \ + 'failed to read Spin Trusted Server app-config blob' EDGEZERO__STORES__CONFIG__TRUSTED_SERVER_CONFIG__NAME=default \ "$SMOKE_TS_BIN" config push \ @@ -121,8 +130,10 @@ EDGEZERO__STORES__CONFIG__TRUSTED_SERVER_CONFIG__NAME=default \ --no-diff CONFIG_PUSHED=true -run_case missing-proxy "$((BASE_PORT + 1))" "$SPIN_PROXY_VAR" 500 -run_case missing-ec "$((BASE_PORT + 2))" "$SPIN_EC_VAR" 500 +run_case missing-proxy "$((BASE_PORT + 1))" "$SPIN_PROXY_VAR" 500 \ + "resolved secret at \`publisher.proxy_secret\` must not be empty" +run_case missing-ec "$((BASE_PORT + 2))" "$SPIN_EC_VAR" 500 \ + "resolved secret at \`ec.hmac.passphrase\` must not be empty" run_case positive "$((BASE_PORT + 3))" "" 200 echo "Spin first-success smoke passed"