diff --git a/.github/workflows/format.yml b/.github/workflows/format.yml index f458572eb7..a8afcdf05d 100644 --- a/.github/workflows/format.yml +++ b/.github/workflows/format.yml @@ -39,6 +39,19 @@ jobs: exit 1 fi + check-core-platform-names: + name: core platform names guard + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + # Core is built on EdgeZero and names no hosting platform or platform + # domain, in code, comments, tests, documents or file names, so one + # platform's change is never a change in core. Each adapter names its + # own platform. + - name: Check that crates/trusted-server-core names no hosting platform + run: python3 scripts/lint-core-platform-names.py + format-rust: name: cargo fmt runs-on: ubuntu-latest diff --git a/AGENTS.md b/AGENTS.md index b1f641b043..a13559dcba 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -496,6 +496,7 @@ Every PR must pass: 9. Native core build-digest test and lint (`cargo test -p trusted-server-core --test template_build_digest` and `cargo clippy -p trusted-server-core --test template_build_digest -- -D warnings`) 10. Wasm clock lint: `scripts/lint-wasm-clock.sh` (crates built into the Cloudflare Worker must use `web_time`, not the `std::time::{Instant,SystemTime}` `now`/`elapsed` methods, which panic there) 11. Documentation proposal scripts: `shellcheck scripts/docs-proposal/*.sh && node --test scripts/docs-proposal/hunks.test.mjs && scripts/docs-proposal/test.sh` +12. Core platform names: `python3 scripts/lint-core-platform-names.py` (no file in `crates/trusted-server-core` names a hosting platform or one of its domains, in code, comments, tests, documents or file names) --- diff --git a/CHANGELOG.md b/CHANGELOG.md index fde797b7d0..f31b37193d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- **Breaking:** the platform's own device module is selected with `[device] module = "platform"`, as `[geo] module = "platform"` selects the platform's geo lookup, where it was `module = "fastly"`. A document that still writes `fastly` is refused at startup as naming a device module no running module supplies. **For crates outside core**, `build_device_module` takes a `build_platform` factory. +- **Breaking for crates outside core:** an adapter declares the request headers its platform sets. `platform::declare_platform_tls_headers` names a header the platform sets on a request that arrived over TLS, which core reads for the scheme and never forwards to an origin. `platform::declare_platform_spoofable_headers` names the platform's other headers a client could also send, which `sanitize_forwarded_headers` strips at entry with the standard forwarding headers, in an adapter that calls it. `platform::declare_platform_internal_headers` names headers that are never passed to a third party. The Fastly adapter declares `fastly-ssl`, `fastly-client-ip` and `x-debug-fastly-pop` before it serves. `SPOOFABLE_FORWARDED_HEADERS` holds the three standard forwarding headers, `HEADER_X_DEBUG_FASTLY_POP` leaves core, and `constants::is_internal_header` answers for the fixed list and the declared headers together. A `[trusted_client_ip]` `ip_header` may start with `x-` or end with `-client-ip`, so `fastly-client-ip` stays valid. On the Axum, Cloudflare and Spin adapters, which declare none of these headers, a client's `fastly-ssl` is no longer removed from the request to the publisher's origin, and on Spin a client's `fastly-ssl` and `fastly-client-ip` are no longer stripped at entry. On those adapters the `x-debug-fastly-pop` header from a client is also no longer kept from third parties, because `INTERNAL_HEADERS` no longer holds it, and is no longer refused as a `[trusted_client_ip]` header name. On the Axum and Cloudflare adapters a client's `fastly-ssl` also no longer makes the request HTTPS. On Fastly, `ts config validate`, which runs with no adapter's declarations, no longer refuses `x-debug-fastly-pop` as a `[trusted_client_ip]` header name, and the service refuses it when it loads the document. +- **Breaking for crates outside core:** an adapter tells core whether its platform runs the service in a staging environment with `platform::declare_platform_staging`, and a module asks `platform::platform_is_staging`. `ENV_FASTLY_IS_STAGING` leaves core, the Fastly adapter reads `FASTLY_IS_STAGING` itself, and the DataDome module's test bypass follows the declaration. On the Axum, Cloudflare and Spin adapters, which declare no staging environment, the bypass no longer follows a `FASTLY_IS_STAGING` variable. +- **Breaking for crates outside core:** a settings document too long for one config store entry is put back together by the platform's config store, which keeps it in parts. The Fastly adapter's `PlatformConfigStore` follows the chunk pointer, reads each chunk and checks every length and hash, where core's settings loader recognized the pointer and read the chunks itself. A deployment's documents read as before. A `PlatformConfigStore` an adapter implements returns each entry whole. +- Core names no hosting platform or platform domain in any line, its comments, tests and README files included. Comments and tests describe what a platform does without naming one, the test that core never uses the Fastly SDK and the test of the Fastly KV Store's metadata limit move to the Fastly adapter, and `scripts/lint-core-platform-names.py`, which the Run Format workflow runs, fails when a file under `crates/trusted-server-core` names a hosting platform, a tool named for one or a domain a platform serves deployments from. - **Breaking for a build that links its own crates:** a source's `auth` table names an origin signer a crate offers. Core keeps the interface, `OriginSigner` in `trusted_server_core::origin_auth`, and signs the publisher's origin requests and each asset route's through it where it signed before. `type` names the signer, `origin_query` is read by core, and the rest of the table is the signer's. The `s3_sigv4` signer moves to `crates/shared/aws`, keeping its settings, defaults and secret-store key references, and every adapter and the `ts` tool link the crate's builder, so a stock build still offers it. A build that composes its own modules adds `trusted_server_shared_aws::builder()`, or a `type` of `s3_sigv4` is refused at startup naming the signers on offer. A table the signer cannot use is refused at startup and by `ts config validate` naming the setting. The deprecated spelling `s3_sig_v4` is no longer read. **For crates outside core**, `AssetOriginAuth` and `S3SigV4AuthConfig` are replaced by `OriginAuth`, `SourceLocation::auth` gives an `OriginSigning`, `OriginSettings::location` and `OriginSource::new` take the signers on offer and the setting's name, and a crate offers a signer with `IntegrationBuilder::with_origin_signer`. - **Breaking for crates outside core:** an adapter answers how it names a backend and bounds its transport timers through `PlatformBackend` alone, which gains `auction_backend_budget` and `transport_timeout_bucket_count` with defaults, and describes itself to auction plan validation through an `AuctionTargetDescriptor` it builds from its name, its backend implementation and its `AuctionTargetCapabilities`. `AuctionPlan::validate_for_target` takes that descriptor, and `BackendNamingPolicy`, `AuctionTargetId`, `PredictedBackend`, `BackendNamingError` and `PlatformBackend::naming_policy` are gone from core, because core names no hosting platform. Each adapter carries its own naming policy beside the code that registers its backends, and the names a deployment sees are unchanged. - Core names no platform's cache header. It removes from a response it makes private only `Surrogate-Control`, `CDN-Cache-Control` and a platform header it wrote its own policy in, so on every adapter `fastly-surrogate-control` and `cloudflare-cdn-cache-control` stay where an origin or an operator wrote them, until a fetch middleware removes them, which none in the stack does yet. The Fastly and Cloudflare adapters declare their own platform's header, which keeps an origin response with a policy in it out of the stored copies on that adapter alone, and each adapter writes its policy in the header it wrote before. **For crates outside core**, `EdgeCacheHeader::CloudflareCdnCacheControl` is `EdgeCacheHeader::Platform(name)`, `EDGE_CACHE_HEADER_NAMES` lists only the two standard headers, the two platform header constants leave core, an adapter declares its platform's headers with `platform::declare_platform_cache_headers`, and a policy written with `CachePolicy::apply_to_response` is removed by `cache_policy::remove_owned_cache_headers` when the response is made private. diff --git a/Cargo.lock b/Cargo.lock index 0547e2226a..1a1357a8e7 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -6738,6 +6738,7 @@ dependencies = [ "futures", "log", "log-fastly", + "regex", "serde", "serde_json", "sha2 0.11.1", @@ -6992,7 +6993,6 @@ dependencies = [ "serde_json", "sha2 0.11.1", "subtle", - "temp-env", "toml", "trusted-server-core", "url", diff --git a/crates/audience/permutive/src/lib.rs b/crates/audience/permutive/src/lib.rs index 12b1a46b6d..9359227b92 100644 --- a/crates/audience/permutive/src/lib.rs +++ b/crates/audience/permutive/src/lib.rs @@ -26,7 +26,7 @@ use http::{Method, StatusCode}; use serde::Deserialize; use validator::Validate; -use trusted_server_core::constants::INTERNAL_HEADERS; +use trusted_server_core::constants::is_internal_header; use trusted_server_core::error::TrustedServerError; use trusted_server_core::integrations::{ AttributeRewriteAction, INTEGRATION_MAX_BODY_BYTES, IntegrationEndpoint, IntegrationProxy, @@ -312,7 +312,7 @@ impl PermutiveIntegration { // Copy any X-* custom headers, skipping TS-internal headers for (name, value) in from { let name_str = name.as_str(); - if name_str.starts_with("x-") && !INTERNAL_HEADERS.contains(&name_str) { + if name_str.starts_with("x-") && !is_internal_header(name_str) { to.append(name.clone(), value.clone()); } } diff --git a/crates/bot-protection/datadome/Cargo.toml b/crates/bot-protection/datadome/Cargo.toml index 026e775ba2..8b43b912a3 100644 --- a/crates/bot-protection/datadome/Cargo.toml +++ b/crates/bot-protection/datadome/Cargo.toml @@ -33,7 +33,6 @@ web-time = { workspace = true } [dev-dependencies] futures = { workspace = true } -temp-env = { workspace = true } toml = { workspace = true } trusted-server-core = { workspace = true, features = ["test-utils"] } diff --git a/crates/bot-protection/datadome/src/lib.rs b/crates/bot-protection/datadome/src/lib.rs index 0e8f982e07..6906d858ce 100644 --- a/crates/bot-protection/datadome/src/lib.rs +++ b/crates/bot-protection/datadome/src/lib.rs @@ -83,7 +83,6 @@ use serde_json::Value as JsonValue; use url::Url; use validator::Validate; -use trusted_server_core::constants::ENV_FASTLY_IS_STAGING; use trusted_server_core::error::TrustedServerError; use trusted_server_core::integrations::{ AttributeRewriteAction, INTEGRATION_MAX_BODY_BYTES, IntegrationDocumentState, @@ -568,7 +567,7 @@ impl DataDomeIntegration { } fn active_protection_test_bypass(&self) -> Option<&ProtectionTestBypassConfig> { - if std::env::var(ENV_FASTLY_IS_STAGING).as_deref() != Ok("1") { + if !trusted_server_core::platform::platform_is_staging() { return None; } @@ -1100,7 +1099,8 @@ fn build( let protection_test_bypass_active = integration.active_protection_test_bypass().is_some(); if protection_test_bypass_configured && !protection_test_bypass_active { log::warn!( - "[datadome] DataDome test bypass is configured but inactive because FASTLY_IS_STAGING is not 1" + "[datadome] DataDome test bypass is configured but inactive because the platform \ + does not run this service in its staging environment" ); } log::info!( diff --git a/crates/bot-protection/datadome/src/protection.rs b/crates/bot-protection/datadome/src/protection.rs index cd78c139b6..7a1de6117c 100644 --- a/crates/bot-protection/datadome/src/protection.rs +++ b/crates/bot-protection/datadome/src/protection.rs @@ -836,7 +836,24 @@ mod tests { use super::*; - static FASTLY_IS_STAGING_ENV_LOCK: Mutex<()> = Mutex::new(()); + static PLATFORM_STAGING_LOCK: Mutex<()> = Mutex::new(()); + + /// Runs `run` with the platform declared as staging or not, putting the + /// declaration back afterwards, even when `run` panics. + fn with_platform_staging(staging: bool, run: impl FnOnce() -> T) -> T { + struct Restore(bool); + impl Drop for Restore { + fn drop(&mut self) { + trusted_server_core::platform::declare_platform_staging(self.0); + } + } + let _guard = PLATFORM_STAGING_LOCK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + let _restore = Restore(trusted_server_core::platform::platform_is_staging()); + trusted_server_core::platform::declare_platform_staging(staging); + run() + } fn protection_config() -> DataDomeConfig { DataDomeConfig { @@ -864,25 +881,16 @@ mod tests { services: &RuntimeServices, request: &mut Request, ) -> RequestFilterDecision { - let _guard = FASTLY_IS_STAGING_ENV_LOCK - .lock() - .expect("should lock staging environment test guard"); - temp_env::with_var( - trusted_server_core::constants::ENV_FASTLY_IS_STAGING, - Some("1"), - || { - futures::executor::block_on(integration.filter_protection_request( - RequestFilterInput { - settings, - services, - request, - geo_info: None, - permissions: None, - is_integration_route: false, - }, - )) - }, - ) + with_platform_staging(true, || { + futures::executor::block_on(integration.filter_protection_request(RequestFilterInput { + settings, + services, + request, + geo_info: None, + permissions: None, + is_integration_route: false, + })) + }) } fn filter_marks_request( @@ -1137,25 +1145,16 @@ mod tests { edgezero_core::http::HeaderValue::from_static("temporary-test-credential-32-bytes!"), ); - let _guard = FASTLY_IS_STAGING_ENV_LOCK - .lock() - .expect("should lock staging environment test guard"); - let decision = temp_env::with_var( - trusted_server_core::constants::ENV_FASTLY_IS_STAGING, - None::<&str>, - || { - futures::executor::block_on(integration.filter_protection_request( - RequestFilterInput { - settings: &settings, - services: &services, - request: &mut request, - geo_info: None, - permissions: None, - is_integration_route: false, - }, - )) - }, - ); + let decision = with_platform_staging(false, || { + futures::executor::block_on(integration.filter_protection_request(RequestFilterInput { + settings: &settings, + services: &services, + request: &mut request, + geo_info: None, + permissions: None, + is_integration_route: false, + })) + }); assert!( matches!(decision, RequestFilterDecision::Continue(_)), diff --git a/crates/device/README.md b/crates/device/README.md index 865cb9671f..3a0741ed5c 100644 --- a/crates/device/README.md +++ b/crates/device/README.md @@ -6,5 +6,5 @@ HTTP/2 signals. Future vendor modules (for example `crates/device/`) slot in alongside it. The built-in default module (User-Agent only) ships in `trusted-server-core` -(`ec::device`). Adapters select and inject the vendor module via -`build_device_module`. +(`ec::device`). An adapter injects its platform's own module through +`build_device_module`, which builds it when `[device] module = "platform"`. diff --git a/crates/device/fastly/src/lib.rs b/crates/device/fastly/src/lib.rs index 3a71e493a5..01914df262 100644 --- a/crates/device/fastly/src/lib.rs +++ b/crates/device/fastly/src/lib.rs @@ -2,7 +2,7 @@ //! //! [`FastlyDeviceModule`] strengthens the built-in User-Agent classification //! with the host's TLS (JA4) and HTTP/2 signals, for deployments on Fastly -//! Compute. It is selected by `[device] module = "fastly"` and wired in by the +//! Compute. It is selected by `[device] module = "platform"` and wired in by the //! Fastly adapter, which injects the captured host signals and hands the module //! the request's module context. //! @@ -66,7 +66,7 @@ impl HostSignals for FastlyHostSignals { } } -/// The Fastly device module, opt-in via `[device] module = "fastly"`. +/// The Fastly device module, opt-in via `[device] module = "platform"`. /// /// Classifies a request with [`DeviceSignals::derive`], which strengthens the /// User-Agent classification with the host signals. It reads the User-Agent diff --git a/crates/identity/lockr/src/lib.rs b/crates/identity/lockr/src/lib.rs index 22fcf1b0ab..4e506337b7 100644 --- a/crates/identity/lockr/src/lib.rs +++ b/crates/identity/lockr/src/lib.rs @@ -30,7 +30,7 @@ use http::{Method, StatusCode}; use serde::Deserialize; use validator::Validate; -use trusted_server_core::constants::INTERNAL_HEADERS; +use trusted_server_core::constants::is_internal_header; use trusted_server_core::error::TrustedServerError; use trusted_server_core::integrations::{ AttributeRewriteAction, INTEGRATION_MAX_BODY_BYTES, IntegrationEndpoint, IntegrationProxy, @@ -322,7 +322,7 @@ impl LockrIntegration { for (name, value) in from { let name_str = name.as_str(); - if name_str.starts_with("x-") && !INTERNAL_HEADERS.contains(&name_str) { + if name_str.starts_with("x-") && !is_internal_header(name_str) { to.append(name.clone(), value.clone()); } } diff --git a/crates/trusted-server-adapter-fastly/Cargo.toml b/crates/trusted-server-adapter-fastly/Cargo.toml index fa51bc0a1c..f682ad1f44 100644 --- a/crates/trusted-server-adapter-fastly/Cargo.toml +++ b/crates/trusted-server-adapter-fastly/Cargo.toml @@ -68,6 +68,7 @@ trusted-server-geo-fastly = { workspace = true } [dev-dependencies] bytes = { workspace = true } edgezero-core = { workspace = true, features = ["test-utils"] } +regex = { workspace = true } toml = { workspace = true } trusted-server-core = { workspace = true, features = ["test-utils"] } # Test fixture only. It proves the integration seam on this adapter from diff --git a/crates/trusted-server-adapter-fastly/build.rs b/crates/trusted-server-adapter-fastly/build.rs new file mode 100644 index 0000000000..bd422fceba --- /dev/null +++ b/crates/trusted-server-adapter-fastly/build.rs @@ -0,0 +1,76 @@ +//! Build script for `trusted-server-adapter-fastly`: lists core's Rust sources +//! for the test that checks core never uses the Fastly SDK. + +#![allow( + clippy::print_stdout, + reason = "Cargo build scripts communicate rebuild inputs on stdout" +)] + +use std::env; +use std::fs; +use std::path::{Path, PathBuf}; + +fn main() { + let manifest_dir = + PathBuf::from(env::var("CARGO_MANIFEST_DIR").expect("should read CARGO_MANIFEST_DIR")); + let src_dir = manifest_dir + .parent() + .expect("should have the crates folder above this crate") + .join("trusted-server-core") + .join("src"); + + // Cargo walks a directory named here, so adding, renaming or deleting a + // core source file reruns this script and the list follows the tree. + println!("cargo:rerun-if-changed=build.rs"); + println!("cargo:rerun-if-changed={}", path_literal(&src_dir)); + + // A checkout without core's sources builds with an empty list, which the + // test that pins the list to core's tree refuses. + let mut sources = Vec::new(); + if src_dir.is_dir() { + collect_rust_sources(&src_dir, &src_dir, &mut sources); + } + sources.sort(); + + let mut generated = String::from( + "// Generated by build.rs. Every Rust source file under core's `src`.\n\ + const CHECKED_SOURCES: &[(&str, &str)] = &[\n", + ); + for relative in &sources { + let absolute = format!("{}/{relative}", path_literal(&src_dir)); + generated.push_str(&format!( + " (\"{relative}\", include_str!(\"{absolute}\")),\n" + )); + } + generated.push_str("];\n"); + + let out_dir = PathBuf::from(env::var("OUT_DIR").expect("should read OUT_DIR")); + fs::write(out_dir.join("core_sources.rs"), generated) + .expect("should write the core source list"); +} + +/// Collects every `.rs` file under `dir`, as a path relative to `root` with +/// forward slashes, so the generated file reads the same on every platform. +fn collect_rust_sources(root: &Path, dir: &Path, sources: &mut Vec) { + let entries = fs::read_dir(dir).expect("should read a source directory"); + for entry in entries { + let entry = entry.expect("should read a source directory entry"); + let path = entry.path(); + if path.is_dir() { + collect_rust_sources(root, &path, sources); + continue; + } + if path.extension().is_some_and(|extension| extension == "rs") { + let relative = path + .strip_prefix(root) + .expect("should strip the source root prefix"); + sources.push(path_literal(relative)); + } + } +} + +/// Renders a path for a Rust string literal, with forward slashes so a Windows +/// path needs no escaping. +fn path_literal(path: &Path) -> String { + path.display().to_string().replace('\\', "/") +} diff --git a/crates/trusted-server-adapter-fastly/src/app.rs b/crates/trusted-server-adapter-fastly/src/app.rs index fbfa3e5ed9..957690c1dd 100644 --- a/crates/trusted-server-adapter-fastly/src/app.rs +++ b/crates/trusted-server-adapter-fastly/src/app.rs @@ -309,6 +309,43 @@ pub const FASTLY_SURROGATE_CONTROL_HEADER: &str = "fastly-surrogate-control"; /// The header core writes a shared cache policy in on this adapter. pub(crate) const EDGE_CACHE_HEADER: EdgeCacheHeader = EdgeCacheHeader::SurrogateControl; +/// The header the entry point sets to `1` on a request that arrived over TLS, +/// after stripping any a client sent. Core reads it for the scheme and never +/// forwards it to an origin. +pub(crate) const FASTLY_TLS_HEADER: &str = "fastly-ssl"; + +/// The header Fastly carries the client's address in, which a client could +/// also send, so it is stripped at entry. +pub(crate) const FASTLY_CLIENT_IP_HEADER: &str = "fastly-client-ip"; + +/// A Fastly debugging header that is never passed on to a third party. +const FASTLY_DEBUG_POP_HEADER: &str = "x-debug-fastly-pop"; + +/// The variable Compute sets to `1` for a version on its staging network. +const FASTLY_IS_STAGING: &str = "FASTLY_IS_STAGING"; + +/// Whether Compute runs this version on its staging network. +pub(crate) fn fastly_is_staging() -> bool { + std::env::var(FASTLY_IS_STAGING).is_ok_and(|value| value == "1") +} + +/// Tells core which headers Compute sets on requests and reads from +/// responses, and whether this version runs on its staging network. +/// +/// [`crate::run`] calls this before serving, ahead of the first settings +/// validation, which refuses an internal header the platform declared. Every +/// state build calls it too, so a state built on its own is declared the +/// same. Declaring twice changes nothing. +pub(crate) fn declare_platform() { + use trusted_server_core::platform as core_platform; + + core_platform::declare_platform_cache_headers(&[FASTLY_SURROGATE_CONTROL_HEADER]); + core_platform::declare_platform_staging(fastly_is_staging()); + core_platform::declare_platform_tls_headers(&[FASTLY_TLS_HEADER]); + core_platform::declare_platform_spoofable_headers(&[FASTLY_CLIENT_IP_HEADER]); + core_platform::declare_platform_internal_headers(&[FASTLY_DEBUG_POP_HEADER]); +} + /// Build the application state from explicit settings, composing the modules /// this adapter links with the externally supplied builders in /// `integrations`. @@ -329,9 +366,7 @@ pub(crate) fn build_state_with_registrations( settings: Settings, integrations: &[IntegrationBuilder], ) -> Result, Report> { - trusted_server_core::platform::declare_platform_cache_headers(&[ - FASTLY_SURROGATE_CONTROL_HEADER, - ]); + declare_platform(); warn_if_certificate_check_disabled(&settings); // The modules this adapter links come first and the deployment's own @@ -410,7 +445,7 @@ fn fastly_platform_identity() -> PlatformIdentity { name: "fastly".to_owned(), service_id: read("FASTLY_SERVICE_ID"), service_version: read("FASTLY_SERVICE_VERSION"), - staging: std::env::var("FASTLY_IS_STAGING").is_ok_and(|value| value == "1"), + staging: fastly_is_staging(), pop: read("FASTLY_POP"), } } diff --git a/crates/trusted-server-adapter-fastly/src/compat.rs b/crates/trusted-server-adapter-fastly/src/compat.rs index 5ccd9c423a..5fab039112 100644 --- a/crates/trusted-server-adapter-fastly/src/compat.rs +++ b/crates/trusted-server-adapter-fastly/src/compat.rs @@ -69,7 +69,11 @@ pub(crate) fn sanitize_fastly_forwarded_headers( req.remove_header(config.auth_header.as_str()); } - for &name in SPOOFABLE_FORWARDED_HEADERS { + let own = [ + crate::app::FASTLY_TLS_HEADER, + crate::app::FASTLY_CLIENT_IP_HEADER, + ]; + for &name in SPOOFABLE_FORWARDED_HEADERS.iter().chain(&own) { if req.get_header(name).is_some() { log::debug!("Stripped spoofable header: {name}"); req.remove_header(name); diff --git a/crates/trusted-server-adapter-fastly/src/config_chunks.rs b/crates/trusted-server-adapter-fastly/src/config_chunks.rs new file mode 100644 index 0000000000..3ceb456772 --- /dev/null +++ b/crates/trusted-server-adapter-fastly/src/config_chunks.rs @@ -0,0 +1,277 @@ +//! A document too long for one Fastly config store entry, read back whole. +//! +//! A Fastly config store entry holds at most 8,000 characters. A longer +//! document is written as chunks under keys of their own, with a pointer at +//! the document's key naming each chunk's key, length and hash and the whole +//! document's length and hash. [`whole_entry`] follows such a pointer and +//! checks every length and hash, so the rest of the service only ever sees +//! whole documents. + +use core::fmt::Write as _; + +use error_stack::Report; +use serde::Deserialize; +use sha2::{Digest as _, Sha256}; +use trusted_server_core::platform::PlatformError; + +/// The `edgezero_kind` a chunk pointer carries. +const CHUNK_POINTER_KIND: &str = "fastly_config_chunks"; + +/// The most characters one Fastly config store entry holds. +const ENTRY_LIMIT: usize = 8_000; + +#[derive(Debug, Deserialize)] +struct ChunkPointer { + chunks: Vec, + edgezero_kind: String, + envelope_len: usize, + envelope_sha256: String, + version: u8, +} + +#[derive(Debug, Deserialize)] +struct ChunkRef { + key: String, + len: usize, + sha256: String, +} + +/// `value`, as read at a key, or the document its chunks hold when it is a +/// chunk pointer, each chunk read through `read`. +/// +/// # Errors +/// +/// When the pointer names a version this does not read, when it or a chunk +/// is longer than an entry can be, when a chunk cannot be read, or when any +/// length or hash disagrees with the pointer. +pub(crate) fn whole_entry( + value: String, + read: impl Fn(&str) -> Result>, +) -> Result> { + let Ok(pointer) = serde_json::from_str::(&value) else { + return Ok(value); + }; + if pointer.edgezero_kind != CHUNK_POINTER_KIND { + return Ok(value); + } + if pointer.version != 1 { + return refused(format!( + "unsupported Fastly config chunk pointer version {}; expected 1", + pointer.version + )); + } + if value.len() > ENTRY_LIMIT { + return refused(format!( + "Fastly config chunk pointer is {} bytes, exceeding the {ENTRY_LIMIT} byte entry \ + limit", + value.len() + )); + } + + let mut declared_len = 0usize; + for chunk in &pointer.chunks { + if chunk.len > ENTRY_LIMIT { + return refused(format!( + "Fastly config chunk `{}` declares {} bytes, exceeding the {ENTRY_LIMIT} byte \ + entry limit", + chunk.key, chunk.len + )); + } + declared_len = match declared_len.checked_add(chunk.len) { + Some(total) => total, + None => return refused("Fastly config chunk lengths overflowed usize".to_string()), + }; + } + if declared_len != pointer.envelope_len { + return refused(format!( + "Fastly config chunk lengths total mismatch: expected envelope length {}, got \ + {declared_len}", + pointer.envelope_len + )); + } + + let mut envelope = String::with_capacity(pointer.envelope_len); + for chunk in pointer.chunks { + let chunk_value = read(&chunk.key)?; + if chunk_value.len() != chunk.len { + return refused(format!( + "Fastly config chunk `{}` length mismatch: expected {}, got {}", + chunk.key, + chunk.len, + chunk_value.len() + )); + } + if envelope.len().saturating_add(chunk_value.len()) > pointer.envelope_len { + return refused(format!( + "Fastly config envelope exceeded declared length {} while reading chunk `{}`", + pointer.envelope_len, chunk.key + )); + } + let chunk_sha = sha256_hex(chunk_value.as_bytes()); + if chunk_sha != chunk.sha256 { + return refused(format!( + "Fastly config chunk `{}` sha mismatch: expected {}, got {chunk_sha}", + chunk.key, chunk.sha256 + )); + } + envelope.push_str(&chunk_value); + } + + if envelope.len() != pointer.envelope_len { + return refused(format!( + "Fastly config envelope length mismatch: expected {}, got {}", + pointer.envelope_len, + envelope.len() + )); + } + let envelope_sha = sha256_hex(envelope.as_bytes()); + if envelope_sha != pointer.envelope_sha256 { + return refused(format!( + "Fastly config envelope sha mismatch: expected {}, got {envelope_sha}", + pointer.envelope_sha256 + )); + } + Ok(envelope) +} + +fn sha256_hex(bytes: &[u8]) -> String { + let digest = Sha256::digest(bytes); + let mut hex = String::with_capacity(digest.len() * 2); + for byte in digest { + write!(hex, "{byte:02x}").expect("should write hex digit to string"); + } + hex +} + +fn refused(message: String) -> Result> { + Err(Report::new(PlatformError::ConfigStore).attach(message)) +} + +#[cfg(test)] +mod tests { + use std::collections::BTreeMap; + + use serde_json::json; + + use super::*; + + const KEY: &str = "trusted_server_config"; + + fn reader( + entries: BTreeMap, + ) -> impl Fn(&str) -> Result> { + move |key: &str| { + entries.get(key).cloned().ok_or_else(|| { + Report::new(PlatformError::ConfigStore).attach(format!("missing key `{key}`")) + }) + } + } + + fn pointer_for(document: &str, chunks: &[(&str, &str)]) -> String { + json!({ + "edgezero_kind": CHUNK_POINTER_KIND, + "version": 1, + "envelope_sha256": sha256_hex(document.as_bytes()), + "envelope_len": document.len(), + "chunks": chunks.iter().map(|(key, piece)| json!({ + "key": key, + "sha256": sha256_hex(piece.as_bytes()), + "len": piece.len(), + })).collect::>(), + }) + .to_string() + } + + #[test] + fn a_document_that_fits_is_returned_as_it_is() { + let document = r#"{"data":{},"sha256":"x"}"#.to_owned(); + let read = reader(BTreeMap::new()); + assert_eq!( + whole_entry(document.clone(), read).expect("should pass it through"), + document + ); + } + + #[test] + fn a_chunked_document_is_read_back_whole() { + let document = r#"{"data":{"publisher":{"domain":"example.com"}},"sha256":"x"}"#; + let (first, second) = document.split_at(document.len() / 2); + let first_key = format!("{KEY}.__edgezero_chunks.test.0"); + let second_key = format!("{KEY}.__edgezero_chunks.test.1"); + let pointer = pointer_for(document, &[(&first_key, first), (&second_key, second)]); + let read = reader(BTreeMap::from([ + (first_key, first.to_owned()), + (second_key, second.to_owned()), + ])); + + assert_eq!( + whole_entry(pointer, read).expect("should reassemble the document"), + document + ); + } + + #[test] + fn a_pointer_whose_lengths_disagree_with_the_document_is_refused() { + let chunk_key = format!("{KEY}.__edgezero_chunks.test.0"); + let pointer = json!({ + "edgezero_kind": CHUNK_POINTER_KIND, + "version": 1, + "envelope_sha256": sha256_hex(b"ab"), + "envelope_len": 1, + "chunks": [{"key": chunk_key, "sha256": sha256_hex(b"ab"), "len": 2}], + }) + .to_string(); + + let error = + whole_entry(pointer, reader(BTreeMap::new())).expect_err("should refuse the pointer"); + assert!( + format!("{error:?}").contains("chunk lengths total mismatch"), + "{error:?}" + ); + } + + #[test] + fn a_chunk_whose_hash_disagrees_is_refused() { + let document = "abcd"; + let chunk_key = format!("{KEY}.__edgezero_chunks.test.0"); + let pointer = pointer_for(document, &[(&chunk_key, document)]); + let read = reader(BTreeMap::from([(chunk_key, "abce".to_owned())])); + + let error = whole_entry(pointer, read).expect_err("should refuse the chunk"); + assert!(format!("{error:?}").contains("sha mismatch"), "{error:?}"); + } + + #[test] + fn another_kind_of_json_at_the_key_is_returned_as_it_is() { + let value = r#"{"edgezero_kind":"something_else","version":1,"chunks":[],"envelope_len":0,"envelope_sha256":""}"#.to_owned(); + assert_eq!( + whole_entry(value.clone(), reader(BTreeMap::new())).expect("should pass it through"), + value + ); + } + + #[test] + fn a_written_document_is_read_back_whole() { + // What `ts config push --local --key a.publisher.example` wrote for a + // document too long for one entry, in the order it wrote them. + let written: Vec<(String, String)> = + serde_json::from_str(include_str!("fixtures/config-push-chunks.json")) + .expect("should parse the fixture"); + assert!(written.len() > 2, "the fixture should hold chunks"); + let (key, pointer) = written.last().cloned().expect("should hold the pointer"); + assert_eq!( + key, "a.publisher.example", + "the pointer should be written last" + ); + + let document = whole_entry(pointer, reader(written.into_iter().collect())) + .expect("should read the written document back whole"); + + let envelope: serde_json::Value = + serde_json::from_str(&document).expect("should read back the envelope"); + assert_eq!( + envelope["data"]["publisher"]["domain"], "a.publisher.example", + "should reassemble the pushed app config" + ); + } +} diff --git a/crates/trusted-server-core/src/migration_guards.rs b/crates/trusted-server-adapter-fastly/src/core_sdk_guard.rs similarity index 64% rename from crates/trusted-server-core/src/migration_guards.rs rename to crates/trusted-server-adapter-fastly/src/core_sdk_guard.rs index 59deccec97..5d0f405ce8 100644 --- a/crates/trusted-server-core/src/migration_guards.rs +++ b/crates/trusted-server-adapter-fastly/src/core_sdk_guard.rs @@ -1,3 +1,9 @@ +//! Checks from outside core that core's code never uses the Fastly SDK. +//! +//! This crate's build script lists every Rust source file under core's `src` +//! and embeds each one by path, so a file added to or removed from core joins +//! or leaves the check on its own. + // Strips lines whose first non-whitespace token is `//`. // // Known limitations (both produce false positives, never false negatives): @@ -7,10 +13,10 @@ // inside a block comment causes a spurious failure; one hidden *outside* a // block comment is still caught by the non-comment portions of the line. // -// False positives are safe for a guard test — they cause a noisy failure that -// forces investigation rather than letting a real regression slip through +// False positives are safe for a guard test, because they cause a noisy failure +// that forces investigation rather than letting a real regression slip through // silently. False negatives are not possible with the current banned-pattern -// list because none of the migrated files use block comments in practice. +// list because none of core's files use block comments in practice. fn strip_line_comments(source: &str) -> String { source .lines() @@ -27,15 +33,11 @@ fn fastly_sdk_pattern() -> regex::Regex { .expect("should compile migration guard regex") } -// The list of files this guard checks is generated by `build.rs` from every -// `.rs` file under `src`, so a file added to core joins the guard on its own -// and a file that leaves core (a vendor integration moving to its own crate, -// for instance) leaves the guard without anyone editing this file. The -// generated file defines `CHECKED_SOURCES`. -// -// `migration_guards.rs` itself is left out, because the banned pattern appears -// here as the regex literal the guard matches with. -include!(concat!(env!("OUT_DIR"), "/migration_guard_sources.rs")); +// The list of files this guard checks is generated by this crate's `build.rs` +// from every `.rs` file under core's `src`, so a file added to core joins the +// guard on its own and a file that leaves core leaves the guard without anyone +// editing this file. The generated file defines `CHECKED_SOURCES`. +include!(concat!(env!("OUT_DIR"), "/core_sources.rs")); fn checked_sources() -> &'static [(&'static str, &'static str)] { CHECKED_SOURCES @@ -49,7 +51,7 @@ fn core_modules_do_not_reference_the_fastly_sdk() { let uncommented = strip_line_comments(source); assert!( !banned.is_match(&uncommented), - "{path} should not reference the Fastly SDK — core is platform-agnostic" + "{path} should not reference the Fastly SDK, because core names no platform" ); } } @@ -58,7 +60,7 @@ fn core_modules_do_not_reference_the_fastly_sdk() { /// guard above passing while checking nothing, so this pins the list to the /// tree it is meant to cover. #[test] -fn the_generated_list_covers_the_crate() { +fn the_generated_list_covers_core() { let sources = checked_sources(); assert!( @@ -76,10 +78,4 @@ fn the_generated_list_covers_the_crate() { .any(|&(path, _)| path == "integrations/registry.rs"), "should check files in subdirectories" ); - assert!( - !sources - .iter() - .any(|&(path, _)| path == "migration_guards.rs"), - "should leave out the guard's own source, which holds the banned pattern" - ); } diff --git a/crates/trusted-server-adapter-fastly/src/ec_kv.rs b/crates/trusted-server-adapter-fastly/src/ec_kv.rs index c427ede96b..e4db6f91b7 100644 --- a/crates/trusted-server-adapter-fastly/src/ec_kv.rs +++ b/crates/trusted-server-adapter-fastly/src/ec_kv.rs @@ -193,11 +193,16 @@ impl EcKvStore for FastlyEcKvStore { mod tests { use std::time::Duration; + use trusted_server_core::ec::kv_types::{KvMetadata, SCHEMA_VERSION}; + use super::*; /// KV store declared for the local simulator in `fastly.toml`. const TEST_STORE: &str = "ec_identity_store"; + /// The most metadata the Fastly KV Store keeps with one entry. + const KV_METADATA_LIMIT_BYTES: usize = 2048; + /// Entry metadata. Opaque to the backend, which only round-trips bytes. const METADATA: &str = "entry-metadata"; @@ -496,4 +501,24 @@ mod tests { backend.delete(key).expect("should delete the entry"); } } + + #[test] + fn identity_graph_metadata_fits_the_store_metadata_limit() { + // Worst case: all fields populated. + let meta = KvMetadata { + ok: false, + country: "XX".to_owned(), + v: SCHEMA_VERSION, + cluster_size: Some(u32::MAX), + is_mobile: Some(2), + known_browser: Some(true), + }; + let json = serde_json::to_string(&meta).expect("should serialize KvMetadata"); + assert!( + json.len() <= KV_METADATA_LIMIT_BYTES, + "metadata must fit in the Fastly KV Store's {KV_METADATA_LIMIT_BYTES}-byte \ + limit, got {} bytes", + json.len() + ); + } } diff --git a/crates/trusted-server-adapter-fastly/src/fixtures/config-push-chunks.json b/crates/trusted-server-adapter-fastly/src/fixtures/config-push-chunks.json new file mode 100644 index 0000000000..1cba533521 --- /dev/null +++ b/crates/trusted-server-adapter-fastly/src/fixtures/config-push-chunks.json @@ -0,0 +1,14 @@ +[ + [ + "a.publisher.example.__edgezero_chunks.da41942742b778decd2a02a332ba43fbbc0c46458d0b55cd173f42f2a13e333c.0", + "{\"data\":{\"publisher\":{\"domain\":\"a.publisher.example\",\"cookie_domain\":\".a.publisher.example\",\"origin_url\":\"https://origin.publisher.example\",\"origin_host_header_override\":null,\"proxy_secret\":\"publisher_proxy_secret\",\"max_buffered_body_bytes\":16777216},\"tester_cookie\":{\"enabled\":false,\"max_age_seconds\":2592000},\"ec\":{\"module\":null,\"resolve_allowed_origins\":[],\"ec_store\":\"ec_identity_store\",\"pull_sync_concurrency\":3,\"cluster_trust_threshold\":10,\"cluster_recheck_secs\":3600,\"partners\":[]},\"response_headers\":{},\"request_signing\":null,\"rewrite\":{\"exclude_domains\":[]},\"auction\":{\"enabled\":false,\"bidders\":{\"example-bidder\":{\"module\":\"pbs_main\"}},\"timeout_ms\":2000,\"creative_store\":\"creative_store\",\"allowed_context_keys\":[]},\"demand\":{\"modules\":[\"pbs_main\"],\"pbs_main\":{\"consent_forwarding\":\"both\",\"debug\":false,\"endpoint\":\"https://prebid.example.com/openrtb2/auction\",\"implementation\":\"auction.prebid-server\",\"notifications\":{\"suppress_all\":false,\"suppress_seats\":[]},\"routing\":\"explicit\",\"test_mode\":false}},\"consent\":{\"mode\":\"interpreter\",\"check_expiration\":true,\"max_consent_age_days\":395,\"us_privacy_defaults\":{\"notice_given\":true,\"lspa_covered\":false,\"gpc_implies_optout\":true},\"conflict_resolution\":{\"mode\":\"restrictive\",\"freshness_threshold_days\":30}},\"cache\":{\"asset_rules\":[]},\"proxy\":{\"certificate_check\":true,\"allowed_domains\":[\"*.assets-0.publisher.example\",\"*.assets-1.publisher.example\",\"*.assets-2.publisher.example\",\"*.assets-3.publisher.example\",\"*.assets-4.publisher.example\",\"*.assets-5.publisher.example\",\"*.assets-6.publisher.example\",\"*.assets-7.publisher.example\",\"*.assets-8.publisher.example\",\"*.assets-9.publisher.example\",\"*.assets-10.publisher.example\",\"*.assets-11.publisher.example\",\"*.assets-12.publisher.example\",\"*.assets-13.publisher.example\",\"*.assets-14.publisher.example\",\"*.assets-15.publisher.example\",\"*.assets-16.publisher.example\",\"*.assets-17.publisher.example\",\"*.assets-18.publisher.example\",\"*.assets-19.publisher.example\",\"*.assets-20.publisher.example\",\"*.assets-21.publisher.example\",\"*.assets-22.publisher.example\",\"*.assets-23.publisher.example\",\"*.assets-24.publisher.example\",\"*.assets-25.publisher.example\",\"*.assets-26.publisher.example\",\"*.assets-27.publisher.example\",\"*.assets-28.publisher.example\",\"*.assets-29.publisher.example\",\"*.assets-30.publisher.example\",\"*.assets-31.publisher.example\",\"*.assets-32.publisher.example\",\"*.assets-33.publisher.example\",\"*.assets-34.publisher.example\",\"*.assets-35.publisher.example\",\"*.assets-36.publisher.example\",\"*.assets-37.publisher.example\",\"*.assets-38.publisher.example\",\"*.assets-39.publisher.example\",\"*.assets-40.publisher.example\",\"*.assets-41.publisher.example\",\"*.assets-42.publisher.example\",\"*.assets-43.publisher.example\",\"*.assets-44.publisher.example\",\"*.assets-45.publisher.example\",\"*.assets-46.publisher.example\",\"*.assets-47.publisher.example\",\"*.assets-48.publisher.example\",\"*.assets-49.publisher.example\",\"*.assets-50.publisher.example\",\"*.assets-51.publisher.example\",\"*.assets-52.publisher.example\",\"*.assets-53.publisher.example\",\"*.assets-54.publisher.example\",\"*.assets-55.publisher.example\",\"*.assets-56.publisher.example\",\"*.assets-57.publisher.example\",\"*.assets-58.publisher.example\",\"*.assets-59.publisher.example\",\"*.assets-60.publisher.example\",\"*.assets-61.publisher.example\",\"*.assets-62.publisher.example\",\"*.assets-63.publisher.example\",\"*.assets-64.publisher.example\",\"*.assets-65.publisher.example\",\"*.assets-66.publisher.example\",\"*.assets-67.publisher.example\",\"*.assets-68.publisher.example\",\"*.assets-69.publisher.example\",\"*.assets-70.publisher.example\",\"*.assets-71.publisher.example\",\"*.assets-72.publisher.example\",\"*.assets-73.publisher.example\",\"*.assets-74.publisher.example\",\"*.assets-75.publisher.example\",\"*.assets-76.publisher.example\",\"*.assets-77.publisher.example\",\"*.assets-78.publisher.example\",\"*.assets-79.publisher.example\",\"*.assets-80.publisher.example\",\"*.assets-81.publisher.example\",\"*.assets-82.publisher.example\",\"*.assets-83.publisher.example\",\"*.assets-84.publisher.example\",\"*.assets-85.publisher.example\",\"*.assets-86.publisher.example\",\"*.assets-87.publisher.example\",\"*.assets-88.publisher.example\",\"*.assets-89.publisher.example\",\"*.assets-90.publisher.example\",\"*.assets-91.publisher.example\",\"*.assets-92.publisher.example\",\"*.assets-93.publisher.example\",\"*.assets-94.publisher.example\",\"*.assets-95.publisher.example\",\"*.assets-96.publisher.example\",\"*.assets-97.publisher.example\",\"*.assets-98.publisher.example\",\"*.assets-99.publisher.example\",\"*.assets-100.publisher.example\",\"*.assets-101.publisher.example\",\"*.assets-102.publisher.example\",\"*.assets-103.publisher.example\",\"*.assets-104.publisher.example\",\"*.assets-105.publisher.example\",\"*.assets-106.publisher.example\",\"*.assets-107.publisher.example\",\"*.assets-108.publisher.example\",\"*.assets-109.publisher.example\",\"*.assets-110.publisher.example\",\"*.assets-111.publisher.example\",\"*.assets-112.publisher.example\",\"*.assets-113.publisher.example\",\"*.assets-114.publisher.example\",\"*.assets-115.publisher.example\",\"*.assets-116.publisher.example\",\"*.assets-117.publisher.example\",\"*.assets-118.publisher.example\",\"*.assets-119.publisher.example\",\"*.assets-120.publisher.example\",\"*.assets-121.publisher.example\",\"*.assets-122.publisher.example\",\"*.assets-123.publisher.example\",\"*.assets-124.publisher.example\",\"*.assets-125.publisher.example\",\"*.assets-126.publisher.example\",\"*.assets-127.publisher.example\",\"*.assets-128.publisher.example\",\"*.assets-129.publisher.example\",\"*.assets-130.publisher.example\",\"*.assets-131.publisher.example\",\"*.assets-132.publisher.example\",\"*.assets-133.publisher.example\",\"*.assets-134.publisher.example\",\"*.assets-135.publisher.example\",\"*.assets-136.publisher.example\",\"*.assets-137.publisher.example\",\"*.assets-138.publisher.example\",\"*.assets-139.publisher.example\",\"*.assets-140.publisher.example\",\"*.assets-141.publisher.example\",\"*.assets-142.publisher.example\",\"*.assets-143.publisher.example\",\"*.assets-144.publisher.example\",\"*.assets-145.publisher.example\",\"*.assets-146.publisher.example\",\"*.assets-147.publisher.example\",\"*.assets-148.publisher.example\",\"*.assets-149.publisher.example\",\"*.assets-150.publisher.example\",\"*.assets-151.publisher.example\",\"*.assets-152.publisher.example\",\"*.assets-153.publisher.example\",\"*.assets-154.publisher.example\",\"*.assets-155.publisher.example\",\"*.assets-156.publisher.example\",\"*.assets-157.publisher.example\",\"*.assets-158.publisher.example\",\"*.assets-159.publisher.example\",\"*.assets-160.publisher.example\",\"*.assets-161.publisher.example\",\"*.assets-162.publisher.example\",\"*.assets-163.publisher.example\",\"*.assets-164.publisher.example\",\"*.assets-165.publisher.example\",\"*.assets-166.publisher.example\",\"*.assets-167.publisher.example\",\"*.assets-168.publisher.example\",\"*.assets-169.publisher.example\",\"*.assets-170.publisher.example\",\"*.assets-171.publisher.example\",\"*.assets-172.publisher.example\",\"*.assets-173.publisher.example\",\"*.assets-174.publisher.examp" + ], + [ + "a.publisher.example.__edgezero_chunks.da41942742b778decd2a02a332ba43fbbc0c46458d0b55cd173f42f2a13e333c.1", + "le\",\"*.assets-175.publisher.example\",\"*.assets-176.publisher.example\",\"*.assets-177.publisher.example\",\"*.assets-178.publisher.example\",\"*.assets-179.publisher.example\",\"*.assets-180.publisher.example\",\"*.assets-181.publisher.example\",\"*.assets-182.publisher.example\",\"*.assets-183.publisher.example\",\"*.assets-184.publisher.example\",\"*.assets-185.publisher.example\",\"*.assets-186.publisher.example\",\"*.assets-187.publisher.example\",\"*.assets-188.publisher.example\",\"*.assets-189.publisher.example\",\"*.assets-190.publisher.example\",\"*.assets-191.publisher.example\",\"*.assets-192.publisher.example\",\"*.assets-193.publisher.example\",\"*.assets-194.publisher.example\",\"*.assets-195.publisher.example\",\"*.assets-196.publisher.example\",\"*.assets-197.publisher.example\",\"*.assets-198.publisher.example\",\"*.assets-199.publisher.example\"],\"asset_routes\":[]},\"creative_opportunities\":{\"gam_network_id\":\"123456789\",\"auction_timeout_ms\":500,\"price_granularity\":\"dense\",\"slot\":[]},\"image_optimizer\":{\"profile_sets\":{}},\"tinybird\":{\"enabled\":false,\"api_host\":\"\",\"auction_dataset\":\"auction_events_raw\",\"access_enabled\":false,\"access_dataset\":\"access_logs_raw\",\"access_sample_rate\":0.0,\"max_body_bytes\":1048576},\"debug\":{\"ja4_endpoint_enabled\":false,\"auction_html_comment\":false,\"inject_adm_for_testing\":false}},\"generated_at\":\"2026-10-16T10:55:33Z\",\"sha256\":\"6fe27c46769f0cd8d9c5ce8dcb085cd1a3c6641b54ce925875effb2eb19fee1d\",\"version\":1}" + ], + [ + "a.publisher.example", + "{\"chunks\":[{\"key\":\"a.publisher.example.__edgezero_chunks.da41942742b778decd2a02a332ba43fbbc0c46458d0b55cd173f42f2a13e333c.0\",\"len\":7000,\"sha256\":\"86a1412d9763eb57f7e5455cd10325ea38092e7476c2aa51a14711c36ee6bf3c\"},{\"key\":\"a.publisher.example.__edgezero_chunks.da41942742b778decd2a02a332ba43fbbc0c46458d0b55cd173f42f2a13e333c.1\",\"len\":1422,\"sha256\":\"ae3b938c81ae511a87b46d90731cc3755acba375624b22038a385ce9dc167fe0\"}],\"data_sha256\":\"6fe27c46769f0cd8d9c5ce8dcb085cd1a3c6641b54ce925875effb2eb19fee1d\",\"edgezero_kind\":\"fastly_config_chunks\",\"envelope_len\":8422,\"envelope_sha256\":\"da41942742b778decd2a02a332ba43fbbc0c46458d0b55cd173f42f2a13e333c\",\"version\":1}" + ] +] diff --git a/crates/trusted-server-adapter-fastly/src/lib.rs b/crates/trusted-server-adapter-fastly/src/lib.rs index 96e7a003dd..e855947a2b 100644 --- a/crates/trusted-server-adapter-fastly/src/lib.rs +++ b/crates/trusted-server-adapter-fastly/src/lib.rs @@ -44,7 +44,10 @@ use trusted_server_device_fastly::{FastlyDeviceModule, FastlyHostSignals}; mod app; mod backend; mod compat; +mod config_chunks; mod config_selection; +#[cfg(test)] +mod core_sdk_guard; mod ec_kv; mod esi_assembly; mod logging; @@ -404,6 +407,7 @@ pub fn run_with_addressing( /// come from the runtime environment before the SDK serving loop is entered; /// an unconfigured or partially configured sandbox stays single-request. pub fn run() { + app::declare_platform(); let (mode, diagnostics) = serve_mode(); // Held outside the sandbox: `serve_custom` owns the `Sandbox` and exposes @@ -739,14 +743,14 @@ fn edgezero_main(mut req: FastlyRequest, sandbox: &mut Sandbox, ordinal: u64, re // sanitization that removes the headers it reads. let resolved_client_ip = compat::resolve_and_sanitize_client_ip(&mut req, trusted_client_ip); - // Re-inject a trusted TLS scheme signal after sanitization has stripped any - // client-sent fastly-ssl header. Setting it from Fastly's native TLS - // metadata here is authoritative. detect_request_scheme in http_util checks - // this header so scheme-sensitive logic produces https URLs on HTTPS traffic. + // Re-inject the TLS header after sanitization has stripped any a client + // sent. Setting it from Fastly's native TLS metadata here is authoritative, + // and core reads it as the TLS header this adapter declares, so + // scheme-sensitive logic produces https URLs on HTTPS traffic. if req.get_tls_protocol().ok().flatten().is_some() || req.get_tls_cipher_openssl_name().ok().flatten().is_some() { - req.set_header("fastly-ssl", "1"); + req.set_header(app::FASTLY_TLS_HEADER, "1"); } // Strip any client-supplied x-ts-tls-* headers before injecting the trusted @@ -1215,8 +1219,8 @@ pub(crate) fn extract_cookie_value(req: &HttpRequest, name: &str) -> Option ModuleRequest<'_> { /// `https` when the entry point marked the request as arriving over TLS, /// which it does from Fastly's own TLS metadata. fn entry_scheme(req: &FastlyRequest) -> &'static str { - if req.get_header_str("fastly-ssl").is_some() { + if req.get_header_str(app::FASTLY_TLS_HEADER).is_some() { "https" } else { "http" @@ -1380,7 +1384,7 @@ mod tests { req.set_header("user-agent", "Mozilla/5.0 (X11; Linux x86_64) Chrome/140.0"); req.set_header("cookie", "example_width=1280"); // The entry point marks a request that arrived over TLS this way. - req.set_header("fastly-ssl", "1"); + req.set_header(crate::app::FASTLY_TLS_HEADER, "1"); req } diff --git a/crates/trusted-server-adapter-fastly/src/middleware.rs b/crates/trusted-server-adapter-fastly/src/middleware.rs index 6794608e64..da79c9e074 100644 --- a/crates/trusted-server-adapter-fastly/src/middleware.rs +++ b/crates/trusted-server-adapter-fastly/src/middleware.rs @@ -11,7 +11,7 @@ use edgezero_core::error::EdgeError; use edgezero_core::http::{HeaderValue, Response}; use edgezero_core::middleware::{Middleware, Next}; use edgezero_core::response::IntoResponse; -use trusted_server_core::constants::{ENV_FASTLY_IS_STAGING, HEADER_X_TS_ENV}; +use trusted_server_core::constants::HEADER_X_TS_ENV; use trusted_server_core::http_util::{ROUTE_NAMESPACE, strip_deployment_headers}; use trusted_server_core::settings::Settings; @@ -91,7 +91,7 @@ impl Middleware for FinalizeResponseMiddleware { /// 4. `X-Robots-Tag: noindex, nofollow` when `[robots-txt] modules` selects /// `refuse_all` pub(crate) fn apply_finalize_headers(settings: &Settings, response: &mut Response) { - if std::env::var(ENV_FASTLY_IS_STAGING).as_deref() == Ok("1") { + if crate::app::fastly_is_staging() { response .headers_mut() .insert(HEADER_X_TS_ENV, HeaderValue::from_static("staging")); diff --git a/crates/trusted-server-adapter-fastly/src/platform.rs b/crates/trusted-server-adapter-fastly/src/platform.rs index 006785595c..ec9f10adb6 100644 --- a/crates/trusted-server-adapter-fastly/src/platform.rs +++ b/crates/trusted-server-adapter-fastly/src/platform.rs @@ -20,7 +20,6 @@ use trusted_server_core::platform::{ PlatformSecretStore, PlatformSelectResult, StoreId, StoreName, policy_entry_text, policy_key, }; use trusted_server_core::settings::TrustedClientIpConfig; -use trusted_server_core::settings_data::resolve_fastly_chunk_pointer; // --------------------------------------------------------------------------- // FastlyPlatformConfigStore @@ -28,17 +27,19 @@ use trusted_server_core::settings_data::resolve_fastly_chunk_pointer; /// Fastly [`ConfigStore`]-backed implementation of [`PlatformConfigStore`]. /// -/// Reads take the store name on each call. The service never writes to a -/// config store, so `put` and `delete` are refused. Signing keys are written -/// by `ts keys`. +/// Reads take the store name on each call, and a document stored in chunks +/// because it is longer than an entry is read back whole, see +/// [`crate::config_chunks`]. The service never writes to a config store, so +/// `put` and `delete` are refused. Signing keys are written by `ts keys`. pub struct FastlyPlatformConfigStore; /// The shared policy entries of a config store, from which a blob the keyed /// push wrote for a host, which carries no policy file, reads the file it /// pins. /// -/// An entry is read as a blob is, through the store and core's chunk reader, -/// and the runtime loader checks the file against the blob's digest. +/// An entry is read as a blob is, through the store, which returns a long +/// entry whole, and the runtime loader checks the file against the blob's +/// digest. pub(crate) struct FastlyPolicyEntries<'a> { config_store: &'a dyn PlatformConfigStore, store_name: &'a StoreName, @@ -66,12 +67,7 @@ impl PlatformPolicySource for FastlyPolicyEntries<'_> { .get(self.store_name, &key) .change_context(PlatformError::PolicySource) .attach(format!("the policy entry `{key}` cannot be read"))?; - let entry = resolve_fastly_chunk_pointer(self.config_store, self.store_name, &value) - .change_context(PlatformError::PolicySource) - .attach(format!( - "the chunks of the policy entry `{key}` cannot be read" - ))?; - policy_entry_text(&entry) + policy_entry_text(&value) } } @@ -82,17 +78,8 @@ impl PlatformConfigStore for FastlyPlatformConfigStore { Report::new(PlatformError::ConfigStore) .attach(format!("failed to open config store '{name}': {e}")) })?; - store - .try_get(key) - .map_err(|e| { - Report::new(PlatformError::ConfigStore).attach(format!( - "lookup for key '{key}' in config store '{name}' failed: {e}" - )) - })? - .ok_or_else(|| { - Report::new(PlatformError::ConfigStore) - .attach(format!("key '{key}' not found in config store '{name}'")) - }) + let value = read_entry(&store, name, key)?; + crate::config_chunks::whole_entry(value, |chunk| read_entry(&store, name, chunk)) } fn put(&self, _: &StoreId, _: &str, _: &str) -> Result<(), Report> { @@ -106,6 +93,21 @@ impl PlatformConfigStore for FastlyPlatformConfigStore { } } +/// The value at `key` in the open config store `store`, named `name`. +fn read_entry(store: &ConfigStore, name: &str, key: &str) -> Result> { + store + .try_get(key) + .map_err(|e| { + Report::new(PlatformError::ConfigStore).attach(format!( + "lookup for key '{key}' in config store '{name}' failed: {e}" + )) + })? + .ok_or_else(|| { + Report::new(PlatformError::ConfigStore) + .attach(format!("key '{key}' not found in config store '{name}'")) + }) +} + // --------------------------------------------------------------------------- // FastlyPlatformSecretStore // --------------------------------------------------------------------------- diff --git a/crates/trusted-server-cli/src/commands/config/keyed_push.rs b/crates/trusted-server-cli/src/commands/config/keyed_push.rs index be2058b22c..29e7525f86 100644 --- a/crates/trusted-server-cli/src/commands/config/keyed_push.rs +++ b/crates/trusted-server-cli/src/commands/config/keyed_push.rs @@ -32,7 +32,7 @@ use edgezero_core::blob_envelope::BlobEnvelope; use edgezero_core::env_config::EnvConfig; use edgezero_core::manifest::{Manifest, ManifestLoader}; use error_stack::Report; -use serde::Serialize; +use serde::{Deserialize, Serialize}; use sha2::{Digest as _, Sha256}; use toml_edit::{DocumentMut, Item, Table}; use trusted_server_core::config::TrustedServerAppConfig; @@ -42,7 +42,6 @@ use trusted_server_core::platform::{ PlatformConfigStore, PlatformError, PolicyEntry, StoreId, StoreName, policy_entry_text, policy_key, }; -use trusted_server_core::settings_data::resolve_fastly_chunk_pointer; use super::policy::{self, PolicyText, PushArgs}; use crate::fastly_cli::{self, FastlyProcess, FastlyRunner}; @@ -85,6 +84,22 @@ struct ChunkRef { sha256: String, } +/// A pointer read back from a store, with the fields [`stored_whole`] +/// checks. +#[derive(Deserialize)] +struct StoredPointer { + chunks: Vec, + edgezero_kind: String, + envelope_sha256: String, +} + +/// One chunk a stored pointer names. +#[derive(Deserialize)] +struct StoredChunk { + key: String, + sha256: String, +} + /// The key this writer pushes for `args`, or `None` when the push is /// `EdgeZero`'s. /// @@ -277,18 +292,43 @@ fn policy_entries(name: &str, text: &str) -> Result { } /// Whether `stored` holds the policy file whose SHA-256 is `sha256` under -/// `key`, read back as the runtime reads it, chunks included. An entry that -/// cannot be read back, or holds another file, does not. +/// `key`, read back whole, chunks included. An entry that cannot be read +/// back, or holds another file, does not. fn holds_policy(stored: &dyn PlatformConfigStore, key: &str, sha256: &str) -> bool { let store_name = StoreName::from(DEFAULT_CONFIG_STORE_ID); stored .get(&store_name, key) .ok() - .and_then(|value| resolve_fastly_chunk_pointer(stored, &store_name, &value).ok()) + .and_then(|value| stored_whole(stored, &store_name, value)) .and_then(|value| policy_entry_text(&value).ok()) .is_some_and(|text| policy_sha256(&text) == sha256) } +/// `value` read back whole: the value itself, or, when it is a pointer +/// [`entries`] wrote, its chunks joined in order. `None` when a chunk is +/// missing or holds other bytes than the pointer names. +fn stored_whole( + stored: &dyn PlatformConfigStore, + store_name: &StoreName, + value: String, +) -> Option { + let Ok(pointer) = serde_json::from_str::(&value) else { + return Some(value); + }; + if pointer.edgezero_kind != POINTER_KIND { + return Some(value); + } + let mut whole = String::new(); + for chunk in pointer.chunks { + let text = stored.get(store_name, &chunk.key).ok()?; + if sha256_hex(&text) != chunk.sha256 { + return None; + } + whole.push_str(&text); + } + (sha256_hex(&whole) == pointer.envelope_sha256).then_some(whole) +} + /// Entries already read from a store, for the chunk reader to read chunks /// from. struct StoredEntries<'a>(&'a BTreeMap); diff --git a/crates/trusted-server-cli/tests/config_push_by_key.rs b/crates/trusted-server-cli/tests/config_push_by_key.rs index ee9f381aeb..c5f8f7ac91 100644 --- a/crates/trusted-server-cli/tests/config_push_by_key.rs +++ b/crates/trusted-server-cli/tests/config_push_by_key.rs @@ -18,9 +18,7 @@ use trusted_server_core::platform::{ StoreId, StoreName, policy_entry_text, policy_key, }; use trusted_server_core::settings::Settings; -use trusted_server_core::settings_data::{ - get_settings_from_config_store_with, resolve_fastly_chunk_pointer, -}; +use trusted_server_core::settings_data::get_settings_from_config_store_with; const HOST: &str = "a.publisher.example"; @@ -178,12 +176,31 @@ fn stored_entries(project: &TempDir) -> Vec<(String, String)> { struct MemoryConfigStore(BTreeMap); -impl PlatformConfigStore for MemoryConfigStore { - fn get(&self, _store_name: &StoreName, key: &str) -> Result> { +impl MemoryConfigStore { + fn entry(&self, key: &str) -> Result> { self.0.get(key).cloned().ok_or_else(|| { Report::new(PlatformError::ConfigStore).attach(format!("missing key `{key}`")) }) } +} + +impl PlatformConfigStore for MemoryConfigStore { + /// The entry at `key`, its chunks joined in order when it is a chunk + /// pointer, as the Fastly config store returns a long entry whole. + fn get(&self, _store_name: &StoreName, key: &str) -> Result> { + let value = self.entry(key)?; + let pointer: serde_json::Value = + serde_json::from_str(&value).unwrap_or(serde_json::Value::Null); + if pointer["edgezero_kind"] != "fastly_config_chunks" { + return Ok(value); + } + pointer["chunks"] + .as_array() + .into_iter() + .flatten() + .map(|chunk| self.entry(chunk["key"].as_str().unwrap_or_default())) + .collect() + } fn put( &self, @@ -240,10 +257,7 @@ impl PlatformPolicySource for SharedPolicyEntries<'_> { POLICY_READS.with(|reads| reads.set(reads.get() + 1)); let store_name = StoreName::from(DEFAULT_CONFIG_STORE_ID); let value = self.0.get(&store_name, &policy_key(&reference.sha256))?; - let entry = resolve_fastly_chunk_pointer(self.0, &store_name, &value).map_err(|error| { - Report::new(PlatformError::PolicySource).attach(format!("{error:?}")) - })?; - policy_entry_text(&entry) + policy_entry_text(&value) } } @@ -370,7 +384,7 @@ fn a_long_document_is_written_as_chunks_then_a_pointer() { &["--local", "--key", HOST, "--permissions", POLICY_FILE], )); - let mut entries = stored_entries(&project); + let entries = stored_entries(&project); let chunk_prefix = format!("{HOST}.__edgezero_chunks."); let chunk_count = entries .iter() @@ -381,40 +395,29 @@ fn a_long_document_is_written_as_chunks_then_a_pointer() { .iter() .position(|(key, _)| key == HOST) .expect("should write the pointer"); - assert!( - entries[..pointer_at] - .iter() - .filter(|(key, _)| key.starts_with(&chunk_prefix)) - .count() - == chunk_count, - "every chunk should be written before the pointer" - ); - let (settings, reads) = read_back_counting_reads(&entries, HOST); - let settings = settings.expect("should read the chunked blob back whole"); - assert_eq!( - reads, 1, - "the read-back should read the policy file from its shared entry" - ); + let written: Vec<&str> = entries[..pointer_at] + .iter() + .map(|(key, _)| key.as_str()) + .filter(|key| key.starts_with(&chunk_prefix)) + .collect(); assert_eq!( - settings.publisher.domain, HOST, - "should reassemble the pushed app config" + written.len(), + chunk_count, + "every chunk should be written before the pointer" ); + // The Fastly adapter reads a written document back whole, which + // `config_chunks.rs` tests against a fixture this writer produced. + let pointer: serde_json::Value = + serde_json::from_str(&entries[pointer_at].1).expect("should write the pointer as JSON"); + let named: Vec<&str> = pointer["chunks"] + .as_array() + .expect("should name the chunks") + .iter() + .map(|chunk| chunk["key"].as_str().expect("should name each chunk's key")) + .collect(); assert_eq!( - settings.proxy.allowed_domains.len(), - 400, - "should reassemble every part of the long blob" - ); - - let pointer = &mut entries[pointer_at].1; - let mut changed: serde_json::Value = - serde_json::from_str(pointer).expect("should write the pointer as JSON"); - changed["chunks"][0]["sha256"] = serde_json::Value::from("0".repeat(64)); - *pointer = changed.to_string(); - let refused = - read_back(&entries, HOST).expect_err("a chunk whose hash differs should be refused"); - assert!( - format!("{refused:?}").contains("sha mismatch"), - "should refuse the chunk for its hash: {refused:?}" + named, written, + "the pointer should name the chunks written, in order" ); } diff --git a/crates/trusted-server-core/Cargo.toml b/crates/trusted-server-core/Cargo.toml index dd3254a019..28e148e42b 100644 --- a/crates/trusted-server-core/Cargo.toml +++ b/crates/trusted-server-core/Cargo.toml @@ -54,9 +54,9 @@ validator = { workspace = true } web-time = { workspace = true } [target.'cfg(all(target_arch = "wasm32", target_os = "unknown"))'.dependencies] -# Enable JS-backed RNG for `wasm32-unknown-unknown` targets (e.g. Cloudflare Workers). -# The Fastly adapter uses `wasm32-wasip1`, which has native POSIX RNG and does not -# need these — this block is intentionally scoped to `target_os = "unknown"` only. +# Enable JS-backed RNG for `wasm32-unknown-unknown` targets, which have no OS +# randomness source. `wasm32-wasip1` has native POSIX RNG and does not need +# these, so this block is scoped to `target_os = "unknown"` only. getrandom = { workspace = true, features = ["js"] } uuid = { workspace = true, features = ["js"] } # `chrono::Utc::now()` calls `std::time::SystemTime::now()`, which panics on diff --git a/crates/trusted-server-core/README.md b/crates/trusted-server-core/README.md index f6a37bf9b1..66cad508e6 100644 --- a/crates/trusted-server-core/README.md +++ b/crates/trusted-server-core/README.md @@ -2,7 +2,9 @@ Portable application core shared by every Trusted Server adapter. It targets both supported WASM environments and native adapter tests, so it must not depend -on an edge SDK, Tokio runtime, filesystem, socket, or host-only process API. +on an edge SDK, Tokio runtime, filesystem, socket, or host-only process API. It +names no hosting platform either, in code, comments, tests or documents, and +`scripts/lint-core-platform-names.py` fails CI when a file here names one. ## Responsibilities @@ -36,15 +38,13 @@ boundary; see the [Integration Guide](../../docs/guide/integration-guide.md). ## Build and test -From the repository root: +From the repository root, run core's own tests on the host target: ```bash -cargo build-fastly -cargo test-fastly +cargo test -p trusted-server-core --target "$(rustc -vV | sed -n 's/host: //p')" ``` -The Fastly aliases compile and test the core for `wasm32-wasip1` through -Viceroy. Cloudflare and native adapter suites exercise the same core under -their target configurations. Use [TESTING.md](../../TESTING.md) for the complete -target matrix and [Architecture](../../docs/guide/architecture.md) for the +Each adapter's test alias builds and tests the same core for that adapter's +target, and [TESTING.md](../../TESTING.md) lists them with the complete target +matrix. [Architecture](../../docs/guide/architecture.md) gives the request-level system view. diff --git a/crates/trusted-server-core/build.rs b/crates/trusted-server-core/build.rs index ba94a65d79..db9c31d27c 100644 --- a/crates/trusted-server-core/build.rs +++ b/crates/trusted-server-core/build.rs @@ -105,13 +105,6 @@ fn resolve_from_local_git() -> Option { }) } -/// Source file that must stay out of the generated list. -/// -/// `migration_guards.rs` holds the banned pattern itself, as the regex literal -/// the guard matches with, so including it would fail the guard on its own -/// text. -const EXCLUDED: &str = "migration_guards.rs"; - fn main() { // Watching the directory also catches newly added and removed source files. println!("cargo:rerun-if-changed=src"); @@ -160,7 +153,6 @@ fn main() { ) .expect("should write the template build digest"); - write_migration_guard_sources(); emit_git_version(); } @@ -273,65 +265,3 @@ fn collect_sources(crate_dir: &Path, relative: &Path, sources: &mut Vec } } } - -/// Lists every Rust source file under `src` for the migration guard, so a -/// file added to or removed from core joins or leaves the guard on its own. -fn write_migration_guard_sources() { - // Cargo walks a directory named here, so adding, renaming or deleting a - // source file reruns this script and the guard list follows the tree. - println!("cargo:rerun-if-changed=src"); - - let manifest_dir = - PathBuf::from(env::var("CARGO_MANIFEST_DIR").expect("should read CARGO_MANIFEST_DIR")); - let src_dir = manifest_dir.join("src"); - - let mut sources = Vec::new(); - collect_rust_sources(&src_dir, &src_dir, &mut sources); - sources.sort(); - - let mut generated = String::from( - "// Generated by build.rs. Every Rust source file under `src`, so a file\n\ - // added to or removed from core joins or leaves the guard on its own.\n\ - const CHECKED_SOURCES: &[(&str, &str)] = &[\n", - ); - for relative in &sources { - let absolute = format!("{}/{relative}", path_literal(&src_dir)); - generated.push_str(&format!( - " (\"{relative}\", include_str!(\"{absolute}\")),\n" - )); - } - generated.push_str("];\n"); - - let out_dir = PathBuf::from(env::var("OUT_DIR").expect("should read OUT_DIR")); - fs::write(out_dir.join("migration_guard_sources.rs"), generated) - .expect("should write the migration guard source list"); -} - -/// Collects every `.rs` file under `dir`, as a path relative to `root` with -/// forward slashes, so the generated file reads the same on every platform. -fn collect_rust_sources(root: &Path, dir: &Path, sources: &mut Vec) { - let entries = fs::read_dir(dir).expect("should read a source directory"); - for entry in entries { - let entry = entry.expect("should read a source directory entry"); - let path = entry.path(); - if path.is_dir() { - collect_rust_sources(root, &path, sources); - continue; - } - if path.extension().is_some_and(|extension| extension == "rs") { - let relative = path - .strip_prefix(root) - .expect("should strip the source root prefix"); - let relative = path_literal(relative); - if relative != EXCLUDED { - sources.push(relative); - } - } - } -} - -/// Renders a path for a Rust string literal, with forward slashes so a Windows -/// path needs no escaping. -fn path_literal(path: &Path) -> String { - path.display().to_string().replace('\\', "/") -} diff --git a/crates/trusted-server-core/src/auction/README.md b/crates/trusted-server-core/src/auction/README.md index fa8271577b..ac079eebb8 100644 --- a/crates/trusted-server-core/src/auction/README.md +++ b/crates/trusted-server-core/src/auction/README.md @@ -29,7 +29,7 @@ flowchart TB ``` Each adapter owns transport routing in its `app.rs`. Core request handling stays -in `auction::endpoints`, so no demand implementation depends on Fastly types. +in `auction::endpoints`, so no demand implementation depends on a platform's types. `handle_auction` performs these steps: @@ -183,11 +183,10 @@ integration registry from the same `Arc`. Tests for one implementation should cover its typed settings, exact request output, response admission, routing, local failures, and target validation. -Run target-matched aliases rather than bare workspace tests: +Run core's tests on the host target, and the adapter aliases +[TESTING.md](../../../../TESTING.md#automated-checks) lists, rather than bare +workspace tests: ```bash -cargo test-fastly -cargo test-axum -cargo test-cloudflare -cargo test-spin +cargo test -p trusted-server-core --target "$(rustc -vV | sed -n 's/host: //p')" ``` diff --git a/crates/trusted-server-core/src/auction/orchestrator.rs b/crates/trusted-server-core/src/auction/orchestrator.rs index a7a5ef9076..43fbb78ccf 100644 --- a/crates/trusted-server-core/src/auction/orchestrator.rs +++ b/crates/trusted-server-core/src/auction/orchestrator.rs @@ -29,7 +29,7 @@ use crate::request_signing::RequestSigner; /// Created by [`AuctionOrchestrator::dispatch_auction`] and consumed by /// [`AuctionOrchestrator::collect_dispatched_auction`]. Carrying this handle /// across `pending_origin.wait()` lets origin response and SSP HTTP requests -/// race in Fastly's native layer, enabling TTFB ≈ origin latency rather than +/// race in the host's native layer, enabling TTFB ≈ origin latency rather than /// TTFB ≈ auction timeout. pub struct DispatchedAuction { pending_requests: Vec, @@ -1139,7 +1139,7 @@ impl AuctionOrchestrator { } // Reject multi-provider fan-out before any request launches when the - // platform executes `send_async` eagerly (e.g. Cloudflare Workers): + // platform executes `send_async` eagerly: // sequential execution would accrue the sum of provider latencies and // blow the auction budget before a later `select` could reject it. if provider_names.len() > 1 && !context.services.http_client().supports_concurrent_fanout() @@ -1850,8 +1850,8 @@ impl AuctionOrchestrator { /// Dispatch SSP bid requests without blocking WASM. /// /// Calls each enabled provider's [`AuctionProvider::request_bids`] (which - /// internally calls Fastly's `send_async`), then returns immediately with a - /// [`DispatchedAuction`] token. The Fastly host begins the SSP round-trips + /// internally calls the platform's `send_async`), then returns immediately with a + /// [`DispatchedAuction`] token. The host begins the SSP round-trips /// while WASM continues to `pending_origin.wait()`. /// /// Returns [`DispatchAuctionOutcome::NotStarted`] when no providers are configured or @@ -1883,10 +1883,9 @@ impl AuctionOrchestrator { } // Mirror run_providers_parallel: reject multi-provider fan-out before - // any request launches when the platform executes `send_async` eagerly - // (e.g. Cloudflare Workers, Spin). Sequential execution would accrue - // the sum of provider latencies before the origin fetch and then fail - // collection with empty bids. + // any request launches when the platform executes `send_async` eagerly. + // Sequential execution would accrue the sum of provider latencies + // before the origin fetch and then fail collection with empty bids. #[cfg(test)] if provider_names.len() > 1 && !context.services.http_client().supports_concurrent_fanout() { diff --git a/crates/trusted-server-core/src/auction/telemetry.rs b/crates/trusted-server-core/src/auction/telemetry.rs index 17517f3d43..e865327bb7 100644 --- a/crates/trusted-server-core/src/auction/telemetry.rs +++ b/crates/trusted-server-core/src/auction/telemetry.rs @@ -223,10 +223,10 @@ impl AuctionObservationContext { /// Return elapsed milliseconds since the observation was created. /// - /// On Cloudflare Workers, [`Instant`] reads `performance.now()`, which only - /// advances after I/O. There this counts time spent waiting on I/O and - /// reads 0 for CPU-only spans, so it is not comparable with the wall-clock - /// value other adapters report. + /// On `wasm32-unknown-unknown`, [`Instant`] reads `performance.now()`, which + /// a platform may advance only after I/O. There this counts time spent + /// waiting on I/O and reads 0 for CPU-only spans, so it is not comparable + /// with the wall-clock value other adapters report. #[must_use] pub fn elapsed_ms(&self) -> u64 { u64::try_from(self.started_at.elapsed().as_millis()).unwrap_or(u64::MAX) diff --git a/crates/trusted-server-core/src/constants.rs b/crates/trusted-server-core/src/constants.rs index b703c553e7..b15e7a348f 100644 --- a/crates/trusted-server-core/src/constants.rs +++ b/crates/trusted-server-core/src/constants.rs @@ -29,7 +29,6 @@ pub const HEADER_X_FORWARDED_FOR: HeaderName = HeaderName::from_static("x-forwar pub const HEADER_X_SUBJECT_ID: HeaderName = HeaderName::from_static("x-subject-id"); pub const HEADER_X_REQUEST_ID: HeaderName = HeaderName::from_static("x-request-id"); pub const HEADER_X_COMPRESS_HINT: HeaderName = HeaderName::from_static("x-compress-hint"); -pub const HEADER_X_DEBUG_FASTLY_POP: HeaderName = HeaderName::from_static("x-debug-fastly-pop"); // Staging / version identification headers. The two version headers are // written by core's `service_version` middleware where a `[[serve]]` entry @@ -47,9 +46,6 @@ pub const TS_GIT_VERSION: Option<&str> = match option_env!("TRUSTED_SERVER__GIT_ _ => None, }; -// Fastly environment variables -pub const ENV_FASTLY_IS_STAGING: &str = "FASTLY_IS_STAGING"; - // Common standard header names used across modules pub const HEADER_USER_AGENT: HeaderName = HeaderName::from_static("user-agent"); pub const HEADER_ACCEPT: HeaderName = HeaderName::from_static("accept"); @@ -107,10 +103,9 @@ const NON_EC_INTERNAL_HEADERS: &[&str] = &[ "x-geo-region", "x-request-id", "x-compress-hint", - "x-debug-fastly-pop", - // Trusted TLS metadata injected by the Fastly EdgeZero entry point. - // Injected after stripping spoofable forwarded headers so they cannot be - // client-supplied. Must not be forwarded to downstream origins. + // Trusted TLS metadata an adapter's entry point injects after stripping + // spoofable forwarded headers, so they cannot be client-supplied. Must + // not be forwarded to downstream origins. "x-ts-tls-protocol", "x-ts-tls-cipher", ]; @@ -159,6 +154,14 @@ const fn join_internal_headers() -> [&'static str; INTERNAL_HEADER_COUNT] { /// in `const` context. pub const INTERNAL_HEADERS: &[&str] = &join_internal_headers(); +/// Whether `name`, in lower case, must not be forwarded to a third party, +/// being one of [`INTERNAL_HEADERS`] or an internal header the running +/// platform declared. +#[must_use] +pub fn is_internal_header(name: &str) -> bool { + INTERNAL_HEADERS.contains(&name) || crate::platform::is_platform_internal_header(name) +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/trusted-server-core/src/creative.rs b/crates/trusted-server-core/src/creative.rs index 578d86d680..e37093b2f3 100644 --- a/crates/trusted-server-core/src/creative.rs +++ b/crates/trusted-server-core/src/creative.rs @@ -785,7 +785,7 @@ pub fn rewrite_css_body(settings: &Settings, css: &str) -> Result that development traffic never reaches. let settings = crate::test_support::tests::create_test_settings(); @@ -3474,31 +3474,31 @@ b{background:url(\"https://cdn.example/c.png\")}"; #[test] fn to_abs_respects_wildcard_domains() { let mut settings = crate::test_support::tests::create_test_settings(); - settings.rewrite.exclude_domains = vec!["*.cloudflare.com".to_owned()]; + settings.rewrite.exclude_domains = vec!["*.cdn.example.com".to_owned()]; // Should exclude base domain - assert_eq!(to_abs(&settings, "https://cloudflare.com/cdn.js"), None); + assert_eq!(to_abs(&settings, "https://cdn.example.com/cdn.js"), None); // Should exclude subdomain assert_eq!( - to_abs(&settings, "https://cdnjs.cloudflare.com/lib.js"), + to_abs(&settings, "https://assets.cdn.example.com/lib.js"), None ); assert_eq!( - to_abs(&settings, "//cloudflare.com/cdn.js"), + to_abs(&settings, "//cdn.example.com/cdn.js"), None, "should exclude a protocol-relative wildcard base domain" ); assert_eq!( - to_abs(&settings, "//cdnjs.cloudflare.com/lib.js"), + to_abs(&settings, "//assets.cdn.example.com/lib.js"), None, "should exclude a protocol-relative wildcard subdomain" ); // Should not exclude different domain assert_eq!( - to_abs(&settings, "https://notcloudflare.com/lib.js"), - Some("https://notcloudflare.com/lib.js".to_owned()) + to_abs(&settings, "https://notcdn.example.com/lib.js"), + Some("https://notcdn.example.com/lib.js".to_owned()) ); } diff --git a/crates/trusted-server-core/src/creative_opportunities.rs b/crates/trusted-server-core/src/creative_opportunities.rs index e84a000095..82d02bb9e1 100644 --- a/crates/trusted-server-core/src/creative_opportunities.rs +++ b/crates/trusted-server-core/src/creative_opportunities.rs @@ -380,9 +380,9 @@ pub struct CreativeOpportunitiesConfig { /// [`Self::origin_is_cookie_independent`], which only ever applies to cookie-bearing /// requests, readthrough admits *cookieless* requests — first-time visitors, which is /// exactly when an origin issues a session cookie. There is no response-side guard on - /// this path: the decision is made before the origin replies, and no post-response - /// hook is reachable on the Fastly adapter. Safety therefore rests on the origin's own - /// `Cache-Control` plus an operator's verification, so enabling it must be a + /// this path: the decision is made before the origin replies, and the platform's cache + /// stores the reply before any hook of core's runs. Safety therefore rests on the + /// origin's own `Cache-Control` plus an operator's verification, so enabling it must be a /// deliberate act rather than a consequence of deploying. /// /// Configured [`Self::template_cache_key_cookies`] disable readthrough because diff --git a/crates/trusted-server-core/src/ec/device.rs b/crates/trusted-server-core/src/ec/device.rs index 756a5c7aab..ded37a3fbd 100644 --- a/crates/trusted-server-core/src/ec/device.rs +++ b/crates/trusted-server-core/src/ec/device.rs @@ -1,7 +1,7 @@ //! Device signal derivation for bot detection and browser classification. //! //! The [`DeviceSignals`] derivation here is pure computation, with no KV I/O or -//! Fastly SDK calls. A [`DeviceModule`] is handed the request's module context +//! platform SDK calls. A [`DeviceModule`] is handed the request's module context //! when `detect` is called and names what it reads from it, the //! [`RequestInfo`] for the User-Agent among them. On a host that supplies //! them, it also reads the [`HostSignals`](crate::evidence::HostSignals) for @@ -131,9 +131,10 @@ fn kept_text(module: &str, field: &str, value: Option) -> Option /// Device signals derived from a single request. /// -/// Computed in the Fastly adapter from raw TLS/H2/UA data, then passed to -/// core for storage and gating decisions. This type lives in core so it -/// can be used in [`KvDevice`] construction and tested without Fastly. +/// Computed by the device module from the User-Agent, and from the raw TLS +/// and HTTP/2 signals where the platform supplies them, then passed to core +/// for storage and gating decisions. This type lives in core so it can be +/// used in [`KvDevice`] construction and tested without any platform. #[derive(Debug, Clone, PartialEq)] pub struct DeviceSignals { /// `0` = desktop, `1` = mobile, `2` = unknown. @@ -149,8 +150,8 @@ pub struct DeviceSignals { pub known_browser: Option, /// Whether the request looks like a real browser, used to gate Edge Cookie /// writes. Computed by the producing module: the built-in module uses a - /// User-Agent-only heuristic, while the Fastly module strengthens it with - /// the TLS and HTTP/2 signals. + /// User-Agent-only heuristic, while the platform's own module strengthens + /// it with the TLS and HTTP/2 signals. /// /// [`human_confidence`](Self::human_confidence) is the module's answer /// about a person, which nothing reads yet. @@ -202,7 +203,7 @@ impl DeviceSignals { /// Derives device signals from the User-Agent alone, with no /// host-specific TLS or HTTP/2 evidence. /// - /// This is the default path. It touches no Fastly-specific API, so device + /// This is the default path. It touches no platform-specific API, so device /// classification stays host-neutral by default. `ja4_class` and /// `h2_fp_hash` are left absent, and the browser/bot decision uses a /// User-Agent-only heuristic (`looks_like_browser_from_ua`). @@ -226,11 +227,11 @@ impl DeviceSignals { /// Derives device signals from the User-Agent strengthened with the /// host's TLS and HTTP/2 signals. /// - /// `ua` is the `User-Agent` header value. `ja4` is the full JA4 hash - /// from `req.get_tls_ja4()`. `h2_fp` is the raw H2 SETTINGS string - /// from `req.get_client_h2_fingerprint()`. These signals are - /// host-specific (Fastly), so only the opt-in Fastly device module - /// uses this path, and the browser/bot gate then requires a TLS signal. + /// `ua` is the `User-Agent` header value. `ja4` is the full JA4 hash and + /// `h2_fp` the raw H2 SETTINGS string, as the platform reports them. + /// These signals are host-specific, so only the platform's own device + /// module, selected with `platform`, uses this path, and the browser/bot + /// gate then requires a TLS signal. #[must_use] pub fn derive(ua: &str, ja4: Option<&str>, h2_fp: Option<&str>) -> Self { let is_mobile = parse_is_mobile(ua); @@ -372,13 +373,13 @@ impl DeviceModule for BuiltinDeviceModule { /// The device module core or the adapter supplies for the `[device] module` /// selector. /// -/// Returns the built-in User-Agent-only module unless the `fastly` selector is -/// set, in which case it builds the host-specific module through the -/// `build_fastly` factory the adapter supplies. The factory runs only when that -/// module is selected, so device classification itself reads no host signals -/// by default (see [`BuiltinDeviceModule`] for the host-neutral default). The -/// Fastly entry point still reads the TLS and HTTP/2 signals on every request to -/// build the host-signal service and client info. +/// Returns the built-in User-Agent-only module unless the `platform` selector +/// is set, in which case it builds the host platform's own module through the +/// `build_platform` factory the adapter supplies. The factory runs only when +/// that module is selected, so device classification itself reads no host +/// signals by default (see [`BuiltinDeviceModule`] for the host-neutral +/// default). An adapter's entry point may still read its platform's signals +/// on every request for other uses, such as the client info. /// /// A module a registration supplies is resolved by the integration registry, /// which refuses a name no module supplies when it is built. A caller uses @@ -387,10 +388,10 @@ impl DeviceModule for BuiltinDeviceModule { #[must_use] pub fn build_device_module( settings: &Settings, - build_fastly: impl FnOnce() -> Box, + build_platform: impl FnOnce() -> Box, ) -> Box { match settings.device.module_key() { - "fastly" => build_fastly(), + "platform" => build_platform(), _ => Box::new(BuiltinDeviceModule::new()), } } @@ -456,8 +457,9 @@ fn parse_platform_class(ua: &str) -> Option { /// /// This is the default, host-neutral gate. It filters obvious non-browser /// traffic but does not resist a bot that forges a complete browser -/// User-Agent. The opt-in Fastly device module strengthens the gate with the -/// TLS and HTTP/2 signals for deployments that need it. +/// User-Agent. The platform's own device module, selected with `platform`, +/// strengthens the gate with the TLS and HTTP/2 signals for deployments that +/// need it. #[must_use] fn looks_like_browser_from_ua(ua: &str, platform_class: Option<&str>) -> bool { platform_class.is_some() && ua.contains("Mozilla/") && !looks_like_bot_ua(ua) @@ -525,7 +527,7 @@ fn compute_h2_fp_hash(raw_h2_fp: &str) -> String { /// `h2_fp_prefix` is the raw H2 SETTINGS string (not the hash) — we /// compare against the hash computed from it. /// -/// Empirically derived from Fastly Compute production responses (2026-04-03). +/// Empirically derived from production traffic at the edge (2026-04-03). const KNOWN_BROWSERS: &[(&str, &str, bool)] = &[ // Chrome/Mac v146 ("t13d1516h2", "1:65536;2:0;4:6291456;6:262144", true), @@ -1086,14 +1088,14 @@ mod tests { OwnedRequestInfo::new(String::new(), headers) } - /// A stand-in for the host-specific module the adapter injects, so the - /// selection logic can be tested in core without the Fastly module crate. - struct StubFastlyModule; + /// A stand-in for the platform's own module the adapter injects, so the + /// selection logic can be tested in core without a platform's module crate. + struct StubPlatformModule; #[async_trait::async_trait(?Send)] - impl DeviceModule for StubFastlyModule { + impl DeviceModule for StubPlatformModule { fn id(&self) -> &'static str { - "fastly" + "stub-platform-module" } async fn detect(&self, _call: ModuleCall<'_>) -> DeviceSignals { @@ -1115,20 +1117,21 @@ mod tests { // injected candidate. let settings = crate::settings::Settings::default(); let default = build_device_module(&settings, || { - Box::new(StubFastlyModule) as Box + Box::new(StubPlatformModule) as Box }); assert_eq!(default.id(), "builtin", "no selector should be UA-only"); - // The `fastly` selector returns the module the adapter's factory builds. - let mut fastly = crate::settings::Settings::default(); - fastly.device.module = Some("fastly".to_owned()); - let selected = build_device_module(&fastly, || { - Box::new(StubFastlyModule) as Box + // The `platform` selector returns the module the adapter's factory + // builds. + let mut platform = crate::settings::Settings::default(); + platform.device.module = Some("platform".to_owned()); + let selected = build_device_module(&platform, || { + Box::new(StubPlatformModule) as Box }); assert_eq!( selected.id(), - "fastly", - "the fastly selector should use the injected module" + "stub-platform-module", + "the platform selector should use the injected module" ); } } diff --git a/crates/trusted-server-core/src/ec/generation.rs b/crates/trusted-server-core/src/ec/generation.rs index 41b5db0c57..41b91b7888 100644 --- a/crates/trusted-server-core/src/ec/generation.rs +++ b/crates/trusted-server-core/src/ec/generation.rs @@ -50,8 +50,8 @@ pub(crate) fn normalize_ip(ip: IpAddr) -> String { /// Generates a random alphanumeric string of the specified length. /// -/// Fastly Compute's `wasm32-wasip1` runtime supplies OS randomness through -/// WASI for `rand::thread_rng`; the CI wasm release build verifies that this +/// On `wasm32-wasip1` the runtime supplies OS randomness through WASI for +/// `rand::thread_rng`, and the CI wasm release build verifies that this /// entropy path remains available for the EC suffix contract. fn generate_random_suffix(length: usize) -> String { let mut rng = rand::thread_rng(); diff --git a/crates/trusted-server-core/src/ec/kv.rs b/crates/trusted-server-core/src/ec/kv.rs index 84011c6c4f..bf9ef422a4 100644 --- a/crates/trusted-server-core/src/ec/kv.rs +++ b/crates/trusted-server-core/src/ec/kv.rs @@ -3,8 +3,7 @@ //! This module provides [`KvIdentityGraph`] which implements the //! read-modify-write operations for the EC identity graph on top of the //! platform-neutral [`EcKvStore`] primitives. The platform adapter supplies -//! the concrete store backend (e.g. the Fastly KV Store implementation in -//! `trusted-server-adapter-fastly`). +//! the concrete store backend. //! //! All methods return `Result` — callers decide whether to swallow errors //! (organic request paths) or propagate them (sync endpoints). See the diff --git a/crates/trusted-server-core/src/ec/kv_backend.rs b/crates/trusted-server-core/src/ec/kv_backend.rs index bfe74ca6f3..4437a62521 100644 --- a/crates/trusted-server-core/src/ec/kv_backend.rs +++ b/crates/trusted-server-core/src/ec/kv_backend.rs @@ -3,8 +3,7 @@ //! [`super::kv::KvIdentityGraph`] owns all identity-graph business logic //! (CAS retry loops, consent tombstone semantics, entry validation) and //! delegates raw store access to an [`EcKvStore`] implementation provided -//! by the adapter crate (e.g. the Fastly KV Store backend in -//! `trusted-server-adapter-fastly`). +//! by the adapter crate. //! //! This trait is intentionally narrow: lookup with a generation marker, //! conditional insert, strong prefix listing, and delete. Conditional writes are diff --git a/crates/trusted-server-core/src/ec/kv_types.rs b/crates/trusted-server-core/src/ec/kv_types.rs index 52bf10b6a0..1a895eb711 100644 --- a/crates/trusted-server-core/src/ec/kv_types.rs +++ b/crates/trusted-server-core/src/ec/kv_types.rs @@ -1,6 +1,6 @@ //! KV identity graph schema types. //! -//! These types define the JSON schema stored in the Fastly KV Store for the +//! These types define the JSON schema stored in the platform's KV store for the //! EC identity graph. Each EC ID (`{64hex}.{6alnum}`) maps to a [`KvEntry`] //! containing consent state, geo location, and accumulated partner IDs. //! @@ -202,8 +202,10 @@ pub struct KvNetwork { /// Compact metadata stored alongside the KV entry body. /// -/// Fastly KV metadata is limited to 2048 bytes and can be read without -/// streaming the full body. Used by batch sync for fast consent checks. +/// Metadata can be read without streaming the full body, and a platform's +/// store may bound its size, so it stays small. An adapter whose store bounds +/// it tests that the largest value fits. Used by batch sync for fast consent +/// checks. #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] pub struct KvMetadata { /// Mirrors [`KvConsent::ok`] — `false` means tombstone. @@ -669,25 +671,6 @@ mod tests { assert!(meta.cluster_size.is_none(), "should default to None"); } - #[test] - fn metadata_fits_in_2048_bytes() { - // Worst case: all fields populated. - let meta = KvMetadata { - ok: false, - country: "XX".to_owned(), - v: SCHEMA_VERSION, - cluster_size: Some(u32::MAX), - is_mobile: Some(2), - known_browser: Some(true), - }; - let json = serde_json::to_string(&meta).expect("should serialize KvMetadata"); - assert!( - json.len() <= 2048, - "metadata must fit in Fastly's 2048-byte limit, got {} bytes", - json.len() - ); - } - #[test] fn new_entry_has_correct_initial_state() { let consent = sample_consent_context(); diff --git a/crates/trusted-server-core/src/ec/mod.rs b/crates/trusted-server-core/src/ec/mod.rs index bf0c51c498..64e9ebcbed 100644 --- a/crates/trusted-server-core/src/ec/mod.rs +++ b/crates/trusted-server-core/src/ec/mod.rs @@ -1365,8 +1365,8 @@ impl EcContext { /// from a consent record the way a deployment does through a signal /// module. It states the answer instead and tests what finalization does /// with it. That a record refusing storage produces this answer is proved - /// where the modules are linked, in the Axum adapter's - /// `permission_signals` test. + /// where the modules are linked, in the `permission_signals` test of an + /// adapter that links them. #[cfg(test)] #[must_use] pub fn with_storage_withdrawn_for_test(mut self, withdrawn: bool) -> Self { @@ -1399,8 +1399,7 @@ fn permissions_for_test(ec_allowed: bool) -> PermissionState { /// /// Uses [`web_time::SystemTime`], which maps to `std::time::SystemTime` on /// native and `wasm32-wasip1` targets and to a JS-backed clock on -/// `wasm32-unknown-unknown` (Cloudflare Workers), where `std::time` is not -/// available. +/// `wasm32-unknown-unknown`, where `std::time` is not available. pub(crate) fn current_timestamp() -> u64 { checked_current_timestamp().unwrap_or(0) } @@ -1567,8 +1566,8 @@ pub(crate) mod tests { #[test] fn read_from_request_does_not_authorize_recovery_from_navigation_headers() { - // Non-Fastly adapters build EC context through the shared read path and - // never call `set_recovery_eligible`. Navigation headers alone must not + // An adapter that builds EC context through the shared read path never + // calls `set_recovery_eligible`. Navigation headers alone must not // authorize orphan recovery or seed KV state. let settings = create_test_settings(); let ec_id = valid_ec_id("b", "CkEc01"); @@ -2862,9 +2861,9 @@ pub(crate) mod tests { /// A geo module whose lookup fails, the state the permission model's /// fail-closed rule exists for. /// - /// No geo module shipped in this workspace can fail: the Fastly SDK's - /// `geo_lookup` returns an `Option`, the Cloudflare module reads request - /// headers, and the Axum and Spin modules resolve nothing at all. The + /// No geo module shipped in this workspace can fail, because each one + /// reads an optional answer from its platform, reads request headers or + /// resolves nothing at all. The /// `Result` on [`PlatformGeo::lookup`] is there for a module that does /// its own fallible lookup, so this stands in for one and proves the floor /// is reached through the seam rather than only from a hand-built status. diff --git a/crates/trusted-server-core/src/ec/module.rs b/crates/trusted-server-core/src/ec/module.rs index fc49f80439..99865b1b38 100644 --- a/crates/trusted-server-core/src/ec/module.rs +++ b/crates/trusted-server-core/src/ec/module.rs @@ -9,10 +9,10 @@ //! service, is built only on a host that supplies that service. Construction //! reads configuration and long-lived services, so a selection this deployment //! cannot satisfy fails at startup rather than leaving it running without an -//! identity. Fastly, Cloudflare and Spin resolve the module once per -//! application state and thread the result. Axum and embedders resolve per -//! request. The host-signal module is the exception, being built per request -//! from that request's TLS and HTTP/2 signals (see +//! identity. An adapter resolves the module once per application state and +//! threads the result, or leaves the request path to resolve it per request, +//! as an embedder does. The host-signal module is the exception, being built +//! per request from that request's TLS and HTTP/2 signals (see //! [`is_request_scoped`](EdgeCookieModule::is_request_scoped)). //! //! Request evidence reaches a module at call time rather than at diff --git a/crates/trusted-server-core/src/ec/rate_limiter.rs b/crates/trusted-server-core/src/ec/rate_limiter.rs index 5a5dac27e5..645673fe1f 100644 --- a/crates/trusted-server-core/src/ec/rate_limiter.rs +++ b/crates/trusted-server-core/src/ec/rate_limiter.rs @@ -2,8 +2,7 @@ //! //! Provides the [`RateLimiter`] trait used by batch sync and pull sync for //! per-partner request rate enforcement. Platform-specific implementations -//! live in the adapter crates (e.g. the Fastly Edge Rate Limiting -//! implementation in `trusted-server-adapter-fastly`). +//! live in the adapter crates. use error_stack::Report; diff --git a/crates/trusted-server-core/src/evidence.rs b/crates/trusted-server-core/src/evidence.rs index 826735da56..6db5cced1b 100644 --- a/crates/trusted-server-core/src/evidence.rs +++ b/crates/trusted-server-core/src/evidence.rs @@ -8,11 +8,11 @@ //! request. //! //! These traits are the service interfaces. Concrete implementations live with -//! the host or vendor that supplies them (for example `FastlyHostSignals` in -//! `trusted-server-device-fastly`). An injected service outlives the borrow of -//! the live request, so an implementation owns its data ([`OwnedRequestInfo`] is -//! the built-in owned snapshot, while [`BorrowedRequestInfo`] is the borrowed -//! view core builds on the request path and must not outlive the request). +//! the host or vendor that supplies them, in an adapter or a device crate. An +//! injected service outlives the borrow of the live request, so an +//! implementation owns its data ([`OwnedRequestInfo`] is the built-in owned +//! snapshot, while [`BorrowedRequestInfo`] is the borrowed view core builds on +//! the request path and must not outlive the request). //! //! [`RequestInfo`] describes what a request carries, not what code in this //! repository happens to read today, so it carries the whole of it whether or @@ -24,8 +24,8 @@ use http::HeaderMap; /// Host-computed client signals that are not carried in request headers. /// -/// A host that can compute them supplies an implementation (Fastly exposes the -/// TLS JA4 and HTTP/2 signals). A module that needs them takes +/// A host that can compute them, such as a platform exposing a connection's +/// TLS and HTTP/2 signals, supplies an implementation. A module that needs them takes /// `Arc` in its constructor, and on a host that supplies none /// the module cannot be built and the request stops. pub trait HostSignals: Send + Sync + core::fmt::Debug { diff --git a/crates/trusted-server-core/src/html_processor.rs b/crates/trusted-server-core/src/html_processor.rs index 7b06d76a69..cc540e799e 100644 --- a/crates/trusted-server-core/src/html_processor.rs +++ b/crates/trusted-server-core/src/html_processor.rs @@ -1667,7 +1667,7 @@ mod tests { // Process - replace test-publisher.com with our edge domain let mut config = create_test_config(); config.origin_host = "www.test-publisher.com".to_owned(); // Match what's in the HTML - config.request_host = "test-publisher-ts.edgecompute.app".to_owned(); + config.request_host = "test-publisher-ts.edge.example".to_owned(); let processor = create_html_processor(config); let pipeline_config = PipelineConfig { @@ -1687,7 +1687,7 @@ mod tests { // Check URL replacements (not all occurrences will be replaced since // we only rewrite attributes, not text/JSON/script bodies) let remaining_urls = result.matches("www.test-publisher.com").count(); - let replaced_urls = result.matches("test-publisher-ts.edgecompute.app").count(); + let replaced_urls = result.matches("test-publisher-ts.edge.example").count(); println!("After processing:"); println!(" Remaining original URLs: {remaining_urls}"); @@ -1750,7 +1750,7 @@ mod tests { // Process with compression let mut config = create_test_config(); config.origin_host = "www.test-publisher.com".to_owned(); // Match what's in the HTML - config.request_host = "test-publisher-ts.edgecompute.app".to_owned(); + config.request_host = "test-publisher-ts.edge.example".to_owned(); let processor = create_html_processor(config); let pipeline_config = PipelineConfig { @@ -1780,7 +1780,7 @@ mod tests { let remaining_urls = decompressed.matches("www.test-publisher.com").count(); let replaced_urls = decompressed - .matches("test-publisher-ts.edgecompute.app") + .matches("test-publisher-ts.edge.example") .count(); assert!(replaced_urls > 0, "Should replace some URLs in attributes"); @@ -1880,7 +1880,7 @@ mod tests { // Process it through our pipeline let mut config = create_test_config(); config.origin_host = "www.test-publisher.com".to_owned(); // Match what's in the HTML - config.request_host = "test-publisher-ts.edgecompute.app".to_owned(); + config.request_host = "test-publisher-ts.edge.example".to_owned(); let processor = create_html_processor(config); let pipeline_config = PipelineConfig { diff --git a/crates/trusted-server-core/src/http_util.rs b/crates/trusted-server-core/src/http_util.rs index a5706f8981..ed7b153449 100644 --- a/crates/trusted-server-core/src/http_util.rs +++ b/crates/trusted-server-core/src/http_util.rs @@ -8,14 +8,14 @@ use std::time::Duration; use subtle::ConstantTimeEq as _; use crate::cache_policy::{CachePolicy, EdgeCacheHeader}; -use crate::constants::INTERNAL_HEADERS; +use crate::constants::is_internal_header; use crate::error::TrustedServerError; use crate::platform::ClientInfo; use crate::settings::{Settings, TrustedClientIpConfig}; /// Copy `X-*` custom headers from one request to another, skipping TS-internal headers. /// -/// This filters out all headers listed in [`INTERNAL_HEADERS`] **and** any header +/// This filters out every header [`is_internal_header`] names **and** any header /// matching the `x-ts-` prefix (case-insensitive) to prevent leaking internal /// identity, geo-enrichment, debugging data, and dynamic `X-ts-` /// headers to downstream third-party services. Integrations that forward custom @@ -27,33 +27,28 @@ pub fn copy_custom_headers(from: &Request, to: &mut Request) // so only the lowercase prefix check is needed. if name_str.starts_with("x-") && !name_str.starts_with("x-ts-") - && !INTERNAL_HEADERS.contains(&name_str) + && !is_internal_header(name_str) { to.headers_mut().append(header_name.clone(), value.clone()); } } } -/// Headers that clients can spoof to hijack URL rewriting or the client address. +/// The standard headers that clients can spoof to hijack URL rewriting. /// -/// On Fastly Compute these values are client-spoofable at request entry. The -/// Fastly adapter may first consume an authenticated `fastly-client-ip`, but -/// removes it before routing along with every other listed header. Stripping -/// them forces [`RequestInfo::from_request`] to fall back to the trustworthy -/// `Host` header and [`ClientInfo`] TLS detection. -pub const SPOOFABLE_FORWARDED_HEADERS: &[&str] = &[ - "forwarded", - "x-forwarded-host", - "x-forwarded-proto", - "fastly-ssl", - "fastly-client-ip", -]; +/// Stripping them at entry forces [`RequestInfo::from_request`] to fall back +/// to the trustworthy `Host` header and [`ClientInfo`] TLS detection. A +/// platform's own spoofable headers are declared by its adapter, see +/// [`crate::platform::declare_platform_spoofable_headers`]. +pub const SPOOFABLE_FORWARDED_HEADERS: &[&str] = + &["forwarded", "x-forwarded-host", "x-forwarded-proto"]; /// Remove the configured client-IP trust headers before routing. /// -/// Only the Fastly adapter consumes these values, but every adapter removes -/// them so a shared configuration cannot expose an authentication secret to -/// publisher or integration request handling. +/// Only an adapter that resolves the trusted client address consumes these +/// values, but every adapter removes them so a shared configuration cannot +/// expose an authentication secret to publisher or integration request +/// handling. pub fn sanitize_trusted_client_ip_headers( req: &mut Request, config: Option<&TrustedClientIpConfig>, @@ -69,11 +64,17 @@ pub fn sanitize_trusted_client_ip_headers( /// Call this at the edge entry point (before routing) to prevent /// `X-Forwarded-Host: evil.com` from hijacking all URL rewriting. /// See . +/// +/// Strips the standard [`SPOOFABLE_FORWARDED_HEADERS`] and every TLS and +/// spoofable header the running platform declared. pub fn sanitize_forwarded_headers(req: &mut Request) { - for header in SPOOFABLE_FORWARDED_HEADERS { - if req.headers().contains_key(*header) { + let declared = crate::platform::platform_tls_headers() + .into_iter() + .chain(crate::platform::platform_spoofable_headers()); + for header in SPOOFABLE_FORWARDED_HEADERS.iter().copied().chain(declared) { + if req.headers().contains_key(header) { log::debug!("Stripped spoofable header: {header}"); - req.headers_mut().remove(*header); + req.headers_mut().remove(header); } } } @@ -121,20 +122,21 @@ pub fn is_navigation_request(req: &Request) -> bool { /// /// This struct captures the effective host and scheme from an incoming request. /// The parser checks forwarded headers (`Forwarded`, `X-Forwarded-Host`, -/// `X-Forwarded-Proto`) as fallbacks, but on the Fastly edge +/// `X-Forwarded-Proto`) as fallbacks, but at the edge /// [`sanitize_forwarded_headers`] strips those headers before this method is -/// called, so the `Host` header and Fastly SDK TLS detection are the effective -/// sources in production. +/// called, so the `Host` header and the TLS status the adapter reports are the +/// effective sources in production. #[derive(Debug, Clone)] pub struct RequestInfo { /// The effective host for URL rewriting (typically the `Host` header after edge sanitization). pub host: String, - /// The effective scheme (typically from Fastly SDK TLS detection after edge sanitization). + /// The effective scheme (typically from the TLS status the adapter reports, after edge + /// sanitization). pub scheme: String, } impl RequestInfo { - /// Extract request info from a Fastly request. + /// Extract request info from a request. /// /// Host fallback order (first present wins): /// 1. `Forwarded` header (`host=...`) @@ -144,10 +146,10 @@ impl RequestInfo { /// in place of a `Host` header /// /// Scheme fallback order: - /// 1. Fastly SDK TLS detection + /// 1. The TLS status the adapter reports in [`ClientInfo`] /// 2. `Forwarded` header (`proto=...`) /// 3. `X-Forwarded-Proto` - /// 4. `Fastly-SSL` + /// 4. A TLS header the platform declares /// 5. Default `http` /// /// In production the forwarded headers are stripped by @@ -270,13 +272,15 @@ fn normalize_scheme(value: &str) -> Option<&'static str> { } } -/// Detects the request scheme (HTTP or HTTPS) using Fastly SDK methods and headers. +/// Detects the request scheme (HTTP or HTTPS) from the adapter's TLS status +/// and headers. /// /// Tries multiple methods in order of reliability: -/// 1. Fastly SDK TLS detection methods (most reliable) +/// 1. The TLS status the adapter reports in [`ClientInfo`] (most reliable) /// 2. Forwarded header (RFC 7239) /// 3. X-Forwarded-Proto header -/// 4. Fastly-SSL header (trusted on `EdgeZero` path; can be spoofed on legacy path) +/// 4. A TLS header the platform declares, which its entry point sets after +/// stripping any a client sent /// 5. Default to HTTP fn detect_request_scheme( req: &Request, @@ -313,22 +317,16 @@ fn detect_request_scheme( return scheme; } - // 4. Check Fastly-SSL header. On the `EdgeZero` path this is injected from - // authoritative Fastly TLS metadata after spoofable headers are stripped, - // so it is reliable. On direct or legacy paths it can be spoofed by clients. - // - // Layering wart: this is a vendor-specific header name living in - // platform-neutral core. It is only a fallback — signal #1 above - // (`ClientInfo::tls_protocol`) is the neutral path adapters populate. The - // `fastly-ssl` fallback (plus its entry in `SPOOFABLE_FORWARDED_HEADERS` - // and the origin-forwarding strip in `publisher::rewrite_origin_request`) - // should be replaced by a platform-neutral scheme signal in a separate - // change, after confirming the legacy path is covered by `ClientInfo`. - if let Some(ssl) = req.headers().get("fastly-ssl") - && let Ok(ssl_str) = ssl.to_str() - && (ssl_str == "1" || ssl_str.to_lowercase() == "true") - { - return "https"; + // 4. Check the TLS headers the platform declared. Its entry point sets one + // from the platform's own TLS metadata after stripping any a client + // sent, so it is reliable there. + for name in crate::platform::platform_tls_headers() { + if let Some(ssl) = req.headers().get(name) + && let Ok(ssl_str) = ssl.to_str() + && (ssl_str == "1" || ssl_str.to_lowercase() == "true") + { + return "https"; + } } // Default to HTTP @@ -783,17 +781,27 @@ mod tests { } #[test] - fn test_request_info_scheme_from_fastly_ssl() { + fn test_request_info_scheme_from_a_declared_tls_header() { + crate::platform::declare_platform_tls_headers(&["example-platform-tls"]); let mut req = build_request(Method::GET, "https://test.example.com/page"); - set_header(&mut req, "fastly-ssl", "1"); + set_header(&mut req, "example-platform-tls", "1"); let info = RequestInfo::from_request(&req, &default_client_info()); assert_eq!( info.scheme, "https", - "Scheme should fall back to Fastly-SSL when other signals are missing" + "Scheme should fall back to the platform's TLS header when other signals are missing" ); } + #[test] + fn an_undeclared_tls_header_says_nothing_about_the_scheme() { + let mut req = build_request(Method::GET, "https://test.example.com/page"); + set_header(&mut req, "example-undeclared-tls", "1"); + + let info = RequestInfo::from_request(&req, &default_client_info()); + assert_eq!(info.scheme, "http"); + } + #[test] fn test_request_info_chained_proxy_scenario() { // Simulate: Client (HTTPS) -> Proxy A -> Trusted Server (HTTP internally) @@ -858,7 +866,10 @@ mod tests { set_header(&mut req, "forwarded", "host=evil.com;proto=https"); set_header(&mut req, "x-forwarded-host", "evil.com"); set_header(&mut req, "x-forwarded-proto", "https"); - set_header(&mut req, "fastly-ssl", "1"); + crate::platform::declare_platform_tls_headers(&["example-platform-tls"]); + crate::platform::declare_platform_spoofable_headers(&["example-platform-client-ip"]); + set_header(&mut req, "example-platform-tls", "1"); + set_header(&mut req, "example-platform-client-ip", "203.0.113.9"); sanitize_forwarded_headers(&mut req); @@ -875,8 +886,12 @@ mod tests { "should strip X-Forwarded-Proto header" ); assert!( - req.headers().get("fastly-ssl").is_none(), - "should strip Fastly-SSL header" + req.headers().get("example-platform-tls").is_none(), + "should strip the platform's TLS header" + ); + assert!( + req.headers().get("example-platform-client-ip").is_none(), + "should strip the platform's spoofable header" ); assert_eq!( req.headers() diff --git a/crates/trusted-server-core/src/integrations/mod.rs b/crates/trusted-server-core/src/integrations/mod.rs index e3e710b4c2..3e9a48118e 100644 --- a/crates/trusted-server-core/src/integrations/mod.rs +++ b/crates/trusted-server-core/src/integrations/mod.rs @@ -226,7 +226,7 @@ pub async fn collect_body_bounded( })); } // Size check runs after chunk is materialized — effective bound is - // ≤ max_bytes + one_chunk (Fastly H2/H3 chunks are ≤ 16 KiB in practice). + // ≤ max_bytes + one_chunk (H2/H3 chunks are ≤ 16 KiB in practice). body_bytes.extend_from_slice(&chunk); } Ok(body_bytes) @@ -244,8 +244,8 @@ pub async fn collect_body_bounded( /// [`TrustedServerError::RequestTooLarge`] (413). /// /// Note: the effective bound for streaming bodies is ≤ `max_bytes` + `one_chunk` -/// because the size check runs after each chunk is materialized. Fastly -/// H2/H3 chunks are ≤ 16 KiB in practice, making the overshoot negligible. +/// because the size check runs after each chunk is materialized. H2/H3 +/// chunks are ≤ 16 KiB in practice, making the overshoot negligible. /// /// Public because an integration crate outside this one reads upstream /// responses too, and an unbounded read of a misbehaving upstream is not a @@ -284,7 +284,7 @@ pub async fn collect_response_bounded( }) })?; // Size check runs after chunk is materialized — effective bound is - // ≤ max_bytes + one_chunk (Fastly H2/H3 chunks are ≤ 16 KiB in practice). + // ≤ max_bytes + one_chunk (H2/H3 chunks are ≤ 16 KiB in practice). if body_bytes.len() + chunk.len() > max_bytes { return Err(Report::new(TrustedServerError::Integration { integration: integration.to_string(), diff --git a/crates/trusted-server-core/src/integrations/registry.rs b/crates/trusted-server-core/src/integrations/registry.rs index b390ee7585..e57a587070 100644 --- a/crates/trusted-server-core/src/integrations/registry.rs +++ b/crates/trusted-server-core/src/integrations/registry.rs @@ -1117,11 +1117,12 @@ const GEO_MODULE_PLATFORM: &str = "platform"; /// The same choice as leaving the selector unset, spelled explicitly. const DEVICE_MODULE_BUILTIN: &str = "builtin"; -/// `[device] module` value opting in to the host module the adapter builds. +/// `[device] module` value opting in to the host platform's own module, which +/// the adapter builds. /// /// Resolved by `build_device_module` in [`device`](crate::ec::device) rather /// than by a module, so the registry supplies nothing for it. -const DEVICE_MODULE_FASTLY: &str = "fastly"; +const DEVICE_MODULE_PLATFORM: &str = "platform"; /// The type folder each of these kinds of module is selected in, which a name /// written in its section may leave off. @@ -1347,9 +1348,9 @@ fn resolve_device_module( let selector = settings.device.module.as_deref(); let resolved = match selector { // Unset and `builtin` both name the module core supplies itself, and - // `fastly` names the host module the adapter builds, so the registry - // supplies nothing for any of the three. - None | Some(DEVICE_MODULE_BUILTIN) | Some(DEVICE_MODULE_FASTLY) => None, + // `platform` names the host platform's own module the adapter builds, + // so the registry supplies nothing for any of the three. + None | Some(DEVICE_MODULE_BUILTIN) | Some(DEVICE_MODULE_PLATFORM) => None, Some(written) => Some(named_module( DEVICE_TYPE, "device", @@ -2469,13 +2470,11 @@ impl IntegrationRegistry { // The served `?v=` hash and its memo trust this value, so a // stale literal is a startup error rather than a stale // script. The cost is one SHA-256 of each carried module - // per registry build, and only the Axum dev server builds - // the registry once, because its `main` builds the router - // before serving. Fastly starts a fresh Wasm instance per - // request, and `edgezero_adapter_cloudflare::run_app` and - // `edgezero_adapter_spin::run_app` both call `build_app` - // inside the per-request entry point, so those three pay - // it on every request. + // per registry build. A native server builds the registry + // once, because its `main` builds the router before + // serving, while an adapter that starts a fresh instance + // per request, or builds the app inside its per-request + // entry point, pays it on every request. let actual = hex::encode(Sha256::digest(module.source.as_bytes())); if actual != module.sha256 { return Err(Report::new(TrustedServerError::Configuration { @@ -4861,7 +4860,7 @@ mod tests { } #[test] - fn handle_proxy_passes_http_request_without_fastly_round_trip() { + fn handle_proxy_passes_http_request_without_a_platform_round_trip() { let settings = create_test_settings(); let registry = IntegrationRegistry::from_routes(vec![( http::Method::GET, diff --git a/crates/trusted-server-core/src/lib.rs b/crates/trusted-server-core/src/lib.rs index 343b5f0ddf..26d98f091c 100644 --- a/crates/trusted-server-core/src/lib.rs +++ b/crates/trusted-server-core/src/lib.rs @@ -1,7 +1,7 @@ //! Common functionality for the trusted server. //! -//! This crate provides shared types, utilities, and abstractions used by both -//! the Fastly edge implementation and local development/testing environments. +//! This crate provides shared types, utilities, and abstractions used by every +//! adapter, on edge platforms and in local development and testing. //! //! # Modules //! @@ -84,6 +84,3 @@ pub mod test_support; pub mod tester_cookie; pub mod tsjs; pub mod tsjs_bundle; - -#[cfg(test)] -mod migration_guards; diff --git a/crates/trusted-server-core/src/module_context.rs b/crates/trusted-server-core/src/module_context.rs index 1f99ff3b68..889997e8ae 100644 --- a/crates/trusted-server-core/src/module_context.rs +++ b/crates/trusted-server-core/src/module_context.rs @@ -409,8 +409,8 @@ impl<'r> ModuleContext<'r> { /// what the geo module in `services` declares, the device signals what the /// device module in `services` declares, and the identifier what the Edge /// Cookie module the state was built with declares. Signals from a device - /// module the adapter builds itself, as Fastly does with none registered, - /// are gated by nothing, so such a module must declare no permission. + /// module an adapter builds itself while none is registered are gated by + /// nothing, so such a module must declare no permission. #[must_use] pub fn with_request_state(self, state: &'r EcContext, services: &'r RuntimeServices) -> Self { let mut context = self diff --git a/crates/trusted-server-core/src/platform/http.rs b/crates/trusted-server-core/src/platform/http.rs index b47abaebbb..208715700e 100644 --- a/crates/trusted-server-core/src/platform/http.rs +++ b/crates/trusted-server-core/src/platform/http.rs @@ -9,11 +9,11 @@ use super::image_optimizer::PlatformImageOptimizerOptions; /// What the caller wants the platform's intermediary cache to do with this request. /// -/// One enum rather than independent flags because on Fastly they are *not* independent: -/// `set_surrogate_key` and `set_ttl` each "override any previous `Request::set_pass` call" -/// (`fastly-0.12.1/src/http/request.rs:2462`, `:2381`). A pair of booleans could express -/// "bypass the cache, and tag it for purge", which on Fastly silently means "do not -/// bypass" — the opposite of how it reads. This type makes that combination unspellable. +/// One enum rather than independent flags because on some platforms they are *not* +/// independent: tagging a request for purge, or giving it a TTL, overrides an earlier +/// cache bypass. A pair of booleans could express "bypass the cache, and tag it for +/// purge", which there silently means "do not bypass", the opposite of how it reads. +/// This type makes that combination unspellable. #[derive(Debug, Clone, PartialEq, Eq, Default)] pub enum PlatformCacheIntent { /// Let the platform apply its default behavior, honoring origin freshness. @@ -38,7 +38,7 @@ impl PlatformCacheIntent { /// The surrogate key to tag the stored object with, when there is one. /// /// `None` for both [`Self::Default`] and [`Self::Bypass`]: attaching a key to a - /// bypassed request would reverse the bypass on Fastly. + /// bypassed request would reverse the bypass on some platforms. #[must_use] pub fn surrogate_key(&self) -> Option<&str> { match self { @@ -51,8 +51,8 @@ impl PlatformCacheIntent { /// Outbound HTTP request paired with a pre-resolved backend name. /// /// Uses `EdgeZero`'s neutral [`EdgeRequest`] type so adapters share one -/// HTTP/body model while still preserving backend correlation for Fastly-style -/// fan-out. +/// HTTP/body model while still preserving backend correlation for fan-out +/// across named backends. #[derive(Debug)] pub struct PlatformHttpRequest { /// Platform-neutral request to send upstream. @@ -91,9 +91,10 @@ impl PlatformHttpRequest { /// Attach Image Optimizer metadata to the request. /// - /// The current Fastly adapter supports this on [`PlatformHttpClient::send`] - /// and rejects it on [`PlatformHttpClient::send_async`] because Fastly IO is - /// not available through the fan-out helper path. + /// An adapter whose platform optimizes images may support this on + /// [`PlatformHttpClient::send`] alone and reject it on + /// [`PlatformHttpClient::send_async`], where the optimizer is not available + /// through the fan-out helper path. #[must_use] pub fn with_image_optimizer(mut self, options: PlatformImageOptimizerOptions) -> Self { self.image_optimizer = Some(options); @@ -169,7 +170,7 @@ impl PlatformResponse { /// bound exists because [`edgezero_core::body::Body`] wraps a /// `LocalBoxStream` that is intentionally `!Send` for wasm32 compatibility — /// wasm32 targets are single-threaded and cannot use `Send` futures. -/// Adapter crates targeting a multi-threaded runtime (e.g. Axum with tokio) +/// Adapter crates targeting a multi-threaded runtime (e.g. tokio on a server) /// would need to wrap state in `Arc` rather than relying on `Send` here. /// /// See [`PlatformHttpClient`] for the trait-level `?Send` design rationale, @@ -277,8 +278,8 @@ pub struct PlatformSelectResult { /// Requests still in flight after the ready result is removed. /// /// Note: entries in this list may have `backend_name == None` on some - /// adapters (e.g., Fastly), because the adapter cannot preserve input - /// identifiers across the platform `select()` call. Orchestrators must + /// adapters, because the adapter cannot preserve input identifiers across + /// the platform `select()` call. Orchestrators must /// not rely on `backend_name()` being set on remaining entries. pub remaining: Vec, /// Backend name of the request that became ready with an error, when @@ -288,8 +289,8 @@ pub struct PlatformSelectResult { } /// A [`PlatformHttpClient`] stand-in used when outbound HTTP is not available -/// on the current platform (e.g. Cloudflare Workers, where the proxy client is -/// managed by the edgezero dispatch layer instead). +/// on the current platform (for example where the proxy client is managed by +/// the edgezero dispatch layer instead). /// /// Every method returns [`PlatformError::HttpClient`], ensuring that code paths /// that reach this stub receive a typed error. Adapter crates should use this @@ -363,8 +364,8 @@ pub trait PlatformHttpClient: Send + Sync { /// pending requests progress concurrently and [`select`](Self::select) /// races them. /// - /// Platforms where `send_async` executes each request eagerly before - /// returning (e.g. Cloudflare Workers) return `false`. On such platforms + /// A platform whose `send_async` executes each request eagerly before + /// returning answers `false`. On such platforms /// multi-request fan-out runs sequentially and accrues the sum of the /// individual latencies, so callers with a latency budget (the auction /// orchestrator) must check this before launching more than one request. @@ -474,8 +475,8 @@ mod tests { #[test] fn cache_intent_cannot_request_bypass_and_a_surrogate_key_at_once() { - // The whole reason this is an enum. On Fastly a surrogate key reverses a prior - // set_pass, so "bypass, and tag for purge" would silently mean "do not bypass". + // The whole reason this is an enum. On some platforms a surrogate key reverses a + // prior bypass, so "bypass, and tag for purge" would silently mean "do not bypass". let bypass = PlatformCacheIntent::Bypass; let shared = PlatformCacheIntent::Shared { surrogate_key: "ts-origin".to_owned(), @@ -526,7 +527,7 @@ mod tests { #[test] fn platform_response_default_has_no_backend_name() { - // On Axum/Cloudflare noop clients return PlatformResponse::new(response) + // An adapter's noop client returns PlatformResponse::new(response) // with no backend_name. Core logic must not panic when backend_name is None. let response = edgezero_core::http::response_builder() .status(200) @@ -543,7 +544,8 @@ mod tests { #[test] fn platform_response_with_backend_name_is_some() { - // On Fastly, responses carry backend_name for error correlation. + // An adapter that sends to named backends tags responses with + // backend_name for error correlation. let response = edgezero_core::http::response_builder() .status(200) .body(edgezero_core::body::Body::empty()) diff --git a/crates/trusted-server-core/src/platform/image_optimizer.rs b/crates/trusted-server-core/src/platform/image_optimizer.rs index 9a3f1709eb..2acba81f58 100644 --- a/crates/trusted-server-core/src/platform/image_optimizer.rs +++ b/crates/trusted-server-core/src/platform/image_optimizer.rs @@ -1,18 +1,19 @@ //! Platform-neutral Image Optimizer request metadata. //! //! These DTOs describe the closed image transformation set that Trusted Server -//! attaches to an outbound asset proxy request. They intentionally avoid Fastly -//! SDK types so the profile-table logic can stay in core while the Fastly -//! adapter remains responsible for translating metadata into -//! `fastly::image_optimizer::ImageOptimizerOptions`. +//! attaches to an outbound asset proxy request. They intentionally avoid any +//! platform SDK's types so the profile-table logic can stay in core while an +//! adapter whose platform optimizes images translates the metadata into that +//! SDK's options. //! //! Unsupported adapters should reject requests carrying this metadata rather //! than silently dropping transformations. /// Platform-neutral Image Optimizer processing region. /// -/// These variants mirror the regions currently exposed by the Fastly SDK while -/// keeping config validation in core independent from adapter-specific types. +/// These variants mirror the processing regions a platform's image optimizer +/// offers, while keeping config validation in core independent from +/// adapter-specific types. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum PlatformImageOptimizerRegion { /// Apply image optimization in the eastern United States. @@ -55,10 +56,10 @@ impl PlatformImageOptimizerRegion { /// Platform-neutral Image Optimizer options for an upstream request. /// -/// Core code stores only a closed transformation set here. The Fastly adapter is -/// responsible for translating these values to SDK-specific -/// `ImageOptimizerOptions`, while adapters without Image Optimizer support -/// should reject requests carrying this metadata. +/// Core code stores only a closed transformation set here. An adapter whose +/// platform optimizes images translates these values to its SDK's options, +/// while adapters without Image Optimizer support should reject requests +/// carrying this metadata. #[derive(Debug, Clone, PartialEq, Eq)] pub struct PlatformImageOptimizerOptions { /// Image Optimizer processing region understood by the target adapter. @@ -176,6 +177,6 @@ impl PlatformImageOptimizerCrop { /// Platform-neutral crop focus mode. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum PlatformImageOptimizerCropMode { - /// Use Fastly IO smart crop mode. + /// Use the image optimizer's smart crop mode. Smart, } diff --git a/crates/trusted-server-core/src/platform/mod.rs b/crates/trusted-server-core/src/platform/mod.rs index c882703cc0..08f73cef39 100644 --- a/crates/trusted-server-core/src/platform/mod.rs +++ b/crates/trusted-server-core/src/platform/mod.rs @@ -1,8 +1,7 @@ //! Platform abstraction layer for `trusted-server-core`. //! //! This module defines platform-neutral service contracts and request-scoped -//! runtime state. Concrete implementations live in adapter crates such as -//! `trusted-server-adapter-fastly`. +//! runtime state. Concrete implementations live in the adapter crates. //! //! ## Traits //! @@ -23,7 +22,7 @@ //! The following components were evaluated for platform-specific behavior //! (verified 2026-03-31; see `docs/superpowers/plans/2026-03-31-pr8-content-rewriting-verification.md`) //! and found to have a platform-agnostic rewriting pipeline. No -//! platform trait is required; future adapters (Cloudflare Workers, Axum, Spin) need not provide +//! platform trait is required, and no adapter needs to provide //! any content-rewriting implementation: //! //! - **Content rewriting** — `html_processor`, `streaming_processor`, @@ -47,7 +46,9 @@ mod image_optimizer; mod kv; mod loading_cache; mod policy_source; +mod request_headers; mod server_key; +mod staging; mod template_assembly; mod template_cache; #[cfg(any(test, feature = "test-utils"))] @@ -86,7 +87,13 @@ pub use policy_source::{ POLICY_KEY_PREFIX, PlatformPolicySource, PolicyEntry, PolicyOrigin, policy_entry_text, policy_key, }; +pub use request_headers::{ + declare_platform_internal_headers, declare_platform_spoofable_headers, + declare_platform_tls_headers, is_platform_internal_header, platform_spoofable_headers, + platform_tls_headers, +}; pub use server_key::ServerKeyPrefix; +pub use staging::{declare_platform_staging, platform_is_staging}; pub use template_assembly::{ PlatformTemplateAssembler, TemplateAssemblyError, UnavailableTemplateAssembler, contains_publisher_esi_directive, diff --git a/crates/trusted-server-core/src/platform/request_headers.rs b/crates/trusted-server-core/src/platform/request_headers.rs new file mode 100644 index 0000000000..2a128f527a --- /dev/null +++ b/crates/trusted-server-core/src/platform/request_headers.rs @@ -0,0 +1,99 @@ +//! The request headers a hosting platform sets itself, named by its adapter. +//! +//! Core reads and strips the standard forwarding headers on every platform. +//! A platform's own are declared by its adapter, once, before it serves: +//! +//! - a header the platform sets to `1` on a request that arrived over TLS, +//! which core reads for the scheme when nothing better says and never +//! forwards to an origin, +//! - any other header of the platform's that a client could also send, such +//! as one carrying the client's address, +//! - the platform's internal headers, which are never passed on to a third +//! party. +//! +//! A header of the first two kinds is removed at entry, with the standard +//! forwarding headers, so only the value the platform set survives. Each +//! list keeps a name declared twice once. + +use std::sync::{Mutex, PoisonError}; + +static TLS: Mutex> = Mutex::new(Vec::new()); +static SPOOFABLE: Mutex> = Mutex::new(Vec::new()); +static INTERNAL: Mutex> = Mutex::new(Vec::new()); + +fn declare(list: &Mutex>, names: &[&'static str]) { + let mut declared = list.lock().unwrap_or_else(PoisonError::into_inner); + for name in names { + if !declared.contains(name) { + declared.push(name); + } + } +} + +fn declared(list: &Mutex>) -> Vec<&'static str> { + list.lock().unwrap_or_else(PoisonError::into_inner).clone() +} + +/// Declares the headers, each in lower case, that the hosting platform sets +/// to `1` on a request that arrived over TLS. +pub fn declare_platform_tls_headers(names: &[&'static str]) { + declare(&TLS, names); +} + +/// The TLS headers the running platform declared, in declaration order. +#[must_use] +pub fn platform_tls_headers() -> Vec<&'static str> { + declared(&TLS) +} + +/// Declares the hosting platform's own headers, each in lower case, that a +/// client could also send, beyond its TLS headers. +pub fn declare_platform_spoofable_headers(names: &[&'static str]) { + declare(&SPOOFABLE, names); +} + +/// The spoofable headers the running platform declared, beyond its TLS +/// headers, in declaration order. +#[must_use] +pub fn platform_spoofable_headers() -> Vec<&'static str> { + declared(&SPOOFABLE) +} + +/// Declares the hosting platform's internal headers, each in lower case, +/// which are never passed on to a third party. +pub fn declare_platform_internal_headers(names: &[&'static str]) { + declare(&INTERNAL, names); +} + +/// Whether the running platform declared `name`, in lower case, as one of +/// its internal headers. +#[must_use] +pub fn is_platform_internal_header(name: &str) -> bool { + INTERNAL + .lock() + .unwrap_or_else(PoisonError::into_inner) + .contains(&name) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_declared_header_is_listed_once_in_its_own_list() { + declare_platform_tls_headers(&["example-twice-tls"]); + declare_platform_tls_headers(&["example-twice-tls"]); + declare_platform_internal_headers(&["example-twice-internal"]); + + assert_eq!( + platform_tls_headers() + .iter() + .filter(|name| **name == "example-twice-tls") + .count(), + 1 + ); + assert!(!platform_spoofable_headers().contains(&"example-twice-tls")); + assert!(is_platform_internal_header("example-twice-internal")); + assert!(!is_platform_internal_header("example-twice-tls")); + } +} diff --git a/crates/trusted-server-core/src/platform/staging.rs b/crates/trusted-server-core/src/platform/staging.rs new file mode 100644 index 0000000000..5875213899 --- /dev/null +++ b/crates/trusted-server-core/src/platform/staging.rs @@ -0,0 +1,23 @@ +//! Whether the service runs in its hosting platform's staging environment. +//! +//! A platform that serves a version under test apart from the live one tells +//! the running service so. Its adapter declares it once, before it serves, +//! and a module that behaves differently under test, such as a test bypass, +//! reads it here rather than reading the platform's own variable. + +use std::sync::atomic::{AtomicBool, Ordering}; + +static STAGING: AtomicBool = AtomicBool::new(false); + +/// Declares whether the service runs in its hosting platform's staging +/// environment. +pub fn declare_platform_staging(staging: bool) { + STAGING.store(staging, Ordering::Relaxed); +} + +/// Whether the adapter declared that the service runs in its hosting +/// platform's staging environment. False until it does. +#[must_use] +pub fn platform_is_staging() -> bool { + STAGING.load(Ordering::Relaxed) +} diff --git a/crates/trusted-server-core/src/platform/template_cache.rs b/crates/trusted-server-core/src/platform/template_cache.rs index 62873d58d2..5a8ca1c4a4 100644 --- a/crates/trusted-server-core/src/platform/template_cache.rs +++ b/crates/trusted-server-core/src/platform/template_cache.rs @@ -29,8 +29,7 @@ use crate::creative_opportunities::AssemblyMode; /// The build digest in the template fingerprint already isolates every core change, /// including the metadata encoding, the seam marker, and Rust-inlined head programs, /// so those need no bump. Bump this only when code outside that digest changes -/// template compatibility, such as entry storage in -/// `trusted-server-adapter-fastly/src/template_cache.rs` or the Rust side of +/// template compatibility, such as an adapter's entry storage or the Rust side of /// `trusted-server-js`, unless another fingerprint input already isolates the change. /// /// | Version | Transform | @@ -772,9 +771,10 @@ impl fmt::Debug for dyn PlatformTemplateCache { /// A platform's shared-template cache. /// -/// Only the Fastly adapter implements this; every other adapter uses -/// [`UnavailableTemplateCache`], which reports [`TemplateCacheMiss::Unsupported`] so -/// the caller transforms every time rather than failing. +/// An adapter whose platform has a shared cache implements this, and every other +/// adapter uses [`UnavailableTemplateCache`], which reports +/// [`TemplateCacheMiss::Unsupported`] so the caller transforms every time rather +/// than failing. /// /// `Send + Sync` on the trait, `?Send` on the futures: `RuntimeServices` is held in a /// `LazyLock` static, so the trait object must cross threads even though the futures @@ -833,8 +833,9 @@ pub struct TemplateEntry { /// The null object, used by every adapter without a template cache. /// /// Reporting [`TemplateCacheMiss::Unsupported`] rather than erroring means the -/// ESI assembly mode degrades to transforming per request on Cloudflare, Axum and Spin -/// instead of failing — the mode stays portable, only the caching is not. +/// ESI assembly mode degrades to transforming per request on an adapter without a +/// template cache instead of failing, so the mode stays portable and only the +/// caching is not. pub struct UnavailableTemplateCache; #[async_trait::async_trait(?Send)] @@ -1558,7 +1559,7 @@ mod tests { #[tokio::test] async fn the_null_object_reports_unsupported_rather_than_failing() { // Degrading to per-request transformation keeps the shared modes portable on - // adapters with no cache; erroring would make them Fastly-only outright. + // adapters with no cache, where erroring would make them unusable. let cache = UnavailableTemplateCache; assert_eq!( cache.get(&key()).await.err(), diff --git a/crates/trusted-server-core/src/platform/test_support.rs b/crates/trusted-server-core/src/platform/test_support.rs index 3daa5fea78..3d4bafa155 100644 --- a/crates/trusted-server-core/src/platform/test_support.rs +++ b/crates/trusted-server-core/src/platform/test_support.rs @@ -349,13 +349,13 @@ pub struct StubHttpClient { // Test-only wall-clock delays applied before each select result is returned. select_delays: Mutex>, // Reported by supports_concurrent_fanout(); set false to emulate - // platforms whose send_async executes eagerly (e.g. Cloudflare Workers). + // platforms whose send_async executes eagerly. concurrent_fanout: std::sync::atomic::AtomicBool, // Reported by has_enforceable_total_request_deadline(); set true to emulate // a future adapter with a hard total request deadline. enforceable_total_request_deadline: std::sync::atomic::AtomicBool, - // Reported by supports_streaming_responses(); set true to emulate Fastly's - // streaming response support. + // Reported by supports_streaming_responses(); set true to emulate a + // platform that streams responses. streaming_responses_supported: std::sync::atomic::AtomicBool, pending_streaming_responses_supported: std::sync::atomic::AtomicBool, image_optimizer_options: Mutex>>, @@ -824,7 +824,7 @@ impl PlatformHttpClient for StubHttpClient { /// Always marks the first pending request in the input as ready (FIFO order). /// - /// This differs from Fastly's production `select()`, which returns whichever + /// This differs from a platform's production `select()`, which returns whichever /// request completes first and makes no ordering guarantees. Tests that rely on /// this stub should not depend on "first-pushed = first-ready" semantics, and /// should document their ordering assumptions explicitly if order matters. @@ -864,9 +864,10 @@ impl PlatformHttpClient for StubHttpClient { let ready_backend_name = stub.backend_name.clone(); - // Strip backend names from remaining to match Fastly production behavior: - // Fastly's select() rebuilds remaining with PlatformPendingRequest::new() - // (no backend_name) — orchestrators must not rely on names being set. + // Strip backend names from remaining to match production behavior, where + // a platform's select() may rebuild remaining with + // PlatformPendingRequest::new() (no backend_name), so orchestrators must + // not rely on names being set. let remaining: Vec = pending_requests .into_iter() .map(|r| match r.downcast::() { @@ -1661,7 +1662,7 @@ mod tests { assert_eq!( result.remaining[0].backend_name(), None, - "should strip backend name from remaining (matches Fastly production behavior)" + "should strip backend name from remaining (matches production select behavior)" ); let names = stub.recorded_backend_names(); diff --git a/crates/trusted-server-core/src/platform/traits.rs b/crates/trusted-server-core/src/platform/traits.rs index 85076321fd..92585c9c4c 100644 --- a/crates/trusted-server-core/src/platform/traits.rs +++ b/crates/trusted-server-core/src/platform/traits.rs @@ -7,8 +7,8 @@ use super::{GeoInfo, PlatformBackendSpec, PlatformError, RuntimeServices, StoreI /// Synchronous, object-safe access to a key-value config store. /// /// Reads use the edge-visible store name. Writes use the platform management -/// store identifier because Fastly separates the runtime store name from the -/// management API store ID. +/// store identifier because a platform may separate the runtime store name +/// from the management API store ID. pub trait PlatformConfigStore: Send + Sync { /// Retrieve a string value from `store_name` by `key`. /// @@ -171,8 +171,8 @@ pub trait PlatformGeo: Send + Sync { /// ISO 3166-1 alpha-2 code (for example `US`) and the region as the /// ISO 3166-2 subdivision code without the country prefix (for example /// `CA`). The permission model keys its country and region rules on these - /// codes, matched case-insensitively, so the Fastly and other geo - /// modules feed the same rules without translation. + /// codes, matched case-insensitively, so every platform's geo module + /// feeds the same rules without translation. /// /// # Errors /// diff --git a/crates/trusted-server-core/src/platform/types.rs b/crates/trusted-server-core/src/platform/types.rs index 4dc09e08e0..4b4f7c53d7 100644 --- a/crates/trusted-server-core/src/platform/types.rs +++ b/crates/trusted-server-core/src/platform/types.rs @@ -216,7 +216,8 @@ pub struct RuntimeServices { /// /// `None` when the adapter resolved nothing here, in which case the request /// path resolves the selection itself, which is what a deployment that - /// selects no module, the Axum adapter, and the core tests all do. + /// selects no module, an adapter that threads none, and the core tests all + /// do. pub(crate) resolved_ec_module: Option>, pub(crate) device_module: Option>, /// The asset cache module `[asset-cache]` selected, which may answer for @@ -478,8 +479,8 @@ impl RuntimeServices { /// Adapters that build their per-request services through a shared helper /// with no application state in hand use this to thread the module the /// composition root resolved. `None` leaves the request path to resolve - /// `[ec] module` for itself, which is what the Axum adapter and a - /// deployment selecting no module both do. + /// `[ec] module` for itself, which is what an adapter that threads no + /// module and a deployment selecting no module both do. #[must_use] pub fn with_resolved_ec_module( self, diff --git a/crates/trusted-server-core/src/proxy.rs b/crates/trusted-server-core/src/proxy.rs index c2970f5dad..cbc991c2d2 100644 --- a/crates/trusted-server-core/src/proxy.rs +++ b/crates/trusted-server-core/src/proxy.rs @@ -119,7 +119,7 @@ pub const ASSET_NO_STORE_PRIVATE_CACHE_CONTROL: &str = NO_STORE_PRIVATE_CACHE_CO /// Cache policy metadata emitted by the asset proxy handler. /// -/// The Fastly router finalizes standard response headers after the asset handler +/// An adapter's router finalizes standard response headers after the asset handler /// returns. This typed policy lets the router reapply protected cache directives /// without depending on an exact header string set by the handler. #[derive(Clone, Copy, Debug, Eq, PartialEq)] @@ -132,8 +132,8 @@ pub enum AssetProxyCachePolicy { /// /// The adapter must call [`Self::apply_after_route_finalization`] after /// standard response and privacy finalization, passing its runtime - /// [`EdgeCacheHeader`]. Asset rehosting is Fastly-only today; a future - /// adapter must preserve this finalization step to emit its edge directive. + /// [`EdgeCacheHeader`]. An adapter that rehosts assets must preserve this + /// finalization step to emit its edge directive. Normalized(CachePolicy), } @@ -218,7 +218,7 @@ impl AssetProxyResponse { self.cache_policy } - /// Consume this wrapper and return a buffered Fastly response. + /// Consume this wrapper and return a buffered response. /// /// # Errors /// @@ -249,7 +249,7 @@ impl AssetProxyResponse { /// /// Returns [`TrustedServerError::Proxy`] when `platform_resp` carries a /// streaming body, which the buffered proxy path cannot materialize. -pub(crate) fn platform_response_to_fastly( +pub(crate) fn platform_response_to_buffered( platform_resp: PlatformResponse, ) -> Result, Report> { let (parts, body) = platform_resp.response.into_parts(); @@ -257,7 +257,7 @@ pub(crate) fn platform_response_to_fastly( EdgeBody::Once(bytes) => bytes.to_vec(), EdgeBody::Stream(_) => { return Err(Report::new(TrustedServerError::Proxy { - message: "streaming platform response body is not supported by Fastly response conversion" + message: "streaming platform response body is not supported by buffered response conversion" .to_string(), })); } @@ -265,7 +265,7 @@ pub(crate) fn platform_response_to_fastly( Ok(Response::from_parts(parts, EdgeBody::from(body_bytes))) } -fn platform_response_to_fastly_asset(platform_resp: PlatformResponse) -> AssetProxyResponse { +fn platform_response_to_asset(platform_resp: PlatformResponse) -> AssetProxyResponse { let (parts, body) = platform_resp.response.into_parts(); match body { @@ -282,8 +282,8 @@ fn platform_response_to_fastly_asset(platform_resp: PlatformResponse) -> AssetPr /// Stream a platform response body directly to a writable client stream. /// -/// Asset routes and Fastly `EdgeZero` publisher fallback both use this bridge -/// after headers have been committed through `stream_to_client()`. +/// Asset routes and an adapter's `EdgeZero` publisher fallback both use this +/// bridge after headers have been committed through `stream_to_client()`. /// /// # Errors /// @@ -313,8 +313,8 @@ pub async fn stream_asset_body( .change_context(TrustedServerError::Proxy { message: "failed to write streaming platform response body".to_string(), })?; - // Flush per chunk: Fastly's `StreamingBody` is a - // `BufWriter`, so a segment smaller than + // Flush per chunk: a platform's client stream may be a + // `BufWriter` over its stream handle, so a segment smaller than // the write buffer would sit in the Wasm heap until a later // write filled it. The publisher stream yields compressed // segments that are commonly smaller than that buffer, and the @@ -971,9 +971,9 @@ async fn send_asset_origin_request( })?; if stream_response { - Ok(platform_response_to_fastly_asset(platform_resp)) + Ok(platform_response_to_asset(platform_resp)) } else { - platform_response_to_fastly(platform_resp).map(AssetProxyResponse::origin_controlled) + platform_response_to_buffered(platform_resp).map(AssetProxyResponse::origin_controlled) } } @@ -1206,9 +1206,9 @@ pub async fn handle_asset_proxy_request( })?; let mut response = if stream_response { - platform_response_to_fastly_asset(platform_resp) + platform_response_to_asset(platform_resp) } else { - platform_response_to_fastly(platform_resp).map(AssetProxyResponse::origin_controlled)? + platform_response_to_buffered(platform_resp).map(AssetProxyResponse::origin_controlled)? }; strip_asset_proxy_response_headers(response.response_mut()); @@ -1793,10 +1793,10 @@ pub async fn handle_first_party_proxy_sign( // protocol-relative sign target inherits it. // // Prefer the URI's explicit scheme, which adapters that deliver absolute - // request targets (Fastly, Spin after normalization) always carry, and fall - // back to TLS/forwarded metadata otherwise. What must never be used is the - // parsed request target: origin-form URIs — what browsers send and the Axum - // adapter forwards verbatim — have no scheme of their own, so parsing would + // request targets always carry, and fall back to TLS/forwarded metadata + // otherwise. What must never be used is the parsed request target: + // origin-form URIs, which browsers send and an adapter may forward + // verbatim, have no scheme of their own, so parsing would // hand back the placeholder base's `https` and make an HTTP dev server sign // an HTTPS target it then proxies over TLS against a plaintext service. let request_scheme = req @@ -2013,7 +2013,7 @@ pub async fn handle_first_party_proxy_rebuild( } else { // Support GET: /first-party/proxy-rebuild?tsclick=...&add=...&del=... // Parse the query component directly rather than the full URI: browsers - // (and the Axum/Spin adapters) deliver origin-form URIs (`/path?query`), + // (and the adapters that forward them) deliver origin-form URIs (`/path?query`), // which `url::Url::parse` rejects as relative. rebuild_request_from_pairs(url::form_urlencoded::parse(req_query.as_bytes()))? }; @@ -2194,9 +2194,9 @@ const REQUEST_TARGET_BASE: &str = "https://request.invalid"; /// Parse a request target that may be either an absolute URL or origin-form. /// -/// Adapters differ in what `Request::uri()` carries: Fastly and the normalized -/// Spin path expose an absolute URL, while browsers send origin-form targets -/// (`/path?query`) that the Axum adapter passes through verbatim. Joining a +/// Adapters differ in what `Request::uri()` carries. Some expose an absolute +/// URL, while browsers send origin-form targets (`/path?query`) that other +/// adapters pass through verbatim. Joining a /// fixed placeholder origin lets one parser serve both, so first-party /// endpoints behave identically across adapters. /// @@ -2882,7 +2882,7 @@ mod tests { fn proxy_sign_inherits_the_request_scheme_for_protocol_relative_urls() { // A protocol-relative target adopts the scheme the visitor is on. It // must come from the request, not from the parser's placeholder base: - // origin-form URIs (browsers, and the Axum adapter verbatim) carry no + // origin-form URIs (browsers, and an adapter forwarding them) carry no // scheme, so an HTTP dev server would otherwise sign an HTTPS target // and proxy TLS against a plaintext service. futures::executor::block_on(async { @@ -3412,7 +3412,7 @@ mod tests { #[test] fn first_party_click_accepts_origin_form_uri() { - // Browsers send origin-form request targets and the Axum adapter passes + // Browsers send origin-form request targets and an adapter may pass // them through verbatim, so the shared signed-target parser must accept // both forms — otherwise the second hop of the rebuild redirect chain // fails instead of reaching the advertiser. @@ -3480,7 +3480,7 @@ mod tests { #[test] fn proxy_rebuild_get_with_origin_form_uri_redirects() { // The opaque-origin creative click guard navigates to this endpoint, - // and browsers (via the Axum/Spin adapters) deliver origin-form URIs + // and browsers (through an adapter that forwards them) deliver origin-form URIs // (`/path?query`) rather than absolute URLs. The handler must parse the // query and answer with the 302 recovery redirect. futures::executor::block_on(async { @@ -6908,7 +6908,7 @@ mod tests { #[test] fn stream_asset_body_flushes_each_chunk_before_polling_the_source_again() { - // Fastly's `StreamingBody` is a `BufWriter`, so a yielded segment only + // A platform's client stream may be a `BufWriter`, so a yielded segment only // reaches the client once the sink is flushed. The publisher stream // awaits the origin (and the auction) between segments, so a segment // smaller than the write buffer must be flushed before the next poll or diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index 4eb5760ce6..1c7496e22e 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -13,7 +13,7 @@ //! **Note on platform coupling:** The handler boundaries use portable HTTP //! types: [`handle_publisher_request`] and [`stream_publisher_body`] take and //! return `http::Request`/`http::Response` over `EdgeBody`, and platform I/O is -//! reached through `RuntimeServices` rather than `fastly::*` directly. The +//! reached through `RuntimeServices` rather than a platform SDK directly. The //! streaming processor itself is generic: `process_response_streaming` writes //! into any [`Write`] (a `Vec` for buffered routes, a streaming writer for //! the streaming route). It is not a content-rewriting concern. @@ -2348,7 +2348,7 @@ enum TemplateStoreOutcome { /// statuses (204, 304) carry no body but may advertise the `GET` representation's /// length, so they skip the buffer and length rewrite. /// -/// Buffered adapters (Axum, Cloudflare, Spin, and non-streaming fallbacks) call +/// Buffered adapters and non-streaming fallbacks call /// this: it drives /// [`stream_publisher_body_async`], which awaits /// [`AuctionOrchestrator::collect_dispatched_auction`], writes the winning bids @@ -2609,9 +2609,10 @@ pub async fn buffer_publisher_response_async( /// Splices this visitor's slots and bids into the seam, if the mode assembles. /// -/// Gives the platform assembler the already-buffered cold document. Fastly resolves a -/// synthetic ESI include with the repaired parser; adapters without an assembler and -/// documents the parser rejects use the same validated byte split as the warm path. +/// Gives the platform assembler the already-buffered cold document. An adapter with an +/// assembler resolves a synthetic ESI include with the repaired parser, while adapters +/// without one, and documents the parser rejects, use the same validated byte split as +/// the warm path. /// /// Called *after* [`store_template_if_authorized`], never before: what is stored must be /// the template every visitor shares. @@ -3049,8 +3050,8 @@ async fn store_template_if_authorized( /// bodies where possible. /// /// Buffered adapters should keep using [`buffer_publisher_response_async`]. -/// Fastly uses this helper before the entry point commits headers, allowing the -/// response body to be pulled lazily by `stream_to_client()`. +/// A streaming adapter uses this helper before the entry point commits headers, +/// allowing the response body to be pulled lazily by `stream_to_client()`. /// /// # Errors /// @@ -3100,8 +3101,8 @@ pub async fn publisher_response_into_streaming_response( match publisher_response { PublisherResponse::Buffered(mut response) => { - // Fastly requests the origin body as a stream before the response is - // classified, so a buffered-unmodified response can still hold an + // A streaming adapter requests the origin body as a stream before + // classification, so a buffered-unmodified response can still hold an // `EdgeBody::Stream`. A bodiless response (HEAD, 204, 205, 304) must // stay bodiless — `send_edgezero_response` streams any // `EdgeBody::Stream` to the client — so drop the body and correct @@ -3820,27 +3821,27 @@ fn should_preload_ec_snapshot(input: &EcSnapshotPreloadInput) -> bool { eligible && !marker_can_skip } +/// Removes every TLS header the platform declared from `req`, being an +/// internal scheme signal its entry point sets. +fn remove_platform_tls_headers(req: &mut Request) { + for name in crate::platform::platform_tls_headers() { + req.headers_mut().remove(name); + } +} + /// Rewrites a downstream request into an outbound publisher-origin request. /// /// Restricts advertised encodings to those the rewrite pipeline can handle, -/// strips the internal `fastly-ssl` scheme signal and the host signal headers -/// so they never leak to the backend, and retargets the URI and `Host` header -/// at the origin. +/// strips the platform's TLS headers and the host signal headers so that no +/// internal signal leaks to the backend, and retargets the URI and `Host` +/// header at the origin. fn rewrite_origin_request( req: &mut Request, target_uri: Uri, origin_host_header: &str, ) -> Result<(), Report> { restrict_accept_encoding(req); - // Layering wart: `fastly-ssl` is a vendor-specific header name. Core only - // knows it because the Fastly adapter re-injects it from trusted TLS - // metadata and `detect_scheme` (see `http_util`) still reads it as a - // fallback scheme signal. The neutral signal (`ClientInfo::tls_protocol`) - // is already the primary path, so this strip — and the whole `fastly-ssl` - // coupling in core — should move behind a platform-neutral header in a - // separate change. Until then we strip it here so the edge signal never - // reaches publisher backends. - req.headers_mut().remove("fastly-ssl"); + remove_platform_tls_headers(req); // The origin request carries the reader's other headers onward, so the // host's TLS and HTTP/2 client signals are removed by name. for name in HOST_SIGNAL_HEADERS { @@ -4666,10 +4667,10 @@ async fn stream_html_with_auction_hold( /// so the two async hold paths cannot drift apart. /// /// No production caller reaches this today: it is only entered through -/// [`buffer_publisher_response_async`], and the buffered adapters (Axum, -/// Cloudflare, Spin) never produce `Body::Stream` because the publisher fetch -/// is gated on `supports_streaming_responses()`. It is groundwork for those -/// adapters' streaming cutover; Fastly uses the lazy stream instead. +/// [`buffer_publisher_response_async`], and the buffered adapters never +/// produce `Body::Stream` because the publisher fetch is gated on +/// `supports_streaming_responses()`. It is groundwork for those adapters' +/// streaming cutover, and a streaming adapter uses the lazy stream instead. #[allow( clippy::too_many_arguments, reason = "stream state remains explicit across the shared adapter driver" @@ -4710,7 +4711,7 @@ async fn body_close_hold_loop_stream( .await? { // Write the ready prefix before collecting the auction, matching the - // lazy Fastly stream: only the held `` tail waits on collection. + // lazy stream: only the held `` tail waits on collection. for encoded in step.ready { write_encoded_segment(writer, &encoded)?; } @@ -4727,7 +4728,7 @@ async fn body_close_hold_loop_stream( } // Write the decoder-finalized prefix before collection, matching the lazy - // Fastly stream: only the held `` tail waits on the auction. + // stream: only the held `` tail waits on the auction. let final_step = hold_finish_ready_segments( processor, &mut decoder, @@ -5292,7 +5293,7 @@ fn apply_origin_cache_intent( request.with_cache_bypass() } else if readthrough_applies { // The reader-facing key and the all-scope key that `ts cache purge` names. - // Fastly accepts multiple space-separated surrogate keys without changing TTL. + // Several space-separated surrogate keys tag one object without changing its TTL. request.with_shared_cache(format!( "{TEMPLATE_CACHE_PURGE_ALL_SURROGATE_KEY} {}", reader_url_surrogate_key(reader_url), @@ -5701,12 +5702,9 @@ pub async fn handle_publisher_request( if matches!(assembly_mode, AssemblyMode::Esi) && !reader_supports_assembly { log::debug!("template_cache bypass: reader accepts no representation TS can assemble"); } - // Strip the internal `fastly-ssl` scheme signal before forwarding to the - // origin. On the EdgeZero path the entry point re-injects this header from - // trusted Fastly TLS metadata so in-process scheme detection works; the - // legacy path never sets it. Either way it is an internal edge signal that - // must not leak to publisher backends. - req.headers_mut().remove("fastly-ssl"); + // The platform's TLS headers are an internal scheme signal its entry point + // sets, which must not leak to publisher backends. + remove_platform_tls_headers(&mut req); // The template cache key is built here, before the request is consumed, because every field // is request-derived and this is the last point where the request is in hand. // @@ -8217,9 +8215,9 @@ fn normalize_page_bids_path(raw: &str) -> String { /// Called by the client-side SPA navigation hook after `pushState` / `popstate`. /// /// `kv` enriches the bid request with server-side EIDs from the EC identity -/// graph. Only the Fastly adapter has a KV identity store, so Axum, Cloudflare, -/// and Spin pass `None`; client cookie EIDs are still resolved and consent-gated -/// on every adapter, so no adapter forwards unconsented EIDs. +/// graph. An adapter with no KV identity store passes `None`, and client cookie +/// EIDs are still resolved and consent-gated on every adapter, so no adapter +/// forwards unconsented EIDs. /// /// # Errors /// @@ -9253,7 +9251,7 @@ mod tests { .method(Method::GET) .uri("https://publisher.example/article?x=1") .header(header::HOST, "publisher.example") - .header("fastly-ssl", "1") + .header("example-platform-tls", "1") .header(header::USER_AGENT, "test-browser") .body(EdgeBody::from("ignored-body")) .expect("should build request"); @@ -9397,12 +9395,16 @@ mod tests { const SCHEDULING_PROVIDER: &str = "scheduling_capture"; + /// A TLS header the scheduling tests declare for the platform, which the + /// auction sees on the client's request and the origin never does. + const SCHEDULING_TLS_HEADER: &str = "example-scheduling-tls"; + #[derive(Debug)] struct CapturedSchedulingAuction { request: AuctionRequest, client_uri: String, client_host: Option, - client_fastly_ssl: Option, + client_platform_tls: Option, client_user_agent: Option, http_calls_at_dispatch: usize, lookups_at_dispatch: usize, @@ -9438,7 +9440,7 @@ mod tests { request: request.clone(), client_uri: context.request.uri().to_string(), client_host: header_value(header::HOST.as_str()), - client_fastly_ssl: header_value("fastly-ssl"), + client_platform_tls: header_value(SCHEDULING_TLS_HEADER), client_user_agent: header_value(header::USER_AGENT.as_str()), http_calls_at_dispatch: self.http.recorded_backend_names().len(), lookups_at_dispatch: self.lookups.load(Ordering::SeqCst), @@ -9521,12 +9523,13 @@ mod tests { } fn navigation_request() -> Request { + crate::platform::declare_platform_tls_headers(&[SCHEDULING_TLS_HEADER]); let mut request = HttpRequest::builder() .method(Method::GET) .uri("https://publisher.example/article") .header(header::HOST, "publisher.example") .header("sec-fetch-dest", "document") - .header("fastly-ssl", "1") + .header(SCHEDULING_TLS_HEADER, "1") .header(header::IF_NONE_MATCH, "\"cached-page\"") .header(header::RANGE, "bytes=0-18") .header(header::USER_AGENT, "scheduling-browser") @@ -9679,7 +9682,7 @@ mod tests { .map(|(_, value)| value.as_str()) }; header_value(header::HOST.as_str()) == Some("origin.test-publisher.com") - && header_value("fastly-ssl").is_none() + && header_value(SCHEDULING_TLS_HEADER).is_none() && header_value(header::IF_NONE_MATCH.as_str()).is_none() && header_value(header::RANGE.as_str()).is_none() }); @@ -9691,7 +9694,7 @@ mod tests { let auction_preserved_client_snapshot = captured_auction.as_ref().is_some_and(|captured| { captured.client_uri == "https://publisher.example/article" && captured.client_host.as_deref() == Some("publisher.example") - && captured.client_fastly_ssl.as_deref() == Some("1") + && captured.client_platform_tls.as_deref() == Some("1") && captured.client_user_agent.as_deref() == Some("scheduling-browser") && captured .request @@ -12969,8 +12972,8 @@ mod tests { /// test that exercised one of them proves nothing about the other. #[derive(Clone, Copy)] enum Finalizer { - /// Fastly's path: headers commit first and the seam is spliced inside the - /// body stream. + /// A streaming adapter's path: headers commit first and the seam is + /// spliced inside the body stream. Streaming, /// Every buffered adapter's path: the seam is spliced eagerly. Buffered, @@ -13996,12 +13999,12 @@ mod tests { // trip. // // This asserts the *intent* recorded on the outbound request, not a cache - // hit, because a hit is not observable here. Measured under Viceroy 0.17 with - // the gate enabled, the request judged shareable, and an origin responding - // `Cache-Control: public, max-age=60` with no `Set-Cookie`: two identical - // navigations still produced two origin fetches. Viceroy does not implement - // the readthrough cache, so the saving this gate exists for cannot be - // demonstrated locally in any form — only the decision that enables it. + // hit, because a hit is not observable here. Measured under the local + // simulator with the gate enabled, the request judged shareable, and an origin + // responding `Cache-Control: public, max-age=60` with no `Set-Cookie`: two + // identical navigations still produced two origin fetches. The simulator does + // not implement the readthrough cache, so the saving this gate exists for + // cannot be demonstrated locally in any form, only the decision that enables it. let stub = Arc::new(StubHttpClient::new()); let services = services_with_cache_and_telemetry( Arc::clone(&stub), @@ -14763,15 +14766,16 @@ mod tests { } #[tokio::test] - async fn fastly_surrogate_control_still_allows_a_cold_fill_and_warm_hit() { + async fn surrogate_control_still_allows_a_cold_fill_and_warm_hit() { let stub = Arc::new(StubHttpClient::new()); let cache = Arc::new(MemoryTemplateCache::default()); let settings = Arc::new(settings_with_mode_and_template_cache_max_age("esi", 1_200)); let services = services(Arc::clone(&stub), Arc::clone(&cache)); - // This is the policy observed on the publisher that exposed the bug. Fastly - // gives the surrogate lifetime edge precedence; the configured safety - // ceiling allows the origin's twenty-minute edge freshness. + // This is the policy observed on the publisher that exposed the bug. + // `Surrogate-Control` gives the edge lifetime precedence, and the + // configured safety ceiling allows the origin's twenty-minute edge + // freshness. stub.push_response_with_headers( 200, b"origin".to_vec(), @@ -18533,7 +18537,7 @@ mod tests { } #[test] - fn observed_fastly_surrogate_policy_uses_edge_freshness_capped_by_configuration() { + fn observed_surrogate_policy_uses_edge_freshness_capped_by_configuration() { let publisher_headers = headers(&[ (header::CACHE_CONTROL, "public, max-age=60"), ( @@ -18553,13 +18557,13 @@ mod tests { &TemplateCachePolicy::for_test(¬hing_covered(), Duration::from_secs(300),), ), Ok(Duration::from_secs(300)), - "Fastly's edge freshness should take precedence over the shorter browser \ + "the edge freshness should take precedence over the shorter browser \ lifetime, while the configured safety ceiling remains authoritative" ); } #[test] - fn fastly_surrogate_freshness_takes_precedence_over_standard_freshness() { + fn surrogate_freshness_takes_precedence_over_standard_freshness() { for (cache_control, surrogate_control, expected) in [ ("public, max-age=300", "max-age=30", 30), ("public, max-age=30", "max-age=300", 300), @@ -18720,7 +18724,7 @@ mod tests { } #[test] - fn surrogate_control_can_authorize_fastly_edge_freshness_without_browser_freshness() { + fn surrogate_control_can_authorize_edge_freshness_without_browser_freshness() { let publisher_headers = headers(&[( header::HeaderName::from_static("surrogate-control"), "max-age=1200", @@ -18737,7 +18741,7 @@ mod tests { &TemplateCachePolicy::for_test(¬hing_covered(), Duration::from_secs(300),), ), Ok(Duration::from_secs(300)), - "Fastly edge freshness should not require browser freshness" + "Surrogate-Control edge freshness should not require browser freshness" ); } @@ -19496,7 +19500,7 @@ mod tests { .method(Method::GET) .uri("https://publisher.example/redirect") .header(header::HOST, "publisher.example") - .header("fastly-ssl", "1") + .header("x-forwarded-proto", "https") .body(EdgeBody::empty()) .expect("should build request") } @@ -23760,7 +23764,7 @@ mod tests { /// The answers to `GET /a.txt` through the publisher path, the origin /// answering `status` with `plain\n` as UTF-8 text and the headers given, - /// finalized as the adapters that buffer do and then as Fastly does. + /// finalized as the adapters that buffer do and then as a streaming adapter does. /// With `marked`, a module leaves state on the request. async fn text_answers_through_the_publisher( settings: Settings, @@ -24717,7 +24721,7 @@ mod tests { } #[tokio::test] - async fn a_made_file_streams_through_the_finalizer_fastly_uses() { + async fn a_made_file_streams_through_the_streaming_finalizer() { use crate::integrations::registry_test_support::middleware_fixture as fixture; let proxied = proxy( @@ -28383,10 +28387,11 @@ mod tests { #[test] fn publisher_response_streaming_finalize_normalizes_bodiless_framing() { - // Fastly requests the origin body as a stream before classification, so - // a buffered-unmodified response can hold an `EdgeBody::Stream`. The - // adapter streams any `EdgeBody::Stream` to the client, so bodiless - // responses must carry no body and correct framing per status. + // A streaming adapter requests the origin body as a stream before + // classification, so a buffered-unmodified response can hold an + // `EdgeBody::Stream`. The adapter streams any `EdgeBody::Stream` to the + // client, so bodiless responses must carry no body and correct framing + // per status. let settings = Arc::new(create_test_settings()); let registry = Arc::new( IntegrationRegistry::with_plan( @@ -28454,8 +28459,8 @@ mod tests { #[test] fn buffer_publisher_response_normalizes_bodiless_framing() { - // The buffered finalizer (Axum/Cloudflare/Spin) must correct bodiless - // framing identically to the streaming finalizer. + // The buffered finalizer must correct bodiless framing identically to + // the streaming finalizer. let settings = create_test_settings(); let registry = IntegrationRegistry::with_plan( &settings, @@ -28625,7 +28630,7 @@ mod tests { ); } - // Streaming finalizer (Fastly). + // Streaming finalizer. let streaming_sink = Arc::new(RecordingTelemetrySink::default()); let response = futures::executor::block_on(publisher_response_into_streaming_response( make_stream_response(), @@ -28639,7 +28644,7 @@ mod tests { assert_eq!(response.status(), StatusCode::NOT_MODIFIED); assert_bodiless_abandoned(&streaming_sink); - // Buffered finalizer (Axum/Cloudflare/Spin). + // Buffered finalizer. let buffered_sink = Arc::new(RecordingTelemetrySink::default()); let buffered_services = noop_services_with_telemetry_sink(Arc::clone(&buffered_sink) as _); let response = futures::executor::block_on(buffer_publisher_response_async( @@ -28950,7 +28955,7 @@ mod tests { } /// Pass-through dispatch contract: the adapter treats `PublisherResponse::PassThrough` - /// by reattaching the origin body unchanged and letting Fastly emit it. + /// by reattaching the origin body unchanged and letting the platform emit it. /// Simulate that step and assert byte identity plus Content-Length /// preservation. Distinct from `pass_through_preserves_body_and_content_length` /// which only tests the header preservation; this one walks the full diff --git a/crates/trusted-server-core/src/request_signing/endpoints.rs b/crates/trusted-server-core/src/request_signing/endpoints.rs index 6308d6d049..5f0f4b4ef1 100644 --- a/crates/trusted-server-core/src/request_signing/endpoints.rs +++ b/crates/trusted-server-core/src/request_signing/endpoints.rs @@ -189,13 +189,13 @@ fn validate_kid_format(kid: &str) -> Result<(), Report> { /// common denominator. Concretely the kid must start with a lowercase ASCII /// letter, on top of the structural [`validate_kid_format`] charset checks. /// -/// That floor is currently set by the Fermyon Spin variable encoder -/// (`spin_variable_name` in the Spin adapter's `platform.rs`), which requires a -/// lowercase ASCII leading character: it would otherwise alias digit-leading keys -/// (`1foo` and `n1foo` both map to `v_n1foo`) or reject uppercase/punctuation -/// leading keys (`KidA`, `_kid`, `-kid`, `.kid`, `:kid`) at storage time. -/// Checking the floor here refuses such a kid for every platform rather than -/// leaving it to fail at storage time on Spin. [`kid_is_creatable`] exposes +/// That floor is set by the strictest variable-name encoder among the +/// adapters, which requires a lowercase ASCII leading character: it would +/// otherwise alias digit-leading keys (`1foo` and `n1foo` both map to +/// `v_n1foo`) or reject uppercase/punctuation leading keys (`KidA`, `_kid`, +/// `-kid`, `.kid`, `:kid`) at storage time. Checking the floor here refuses +/// such a kid for every platform rather than leaving it to fail at storage +/// time on one. [`kid_is_creatable`] exposes /// this predicate so adapter crates can pin the contract with a test, because /// core must never accept a kid its encoder rejects. System-generated KIDs /// (`ts-YYYY-MM-DD`) start with `t` and are unaffected. @@ -534,9 +534,10 @@ mod tests { #[test] fn validate_kid_rejects_non_lowercase_leading_ids() { - // The Spin variable encoder requires a lowercase ASCII leading character, - // so uppercase- and punctuation-leading KIDs that pass validate_kid_format - // must be rejected for a new key, which Spin could not store. + // The strictest adapter's variable encoder requires a lowercase ASCII + // leading character, so uppercase- and punctuation-leading KIDs that + // pass validate_kid_format must be rejected for a new key, which that + // adapter could not store. for kid in &["KidA", "_kid", "-kid", ".kid", ":kid"] { assert!( validate_kid(kid).is_err(), diff --git a/crates/trusted-server-core/src/request_signing/mod.rs b/crates/trusted-server-core/src/request_signing/mod.rs index 8e0d20c1f7..7cde68094f 100644 --- a/crates/trusted-server-core/src/request_signing/mod.rs +++ b/crates/trusted-server-core/src/request_signing/mod.rs @@ -10,8 +10,8 @@ //! - **Store name** ([`JWKS_CONFIG_STORE_NAME`], [`SIGNING_SECRET_STORE_NAME`]): //! used for runtime reads via [`crate::platform::PlatformConfigStore::get`] //! and [`crate::platform::PlatformSecretStore::get_bytes`] through -//! [`crate::platform::RuntimeServices`]. These names are configured in -//! `fastly.toml` for the Fastly adapter. +//! [`crate::platform::RuntimeServices`]. Each adapter's platform +//! configuration declares stores under these names. //! //! - **Store ID**: used for write operations via //! [`crate::platform::PlatformConfigStore::put`] / @@ -35,14 +35,14 @@ pub mod signing; /// Config store name for JWKS public keys used by runtime read operations. /// -/// This must match the store name declared in `fastly.toml` under -/// `[local_server.config_stores]`. +/// This must match the config store name the adapter's platform +/// configuration declares. pub const JWKS_CONFIG_STORE_NAME: &str = "jwks_store"; /// Secret store name for Ed25519 signing keys used by runtime read operations. /// -/// This must match the store name declared in `fastly.toml` under -/// `[local_server.secret_stores]`. +/// This must match the secret store name the adapter's platform +/// configuration declares. pub const SIGNING_SECRET_STORE_NAME: &str = "signing_keys"; /// Lazily constructed [`StoreName`] for JWKS config-store reads. diff --git a/crates/trusted-server-core/src/settings.rs b/crates/trusted-server-core/src/settings.rs index e1958b7a5e..cf22a6c95f 100644 --- a/crates/trusted-server-core/src/settings.rs +++ b/crates/trusted-server-core/src/settings.rs @@ -19,7 +19,7 @@ use validator::{Validate, ValidationError, ValidationErrors, ValidationErrorsKin use crate::auction_config_types::AuctionConfig; use crate::cache_policy::{CachePolicy, CacheVisibility}; use crate::consent_config::ConsentConfig; -use crate::constants::INTERNAL_HEADERS; +use crate::constants::is_internal_header; use crate::creative_opportunities::CreativeOpportunitiesConfig; use crate::ec::module::{ EcModuleSelection, HMAC_MODULE_KEY, HOST_SIGNALS_MODULE_KEY, check_named_module_configuration, @@ -77,7 +77,7 @@ pub struct Publisher { /// exceeding it fails the response rather than allocating past the cap. /// Defaults to 16 MiB — a conservative cap that prevents Wasm-heap OOM. /// - /// Fastly origin bodies are preserved as streams on the publisher path, so + /// Streaming adapters keep origin bodies as streams on the publisher path, so /// this setting also caps the streaming pipeline twice over: cumulative /// raw (still compressed) bytes pulled from origin, and cumulative decoded /// bytes emitted by the decompressor — the latter so a decompression bomb @@ -825,7 +825,7 @@ pub struct Ec { #[serde(default)] pub resolve_allowed_origins: Vec, - /// Fastly KV store name for the EC identity graph. + /// KV store name for the EC identity graph. #[serde(default, serialize_with = "crate::redacted::sensitive")] pub ec_store: Option, @@ -1509,10 +1509,10 @@ pub struct DeviceConfig { /// /// Defaults to the built-in `builtin` module when absent, which classifies /// from the User-Agent alone, so device classification itself makes no - /// host-specific call. The opt-in `fastly` module strengthens the - /// browser/bot gate with the host's TLS and HTTP/2 signals, which the Fastly - /// entry point reads on every request regardless of this selector. Override - /// it with the + /// host-specific call. The opt-in `platform` module, which an adapter + /// supplies where its platform reports the signals, strengthens the + /// browser/bot gate with the host's TLS and HTTP/2 signals. Override it + /// with the /// `TRUSTED_SERVER__device__module` environment variable so the same /// compiled WebAssembly can switch modules at deployment. An unknown key is /// rejected at startup by @@ -1531,7 +1531,7 @@ impl DeviceConfig { /// Validates the selected device-detection module. /// - /// `builtin` and `fastly` are resolved by the core and the Fastly adapter. + /// `builtin` and `platform` are resolved by core and by the adapter. /// Any other key names an integration module that declares a device /// module, and the registry rejects a key no module supplies, so this /// cannot be a closed list without shutting modules out of device detection @@ -1916,9 +1916,9 @@ pub struct AssetImageOptimizerConfig { pub profile_set: String, /// Query-string handling policy for the origin request. /// - /// `preserve` is rejected while Image Optimizer is enabled because Fastly `IO` - /// can interpret arbitrary request query parameters as transformation - /// inputs outside the configured profile table. + /// `preserve` is rejected while Image Optimizer is enabled because the + /// platform's image optimizer can interpret arbitrary request query + /// parameters as transformation inputs outside the configured profile table. #[serde(default)] pub origin_query: Option, } @@ -2204,7 +2204,7 @@ impl ImageOptimizerAspectRatioConfig { #[derive(Debug, Clone, Copy, Default, Deserialize, Eq, PartialEq, Serialize)] #[serde(rename_all = "snake_case")] pub enum MissingCropOffsetMode { - /// Append Fastly `IO` `smart` crop mode. + /// Append the image optimizer's `smart` crop mode. #[default] Smart, /// Leave the crop as-is. @@ -3506,8 +3506,9 @@ pub struct DebugConfig { /// Expose the JA4/TLS probabilistic identifier debug endpoint at `GET /_ts/debug/ja4`. /// /// When `false` (the default), the endpoint returns 404. Enable only for - /// intentional Fastly/browser TLS investigation. The endpoint reflects - /// Fastly-observed TLS details that browser JS cannot normally read. + /// an intentional comparison of the TLS details the edge and the browser + /// see. The endpoint reflects TLS details observed at the edge that + /// browser JS cannot normally read. #[serde(default)] pub ja4_endpoint_enabled: bool, @@ -3984,7 +3985,7 @@ impl TrustedClientIpConfig { /// use trusted_server_core::settings::TrustedClientIpConfig; /// /// let config = TrustedClientIpConfig { - /// ip_header: "fastly-client-ip".to_owned(), + /// ip_header: "x-ts-client-ip".to_owned(), /// auth_header: "x-trusted-client-auth".to_owned(), /// shared_secret: Redacted::new("fictional-shared-secret-0123456789".to_owned()), /// }; @@ -4012,12 +4013,14 @@ fn validate_trusted_client_ip(config: &TrustedClientIpConfig) -> Result<(), Vali } for header in [&ip_header, &auth_header] { - if INTERNAL_HEADERS.contains(&header.as_str()) { + if is_internal_header(header.as_str()) { return Err(ValidationError::new("reserved_trusted_client_ip_header")); } } - if ip_header.as_str() != "fastly-client-ip" && !ip_header.as_str().starts_with("x-") { + // A dedicated `x-` header, or a platform's own client address header, + // which by convention ends `-client-ip`. + if !ip_header.as_str().starts_with("x-") && !ip_header.as_str().ends_with("-client-ip") { return Err(ValidationError::new("unsafe_trusted_client_ip_header")); } if !auth_header.as_str().starts_with("x-") { @@ -5588,7 +5591,7 @@ module = \"none\"", #[test] fn trusted_client_ip_parses_and_redacts_shared_secret_in_debug_output() { let settings = Settings::from_toml(&trusted_client_ip_toml( - "fastly-client-ip", + "example-client-ip", "x-trusted-client-auth", "fictional-shared-secret-0123456789", )) @@ -5597,7 +5600,7 @@ module = \"none\"", .trusted_client_ip .expect("should retain trusted client IP configuration"); - assert_eq!(config.ip_header, "fastly-client-ip"); + assert_eq!(config.ip_header, "example-client-ip"); assert_eq!(config.auth_header, "x-trusted-client-auth"); let debug = format!("{config:?}"); assert!( @@ -5657,7 +5660,7 @@ module = \"none\"", }]; settings.trusted_client_ip = Some(TrustedClientIpConfig { - ip_header: "fastly-client-ip".to_string(), + ip_header: "example-client-ip".to_string(), auth_header: "x-trusted-client-auth".to_string(), shared_secret: Redacted::new(CANARY_TRUSTED_CLIENT_IP_SHARED_SECRET.to_string()), }); @@ -5750,7 +5753,7 @@ module = \"none\"", #[test] fn trusted_client_ip_authentication_requires_an_exact_match() { let settings = Settings::from_toml(&trusted_client_ip_toml( - "fastly-client-ip", + "example-client-ip", "x-trusted-client-auth", "fictional-shared-secret-0123456789", )) @@ -5806,7 +5809,12 @@ module = \"none\"", "unsafe_trusted_client_ip_header", ), ( - "fastly-client-ip", + "client-ip", + "x-trusted-client-auth", + "unsafe_trusted_client_ip_header", + ), + ( + "example-client-ip", "authorization", "unsafe_trusted_client_ip_auth_header", ), @@ -5830,16 +5838,33 @@ module = \"none\"", } } + #[test] + fn trusted_client_ip_rejects_an_internal_header_the_platform_declared() { + crate::platform::declare_platform_internal_headers(&["x-example-platform-debug"]); + + let error = Settings::from_toml(&trusted_client_ip_toml( + "example-client-ip", + "x-example-platform-debug", + "fictional-shared-secret-0123456789", + )) + .expect_err("should reject an internal header the platform declared"); + + assert!( + format!("{error:?}").contains("reserved_trusted_client_ip_header"), + "should identify the platform's internal header as reserved" + ); + } + #[test] fn trusted_client_ip_rejects_reserved_internal_headers() { for (ip_header, auth_header) in [ ("x-ts-tls-protocol", "x-trusted-client-auth"), ("x-ts-tls-cipher", "x-trusted-client-auth"), - ("fastly-client-ip", "x-ts-tls-protocol"), - ("fastly-client-ip", "x-ts-tls-cipher"), + ("example-client-ip", "x-ts-tls-protocol"), + ("example-client-ip", "x-ts-tls-cipher"), ("x-forwarded-for", "x-trusted-client-auth"), ("x-geo-info-available", "x-trusted-client-auth"), - ("fastly-client-ip", "x-ts-ec"), + ("example-client-ip", "x-ts-ec"), ] { let error = Settings::from_toml(&trusted_client_ip_toml( ip_header, @@ -5858,7 +5883,7 @@ module = \"none\"", #[test] fn trusted_client_ip_rejects_empty_secret_malformed_names_and_incomplete_sections() { let empty_secret = Settings::from_toml(&trusted_client_ip_toml( - "fastly-client-ip", + "example-client-ip", "x-trusted-client-auth", "", )); @@ -5874,7 +5899,7 @@ module = \"none\"", "invalid_trusted_client_ip_header", ), ( - "fastly-client-ip", + "example-client-ip", "invalid header", "invalid_trusted_client_ip_auth_header", ), @@ -5893,9 +5918,9 @@ module = \"none\"", for section in [ "[trusted_client_ip]\nauth_header = \"x-trusted-client-auth\"\nshared_secret = \"fictional-shared-secret-0123456789\"", - "[trusted_client_ip]\nip_header = \"fastly-client-ip\"\nshared_secret = \"fictional-shared-secret-0123456789\"", - "[trusted_client_ip]\nip_header = \"fastly-client-ip\"\nauth_header = \"x-trusted-client-auth\"", - "[trusted_client_ip]\nip_header = \"fastly-client-ip\"\nauth_header = \"x-trusted-client-auth\"\nshared_secret = \"fictional-shared-secret-0123456789\"\nunknown_field = true", + "[trusted_client_ip]\nip_header = \"example-client-ip\"\nshared_secret = \"fictional-shared-secret-0123456789\"", + "[trusted_client_ip]\nip_header = \"example-client-ip\"\nauth_header = \"x-trusted-client-auth\"", + "[trusted_client_ip]\nip_header = \"example-client-ip\"\nauth_header = \"x-trusted-client-auth\"\nshared_secret = \"fictional-shared-secret-0123456789\"\nunknown_field = true", ] { let result = Settings::from_toml(&format!("{}\n{section}\n", crate_test_settings_str())); @@ -5914,7 +5939,7 @@ module = \"none\"", ) .expect("should serialize base settings for JSON validation"); settings["trusted_client_ip"] = json!({ - "ip_header": "fastly-client-ip", + "ip_header": "example-client-ip", "auth_header": "x-trusted\u{0000}client-auth", "shared_secret": "fictional-control-byte-secret-0123", }); @@ -5937,7 +5962,7 @@ module = \"none\"", fn trusted_client_ip_rejects_a_31_byte_shared_secret_without_exposing_it() { let shared_secret = "1234567890123456789012345678901"; let error = Settings::from_toml(&trusted_client_ip_toml( - "fastly-client-ip", + "example-client-ip", "x-trusted-client-auth", shared_secret, )) @@ -5958,7 +5983,7 @@ module = \"none\"", fn trusted_client_ip_accepts_an_exactly_32_byte_ascii_graphic_shared_secret() { let shared_secret = "0123456789abcdef0123456789ABCDEF"; let settings = Settings::from_toml(&trusted_client_ip_toml( - "fastly-client-ip", + "example-client-ip", "x-trusted-client-auth", shared_secret, )) @@ -5978,7 +6003,7 @@ module = \"none\"", fn trusted_client_ip_rejects_a_non_ascii_shared_secret_without_exposing_it() { let shared_secret = "ascii-graphic-secret-0123456789é"; let error = Settings::from_toml(&trusted_client_ip_toml( - "fastly-client-ip", + "example-client-ip", "x-trusted-client-auth", shared_secret, )) @@ -5999,7 +6024,7 @@ module = \"none\"", fn trusted_client_ip_rejects_a_shared_secret_with_an_embedded_space_without_exposing_it() { let shared_secret = "valid-shared-secret-with space-012345"; let error = Settings::from_toml(&trusted_client_ip_toml( - "fastly-client-ip", + "example-client-ip", "x-trusted-client-auth", shared_secret, )) @@ -6025,7 +6050,7 @@ module = \"none\"", ) .expect("should serialize base settings for JSON validation"); settings["trusted_client_ip"] = json!({ - "ip_header": "fastly-client-ip", + "ip_header": "example-client-ip", "auth_header": "x-trusted-client-auth", "shared_secret": shared_secret, }); @@ -6053,7 +6078,7 @@ module = \"none\"", ) .expect("should serialize base settings for JSON validation"); settings["trusted_client_ip"] = json!({ - "ip_header": "fastly-client-ip", + "ip_header": "example-client-ip", "auth_header": "x-trusted-client-auth", "shared_secret": shared_secret, }); @@ -6081,7 +6106,7 @@ module = \"none\"", ) .expect("should serialize base settings for JSON validation"); settings["trusted_client_ip"] = json!({ - "ip_header": "fastly-client-ip", + "ip_header": "example-client-ip", "auth_header": "x-trusted-client-auth", "shared_secret": shared_secret, }); @@ -6113,7 +6138,7 @@ module = \"none\"", ); let settings = Settings::from_toml(&trusted_client_ip_toml( - "fastly-client-ip", + "example-client-ip", "x-trusted-client-auth", placeholder, )) @@ -7497,12 +7522,12 @@ module = \"none\"", .validate_module_selection() .expect("should validate the built-in default"); - let fastly = DeviceConfig { - module: Some("fastly".to_owned()), + let platform = DeviceConfig { + module: Some("platform".to_owned()), }; - fastly + platform .validate_module_selection() - .expect("should validate the fastly opt-in"); + .expect("should validate the platform opt-in"); // As with geo, a key core does not know is not a settings error, // because a module's name is a legitimate value here too. diff --git a/crates/trusted-server-core/src/settings_data.rs b/crates/trusted-server-core/src/settings_data.rs index f915cfe1ff..e72ba198d9 100644 --- a/crates/trusted-server-core/src/settings_data.rs +++ b/crates/trusted-server-core/src/settings_data.rs @@ -1,7 +1,5 @@ use edgezero_core::env_config::EnvConfig; use error_stack::{Report, ResultExt}; -use serde::Deserialize; -use sha2::{Digest as _, Sha256}; pub use crate::config_payload::DEFAULT_CONFIG_STORE_ID; use crate::config_payload::{DEFAULT_SECRET_STORE_ID, settings_from_config_blob_with}; @@ -10,25 +8,6 @@ use crate::integrations::IntegrationBuilder; use crate::platform::{PlatformConfigStore, PlatformSecretStore, PolicyOrigin, StoreName}; use crate::settings::Settings; -const FASTLY_CHUNK_POINTER_KIND: &str = "fastly_config_chunks"; -const FASTLY_CONFIG_ENTRY_LIMIT: usize = 8_000; - -#[derive(Debug, Deserialize)] -struct FastlyChunkPointer { - chunks: Vec, - edgezero_kind: String, - envelope_len: usize, - envelope_sha256: String, - version: u8, -} - -#[derive(Debug, Deserialize)] -struct FastlyChunkRef { - key: String, - len: usize, - sha256: String, -} - /// Resolves the `EdgeZero` app-config store name from runtime configuration. #[must_use] pub fn config_store_name(env: &EnvConfig) -> StoreName { @@ -43,10 +22,10 @@ pub fn config_key(env: &EnvConfig) -> String { /// Resolves the `EdgeZero` app-config store name from the process environment. /// -/// Native adapters such as Axum use this wrapper. Fastly reads no selector at -/// runtime: it opens the store under the logical ID that a managed deployment -/// links to the selected physical store. Without an override, this wrapper -/// also returns the manifest default logical store ID. +/// Native adapters use this wrapper. An adapter whose platform +/// links each store to the deployment under its logical ID opens the store +/// under that ID and reads no store name at runtime. Without an override, +/// this wrapper also returns the manifest default logical store ID. #[must_use] pub fn default_config_store_name() -> StoreName { config_store_name(&EnvConfig::from_env()) @@ -54,8 +33,10 @@ pub fn default_config_store_name() -> StoreName { /// Resolves the app-config blob key from the process environment. /// -/// Native adapters such as Axum use this wrapper and honour a `__KEY` -/// override. +/// Native adapters use this wrapper and honour a `__KEY` +/// override, which must match `ts config push --key`, because an ordinary +/// push without that flag writes at the logical store ID. An adapter that +/// chooses its key for each request resolves the key itself. #[must_use] pub fn default_config_key() -> String { config_key(&EnvConfig::from_env()) @@ -105,6 +86,10 @@ pub fn get_settings_from_config_store( /// against the built-in integrations followed by `extra_integrations`, with /// the permission policy they name read from `policy_origin`. /// +/// The entry is read as the platform's config store returns it, whole. A +/// platform that stores a long document in parts reassembles it in its own +/// config store read. +/// /// # Errors /// /// Returns [`TrustedServerError::Configuration`] when the config blob is @@ -120,8 +105,7 @@ pub fn get_settings_from_config_store_with( policy_origin: PolicyOrigin<'_>, extra_integrations: &[IntegrationBuilder], ) -> Result> { - let raw_value = read_config_entry(config_store, store_name, key)?; - let envelope_json = resolve_fastly_chunk_pointer(config_store, store_name, &raw_value)?; + let envelope_json = read_config_entry(config_store, store_name, key)?; settings_from_config_blob_with( &envelope_json, secret_store, @@ -144,117 +128,6 @@ fn read_config_entry( .change_context(TrustedServerError::Configuration { message }) } -/// The value a Fastly config store entry holds, put back together from its -/// chunks when the entry is a pointer to them, as a long value is written, -/// and the value itself otherwise. -/// -/// # Errors -/// -/// [`TrustedServerError::Configuration`] when a pointer is malformed, or a -/// chunk cannot be read or has the wrong length or SHA-256. -pub fn resolve_fastly_chunk_pointer( - config_store: &dyn PlatformConfigStore, - store_name: &StoreName, - value: &str, -) -> Result> { - let Ok(pointer) = serde_json::from_str::(value) else { - return Ok(value.to_string()); - }; - if pointer.edgezero_kind != FASTLY_CHUNK_POINTER_KIND { - return Ok(value.to_string()); - } - if pointer.version != 1 { - return configuration_error(format!( - "unsupported Fastly config chunk pointer version {}; expected 1", - pointer.version - )); - } - if value.len() > FASTLY_CONFIG_ENTRY_LIMIT { - return configuration_error(format!( - "Fastly config chunk pointer is {} bytes, exceeding the {} byte entry limit", - value.len(), - FASTLY_CONFIG_ENTRY_LIMIT - )); - } - - let mut declared_envelope_len = 0usize; - for chunk in &pointer.chunks { - if chunk.len > FASTLY_CONFIG_ENTRY_LIMIT { - return configuration_error(format!( - "Fastly config chunk `{}` declares {} bytes, exceeding the {} byte entry limit", - chunk.key, chunk.len, FASTLY_CONFIG_ENTRY_LIMIT - )); - } - declared_envelope_len = match declared_envelope_len.checked_add(chunk.len) { - Some(total) => total, - None => { - return configuration_error( - "Fastly config chunk lengths overflowed usize".to_string(), - ); - } - }; - } - if declared_envelope_len != pointer.envelope_len { - return configuration_error(format!( - "Fastly config chunk lengths total mismatch: expected envelope length {}, got {}", - pointer.envelope_len, declared_envelope_len - )); - } - - let mut envelope_json = String::with_capacity(pointer.envelope_len); - let mut actual_envelope_len = 0usize; - for chunk in pointer.chunks { - let chunk_value = read_config_entry(config_store, store_name, &chunk.key)?; - let chunk_len = chunk_value.len(); - if chunk_len != chunk.len { - return configuration_error(format!( - "Fastly config chunk `{}` length mismatch: expected {}, got {}", - chunk.key, chunk.len, chunk_len - )); - } - actual_envelope_len = actual_envelope_len.saturating_add(chunk_len); - if actual_envelope_len > pointer.envelope_len { - return configuration_error(format!( - "Fastly config envelope exceeded declared length {} while reading chunk `{}`", - pointer.envelope_len, chunk.key - )); - } - let chunk_sha = sha256_hex(chunk_value.as_bytes()); - if chunk_sha != chunk.sha256 { - return configuration_error(format!( - "Fastly config chunk `{}` sha mismatch: expected {}, got {}", - chunk.key, chunk.sha256, chunk_sha - )); - } - envelope_json.push_str(&chunk_value); - } - - if envelope_json.len() != pointer.envelope_len { - return configuration_error(format!( - "Fastly config envelope length mismatch: expected {}, got {}", - pointer.envelope_len, - envelope_json.len() - )); - } - let envelope_sha = sha256_hex(envelope_json.as_bytes()); - if envelope_sha != pointer.envelope_sha256 { - return configuration_error(format!( - "Fastly config envelope sha mismatch: expected {}, got {}", - pointer.envelope_sha256, envelope_sha - )); - } - - Ok(envelope_json) -} - -fn sha256_hex(bytes: &[u8]) -> String { - hex::encode(Sha256::digest(bytes)) -} - -fn configuration_error(message: String) -> Result> { - Err(Report::new(TrustedServerError::Configuration { message })) -} - #[cfg(test)] mod tests { use super::*; @@ -263,7 +136,6 @@ mod tests { use crate::settings::Settings; use crate::test_support::tests::{crate_test_settings_str, sample_permissions}; use edgezero_core::blob_envelope::BlobEnvelope; - use serde_json::json; use std::collections::BTreeMap; struct MemoryConfigStore { @@ -437,93 +309,6 @@ mod tests { ); } - #[test] - fn loads_settings_from_fastly_chunk_pointer() { - let mut settings = - Settings::from_toml(&crate_test_settings_str()).expect("should parse test settings"); - settings.proxy.allowed_domains = vec!["*.example".to_owned(), "*.example.com".to_owned()]; - let envelope_json = envelope_json(&settings); - // Chunks as long as an entry may be, cut where a character ends, so a - // blob carrying the sample policy takes several. - let mut chunks: Vec<(String, String)> = Vec::new(); - let mut rest = envelope_json.as_str(); - while !rest.is_empty() { - let mut end = rest.len().min(FASTLY_CONFIG_ENTRY_LIMIT); - while !rest.is_char_boundary(end) { - end -= 1; - } - let (chunk, remainder) = rest.split_at(end); - let key = format!("{CONFIG_BLOB_KEY}.__edgezero_chunks.test.{}", chunks.len()); - chunks.push((key, chunk.to_owned())); - rest = remainder; - } - assert!(chunks.len() > 2, "the blob should need several chunks"); - let pointer = json!({ - "edgezero_kind": FASTLY_CHUNK_POINTER_KIND, - "version": 1, - "envelope_sha256": sha256_hex(envelope_json.as_bytes()), - "envelope_len": envelope_json.len(), - "chunks": chunks - .iter() - .map(|(key, chunk)| json!({ - "key": key, - "sha256": sha256_hex(chunk.as_bytes()), - "len": chunk.len() - })) - .collect::>() - }) - .to_string(); - let mut entries = BTreeMap::from([(CONFIG_BLOB_KEY.to_string(), pointer)]); - entries.extend(chunks); - let store = MemoryConfigStore { entries }; - - let loaded = load_settings( - &store, - &StoreName::from(DEFAULT_CONFIG_STORE_ID), - CONFIG_BLOB_KEY, - ) - .expect("should load settings"); - - assert_eq!( - loaded.publisher.domain, settings.publisher.domain, - "should reconstruct chunked envelope" - ); - } - - #[test] - fn rejects_chunk_pointer_when_declared_lengths_do_not_match_envelope_len() { - let chunk_key = format!("{CONFIG_BLOB_KEY}.__edgezero_chunks.test.0"); - let pointer = json!({ - "edgezero_kind": FASTLY_CHUNK_POINTER_KIND, - "version": 1, - "envelope_sha256": sha256_hex(b"ab"), - "envelope_len": 1, - "chunks": [ - { - "key": chunk_key, - "sha256": sha256_hex(b"ab"), - "len": 2 - } - ] - }) - .to_string(); - let store = MemoryConfigStore { - entries: BTreeMap::from([(CONFIG_BLOB_KEY.to_string(), pointer)]), - }; - - let err = load_settings( - &store, - &StoreName::from(DEFAULT_CONFIG_STORE_ID), - CONFIG_BLOB_KEY, - ) - .expect_err("should reject malformed chunk length metadata"); - - assert!( - err.to_string().contains("chunk lengths total mismatch"), - "error should explain chunk length mismatch: {err:?}" - ); - } - #[test] fn fails_when_blob_key_is_missing() { let store = MemoryConfigStore { diff --git a/crates/trusted-server-core/src/streaming_processor.rs b/crates/trusted-server-core/src/streaming_processor.rs index e3d9272e6b..ec5ba1ac8f 100644 --- a/crates/trusted-server-core/src/streaming_processor.rs +++ b/crates/trusted-server-core/src/streaming_processor.rs @@ -9,13 +9,13 @@ //! # Platform notes //! //! This module is **platform-agnostic** (verified 2026-03-31; see -//! `docs/superpowers/plans/2026-03-31-pr8-content-rewriting-verification.md`). It has zero -//! `fastly` imports. [`StreamingPipeline::process`] is generic over +//! `docs/superpowers/plans/2026-03-31-pr8-content-rewriting-verification.md`). It imports +//! no platform SDK. [`StreamingPipeline::process`] is generic over //! `R: Read + W: Write` — any reader or writer works, including //! any platform body type (which implements `std::io::Read`) or standard //! `std::io::Cursor<&[u8]>`. //! -//! Future adapters (Cloudflare Workers, Axum, Spin) do not need to implement any compression or +//! Adapters do not need to implement any compression or //! streaming interface. See `crate::platform` module doc for the //! authoritative note. diff --git a/crates/trusted-server-core/src/tsjs_bundle.rs b/crates/trusted-server-core/src/tsjs_bundle.rs index 4e98e491c8..ace2e0152c 100644 --- a/crates/trusted-server-core/src/tsjs_bundle.rs +++ b/crates/trusted-server-core/src/tsjs_bundle.rs @@ -120,12 +120,10 @@ pub fn compose(parts: &[JsModulePart]) -> String { /// only sets derived from configuration, never sets derived from request /// input. /// -/// The memo can only hit where the process outlives the request, which of the -/// four adapters means the Axum dev server alone, because its `main` builds -/// the router once before serving. Fastly starts a fresh Wasm instance per -/// request, and `edgezero_adapter_cloudflare::run_app` and -/// `edgezero_adapter_spin::run_app` both call `build_app` inside the -/// per-request entry point, so on those three every call is a miss. A miss +/// The memo can only hit where the process outlives the request, as on a +/// native server whose `main` builds the router once before serving. On an +/// adapter that starts a fresh instance per request, or builds the app inside +/// its per-request entry point, every call is a miss. A miss /// costs a key vector, two mutex locks and a stored copy of the hash, all /// beside a SHA-256 over the whole bundle, so the memo is close to free where /// it cannot hit and removes the hash entirely where it can. diff --git a/crates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rs b/crates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rs index 1a2bb17443..c0b2df3148 100644 --- a/crates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rs +++ b/crates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rs @@ -454,13 +454,32 @@ mod tests { /// The entries a config store holds, as Viceroy serves them. struct GeneratedConfigStore(HashMap); - impl PlatformConfigStore for GeneratedConfigStore { - fn get(&self, _store_name: &StoreName, key: &str) -> Result> { + impl GeneratedConfigStore { + fn entry(&self, key: &str) -> Result> { self.0 .get(key) .cloned() .ok_or_else(|| Report::new(PlatformError::ConfigStore)) } + } + + impl PlatformConfigStore for GeneratedConfigStore { + /// The entry at `key`, its chunks joined in order when it is a chunk + /// pointer, as the Fastly config store returns a long entry whole. + fn get(&self, _store_name: &StoreName, key: &str) -> Result> { + let value = self.entry(key)?; + let pointer: serde_json::Value = + serde_json::from_str(&value).unwrap_or(serde_json::Value::Null); + if pointer["edgezero_kind"] != CHUNK_POINTER_KIND { + return Ok(value); + } + pointer["chunks"] + .as_array() + .into_iter() + .flatten() + .map(|chunk| self.entry(chunk["key"].as_str().unwrap_or_default())) + .collect() + } fn put( &self, diff --git a/docs/guide/configuration-rules.md b/docs/guide/configuration-rules.md index 28a71189cc..86dfa63ee3 100644 --- a/docs/guide/configuration-rules.md +++ b/docs/guide/configuration-rules.md @@ -50,7 +50,7 @@ setting = "value" | `source` | one | `module`, a string | `origin`, an HTTP origin, or a module that supplies a source | | `ec` | one | `module`, a string | `hmac`, `host_signals`, `client_fixed` (demonstration builds), or an integration that supplies identity | | `geo` | one | `module`, a string | `platform`, `none`, or an integration that supplies location | -| `device` | one | `module`, a string | `builtin` (the default), `fastly`, or an integration that supplies device signals | +| `device` | one | `module`, a string | `builtin` (the default), `platform`, or an integration that supplies device signals | | `permission-signal` | several, in order | `modules`, a list | `gpc`, `gpp`, `us-privacy`, `tcf` | | `demand` | several | `modules`, a list | `auction-protocol.openrtb`, `auction.prebid-server`, `auction.aps` | | `ad-server` | one | `module`, a string | `iabtl` | @@ -300,6 +300,7 @@ endpoint = "https://house.example.com/openrtb2/auction" | --------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | | `[integrations.]` with `enabled = true` | the module in the section of its type, and `[.]` only for settings | | `[ec.providers.]` | `[ec.]` | +| `[device] module = "fastly"` | `[device] module = "platform"` | | `[permission_signal] sources` | `[permission-signal] modules` | | `host-signals`, `client-fixed`, `gpp-sale-opt-out`, `gpp_sale_opt_out`, `us_privacy` | `host_signals`, `client_fixed`, `gpp`, `us-privacy` | | `[auction.providers.]` with `protocol`, `profile` and `profile_config` | `[demand] modules` and `[demand.]`, with `implementation` and the settings flat in the table | diff --git a/docs/guide/configuration.md b/docs/guide/configuration.md index 75304933af..bae0ef2719 100644 --- a/docs/guide/configuration.md +++ b/docs/guide/configuration.md @@ -748,13 +748,15 @@ trimmed or normalized. Missing, empty, duplicate, non-UTF-8, mismatched, or malf values do not reject the request; Trusted Server safely falls back to the immediate peer address. Both configured headers are removed before routing. -Header names are validated case-insensitively. `ip_header` must be -`fastly-client-ip` or start with `x-`, while `auth_header` must start with `x-`. -The names must differ. Neither field may use a header name reserved for -Trusted Server's own internal signals (for example `x-forwarded-for`, -`x-geo-info-available`, `x-ts-ec`, `x-ts-tls-protocol`, or `x-ts-tls-cipher`); -the full reserved set is the internal-header list that Trusted Server strips -before forwarding to third parties. These restrictions exclude standard +Header names are validated case-insensitively. `ip_header` must start with +`x-` or end with `-client-ip`, as a platform's own client address header such as +`fastly-client-ip` does, while `auth_header` must start with `x-`. The names +must differ. Neither field may use a header name reserved for Trusted Server's +own internal signals, for example `x-forwarded-for`, `x-geo-info-available`, +`x-ts-ec`, `x-ts-tls-protocol`, or `x-ts-tls-cipher`. The full reserved set is +the internal-header list that Trusted Server strips before forwarding to third +parties, together with the internal headers the adapter declares for its +platform. These restrictions exclude standard sensitive headers such as `Host`, `Content-Length`, `Cookie`, and `Authorization`, as well as every Trusted Server internal header. Choose dedicated `x-` names that no other application or routing logic uses, because @@ -1010,17 +1012,17 @@ Selects how a request is classified into the coarse device signals the Edge Cook ### `[device]` -| Field | Type | Required | Description | -| -------- | -------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `module` | String or null | No | Name of the device-detection module: `builtin` (the default, User-Agent only, no host-specific call), `fastly` to add the host's TLS (JA4) and HTTP/2 probabilistic identifiers, or the name of a module a crate outside core supplies | +| Field | Type | Required | Description | +| -------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `module` | String or null | No | Name of the device-detection module: `builtin` (the default, User-Agent only, no host-specific call), `platform` to add the host's TLS (JA4) and HTTP/2 probabilistic identifiers where the platform supplies them, or the name of a module a crate outside core supplies | -The default `builtin` module classifies from the User-Agent alone and makes no host-specific call, so the default path stays host-neutral. Neither `builtin` nor `fastly` has settings, and `[device]` holds no module settings table. A module from a crate at `crates/device/` is written `` or `device.`, and the crate's own module has to be selected in the section of its type as well. Naming a module this deployment does not run fails at startup, with the device modules it does run. +The default `builtin` module classifies from the User-Agent alone and makes no host-specific call, so the default path stays host-neutral. Neither `builtin` nor `platform` has settings, and `[device]` holds no module settings table. A module from a crate at `crates/device/` is written `` or `device.`, and the crate's own module has to be selected in the section of its type as well. Naming a module this deployment does not run fails at startup, with the device modules it does run. **Example**: ```toml [device] -module = "builtin" # or "fastly" to add TLS and HTTP/2 evidence +module = "builtin" # or "platform" to add TLS and HTTP/2 evidence ``` **Environment Override**: diff --git a/docs/guide/fastly.md b/docs/guide/fastly.md index f6bcfbba1a..936bd1fa0f 100644 --- a/docs/guide/fastly.md +++ b/docs/guide/fastly.md @@ -118,8 +118,8 @@ carries other traffic, a dedicated `x-` name is safer: the front-door VCL then never modifies `Fastly-Client-IP`, so security rules, rate limiters, logging formats, and vendor snippets that read it keep working unchanged. -Both names are validated at startup. `ip_header` must be `fastly-client-ip` or -begin with `x-`, `auth_header` must begin with `x-`, the two must differ, and +Both names are validated at startup. `ip_header` must begin with `x-` or end +with `-client-ip`, as `fastly-client-ip` does, `auth_header` must begin with `x-`, the two must differ, and neither may reuse a Trusted Server internal header name such as `x-forwarded-for` or `x-ts-ec`. Use lowercase in the TOML. diff --git a/docs/guide/integrations/datadome.md b/docs/guide/integrations/datadome.md index 8b275a5a13..a2cd9a6e8c 100644 --- a/docs/guide/integrations/datadome.md +++ b/docs/guide/integrations/datadome.md @@ -174,7 +174,7 @@ A request is protected when all of the following are true: 5. The client IP does not match `protection_excluded_ip_cidrs` or any Config Store-backed CIDR source. 6. The client ASN is not listed in `protection_excluded_asns`. 7. No `protection_exclusion_rules` match. -8. The request does not contain a matching enabled `protection_test_bypass` credential while `FASTLY_IS_STAGING=1`. +8. The request does not contain a matching enabled `protection_test_bypass` credential in a staging environment. Static assets are excluded by default using a case-insensitive file-extension regex. Trusted Server internal routes such as `/static/tsjs=`, `/integrations/`, `/first-party/`, discovery routes, and signature-verification routes are also excluded by default. @@ -187,7 +187,7 @@ can configure a static header credential that skips only the server-side Protection API: ```toml -# Runtime activation also requires FASTLY_IS_STAGING=1. +# Runtime activation also requires a staging environment. [bot-protection] module = "datadome" @@ -197,7 +197,8 @@ credential_secret_name = "datadome_test_bypass" ``` `protection_test_bypass` requires `enable_protection = true`; it is disabled -when omitted and is runtime-active only when `FASTLY_IS_STAGING=1`. +when omitted and is runtime-active only in a staging environment, which only the +Fastly adapter declares, when `FASTLY_IS_STAGING=1`. `FASTLY_IS_STAGING` is supplied at runtime by Fastly (`1` in staging and `0` in production); it is not compiled into or promoted with the Wasm artifact. Verify staging through the `X-TS-ENV: staging` response signal and the integration diff --git a/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md new file mode 100644 index 0000000000..25d4e8e6b3 --- /dev/null +++ b/docs/superpowers/specs/2026-10-15-fastly-names-in-core-design.md @@ -0,0 +1,189 @@ +# Design Spec: The last Fastly names leave core + +**Status:** Proposed, 2026-10-15. Raised as pull request IABTechLab/trusted-server#1399 (from jwrosewell/trusted-server#44), for Tech Lab review. +**Author:** 51Degrees (contributed), for Tech Lab review +**Related specs:** `2026-10-14-last-names-in-core-design.md`, `2026-10-14-backend-naming-design.md` +**Related PRs:** IABTechLab/trusted-server#1399, jwrosewell/trusted-server#44 +**Issues:** IABTechLab/trusted-server#777 (its Fastly row) +**Last updated:** 2026-10-27 + +> **Why this spec exists.** After the series before this one, core's shipped code still names one hosting platform, Fastly, in small pieces. They are the format of a long configuration document, a staging variable, three request headers, a device module selector and the names of two functions. Each is a place where a change by one platform is a change in core. Core's comments, tests and README files name Fastly, Cloudflare, Spin, Fermyon, Viceroy and Axum as well, and three tests use a host under Fastly's `edgecompute.app` domain. After this change no line of core names a hosting platform, and a guard outside core keeps it so (section 2.6). One Fastly header, `x-compress-hint`, whose name does not name the platform, is still set by core after this change (section 3). + +## 1. Problem + +These names remain in core, each with the files that hold it: + +| What core names | Where | +| ---------------------------------------------------------------------- | --------------------------------------------------------------------------------- | +| The Fastly config store's format for a document too long for one entry | `settings_data.rs` | +| Fastly's staging variable | `constants.rs`, the DataDome module | +| Fastly's TLS, client address and debug request headers | `http_util.rs`, `publisher.rs`, `settings.rs`, `constants.rs` | +| The `fastly` device module selector | `ec/device.rs`, `integrations/registry.rs`, and the doc comments of `settings.rs` | +| Fastly in the names of the asset proxy's response conversions | `proxy.rs` | + +Each name is small. Each one means that another platform, or a change in how Fastly stores, flags or forwards something, is an edit to core rather than to the adapter beside it. + +Core's comments, doc comments, tests and README files named platforms too. Each told a reader that core knows which platform runs it or serves its origin. Section 2.6 counts them. + +## 2. Design + +Each name moves to the adapter that reads it, and core keeps a neutral declaration or interface in its place. The settings a deployment writes do not change, except the device selector. + +### 2.1 A platform's config store returns a long document whole + +A Fastly config store entry holds at most 8,000 characters, so a longer settings document is written as chunks, with a pointer at its key naming each chunk's key, length and hash. Core's settings loader recognized the pointer and read the chunks itself. + +**The interface.** `PlatformConfigStore::get` returns an entry whole. The Fastly adapter's store follows a pointer, reads each chunk and checks every length and hash before it returns the document, so core reads what any platform's store returns. + +**What each adapter does.** The Fastly adapter's store is the only one that follows a pointer. The Axum adapter loads its settings through the same core function (`get_settings_from_config_store_with`) from a store that reads an environment variable, so it followed a pointer before this change and now reads the entry as it finds it. The Cloudflare and Spin adapters load their settings from the blob they read (`settings_from_config_blob_with`) and never followed a pointer. + +**What changes for an operator.** A long document reads as before on Fastly, and one whose chunks disagree with the pointer is still refused. The detail of the refusal moves. Core's loader put the detail in the message of its own `Configuration` error. The Fastly store now attaches the detail to a config store error, which the loader wraps in a general `Configuration` error saying that the key could not be read. The detail therefore shows when the error is printed with `{:?}`, as the Fastly adapter does when it logs that it failed to build its state, and not in the error's `to_string()`, which the removed core test read. A refused document stops the service from building its application, so every route that reads the document answers 500 with `An internal error occurred`, apart from `/health`, which is answered before any settings are read. + +### 2.2 The platform says whether it runs in a staging environment + +Core named Fastly's `FASTLY_IS_STAGING` variable, and the DataDome module read it so that its test bypass works only on a staging version. + +**The interface.** An adapter declares whether its platform runs the service in a staging environment with `platform::declare_platform_staging`, and a module asks `platform::platform_is_staging`, which answers false until an adapter declares otherwise. `ENV_FASTLY_IS_STAGING` is gone from core, and a crate outside core that used it stops building. + +**What each adapter declares.** The Fastly adapter declares staging when `FASTLY_IS_STAGING` is `1`. The Axum, Cloudflare and Spin adapters declare nothing. + +**What changes for an operator.** Nothing on Fastly. On the Axum, Cloudflare and Spin adapters the DataDome bypass no longer follows a `FASTLY_IS_STAGING` variable and stays inactive, because none of them declares a staging environment. The warning DataDome logs when a bypass is configured and inactive now says the platform does not run the service in its staging environment, where it said `FASTLY_IS_STAGING` is not 1. The DataDome guide (`docs/guide/integrations/datadome.md`) names the condition in the same terms. + +### 2.3 The platform names the request headers it sets + +Core read Fastly's `fastly-ssl` header for the scheme and kept it from the origin, stripped `fastly-ssl` and `fastly-client-ip` as headers a client could send, accepted `fastly-client-ip` by name as a trusted client address header, and listed `x-debug-fastly-pop` among the headers never passed to a third party. + +**The interface.** An adapter declares three lists before it serves. + +- `platform::declare_platform_tls_headers` names a header its platform sets on a request that arrived over TLS. Core reads it for the scheme after the TLS status the adapter reports and the standard forwarding headers, and removes it from the request to the publisher's origin (`rewrite_origin_request` and `handle_publisher_request` in `publisher.rs`). The asset proxy and the first-party proxy build their origin requests from fixed lists of headers to copy (`ASSET_PROXY_FORWARD_HEADERS` and `PROXY_FORWARD_HEADERS` in `proxy.rs`), and neither list holds a TLS header. +- `platform::declare_platform_spoofable_headers` names the platform's other headers a client could also send. `sanitize_forwarded_headers` strips them, and the TLS headers, with the three standard forwarding headers. +- `platform::declare_platform_internal_headers` names headers never passed to a third party. `constants::is_internal_header` answers for core's fixed list and these together, and the trusted client address settings refuse any of them. + +The Fastly adapter makes its declarations in `app::declare_platform()`. `run()` calls the function before the adapter serves, which is before the first settings load and the validation that refuses a declared internal header. Every state build calls the function again, so a state built without `run()` is declared the same, and declaring twice changes nothing. + +A trusted client address header must start with `x-` or end with `-client-ip`. That is the shape of Fastly's `fastly-client-ip`. Another platform's own client address header can take another shape, as Cloudflare's `cf-connecting-ip` does, and was refused before this change and is refused now. The rule is a pattern because `ts config validate` checks the settings where no adapter has declared anything. The pattern accepts any name ending `-client-ip`, where only `fastly-client-ip` was accepted before. + +**What reads the declarations.** Each declared list has these readers in production. + +| Declared list | What reads it | Runs on | +| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------- | +| TLS headers | `detect_request_scheme` in `http_util.rs`, for the scheme | Every adapter | +| TLS headers | `remove_platform_tls_headers` in `publisher.rs`, for the request to the publisher's origin | Every adapter | +| TLS and spoofable headers | `sanitize_forwarded_headers` in `http_util.rs`, for the strip at entry | The Spin adapter only | +| Internal headers | `constants::is_internal_header`, for the copy of a client's `x-` headers to a third party by Permutive and Lockr, and for the check of `[trusted_client_ip]` | Every adapter | + +The Spin adapter is the only adapter that calls `sanitize_forwarded_headers` (`normalize_spin_request` in its `app.rs`), and it declares nothing. The Fastly adapter does not call the function. Its entry point strips with `sanitize_fastly_forwarded_headers` in its `compat.rs`, which removes the three standard headers and its own `fastly-ssl` and `fastly-client-ip` by name and does not read the declared lists. The Axum and Cloudflare adapters call neither function. So the one production call that reads the spoofable list is on an adapter that declares nothing, and the Fastly adapter's declaration of `fastly-client-ip` as spoofable is read by nothing. An adapter that declares a TLS or spoofable header gets the header stripped at entry only if the adapter calls `sanitize_forwarded_headers`, so declaring is not enough on its own. + +**What each adapter declares.** The Fastly adapter declares `fastly-ssl` as a TLS header, `fastly-client-ip` as a spoofable header and `x-debug-fastly-pop` as an internal header. The Axum, Cloudflare and Spin adapters declare nothing. + +**What changes for an operator.** On Fastly the scheme, the strip at entry, the removal from the origin request and the keeping of `x-debug-fastly-pop` from third parties all behave as before, and `ip_header = "fastly-client-ip"` stays valid. One refusal moves. `ts config validate` no longer refuses `x-debug-fastly-pop` as an `ip_header` or `auth_header` name, and the Fastly service refuses it when it loads the document (see "Where a refusal happens" below). On the Axum, Cloudflare and Spin adapters, which declare nothing, these change. + +1. On the Axum and Cloudflare adapters a client's `fastly-ssl` is no longer removed from the request to the publisher's origin. +2. On the Axum and Cloudflare adapters a request that carries `fastly-ssl: 1` and no scheme in `Forwarded` or `X-Forwarded-Proto` is no longer taken to be HTTPS. Neither adapter reports a TLS status (the `ClientInfo` each builds in its `platform.rs` carries none) and neither strips a client's forwarding headers, so core read the header there as the scheme and a client could send it. The scheme is now `http` unless one of those two headers says otherwise. The Spin adapter's scheme is unchanged, because Spin removed the header at entry and takes the scheme from the runtime URL it trusts. +3. On the Spin adapter a client's `fastly-ssl` and `fastly-client-ip` are no longer stripped at entry, so both now reach the publisher's origin with the rest of the client's headers, except that `fastly-client-ip` is still removed where `[trusted_client_ip]` names it as `ip_header`. The request middleware of each of the Axum, Cloudflare and Spin adapters removes the configured `ip_header` and `auth_header` (`middleware.rs`), before this change and after, so nothing changes for `fastly-client-ip` on the Axum and Cloudflare adapters. +4. On the Axum, Cloudflare and Spin adapters a client's `x-debug-fastly-pop` is no longer kept from third parties. Permutive and Lockr copy a client's `x-` headers that are not internal, so both now pass a client-sent copy on, and the header is no longer refused as an `ip_header` or `auth_header` name. Only a client can send the header on these adapters, so the effect is small. + +**Where a refusal happens.** The settings check refuses a declared internal header named as `ip_header` or `auth_header`, and an `ip_header` that neither starts with `x-` nor ends with `-client-ip`. The second refusal needs no declaration, so `ts config validate` applies it. The first depends on what the running adapter declared, and `ts config validate` runs with no adapter's declarations, so a document that names `x-debug-fastly-pop` passes `ts config validate` and is refused when the Fastly service loads the document. Before this change `ts config validate` refused the name, because the fixed list held it. A refused document stops the service from building its application, so every route that reads the document answers 500 with `An internal error occurred`, apart from `/health`, and the log carries the reason. + +**For crates outside core.** `SPOOFABLE_FORWARDED_HEADERS` holds the three standard forwarding headers and no longer holds `fastly-ssl` and `fastly-client-ip`, and `INTERNAL_HEADERS` no longer holds `x-debug-fastly-pop`. A crate that reads either constant still builds and now sees the shorter list. To see what the running platform added, the crate reads `platform::platform_tls_headers` and `platform::platform_spoofable_headers`, or calls `constants::is_internal_header`. `HEADER_X_DEBUG_FASTLY_POP` is gone, and a crate that used it stops building. + +**Open questions.** Both ask for a code change, and this design chooses neither. + +1. Should the Fastly adapter's `sanitize_fastly_forwarded_headers` read the declared TLS and spoofable lists, so that what an adapter declares and what it strips cannot differ? The other choice is for the spoofable list to leave the declarations, so that each adapter strips its own headers by name as the Fastly adapter does today. +2. Is it acceptable that the Axum, Cloudflare and Spin adapters pass a client-sent `x-debug-fastly-pop` to third parties? The other choice is to keep it from third parties on every adapter, as before this change. + +### 2.4 The platform's own device module is `platform` + +`[device] module = "fastly"` selected the module the Fastly adapter builds from its TLS and HTTP/2 signals, so core's device selection and the registry both named Fastly. + +**The interface.** `[device] module = "platform"` selects the platform's own module, as `[geo] module = "platform"` selects the platform's geo lookup, and `build_device_module` builds it through the `build_platform` factory the adapter passes. + +**What each adapter does.** The Fastly adapter is the only one that calls `build_device_module`, and its factory builds the Fastly device module from the request's TLS and HTTP/2 signals. The Axum, Cloudflare and Spin adapters never call the function, so `module = "platform"` is accepted there and selects nothing, as `module = "fastly"` was before. + +**What changes for an operator.** A document that wrote `module = "fastly"` now writes `module = "platform"`. The registry refuses the old value as a device module no running module supplies, when the service builds its modules as it starts. `ts config validate` does not check the device selector, so it passes a document that still writes `fastly`, and the service then answers 500 on every route that reads the document, apart from `/health`, as it does for any refused document. + +**Rollout.** The two values do not overlap. A package from before this change refuses `platform` and a package from this change refuses `fastly`, each at startup, and `ts config validate` passes either value. The package and the document therefore have to change together, and the live change needs the same pairing because the active version reads one document. A deployment can try the pair first on a staged Fastly version, since `ts deploy --staging` and `ts config push --staging` give the staged version a document of its own that the active version does not read (`docs/guide/cli.md`). A deployment that selects no device module, or `builtin`, changes nothing. + +### 2.5 Names that were not Fastly's + +`platform_response_to_fastly` returned a buffered response and `platform_response_to_fastly_asset` an asset response that keeps a stream, and neither touched Fastly. They are `platform_response_to_buffered` and `platform_response_to_asset`. + +**What changes for an operator.** One error message changes with the names. The `Proxy` error for a streaming body in the buffered conversion (`proxy.rs`) now reads `streaming platform response body is not supported by buffered response conversion`, where it named a Fastly response conversion. A `Proxy` error answers 502 with `An internal error occurred`, so the message reaches the log and not the response. Both functions are internal to core. + +### 2.6 Core names no hosting platform + +Core is built on EdgeZero and names no hosting platform in any line, whether code, comment, test, document or file name. EdgeZero's own names stay, because EdgeZero is the platform layer core is built on. An adapter may name its own platform. + +**What changes.** + +- Comments and doc comments say what they meant without the platform's name, for example a streaming adapter, an adapter whose platform has a shared cache, or a platform whose `send_async` runs each request eagerly. Where one cited a platform SDK's source, the adapter that maps core's type onto that SDK already documents it (`apply_fastly_cache_intent` in the Fastly adapter's `platform.rs`). +- The `Surrogate-Control` tests in `publisher.rs` test the standard header core reads for every platform, so their names and assertion messages say `Surrogate-Control`, or edge freshness, where they said Fastly. The wildcard exclusion test in `creative.rs` excludes `*.cdn.example.com` where it excluded a platform's domain. A registry test name, a publisher test name and a stub's assertion message drop the platform they named. +- The test that the largest `KvMetadata` serializes within 2048 bytes moves to the tests of the Fastly adapter's store (`ec_kv.rs`), because the limit is the Fastly KV Store's. +- The test that core's code never uses the Fastly SDK moves from core's `migration_guards.rs` to the Fastly adapter's `core_sdk_guard.rs`. A build script in the adapter embeds every Rust source under core's `src` by path, so a file added to core is checked with no edit to the test. Core's build script no longer lists its own sources. +- Core's README and the auction README show the command that runs core's tests on the host and point at `TESTING.md` for the adapter aliases, where they listed each alias by its platform. +- Three HTML processor tests use `test-publisher-ts.edge.example` as the request host, where they used a host under `edgecompute.app`. +- `scripts/lint-core-platform-names.py` fails on any line or path under `crates/trusted-server-core` that names a hosting platform, a tool named for one, or a domain a platform serves deployments from. It splits text into words at every character that is not a letter and at each change from lower to upper case, so `FASTLY_IS_STAGING` and `edgezero_adapter_cloudflare` each name a platform. Six names, among them `spin`, `axum` and `aws`, count only as words of their own, because ordinary words such as `spinner` and `withdraws` contain them. Eighteen, among them `fastly`, `cloudflare`, `akamai`, `vercel` and `netlify`, also count at the start of a word, so `akamaized` counts. Sixteen domains, among them `edgecompute.app`, `workers.dev` and `fermyon.app`, count where they are not part of a longer host name. It runs its matcher on every name and domain and on known non-names before it reads core, and stops with exit 2 when the matcher is wrong. The Run Format workflow runs it as a job of its own, and `AGENTS.md` lists it among the CI gates. + +**The count.** Lines of `crates/trusted-server-core` that name each platform on this branch before and after this change, counted as a whole word with `git grep -i -w`, and with the guard, which also counts a name inside an identifier such as `FASTLY_IS_STAGING`. + +| Name | `git grep -i -w` before (files) | Guard before | After, either count | +| -------------------------- | ------------------------------- | ------------ | ------------------- | +| Fastly | 100 (32) | 110 | 0 | +| Cloudflare | 26 (14) | 28 | 0 | +| Axum | 27 (14) | 27 | 0 | +| Spin | 21 (10) | 23 | 0 | +| `edgecompute.app` | 5 (1) | 5 | 0 | +| Viceroy | 4 (3) | 4 | 0 | +| Fermyon | 1 (1) | 1 | 0 | +| Every other name or domain | 0 | 0 | 0 | + +At `13d57b03f`, this row's last commit before its comments, documents and tests were changed, the guard finds 173 lines. One is the `_vercel` entry of the attestation prefixes, which core no longer has, and the other 172, some of them naming two platforms, are comments, documents or test code. At this row's head the guard finds none. + +**What changes for an operator.** Nothing that core does when it runs changes. + +**For crates outside core.** Nothing in core's API changes. A change that adds a hosting platform's name to core, in a comment or a test as much as in code, fails the guard. The Fastly adapter gains a build script and a `regex` dev-dependency for the test of core's sources. + +## 3. What is not in this change + +- The `x-compress-hint` header, a Fastly compression hint. Core sets it to `on` on the script bundle responses it serves (`HEADER_X_COMPRESS_HINT` in `constants.rs`, used in `publisher.rs`) and lists it among the headers never passed to a third party (`constants.rs`). The header's name does not name the platform, so the guard of 2.6 passes it. +- The comment at the top of `platform/request_headers.rs`, which says a TLS or spoofable header is removed at entry, without the precondition in 2.3 that the adapter calls `sanitize_forwarded_headers`. It is a code comment and stays for a later change. +- The Axum, Cloudflare and Spin adapters' comments comparing their behavior with the Fastly adapter's, and their two errors that suggest the Fastly adapter for parallel bidder fan-out. +- The `ts` development proxy's removal of `fastly-ssl`. + +## 4. Tests + +- The Fastly adapter's `whole_entry` returns a document that fits, and other JSON at the key, as they are, reads a chunked document back whole, and refuses a pointer whose lengths disagree with the document and a chunk whose hash disagrees. +- The DataDome test bypass applies while the platform declares staging and not otherwise. +- A declared TLS header sets the scheme and an undeclared one does not, the declared TLS and spoofable headers are stripped with the standard ones, the auction sees the client's declared TLS header where the origin request does not, a declared internal header is refused in the `auth_header` slot of `[trusted_client_ip]`, and a bare `client-ip` is refused where `example-client-ip` is accepted. +- The `platform` device selector builds the module the adapter's factory supplies, and no selector builds the built-in one. +- The asset proxy's tests pass unchanged through the renamed conversions. +- The guard of 2.6 passes on core after this change and fails on core before it. Each of its 40 names and domains, planted once on a line of its own in a file git does not yet track, fails it on that line, and so does the `edgecompute.app` test host put back where it was. It passes `spinner`, `SpinyAI`, `steadfastly`, `denominator`, `withdraws`, `EdgeZero`, `notworkers.dev` and `butterfly.io`. With its word pattern or its domain pattern broken to match nothing, it stops with exit 2 before it reads core. +- The Fastly adapter checks that core's code never uses the Fastly SDK, that the list of core's sources it embeds covers core, and that the largest `KvMetadata` fits the Fastly KV Store's 2048-byte metadata limit. + +Not tested: + +- `FastlyPlatformConfigStore::get`. The tests call `whole_entry` with a reader of their own, and `get` needs Fastly's config store, so nothing proves `get` passes each entry through `whole_entry`. +- That the Fastly adapter makes its declarations in `declare_platform()`. No test reads the lists back from the Fastly adapter, and core's tests declare example names of their own, so deleting a declaration passes every test. Deleting the TLS declaration sends `fastly-ssl` to the publisher's origin and stops core reading it for the scheme, deleting the internal declaration sends a client's `x-debug-fastly-pop` to third parties, and deleting the staging declaration switches the DataDome bypass off on Fastly staging. +- A declared internal header in the `ip_header` slot. The one test uses the `auth_header` slot, and one loop checks both slots. +- Any change on the Axum, Cloudflare and Spin adapters listed in 2.1 to 2.4. No adapter test sends a client's `fastly-ssl`, `fastly-client-ip` or `x-debug-fastly-pop` to those adapters, or selects `module = "platform"` on them. +- The strip at entry of a declared header through an adapter. Core's test declares example headers and calls `sanitize_forwarded_headers` directly, and the one adapter that calls the function declares none. +- The guard's job in the Run Format workflow, which only a pull request's checks run. +- That the guard's list of names is complete. It holds the platforms the stack has adapters or tools for and other common edge, CDN and hosting platforms, so a platform outside the list has to be added to it. + +## Revision record + +| Date | Change | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2026-10-15 | First draft. | +| 2026-10-15 | Fixes from review round 1. The readers of each declared list and the open questions on them, `x-debug-fastly-pop`, the scheme change, the effects on the other adapters, where each refusal happens, the rollout, the narrowed constants, `x-compress-hint`, the file count, the untested paths and the review log. | +| 2026-10-15 | Corrections from review round 2. The Fastly sentence of 2.3 and the changelog on the `x-debug-fastly-pop` refusal, the changelog on the spoofable strip and the scheme change, when the Axum, Cloudflare and Spin adapters strip `fastly-client-ip`, and the comment in `request_headers.rs` left for a later change. | +| 2026-10-16 | Core names no hosting platform or platform domain in any line. Section 2.6 with the counts, the `_vercel` prefix and the test host, the guard and the tests that moved to the Fastly adapter, and section 3 no longer leaves core's comments and tests for later. | +| 2026-10-27 | Row 12 no longer adds `ORIGIN_ASSET_PREFIXES`, so core holds no `_vercel` entry before this change. Section 2.6 names the commit it counts at, `13d57b03f`, where the guard finds 173 lines, that entry and 172 comments, documents or test code, and no other section names the prefix. | +| 2026-10-27 | The header no longer calls the pull request a draft. | + +## Review log + +1. **Round 1, 2026-10-15.** A review in a fresh session, against `split/49-host-features` at `69ba03219`. Issues found, two of them blocking. The page left out the change to `x-debug-fastly-pop` on the Axum, Cloudflare and Spin adapters, and described the spoofable header list as stripping headers although the one adapter that calls the function reading it declares nothing. This revision states both as the code has them today, adds the effects on the other adapters, the scheme change, where each refusal happens, the order for changing the package and the document, the name that stays and the untested paths, and corrects the partly true claims, whereas the two code fixes wait on the open questions in section 2.3. +2. **Round 2, 2026-10-15.** A lighter review in a fresh session, against this page at `b319bf039`, of the round 1 fixes and the new text. No blocking issues. This revision names in the Fastly sentence of 2.3 that `ts config validate` no longer refuses `x-debug-fastly-pop`, corrects when the Axum, Cloudflare and Spin adapters strip `fastly-client-ip`, lists the comment at the top of `platform/request_headers.rs` among the comments left for a later change, and corrects the changelog entry on the spoofable strip while adding the scheme change and the `x-debug-fastly-pop` change on Fastly to it. +3. **Round 3.** A review in a fresh session, against `split/49-host-features`, of the changes that follow attestation leaving row 12. No issues found. The round ran the guard's own matcher on core at the commit before this row's guard commits and found 173 lines, one of them the `_vercel` entry of `ORIGIN_ASSET_PREFIXES`, which row 12 no longer adds, so the count of 172 lines before this change, all of them comments, documents or test code, holds. This revision also stops calling the pull request a draft, as IABTechLab/trusted-server#1399 is not one. +4. **Round 4.** A review in a fresh session, checking the round 3 fixes. No issues found. The round found the header right, because IABTechLab/trusted-server#1399 is not a draft, and found nothing else to change in this design. diff --git a/scripts/README.md b/scripts/README.md index 2a86c6df69..1047e04927 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -24,6 +24,7 @@ their documented orchestration handles a probe deliberately. | `docs-proposal/validate.sh` | Merge SHA, work dir from `propose.sh`; Node, `npm`; no write credentials | Applies the patch, rejects paths outside the allowlist, and runs the docs gates. Leaves the edits staged. | | `docs-proposal/publish.sh` | Merge SHA, work dir from `propose.sh`; `gh` with write `GH_TOKEN` | Re-checks the allowlist without building, pushes `docs/auto/` unless a maintainer added to it, opens or edits its PR, and reviews each new commit per hunk. An empty rerun deletes the branch under a lease, then closes the PR; a failed branch lookup stops the run. Never writes `main`. | | `docs-proposal/test.sh` | Git, Node, and `jq` | Tests the helpers and the propose, validate, and publish flows against a temporary repository with stubbed `copilot`, `gh`, and `npm`; removes it on exit. | +| `lint-core-platform-names.py` | Python 3 and `git` | Reads every file under `crates/trusted-server-core` and fails on a hosting platform's name or domain. Writes nothing. | The four adapter smoke contracts are documented in the [deployment guides](../docs/guide/integrations-overview.md#adapter-support). diff --git a/scripts/lint-core-platform-names.py b/scripts/lint-core-platform-names.py new file mode 100755 index 0000000000..d015c449c2 --- /dev/null +++ b/scripts/lint-core-platform-names.py @@ -0,0 +1,250 @@ +#!/usr/bin/env python3 +"""Fail when a file under crates/trusted-server-core names a hosting platform. + +Core is built on EdgeZero and names no hosting platform, in code, comments, +tests, documents or file names, so that one platform's change is never a +change in core. Each adapter names its own platform. This check reads every +file git tracks, or would track, under crates/trusted-server-core and fails on +any line or path that holds a platform's name or one of its domains. + +Text is split into words at every character that is not a letter and at each +change from a lower case to an upper case letter, so `FASTLY_IS_STAGING`, +`FastlyStore`, `x-fastly-pop` and `adapter_fastly` each hold the word `fastly`. +A name in WHOLE_WORDS counts only as a word of its own, because each is also +part of ordinary words, so `spinner` holds no `spin`. A name in STEMS also +counts at the start of a word, so `akamaized` and `cloudflareinsights` count. +A domain in DOMAINS counts where it is not part of a longer host name, so +`example.workers.dev` counts and `notworkers.dev` does not. EdgeZero's own +names hold none of these. + +Before it reads core, the check runs its matcher on every name and domain and +on known non-names, so a broken pattern stops the check rather than passing +everything. + +Usage: python3 scripts/lint-core-platform-names.py +Exit status: 0 when core names no platform, 1 when it names one, 2 when the +check cannot run. +""" + +import os +import re +import subprocess +import sys +import traceback +from pathlib import Path + +CORE = "crates/trusted-server-core" + +# Hosting platforms, and tools named for one, that count only as words of +# their own. +WHOLE_WORDS = ("aws", "axum", "deno", "gcore", "spin", "wrangler") + +# Hosting platforms, and tools named for one, that also count at the start of +# a word. +STEMS = ( + "akamai", + "amazonaws", + "azure", + "bunnycdn", + "cloudflare", + "cloudfront", + "edgecast", + "edgeworkers", + "edgio", + "fastly", + "fermyon", + "heroku", + "miniflare", + "netlify", + "stackpath", + "vercel", + "viceroy", + "workerd", +) + +# Domains a platform serves its customers' deployments from. +DOMAINS = ( + "b-cdn.net", + "bunny.net", + "edg.io", + "edgecompute.app", + "edgekey.net", + "edgesuite.net", + "fermyon.app", + "fly.dev", + "fly.io", + "llnwd.net", + "netlify.app", + "onrender.com", + "pages.dev", + "railway.app", + "vercel.app", + "workers.dev", +) + +# Samples that hold nothing, or exactly the names given. +KNOWN_SAMPLES = ( + ("FASTLY_IS_STAGING", {"fastly"}), + ("x-debug-fastly-pop", {"fastly"}), + ("FastlyStore", {"fastly"}), + ("trusted_server_adapter_fastly::app", {"fastly"}), + ("edgezero_adapter_cloudflare::run_app", {"cloudflare"}), + ("cdn.akamaized.net", {"akamai"}), + ("Akamai EdgeWorkers", {"akamai", "edgeworkers"}), + ("https://test.edgecompute.app/", {"edgecompute.app"}), + ("a spinner, SpinyAI, steadfastly, a denominator, withdraws", set()), + ("EdgeZero edge workers on notworkers.dev and butterfly.io", set()), +) + +LETTERS = re.compile(r"[A-Za-z]+") +# An upper case run not followed by a lower case letter, such as an acronym, +# or a lower case run with at most one capital before it. +CAMEL = re.compile(r"[A-Z]+(?![a-z])|[A-Z]?[a-z]+") +DOMAIN_PATTERNS = tuple( + ( + domain, + re.compile( + r"(?