Skip to content

Verifier result contract is not platform-uniform: the same fields mean different things on each TEE variant, and nothing tells the relying party #1292

Description

@kvinwang

Summary

Classification: DESIGN. dstack supports five TEE variants. The /verify result contract — the
fields a relying party reads to decide whether to trust a CVM — uses the same field names for
materially different guarantees on each of them, and nothing in the response, the README, the SDKs
or the contract ABI says which guarantee is in force.

One observation in here is demonstrated with a test against a committed real fixture; the rest is
read from code and marked as argued. Nothing here is being called a vulnerability.

Tree: origin/next @ 030fbb2183.


1. What the design currently is

1a. event_log_verified is set unconditionally, on two platforms that replay no event log

details.event_log_verified = true at dstack/verifier/src/verification.rs:773, once
decode_app_info_ex returns, for every variant. Where the replay actually happens:

variant guest extends verifier replays and compares
dstack TDX RTMR3 (dstack/dstack-attest/src/lib.rs:189-193) yes — attestation.rs:1808-1815
GCP TDX RTMR3 + TPM PCR14 (lib.rs:196-201) both — attestation.rs:1808-1815, :1090-1101
AWS NitroTPM TPM PCR14 (lib.rs:202-206) yes — attestation.rs:1300-1314
SEV-SNP nothing (dstack/dstack-types/src/lib.rs:1355) nothing — the SevSnp arm attestation.rs:1144-1155 never reads runtime_events
Nitro Enclave nothing (same line) nothingattestation.rs:1107-1132 never reads runtime_events

The field's own doc comment (dstack/verifier/src/types.rs:83-88) and dstack/verifier/README.md:240
both describe RTMR3 / PCR14 digest-and-payload replay.

Demonstrated. Taking the committed dstack/verifier/fixtures/sev-snp-attestation.json, replacing
its 9 real runtime events with 3 fabricated ones and re-encoding, /verify still returns:

is_valid           = true
event_log_verified = true
os_image_hash_ver. = true
app_id             = 86e59625be93207bc2351c4d1bba20037cec8e16      (unchanged)

The app_id is unchanged because on SEV-SNP identity comes from the HOST_DATA-bound MrConfigV3
document (attestation.rs:1673-1686), not from the event log — so on SNP nothing is exploitable and
the finding is purely that the response says a check ran that did not. Test source at the bottom.

1b. tcb_status carries four different meanings under one name

policy_tcb_fields, dstack/verifier/src/verification.rs:44-68:

variant what "UpToDate" means there line
TDX / GCP TDX Intel's signed TCBInfo says this platform is at the current TCB level :46-53
SEV-SNP the report's four TCB version fields agree with each other. Says nothing about AMD's current level :56-59dstack/sev-snp-qvl/src/lib.rs:236-244
AWS NitroTPM nothing — the string is a literal :63
Nitro Enclave "", which fails an UpToDate gate closed :66

DstackApp.requireTcbUpToDate (DstackApp.sol:196-201) and DstackKms's hard
tcbStatus == "UpToDate" (DstackKms.sol:258-260) apply one predicate across all four, and cannot
do better: teeVariant is present in PolicyBootInfo (dstack/verifier/src/types.rs:35) but is
dropped at both auth-backend schemas (kms/auth-eth/src/server.ts:28-43,
kms/auth-eth-bun/index.ts:19-31) and is absent from IAppAuth.AppBootInfo (IAppAuth.sol:36-46).
auth-simple parses it (auth-simple/index.ts:12) and never reads it again. keyProviderInfo is
dropped at the same hop.

docs/security/security-model.md:188 says AWS NitroTPM "does not expose a dstack-style TCB status,
so policy must rely on …" — while the code exposes one that reads "UpToDate". This overlaps
AUDIT-BACKLOG K-b; PR #1268 confirms it with
aws_nitro_tpm_tcb_status_is_synthesized_and_no_backend_can_tell and states the fix is an ABI change
deliberately not in that diff. The four-way divergence and the SEV-SNP semantics are not covered
there.

1c. Nothing outside /verify names the variant at all

surface names the variant? says what it does not cover?
/verify JSON yes — details.tee_variant verifier/src/types.rs:108 partly; see 1d
verifier/README.md yes no — and inaccurate in five places, see 1d
guest Info v0/v1 no no. Nearest are two raw, unattested DMI strings, cloud_vendor/cloud_product (guest-agent/rpc/proto/agent_rpc_v1.proto:365-367) — the inputs to Platform::detect_from_dmi, not the resolved variant
Rust / Python / Go / JS SDKs no — none of the four no. sdk/rust/types/src/dstack_v1.rs:288-327, sdk/python/src/dstack_sdk/dstack_client_v1.py:191-192, sdk/go/dstack/client_v1.go:100-118, sdk/js/src/client-v1.ts:191-212. No SDK parses PlatformEvidence's kind tag (dstack-attest/src/v1.rs:64-87)
KMS GetAttestationInfo yes — kms/rpc/proto/kms_rpc.proto:170 no; operator onboarding UI, not app-facing
contract ABI no no — see 1b

1d. The right pattern already exists in the tree, applied once

acpi_tables_verified's doc comment names the platforms it does not apply to
(verifier/src/types.rs:92-101) and is pinned by a test. event_log_verified,
os_image_hash_verified and os_image_is_dev do not. PR #1266 introduces exactly the right
construction for a fourth field — os_image_hash_anchor, filled from one exhaustive match so a new
variant fails the build — but it is one field on one PR, and os_image_hash_anchor does not exist on
next today.

README inaccuracies found while checking this, three of which PR #1266 does not cover:

  • :240 claims replay "for RTMR3 and AWS NitroTPM PCR14"; silent on SNP / Nitro Enclave where
    event_log_verified is still true.
  • :48-62 and :215-229 document mrtd / rtmr0..3 fields on app_info; AppInfo
    (dstack-attest/src/attestation.rs:2580-2612) has no such fields, and its init_script_hashes is
    undocumented.
  • :28-30 tells the reader to use the NitroTPM nonce for challenge-response. The nonce is captured
    (attestation.rs:605, :1849) and never checked and never returned — there is no
    VerificationDetails field for it, so the documented workflow is not achievable.
  • :256 says acpi_tables_verified is false only for "non-TDX platforms"; GCP TDX is TDX and is
    always false.
  • :290-297 lists GCP TDX and Nitro Enclave as the os_image_is_dev null cases; TDX-lite, SNP and
    AWS are null too (AUDIT-BACKLOG V-c; feat(verifier): say which trust anchor os_image_hash_verified came from #1266 fixes this one).

1e. Minor, same shape

AWS NitroTPM's guest extends a MrConfig::V2 commitment into PCR8 with a careful read-back check
(dstack-util/src/system_setup.rs:2946-2961, dstack-attest/src/lib.rs:225-246), and no verifier
reads PCR8 — AWS_NITRO_TPM_BOOT_PCRS = [4,7,12], AWS_NITRO_TPM_EVENT_PCR = 14
(attestation.rs:1273-1275). Its only consumer is the local TPM key-provider seal policy
(tpm-attest/src/lib.rs:38-46). App identity on AWS is soundly bound anyway, by PCR14, so this is an
unenforced belt beside working braces. The cost is the rationale comment at
dstack-util/src/system_setup/config_id_verifier.rs:93-96, which says the pin "is enforced by
verify_key_provider_id" — true only in the local-provider case.


2. The steelman

None of this was decided wrongly in isolation.

  • event_log_verified predates the variants that have no event-log lane. Set after
    decode_app_info_ex succeeds, it reads naturally as "app info decoded without error", and on the
    three variants that replay, that is exactly equivalent.
  • Each tcb_status arm is individually defensible. SEV-SNP genuinely carries no upstream status
    string, so deriving one from the report's TCB quads is the only self-contained option. The AWS
    normalization is documented in-code with its reason (verification.rs:60-62): keep the verifier's
    boot info byte-identical to the KMS bootAuth payload so one shared gate works. Nitro Enclave's
    empty string is the honest arm and it fails closed.
  • The SDKs treat the attestation as opaque on purpose, because verification belongs to the KMS and
    dstack-auth, not to a client library. PR feat(verifier): say which trust anchor os_image_hash_verified came from #1266 argues this explicitly and the argument is good.
  • The ABI omission is a consequence of the contract predating three of the five variants.

The divergence is emergent. That is what makes it worth an issue rather than a patch.

3. What it costs

A relying party reading is_valid && event_log_verified && os_image_hash_verified && tcb_status == "UpToDate" — the obvious reading, and the one the README encourages — gets four different sets of
guarantees depending on a variant it cannot see from any SDK, and cannot distinguish them by reading
the response. On SEV-SNP, event_log_verified is simply untrue while the substance is fine. An
operator who sets requireTcbUpToDate = true gets a real Intel check on two platforms, a
self-consistency check on one, a no-op on one, and a hard deny on one.

4. Improvement direction

Roughly in cost order. (1)–(3) are cheap and break nothing; (5) is a contract upgrade.

  1. Free — documentation. State per variant, in verifier/README.md and
    docs/security/security-model.md, what each result field means and does not mean; correct the
    five inaccuracies in 1d and make security-model.md:188 match the code. Composes with feat(verifier): say which trust anchor os_image_hash_verified came from #1266,
    which already fixes two of them.
  2. Cheap, no wire break — generalize feat(verifier): say which trust anchor os_image_hash_verified came from #1266's construction. One exhaustive match on TeeVariant
    per result field, so a new variant fails the build in every place a guarantee is claimed rather
    than silently inheriting another platform's default. Applies to event_log_verified,
    os_image_hash_verified and tcb_status. This alone would have caught 1a when the SNP and Nitro
    arms were added.
  3. Cheap, additive — say what was checked. A guarantees object in the response naming
    image_anchor, runtime_event_lane, tcb_source and revocation_checked (the last composes
    with Attestation: certificate validity and revocation checks differ per platform, and SEV-SNP has none #1140), so a relying party reads one field instead of inferring from five. Optional field;
    ct_monitor/src/main.rs:56 is the only in-tree consumer and has no deny_unknown_fields.
    Alternatively, replace event_log_verified with a small enum (replayed_rtmr3 |
    replayed_pcr14 | not_applicable_launch_bound | not_applicable_no_lane) and keep the boolean
    as a deprecated alias for one release.
  4. Moderate — surface the variant where integrators actually look. A typed variant field on
    Info and in the four SDKs. Additive proto field; the four-language parity obligation is the real
    cost. Related: Go and JS omit dstack-aws-nitro-tpm from their RequirementPlatform input
    enums (sdk/go/dstack/compose_hash.go:33-40, sdk/js/src/get-compose-hash.ts:39-43) although the
    guest enforcer accepts it.
  5. Deployment event — carry teeVariant to the chain. Add it to both auth-backend schemas and to
    IAppAuth.AppBootInfo. This is the only option that lets an on-chain policy express "UpToDate
    means Intel's TCBInfo, and I do not accept the other three." It is a contract upgrade and should
    be paired with a decision on 1b's SNP and AWS arms, since changing either string in isolation
    turns a passing gate into a failing one for existing deployments.

Explicitly not proposed: making tcb_status uniform by weakening the TDX arm, or making
event_log_verified false on SNP without saying why — on SNP the launch-time HOST_DATA binding is
arguably stronger than a runtime lane, and a bare false would understate it as badly as true
overstates it.

5. Dedup

6. Reachability

  • 1a: any /verify caller; no credential (/verify is unauthenticated); caller-controlled
    frequency. Impact is on what a response asserts, not on what is released.
  • 1b: an operator acting on a reasonable misreading; contract owner; standing condition rather than
    an event.

Appendix — the test for 1a, as run

Inserted in dstack/verifier/src/verification.rs's mod tests, run with
cargo test -p dstack-verifier --lib audit_sev_snp_event_log -- --nocaptureok. It passes on
origin/next
: it is a characterization test recording current behaviour, not a regression test.
Reverted afterwards; nothing is committed.

#[tokio::test]
async fn audit_sev_snp_event_log_verified_is_true_for_a_fabricated_event_log() {
    use ra_tls::attestation::{StackEvidence, VersionedAttestation};
    let request: VerificationRequest =
        serde_json::from_str(include_str!("../fixtures/sev-snp-attestation.json")).unwrap();
    let original = request.attestation.clone().unwrap();
    let mut v1 = VersionedAttestation::from_bytes(&original).unwrap().into_v1();
    let fabricated = vec![
        cc_eventlog::RuntimeEvent::new("app-id".into(), vec![0xff; 20], cc_eventlog::EventLogVersion::V1),
        cc_eventlog::RuntimeEvent::new("compose-hash".into(), vec![0xff; 32], cc_eventlog::EventLogVersion::V1),
        cc_eventlog::RuntimeEvent::new("this-event-never-existed".into(), b"junk".to_vec(), cc_eventlog::EventLogVersion::V1),
    ];
    v1.stack = match v1.stack {
        StackEvidence::Dstack { report_data, config, runtime_events } => {
            println!("original runtime events: {}", runtime_events.len());
            StackEvidence::Dstack { report_data, config, runtime_events: fabricated }
        }
        other => other,
    };
    let tampered = VersionedAttestation::V1 { attestation: v1 }.to_bytes().unwrap();
    let verifier = CvmVerifier::new(
        tempfile::tempdir().unwrap().path().display().to_string(),
        "http://127.0.0.1:9/should-not-download/{OS_IMAGE_HASH}.tar.gz".to_string(),
        Duration::from_secs(1), test_attestation_verifier());
    let response = verifier
        .verify(VerificationRequest { attestation: Some(tampered), ..request.clone() })
        .await.unwrap();
    assert!(response.is_valid, "{:?}", response.reason);
    assert!(response.details.event_log_verified);
    let info = response.details.app_info.unwrap();
    assert_ne!(info.app_id, vec![0xff; 20]); // SNP identity comes from HOST_DATA, not the log
}

A full per-platform parity matrix backing this issue — hardware root, revocation, what commits
firmware/kernel/cmdline/initrd/rootfs, the runtime-event lane, report_data, device_id,
os_image_hash anchoring, TCB, and which fields reach policy, with a file:line or an explicit
"nothing" in every cell — is in .agent/TRUST-CHAIN-platform-parity.md.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions