TEE case study: binding AI gateway billing receipts to an attested runtime

[wangyi160]

An LLM gateway sits between a client and a model provider. It receives sensitive inputs, holds upstream access credentials, chooses a route, and calculates charges. Moving that gateway into a confidential VM raises a practical question: what evidence can a client retain after the request finishes?

This case examines two connected mechanisms: attestation of the gateway endpoint before sending a request, and verification of its accounting records afterwards. The model itself runs at an external provider, outside the gateway's TEE.

The implementation discussed here is TeeRouter, a Rust gateway deployed on dstack with Intel TDX. I maintain the project. This write-up was prepared with AI assistance.

The trust boundary

Consider a user disputing a charge. The operator can show a database row containing token counts and a dollar amount, but the user cannot tell whether that row was produced by the expected gateway, changed later, or calculated using different prices.

There are three distinct questions:

• Did the request reach an instance running software that the client approves?
• Are the presented accounting records the bytes that instance committed to?
• Does the calculation follow the recorded usage and pricing rules?

A TEE contributes to the first question. Signatures and commitments address the second. The third requires inspecting the billing rules and their inputs. Treating a valid quote as an answer to all three would overstate what attestation provides.

The host and relay storage are outside the intended confidentiality boundary. The hardware root, accepted TCB, measured guest software, and application remain part of the trust assumptions. The upstream model provider still receives the prompt and must be trusted for its own processing and usage reports.

Binding a connection and a receipt key

The client supplies a fresh challenge. The gateway obtains local TDX evidence binding that challenge to the current TLS certificate. The client checks the certificate binding, Intel collateral and TCB, and measurements against an independently approved policy. It then sends the request over that same verified TLS connection.

This closes a specific gap: verifying evidence from one endpoint and accidentally sending the prompt to another. It does not remove the need to review the software and establish trustworthy reference measurements. Publishing source code makes review possible; it does not automatically establish the identity of a deployed binary.

Accounting uses a separate Ed25519 key generated inside the router process. Hardware evidence binds that public key, a boot identifier, and a challenge. Once a verifier accepts that key's attested identity, ordinary batches can use software signatures without obtaining a new hardware quote for every model call.

Client -- attested TLS --> Gateway in TDX -- HTTPS --> Model provider
                                 |
                           signed usage batch
                                 |
                           external storage
                                 |
                         independent verifier

Following one charge

At request admission, the gateway freezes the applicable price rules and account-pool multiplier. It records the outcome and provider-specific usage categories when available. A later pricing change should not rewrite the quote used for an earlier request.

For a simplified example, assume these illustrative rates, not current prices for any particular model. The ordinary-input count below excludes cached input:

Component	Tokens	USD per million tokens	Charge in USD
Ordinary input	8,000	2.00	0.0160
Cached input	2,000	0.20	0.0004
Output	1,000	8.00	0.0080

The base charge is $0.0244. At a pool multiplier of 0.8, the charge before final rounding is $0.01952. Under this implementation's rule of rounding upward once to a micro-USD, it becomes $0.019520.

The arithmetic is straightforward only after the counters have been interpreted correctly. Providers differ in whether input totals include cached tokens, how cache creation is reported, and how reasoning tokens relate to output totals. Those normalization rules are part of the billing implementation.

Publishing the usage, pricing snapshot, multiplier, and result makes such discrepancies inspectable. It does not prove that the provider reported truthful usage. For interrupted requests, an estimate also needs an explicit provenance label; signing an estimate cannot turn it into an official measurement.

Committing the accounting events

The default publication cycle is five minutes. The router groups bounded sets of records into batches and hashes the exact UTF-8 bytes of the records JSON. A manifest commits to that digest, the record count, sequence range, charge total, signing identity, and previous batch hash. A domain-separated Ed25519 signature covers the manifest.

Changing a record changes the committed digest. Comparing successive batches can reveal a missing predecessor or inconsistent sequence boundary. The present public format uses a full-batch digest, so checking the details requires the complete original records string; it does not provide individual Merkle inclusion proofs.

The public record includes usage and billing metadata, not prompts, responses, tokens used for authentication, or refusal text. User pseudonyms rotate by batch using a private random identity and a public batch nonce. This reduces direct cross-batch linkage, while leaving timing and traffic-pattern correlation possible.

The connection to blockchain is the idea of exposing transactions and accounting rules for independent verification. The primitives themselves predate blockchains. Here, a single attested process produces the log: there is no validator consensus or globally agreed latest head. The same mechanism cannot establish that every event was published or that every observer received the same history.

Recovery is a separate problem

Signed bytes still need durable storage. The optional accounting journal preserves committed events and frozen batches across ordinary crashes. An unsuccessful upload is retained for the next publication cycle, using the same signed bytes.

A storage acknowledgment and a settlement acknowledgment have different meanings. Storage confirms receipt of a batch. Settlement names the batch boundary through which consumption is incorporated into a balance update; newer consumption must remain outstanding. Conflating these two steps can lose charges even when every signature is valid.

The journal cannot recover an event that was never committed, and local authentication cannot by itself detect rollback of the entire stored history. Deposits, withdrawals, and externally supplied balances are also outside the consumption proof's scope.

What a verifier should report

The implementation's separate verifier distinguishes signature verification, record verification, and TEE verification. Its checks include the record digest, count, amount sum, and available predecessor linkage. Recomputing each provider-specific charge is a separate check of the recorded pricing inputs and billing rules; an amount-sum check alone is insufficient.

An accepted batch therefore supports a bounded statement: these are the records committed to by a particular signing key, whose runtime identity can be checked against a trusted policy. It does not establish the model weights used upstream, correctness of the upstream usage report, a complete history, or a universally latest state.

For this design, the remaining distributed-systems question is how independent observers should retain and compare accepted batch heads. That could help detect divergent histories, but it is additional machinery rather than a property obtained merely by placing a signing key in a TEE.

Implementation references: channel and trust boundaries, public batch protocol, and verifier.