fix(vertex): serialize the body static-fields-first so prompt caching survives routing

[nkato]

Problem

On Google Cloud's Agent Platform (Vertex AI), prompt caching misses during the first turns of a conversation even though the cached prefix never changes.

A request is served by one of several independent cache stores, and which store is chosen depends on a hash of the head of the request body. The body serializes messages second, so while a conversation is still short the growing history sits inside the hashed region. Each request of that conversation hashes differently, can reach a store that has not seen the prefix, and writes the prefix again instead of reading it.

Reproduction

Requires anthropic[vertex] and application default credentials.

import uuid

import anthropic

client = anthropic.AnthropicVertex(project_id="YOUR_PROJECT", region="us")

# The uuid only makes the cached prefix unique per run, so every run starts cold.
system = [
    {
        "type": "text",
        "text": f"{uuid.uuid4()} " + "You are a helpful assistant. " * 700,
        "cache_control": {"type": "ephemeral"},
    }
]

messages = []
for i in range(6):
    messages.append({"role": "user", "content": f"Question {i}. Answer in one word."})
    usage = client.messages.create(
        model="claude-sonnet-5", max_tokens=16, system=system, messages=messages
    ).usage
    print(f"request {i}: cache_read={usage.cache_read_input_tokens} cache_creation={usage.cache_creation_input_tokens}")
    messages.append({"role": "assistant", "content": "Answer."})

The system prefix is 7k tokens and never changes, so requests 1 to 5 should all read it. On main at 1.5.0, three runs missed on 2, 1 and 2 of those 5 requests. The first run:

request 0: cache_read=0    cache_creation=7032
request 1: cache_read=0    cache_creation=7032
request 2: cache_read=0    cache_creation=7032
request 3: cache_read=7032 cache_creation=0
request 4: cache_read=7032 cache_creation=0
request 5: cache_read=7032 cache_creation=0

With this branch, three runs of the same script missed on none of them. The first run:

request 0: cache_read=0    cache_creation=7029
request 1: cache_read=7029 cache_creation=0
request 2: cache_read=7029 cache_creation=0
request 3: cache_read=7029 cache_creation=0
request 4: cache_read=7029 cache_creation=0
request 5: cache_read=7029 cache_creation=0

The behaviour is the same on the us multi-region endpoint and on global.

Measurements

The hashed region is about 16 KB. Recording how many leading bytes each request shared with the previous one, requests that shared 9 KB missed and requests that shared 19 KB or more hit, on every step of two chains. Serializing system first but keeping it at 12 KB still missed, and 15 KB or larger did not.

Keeping the current key order and instead putting a 20 KB block of unchanging text at the start of the first user message also removes the misses. What matters is which bytes are at the front of the body, not which field they belong to.

On one failing chain a request read back the prefix that had been written three requests earlier rather than the most recent one, so the conversation was spread over more than one cache store.

Change

_prepare_options reorders the assembled body so anthropic_version, system and tools come first and messages comes last. JSON object key order does not change the meaning of the request.

The added tests assert the serialized key order for the sync and the async Vertex client, and fail without the change.

This reduces cold prefixes rather than removing them, since the assignment of a request to a store can still change on the platform side.