[rohangavankar]

Summary

Operations with an idempotency token now get one filled in automatically when the caller leaves it unset. The request pipeline fills it with a UUID v4 right after the whole modify_before_serialization interceptor chain, before the request is serialized, so it works for every protocol whether the token goes in the body, a header or the query string. This matches smithy-java, which fills a copy in modifyBeforeSerialization.

The caller's input is never changed: the token goes on a shallow copy, so reusing one input across calls sends a new token each call. Retries resend the same token since the request is only serialized once. A token set by the caller or by an interceptor is sent as is, and an interceptor that returns a new input still gets a token. Interceptors from read_before_serialization on, and logs, can see the generated token. Because the fill only happens in the pipeline, other serializations such as Document.from_shape(input) and types-only packages never get a token. Hand-written inputs that aren't dataclasses with the token field are sent unchanged, with a warning.

The generator is kept private, because a custom generator still can't dedupe across calls. Callers who need that should pass their own token.

This also turns on the idempotency token autofill protocol tests for restJson1, restXml, awsQuery and ec2Query.

Testing

Unit and protocol tests pass. I also generated a Secrets Manager client and ran it against the real service, where the token becomes the secret's version ID:

• An unset token was filled with a UUID v4 that matched the version ID the service returned
• Reusing one input for two calls sent two different tokens
• A caller token was sent unchanged, and repeating the call with it did not create a new version

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[jonathan343]

Thanks @rohangavankar! I think these changes mostly look good.

I think our code generator should detect members with this trait and append additional document to describe the behavior. Botocore adds This field is autopopulated if not provided.. We could add something similar?

[jonathan343]

I know we discussed the tradeoffs offline and decided on the serializer approach. After more thought, I think filling the token in the pipeline before serialization is better.

Both work, but serialize_members doesn't know whether it's building a request. So any serialization creates a token. For example, Document.from_shape(input) gives a new random clientToken each time, and types-only packages would too.

As you're aware, filling in the pipeline also:

• Lets interceptors and logs see the token, so users don't have to parse the request for each protocol.
• Matches smithy-java, which fills a copy in modifyBeforeSerialization.

It just needs to run after the whole modify_before_serialization chain, so an interceptor that rebuilds the input can't drop the token.

It's a two-way door, but cheapest to change before release. Did you look into this any more after our discussion?

[jonathan343]

Nit: I think a few of these could be merged:

• test_unset_idempotency_token_is_generated_once and test_caller_input_is_not_mutated_by_token_fill check the same input and token.
• test_set_idempotency_token_is_sent_unchanged and test_empty_string_idempotency_token_is_sent_unchanged could be one test parametrized on "caller-token" and "".
• test_non_dataclass_input_is_passed_through and test_input_without_the_derived_field_is_passed_through both hit the "can't fill" path, but only the second checks the warning.

[rohangavankar]

Merged all three in 386e3be:

• test_unset_idempotency_token_is_generated_once_on_a_copy now also checks the caller's input isn't mutated.
• test_set_idempotency_token_is_sent_unchanged is parametrized on "caller-token" and "".
• test_input_that_cannot_be_filled_is_passed_through_with_a_warning is parametrized on the non-dataclass input and the misnamed field, and checks the warning for both.