Wire request_id through CallbackInfo for linking

[chrsmith]

⚠️ This is to be merged into the feature/worker-callbacks branch, and not main. Only after the feature is complete will that branch be rebased and merged into main.

---

This PR makes three changes, all so that resources spanwed from the invocation of a worker callback can be linked correctly.

(1) Remove the Link_NexusOperationCallback variant with a more general Link_Callback proto

Previously we were scoping the feature to only be applicable for SANO callbacks. But if we are going to support worker callbacks for any async operation, having a general link type (that uses the existing Execution proto) will avoid needing to create additional link variants in the future.

(2) Add a callbackpb.CallbackInfo::request_id field

This type is used in the Describe- operations for standalone Activities and standalone Nexus operations. Without it, there would be no way to determine which completion callback is being referred to. (Instead, we couldn't be any more accurate than to have the link point to "one of these N" callbacks.)

(3) Add workflowpb.CallbackInfo::{request_id, result}

The workflowpb namespace forked rather than embedded the callbackpb.CallbackInfo message. The changes here add the missing fields, so that DescribeWorkflowExecution can disambiguate callbacks as well. (In addition to carrying the result of those callbacks.)

Why?

With these changes, the server will be able to properly cross-link resources spawned from completion callbacks.

On the Caller-side, any resources spawned from the completion callbacks would be available on the commonpb.Callback::links field. (*)

query GetSpawnedResourceLinks(workflowID: string {
  DescribeWorkflowExecution(workflowID) {
    completion_callbacks {
      callback {
        links
      }
    }
  }
}

(*) Only the resources initially created from the worker callback invocation will be present. e.g. the Workflow that backs an asynchronous Nexus handler. It would not contain links for any subsequent resources created.

On the Handler-side, a single Link_Callback would be supplied to the Nexus handler receiving the worker callback. (This would be in the form of a nexuspb.Link.)

Breaking changes

Yes, this PR contains breaking proto changes. However, in the context of a PR into a long-lived feature branch for an unshipped feature this is safe. (The protos haven't ever been persisted by a production service.)

Server PR

It isn't out yet, but will be stacked on top of this:
temporalio/temporal#11589

[VegetarianOrc]

I think we should consider creating a "callback ID" as fully supported thing. The ambiguity where the request ID can be shared across multiple callbacks feels a little awkward.

[chrsmith]

Strong +1 to this. We just need a way to uniquely identify a worker callback within the scope of an execution, and having an entirely server-side generated ID removes any potential confusion.

Moreover, having it be an entirely Temporal-managed CallbackInfo::callback_id and not a user-editable Callback::id simplifies things even more.

[bergundy]

There's no user-editable callback::id. I'd be more than okay using the request ID as the callback ID since it is a unique identifier as long as we document it.

[bergundy]

I would decouple the user provided request ID from the one the system generates. The start request is not the same as the callback delivery request and those should have different IDs. I would be confused if I saw the same request ID provided in a log for different purposes.

[bergundy]

I am concerned that this will be confused with the request ID of the start request used to attach this callback. Document the semantics as we did here:

api/temporal/api/nexus/v1/message.proto

Lines 308 to 311 in 7868510

	// Server-generated request ID used as an idempotency token when submitting start requests to
	// the handler. Distinct from the request_id in StartNexusOperationRequest, which is the
	// caller-side idempotency key for the StartNexusOperation RPC itself.
	string request_id = 21;

[chrsmith]

Thanks for linking to the comment, I agree that's exactly how we'd want to define it.

[bergundy]

There's no user-editable callback::id. I'd be more than okay using the request ID as the callback ID since it is a unique identifier as long as we document it.

[bergundy]

This is not an "execution", it's a component within an execution.

[bergundy]

Suggested change

	EXECUTION_TYPE_NEXUS = 3;
	EXECUTION_TYPE_NEXUS_OPERATION = 3;

[chrsmith]

Addressed PR feedback. However, after the blueprint review we will scope worker callbacks to just SANO operations. So I removed the new fields added to workflowpb.CallbackInfo and avoided adding anything else to Link_Callback (which would have been required to support links to callbacks attached to workflow updates).