Skip to content

Add Functions gRPC core helpers#282

Open
YunchuWang wants to merge 31 commits into
mainfrom
yunchuwang/functions-grpc-support
Open

Add Functions gRPC core helpers#282
YunchuWang wants to merge 31 commits into
mainfrom
yunchuwang/functions-grpc-support

Conversation

@YunchuWang

@YunchuWang YunchuWang commented Jun 29, 2026

Copy link
Copy Markdown
Member

What

Adds the minimal host-integration surface so Azure Functions can drive a single durable work item per invocation, without running the long-lived gRPC worker loop. This mirrors the Azure Functions consolidation done for Python in durabletask-python PR #155.

API

Two methods on TaskHubGrpcWorker:

  • processOrchestratorRequest(request: Uint8Array): Promise<Uint8Array>
  • processEntityBatchRequest(request: Uint8Array): Promise<Uint8Array>

Each deserializes a TaskHubSidecarService protobuf payload, executes one work item, and returns the serialized response. Host integrations own any transport encoding (for example base64); base64 stays out of the core SDK.

Design (aligned with Python #155)

Rather than refactoring the worker, these methods reuse the worker's existing internal execution path (_executeOrchestratorInternal / _executeEntityInternal) and pass an in-process CapturingSidecarStub that records the completion payload instead of sending it over gRPC. This is the same null-stub pattern Python uses (AzureFunctionsNullStub).

Deliberately NOT included (kept minimal)

  • No base64 protobuf helper module (transport detail belongs in the host).
  • No object-level execute* public methods (the byte processors are the single entry point).
  • No V2 EntityRequest host path (not present in Python fix: Reset customStatus on continue-as-new in InMemoryOrchestrationBackend #155; the worker's internal V2 handling for the DTS backend is unchanged).
  • No new client options; task-hub routing metadata continues to flow through the existing metadataGenerator.

Tests

packages/durabletask-js/test/functions-grpc-support.spec.ts covers both byte processors end to end.

YunchuWang and others added 4 commits June 29, 2026 15:30
Expose low-level protobuf codecs and single work-item execution helpers for Azure Functions Durable JS gRPC consolidation without adding Functions metadata support.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Expose stable byte-oriented worker processing methods and endpoint/taskHub client options for Azure Functions Durable JS gRPC consolidation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the work-item-executor extraction and base64 protobuf helpers with a
minimal, Python-aligned implementation. processOrchestratorRequest and
processEntityBatchRequest now reuse the existing worker execution path via an
in-process capturing stub (mirroring durabletask-python PR #155's null-stub
pattern) instead of refactoring the worker. Removed the base64 helper module,
the object-level execute* methods, the V2 EntityRequest host path, and the
related exports and tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@YunchuWang YunchuWang marked this pull request as ready for review June 30, 2026 22:02
Copilot AI review requested due to automatic review settings June 30, 2026 22:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a minimal “single work-item” execution surface to the Durable Task JS worker so host integrations (e.g., Azure Functions) can execute one orchestrator/entity batch request per invocation without running the long-lived gRPC streaming loop. This aligns with the SDK’s role as a low-level Durable Task Scheduler (sidecar) client/worker implementation.

Changes:

  • Added byte-level helper methods on TaskHubGrpcWorker to process serialized OrchestratorRequest / EntityBatchRequest payloads and return serialized responses.
  • Introduced an in-process CapturingSidecarStub that captures completion payloads instead of sending them over gRPC.
  • Added end-to-end tests and README documentation for the new host-integration surface.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
README.md Documents the new low-level host integration APIs and clarifies the non–Durable Functions programming model scope.
packages/durabletask-js/test/functions-grpc-support.spec.ts Adds e2e-style tests covering the new byte processors for orchestrations and entities.
packages/durabletask-js/src/worker/task-hub-grpc-worker.ts Implements the new public byte-processing methods and the capturing sidecar stub.

Comment thread packages/durabletask-js/src/worker/task-hub-grpc-worker.ts
Comment thread packages/durabletask-js/src/worker/task-hub-grpc-worker.ts
Comment thread packages/durabletask-js/src/worker/task-hub-grpc-worker.ts
YunchuWang and others added 5 commits July 1, 2026 15:34
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ator

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…t glue

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ease pipeline

- Add DurableOrchestrationContext/DurableEntityContext (context.df.* adapters) + wrapOrchestrator/wrapEntity
- Add RetryOptions, callHttp (throws), parentInstanceId; align newGuid/callSubOrchestrator with v3 signatures
- Add v3 client query-return types (DurableOrchestrationStatus, OrchestrationRuntimeStatus, EntityStateResponse, PurgeHistoryResult)
- Add client getStatus/getStatusAll/getStatusBy/readEntityState/purgeInstanceHistory/startNew/waitForCompletionOrCreateCheckStatusResponse
- Add deprecated DurableOrchestrationClient alias
- Wire host-provided maxGrpcMessageSizeInBytes into gRPC channel options (Python parity)
- Remove dead CapturingSidecarStub.abandonRequest field (keep required no-op method)
- Add durable-functions package to build/release pipelines
Comment thread packages/azure-functions-durable/CHANGELOG.md Outdated
Comment thread packages/azure-functions-durable/README.md
"dependencies": {
"@azure/functions": "^4.16.1",
"@grpc/grpc-js": "^1.14.4",
"@microsoft/durabletask-js": "0.3.0"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: remember to bump this to 0.4.0, and we need to release @microsoft/durabletask-js first

also confirm whether @azure/functions 4.16.1 has the extension changes required,

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in f935bce

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Release-readiness findings (version bump to 0.4.0 is done in code; the two open items below are release-time actions):

1) @azure/functions 4.16.1 floor — confirmed adequate.

  • v4.16.0 carried the substantive change: bumped @azure/function-extensions-base 0.2.0 → 0.3.0 and dropped Node 18 from CI (which lines up with this package's Node >= 22 floor).
  • v4.16.1 is a security-servicing release over 4.16.0 (MSRC updates); v4.16.2 is functionally identical to 4.16.1 (only reclaimed the npm latest tag from an erroneously published 3.6.0).
  • This package uses only stable generic-binding APIs (trigger.generic / input.generic carrying durableRequiresGrpc), available since v4.0, and does not depend on @azure/function-extensions-base directly. So ^4.16.1 is a sound "recent + security-patched" floor (it resolves to 4.16.2 today). The actual durable-gRPC enablement (durableRequiresGrpc → gRPC channel) is host-extension-side, not gated by the npm library version.

2) Core release ordering — a real blocker to verify before publish.

  • The core package (@microsoft/durabletask-js) on this branch still declares version: 0.3.0, but it adds new public APIs this package relies on (processOrchestratorRequest, addNamedOrchestrator, CapturingSidecarStub, …). The published 0.3.0 on npm does NOT contain those.
  • This package's dependency is pinned to "@microsoft/durabletask-js": "0.3.0". Locally the monorepo workspace links the in-tree core (which has the new APIs), so E2E passes; but once published to npm, 0.3.0 would resolve to the old published core and fail at runtime (processOrchestratorRequest is not a function).
  • Release checklist: (a) bump the core version (e.g. 0.4.0) and publish @microsoft/durabletask-js first; (b) update this package's dep from 0.3.0 to the new core version (0.4.0 / ^0.4.0); (c) then publish durable-functions@0.4.0.

Leaving this thread open until the core is released and the dependency is bumped.

YunchuWang and others added 7 commits July 7, 2026 16:25
Drop rootDir, outDir, include, and exclude since they are inherited from ./tsconfig.json. Keep only the baseUrl + paths override that points @microsoft/durabletask-js imports at the built dist output.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… Functions

- Introduced `EntityId` class for classic v3 entity identifiers.
- Enhanced `DurableEntityContext` to include `isNewlyConstructed` and `entityId` properties.
- Implemented `signalEntity` method in `DurableEntityContext` for signaling other entities.
- Updated `DurableOrchestrationContext` to track and set custom status.
- Added unit tests for new features and behaviors.
Collapse the 4.0.0-alpha.0 changelog to a single bullet and rewrite the README to focus on what the package supports and why it is needed, dropping the implementation-plan / phase-status / open-questions sections.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
YunchuWang and others added 4 commits July 8, 2026 15:42
…onContext type aliases

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rator

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… context.log to classic orchestration context

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…3-compatible)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…InstanceHistoryBy (v3-compat)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

@andystaples andystaples left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Two things ship here: (1) the advertised core gRPC byte-processors (processOrchestratorRequest / processEntityBatchRequest + CapturingSidecarStub) in task-hub-grpc-worker.ts, and (2) an entire new durable-functions@4.0.0-alpha.0 package (packages/azure-functions-durable) — the intended next-major replacement for Azure/azure-functions-durable-js v3.

The core helpers are clean, and reusing the existing execution path with an in-process capturing stub is a nice low-risk approach. The v3 compat adapters (context.df.* for orchestrations/entities and the client method surface) are genuinely broad and well documented.

My main concern is the "drop-in / run unchanged" framing in the README: this is a strong compatibility shim for the common method surface, but several heavily-used v3 patterns and the entire durable-client authoring model are not covered — and there's one silent-failure bug in the orchestrator adapter. Details are inline; grouped here:

Bug (please fix)

  • wrapOrchestrator picks classic-vs-native by parameter arity, which silently mis-runs a native single-arg orchestrator (async function*(ctx)) — the body never executes and no error is thrown.

Behavioral breaks vs v3 (worth closing or documenting)

  • createTimer returns a non-cancelable Task; the Task.any + timeoutTask.cancel() timeout pattern breaks, and the Task result shape changed (isCompleted/isFaulted/resultisComplete/isFailed/getResult()).
  • Orchestration/entity contexts no longer extend InvocationContext.
  • Missing top-level exports: DummyOrchestrationContext/DummyEntityContext (breaks users' tests), error types (DurableError/AggregatedError/TaskFailedError) for instanceof, ManagedIdentityTokenSource/TokenSource; the app.client.* durable-client trigger helpers and setExceptionPropertiesProvider are also gone.
  • Client deltas: getStatus returns | undefined and ignores showHistory; createCheckStatusResponse now requires a defined request; startNew drops the version option.

Nits

  • parseJson doesn't guard "" (JSON.parse("") throws).
  • Core: a version-mismatch abandon makes processOrchestratorRequest throw a generic error.
  • Spurious deserializeBinary reorder in the generated proto .d.ts.
  • engines.node >= 22 vs @types/node@^18; the Node-22 floor is a notable consumer bump.

Overall a promising foundation — I'd fix the arity bug and soften/annotate the compatibility claims before this reads as a transparent v3 replacement. Leaving this as comments (non-blocking).

Comment thread packages/azure-functions-durable/src/orchestration-context.ts Outdated
Comment thread packages/azure-functions-durable/src/orchestration-context.ts Outdated
Comment thread packages/azure-functions-durable/src/orchestration-context.ts
Comment thread packages/azure-functions-durable/src/client.ts Outdated
Comment thread packages/azure-functions-durable/src/client.ts Outdated
Comment thread packages/azure-functions-durable/src/orchestration-status.ts Outdated
Comment thread packages/azure-functions-durable/src/index.ts
Comment thread packages/durabletask-js/src/worker/task-hub-grpc-worker.ts
Comment thread packages/durabletask-js/src/proto/orchestrator_service_pb.d.ts
Comment thread packages/azure-functions-durable/package.json
… empty-string parseJson

Two self-contained fixes in the durable-functions v4 compat package:

- rewind(): delegate to core TaskHubGrpcClient.rewindInstance() instead of throwing 'rewind is not yet supported'; the optional reason defaults to an empty string.

- parseJson(): treat empty-string serialized input/output/customStatus as undefined so getStatusAll/getStatusBy no longer throw 'Unexpected end of JSON input'.

Adds one regression test per fix (durable-functions unit suite: 67 tests, all green).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
YunchuWang and others added 7 commits July 14, 2026 13:44
Brings the PR branch up to date with main so the compat layer can build against the current core public API: cancellable TimerTask (#293), native AggregateError WhenAll semantics (#302), rewind support (#296), and related fixes. Clean auto-merge (proto + worker).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolves andystaples' review comments plus a ListAll E2E bug on
packages/azure-functions-durable:

- #1 orchestrator arity: detect generator kind (sync=classic v3, async=core
  native) instead of an arity heuristic; driver now handles sync+async
  generators so single-param native orchestrators actually execute.
- #2 parseJson tolerates non-JSON serializedOutput (fixes getStatusAll 400).
- #3 createCheckStatusResponse accepts undefined request (baseUrl fallback).
- #4 createTimer returns TimerTask (cancelable) in its type signature.
- #5 re-export TaskFailedError from core + compat; document removed v3-only
  exports in CHANGELOG/README.
- #6/#7 document getStatus + classic-context breaking changes.
- #8 bump @types/node to ^22 to match engines.node>=22.
- #9 set version 0.4.0 + document release order.
- #10 distinct abandon error for the single work-item execution path.
- #11 revert proto codegen churn to origin/main.

Adds regression tests for items #1/#2/#3/#5/#10.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…, startNew version

- (1) orchestration-context: detect classic vs core-native orchestrators by generator/async kind (async* and async => native; sync generator => classic; plain sync => arity fallback) and drive both sync/async generators; add end-to-end regression tests through the real core executor.

- (2) createTimer already returns cancelable TimerTask (type now visible); add type/Task.any cancel test.

- (3) doc-only: classic contexts no longer extend InvocationContext (entity ctx is {df}); add replay-nondeterminism rationale to CHANGELOG/README.

- (4) createCheckStatusResponse already accepts undefined request with baseUrl fallback (v3-faithful).

- (5) getStatus returns non-optional DurableOrchestrationStatus and throws on not-found (v3 parity); showHistory populates core history; showInput maps to fetchPayloads; startNew forwards v3 version option. Docs note gRPC-path boundaries.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- getStatus throws on not-found with a v3-style 'DurableClient error:' message (verified v3 DurableClient.getStatus throws on the extension's HTTP 404; returns non-optional DurableOrchestrationStatus).

- showInput now gates only the top-level input (payloads always fetched); output/customStatus are always returned, matching v3.

- showHistoryOutput now strips input/result payloads from history entries when falsy and keeps them when true. history stays core HistoryEvent[] (v3 types history as Array<unknown>).

- startNew forwards the v3 version option (already wired to core scheduleNewOrchestration).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
v3 DurableClient.getStatus throws on the extension's HTTP 404 with 'This usually means we could not find any data associated with the instanceId provided: <id>.'. Mirror that sentence verbatim (only the HTTP-404-specific first sentence is replaced, since the gRPC path has no HTTP 404). Update the not-found tests to assert the v3 message.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The single-work-item abandon path is JS-specific for the Azure Functions host integration. durabletask-python has no equivalent single-work-item helper, and its worker-loop abandon hands the item back over a real sidecar stub rather than no-opping, so the prior 'matches the Python provider, whose null stub no-ops' note was inaccurate. Comment-only change.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The prior note referenced durabletask-python behavior, which is fragile: the Python single-work-item support lives on an unmerged fork PR and its details can drift. Trim the comment to the JS-only rationale for the no-op abandon on the single-work-item host path. Comment-only change.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
YunchuWang and others added 2 commits July 15, 2026 14:05
…pers for v3 parity

Restores the 10 client-starter registration helpers (http, timer, storageBlob, storageQueue, serviceBusQueue, serviceBusTopic, eventHub, eventGrid, cosmosDB, generic). Each adds the durableClient input binding and injects a DurableFunctionsClient as the handler's 2nd arg, so (trigger, client, context) handlers work without manual df.input.durableClient() + df.getClient(context).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Surface the classic v3 client.startNew alias alongside the canonical scheduleNewOrchestration in the README examples, and add a verified alias -> core method table. Docs-only; these aliases already ship in client.ts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants