Merge latest main into Python 3.11 migration

Preserve upstream payload preflight, entity identity, dispatch recovery, and task hub filtering improvements alongside the Python 3.11 baseline.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e6b07f3f-b3e1-49dd-9121-318968272d31

Author: Bernd Verst

.github/workflows/codeql.yml

@@ -67,7 +67,7 @@ jobs:
 
     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
+      uses: github/codeql-action/init@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
       with:
         languages: ${{ matrix.language }}
         build-mode: ${{ matrix.build-mode }}
@@ -96,6 +96,6 @@ jobs:
         exit 1
 
     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
+      uses: github/codeql-action/analyze@6f2ac81f5cd81fe351a197ba0b00d91d03cd2aee # v4.39.0
       with:
         category: "/language:${{matrix.language}}"

.github/workflows/durabletask-azurefunctions.yml

@@ -78,7 +78,7 @@ jobs:
           python-version: "3.13"
 
       - name: Set up Node.js (needed for Azurite and Functions Core Tools)
-        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
         with:
           node-version: "22.x"
 
@@ -106,7 +106,7 @@ jobs:
       # pytest summary does not surface, so upload it for diagnosis.
       - name: Upload Functions host logs
         if: failure()
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
         with:
           name: func-host-logs-strict-${{ matrix.strict-typing }}
           path: tests/azure-functions-durable/e2e/apps/*/_func_host.log

.github/workflows/durabletask.yml

@@ -46,7 +46,7 @@ jobs:
           python-version: ${{ matrix.python-version }}
 
       - name: Set up Node.js (needed for Azurite)
-        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
         with:
           node-version: '22.x'
 

CHANGELOG.md

@@ -7,6 +7,11 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## Unreleased
 
+ADDED
+
+- Added `OrchestrationQuery.task_hub_names` to filter orchestration instances
+by one or more task hubs.
+
 CHANGED
 
 - Dropped Python 3.10 support. Python 3.11 or later is now required.
@@ -18,9 +23,27 @@ failure is still raised directly.
 
 FIXED
 
-- Orchestrator generators are now closed when a dispatch ends, so cleanup in
-`finally` blocks runs deterministically, including for async bridges that would
-otherwise retain tasks, event loops, and orchestration contexts.
+- Implemented `__hash__` and `__repr__` for `EntityInstanceId`, making identity attributes immutable to guarantee hash stability when used in sets or dict keys.
+- Large-payload externalization now checks every eligible payload's maximum
+size before uploading or replacing any protobuf values, including carryover
+events and entity payloads, for both synchronous and asynchronous calls.
+Oversized messages remain unchanged and cause no uploads; their size error
+now takes precedence over hypothetical earlier storage failures.
+- Orchestrations that exceed the configured external payload storage limit now
+fail without retrying and release their entity locks, allowing other callers to
+progress. Transient payload-store failures remain retryable.
+- Previously unhandled non-cancellation `BaseException` exits, including result
+serialization and generator teardown failures, now fail with a clear reason instead of leaving an accepted dispatch
+unfinished. Cancellation and process-control signals retain their semantics.
+The gRPC worker abandons work that exits before an outcome is sent and resets
+the originating work-item stream after an uncertain completion or abandonment
+RPC, allowing scheduler ownership recovery without retrying an uncertain outcome.
+- Orchestrator generators are now closed at the end of each dispatch, including
+while waiting for durable work, releasing resources held by async bridges.
+`finally` blocks and context-manager exits can run before orchestration completion
+and repeat across dispatches; see [cleanup semantics](docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks).
+- Async cancellation raised by generator teardown no longer replaces dispatch
+responses or version-rejection abandonment.
 
 ## v1.12.0
 

azure-functions-durable/CHANGELOG.md

@@ -27,9 +27,18 @@ directly.
 
 FIXED
 
+- With the corresponding core `durabletask` SDK fix, oversized messages fail
+large-payload validation before any uploads or payload replacements. Size
+errors now take precedence over hypothetical earlier storage failures.
+- With the corresponding core `durabletask` SDK fix, previously unhandled non-cancellation
+`BaseException` exits from orchestration execution, result serialization, or teardown now produce a failed response
+with the exception's type, message, and stack trace instead of losing the response.
 - With the corresponding core `durabletask` SDK fix, orchestrator generators
-are closed when a dispatch ends, so cleanup in `finally` blocks runs
-deterministically, including for async bridges.
+are closed at the end of each dispatch, including while waiting for durable work.
+See [cleanup semantics](../docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks)
+for repeated finalizers, discarded teardown actions/status, and yielding restrictions.
+- With the corresponding core `durabletask` SDK fix, async cancellation raised by
+generator teardown no longer replaces dispatch responses or version-rejection abandonment.
 
 ## v2.0.0rc3
 

docs/supported-patterns.md

@@ -22,6 +22,74 @@ def sequence(ctx: task.OrchestrationContext, _):
 
 See the full [function chaining example](../examples/activity_sequence.py).
 
+## Cleanup in orchestrator finally blocks
+
+The SDK closes an orchestrator generator at the end of every dispatch, including
+when it is waiting for durable work. Closing injects `GeneratorExit`, running
+active `finally` blocks and context-manager exits. Replay creates a fresh
+generator, so this cleanup may run many times before the orchestration completes.
+
+Use these hooks to release dispatch-local resources, not as orchestration-completion
+or exactly-once compensation hooks. Do not release resources that an activity
+still needs: that activity may still be running when the dispatch ends.
+
+The response is built before this forced teardown. Durable actions, timer
+cancellation, custom-status changes, and `continue_as_new()` issued only during
+teardown do not change that response. Keep workflow-level cleanup in normal
+orchestration control flow, as a yielded activity:
+
+```python
+from io import StringIO
+from typing import Any, Generator
+from durabletask import task
+
+
+def orchestrator(
+    ctx: task.OrchestrationContext, value: str,
+) -> Generator[task.Task[Any], Any, Any]:
+    scratch = StringIO(value)
+    try:
+        result = yield ctx.call_activity("process", input=scratch.getvalue())
+    finally:
+        scratch.close()  # Local buffer cleanup runs at each dispatch boundary.
+
+    yield ctx.call_activity("cleanup_workflow", input=value)
+    return result
+```
+
+`ctx.is_replaying` is normally false after new events are processed. Neither
+`not ctx.is_replaying` nor replay-safe logging makes teardown completion-only.
+Yielding during forced close raises `RuntimeError: generator ignored GeneratorExit`;
+the SDK logs a warning, but the generator remains open and may retain resources.
+A `finally` reached through normal execution, rather than forced close, can
+still yield durable work.
+
+Garbage collection could already trigger finalization; completion-only timing
+was never guaranteed. Explicit dispatch teardown does not guarantee cleanup
+across process failure.
+
+Unexpected non-cancellation `BaseException` exits from orchestration execution,
+result serialization, or forced generator teardown fail the orchestration with the exception's type,
+message, and stack trace. An existing orchestration failure is retained if
+teardown also fails. Outstanding acquired entity locks are released with the
+failed response; durable work and status changes made only by teardown are not
+committed.
+
+The gRPC worker does not turn `asyncio.CancelledError`, `GeneratorExit`,
+`SystemExit`, or `KeyboardInterrupt` into orchestration failures. This also applies
+to nested exception groups containing these signals on Python 3.11 and later.
+The existing warning-only handling of direct `asyncio.CancelledError` during
+forced teardown is unchanged. Other cancellation or process-control exits
+propagate after the worker attempts to abandon the accepted dispatch.
+
+If a completion or abandonment RPC raises, its server-side outcome may be
+unknown. The worker logs the uncertainty and closes the dispatch's originating
+work-item stream so the scheduler can recover ownership. It does not retry an
+uncertain RPC or send a conflicting completion or abandonment. Other work owned
+by that stream can also be redelivered. This is not an exactly-once execution
+guarantee or a bounded recovery-time guarantee. Cancelling an asyncio await of a
+still-running executor thread does not abandon that thread's dispatch.
+
 ## Fan-out/fan-in
 
 An orchestration can fan-out a dynamic number of function calls in parallel and then fan-in the

durabletask-azuremanaged/CHANGELOG.md

@@ -18,9 +18,23 @@ directly.
 
 FIXED
 
+- With the corresponding core `durabletask` SDK fix, oversized messages fail
+large-payload validation before any uploads or payload replacements. Size
+errors now take precedence over hypothetical earlier storage failures.
+- With the corresponding core `durabletask` SDK fix, orchestrations that exceed
+the configured external payload storage limit fail without retrying and release
+their entity locks. Transient payload-store failures remain retryable.
+- With the corresponding core `durabletask` SDK fix, previously unhandled
+non-cancellation `BaseException` exits, including result serialization failures,
+now fail with a clear reason. Cancellation and process-control
+exits recover dispatch ownership, and uncertain completion or abandonment RPCs
+reset the originating work-item stream rather than retaining unfinished work.
 - With the corresponding core `durabletask` SDK fix, orchestrator generators
-are closed when a dispatch ends, so cleanup in `finally` blocks runs
-deterministically, including for async bridges.
+are closed at the end of each dispatch, including while waiting for durable work.
+See [cleanup semantics](../docs/supported-patterns.md#cleanup-in-orchestrator-finally-blocks)
+for repeated finalizers, discarded teardown actions/status, and yielding restrictions.
+- With the corresponding core `durabletask` SDK fix, async cancellation raised by
+generator teardown no longer replaces dispatch responses or version-rejection abandonment.
 
 ## v1.12.0
 

durabletask/client.py

@@ -197,6 +197,7 @@ class OrchestrationQuery:
     max_instance_count: int | None = (1 << 31) - 1
     fetch_inputs_and_outputs: bool = False
     instance_id_prefix: str | None = None
+    task_hub_names: list[str] | None = None
 
 
 @dataclass

durabletask/entities/entity_instance_id.py

@@ -1,12 +1,42 @@
 # Copyright (c) Microsoft Corporation.
 # Licensed under the MIT License.
 
+
 class EntityInstanceId:
+    __slots__ = ("_entity", "_key")
+
     def __init__(self, entity: str, key: str):
         EntityInstanceId.validate_entity_name(entity)
         EntityInstanceId.validate_key(key)
-        self.entity = entity.lower()
-        self.key = key
+        object.__setattr__(self, "_entity", entity.lower())
+        object.__setattr__(self, "_key", key)
+
+    @property
+    def entity(self) -> str:
+        return self._entity
+
+    @property
+    def key(self) -> str:
+        return self._key
+
+    def __setattr__(self, name: str, value: object) -> None:
+        raise AttributeError(
+            f"'{type(self).__name__}' object does not support attribute assignment"
+        )
+
+    def __delattr__(self, name: str) -> None:
+        raise AttributeError(
+            f"'{type(self).__name__}' object does not support attribute deletion"
+        )
+
+    def __copy__(self) -> "EntityInstanceId":
+        return self
+
+    def __deepcopy__(self, memo: object) -> "EntityInstanceId":
+        return self
+
+    def __reduce__(self) -> tuple[type["EntityInstanceId"], tuple[str, str]]:
+        return (type(self), (self._entity, self._key))
 
     def __str__(self) -> str:
         return f"@{self.entity}@{self.key}"
@@ -21,6 +51,12 @@ def __lt__(self, other: object) -> bool:
             return NotImplemented
         return str(self) < str(other)
 
+    def __hash__(self) -> int:
+        return hash((self.entity, self.key))
+
+    def __repr__(self) -> str:
+        return f"EntityInstanceId(entity={self.entity!r}, key={self.key!r})"
+
     @staticmethod
     def parse(entity_id: str) -> "EntityInstanceId":
         """Parse a string representation of an entity ID into an EntityInstanceId object.

durabletask/internal/client_helpers.py

@@ -103,6 +103,7 @@ def build_query_instances_req(
             runtimeStatus=[status.value for status in orchestration_query.runtime_status] if orchestration_query.runtime_status else None,
             createdTimeFrom=helpers.new_timestamp(orchestration_query.created_time_from) if orchestration_query.created_time_from else None,
             createdTimeTo=helpers.new_timestamp(orchestration_query.created_time_to) if orchestration_query.created_time_to else None,
+            taskHubNames=[wrappers_pb2.StringValue(value=name) for name in orchestration_query.task_hub_names] if orchestration_query.task_hub_names else None,
             instanceIdPrefix=helpers.get_string_value(orchestration_query.instance_id_prefix),
             maxInstanceCount=orchestration_query.max_instance_count,
             fetchInputsAndOutputs=orchestration_query.fetch_inputs_and_outputs,