Skip to content

Show how to run SDK workers on Kubernetes #336

Closed

Description

@rmcdaniel

Current state: delivered in all8 live guides. PR393 merge2a07fcb9c9ce273b6f4d7e383ac44bc5bed4a23e; exact docsCI39968613613/Kubernetes39968613525/artifact39972508095/actualPages39975647870 PASS. All8 browser/copy/metadata/rollout checks,64 exact source paragraphs and en/ja/zh mobile themes pass. Actual qualification is3 completed workflows/activities,6 Pod registrations,6 clean idle SIGTERM+deregistrations. Earlier failures retained; active interruption/HA/production and tuple promotion are not claimed.

Problem

The deployment guide explains how to run the Server on Kubernetes with the Helm chart or the raw manifests. It does not show how to run your own SDK workers there. The guide says only that workers should use the load-balanced API endpoint. The chart's upgrade notes assume external workers can be drained, but no page shows how to deploy them.

Users running PHP, Python or Rust workers on Kubernetes have to work out for themselves:

  • which Server URL to use: the in-cluster Service name when workers run in the same cluster, or the ingress or load balancer URL with TLS when they run elsewhere;
  • how to pass the worker token from a Secret;
  • how to stop workers safely. The Python worker drains accepted work only when the process wires SIGTERM to Worker.stop(), and the pod's termination grace period must be longer than the worker's shutdown budget.

Acceptance criteria

  • The deployment guide has a "Running SDK workers on Kubernetes" section with a minimal worker Deployment. It shows the Server URL for in-cluster and out-of-cluster workers, the token Secret, and resource and replica settings.
  • The section explains SIGTERM handling and the termination grace period for each SDK, and links the rollout guide for moving workers between builds.
  • The example manifest is checked against a published Server image and a published SDK worker, with each worker registering and completing a workflow.

Activity

  1. added
    documentationImprovements or additions to documentation
    priority:P3Lower-priority product improvement
    on Oct 20, 2026
  2. rmcdaniel commented on Oct 20, 2026

    @rmcdaniel
    MemberAuthor

    Accepted as documentation and deployment acceptance for existing worker behavior. The first step is to verify the published PHP, Python and Rust shutdown contracts; a worker that cannot drain safely would require a product fix rather than a manifest that conceals the gap.

    The guide will include a minimal Deployment, in-cluster Service and external TLS URLs, Secret-based tokens, explicit resource/replica settings, per-SDK SIGTERM wiring and a termination grace period longer than the configured drain budget. Build changes will link the existing rollout guidance. The example must register and finish a genuine workflow using published artifacts before delivery, and the explanation must ship in English and all seven translated locales.

    This is queued after the current release/installed-consumer work. No worker controller, new chart or autoscaler is being introduced by this docs issue; queue scale-from-zero and throttling signals remain server#534.

  3. rmcdaniel commented on Oct 23, 2026

    @rmcdaniel
    MemberAuthor

    The published SDK source review confirms that Pod termination needs explicit application wiring, with different lifecycle limits:

    • PHP's managed loop installs SIGTERM only with pcntl and requests shutdown before managed cleanup. A custom tick loop does not provide that lifecycle. Published PHP3.4.1 Worker.
    • Python must await stop and the run task. Its shared drain deadline does not make a running synchronous thread forcibly cancellable. Published Python2.9 lifecycle.
    • Rust needs a Unix SIGTERM future supplied to run_until and its returned result awaited. Ctrl-C wiring alone does not handle Kubernetes termination. Published Rust4.7 Worker.

    Draft #393 adds the minimal Deployment and stages an actual published-artifact Kubernetes qualification. All seven translations/source hashes and the runtime controls are still pending. The guide must distinguish Pod availability, SDK registration, completed work and successful cleanup. No cluster, published package or deployment pass is being claimed from source review alone; the selected compatibility tuple remains unchanged.

  4. rmcdaniel commented on Oct 23, 2026

    @rmcdaniel
    MemberAuthor

    The documented worker Deployment passed actual published-artifact Kubernetes qualification at exact Website head 4faac223522656d9c6eb69a641214e41ba26ae8c. Bounded evidence artifact retains the normal install locks/reports, manifests, logs, synthetic durable-state summary and receipt for three days.

    The extracted English YAML has SHA256 91f7f9bb44e60b5d50134453933137db475e853e488079d6fd785c6f4b76ad87. All eight guides use identical executable blocks and seven reviewed-source hashes. The fixture installs ordinary published PHP 3.4.1, Python 2.9.0 and Rust 4.7.0 packages, checks their package identities, and runs against immutable published PHP Server 2.11.0 image sha256:b29157eb2cf844916debbbb3c01b3257672e0dece778a9c6fa539043a3038f02. This is a focused qualification cohort, not a change to the site's selected tuple.

    Observed controls: six distinct Pod-derived worker IDs registered; one original workflow and remote activity per SDK completed with the correct SDK/name and a worker ID from that SDK's roster. Scaling each two-Pod Deployment to zero delivered SIGTERM; all six logs reached successful managed worker_stopped, Pods disappeared, and the normal Server API confirmed all six original registrations were gone. Observed idle shutdown durations were PHP1.25s, Python1.01s and Rust1.97s, within the example's 75-second grace. These observations are not universal shutdown bounds.

    Earlier fixture failures remain failed observations: omitted required PHP ActivityContext, fresh SQLite volume ownership, and an unsuitable process-local array cache were corrected in the fixture. Run39965841032 subsequently timed out waiting45s for its PHP workflow; the current successful run added read-only diagnostics but did not change the application runtime, so no cause or product fix is inferred for that timeout. The successful receipt is the delivery control; earlier failures have not been relabeled as passes.

    The fixture uses ephemeral SQLite and an actual scheduler on an isolated kind1.35.0 cluster. It checks idle termination only. Active-task interruption, forced kill, recovery, production databases/networking and HA remain separate qualification. Exact all-locale documentation CI39968613613, protocol/boundary/CodeQL and Sonar checks pass; the reviewed Sonar summary reports zero new issues, accepted issues or security hotspots. Normal merge, actual Pages publication and eight-locale live checks remain before Website delivery.

  5. rmcdaniel commented on Oct 23, 2026

    @rmcdaniel
    MemberAuthor

    Delivered in English and all seven translations: Running SDK workers on Kubernetes.

    The guide covers actual in-cluster Service names and external HTTPS URLs, Secret-based worker credentials, application-owned environment wiring, Pod-derived worker IDs, resource/replica examples, apply/rollout commands and the existing build rollout guide. PHP managed-loop/pcntl, Python stop-and-await and Rust Unix-SIGTERM/run_until behavior are distinguished. The grace period must include polling, accepted work and cleanup; Pod availability alone does not prove SDK registration or successful execution.

    Delivery evidence:

    • PR docs: run SDK workers on Kubernetes with explicit shutdown #393 merged normally at 2a07fcb9c9ce273b6f4d7e383ac44bc5bed4a23e. Exact all-locale CI, protocol/boundary/CodeQL and Sonar pass; the reviewed Sonar summary has zero new issues, accepted issues or hotspots.
    • Actual published-artifact Kubernetes qualification passes. Its full qualification receipt records normal PHP3.4.1/Python2.9.0/Rust4.7.0 installations against immutable PHP Server2.11.0: six registered Pod IDs, three original completed workflows and remote activities, six successful idle SIGTERM shutdowns, and confirmed removal of all six original registrations. Earlier failed fixture attempts remain recorded separately.
    • Site artifact and actual Pages deployment succeeded.
    • All eight live routes passed HTTP200, section/TOC, per-SDK shutdown tokens, canonical and nine alternate URLs, and actual localized rollout-link navigation. The real Copy button returned the qualified manifest bytes in every locale, SHA256 91f7f9bb44e60b5d50134453933137db475e853e488079d6fd785c6f4b76ad87. All64 new rendered paragraphs/list items additionally match the exact merged source. English/Japanese/Chinese 390px light/dark views were inspected with no document overflow.

    This completes Website #336. The runtime control is an isolated kind/ephemeral-SQLite fixture and tests idle termination. Active-task interruption, forced kill, recovery, production databases/networking and HA are not claimed. The selected product tuple is unchanged; this guide does not qualify a new Server deployment or product fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationpriority:P3Lower-priority product improvement

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions