Repository navigation

helm

v0.9.56: PII masking for long texts, Agent block on the shared model fallback, Computer Use opens to every app with sensitive apps protected, AWS Marketplace container license check, slimmer realtime and migrations images, desktop browser rendering waits #1310

Workflow file for this run

	name: helm

	on:
	push:
	branches: [main, staging, dev]
	paths:
	- 'helm/sim/**'
	# Repository-level Artifact Hub metadata, republished by the publish job.
	- 'helm/artifacthub-repo.yml'
	- '.github/workflows/helm.yml'
	# The image inventory is generated from the chart and checked here, so a
	# change to its generator has to run this workflow too.
	- 'scripts/generate-image-manifest.ts'
	- 'package.json'
	pull_request:
	branches: [main, staging, dev]
	paths:
	- 'helm/sim/**'
	# Repository-level Artifact Hub metadata, republished by the publish job.
	- 'helm/artifacthub-repo.yml'
	- '.github/workflows/helm.yml'
	# The image inventory is generated from the chart and checked here, so a
	# change to its generator has to run this workflow too.
	- 'scripts/generate-image-manifest.ts'
	- 'package.json'

	# Pull requests cancel a superseded run; pushes never do, so a newer push cannot cut off a chart
	# publish already in flight.
	concurrency:
	group: helm-${{ github.ref }}
	cancel-in-progress: ${{ github.event_name == 'pull_request' }}

	permissions:
	contents: read

	jobs:
	chart:
	name: chart
	runs-on: &runner-2vcpu ${{ (vars.CI_PROVIDER == '' \|\| vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' \|\| 'ubuntu-latest' }}
	timeout-minutes: 15
	steps:
	- name: Checkout code (mirror)
	if: github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork
	uses: useblacksmith/checkout@2b9fe87d796feea153b455ec2f3125b471a7b0e7 # v1.8.2
	with: &chart-checkout-options
	# Chart version checks need full history to compare against the PR base.
	fetch-depth: 0
	persist-credentials: false
	- name: Checkout code
	if: github.event_name != 'pull_request' \|\| github.event.pull_request.head.repo.fork
	uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
	with: *chart-checkout-options

	- name: Set up Helm
	uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5.0.1
	with:
	version: v3.16.4

	- name: Set up chart-testing
	uses: helm/chart-testing-action@720270857a28afd41e2426521a7801db09e9b510 # v2.9.0

	- name: Setup Bun
	uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
	with:
	bun-version: 1.4.4

	# Docker Compose and Kubernetes must run the same background jobs on the
	# same schedules; this fails the build if the two drift apart. The script
	# imports only node builtins, so this job installs no dependencies.
	- name: Scheduler parity (docker/crontab vs helm cronjobs)
	run: bun run scripts/check-cron-parity.ts

	# helm/sim/images.yaml is what an operator mirrors into a disconnected
	# registry, so a chart change that adds an image has to update it. Lives
	# here rather than in `check:audits` because it renders the chart, and the
	# audits job has no Helm.
	- name: Image inventory is current
	run: bun run images:check

	# ct is the CNCF chart linter (ingress-nginx, prometheus-community and
	# external-secrets all gate on it). Beyond `helm lint` it runs yamllint over
	# Chart.yaml and every values file, validates Chart.yaml against a schema,
	# and — the reason the hand-rolled version-bump job is gone — enforces that
	# the chart version increases whenever chart content changes.
	#
	# `--chart-dirs helm --target-branch` is load-bearing. Passing `--charts`
	# instead silently DISABLES the version-increment check ("Version increment
	# checking disabled.") and still exits 0, which would leave a gate that
	# never fails. On a push there is no base to diff, so `--charts` is correct
	# there and the version check simply does not apply.
	#
	# Maintainer validation is off because it resolves `maintainers[].name`
	# against real forge accounts, and ours is the display name "Sim Team".
	# Turning it on means changing what Artifact Hub shows.
	- name: Chart lint (ct)
	env:
	BASE_REF: ${{ github.base_ref }}
	run: \|
	set -euo pipefail
	args=(--validate-maintainers=false)
	if [ "${GITHUB_EVENT_NAME}" = "pull_request" ]; then
	args+=(--chart-dirs helm --target-branch "${BASE_REF}")
	else
	args+=(--charts helm/sim)
	fi
	ct lint "${args[@]}"

	- name: Helm unit tests
	run: \|
	# Official helm-unittest image, pinned by immutable digest (tag 3.17.3-0.8.2).
	# Run as the runner's UID so the container can write into the bind
	# mount (it creates tests/__snapshot__), with a writable HOME for helm.
	docker run --rm --user "$(id -u):$(id -g)" -e HOME=/tmp \
	-v "$PWD/helm/sim:/apps" \
	helmunittest/helm-unittest@sha256:b653db7d5665bc6cec677b15c5eaa1c0377c0de8ac4eb1df58b924478baa21e1 .

	- name: Install kubeconform
	run: \|
	curl -sSL -o /tmp/kubeconform.tar.gz \
	https://github.com/yannh/kubeconform/releases/download/v0.6.7/kubeconform-linux-amd64.tar.gz
	echo "95f14e87aa28c09d5941f11bd024c1d02fdc0303ccaa23f61cef67bc92619d73 /tmp/kubeconform.tar.gz" \| sha256sum -c -
	tar -xzf /tmp/kubeconform.tar.gz -C /tmp kubeconform

	- name: Render and validate manifests (default configuration)
	run: \|
	helm template sim helm/sim --namespace sim \
	--values helm/sim/ci/default-values.yaml \
	\| /tmp/kubeconform -strict -summary \
	-kubernetes-version 1.29.0 \
	-schema-location default \
	-schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json'

	- name: Render and validate manifests (all components enabled)
	run: \|
	helm template sim helm/sim --namespace sim \
	--values helm/sim/ci/full-values.yaml \
	\| /tmp/kubeconform -strict -summary \
	-kubernetes-version 1.29.0 \
	-schema-location default \
	-schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json'

	- name: Render every example values file
	run: \|
	set -euo pipefail
	for f in helm/sim/examples/values-*.yaml; do
	echo "--- $f"
	# Examples intentionally omit secrets (their headers document the
	# required --set flags), so supply the CI dummies alongside each.
	helm template sim helm/sim --namespace sim \
	--values "$f" \
	--values helm/sim/ci/default-values.yaml \
	--set copilot.postgresql.auth.password=ci-dummy-password \
	--set copilot.server.env.AGENT_API_DB_ENCRYPTION_KEY=cicicicicicicicicicicicicicicicicicicicicicicicicicicicicicicici \
	--set copilot.server.env.INTERNAL_API_SECRET=cicicicicicicicicicicicicicicicicicicicicicicicicicicicicicicici \
	--set copilot.server.env.LICENSE_KEY=ci-dummy-license \
	--set copilot.server.env.SIM_BASE_URL=https://ci.example.com \
	--set copilot.server.env.SIM_AGENT_API_KEY=ci-dummy-agent-key \
	--set copilot.server.env.REDIS_URL=redis://ci-redis:6379 \
	--set copilot.server.env.OPENAI_API_KEY_1=ci-dummy-openai-key \
	--set externalDatabase.password=ci-dummy-password > /dev/null
	done

	install:
	name: install
	needs: chart
	runs-on: ${{ (vars.CI_PROVIDER == '' \|\| vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' \|\| 'ubuntu-latest' }}
	timeout-minutes: 25
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
	with:
	persist-credentials: false

	- name: Set up Helm
	uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5.0.1
	with:
	version: v3.16.4

	- name: Create kind cluster
	uses: helm/kind-action@ea822c7bf1874f08e5a491df991c228edffc3376 # v1.15.2
	with:
	version: v0.24.0

	- name: Install chart
	run: \|
	helm install sim helm/sim \
	--namespace sim --create-namespace \
	--values helm/sim/ci/default-values.yaml \
	--values helm/sim/ci/kind-overlay.yaml \
	--wait --timeout 15m

	- name: Run helm test
	run: helm test sim --namespace sim --timeout 5m

	# After the test, so it also explains a failed probe, not only a failed
	# install. A failed test pod is kept (the hook deletes it only on success),
	# so its log says which URL refused.
	- name: Diagnostics on failure
	if: failure()
	run: \|
	kubectl -n sim get pods -o wide \|\| true
	kubectl -n sim get events --sort-by=.lastTimestamp \| tail -40 \|\| true
	kubectl -n sim describe pods \| tail -100 \|\| true
	kubectl -n sim logs deploy/sim-app -c migrations --tail=50 \|\| true
	kubectl -n sim logs deploy/sim-app --tail=80 \|\| true
	kubectl -n sim logs deploy/sim-realtime --tail=80 \|\| true
	kubectl -n sim logs sim-test-connection \|\| true

	# Resolved once and shared, so the two publish paths cannot package the same
	# immutable chart version with different appVersions -- they run in parallel,
	# and a release becoming public between two independent API calls would be
	# enough to make the OCI artifact and the HTTP repo install different Sims.
	release-version:
	name: release-version
	if: github.event_name == 'push' && github.ref == 'refs/heads/main' && github.repository == 'simstudioai/sim'
	runs-on: *runner-2vcpu
	timeout-minutes: 5
	permissions:
	contents: read # Read the release list.
	outputs:
	version: ${{ steps.resolve.outputs.version }}
	steps:
	# On a release merge the tag does not exist yet -- this commit is what cuts
	# it -- so the subject is the only source available, and the latest release
	# is the source of truth for every other push.
	#
	# The subject pattern MUST stay identical to the `version` job in ci.yml, which
	# is what actually creates the tag and builds the images. If this one
	# matched a subject that one rejects, the chart would publish naming a
	# release that was never cut.
	- name: Resolve the app release
	id: resolve
	env:
	HEAD_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
	GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
	run: \|
	set -euo pipefail
	subject=${HEAD_COMMIT_MESSAGE%%$'\n'*}
	if [[ "$subject" =~ ^(v[0-9]+\.[0-9]+\.[0-9]+): ]]; then
	resolved="${BASH_REMATCH[1]}"
	echo "Release commit; shipping the chart with ${resolved}."
	else
	resolved=$(gh api "repos/${GITHUB_REPOSITORY}/releases/latest" --jq .tag_name)
	echo "Not a release commit; shipping the chart with the latest release ${resolved}."
	fi
	if [ -z "$resolved" ]; then
	echo "::error::Could not resolve an app release to ship this chart with."
	exit 1
	fi
	echo "version=${resolved}" >> "$GITHUB_OUTPUT"

	# Publishes the chart to GHCR as an OCI artifact. Self-hosters cannot admit a
	# chart pulled from a git checkout — they need an immutable, versioned artifact
	# they can pin by digest and mirror into an internal registry — so shipping the
	# chart in-repo only is the same as not shipping it.
	#
	# Lives here rather than in a `publish-*.yml` of its own so it can gate on the
	# jobs above: nothing is published unless the chart linted, unit-tested,
	# rendered clean under kubeconform, and actually installed on a kind cluster.
	# A separate workflow would race those instead of waiting for them.
	publish:
	name: publish
	needs: [chart, install, release-version]
	if: github.event_name == 'push' && github.ref == 'refs/heads/main' && github.repository == 'simstudioai/sim'
	runs-on: *runner-2vcpu
	timeout-minutes: 15
	permissions:
	contents: read # Read the chart source.
	packages: write # Push the chart, its signature, and its attestations to GHCR.
	id-token: write # Sigstore signs against the runner's OIDC identity; no key material is stored.
	attestations: write # Let actions/attest record the SLSA provenance.
	artifact-metadata: write # And its artifact storage record.
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
	with:
	persist-credentials: false

	- name: Set up Helm
	uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5.0.1
	with:
	version: v3.16.4

	# oras also reads ~/.docker/config.json, so this one login covers both the
	# chart push and the Artifact Hub metadata push below.
	- name: Login to GHCR
	uses: docker/login-action@8aa5831fb8d48d8f76594c7ddde61ae58019f550 # v4
	with:
	registry: ghcr.io
	username: ${{ github.repository_owner }}
	password: ${{ secrets.GITHUB_TOKEN }}

	- name: Setup Bun
	uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
	with:
	bun-version: 1.4.4

	# Derives appVersion rather than checking it, so the published chart cannot
	# be pinned to an older Sim than the release it ships with. The committed
	# value is kept current too, but nothing depends on a human remembering.
	- name: Sync chart appVersion to the release
	env:
	APP_VERSION: ${{ needs.release-version.outputs.version }}
	run: bun run scripts/sync-chart-appversion.ts --version "${APP_VERSION}"

	- name: Package chart
	id: package
	run: \|
	set -euo pipefail
	chart=$(helm show chart helm/sim)
	name=$(printf '%s\n' "$chart" \| awk '/^name:/ {print $2}')
	version=$(printf '%s\n' "$chart" \| awk '/^version:/ {print $2}')
	helm package helm/sim --destination dist
	{
	echo "name=${name}"
	echo "version=${version}"
	echo "path=dist/${name}-${version}.tgz"
	echo "repository=ghcr.io/${GITHUB_REPOSITORY_OWNER}/charts/${name}"
	} >> "$GITHUB_OUTPUT"

	# Chart versions are immutable once published: whoever pinned a version
	# must keep resolving the same bytes forever. The PR gate above already
	# forces a version bump on every chart change, so a version that is
	# already in the registry means this commit changed something outside
	# `helm/sim/`.
	#
	# The lookup must fail CLOSED. Treating every non-zero exit as "absent"
	# would let a transient 5xx, an expired token, or a DNS blip re-push an
	# existing version and move a tag consumers have already pinned — and
	# same-version runs are routine, since the path filter also fires on
	# `package.json` and workflow edits.
	#
	# Verified against the pinned Helm (v3.16.4): an absent version AND an
	# absent repository both report `<ref>: not found`, so a first publish
	# still proceeds, while `denied`, `unauthorized`, and `dial tcp` failures
	# do not match and stop the job instead.
	- name: Skip if this version is already published
	id: exists
	env:
	REPOSITORY: ${{ steps.package.outputs.repository }}
	NAME: ${{ steps.package.outputs.name }}
	VERSION: ${{ steps.package.outputs.version }}
	run: \|
	set -euo pipefail
	if err=$(helm show chart "oci://${REPOSITORY}" --version "${VERSION}" 2>&1 >/dev/null); then
	echo "already=true" >> "$GITHUB_OUTPUT"
	echo "::notice::${NAME} ${VERSION} is already published; skipping."
	elif printf '%s\n' "$err" \| grep -q ': not found'; then
	echo "already=false" >> "$GITHUB_OUTPUT"
	else
	printf '%s\n' "$err"
	echo "::error::Could not determine whether ${NAME} ${VERSION} is already published. Refusing to push, because an unchecked push can overwrite a published version."
	exit 1
	fi

	# `helm push` takes the namespace only — it derives the repository
	# basename from the chart's name and the tag from its version, so the
	# result is ghcr.io/<owner>/charts/sim:<version>.
	- name: Push chart
	id: push
	if: steps.exists.outputs.already == 'false'
	env:
	CHART_PATH: ${{ steps.package.outputs.path }}
	run: \|
	set -euo pipefail
	output=$(helm push "${CHART_PATH}" "oci://ghcr.io/${GITHUB_REPOSITORY_OWNER}/charts" 2>&1)
	printf '%s\n' "$output"
	digest=$(printf '%s\n' "$output" \| grep -oE 'sha256:[a-f0-9]{64}' \| head -1 \|\| true)
	if [ -z "$digest" ]; then
	echo "::error::helm push did not report a digest; refusing to sign an unidentified artifact"
	exit 1
	fi
	echo "digest=${digest}" >> "$GITHUB_OUTPUT"

	- name: Install Cosign
	if: steps.exists.outputs.already == 'false'
	uses: sigstore/cosign-installer@756fe2d284c7292c4e9b62c8a28f1e5629c59532 # v4.1.3

	# Signed by digest, never by tag: a tag is a mutable pointer, so signing
	# one would attest to whatever it happens to reference later. The verify
	# is not ceremony — it fails the run if the signature we just wrote cannot
	# be read back with the identity we expect, which is the whole point of
	# publishing a signature at all.
	- name: Sign and verify chart
	if: steps.exists.outputs.already == 'false'
	env:
	REPOSITORY: ${{ steps.package.outputs.repository }}
	DIGEST: ${{ steps.push.outputs.digest }}
	run: \|
	set -euo pipefail
	ref="${REPOSITORY}@${DIGEST}"
	cosign sign --yes "$ref"
	cosign verify "$ref" \
	--certificate-identity-regexp "^https://github.com/${GITHUB_REPOSITORY}/" \
	--certificate-oidc-issuer https://token.actions.githubusercontent.com

	# Stored alongside the chart so a mirrored registry carries the
	# attestation with it, rather than only being retrievable from GitHub.
	- name: Attest build provenance
	if: steps.exists.outputs.already == 'false'
	uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
	with:
	subject-name: ${{ steps.package.outputs.repository }}
	subject-digest: ${{ steps.push.outputs.digest }}
	push-to-registry: true

	- name: Set up ORAS
	uses: oras-project/setup-oras@5e2f2482b30b0cb3e14e67f89a80c88c659a7c63 # v2.0.3

	# Artifact Hub reads repository metadata from the reserved `artifacthub.io`
	# tag on the chart's own OCI repository. Pushed on every run, including
	# version-skip runs, so an edit to the metadata file alone still lands.
	- name: Publish Artifact Hub metadata
	env:
	REPOSITORY: ${{ steps.package.outputs.repository }}
	# Run from `helm/` so the layer's title annotation is the bare
	# `artifacthub-repo.yml`, matching Artifact Hub's documented command. A
	# path-qualified argument records `helm/artifacthub-repo.yml` instead.
	working-directory: helm
	run: \|
	set -euo pipefail
	oras push "${REPOSITORY}:artifacthub.io" \
	--config /dev/null:application/vnd.cncf.artifacthub.config.v1+yaml \
	artifacthub-repo.yml:application/vnd.cncf.artifacthub.repository-metadata.layer.v1.yaml

	- name: Summary
	env:
	ALREADY: ${{ steps.exists.outputs.already }}
	REPOSITORY: ${{ steps.package.outputs.repository }}
	VERSION: ${{ steps.package.outputs.version }}
	DIGEST: ${{ steps.push.outputs.digest }}
	run: \|
	{
	if [ "${ALREADY}" = "true" ]; then
	echo "### Chart ${VERSION} was already published — nothing to do"
	else
	echo "### Published chart ${VERSION}"
	echo
	echo "Digest: \`${DIGEST}\`"
	fi
	echo
	echo '```bash'
	echo "helm install sim oci://${REPOSITORY} --version ${VERSION}"
	echo '```'
	} >> "$GITHUB_STEP_SUMMARY"

	# The classic HTTP repo, published alongside the OCI artifact above. Both is
	# what the ecosystem actually does: Bitnami, cert-manager, ingress-nginx,
	# prometheus-community, Grafana, Argo and external-secrets all still serve an
	# index.yaml, because plenty of clusters, GitOps configs and mirroring tools
	# only speak `helm repo add`. OCI is the modern path, not yet the only one.
	#
	# Separate from the OCI job on purpose: chart-releaser needs `contents: write`
	# to cut a release and push the index, and there is no reason to hand that to
	# the job holding the signing identity.
	publish-http:
	name: publish-http
	needs: [chart, install, release-version]
	if: github.event_name == 'push' && github.ref == 'refs/heads/main' && github.repository == 'simstudioai/sim'
	runs-on: *runner-2vcpu
	timeout-minutes: 15
	permissions:
	contents: write # Cut the chart release and push index.yaml to the pages branch.
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v6
	with:
	# chart-releaser diffs against the previous tag to decide which charts
	# changed, so it needs the full history rather than a shallow clone.
	fetch-depth: 0
	# chart-releaser authenticates with CR_TOKEN, not the checkout credential.
	persist-credentials: false

	# Creating the pages branch and turning on GitHub Pages are one-time
	# manual steps that no workflow can do for itself. Skip loudly rather than
	# failing main when they have not happened yet -- the OCI publish is
	# independent and must not be held hostage to this.
	- name: Check the pages branch exists
	id: pages
	run: \|
	set -euo pipefail
	if git ls-remote --exit-code --heads origin gh-pages >/dev/null 2>&1; then
	echo "exists=true" >> "$GITHUB_OUTPUT"
	else
	echo "exists=false" >> "$GITHUB_OUTPUT"
	echo "::warning::No gh-pages branch, so the HTTP chart repo was not updated. Create it and point GitHub Pages at it to activate this job. The OCI publish is unaffected."
	fi

	- name: Setup Bun
	uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
	with:
	bun-version: 1.4.4

	# Derives appVersion rather than checking it, so the published chart cannot
	# be pinned to an older Sim than the release it ships with. The committed
	# value is kept current too, but nothing depends on a human remembering.
	- name: Sync chart appVersion to the release
	if: steps.pages.outputs.exists == 'true'
	env:
	APP_VERSION: ${{ needs.release-version.outputs.version }}
	run: bun run scripts/sync-chart-appversion.ts --version "${APP_VERSION}"

	- name: Configure Git
	if: steps.pages.outputs.exists == 'true'
	env:
	ACTOR: ${{ github.actor }}
	run: \|
	set -euo pipefail
	git config user.name "${ACTOR}"
	git config user.email "${ACTOR}@users.noreply.github.com"

	# chart-releaser writes index.yaml to the pages branch and attaches the
	# .tgz to a GitHub release, which is where index.yaml points -- so the
	# packages stay reachable no matter which domain serves the index.
	- name: Run chart-releaser
	if: steps.pages.outputs.exists == 'true'
	uses: helm/chart-releaser-action@92c6b97fa9f664492a2592e0d1d80bc6bf378957 # v1.9.0
	with:
	charts_dir: helm
	# Re-running on an already-released version must be a no-op, the same
	# way the OCI publish above refuses to move a published version.
	skip_existing: true
	# A chart release must never take the "Latest" badge from the
	# application release it packages.
	mark_as_latest: false
	env:
	CR_TOKEN: ${{ secrets.GITHUB_TOKEN }}
	# Keeps chart releases visually distinct from the vX.Y.Z app releases
	# they share the list with.
	CR_RELEASE_NAME_TEMPLATE: "helm-chart-{{ .Version }}"

Navigation Menu

Repository navigation

v0.9.56: PII masking for long texts, Agent block on the shared model fallback, Computer Use opens to every app with sensitive apps protected, AWS Marketplace container license check, slimmer realtime and migrations images, desktop browser rendering waits #1310

Workflow file

v0.9.56: PII masking for long texts, Agent block on the shared model fallback, Computer Use opens to every app with sensitive apps protected, AWS Marketplace container license check, slimmer realtime and migrations images, desktop browser rendering waits #1310

Uh oh!

Workflow file for this run