Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ jobs:
- name: Sandbox-runner metrics discovery and network policy
run: tests/sandbox_runner_metrics.sh

- name: Redis TLS environment
run: tests/redis_tls_env.sh

- name: Bridge pairing rollout safety
run: tests/bridge_pairing_rollout.sh

Expand Down
2 changes: 1 addition & 1 deletion helm/codeapi/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ apiVersion: v2
name: codeapi
description: A Helm chart for Code Interpreter API - scalable code execution service
type: application
version: 0.3.2 # Chart version (bump this when you change the chart)
version: 0.4.0 # Chart version (bump this when you change the chart)
appVersion: "2.0.0" # App version (bump this when you change the app)

# Keywords for searching
Expand Down
33 changes: 31 additions & 2 deletions helm/codeapi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,8 +197,37 @@ api:
For development only, `LOCAL_MODE=true` bypasses authentication — see
`values-local.yaml`.

**TLS to Redis.** Set `REDIS_TLS=true` via `extraEnv` on each component when
your Redis (e.g. a managed cache) requires TLS.
**External Redis and TLS.** Set `redis.enabled=false` to use your own Redis
instead of the bundled subchart, and configure it under `redis.external`:

| Value | Default | Description |
|-------|---------|-------------|
| `redis.external.host` | `""` | Redis hostname. |
| `redis.external.port` | `6379` | Redis port. |
| `redis.external.password` | `""` | Redis password / AUTH token. |
| `redis.external.tls` | `false` | Connect over TLS (sets `REDIS_TLS=true`). Required by managed caches with in-transit encryption, e.g. AWS ElastiCache. |
| `redis.external.useAlternativeDnsLookup` | `false` | Skip DNS resolution of the host before the TLS handshake (sets `REDIS_USE_ALTERNATIVE_DNS_LOOKUP=true`), as ioredis recommends for ElastiCache with TLS. |

The TLS settings are rendered on every control-plane container that talks to
Redis (api, service-worker, file-server, tool-call-server, egress-gateway). The
sandbox-runner never receives Redis configuration. They only apply when
`redis.enabled=false`; the bundled subchart is not configured for TLS.

AWS ElastiCache only allows AUTH with in-transit encryption enabled, so use
both options together:

```yaml
redis:
enabled: false
external:
host: master.my-group.xxxxxx.euc1.cache.amazonaws.com
port: 6379
password: <auth token> # better: --set redis.external.password=...
tls: true
useAlternativeDnsLookup: true
```

The clients do not yet verify the Redis server certificate.

**Package delivery.** KVM deployments default to
`workerSandbox.packages.source=image`. Build and publish the baked runner target
Expand Down
19 changes: 19 additions & 0 deletions helm/codeapi/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,25 @@ Redis port
{{- end }}
{{- end }}

{{/*
Redis TLS env vars for control-plane containers. Only applies to external
Redis; the bundled subchart is not configured for TLS. Never include this in
the sandbox-runner container.
*/}}
{{- define "codeapi.redis.tlsEnv" -}}
{{- $external := .Values.redis.external | default dict }}
{{- if not .Values.redis.enabled }}
{{- if (get $external "tls") }}
- name: REDIS_TLS
value: "true"
{{- end }}
{{- if (get $external "useAlternativeDnsLookup") }}
- name: REDIS_USE_ALTERNATIVE_DNS_LOOKUP
value: "true"
{{- end }}
{{- end }}
{{- end }}

{{/*
Redis NetworkPolicy egress. Kubernetes NetworkPolicy cannot match DNS names,
so external Redis can be scoped with CIDRs when available; otherwise the chart
Expand Down
1 change: 1 addition & 0 deletions helm/codeapi/templates/api-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ spec:
secretKeyRef:
name: {{ include "codeapi.fullname" . }}-secrets
key: redis-password
{{- with include "codeapi.redis.tlsEnv" . }}{{ . | trim | nindent 12 }}{{ end }}
- name: CODEAPI_BRIDGE_PAIRING_ROLLBACK_EPOCH
valueFrom:
configMapKeyRef:
Expand Down
1 change: 1 addition & 0 deletions helm/codeapi/templates/egress-gateway-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ spec:
secretKeyRef:
name: {{ include "codeapi.fullname" . }}-secrets
key: redis-password
{{- with include "codeapi.redis.tlsEnv" . }}{{ . | trim | nindent 12 }}{{ end }}
livenessProbe:
httpGet:
path: /live
Expand Down
1 change: 1 addition & 0 deletions helm/codeapi/templates/file-server-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ spec:
secretKeyRef:
name: {{ include "codeapi.fullname" . }}-secrets
key: redis-password
{{- with include "codeapi.redis.tlsEnv" . }}{{ . | trim | nindent 12 }}{{ end }}
- name: CODEAPI_INTERNAL_SERVICE_TOKEN
valueFrom:
secretKeyRef:
Expand Down
1 change: 1 addition & 0 deletions helm/codeapi/templates/tool-call-server-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ spec:
secretKeyRef:
name: {{ include "codeapi.fullname" . }}-secrets
key: redis-password
{{- with include "codeapi.redis.tlsEnv" . }}{{ . | trim | nindent 12 }}{{ end }}
- name: CODEAPI_INTERNAL_SERVICE_TOKEN
valueFrom:
secretKeyRef:
Expand Down
1 change: 1 addition & 0 deletions helm/codeapi/templates/worker-sandbox-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ spec:
secretKeyRef:
name: {{ include "codeapi.fullname" . }}-secrets
key: redis-password
{{- with include "codeapi.redis.tlsEnv" . }}{{ . | trim | nindent 12 }}{{ end }}
- name: CODEAPI_INTERNAL_SERVICE_TOKEN
valueFrom:
secretKeyRef:
Expand Down
14 changes: 9 additions & 5 deletions helm/codeapi/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -452,11 +452,15 @@ redis:
enabled: true
size: 1Gi

# If using external Redis, configure here:
# external:
# host: redis.example.com
# port: 6379
# password: secret
# External Redis, used only when redis.enabled is false.
external:
host: ""
port: 6379
password: ""
# Connect over TLS (required by e.g. AWS ElastiCache with in-transit encryption).
tls: false
# Skip DNS resolution of the host before the TLS handshake (recommended by ioredis for ElastiCache + TLS).
useAlternativeDnsLookup: false

# =============================================================================
# MINIO (S3-compatible storage)
Expand Down
74 changes: 74 additions & 0 deletions tests/redis_tls_env.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
#!/usr/bin/env bash
set -euo pipefail

ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
TMP_DIR="$(mktemp -d)"
trap 'rm -rf "$TMP_DIR"' EXIT
command -v helm >/dev/null

# Render only this chart's templates so the test does not download the Redis
# or MinIO dependencies.
mkdir "$TMP_DIR/chart"
cp "$ROOT/helm/codeapi/values.yaml" "$TMP_DIR/chart/values.yaml"
cp -R "$ROOT/helm/codeapi/templates" "$TMP_DIR/chart/templates"
awk '/^dependencies:/{exit} {print}' "$ROOT/helm/codeapi/Chart.yaml" > "$TMP_DIR/chart/Chart.yaml"

render() {
helm template redis-tls "$TMP_DIR/chart" \
--set executionManifest.privateKey=test \
--set executionManifest.publicKey=test "$@"
}

# Print "<container> <env name>" for every REDIS_TLS* / alternative DNS env var.
redis_tls_env() {
awk '
/^ - name: / { container=$3 }
/^ - name: REDIS_TLS$/ { print container, $3 }
/^ - name: REDIS_USE_ALTERNATIVE_DNS_LOOKUP$/ { print container, $3 }
' "$1" | sort
}

fail() {
echo "FAIL: $*" >&2
exit 1
}

# External Redis with TLS: exactly the five control-plane containers get both
# vars; sandbox-runner must never receive Redis configuration.
render --set redis.enabled=false \
--set redis.external.host=redis.example.com \
--set redis.external.tls=true \
--set redis.external.useAlternativeDnsLookup=true > "$TMP_DIR/tls.yaml"
expected="$(for c in api egress-gateway file-server service-worker tool-call-server; do
echo "$c REDIS_TLS"
echo "$c REDIS_USE_ALTERNATIVE_DNS_LOOKUP"
done | sort)"
actual="$(redis_tls_env "$TMP_DIR/tls.yaml")"
[ "$actual" = "$expected" ] || fail "unexpected Redis TLS env placement:
$actual"
if awk '/^ - name: sandbox-runner$/{s=1;next} /^ - name: /{s=0} s && /REDIS_/' "$TMP_DIR/tls.yaml" | grep -q .; then
fail "sandbox-runner received REDIS_* env"
fi

# Each option is independent.
render --set redis.enabled=false --set redis.external.host=x \
--set redis.external.tls=true > "$TMP_DIR/tls-only.yaml"
[ "$(redis_tls_env "$TMP_DIR/tls-only.yaml" | grep -c ' REDIS_TLS$')" = 5 ] \
|| fail "redis.external.tls did not render REDIS_TLS on all five containers"
! redis_tls_env "$TMP_DIR/tls-only.yaml" | grep -q REDIS_USE_ALTERNATIVE_DNS_LOOKUP \
|| fail "REDIS_USE_ALTERNATIVE_DNS_LOOKUP rendered without useAlternativeDnsLookup"

# TLS is ignored for the bundled subchart, which has no TLS.
render --set redis.enabled=true --set redis.external.tls=true \
--set redis.external.useAlternativeDnsLookup=true > "$TMP_DIR/bundled.yaml"
[ -z "$(redis_tls_env "$TMP_DIR/bundled.yaml")" ] \
|| fail "bundled Redis must not get REDIS_TLS"

# Defaults and older values files without redis.external render no TLS env.
render > "$TMP_DIR/default.yaml"
[ -z "$(redis_tls_env "$TMP_DIR/default.yaml")" ] || fail "default render sets Redis TLS env"
render --set redis.external=null > "$TMP_DIR/no-external.yaml"
cmp -s "$TMP_DIR/default.yaml" "$TMP_DIR/no-external.yaml" \
|| fail "render without redis.external differs from default"

echo 'PASS: Redis TLS env reaches every control-plane Redis client and never sandbox-runner'