Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions charts/sourcegraph/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ Use `**BREAKING**:` to denote a breaking change

## Unreleased

- **BREAKING**: The chart now manages `/etc/redis/redis.conf` for `redis-cache` and `redis-store`. It vendors the config that the images bake in (`charts/sourcegraph/files/redis-{cache,store}/redis.conf`), renders it into a `redis-{cache,store}-conf` ConfigMap and mounts it over the image path. Operators who inject their own `redis.conf` through `redisCache.extraVolumeMounts` / `redisStore.extraVolumeMounts` must move to the new `redisCache.config.existingConfig` / `redisStore.config.existingConfig`; the chart fails the render with a migration message otherwise.
- Added `redisCache.config` and `redisStore.config` (`existingConfig`, `maxmemory`, `maxmemoryRatio`, `maxmemoryPolicy`, `additionalConfig`). By default the chart sizes redis `maxmemory` at `maxmemoryRatio` (0.75) of `resources.limits.memory` instead of leaving the images' hardcoded `maxmemory 6gb`, so lowering the memory limit no longer OOM-kills the pod before eviction runs. At the default 7Gi limit this lowers the effective cap from 6GiB to 5.25GiB: `redis-cache` only evicts earlier, but `redis-store` uses `noeviction`, so its write-error ceiling drops too. Set `redisStore.config.maxmemory: 6gb` or raise `redisStore.config.maxmemoryRatio` to keep the old ceiling.
- Added `searcher.autoCacheSize` (default `false`) to omit the `SEARCHER_CACHE_SIZE_MB` and `SYMBOLS_CACHE_SIZE_MB` env vars, letting `searcher` auto-size its cache to ~45% of the live cache volume so it tracks PVC expansion instead of staying frozen to the initial `storageSize`
- Added support for ordering trace processors via `openTelemetry.gateway.config.traces.tracePipelineProcessors`, falling back to processors ordered by name when unset
- Removed the unused executor controller `/data` PersistentVolumeClaim from the Kubernetes-native executor chart (`sourcegraph-executor/k8s`), along with the now-orphaned `storageClass` and `executor.storageSize` values and the vestigial `EXECUTOR_KUBERNETES_PERSISTENCE_VOLUME_NAME` env var. Since single-job-pod became the only k8s execution mode, job pods use their own ephemeral `emptyDir` volume and the controller writes nothing to `/data`.
Expand Down
10 changes: 10 additions & 0 deletions charts/sourcegraph/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,11 @@ In addition to the documented values, all services also support the following va
| prometheus.storageAnnotations | object | `{}` | Optional annotations to add to the `prometheus` PVC |
| prometheus.storageSize | string | `"200Gi"` | PVC Storage Request for `prometheus` data volume |
| prometheus.storageSubPath | string | `""` | Optional subPath for the `prometheus` primary data volume mount |
| redisCache.config.additionalConfig | string | `""` | Additional raw redis directives appended to the vendored `redis-cache` config. Notes: This is expecting a multiline string. It renders into a ConfigMap in plaintext, so do not put secrets such as `requirepass` here; use `existingConfig` instead. |
| redisCache.config.existingConfig | string | `""` | Name of an existing ConfigMap for `redis-cache`. It must contain a `redis.conf` key. When set, the chart-managed ConfigMap is not rendered and this one is mounted instead, so the chart no longer sizes `maxmemory`. Mutually exclusive with `additionalConfig`. |
| redisCache.config.maxmemory | string | `""` | Explicit redis `maxmemory` for `redis-cache` (for example `6gb`). Overrides the auto-computed value. Empty means compute it from the container memory limit. |
| redisCache.config.maxmemoryPolicy | string | `""` | Override the redis `maxmemory-policy` for `redis-cache`. Empty keeps the vendored default (`allkeys-lru`). |
| redisCache.config.maxmemoryRatio | float | `0.75` | Fraction of `redisCache.resources.limits.memory` used for `maxmemory` when `maxmemory` is empty. Ignored when no memory limit is set. |
| redisCache.connection.endpoint | string | `"redis-cache:6379"` | Endpoint to use for redis-cache. Supports either host:port or IANA specification |
| redisCache.connection.existingSecret | string | `""` | Name of existing secret to use for Redis endpoint The secret must contain the key `endpoint` and should follow IANA specification learn more from the [Helm docs](https://docs.sourcegraph.com/admin/install/kubernetes/helm#using-external-redis-instances) |
| redisCache.containerSecurityContext | object | `{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":1000,"runAsUser":999}` | Security context for the `redis-cache` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
Expand All @@ -307,6 +312,11 @@ In addition to the documented values, all services also support the following va
| redisExporter.image.defaultTag | string | `"6.0.0@sha256:b2ec48fc6adef31f36d525170138dec303c1c0c20c530d659f1fb7c6c54698af"` | Docker image tag for the `redis-exporter` image |
| redisExporter.image.name | string | `"redis_exporter"` | Docker image name for the `redis-exporter` image |
| redisExporter.resources | object | `{"limits":{"cpu":"10m","memory":"100Mi"},"requests":{"cpu":"10m","memory":"100Mi"}}` | Resource requests & limits for the `redis-exporter` sidecar container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
| redisStore.config.additionalConfig | string | `""` | Additional raw redis directives appended to the vendored `redis-store` config. Notes: This is expecting a multiline string. It renders into a ConfigMap in plaintext, so do not put secrets such as `requirepass` here; use `existingConfig` instead. |
| redisStore.config.existingConfig | string | `""` | Name of an existing ConfigMap for `redis-store`. It must contain a `redis.conf` key. When set, the chart-managed ConfigMap is not rendered and this one is mounted instead, so the chart no longer sizes `maxmemory`. Mutually exclusive with `additionalConfig`. |
| redisStore.config.maxmemory | string | `""` | Explicit redis `maxmemory` for `redis-store` (for example `6gb`). Overrides the auto-computed value. Empty means compute it from the container memory limit. |
| redisStore.config.maxmemoryPolicy | string | `""` | Override the redis `maxmemory-policy` for `redis-store`. Empty keeps the vendored default (`noeviction`). |
| redisStore.config.maxmemoryRatio | float | `0.75` | Fraction of `redisStore.resources.limits.memory` used for `maxmemory` when `maxmemory` is empty. Ignored when no memory limit is set. |
| redisStore.connection.endpoint | string | `"redis-store:6379"` | Endpoint to use for redis-store. Supports either host:port or IANA specification |
| redisStore.connection.existingSecret | string | `""` | Name of existing secret to use for Redis endpoint The secret must contain the key `endpoint` and should follow IANA specification learn more from the [Helm docs](https://docs.sourcegraph.com/admin/install/kubernetes/helm#using-external-redis-instances) |
| redisStore.containerSecurityContext | object | `{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":1000,"runAsUser":999}` | Security context for the `redis-store` container, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
Expand Down
17 changes: 17 additions & 0 deletions charts/sourcegraph/files/redis-cache/redis.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Vendored from the `redis-cache` image (`/etc/redis/redis.conf`).
# Source: sourcegraph/sourcegraph docker-images/redis-cache/redis.conf
# Seeded from image tag 6.0.0. Keep this file in sync when the image tag changes.
#
# The chart mounts this file over /etc/redis/redis.conf, so the image entrypoint
# (`redis-server /etc/redis/redis.conf`) reads it. The chart appends the computed
# `maxmemory` and any `redisCache.config` overrides after this body.

# allow access from all instances
protected-mode no
# limit memory usage, discard unused keys when hitting limit
maxmemory 6gb
maxmemory-policy allkeys-lru
# snapshots on disk every minute
dir /redis-data/
appendonly no
save 60 1
18 changes: 18 additions & 0 deletions charts/sourcegraph/files/redis-store/redis.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Vendored from the `redis-store` image (`/etc/redis/redis.conf`).
# Source: sourcegraph/sourcegraph docker-images/redis-store/redis.conf
# Seeded from image tag 6.0.0. Keep this file in sync when the image tag changes.
#
# The chart mounts this file over /etc/redis/redis.conf, so the image entrypoint
# (`redis-server /etc/redis/redis.conf`) reads it. The chart appends the computed
# `maxmemory` and any `redisStore.config` overrides after this body.

# allow access from all instances
protected-mode no
# limit memory usage, return error when hitting limit
maxmemory 6gb
maxmemory-policy noeviction
# live commit log to disk, additionally snapshot every 5 minutes
dir /redis-data/
appendonly yes
aof-use-rdb-preamble yes
save 300 1
57 changes: 57 additions & 0 deletions charts/sourcegraph/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -312,3 +312,60 @@ checksum/auth: {{ toJson $checksum | sha256sum }}
{{- $checksum := append $checksum .Values.redisCache.connection -}}
checksum/redis: {{ toJson $checksum | sha256sum }}
{{- end -}}

{{/*
Resolve the redis `maxmemory` directive for a service.
Usage: include "sourcegraph.redis.maxmemory" (list . "redisCache")

Resolution order:
1. <service>.config.maxmemory, used verbatim.
2. floor(<service>.config.maxmemoryRatio * <service>.resources.limits.memory),
rendered as a plain byte count.
3. Empty string, when there is no memory limit or the quantity is not
recognised. The caller then emits no `maxmemory` and the vendored default
stands.
*/}}
{{- define "sourcegraph.redis.maxmemory" -}}
{{- $top := index . 0 -}}
{{- $service := index . 1 -}}
{{- $values := index $top.Values $service -}}
{{- $config := $values.config | default dict -}}
{{- if $config.maxmemory -}}
{{- $config.maxmemory -}}
{{- else -}}
{{- $limit := dig "resources" "limits" "memory" "" $values | toString -}}
{{- $number := regexReplaceAll "^([0-9]+(\\.[0-9]+)?).*$" $limit "${1}" -}}
{{- $suffix := regexReplaceAll "^[0-9]+(\\.[0-9]+)?" $limit "" -}}
{{- /* Kubernetes quantity suffixes: binary (1024^n) and decimal (1000^n). */ -}}
{{- $units := dict "" 1.0 "k" 1e3 "M" 1e6 "G" 1e9 "T" 1e12 "P" 1e15 "E" 1e18 "Ki" 1024.0 "Mi" 1048576.0 "Gi" 1073741824.0 "Ti" 1099511627776.0 "Pi" 1125899906842624.0 "Ei" 1152921504606846976.0 -}}
{{- if and (regexMatch "^[0-9]+(\\.[0-9]+)?$" $number) (hasKey $units $suffix) -}}
{{- $ratio := $config.maxmemoryRatio | default 0.75 | float64 -}}
{{- $bytes := floor (mulf (float64 $number) (index $units $suffix) $ratio) -}}
{{- if gt $bytes 0.0 -}}
{{- printf "%d" (int64 $bytes) -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- end -}}

{{/*
Fail the render when a service's extra volumes or volume mounts collide with the
chart-managed redis config mount. Without this the collision only surfaces as an
opaque "must be unique" rejection from the API server.
Usage: include "sourcegraph.redis.assertNoConfClash" (list . "redisCache")
*/}}
{{- define "sourcegraph.redis.assertNoConfClash" -}}
{{- $top := index . 0 -}}
{{- $service := index . 1 -}}
{{- $values := index $top.Values $service -}}
{{- range ($values.extraVolumeMounts | default list) -}}
{{- if has .mountPath (list "/etc/redis/redis.conf" "/etc/redis" "/etc/redis/") -}}
{{- fail (printf "%s.extraVolumeMounts must not mount over /etc/redis/redis.conf; the chart now manages that file. Move your custom redis config to %s.config.existingConfig or %s.config.additionalConfig." $service $service $service) -}}
{{- end -}}
{{- end -}}
{{- range (concat ($values.extraVolumes | default list) ($values.extraVolumeMounts | default list)) -}}
{{- if eq (.name | toString) "redis-conf" -}}
{{- fail (printf "%s must not define a volume named 'redis-conf'; the chart reserves that name for the redis config mount." $service) -}}
{{- end -}}
{{- end -}}
{{- end -}}
35 changes: 35 additions & 0 deletions charts/sourcegraph/templates/redis/redis-cache.ConfigMap.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{{- if and .Values.redisCache.enabled (and .Values.redisCache.config.existingConfig .Values.redisCache.config.additionalConfig) -}}
{{- fail "You can only define one of 'redisCache.config.existingConfig' and 'redisCache.config.additionalConfig' at a time" }}
{{- end }}
{{- if and .Values.redisCache.enabled (not .Values.redisCache.config.existingConfig) -}}
{{- $body := .Files.Get "files/redis-cache/redis.conf" | trimSuffix "\n" -}}
{{- $overrides := list -}}
{{- with include "sourcegraph.redis.maxmemory" (list . "redisCache") -}}
{{- $overrides = append $overrides (printf "maxmemory %s" .) -}}
{{- end -}}
{{- with .Values.redisCache.config.maxmemoryPolicy -}}
{{- $overrides = append $overrides (printf "maxmemory-policy %s" .) -}}
{{- end -}}
{{- with .Values.redisCache.config.additionalConfig -}}
{{- $overrides = append $overrides (tpl . $ | trim) -}}
{{- end -}}
{{- if $overrides -}}
{{- $body = printf "%s\n\n# Chart-managed directives. Redis applies the last occurrence of a directive.\n%s" $body (join "\n" $overrides) -}}
{{- end -}}
apiVersion: v1
kind: ConfigMap
metadata:
annotations:
description: Configuration for redis-cache
labels:
{{- include "sourcegraph.labels" . | nindent 4 }}
{{- if .Values.redisCache.labels }}
{{- toYaml .Values.redisCache.labels | nindent 4 }}
{{- end }}
deploy: sourcegraph
app.kubernetes.io/component: redis
name: {{ .Values.redisCache.name }}-conf
data:
redis.conf: |
{{- $body | nindent 4 }}
{{- end }}
14 changes: 14 additions & 0 deletions charts/sourcegraph/templates/redis/redis-cache.Deployment.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{{- if .Values.redisCache.enabled -}}
{{- include "sourcegraph.redis.assertNoConfClash" (list . "redisCache") -}}
apiVersion: apps/v1
kind: Deployment
metadata:
Expand Down Expand Up @@ -26,6 +27,9 @@ spec:
metadata:
annotations:
kubectl.kubernetes.io/default-container: redis-cache
{{- if not .Values.redisCache.config.existingConfig }}
checksum/redis-config: {{ include (print $.Template.BasePath "/redis/redis-cache.ConfigMap.yaml") . | sha256sum }}
{{- end }}
{{- if .Values.sourcegraph.podAnnotations }}
{{- toYaml .Values.sourcegraph.podAnnotations | nindent 8 }}
{{- end }}
Expand Down Expand Up @@ -100,6 +104,10 @@ spec:
{{- if .Values.redisCache.storageSubPath }}
subPath: {{ .Values.redisCache.storageSubPath }}
{{- end }}
- mountPath: /etc/redis/redis.conf
name: redis-conf
subPath: redis.conf
readOnly: true
{{- if .Values.redisCache.extraVolumeMounts }}
{{- toYaml .Values.redisCache.extraVolumeMounts | nindent 8 }}
{{- end }}
Expand Down Expand Up @@ -139,6 +147,12 @@ spec:
- name: redis-data
persistentVolumeClaim:
claimName: redis-cache
- name: redis-conf
configMap:
name: {{ default (print .Values.redisCache.name "-conf") .Values.redisCache.config.existingConfig }}
items:
- key: redis.conf
path: redis.conf
{{- if .Values.redisCache.extraVolumes }}
{{- toYaml .Values.redisCache.extraVolumes | nindent 6 }}
{{- end }}
Expand Down
35 changes: 35 additions & 0 deletions charts/sourcegraph/templates/redis/redis-store.ConfigMap.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{{- if and .Values.redisStore.enabled (and .Values.redisStore.config.existingConfig .Values.redisStore.config.additionalConfig) -}}
{{- fail "You can only define one of 'redisStore.config.existingConfig' and 'redisStore.config.additionalConfig' at a time" }}
{{- end }}
{{- if and .Values.redisStore.enabled (not .Values.redisStore.config.existingConfig) -}}
{{- $body := .Files.Get "files/redis-store/redis.conf" | trimSuffix "\n" -}}
{{- $overrides := list -}}
{{- with include "sourcegraph.redis.maxmemory" (list . "redisStore") -}}
{{- $overrides = append $overrides (printf "maxmemory %s" .) -}}
{{- end -}}
{{- with .Values.redisStore.config.maxmemoryPolicy -}}
{{- $overrides = append $overrides (printf "maxmemory-policy %s" .) -}}
{{- end -}}
{{- with .Values.redisStore.config.additionalConfig -}}
{{- $overrides = append $overrides (tpl . $ | trim) -}}
{{- end -}}
{{- if $overrides -}}
{{- $body = printf "%s\n\n# Chart-managed directives. Redis applies the last occurrence of a directive.\n%s" $body (join "\n" $overrides) -}}
{{- end -}}
apiVersion: v1
kind: ConfigMap
metadata:
annotations:
description: Configuration for redis-store
labels:
{{- include "sourcegraph.labels" . | nindent 4 }}
{{- if .Values.redisStore.labels }}
{{- toYaml .Values.redisStore.labels | nindent 4 }}
{{- end }}
deploy: sourcegraph
app.kubernetes.io/component: redis
name: {{ .Values.redisStore.name }}-conf
data:
redis.conf: |
{{- $body | nindent 4 }}
{{- end }}
14 changes: 14 additions & 0 deletions charts/sourcegraph/templates/redis/redis-store.Deployment.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{{- if .Values.redisStore.enabled -}}
{{- include "sourcegraph.redis.assertNoConfClash" (list . "redisStore") -}}
apiVersion: apps/v1
kind: Deployment
metadata:
Expand Down Expand Up @@ -26,6 +27,9 @@ spec:
metadata:
annotations:
kubectl.kubernetes.io/default-container: redis-store
{{- if not .Values.redisStore.config.existingConfig }}
checksum/redis-config: {{ include (print $.Template.BasePath "/redis/redis-store.ConfigMap.yaml") . | sha256sum }}
{{- end }}
{{- if .Values.sourcegraph.podAnnotations }}
{{- toYaml .Values.sourcegraph.podAnnotations | nindent 8 }}
{{- end }}
Expand Down Expand Up @@ -99,6 +103,10 @@ spec:
{{- if .Values.redisStore.storageSubPath }}
subPath: {{ .Values.redisStore.storageSubPath }}
{{- end }}
- mountPath: /etc/redis/redis.conf
name: redis-conf
subPath: redis.conf
readOnly: true
{{- if .Values.redisStore.extraVolumeMounts }}
{{- toYaml .Values.redisStore.extraVolumeMounts | nindent 8 }}
{{- end }}
Expand Down Expand Up @@ -138,6 +146,12 @@ spec:
- name: redis-data
persistentVolumeClaim:
claimName: redis-store
- name: redis-conf
configMap:
name: {{ default (print .Values.redisStore.name "-conf") .Values.redisStore.config.existingConfig }}
items:
- key: redis.conf
path: redis.conf
{{- if .Values.redisStore.extraVolumes }}
{{- toYaml .Values.redisStore.extraVolumes | nindent 6 }}
{{- end }}
Expand Down
Loading
Loading