redis: vendor redis.conf and auto-size maxmemory from the memory limit - #922
Draft
michaellzc wants to merge 1 commit into
Draft
redis: vendor redis.conf and auto-size maxmemory from the memory limit#922michaellzc wants to merge 1 commit into
michaellzc wants to merge 1 commit into
Conversation
`redis-cache` and `redis-store` relied entirely on the image's baked
`/etc/redis/redis.conf`, which hardcodes `maxmemory 6gb`. The chart never
told redis about the pod memory limit, so lowering
`resources.limits.memory` below 6GiB made the kubelet OOM-kill the pod
before LRU eviction ran, and the pod restarted into the same wall.
Follow the pgsql pattern: vendor the images' config in-repo under
`files/redis-{cache,store}/redis.conf`, render it into a
`redis-{cache,store}-conf` ConfigMap and mount it over the image path
with `subPath`. The image entrypoint (`redis-server
/etc/redis/redis.conf`) reads it, so no `command`/`args` override is
needed. The vendored files are copied verbatim from the pinned 6.0.0
image tags.
The chart appends a `maxmemory` sized at
`config.maxmemoryRatio` (0.75) of `resources.limits.memory`. A new
`config` block adds `existingConfig`, `maxmemory`, `maxmemoryRatio`,
`maxmemoryPolicy` and `additionalConfig` for both services. The quantity
parser fails safe: an unset or unrecognised limit emits no `maxmemory`
and the vendored default stands.
`sourcegraph.redis.assertNoConfClash` fails the render with a migration
message when `extraVolumeMounts` already targets `/etc/redis/redis.conf`
or an extra volume reuses the reserved `redis-conf` name, instead of
letting it surface as an opaque duplicate-mount rejection.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both redis images bake
maxmemory 6gbinto/etc/redis/redis.confand the chart never told redis about the pod memory limit, so loweringresources.limits.memorybelow 6GiB made the kubelet OOM-kill the pod before LRU eviction ran, restarting it into the same wall. Following the existing pgsql pattern, this vendors the images' config in-repo, renders it into aredis-{cache,store}-confConfigMap mounted over the image path withsubPath, and appends amaxmemorysized atconfig.maxmemoryRatio(0.75) of the memory limit — the image entrypoint reads it, so there is nocommand/argsoverride. A newconfigblock on both services addsexistingConfig,maxmemory,maxmemoryRatio,maxmemoryPolicyandadditionalConfig, and the quantity parser fails safe: an unset or unrecognised limit emits nomaxmemoryso the vendored default stands.Breaking: operators who inject their own
redis.confviaredisCache/redisStore.extraVolumeMountsmust move toconfig.existingConfig; a render-time guard fails with a migration message rather than letting it surface as an opaque duplicate-mount rejection. Also note the effective cap drops 6GiB → 5.25GiB at the default 7Gi limit, which forredis-store(noeviction) lowers the write-error ceiling —config.maxmemory: 6gbrestores the old behaviour.Checklist
Test plan
helm lintclean;helm unittest26 suites / 126 tests pass (two new suites cover the parser matrix, bothfailguards, and the mount/volume/checksum wiring). The HEAD~1-vs-HEAD rendered diff is purely additive — two ConfigMaps plus the per-Deployment mount, volume and checksum; every other resource is byte-identical.kubeconform -strictpasses 71/71.Provenance: extracted
/etc/redis/redis.conffrom both pinned 6.0.0 image digests withcrane— identical to the vendored files. Entrypoint confirmed as["redis-server","/etc/redis/redis.conf"]; neither image bakes arequirepass.Deployed to a local kind cluster and inspected every path with
redis-cli CONFIG GET:maxmemoryRatio: 0.5;limits: nulland an unparseable quantity both fall back to the vendored6gb; explicitmaxmemory: 300mbandmaxmemoryPolicy: volatile-lruapplied verbatim.maxmemory 6gb) wasOOMKilledwithevicted_keys:0, reproducing the loop; auto-sized (384Mi) plateaued at exactly384.00M, evicted 154,057 keys, zero restarts, still serving writes. Storenoevictionrejected real writes at 189.20M withOOM command not allowed...while staying healthy.additionalConfigand viaexistingConfig, redis enforces auth and the readiness probe still passes (replaying the probe script showed the grep finding the password in the chart-managed file). For a custom image that bakesrequirepass, verified side-by-side that the old chart keeps auth while the new chart shadows it and starts unauthenticated — the documented regression, real and silent.existingConfigskips the chart ConfigMap and the checksum annotation; changingmaxmemoryRatiorolls the pod viachecksum/redis-config(needed becausesubPathmounts never hot-reload); a real old→newhelm upgradeswung maxmemory 6442450944 → 5637144576 with data intact; all four guards fail onhelm install --dry-run=serverwith an unrelatedextraVolumeMountsnegative control passing.helm upgrade --installon kind came up healthy (frontend/healthz→ 200) with the app writing to both redis instances.Two things reviewers should know: with
requirepassset theredis-exportersidecar reportsredis_up 0while the pod still shows 2/2 Ready, so metrics go dark silently (pre-existing, but newly easy to trigger —redisExporter.env.REDIS_PASSWORDfixes it); and underlocalDevMode: truethe containerresourcesare stripped yetmaxmemoryis still computed from values.🤖 Generated with Claude Code