Repository navigation
feat: add settings to turn on ClickHouse test steps storage - #45
Merged
Merged
Conversation
STEPS_CLICKHOUSE_DUAL_WRITE_FROM, STEPS_CLICKHOUSE_ONLY_FROM and STEPS_RETENTION_DAYS go in .env, which every service reads, so director and write-worker get the same times. They are commented out in .env.example, so nothing changes until an operator sets them. ai-queue-worker reads test steps from ClickHouse but had no CLICKHOUSE_PASSWORD; it now gets it from CLICKHOUSE_CURRENTS_PASSWORD, as api and write-worker do. The three compose files are regenerated. The configuration reference gets the order to turn it on. It needs Currents images that include currents-dev/currents#5007. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011baD3jVnrAJyGsQvfW3Bvd
|
| Topic | Details | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ClickHouse Step Storage | Document and expose opt-in test-step storage settings, rollout order, retention management, validation, and rollback behavior through .env.example and operator guidance.Modified files (3)
Latest Contributors(2)
| |||||||||
| Compose Credentials | Provide ai-queue-worker with CLICKHOUSE_PASSWORD from CLICKHOUSE_CURRENTS_PASSWORD across generated Compose variants so AI step reads authenticate correctly.Modified files (4)
Latest Contributors(2)
|
Merger ✅ Ready to Merge
MERGE: The PR delivers the documented ClickHouse test-step settings and required worker password wiring. CI ran successfully, and there are no unresolved discussions.
The dashboard reads object storage for tests written in dual-write, so it cannot show whether ClickHouse has their steps. Step 4 now queries test_attempt_steps and compares step_count with the object storage file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011baD3jVnrAJyGsQvfW3Bvd
The writer looks up a record's stored expiry and inserts in separate queries, so two writers on different retention values can put one test in two partitions. Changing the value needs all writers stopped first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011baD3jVnrAJyGsQvfW3Bvd
…ntion Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011baD3jVnrAJyGsQvfW3Bvd
twk3
approved these changes
Nov 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Owner: @agoldis
Linear: ENG-2238
Do not release before https://github.com/currents-dev/currents/pull/5007 (allow ClickHouse test steps on-prem) ships in a Currents release. Sibling PR for Helm: currents-dev/helm-charts#81.
What and why
Lets a Compose install turn on test steps in ClickHouse. The variables are commented out in
.env.example, so nothing changes until an operator sets them.STEPS_CLICKHOUSE_DUAL_WRITE_FROM,STEPS_CLICKHOUSE_ONLY_FROMandSTEPS_RETENTION_DAYSgo in.env. Every service hasenv_file: .env, sodirectorandwrite-workerget the same values from one place. No per-service entries are needed.ai-queue-workergetsCLICKHOUSE_PASSWORD: ${CLICKHOUSE_CURRENTS_PASSWORD}(see below). The three compose files are regenerated withsync-templates.sh.docs/configuration.mdhas a new section, "Test Steps in ClickHouse (Optional)", with what each setting does and the order to turn it on..env.exampleandon-prem/CHANGELOG.md(Unreleased) are updated.VERSIONand the image tag are not changed; the release PR does that.Service mapping
director/directordirectorruns the test record code itself.write-worker/writerapi/apiai-queue-worker/ai-queue-worker(profileai)ClickHouse access
apiandwrite-workeralready getCLICKHOUSE_URL,CLICKHOUSE_USERNAME(from.env) andCLICKHOUSE_PASSWORD.ai-queue-workergotCLICKHOUSE_URLandCLICKHOUSE_USERNAMEfrom.envbut noCLICKHOUSE_PASSWORD: the template sets it only onapi,changestreams-worker,write-workerandscheduler, and.envholds the password only asCLICKHOUSE_CURRENTS_PASSWORD. Without it the worker could not authenticate to read steps. This PR adds it. This is the one change that applies to existing installs: for those with theaiprofile on, the worker's environment gains a password it did not have.Checks
docker compose -f docker-compose.{full,database,cache}.yml config --quiet(Compose v5.6.1): all three valid.on-prem/scripts/sync-templates.shrun; the compose files differ frommainonly by the newai-queue-workerline.on-prem/scripts/check-env.shagainst a copy of.env.example: no missing variables.🤖 Generated with Claude Code
https://claude.ai/code/session_011baD3jVnrAJyGsQvfW3Bvd