feat: logs search v2 by carderne · Pull Request #4615 · triggerdotdev/trigger.dev

carderne · 2026-08-14T08:43:50Z

Summary

Improves global Logs search with a smaller, bounded search representation that is indexed asynchronously. Source event inserts are no longer coupled to search indexing, and new logs can become searchable within seconds while a slower pass preserves complete coverage.

The Logs page reads the v2 search table directly.

Design

A best-effort preview pass processes complete five-second windows after a short safety delay. An authoritative pass processes one-minute windows after a longer delay and records each completed window as an append-only PostgreSQL checkpoint.

The initial projection boundary is also an append-only checkpoint, so PostgreSQL has no mutable projector state row. Redis coordinates projection work and stores the disposable preview watermark.

LOGS_SEARCH_PROJECTOR_ENABLED is the operational switch for projection. Both passes use LOGS_CLICKHOUSE_URL when configured and otherwise fall back to the primary CLICKHOUSE_URL. They share a strengthened, bounded projection fingerprint so retries and preview/finalized overlap do not produce duplicate results, while distinct logs that share a span and timestamp remain separate.

The destination is partitioned by insertion date to keep each projection insert localized, while retention remains based on the event timestamp.

changeset-bot · 2026-08-14T08:43:56Z

⚠️ No Changeset found

Latest commit: fdc20f4

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

coderabbitai · 2026-08-14T08:44:24Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 010f945c-8ee2-41e7-a121-f32d69678646

📥 Commits

Reviewing files that changed from the base of the PR and between d789149 and cc3213c.

📒 Files selected for processing (1)

internal-packages/clickhouse/schema/039_create_task_events_search_v2.sql

📜 Recent review details

⏰ Context from checks skipped due to timeout. (23)

GitHub Check: report
GitHub Check: webapp / 🧪 Unit Tests: Webapp (11, 12)
GitHub Check: webapp / 🧪 Unit Tests: Webapp (12, 12)
GitHub Check: webapp / 🧪 Unit Tests: Webapp (8, 12)
GitHub Check: webapp / 🧪 Unit Tests: Webapp (7, 12)
GitHub Check: webapp / 🧪 Unit Tests: Webapp (4, 12)
GitHub Check: webapp / 🧪 Unit Tests: Webapp (6, 12)
GitHub Check: webapp / 🧪 Unit Tests: Webapp (1, 12)
GitHub Check: webapp / 🧪 Unit Tests: Webapp (5, 12)
GitHub Check: webapp / 🧪 Unit Tests: Webapp (10, 12)
GitHub Check: webapp / 🧪 Unit Tests: Webapp (9, 12)
GitHub Check: webapp / 🧪 Unit Tests: Webapp (3, 12)
GitHub Check: webapp / 🧪 Unit Tests: Webapp (2, 12)
GitHub Check: obsmap / 🧪 Unit Tests: Observability Map
GitHub Check: fk-cascade-guard / fk-cascade-guard
GitHub Check: typecheck / typecheck
GitHub Check: internal / 🧪 Unit Tests: Internal
GitHub Check: e2e-webapp / 🧪 E2E Tests: Webapp
GitHub Check: runops-guard / runops-guard
GitHub Check: code-quality / code-quality
GitHub Check: audit
GitHub Check: audit
GitHub Check: Analyze (javascript-typescript)

🧰 Additional context used

📓 Path-based instructions (1)

internal-packages/clickhouse/schema/[0-9][0-9][0-9]_*.sql

📄 CodeRabbit inference engine (internal-packages/clickhouse/CLAUDE.md)

internal-packages/clickhouse/schema/[0-9][0-9][0-9]_*.sql: Migration file numbering: name files as 0(N+1)_descriptive_name.sql where N is the largest existing migration number in schema/; rebase and renumber if main adds migrations before opening a PR
DDL in migrations must be idempotent: use ALTER TABLE ... ADD COLUMN IF NOT EXISTS, CREATE TABLE IF NOT EXISTS, DROP TABLE IF EXISTS, ADD INDEX IF NOT EXISTS, DROP INDEX IF EXISTS, and CREATE MATERIALIZED VIEW IF NOT EXISTS forms to allow out-of-order and retry-safe application

Files:

internal-packages/clickhouse/schema/039_create_task_events_search_v2.sql

🔇 Additional comments (1)

internal-packages/clickhouse/schema/039_create_task_events_search_v2.sql (1)

3-3: LGTM!

Walkthrough

Added a ClickHouse V2 search table and projection pipeline with persistent state, leases, backfills, telemetry, scheduled processing, and admin controls. Updated log search normalization, minimum-length validation, bounded pagination, cursor handling, and period expansion. Centralized logs access checks across navigation and routes. Added ClickHouse command support, runtime configuration, database state storage, and integration tests.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 7.89% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	⚠️ Warning	The description explains the design, but it omits the required issue reference, checklist, testing, changelog, and screenshots sections.	Add the required template sections, include the issue reference and checklist, document testing steps, provide a changelog entry, and add screenshots or state that they are not applicable.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly identifies the main change: adding Logs search v2.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/log-search-v2

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-08-14T08:45:48Z

Observability map

As of fdc20f4.

19/100 over 443 measured of 459 entry points (base 19, no change)

What this PR changed

route	base	head	now failing
/admin/api/v1/logs-search-projector	new	0	request-context
/resources/orgs/:organizationSlug/projects/:projectParam/env/:envParam/logs	0	50

FIX FIRST

/api/v1/projects/:projectRef/envvars (sensitive) - auth-boundary, request-context
/auth/sso (sensitive) - auth-boundary, request-context
/_app/orgs/:organizationSlug/settings/team (sensitive) - error-classification, auth-scope, request-context

AUDIT 3 of 50 sensitive mutations record an actor. 47 without one.
CONTEXT 22 of 443 entry points name a tenant on a failure path. 342 appear only here, 39 of them sensitive, in the JSON rather than the fix list.

What the score is made of

CHECKS
  error-classification  179 applicable, 102 pass,   0 sole, global without it 12
  auth-boundary          62 applicable,  57 pass,   0 sole, global without it 16
  auth-scope             19 applicable,  17 pass,   0 sole, global without it 19
  request-context       443 applicable,  22 pass, 242 sole, global without it 64
  audit-trail            50 applicable,   3 pass,   0 sole, not in the score

The score and findings here are report-only and never gate the merge. Separately, a required test suite keeps this tool's symbol and route lists in sync with the code they name, and can fail a pull request that renames or removes a symbol they reference, or that adds the first route with a segment they anticipate. Each failure names the list to edit. The rules and their reasons: internal-packages/observability-map/README.md.

pkg-pr-new · 2026-08-17T14:41:04Z

Open in StackBlitz

@trigger.dev/build

npm i https://pkg.pr.new/@trigger.dev/build@7c8b416

trigger.dev

npm i https://pkg.pr.new/trigger.dev@7c8b416

@trigger.dev/core

npm i https://pkg.pr.new/@trigger.dev/core@7c8b416

@trigger.dev/python

npm i https://pkg.pr.new/@trigger.dev/python@7c8b416

@trigger.dev/react-hooks

npm i https://pkg.pr.new/@trigger.dev/react-hooks@7c8b416

@trigger.dev/redis-worker

npm i https://pkg.pr.new/@trigger.dev/redis-worker@7c8b416

@trigger.dev/rsc

npm i https://pkg.pr.new/@trigger.dev/rsc@7c8b416

@trigger.dev/schema-to-json

npm i https://pkg.pr.new/@trigger.dev/schema-to-json@7c8b416

@trigger.dev/sdk

npm i https://pkg.pr.new/@trigger.dev/sdk@7c8b416

commit: 7c8b416

Project closed source windows with durable watermarks and leases. Keep v2 reads and backfill disabled by default until sufficient history exists.

Fetch bounded extra rows and remove duplicate projection identities in the application. Keep exact keyset pagination while background merges collapse physical copies.

Create the scheduled-projector schema directly in migration 038 and remove the intermediate migration.

Renumber search_v2 table migration to 039 and drop the inserted_at index DDL, which now ships as standalone migration 038.

Read v2 directly, use the projector enable setting as the operational switch, and store initialization in append-only checkpoints.

devin-ai-integration

Devin Review found 1 new potential issue.

040 was already claimed by 040_create_task_events_search_v2.sql (#4615) by the time #4661 landed, so `goose up` panics with "duplicate version 40 detected" while collecting the migration directory and applies nothing. Renumber this one rather than the other: goose keys on the version number and not the filename, so version 40 is already recorded wherever 040 has been applied, and renaming that file would re-run an applied migration. This one has never been applied anywhere, because collection aborts before any SQL runs.

…#4734) ## Summary `goose up` against `internal-packages/clickhouse/schema` panics on `main` today, so ClickHouse migrations cannot be applied from a fresh checkout. Renumbering the external deployment id migration from 040 to 041 clears it. ## Root cause Two migrations claim version 40. [#4615](#4615) added `040_create_task_events_search_v2.sql`, and [#4661](#4661) added `040_add_task_runs_v2_external_deployment_id.sql` a day later. #4661 was opened before #4615 merged, so 040 was genuinely free at branch time, and because the two files have different names there is no textual conflict for git or a rebase to surface. Both merged green, and no workflow in this repo runs `goose`, so the collision only shows up the first time someone actually migrates. goose parses the numeric filename prefix as the version and refuses duplicates: ``` panic: goose: duplicate version 40 detected: .../040_create_task_events_search_v2.sql .../040_add_task_runs_v2_external_deployment_id.sql ``` It aborts while collecting the directory, before executing any SQL, so nothing was half applied and there is no migration state to repair. This migration gets renumbered rather than the `task_events_search_v2` one because goose keys on the version number and not the filename: version 40 is already recorded wherever 040 has been applied, so renaming that file would re-run an applied migration. Verified with a full `goose up` against ClickHouse 26.2.19.43 (the image pinned in `internal-packages/testcontainers`): migrations apply cleanly through version 41, and `task_runs_v2.external_deployment_id` lands as `String DEFAULT ''`.

This comment was marked as resolved.

Sign in to view

carderne marked this pull request as ready for review August 14, 2026 09:27

This comment was marked as resolved.

Sign in to view

carderne marked this pull request as draft August 14, 2026 13:23

This comment was marked as resolved.

Sign in to view

carderne force-pushed the feat/log-search-v2 branch from cc3213c to 70280f5 Compare August 17, 2026 14:11

carderne force-pushed the feat/log-search-v2 branch from 4ebd9ac to feafde8 Compare August 18, 2026 10:16

carderne marked this pull request as ready for review August 18, 2026 10:21

This comment was marked as resolved.

Sign in to view

carderne force-pushed the feat/log-search-v2 branch from e2b2c18 to 881478b Compare August 18, 2026 10:47

This comment was marked as resolved.

Sign in to view

carderne force-pushed the feat/log-search-v2 branch 2 times, most recently from fd3c5d8 to 7272e21 Compare August 18, 2026 11:37

carderne added 8 commits August 18, 2026 12:59

feat(webapp,clickhouse): add bounded global log search

afc53fc

feat(webapp,clickhouse): decouple log search indexing from event inserts

5bc48eb

Project closed source windows with durable watermarks and leases. Keep v2 reads and backfill disabled by default until sufficient history exists.

perf(webapp,clickhouse): preserve early exit when hiding log retries

51666e4

Fetch bounded extra rows and remove duplicate projection identities in the application. Keep exact keyset pagination while background merges collapse physical copies.

refactor(clickhouse): consolidate log search v2 schema migration

4c98dd2

Create the scheduled-projector schema directly in migration 038 and remove the intermediate migration.

fix coderabbit comments

5a30062

fix test

40ef542

respond to devin comments

e8c54ec

fix(webapp): make log search normalization locale independent

4491c19

carderne added 8 commits August 18, 2026 12:59

refactor(clickhouse): split inserted_at index into its own migration

4972f60

Renumber search_v2 table migration to 039 and drop the inserted_at index DDL, which now ships as standalone migration 038.

feat(webapp,clickhouse,database): add fast and finalized log projection

97db6b8

refactor(webapp): reuse logs ClickHouse connection for projection

d8eb4f4

fix(webapp): keep log projection writes off read replicas

c583e25

chore(webapp): remove unused projector exports

9eb55c3

fix(webapp): keep log projector lag metrics fresh

108031d

fix(webapp): return validation errors from logs pagination

03fdcfc

fix(database): use Prisma checkpoint index name

7c8b416

carderne force-pushed the feat/log-search-v2 branch from 7272e21 to 7c8b416 Compare August 18, 2026 11:59

This comment was marked as resolved.

Sign in to view

refactor(webapp,clickhouse,database): make logs search v2-only

fdc20f4

Read v2 directly, use the projector enable setting as the operational switch, and store initialization in append-only checkpoints.

devin-ai-integration Bot reviewed Aug 18, 2026

View reviewed changes

Comment thread internal-packages/clickhouse/src/taskEvents.ts

ericallam approved these changes Aug 18, 2026

View reviewed changes

carderne merged commit b4313c8 into main Aug 18, 2026
63 of 65 checks passed

carderne deleted the feat/log-search-v2 branch August 18, 2026 13:59

github-actions Bot mentioned this pull request Aug 18, 2026

chore: release v4.5.12 #4610

Merged

0ski mentioned this pull request Aug 20, 2026

fix(clickhouse): renumber the external deployment id migration to 041 #4734

Merged

Uh oh!

Uh oh!

Conversation

carderne commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Design

Uh oh!

changeset-bot Bot commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

coderabbitai Bot commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

❌ Failed checks (2 warnings)

Uh oh!

github-actions Bot commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Observability map

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

pkg-pr-new Bot commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

carderne commented Aug 14, 2026 •

edited

Loading

changeset-bot Bot commented Aug 14, 2026 •

edited

Loading

coderabbitai Bot commented Aug 14, 2026 •

edited

Loading

github-actions Bot commented Aug 14, 2026 •

edited

Loading

pkg-pr-new Bot commented Aug 17, 2026 •

edited

Loading