Skip to content

feat(cli): add supabase config validate (CLI-2866) - #7553

Open
Coly010 wants to merge 16 commits into
developfrom
columferry/cli-2866-add-supabase-config-validate
Open

Coly010 wants to merge 16 commits into
developfrom
columferry/cli-2866-add-supabase-config-validate

Conversation

@Coly010

@Coly010 Coly010 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Current Behavior

There's no way to check a config file on its own. Problems only show up when some other command loads the config, and the load stops at the first one it hits, so fixing a broken config means running a command, fixing one thing, and running it again.

Expected Behavior

supabase config validate checks the selected config file (config.toml, config.json or config.jsonc) through the same load and validateResolvedConfig path every command uses, and reports every problem it finds in one go:

  • syntax errors, the schema version check, the two-files rules, removed keys, unknown keys (with a "did you mean" suggestion), type/enum/value errors and the cross-field rules from config-validate.ts
  • every [remotes.*] block, each validated as its own merged config and labelled with the remote it came from
  • the file, line and column of each problem, worked out from the raw TOML/JSON text

Text output follows the config convert report style from #7517: one block per problem with the key, what's wrong, its file:line:column, and a help: line where there's a fix to suggest. It never prints source text or configured values, so a report is safe to paste into an issue or keep in a CI log; ADR 0034 records that rule for config diagnostics. --output-format json emits a versioned payload with codes, key paths and positions.

Unresolved env() references are warnings rather than errors, since the variable may only exist in CI or on the hosted project. The command exits 1 when there are errors and needs no network, Docker, login or linked project.

A few things worth calling out for review:

  • config-validate.ts now collects every issue instead of throwing on the first. The throwing wrappers that other commands use keep their existing messages, and the parity test covers both pipelines.
  • Having both config.toml and config.json is an error in validate, while other commands still warn and pick one. It felt wrong for a validator to pass a project that's ambiguous about which file it reads, but happy to change that if it's too strict.
  • @supabase/config/internal gains scanCliConfigDocument, which locates a key in raw TOML/JSON/JSONC text, and the TOML scanner from config-edit.ts is now exported. feat(cli): add supabase config convert (CLI-2867) #7517 moves that scanner into its own file, so whichever lands second needs a one-line import change.
  • A run that finds problems exits 1, and telemetry records it as a failed command classified as a user-actionable invalid_config error (ConfigValidateProblemsError), the same way db lint classifies its failures.

Related Issue(s)

Fixes CLI-2866

Validates the selected config file (TOML, JSON or JSONC) through the same
load and validateResolvedConfig path commands use, including every
[remotes.*] block, and reports every problem with its file, line and
column instead of stopping at the first.

Text output renders each problem with a masked source frame and a
suggestion; --output-format json emits a versioned payload of codes,
key paths and positions. Unresolved env() references are warnings.
The command exits 1 when the config has errors and needs no network,
Docker, login or linked project.
@Coly010 Coly010 self-assigned this Nov 10, 2026
@Coly010
Coly010 marked this pull request as ready for review November 10, 2026 13:24
@Coly010
Coly010 requested a review from a team as a code owner November 10, 2026 13:24
@Coly010
Coly010 requested a review from jgoux November 10, 2026 13:24
@github-actions

Copy link
Copy Markdown
Contributor

CLI benchmark (Lightweight Commands): 1 thing to check

Commit 82cc401 against develop at 999550e · 10 runs each · report only, never blocks merging

⚠️ supabase config validate exits with different codes before and after, so its timing is not compared.

⚠️ supabase config validate exited with an error before; its timing is not compared.

Command Workload Before After Change
supabase --version version 210 ms 227 ms no real change (+9%, within normal variation)
supabase --help help 243 ms 246 ms no real change
supabase init init 222 ms 244 ms no real change (+10%, within normal variation)
supabase migration new bench migration-new 220 ms 210 ms no real change
supabase config validate config-validate 197 ms 224 ms not compared
supabase gen types --lang typescript --project-id replayfixtureprojref gen-types 191 ms 195 ms no real change
supabase projects list projects-list 191 ms 193 ms no real change
supabase functions list --project-ref replayfixtureprojref functions-list 241 ms 237 ms no real change

API calls: unchanged for all 8 commands.

Binary size: 135.0 MB, up 112.0 KB from 134.9 MB.

Details: timing ranges, API calls per command, slowest phases

Timing ranges

Command Workload Before range After range Change Verdict
supabase --version version 181-290 ms 187-361 ms +18 ms (+9%) within noise
supabase --help help 213-326 ms 198-306 ms +3 ms (+1%) within noise
supabase init init 210-265 ms 222-267 ms +21 ms (+10%) within noise
supabase migration new bench migration-new 200-228 ms 200-244 ms -11 ms (-5%) within noise
supabase config validate config-validate 157-261 ms 178-304 ms +26 ms (+13%) not comparable
supabase gen types --lang typescript --project-id replayfixtureprojref gen-types 181-287 ms 188-235 ms +3 ms (+2%) within noise
supabase projects list projects-list 170-261 ms 175-255 ms +3 ms (+1%) within noise
supabase functions list --project-ref replayfixtureprojref functions-list 201-353 ms 183-390 ms -4 ms (-2%) within noise

API calls per command

Command Workload Before After Endpoint changes
supabase --version version 0 0
supabase --help help 0 0
supabase init init 0 0
supabase migration new bench migration-new 0 0
supabase config validate config-validate 0 0
supabase gen types --lang typescript --project-id replayfixtureprojref gen-types 2 2
supabase projects list projects-list 1 1
supabase functions list --project-ref replayfixtureprojref functions-list 2 2

Slowest phases

No phase deltas are available; some commands were not comparable.

Workflow run

Comment /bench to run the lightweight benchmark (this comment) again on the latest commit, or /bench full to also run the integration-heavy benchmark, which posts in its own comment.

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by a newer AI review

🤖 AI Review

Verified all nine input findings against the code and merged the duplicate multiline-secret finding into one entry. Eight findings are confirmed: two critical security issues, three major validation issues, and three minor diagnostic issues. Isolated probes reproduced the disclosure, suppression, inheritance, and source-location defects; the full test suite was unavailable because dependencies are absent.

Findings

Severity Location Category Sources Claim
🔴 CRITICAL apps/cli/src/commands/config/validate/validate.format.ts:89 security claude+codex Code frames disclose multiline TOML secrets when continuation lines resemble comments, headers, or keys. Comment-looking secret lines print unchanged, and key-looking lines expose their prefix.
🔴 CRITICAL apps/cli/src/commands/config/validate/validate.handler.ts:589 security codex Unresolved-reference warnings can print sibling secrets when source lookup falls back to an ancestor value.
🟠 MAJOR apps/cli/src/commands/config/validate/validate.model.ts:183 validation codex An unresolved env reference suppresses unrelated errors at the same key, allowing invalid configurations to pass validation.
🟠 MAJOR apps/cli/src/config/cli-config-values.layer.ts:427 validation codex A non-table remotes value bypasses structural validation and can be reported as valid.
🟠 MAJOR apps/cli/src/commands/config/validate/validate.model.ts:186 env-resolution codex Unresolved base references inherited by a remote produce hard errors instead of warnings when the remote enables their section.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:465 error-handling claude Environment-loading failures discard the actual file and cause, always reporting supabase/.env* as unreadable even for root-level files or parse failures.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:263 error-reporting codex Semantic errors caused by SUPABASE_* overrides are incorrectly attributed to config-file values.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:180 source-location codex Literal dots in dynamic key names cause diagnostics to point at the wrong source location.

Stats

Claude findings: 2 · Codex findings: 7 · Confirmed: 8 · Refuted: 0 · Uncertain: 0


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: auto · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

- Mask code frames from parsed value spans so multi-line strings,
  comment-looking continuation lines and syntax-error documents never
  print a value; show a located value only when the location is exact.
- Suppress only value errors under an unresolved env() reference, never
  unknown keys or SUPABASE_* override failures, and evaluate inherited
  base references per remote.
- Report a non-table remotes or remotes.<name> as a type error.
- Name the real env file and whether it failed to read or parse.
- Attribute semantic errors from SUPABASE_* overrides to the variable.
- Keep key path segments so dotted table names locate correctly.
@Coly010

Coly010 commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

/ai-review

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by a newer AI review

🤖 AI Review

Both independent reviews were available. All 11 submitted findings were verified, yielding 10 confirmed findings after merging the duplicate typo. These include two secret-disclosure paths, an unresolved reference that incorrectly fails validation, four diagnostic correctness issues, and three nits. The help-text finding is narrowed because Postgres-backend users do not receive the reported analytics errors.

Findings

Severity Location Category Sources Claim
🔴 CRITICAL apps/cli/src/commands/config/validate/validate.format.ts:158 security codex A malformed value at a public setting can expose nested secrets in text reports. For example, api.max_rows = { secret = "hunter2" } exempts the nested secret from masking.
🔴 CRITICAL apps/cli/src/commands/config/validate/validate.model.ts:80 security claude URL-valued public settings can expose embedded credentials in text reports, including when a valid credential-bearing URL appears immediately before another problem.
🟠 MAJOR apps/cli/src/commands/config/validate/validate.model.ts:195 correctness codex An unset env() reference used as a remote project_id produces an error and exit code 1 instead of warning-only unresolved-reference behavior.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:178 correctness claude A remote missing project_id is reported against the unrelated base project_id value when that base key exists.
🟡 MINOR apps/cli/src/command-internal/config-validate.ts:291 error-messages claude Generated missing-key diagnostics omit the BigQuery condition for analytics.gcp_project_id and gcp_project_number, and give ambiguous advice to set enabled = false for WebAuthn fields whose switch actually belongs to auth.passkey.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:650 correctness codex Unresolved-reference warnings are emitted even when a SUPABASE_* override replaces the reference, incorrectly claiming the effective key contains literal env(NAME) text.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:521 error-handling codex A removed environment-variable error prevents subsequent schema and semantic validation while the final JSON report still claims complete: true.
⚪ NIT apps/cli/src/commands/config/validate/validate.handler.ts:431 code-quality claude The schema-version error branch reads the same config source twice.
⚪ NIT apps/cli/src/commands/config/validate/validate.format.ts:262 output-formatting codex A long invalid public value produces a caret underline extending beyond its clipped source frame.
⚪ NIT apps/cli/src/config/cli-config-remote.ts:46 documentation claude+codex The new JSDoc ends with the punctuation typo "one failure per remote,.".

Stats

Claude findings: 5 · Codex findings: 6 · Confirmed: 10 · Refuted: 0 · Uncertain: 0


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: manual · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

- Show a located value only when it is a scalar or a flat array of
  scalars, and redact URL credentials and query strings in any shown
  value or context line.
- Skip a remote whose project_id is an unset env() reference with a
  warning, and locate remote project_id failures inside that remote.
- Stay quiet about an unresolved reference a SUPABASE_* variable
  overrides, and report a removed env variable as an incomplete run.
- Name the bigquery backend and auth.passkey.enabled in missing-key help.
- Clamp the caret to the windowed frame and read the source once on a
  schema-version stop.
@Coly010

Coly010 commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

/ai-review

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by a newer AI review

🤖 AI Review

Verified all nine findings: five distinct secret-disclosure paths, two diagnostic correctness issues, and two conditional defensive-path diagnostic gaps. Raised the JSON context disclosure to critical. Source inspection and isolated scanner/formatter probes supported the findings; the full test suite was unavailable because dependencies are absent.

Findings

Severity Location Category Sources Claim
🔴 CRITICAL apps/cli/src/commands/config/validate/validate.format.ts:296 security claude JSON/JSONC context lines can expose a secret when the line finishes an earlier property's multiline value and starts a public setting.
🔴 CRITICAL packages/config/src/config-locate.ts:97 security codex Triple-quoted TOML strings inside inline tables can expose parts of secrets in code frames.
🔴 CRITICAL packages/config/src/config-locate.ts:321 security codex Nested TOML arrays are classified as safe context lines, exposing their nested values.
🔴 CRITICAL apps/cli/src/commands/config/validate/validate.format.ts:91 security codex Valid JSON string escapes can bypass URL credential and query-string redaction.
🔴 CRITICAL apps/cli/src/commands/config/validate/validate.handler.ts:497 security codex Unrecognized schema warnings disclose credentials and query secrets embedded in schema URLs in both text and JSON output.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:334 error-attribution codex Invalid remote project-ID environment overrides are incorrectly attributed to the config file.
🟡 MINOR packages/config/src/config-locate.ts:129 source-location codex Escaped TOML keys inside inline tables cannot be located by their decoded config paths.
🟡 MINOR apps/cli/src/config/cli-config-values.layer.ts:604 error-handling claude If validation pruning reaches MAX_PRUNE_ROUNDS without converging, earlier schema issues are discarded and the resulting incomplete validation is reported without an incomplete marker.
⚪ NIT apps/cli/src/command-internal/local-config-values.ts:874 error-handling claude The collector's defensive catch-all produces a pathless could not be checked problem with no actionable explanation.

Stats

Claude findings: 3 · Codex findings: 6 · Confirmed: 9 · Refuted: 0 · Uncertain: 0


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: manual · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread packages/config/src/config-locate.ts Resolved
Comment thread packages/config/src/config-locate.ts Resolved
Comment thread packages/config/src/config-locate.ts Resolved
- Show a value only when it is the sole value on its line and a plain
  single-line scalar; mask arrays, tables, escaped and multi-line
  strings, and treat an inline table holding a triple-quoted string as
  one span.
- Give the unrecognised schema warning a value-free reason.
- Report an invalid env-sourced remote project_id against its variable.
- Decode escaped inline-table keys when locating a path.
- Stop with an explicit schema_limit stage, keeping the issues found,
  when schema pruning does not converge.
- Give the collector's fallback issue value-free help.
@Coly010

Coly010 commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

/ai-review

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by a newer AI review

🤖 AI Review

Verified all nine findings against the checked-out code; all are confirmed and none overlap. The principal issues are credential disclosure in code frames and suppressed decryption failures. Focused executions reproduced these and the diagnostic bugs. The cache finding is limited to retaining the most recently scanned document. Full test suites were unavailable because dependencies are not installed.

Findings

Severity Location Category Sources Claim
🔴 CRITICAL apps/cli/src/commands/config/validate/validate.format.ts:98 security codex URL credentials and query secrets containing apostrophes can print unredacted in code frames. A public setting containing "https://user:pw'secret@host.test/x?token=tok-secret" exposes the entire URL when shown as context.
🟠 MAJOR apps/cli/src/commands/config/validate/validate.handler.ts:266 error-handling codex An environment-supplied secret that fails decryption can be suppressed as an unresolved file reference, allowing validation to report success despite the invalid override.
🟡 MINOR apps/cli/src/commands/config/validate/validate.model.ts:82 redaction codex The public-value allowlist conflates literal dotted keys with nested schema paths, allowing unknown values to print in context frames.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:715 diagnostics codex Disabled custom auth providers produce unresolved-env warnings because a more distant gate in the decoded config overrides the nearer gate in the raw document.
🟡 MINOR apps/cli/src/command-internal/config-validate.ts:669 diagnostics codex Invalid function names containing dots can point to another function's source location because their path segments are lost.
🟡 MINOR apps/cli/docs/supabase/config/validate.md:9 documentation claude The documentation reverses which file is shadowed when JSON and TOML coexist, contradicting the same page's later explanation.
⚪ NIT apps/cli/src/commands/config/validate/validate.handler.ts:131 error-messages claude Removed-table rename hints inside remotes omit the remote prefix, so following the displayed TOML header literally targets a base table.
⚪ NIT apps/cli/src/commands/config/validate/validate.handler.ts:262 consistency claude Codec failures always receive invalid_value, including type and enum failures that the schema pipeline distinguishes as invalid_type or invalid_enum. Enum expectations also use inconsistent quoting.
⚪ NIT packages/config/src/config-locate.ts:236 code-quality claude The module-level locator cache strongly retains the most recently scanned config text, including secrets, until another source replaces it or the process exits.

Stats

Claude findings: 4 · Codex findings: 5 · Confirmed: 9 · Refuted: 0 · Uncertain: 0


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: manual · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread packages/config/src/config-locate.ts Resolved
- Print a URL value only after parsing it, with userinfo, query and
  fragment redacted; mask the whole value when it does not parse or
  carries @, ? or # without a scheme.
- Treat every flag, shell or project env failure, including a failed
  decryption, as external so an unresolved file reference cannot hide it.
- Keep segment boundaries in the public-value check, the gate lookup
  and dynamic function and provider names.
- Use remote-scoped headers in rename hints and classify codec failures
  as invalid_type, invalid_enum or invalid_value.
- Replace the module-level scan cache with a caller-owned document scan.
- Correct which file the docs say is shadowed.
Code frames now mask every value and the text of every comment, so a
report never prints configured values. This removes the public-value
allowlist, URL redaction and per-problem value display, and the
isSecretPath export from @supabase/config/internal that only they used.
Text output now gives each problem's key path, reason, file:line:column
and help, and never prints source text or values. Masking source lines
could not be made reliable across every TOML, JSON and JSONC encoding of
a value, so the code frames and their value-span scanning are removed.
ADR 0034 records the rule for config diagnostics.
@Coly010

Coly010 commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

/ai-review

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by a newer AI review

🤖 AI Review

Reconciled all eight findings into seven entries: five confirmed, one refuted, and one uncertain. Confirmed issues concern hidden remote validation errors, incorrect source locations, repeated JSON parsing, and outdated documentation. Custom-provider boolean references are handled upstream, refuting that finding. The potential schema-message disclosure remains uncertain because the Effect formatter's handling of custom filter messages could not be verified.

Findings

Severity Location Category Sources Claim
🟠 MAJOR apps/cli/src/commands/config/validate/validate.handler.ts:682 correctness codex A remote table override can have its validation error suppressed by an unresolved base env() reference.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:262 correctness claude Splitting CliConfigValueError.path on dots loses dynamic key boundaries and can locate an unrelated key as an exact match.
🟡 MINOR packages/config/src/config-locate.ts:97 source-location codex Triple-quoted TOML strings can make inline-table diagnostics point into string contents and mark the location as exact.
🟡 MINOR packages/config/src/config-locate.ts:301 performance codex Every JSON diagnostic lookup reparses the entire config document and rebuilds its line index.
🟡 MINOR apps/cli/docs/supabase/config/validate.md:34 documentation claude+codex The text-report example includes source lines and a caret frame that the command does not emit, contradicting the page's stated output behavior.
🟡 MINOR apps/cli/src/config/cli-config-schema-issues.ts:71 security claude The classifier fallback omits quote sanitization, potentially exposing configured values when a schema filter's value-bearing message bypasses the leaf hook.
Refuted findings (kept for transparency, not posted as review comments)
  • apps/cli/src/command-internal/local-config-values.ts:814 (correctness): Unset boolean env() references in custom auth providers become hard errors instead of warnings.
    Refuted: The claimed failure is handled before the cited catch. Validation enables tolerateInvalid at cli-config-values.layer.ts:743, and the registry rejects and prunes each unresolved boolean at its full field path. rawUnmodeledBool receives undefined and returns false at local-config-values.ts:744. The remaining field-level error matches the unresolved reference and is suppressed by validate.model.ts:157-166. Isolated probes verified both pruning and suppression.

Stats

Claude findings: 3 · Codex findings: 5 · Confirmed: 5 · Refuted: 1 · Uncertain: 1


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: manual · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread packages/config/src/config-locate.ts Resolved
Comment thread packages/config/src/config-locate.ts Resolved
…ating

- A remote that replaces an unset base env() with a table reports its own errors.
- Value errors carry key segments, so names containing dots locate correctly.
- The TOML scanner recognises triple-quoted strings inside inline tables.
- JSON documents are parsed once per scan handle.
- A present key with an invalid value is no longer also reported as missing.
- The docs example shows no source text, matching ADR 0034.
@Coly010

Coly010 commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

/ai-review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

Both independent reviews were available. All eight findings are confirmed and distinct. Each is a minor correctness concern; the cache invalidation issue is latent with current callers. Isolated helper checks reproduced four cases. The full test suite was not run because dependencies are absent.

Findings

Severity Location Category Sources Claim
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:666 correctness claude A remote with a duplicate but well-formed project_id is skipped as having an invalid project_id, leaving its block's type and semantic problems unchecked.
🟡 MINOR apps/cli/src/commands/config/validate/validate.format.ts:172 output-contract claude An environment-provider failure without an associated file emits "file": "environment variables" in the versioned JSON payload, placing a display label in a file-path field.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:748 correctness codex Unresolved references that cause value errors produce warnings inside disabled sections. For example, api.enabled = false with max_rows = "env(UNSET_ROWS)" warns despite the disabled-section warning gate.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:651 correctness codex A remote project_id reference produces an unresolved warning even when a SUPABASE_REMOTES_* override supplies the rejected value. With project_id = "env(UNSET_REF)" and SUPABASE_REMOTES_PROD_PROJECT_ID=nope, the report incorrectly advises setting UNSET_REF alongside the override error.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:121 correctness codex Removed-key diagnostics misidentify remote names containing dots and generate a rename instruction that changes TOML table structure. The section [remotes."prod.eu".inbucket] is attributed to remote prod with path eu.inbucket.
🟡 MINOR apps/cli/src/commands/config/validate/validate.handler.ts:347 correctness codex Duplicate project IDs are diagnosed as malformed refs when either remote name contains a closing bracket, causing advice to replace an already valid project ref.
🟡 MINOR packages/config/src/config-locate.ts:88 correctness codex TOML header spans end prematurely when a quoted key contains a closing bracket. For [storage.buckets."photos]archive"], the reported span stops inside the quoted bucket name.
🟡 MINOR apps/cli/src/config/cli-config-values.layer.ts:732 caching claude invalidate and writeThrough leave validationParsedDocuments cached, so a subsequent validation load in the same service instance can read stale config after a write. Current validation callers do not write, making this latent.

Findings outside the diff

  • 🟡 MINOR apps/cli/src/config/cli-config-values.layer.ts:732 — invalidate and writeThrough leave validationParsedDocuments cached, so a subsequent validation load in the same service instance can read stale config after a write. Current validation callers do not write, making this latent.

Stats

Claude findings: 3 · Codex findings: 5 · Confirmed: 8 · Refuted: 0 · Uncertain: 0


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: manual · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread packages/config/src/config-locate.ts Resolved
…add-supabase-config-validate

# Conflicts:
#	apps/cli/bench/workloads.pr.json
#	apps/cli/docs/tracing-monitoring.md
#	apps/cli/src/commands/config/config.command.ts
#	packages/config/src/config-edit.ts
…names

- Remote failures carry a structured kind; a duplicate project_id still validates its remote.
- Removed-key diagnostics carry segmented paths and the remote, quoting TOML segments in hints.
- TOML header spans end at the key path, so a quoted `]` no longer truncates them.
- Unresolved env() warnings respect the enabled gate for every reference and skip remote
  project_ids supplied by their override variable.
- JSON problems omit `file` when no file is associated.
- Invalidating config values also clears the validation document cache.

@jgoux jgoux left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The command covers a lot of ground, and the output is clean: I couldn't get any configured value or source text to leak into either output format.

Requesting changes for two validation-contract problems (inline):

  • A skipped [remotes.*] block (unset env() or invalid project_id) hides its other errors, and the run reports valid/complete.
  • An unresolved env() reference still fails the run when a cross-field check depends on it.

The rest are P3 diagnostics and docs issues. Every scenario was reproduced by running config validate from source at 44d27f9.

Comment thread packages/config/src/config-locate.ts Resolved
…failing a run

- Every [remotes.*] block is validated even when its project_id is unusable; the
  skipped_remotes field is gone.
- A cross-key error that depends on an unresolved env() reference is suppressed.
- Keys inside [auth.passkey] and [auth.webauthn] are checked against the schema.
- Provider `enabled` problems are reported under the key the document uses.
- Duplicate JSON keys locate the occurrence the parser keeps.
- A run with problems is recorded as an invalid_config failure (ConfigValidateProblemsError).
- env() warnings check overrides on ancestor keys and read gates from the document
  when a table was pruned.
- The empty enum choice renders as "", and the content_path help names the right key.
@Coly010
Coly010 requested a review from jgoux November 12, 2026 09:42
…add-supabase-config-validate

# Conflicts:
#	apps/cli/src/config/cli-config-values.layer.ts

@jgoux jgoux left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 1cefeef (c56ab3d plus a clean merge of develop).

All ten findings from my previous review are fixed. I reproduced each one again from source, and the affected suites pass. Replies are on those threads.

The new cross-key suppression introduces three regressions where an invalid config reports valid: true and exits 0 (inline, P2). There are also a few smaller items, including a correction to a help-text suggestion I made last round.

…eck reads

- Cross-key validation issues declare the keys they read; an unresolved env() suppresses
  only issues that read it, so independent errors under a gated table still fail.
- References replaced by a SUPABASE_* override no longer suppress errors.
- Suppression runs per pass before deduplication, so a remote's own error survives.
- An unset env() on an `enabled` key reads its gate from the parent table.
- The content_path help points at an existing file under the project root.
@Coly010
Coly010 requested a review from jgoux November 12, 2026 10:14

@jgoux jgoux left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 93200c8.

All six findings from my previous review are fixed. I ran each case again from source, and replied on those threads.

Moving suppression to per-check reads dropped the gate for checks that run outside collectConfigValidationIssues's annotated ranges. An unset env() on api.enabled or auth.enabled now fails the run on file-read and custom-provider errors that don't occur when that switch is off (two P2s, inline).

- File-read issues (api.tls files, signing keys, email template content) declare the
  enabled keys their read runs behind.
- Auth-gated sms and external provider issues declare auth.enabled.
- Schema missing_required problems carry their gate.
…add-supabase-config-validate

# Conflicts:
#	apps/cli/src/command-internal/config-validate.ts
…add-supabase-config-validate

# Conflicts:
#	apps/cli/src/config/cli-config-failure-key.unit.test.ts
#	packages/config/src/removed-config-keys.ts

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants