Skip to content

Add pscale insights and pscale inspect commands for database diagnostics#1308

Open
nickvanw wants to merge 8 commits into
mainfrom
nick/inspect-insights
Open

Add pscale insights and pscale inspect commands for database diagnostics#1308
nickvanw wants to merge 8 commits into
mainfrom
nick/inspect-insights

Conversation

@nickvanw

@nickvanw nickvanw commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds two complementary read-only diagnostic surfaces to the CLI, for both MySQL/Vitess and PostgreSQL databases:

  • pscale insights — surfaces PlanetScale's server-side analysis of production traffic, which until now was only reachable through the MCP server or raw pscale api calls:
    • insights queries <db> <branch> — top queries ranked by any insights metric (--sort totalTime|count|p99Latency|rowsRead|rowsReadPerReturned|…, plus --period, --dir, --limit)
    • insights errors <db> <branch> — failing query patterns with error messages
    • insights anomalies <db> <branch> — detected resource anomalies
    • insights recommendations <db> — schema recommendations with ready-to-apply DDL
  • pscale inspect — live, point-in-time checks run over a direct connection using the same ephemeral-credential model as pscale sql (always read-only, always bounded): table-sizes, index-sizes, unused-indexes, redundant-indexes, invalid-indexes, seq-scans, long-running-queries, locks, outliers, calls, bloat, vacuum-stats, replication-slots, subscriptions — and inspect all, which runs every applicable check over a single connection and prints a combined report.

All commands support --format json (and csv for single checks) for scripting and agents. Every command's --help ends with a footer pointing agents at pscale agent-guide --format json, and the embedded guide is readable via pscale help agents.

Why

Users (and increasingly, AI agents) diagnosing a slow or unhealthy database today have to bounce between the web app, the MCP server, and hand-written SQL through pscale shell. The platform already computes rich, traffic-aware analysis — query stats, anomalies, schema recommendations — but none of it was exposed as first-class CLI commands, and there was no safe, canned way to ask the database itself the standard DBA questions (what's big, what's unused, what's locked, what's bloated).

This gives both audiences a single obvious front door. The two surfaces deliberately cross-reference each other (help text, human output, and next_steps in JSON) so an agent that lands on either one discovers the other: insights for historical, traffic-aware analysis; inspect for live state and physical layout. When a check isn't available for an engine — or needs a PostgreSQL extension that isn't installed (e.g. outliers needs pg_stat_statements) — the output points at the insights command that answers the same question server-side with no setup.

Changes

  • internal/planetscale/insights.go — new QueryInsights service (ListQueries/ListErrors/ListAnomalies) plus WithSort/WithPeriod list options; the already-present SchemaRecommendations service is now actually used by a command
  • internal/cmd/insights/ — the four insights subcommands
  • internal/cmd/inspect/ — check catalog (per-engine SQL with invariants enforced by tests: single statement, read-only, LIMIT-bounded, system/internal schemas filtered) and the command tree
  • internal/sqlquery/session.go — reusable Session (one connection, many queries) extracted from Execute; connection setup for both engines refactored into openMySQL/openPostgres with no behavior change to pscale sql
  • internal/printer — expose the resource writer so commands with dynamic column sets can render CSV
  • AGENTS.md — documents both surfaces and when to use which

The PostgreSQL query catalog was sharpened from review feedback:

  • table-sizes uses pg_table_size (heap + TOAST + FSM + VM; indexes are itemized by index-sizes, so counting them here would double-count), rolls partitions up into their root, and includes materialized views
  • index-sizes names indexes as table.index
  • long-running-queries excludes walsenders (replication connections are "active" forever and made the check permanently noisy) and bounds query text
  • locks reports only the roots of blocking trees via pg_blocking_pids() with a count of sessions stuck behind each, instead of dumping every granted lock
  • bloat covers index bloat as well as table bloat, with object size and a bloat percentage
  • vacuum-stats honors per-table autovacuum reloptions (including autovacuum_enabled=false, reported as disabled) and tracks analyze alongside vacuum
  • replication-slots separates retained WAL from unconfirmed WAL and adds wal_status/safe_wal_size (headroom before max_slot_wal_keep_size invalidates the slot)
  • invalid-indexes is a new check for indexes left invalid by failed CREATE INDEX CONCURRENTLY builds

Notes for reviewers:

  • On sharded Vitess databases, inspect statistics reflect one shard's MySQL instance per run. --keyspace accepts vtgate shard targets (mykeyspace/-80, mykeyspace/-80@replica) to pin an exact shard — shard-targeted connections bypass the planner and read that shard's statistics directly. Enumerate shards with SHOW VITESS_SHARDS. Databases can have hundreds of shards, so nothing fans out automatically.
  • On PostgreSQL, stats are scoped to one database; --dbname selects it, and the reader role can lack CONNECT on non-default databases — the error suggests --role admin.
  • All queries verified live against a sharded Vitess database and PostgreSQL databases (all three output formats, extension-missing and engine-mismatch paths).

nickvanw added 2 commits July 22, 2026 18:22
pscale insights surfaces PlanetScale's server-side analysis of production
traffic: top queries by latency/rows-read/time (queries), failing queries
(errors), resource anomalies (anomalies), and schema recommendations with
ready-to-apply DDL (recommendations). The insights API endpoints were
previously only reachable via the MCP server or raw pscale api calls; this
adds a typed QueryInsights service to the internal SDK and wires the
already-present-but-unused SchemaRecommendations service to a command.

pscale inspect runs read-only, bounded diagnostic checks over a direct
connection using the same ephemeral-credential model as pscale sql:
table/index sizes, unused and redundant indexes, full-table scans,
long-running queries, locks, bloat/fragmentation, vacuum stats, and
replication state. Checks adapt per engine (information_schema/sys for
MySQL/Vitess, pg_catalog/pg_stat for PostgreSQL); checks that don't apply
to an engine point at the equivalent surface instead. pscale inspect all
runs every applicable check over a single connection and prints a combined
report.

The two surfaces cross-reference each other in help text, human output,
and JSON next_steps so agents discover both: insights for traffic-aware,
historical analysis; inspect for live state and physical layout.

Supporting changes: sqlquery gains a reusable Session (one connection,
many queries) extracted from Execute; the printer exposes its resource
writer for commands that render dynamic column sets.
Agents that discover pscale via --help (rather than a repo AGENTS.md) had
no pointer to the embedded agent guide outside the root command's help and
JSON error next_steps. Every command's help/usage output now ends with a
footer pointing at pscale agent-guide, and the full guide is readable as a
help topic via pscale help agents.

--keyspace on sql and inspect now accepts vtgate shard targets, e.g.
mykeyspace/-80 or mykeyspace/-80@replica. Shard-targeted connections
bypass the query planner and send statements directly to that shard's
MySQL instance, so inspect checks can read one specific shard's
statistics on sharded databases (enumerate shards with SHOW
VITESS_SHARDS; rows are keyspace/shard). Databases can have hundreds of
shards, so nothing fans out automatically — one target per run. The DSN
is now built with the driver's Config/FormatDSN so the slash and at-sign
in shard targets survive; previously they produced an invalid-DSN error.
@nickvanw
nickvanw force-pushed the nick/inspect-insights branch from 03c5895 to 25cac7b Compare July 23, 2026 01:35
@nickvanw
nickvanw force-pushed the nick/inspect-insights branch from 7438a71 to 19ee257 Compare July 23, 2026 15:19
nickvanw added 2 commits July 23, 2026 13:03
table-sizes rolls partitions up into their root (a 100-partition table
now reads as one entity, not 100 rows) and includes materialized views,
which also consume storage.

index-sizes includes the table each index belongs to; an index name
alone is often ambiguous.

long-running-queries excludes walsenders — replication connections stay
'active' forever and made the check permanently noisy on any database
doing CDC — and bounds query text to 200 chars like the MySQL variant.

locks now reports only the roots of blocking trees (via
pg_blocking_pids) with a count of sessions stuck behind each, instead of
dumping every granted lock; routine AccessShareLocks drowned out the
signal.

bloat covers index bloat in addition to table bloat and adds object size
and a bloat percentage, so the waste column has context.

vacuum-stats honors per-table autovacuum reloptions (including
autovacuum_enabled=false, reported as 'disabled') instead of only the
global settings, and tracks analyze alongside vacuum.

replication-slots separates retained WAL (restart_lsn) from unconfirmed
WAL (confirmed_flush_lsn) — they measure different failure modes — and
adds wal_status and safe_wal_size, the headroom before
max_slot_wal_keep_size invalidates the slot.

New invalid-indexes check finds indexes left invalid by failed CREATE
INDEX CONCURRENTLY builds: they consume space, are never used, and are
easy to miss.

All queries verified against live databases on both engines.
Index storage is itemized by the index-sizes check (and index bloat by
the bloat check), so including indexes in table-sizes counted the same
bytes twice across checks. pg_table_size reports the table's own
storage: heap + TOAST + FSM + VM.
@nickvanw
nickvanw marked this pull request as ready for review July 23, 2026 20:13
@nickvanw
nickvanw requested a review from a team as a code owner July 23, 2026 20:13
Comment thread internal/cmd/inspect/inspect.go
Comment thread internal/cmd/inspect/inspect.go
Bugbot flagged two gaps in human output: a skipped check printed only
the catalog hint, whose embedded command lacks --org and fails when
pasted, even though runCheck had already built fully-flagged NextSteps;
and inspect all never showed the per-check follow-ups (e.g. the
rowsReadPerReturned sort for seq-scans), only the generic footer.

Skipped checks now print their next steps in both single-check and
combined output, and combined-report sections show a compact one-line
follow-up when a check found something. Empty sections stay quiet — the
report footer already covers the generic pointers.
Comment thread internal/cmd/inspect/inspect.go
When a check errored (timeout, permission, SQL failure), the synthetic
result carried the error text but dropped the check's next steps, so
JSON consumers lost the cross-reference exactly when the live check
couldn't answer — the case where the server-side alternative matters
most.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7d4f593. Configure here.

Comment thread internal/cmd/inspect/checks.go
The check listed every user table ordered by seq_scan, including rows
with zero scans, implying activity where none occurred — and the empty
state could never appear. The MySQL variant already returns only tables
that were actually scanned.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants