Repository navigation
Conversation
|
Important Review skippedToo many files! This PR contains 773 files, which is 473 over the limit of 300. To get a review, reduce the PR to 300 files or fewer by splitting it into smaller PRs or changing its base branch. Usage-priced reviews support at most 300 files.
 |
zinic
force-pushed
the
pg-v2
branch
3 times, most recently
from
August 25, 2026 21:06
3609d9e to
5149abc
Compare
zinic
force-pushed
the
pg-v2
branch
2 times, most recently
from
September 8, 2026 21:37
e3ff391 to
bf50f00
Compare
zinic
changed the title
zinic
force-pushed
the
pg-v2
branch
3 times, most recently
from
October 26, 2026 22:54
6f7e8b6 to
3057711
Compare
zinic
force-pushed
the
pg-v2
branch
3 times, most recently
from
November 2, 2026 18:01
cdfadee to
4790a2c
Compare
Add explicit destructive-test opt-in and a credential-free target allowlist. Wire the guard into integration commands and CI, and document the required environment variables.
Add shared fixtures, typed parameters, mutation post-state templates and legacy-builder regressions. Track query-form coverage and validate the regression manifest.
Add traversal admission plans, endpoint and suffix strategies, shortest-path lowering, path functions and graph-scoped SQL rendering. Extend parser, builder and SQL regression fixtures.
Add bounded parse/translation caches, graph-scoped execution, traversal workspaces and manifest-backed routing policies. Add topology metadata, schema upgrades and driver regression coverage.
Introduce the v2 pool/runtime hooks, per-connection translation retention, shared shortest-path templates and cache diagnostics. Cover lifecycle, configuration and live execution.
Add PostgreSQL traversal-plan, inline all-shortest, deletion and graph-wipe controls. Extend the PostgreSQL walker for the emitted query forms.
Add reproducible traversal scenarios, plan capture and reporting for both PostgreSQL runtimes. Include scale fixtures and shortest-path policy selection.
Add capture bundles, A/A resolution, paired confirmation, resource and operational gates, and manifest-backed promotion evidence. Include traversal telemetry, selector studies and reference tournaments.
Capture backend-specific plan diagnostics from the shared corpus, normalize plan structure and report comparable deltas. Extend command options, output provenance and regression coverage.
Record traversal experiments, routing and promotion contracts, source-parity procedures and operational limits. Update translation, development and repository guidance.
Admit the first-use fixed-suffix policy to guarded suffix lowering and emit the transaction-retry form. Add a regression for the versioned selector.
Hydrate repeated expansion endpoints once and reverse raw path-ID segments with the other path components. Cover self-loop aliases and reversed suffix paths.
Select the PostgreSQL-only adjacency feasibility probe by CONNECTION_STRING so Neo4j integration runs skip it.
Move the v2 runtime, pool hooks, translation cache and diagnostics into drivers/pg. Remove the separate v2 driver and obsolete shared SQL translation implementation. BREAKING CHANGE: callers of drivers/pg/v2 must migrate to drivers/pg and construct pools with pg.NewPool or pg.NewPoolWithRuntimeConfig.
Distinguish SP and ASP endpoint-array offsets so singleton all-shortest calls retain their maximum-depth argument. Add a compiler regression and the crypto module requirement.
Retain JSON scalar types in property collections, list operations and UNWIND. Cover heterogeneous values, committed properties and complete mutation state. Refs: DAWGS-054
Carry node/relationship element identity through collection projections and scope copies so overlapping numeric IDs cannot match the wrong entity type. Add shared membership and committed-state controls. Refs: DAWGS-056
Classify local timestamps by location identity rather than zone name. Cover parameter cache rebinding, temporal error boundaries and backend-selected integration tags without requiring a database for unit tests.
Preserve exact integer inputs while truncating fractional literals, parameters, properties and aliases toward zero. Cover signed boundaries, result types and mutation expressions. Refs: DAWGS-015
Treat percent, underscore and escape characters literally for CONTAINS, STARTS WITH and ENDS WITH across supported operand shapes. Add shared read and complete mutation-state templates. Refs: DAWGS-014
Materialize read-only WITH ordering, alias resolution, SKIP and LIMIT before later filtering and writes. Preserve input cardinality across multipart boundaries. Refs: DAWGS-040
Supply one implicit input row and retain matched-side predicates so unmatched patterns yield null bindings. Cover aggregation, chained optionals, paths and mutations. Refs: DAWGS-021
Copy stored JSON values into CREATE maps, retain native-array UNWIND element types and read multipart OPTIONAL inputs from the preceding frame. Validate corpus record assertions, support template schema kinds and normalize tagged instants.
Resolve predicate futures once with their type constraints and materialize pending filters before a later MATCH changes the correlation frame. Cover repeated predicates and exact mutation selection. Refs: DAWGS-036
Include node and relationship references in optimizer dependencies so unresolved bindings stay pinned. Apply deferred MATCH filters before mutations, including writes without RETURN. Refs: DAWGS-035
Read preceding materialized frames instead of enclosing WITH outputs and keep duplicate/empty streams through literal projections. Cover exact paths and SET/CREATE post-state. Refs: DAWGS-046
Reject plain node DELETE when incident relationships remain outside the deletion set. Apply DETACH cleanup on parent and partition paths with graph-scoped endpoint identity. Refs: DAWGS-039
Expand nested aliases with grouping, qualify bare UNWIND inputs and carry hidden DISTINCT sort keys. Preserve visible columns, ordering, pagination and mutation selection. Refs: DAWGS-047
Preserve JSON element types through direct properties, WITH aliases and nested lists. Materialize pending UNWIND inputs before SET while retaining duplicates and empty streams. Refs: DAWGS-042
Reserve DELETE CTE names without advancing the input stream so final projections retain cardinality and captured values. Stage preceding SET/CREATE writes before DELETE in the same transaction. Refs: DAWGS-043
Refresh parser mutation cases to canonical formatting and remove a duplicate license header from the generated property-UNWIND SQL fixture.
Resolve absent kinds without registration while preserving alternatives, conjunctions, negation and OPTIONAL MATCH. Bypass translation retention for missing-kind plans so later registration becomes visible. Refs: DAWGS-023
Return a located parser error and isolate the indexing subtree before PostgreSQL execution. Cover read/mutation rejection and native Neo4j pass-through; full shared indexing support remains open. Refs: DAWGS-001
Return a located parser error for omitted, negative, parameterized and chained slice bounds. Cover rejection before writes and native Neo4j slicing; full shared slicing support remains open. Refs: DAWGS-002
Return a located parser error for explicit and implicit procedure invocations, including YIELD, instead of a successful nil AST. Preserve native Neo4j procedure execution. Refs: DAWGS-017
Isolate expression subtrees and return located parser errors instead of projecting the terminal node. Retain supported MATCH syntax and native Neo4j expressions. Refs: DAWGS-010
Return located errors for CREATE UNIQUE and LOAD CSV before PostgreSQL text execution, including preceding writes. Preserve native Neo4j query text and diagnostics. Refs: DAWGS-013
Isolate the invocation subtree and return a located parser error instead of dropping arguments or YIELD filters from PostgreSQL execution. Cover read/mutation rejection and native procedure controls. Refs: DAWGS-103
Intersect positive conjunctive kind lists and retain an explicit false predicate for disjoint sets. Keep OR, XOR and NOT filters in WHERE so reads and mutations select the intended relationships. Refs: DAWGS-059
Render qualified function separators and conjunctive parsed labels while retaining builder any-kind OR behavior and float literal types. Add shared read/mutation and native namespaced-function controls. Refs: DAWGS-060
Parenthesize builder XOR groups inside AND and group PostgreSQL XOR operands and results. Add render, SQL and shared truth-table read/mutation controls. Refs: DAWGS-085
Decode Cypher identifier escapes, retain case and double embedded SQL quotes. Apply the same rendering to CTE names and cover exact result keys and mutation targets. Refs: DAWGS-055
Separate graph selection from schema reconciliation and use transaction-owned connections for kind mapping with commit-only metadata publication. Remove implicit open-traversal depth caps, add explicit resource failures and rollback, and repair shortest-path length and bounded endpoint workspaces. Quote Neo4j upsert identifiers and reconcile the active/completed variance ledgers. Preserve the regex default and recorded capacity/performance limits; the ledger does not qualify the complete PostgreSQL release gate as passing. Refs: DAWGS-019, DAWGS-061, DAWGS-063, DAWGS-065
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Delivers PostgreSQL driver v2 in
drivers/pg, combining connection-local runtime state with broader CySQL correctness coverage. It also repairs shared graph APIs, Neo4j query handling, and import behavior.pg.NewPoolstill returns the caller-owned*pgxpool.Poolpassed topg.NewDriverordawgs.Open.docs/; commit-message maintenance scripts and tests are included.Manifest-backed traversal canaries and topology routing remain default-off and require separate qualification. The benchmark results below use default runtime policies; capture revisions are identified separately.
Compatibility and upgrade notes
pg.NewPoolorpg.NewPoolWithRuntimeConfigto install the connection lifecycle hooks. Defaults are 64 retained translations per live connection, 128 shared shortest-path templates, and a 5–50 connection pool; cache capacities and pool bounds are configurable.Driver.TranslationCacheStats()now returns runtimepg.Stats. Migrate consumers of the previous compiler-cache counters toDriver.CompilationCacheStats()andpg.CompilationCacheStats.AssertSchemawhen upgrading to install the SQL runtime helpers, topology metadata, and graph-scoped cleanup triggers. Schema assertion performs DDL and resets pooled type metadata; reset or recreate pools after out-of-band schema changes affecting generated SQL or types.driver.RefreshKinds(ctx)on every affected driver. An unknown result kind ID signals later recovery without retrying the failed query.GetKindIDsByKindstill returns an independently mutable copy.int64, including nested values. Untyped consumers that expectedfloat64must handleint64; after draining query results, checkResult.Error()for decoding/iteration errors. Result-column aliases preserve their requested case and spelling.translate.NewExpansionBuilder. Consumers rendering staged multipart writes must execute every statement fromtranslate.TranslatedStatementsin the same explicit transaction. Treat batch identity keys and property hashes as opaque runtime values.Latest recorded benchmarks
The latest PostgreSQL captures compare
upstream/main2d3c62b7, the original PR 4790a2c8, a diagnostic build removing the two eager reload calls, and the P1 kind-cache implementation. Results were recorded on 2026-11-02; upstream was rechecked during that capture and was unchanged.The current changeset is fae5140f. The captures used an implementation working tree based on
4790a2c8; its recorded source hashes differ from the current changeset, which also includes subsequent initialization, snapshot-adoption and query-boundary fixes. These are the latest available implementation measurements, not a benchmark of the exact current commit.PostgreSQL: warmed kind-cache comparison
Cells show p50 / p95. Warm scalar latency fell from 446.42µs to 33.06µs; fluent and Cypher node-count medians fell from 488.31µs and 464.15µs to 29.53µs and 19.90µs. Every final warm implementation trace contained zero kind-catalog statements, versus one full 715-row reload per builder/Cypher request in the original PR.
Scalar client allocation fell from approximately 30,284 bytes/request in the no-refresh diagnostic to 2,922 bytes/request; decoded-entity allocation fell from 37,566 to 10,075 bytes/request. These totals include all client work. Real absent-name probes issued one targeted lookup per request (141.10µs / 284.79µs), and signaled-refresh scalar probes issued one full reload (450.21µs / 830.97µs).
Component benchmarks used five repetitions with
-benchmem -benchtime=200ms -cpu=1. Wrapper construction used a nil physical connection; commit bookkeeping excluded database I/O. Medians follow.Overlay lookups and unchanged commit bookkeeping allocated zero bytes. Dictionary copies remain on publication or explicitly requested owned copies. A separate control warmed all 50 pool connections before measuring empty transactions: implementation throughput was 653k–861k requests/sec versus 147k–167k for the diagnostic, with approximately 666–696 versus 27,966–27,988 client bytes/request.
These are descriptive local measurements on small fixtures. Identical-binary controls showed substantial host variation, including scalar p50 of 12.88–34.17µs. The final implementation scalar median remained 3.11µs above main; Raw-count p95 was 96.00µs versus 51.17µs on main and 72.85µs in the diagnostic. Earlier cohorts reversed that tail difference, so precise latency equivalence and elimination of tail risk are not established. Production-scale performance and default-off traversal policies require separate qualification. Cold optimizer overhead remains unresolved.
Neo4j: matched-query follow-up
The follow-up review used identical parameterized statements, binding types and fixed physical IDs on main and the original PR, with Neo4j Community 4.4.44. It predates the PostgreSQL-only P1 fix.
The earlier committed-harness comparison used literal endpoints on main and parameters on the PR; parameterized
Queryparsing adds 0.33–0.42ms on both revisions. Earlier diamond/disconnected regressions did not reproduce with matched inputs. The base offset remains unresolved, and identical-binary controls produced apparent median changes of −10.6% to +1.9%. These observations do not establish a repeatable Neo4j implementation regression; batch-write performance was outside the read measurements.Local capture protocols, raw samples, source hashes, allocation summaries and validation logs are retained under
.coverage/p1/and.coverage/pr125/regression-review/. These ignored artifacts are available in the measurement workspace and are not part of the PR.PostgreSQL vs. Neo4j: latest available captures
The table compares the P1 PostgreSQL implementation capture above (800 observations per case) with the latest recorded Neo4j original-PR capture (
4790a2c8, 600 observations per case). Both used the same host and three-node/two-edgebasefixture, four balanced rounds and 100 warm-ups. They were separate capture cohorts with different harness versions; this is a descriptive comparison of the latest available results, not a simultaneous comparison of the exact current commit. The P1 changes affect PostgreSQL, but Neo4j has not been recaptured at the current commit.Timings are p50 / p95. Delta is (PG / Neo4j − 1) × 100%: negative values mean lower PostgreSQL latency. Raw cases use native SQL on PostgreSQL and native Cypher on Neo4j for the same scalar/count result.
PostgreSQL scalar/count/projection p50 was 95.9–97.8% lower in these captures; base all-shortest-path p50 was 293.7% higher (3.94× Neo4j). The path query and parameter types match, but the P1 harness additionally checks decoded path shape and kinds. Host variation, differing capture windows and that additional validation limit causal claims; these small fixtures do not establish production-scale backend performance.
PostgreSQL vs. Neo4j: full shared-scenario spread
The broader committed-harness capture covers all 21 shared scenarios on both backends at
upstream/main(2d3c62b7) and the original PR (4790a2c8). Each scenario/backend/revision has 200 observations across four balanced rounds, with one worker, one warm-up and 50 timed iterations per round. Fixtures were reloaded per capture; all expected row counts matched. PostgreSQL 18.6 and Neo4j Community 4.4.44 ran sequentially on the same host with Go 1.27.3.These captures predate the P1 kind-cache fix. They provide the complete recorded scenario spread; the newer P1 results above cover a smaller selection and should be used for that implementation. Warm-up drift affected the broad capture, and shortest-path endpoints were literals on main and parameters on the original PR. Each within-revision backend comparison uses the same query form, but changes between revisions also reflect that workload difference. These tables are descriptive fixture measurements.
Every timing cell is p50 / p95 in µs; every delta cell is p50 / p95, calculated as (PG / Neo4j − 1) × 100%. Negative deltas mean lower PostgreSQL latency. Main graph scoping differs from the PR, so main and PR timings are shown as separate backend comparisons rather than as a graph-query improvement claim.
Base fixture — 3 nodes, 2 edges
Traversal-shapes fixture — 45 nodes, 41 edges
The full-spread tables are derived from
.coverage/pr125/benchmark-summary.json; newer P1 and matched-query results retain their own capture protocols and sample counts.Compiler microbenchmarks
These earlier measurements compare main with the original PR (
4790a2c8); the P1 fix does not change the optimizer. The six shared optimizer/compiler-cache benchmarks used identical source and query inputs, five repetitions, and-benchtime=200ms -benchmem. The table reports the median observation per benchmark. Optimizer costs increased about 61–64%, uncached translation increased 13.3%, and cache-hit timings stayed close to baseline. These benchmark functions exercise the shared compiler cache.Type of Change
Testing
Validation includes recorded unit and integration results plus a targeted PostgreSQL telemetry run against
fae5140f. Full suites were not rerun for this description update.CONNECTION_STRINGsetmake test_bdd_integration)-race -tags "manual_integration integration" -p=1 -parallel=1 ./drivers/pg/... ./integration -count=1make test_all)make lintand formatting check of changed Go filesgo mod verifyandgo mod tidy -diffBDD pass counts refer to enforced scenarios;
@report-onlycompatibility cases remain non-gating.Screenshots
N/A
Driver Impact
drivers/pg)drivers/neo4j)Checklist
go.mod/go.sumare up to date