Skip to content

fix(shortener): retry id generation on same-second create collisions - #325

Merged
stakach merged 4 commits into
masterfrom
PPT-2644-shortener-id-collision
Aug 11, 2026
Merged

fix(shortener): retry id generation on same-second create collisions#325
stakach merged 4 commits into
masterfrom
PPT-2644-shortener-id-collision

Conversation

@camreeves

Copy link
Copy Markdown
Contributor

Stacking note: based on the M1 branch tip so downstream PPT-2644 verification picks up both changes together. Two ways to land it — either keep it as its own small PR opened after M1 merges (it will then show only the one commit), or cherry-pick / fold 4710f0a into M1. The fix itself has no dependency on the FTS work; it stacks purely for verification convenience. It could equally be rebased onto master as a standalone PR if you want it to land first.


The defect (pre-existing, unrelated to PPT-2644)

Shortener#short_id (src/placeos-models/shortener.cr) generates the primary key in a before_create:

time = ((Time.utc.to_unix - TIME_OFFSET) << 6) + rand(63)
@id = "uri-#{time.to_s(62)}"

The creation second fills all but the low 6 bits, and rand(63) fills those — so there are only 63 possible ids per wall-clock second. Two creates in the same second collide on shortener_pkey with probability ~1/63, and the caller gets an unhandled PgORM::Error::RecordNotSaved (duplicate key value violates unique constraint "shortener_pkey") → a 500 from rest-api.

Surfaced by PPT-2644 verification: the ES-free rest-api spec harness no longer sleeps between creates, so spec/controllers/short_url_spec.cr (which creates 2 shorteners back-to-back in two tests plus the CRD helper) started failing ~1 run in 2. It is also a real production bug for concurrent shortener creation — burst QR/short-URL provisioning will 500 at ~1/63 per same-second pair.

The fix

The compact id format is deliberate (the id is the public short URL), so it is kept byte-for-byte identical. Instead, Shortener overrides save! to retry on exactly this collision:

  • save! loops calling super, rescuing PgORM::Error::RecordNotSaved only when the record is still new_record? and the message names shortener_pkey; anything else re-raises immediately.
  • pg-orm's __create re-runs the before_create callbacks on every attempt (run_create_callbacks__before_create, verified in active-model's callback macro), so each retry rolls a fresh rand(63) — no extra regeneration code needed.
  • Bounded at CREATE_ID_ATTEMPTS = 10; a failed attempt is one cheap single-row INSERT round-trip.
  • Overriding save! covers every path: save delegates to save! (this is what rest-api's controller uses — url.save), create!/create call record.save!, and the specs call .save! directly. update! also routes through save! but can never satisfy the new_record? guard. Same override pattern as Edge#save! / Survey::Question#save!.

Spec

spec/shortener_spec.cr: freezes time with Timecop and creates 16 shorteners — with the timestamp component pinned, all 16 draw from the same 63 ids, so at least one collision is near-certain (~88%; probabilistic by design, rand deliberately not stubbed). Asserts all 16 persist with unique ids and that the format is unchanged (uri- prefix + base62).

Without the fix this scenario is the rest-api failure amplified: on the old code the batch 500s almost every run.

Evidence

  • Before: rest-api spec/controllers/short_url_spec.cr failed ~1 in 2 full-harness runs (VERIFICATION-2026-08-10.md, "rest-api short_url triage": short_url_spec:6 duplicate key … shortener_pkey).
  • models, containerised ./test spec/shortener_spec.cr ×3: 3/3 green (3 examples, 0 failures each).
    • Run 2's db DEBUG log proves the mechanism: 22 shortener INSERT attempts for 18 distinct ids — 4 colliding attempts (uri-5o05sg ×3, uri-5o05sl ×2, uri-5o05sc ×2) each absorbed by a retry, 0 surfaced errors.
  • rest-api, containerised ./test spec/controllers/short_url_spec.cr ×4 with shard.override.yml temporarily pointing placeos-models at this branch: 4/4 green (previously ~1 in 2 failed). Override reverted afterwards; rest-api worktree left untouched.

camreeves and others added 4 commits August 10, 2026 21:43
…earch (PPT-2644)

Adds a generated, stored tsvector column + GIN index to every table whose
rest-api index route is searchable, replacing the Elasticsearch write path.
'simple' text-search config on both sides (no stemming — entity names,
identifiers, emails; matches the guests.tsv_search precedent). Secrets and
encrypted content are deliberately excluded from every vector.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Elasticsearch's analyzer segmented uri/file-path fields, so their parts
were individually searchable — PG 'simple' keeps '/path' as one lexeme.
Split on punctuation (keeping the raw value) for mod.uri,
driver.file_name/default_uri, repo.uri, shortener.uri and
signage_plugin.uri.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t migration

master's 9ff188d renumbered the AI-support migration to 20260806100500000,
the exact version this migration was using. Micrate versions must be
unique, and the search-vector migration must sort after everything on
master, so it moves to 20260810100500000.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Shortener#short_id derives the primary key from the creation second plus
rand(63), i.e. only 63 possible ids per wall-clock second, so two
same-second creates collide on shortener_pkey with probability ~1/63 and
the caller gets a 500 (PgORM::Error::RecordNotSaved). The compact format
is deliberate (the id is the public short URL), so keep it and retry the
insert instead: each attempt re-runs the before_create callback which
rolls a fresh random component. Bounded at 10 attempts; retries are
cheap single-row inserts.

Surfaced by rest-api short_url_spec failing ~1 in 2 runs once the
ES-free spec harness (PPT-2644) removed the inter-create ES sleeps, but
the defect is pre-existing and also affects concurrent production
creates.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions github-actions Bot added the type: bug something isn't working label Aug 11, 2026
Base automatically changed from PPT-2644-pg-full-text-search to master August 11, 2026 05:13
@stakach
stakach merged commit be62b5e into master Aug 11, 2026
8 of 9 checks passed
@stakach
stakach deleted the PPT-2644-shortener-id-collision branch August 11, 2026 05:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: bug something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants