Skip to content

fix(db): prevent runtime lock convoys - #382

Merged
brokermr810 merged 46 commits into
mainfrom
test
Nov 14, 2026
Merged

brokermr810 merged 46 commits into
mainfrom
test

Conversation

@brokermr810

Copy link
Copy Markdown
Collaborator

Root cause

A startup admin probe kept a read transaction open while calling helpers that acquired another pooled connection. Concurrent request-time DDL on qd_users then waited for an access-exclusive lock, and later API and worker sessions queued behind that lock. PostgreSQL could not detect the application-level cross-session cycle, so the lock convoy persisted until the blocker was terminated.

Changes

  • sanitize pooled PostgreSQL connections on checkout and return
  • ensure health probes never leave transactions open
  • enforce bounded idle-transaction and lock-wait timeouts for API and workers
  • release the startup read transaction before nested database helpers run
  • remove request-time user and billing DDL; canonical schema changes remain migration-owned
  • add regression guards for transaction cleanup, timeout configuration, bootstrap ordering, and runtime DDL

Validation

  • 61 database and billing regression tests passed
  • 53 database, user, referral, and admin regression tests passed
  • focused final regression: 20 tests passed
  • Ruff, Python compilation, Docker Compose validation, and git diff --check passed

Deployment impact

Allow the migration service to finish, then rebuild and recreate the backend and Python worker containers so both the code changes and new PostgreSQL session options take effect. A plain container restart is not sufficient when the image has not been rebuilt.

tianhe added 30 commits November 1, 2026 17:17
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>

# Conflicts:
#	backend_api_python/app/routes/community.py
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
tianhe and others added 16 commits November 6, 2026 15:07
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Co-authored-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
#362 normalized the Binance protection orders, but Bitget (UTA and
classic) and Bybit still sent percentage-derived prices with str(), e.g.
stopLoss=77951.14749999999 for a 2.5% stop on a 76049.9 short. Those are
off the contract tick, so the venue rejects the protection order.

Run the stop-loss, take-profit and trailing activation prices (and the
Bybit trailingStop distance) through each client's _normalize_price, the
same way #362 does for Binance, and raise when the result falls below the
minimum.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: aliassar <ali.assar8@gmail.com>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
Signed-off-by: TIANHE <TIANHE@GMAIL.COM>
@sonarqubecloud

Copy link
Copy Markdown

@codecov

codecov Bot commented Nov 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 56.00000% with 22 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
backend_api_python/app/utils/db_postgres.py 45.00% 15 Missing and 7 partials ⚠️

📢 Thoughts on this report? Let us know!

@brokermr810
brokermr810 merged commit c4add1e into main Nov 14, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants