fix: keep IBKR sessions usable across API workers and threads

[johnsontamiwt]

What changed and why

ib_insync binds a connection to two things: the process that opened it (TWS and IB Gateway allow one session per client id) and the asyncio event loop of the thread that created it. The API served IBKR from gunicorn workers that shared neither — each process kept its own in-memory session map and defaulted to client id 1, and each gthread request thread created a fresh event loop in _ensure_event_loop().

The result, with the stock GUNICORN_WORKERS=2:

Connecting to host:4002 with clientId 1...
Logged on to server version 176
Error 326, reqId -1: Unable to connect as the client id is already in use.
Peer closed connection. clientId 1 already in use?
API connection failed: TimeoutError()          <- 20s later
POST /api/ibkr/connect -> 400
GET  /api/ibkr/positions -> 400

Worker A holds client id 1; worker B's connect is refused. Because requests are balanced across workers, GET /api/ibkr/status alternates between a connected and a disconnected view of the same account, and the data endpoints fail at random. Calls that did reach a live client from another thread ran against an idle loop instead.

Alpaca is unaffected throughout: it is a stateless REST broker with no session and no client id, which is why the same UI works there.

A second, independent bug surfaced once the connection held: the account panel showed the account id but rendered every metric as --. IB.accountSummary() returns tag rows whose values are strings paired with a currency, while BrokerAccountCard reads flat numeric fields (net_liquidation, buying_power, …) — the shape the Alpaca route already returns.

Changes

File	Change
ibkr_trading/session.py (new)	Pin each connection to a dedicated thread whose event loop stays alive and marshal every call onto it, which also serialises access to the socket. A process-wide registry keyed by (host, port, client_id) reuses the live session, so the process cannot answer its own connect with Error 326.
ibkr_trading/config.py (new)	Resolve connection settings from the stored credential instead of querying credential tables from the route (per MODULE_BOUNDARIES.md). The UI's 127.0.0.1 placeholder is unreachable from a container, so it falls back to the saved host; an explicit host or port is always honoured; a blank client id no longer raises. The credential's ibkr_client_id belongs to the order session (default 7), so the UI session never borrows it.
ibkr_trading/account.py (new)	Expose the flat numeric fields the account UI reads, alongside the existing summary map which stays for agent/MCP callers. Missing tags are omitted rather than zeroed — a margin requirement of 0 reads very differently from "unknown".
routes/ibkr.py	Reconnect on demand from the stored credential so every endpoint works regardless of which worker serves it; drop a sleep-based retry that could not fix a cross-process session; a failed connect from inside a container now explains the loopback trap instead of only suggesting TWS may be down; a failed summary read returns 400 instead of reporting success with an empty payload.
live_trading/account_snapshot.py	Read IBKR over one short-lived session instead of the crypto path's swap+spot pair (which collides on the same client id), and serialise snapshots so concurrent refreshes cannot answer each other with Error 326.

How to test

cd backend_api_python
pytest tests/test_ibkr_session_and_account.py

14 tests covering session reuse, thread affinity, failed-connect cleanup, credential merge precedence, and summary flattening.

Full suite is unchanged: 47 failed / 1335 passed before, 47 failed / 1349 passed after — the same pre-existing failures, plus the new tests.

Verified end to end against a live IB Gateway paper account (backend in Docker, Gateway on the host):

• connect with an empty body, with the UI's 127.0.0.1 defaults, and twice in a row — all 200; the second connect previously produced Error 326
• account, positions, orders, status, disconnect — all 200
• a data endpoint called with no prior /connect — 200, via on-demand reconnect
• two concurrent fetch_account_snapshot calls — no warnings, no Error 326
• grep -c "Error 326" over the backend logs — 0
• the account panel renders net liquidation, cash, buying power and both margin requirements

ruff check passes on all touched files.

Backward compatibility

• Response fields are additive; summary and account are unchanged, so agent/MCP callers are unaffected.
• GET /api/ibkr/account now returns 400 instead of 200 when the summary read fails. The OpenAPI contract already documents 400 for this operation, and scripts/export_openapi.py produces no IBKR diff.
• No configuration or migration changes.

Note for maintainers

Operators running more than one gunicorn worker were hitting this constantly. The fix makes a multi-worker deployment behave correctly, but each worker still holds its own IB session under the same client id policy. Moving the session into the trading worker (which already owns runtime leases) and reaching it over the command queue would give the deployment a single IB connection — happy to open a discussion if that direction is wanted.

[johnsontamiwt]

Follow-up commit: verified the account panel against a live futures position rather than an empty paper account, which exposed three more payload problems on the same surface.

Held: 1x MGCV6 (Micro Gold OCT'26, multiplier 10) filled at 4433.30, with a protective stop #40 SELL 1 MGCV6 STP @ 4350.

Symptom	Cause
Entry price shown as 44,332.96 for a 4,433.30 fill	Position.avgCost is reported per contract, i.e. already multiplied. Correct for stocks (multiplier 1), 10x off for MGC.
Protective stop rendered with no price	A stop carries its trigger in auxPrice; lmtPrice is 0 for a plain STP, and the order table falls through limit_price → limitPrice → price to nothing.
Order id column empty, cancel disabled	The table keys rows on id; only orderId was returned, and canCancel requires !!row.id.

Also carrying localSymbol and lastTradeDate now: futures share a symbol across expiries, so MGC alone cannot tell MGCV6 from MGCZ6.

Verified end to end again on the same account — entry price 4433.296, stop 4350.0, order id 40, position named MGCV6. 7 more tests cover futures vs stock multipliers, STP vs LMT price selection, and the order id key; the suite is unchanged apart from the additions.

All fields are additive: avgCost, limitPrice, orderId and symbol keep their existing meaning.

[johnsontamiwt]

Third commit: pending_order_position_sync reads the same IBKRClient.get_positions() payload and fed avgCost into the L1 account mirror, so a futures entry was recorded inflated by the contract multiplier there too. Extracted position_entry_price() into ibkr_trading/account.py and both call sites now use it, so the two paths cannot drift again.

Scoped deliberately:

• The reconciliation key stays on symbol/ib_symbol — it has to match the symbol the strategy stored, so switching it to localSymbol would scatter reconciliation. Only the price changed, and the constraint is now a comment at that line.
• Alpaca's branch in the same function is untouched: its avgCost is avg_entry_price, already per unit.
• strategy_v2/runtime.py uses avgCost as a key name too, but that value comes from _next_average_cost(..., fill_price) with a per-unit fill price and is restored from the runtime's own avg_cost state, not from a broker payload. No change needed.

One correction to my earlier comment: the follow-up commit added 4 tests, not 7. Current totals: 21 in tests/test_ibkr_session_and_account.py, and the full suite goes from 1335 passed on the base commit to 1356 with these changes, with the same 47 pre-existing failures in both runs.

Re-verified on the live account: MGCV6 avgCost=44332.96 x10.0 -> entry_price=4433.296.

[johnsontamiwt]

Gentle ping on the CI state: the checks here are sitting at action_required, which means the workflows are waiting on a maintainer to approve the run — the usual gate for a first-time contributor pushing from a fork. Nothing has failed; the runs simply haven't started.

To make that approval cheap, here is what was verified locally against this repo's own tooling:

• ruff check passes on every touched file, using the version pinned in basic-ci.yml.
• Full backend suite: 1335 passed on the base commit, 1356 passed on this branch — the same 47 pre-existing failures in both runs, plus 21 new tests in tests/test_ibkr_session_and_account.py.
• scripts/export_openapi.py produces no diff for any /api/ibkr/* path.
• End to end against a live IB Gateway paper account, including a real futures position (MGCV6, multiplier 10) held with a protective stop, which is what surfaced the second and third commits.

Scope is 3 commits across 8 backend files, and the response changes are additive — summary, account, avgCost, limitPrice and orderId all keep their existing meaning. The one behavioural change is GET /api/ibkr/account returning 400 instead of 200 when the summary read fails, which the OpenAPI contract already documents.

Happy to rebase, split the commits, or narrow the scope if that would make review easier.