Skip to content

fix(api): harden core ASGI routes against custom-node mutation - #10211

Merged
JPPhoto merged 8 commits into
invoke-ai:mainfrom
JPPhoto:asgi-hardening
Nov 11, 2026
Merged

JPPhoto merged 8 commits into
invoke-ai:mainfrom
JPPhoto:asgi-hardening

Conversation

@JPPhoto

@JPPhoto JPPhoto commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Custom-node code could mutate shared FastAPI route state, register HTTP endpoints, or replace the UI mount before requests reached the application. InvokeAI now captures core route configuration and the original frontend before loading nodes, then serves through a guard outside the mutable router. The production Socket.IO mount is also snapshotted: its wrapper and dispatch configuration are detached while the live Engine.IO server is retained. Core routes, frontend assets, and SPA navigation continue to work; unregistered ASGI routes return 404. Registered invocation classes remain in the OpenAPI schema.

Related Issues / Discussions

None.

QA Instructions

  • UV_CACHE_DIR=/tmp/uv-cache PYTHONDONTWRITEBYTECODE=1 uv run --active --no-sync pytest tests/app/api/test_trusted_route_guard.py -k trusted_socketio_mount -q -p no:cacheprovider - 1 passed, 20 deselected. Before the fix, the same regression failed because /ws/custom returned 200. The original app serves the injected static file with 200; the guard returns 404.
  • Guard tests excluding the two TestClient-based lifespan cases - 19 passed, 2 deselected. Those two tests stall in TestClient.__enter__ in this environment; a plain FastAPI app reproduces the same AnyIO portal stall.
  • uv run --no-sync pytest -q tests/app/api/test_trusted_route_guard.py -k 'included_websocket_route_keeps_captured_dependency_override or direct_websocket_route_rebinds_dependency_handler or socketio_mount_detaches_dispatcher_configuration' - 3 passed. The regression failed before the fix because the guard called the default dependency; it now verifies that the original app uses a late override while the guard retains the captured override.
  • UV_CACHE_DIR=/tmp/uv-cache uv run --no-sync pytest tests/app/api/test_trusted_route_guard.py -k nested_websocket_routes_keep_their_prefix_clone -q - 1 passed, 20 deselected. Before the fix, this regression failed twice: the nested WebSocket closed without accepting when the clone cache reused another prefix's route.
  • UV_CACHE_DIR=/tmp/uv-cache PYTHONDONTWRITEBYTECODE=1 uv run --active --no-sync pytest tests/app/test_socketio_disconnect.py -q -p no:cacheprovider - 5 passed.
  • A guarded /invoke polling handshake passed via ASGITransport, with the session on the original Engine.IO server.
  • The preceding revision passed Ruff 0.11.2 check/format. R7 and R8 follow-ups passed Ruff 0.11.13 check/format, git diff --check, and commit hooks. Ruff 0.11.2 could not be rerun for follow-ups because the sandbox could not fetch it (R7 DNS failure; R8 tool-cache write blocked).
  • From docs/, pnpm test - 29 passed; pnpm build - 736 pages built and all internal links valid (run on the preceding revision; docs have not changed since).
  • The full test suite was not run. The TestClient-based subpath suite could not complete in this environment; the guarded subpath handshake was verified directly.

Review

The frontend/static, route/middleware/dependency, OpenAPI cache, root-path/lifespan, API fallback, Socket.IO dispatch, and included WebSocket dependency-override findings are resolved. The included WebSocket regression verifies that the original app can accept a later override while the guard keeps the provider captured before snapshot. The clone-cache ID-reuse finding is resolved by retaining each original router and route alongside its clone until snapshot completion. A weakref regression verifies reused IDs cannot misroute nested WebSocket prefixes. This is an ASGI routing boundary, not a Python sandbox: node-pack code still runs in-process and can mutate shared Python objects and services. The snapshot relies on FastAPI 0.141.1 private routing and lifespan internals.

Compatibility / Rollout

Invocation-based node packs continue to use core endpoints, and registered invocation classes appear in OpenAPI. ASGI routes added by node packs during loading or reload are not served; unregistered endpoints return 404. Slash redirects and wrong-method 405 responses remain intact. No persisted-data migration or generated API contract update is required.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Meaningful regression coverage added / updated where needed; obsolete tests/code removed
  • Persisted-state and API changes include required migrations / compatibility validation
  • Relevant performance/efficiency opportunities considered; material claims have evidence
  • Material review findings resolved and relevant checks rerun
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

JPPhoto and others added 3 commits November 8, 2026 19:53
Capture core routes and frontend state before custom-node loading, then serve through a detached guard.

Reject unknown API paths and isolate middleware, dependency overrides, and OpenAPI behavior.
@github-actions github-actions Bot added api python PRs that change python files python-tests PRs that change python tests docs PRs that change docs labels Nov 9, 2026
@JPPhoto JPPhoto changed the title Harden core ASGI routes against custom-node mutation fix(api): harden core ASGI routes against custom-node mutation Nov 9, 2026
@JPPhoto JPPhoto changed the title fix(api): harden core ASGI routes against custom-node mutation fix(api): harden core ASGI routes against custom-node mutation Nov 9, 2026
@JPPhoto JPPhoto moved this to 7.0.0: UX overhaul in Invoke - Community Roadmap Nov 9, 2026
@JPPhoto JPPhoto added the 7.0.0 label Nov 9, 2026
@JPPhoto
JPPhoto requested a review from joshistoast November 9, 2026 02:12

@joshistoast joshistoast left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach blocks route insertion and mount replacement, but its “immutable core route graph” claim exceeds the protection achieved. The main quality concern is incomplete copying of framework state, compounded by reliance on FastAPI internals. Custom-node API endpoints also intentionally become unavailable—a meaningful compatibility change.

Comment thread invokeai/app/api/trusted_routes.py Resolved
Comment thread invokeai/app/api/trusted_routes.py Resolved
Comment thread invokeai/app/api/trusted_routes.py Resolved
@JPPhoto
JPPhoto requested a review from joshistoast November 9, 2026 04:50

@joshistoast joshistoast left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, the revision substantially improves isolation and restores normal routing behavior. This remaining gap still contradicts the documented restriction on custom-node ASGI routes. Heavy reliance on FastAPI internals remains a maintenance cost.

Comment thread invokeai/app/api/trusted_routes.py Resolved
Clone the mounted dispatcher and isolate its static/fallback route configuration.

Preserve the live Engine.IO server and its configured path.
@JPPhoto
JPPhoto requested a review from joshistoast November 9, 2026 12:36
Comment thread invokeai/app/api/trusted_routes.py Resolved
Rebuild included WebSocket handlers with the captured dependency override provider.\nTest that guarded routes retain captured overrides after the original app changes.
@JPPhoto
JPPhoto requested a review from joshistoast November 10, 2026 02:44

@joshistoast joshistoast left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This predates the latest fix. Current InvokeAI uses a root Socket.IO mount, so no current-production merge blocker or new regression was found.

Comment thread invokeai/app/api/trusted_routes.py Resolved
Retain source routers and routes beside clones during snapshot creation.\nAdd a nested WebSocket regression for reused IDs across prefixes.
@JPPhoto
JPPhoto requested a review from joshistoast November 10, 2026 13:02
@JPPhoto
JPPhoto enabled auto-merge November 11, 2026 00:20
@JPPhoto
JPPhoto added this pull request to the merge queue Nov 11, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 11, 2026
@JPPhoto
JPPhoto added this pull request to the merge queue Nov 11, 2026
Merged via the queue into invoke-ai:main with commit 5b776e3 Nov 11, 2026
21 checks passed
@JPPhoto
JPPhoto deleted the asgi-hardening branch November 11, 2026 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

7.0.0 api docs PRs that change docs python PRs that change python files python-tests PRs that change python tests

Projects

Status: 7.0.0: UX overhaul

Development

Successfully merging this pull request may close these issues.

2 participants