[Pfannkuchensack]

Summary

FLUX.3's prompt upsampler could only be reached through Expand Prompt, with a system prompt pasted by hand, and never
saw the reference images an edit needs. A FLUX.3 generation can now run it itself.

• Prompt upsampler switch in the Render section (FLUX.3 only, off by default) and a Prompt Upsampler model slot
  under Components, required while the switch is on.
• With the switch on, the new node flux3_prompt_upsampler rewrites the prompt into the caption the text encoder
  encodes. It reads the reference images (an edit) and the Regional Guidance boxes (a layout) and uses the matching
  one of Black Forest Labs' four system prompts. It takes the place of the layout prompt node: ids and boxes of the
  regions, and of a box table typed at the end of the prompt, are kept exactly.
• The caption is saved as flux3_caption and the model as flux3_prompt_upsampler. Recall restores the typed prompt,
  the switch and the model; a FLUX.3 image made without the upsampler turns the switch off.
• Expand Prompt ships BFL's t2i system prompt as FLUX.3 Prompt Upsampler and preselects it, with an installed
  upsampler, when a FLUX.3 model is selected.
• With the switch off the FLUX.3 graph is unchanged.

The node follows diffusers' upsample_prompt (greedy, no presence penalty) and reuses its system prompts, scaffolds,
reference preparation and box-table reconciliation.

Related Issues / Discussions

Follow-up to #10092 (the upsampler as a Text LLM) and #10095 (FLUX.3 layout boxes).

QA Instructions

Automated

• uv run --no-sync pytest tests/backend/flux3 tests/app/invocations/test_flux3_prompt_upsampler.py tests/app/invocations/test_flux3_nodes.py tests/app/invocations/test_node_discovery.py tests/app/invocations/test_frontend_flux3_layout_graphs.py tests/app/invocations/test_frontend_graph_node_types.py tests/app/services/shared/sqlite_migrator tests/backend/model_manager/test_starter_models.py tests/backend/model_manager/test_starter_models_package.py tests/app/routers/test_starter_model_is_installed.py: 1468 passed, 10 skipped.
• Ruff check and format check: clean.
• Frontend: pnpm lint passes; the touched unit and browser test files pass (graph, compileCanvasGraph,
  baseGenerationPolicies, settings, graphCoverage, imageRecall, GenerateSettingsForm.browser,
  ExpandPromptSuggestion.browser). test:performance:build, test:performance:browser and
  test:project-files:browser pass.
• test:accessibility:browser: every journey passes except the last one, launchpad-hidden-maintenance, which times
  out waiting for its dialog to close. It fails the same way on unchanged main (83507c6) on this Windows
  machine; CI is green for that commit.
• Not run locally: the full unit and browser suites.
• Docs build: 736 pages, all internal links valid; the app search index is re-synced.

On an RTX 4090 (Windows), bf16 upsampler folder unless noted:

• Captions equal diffusers' upsample_prompt, character for character, for a text prompt, one and two reference
  images, a layout with two regions, and a layout with one reference image and one region, when the reference process
  ranks the SDPA kernels as InvokeAI does. Under torch's stock ranking the three image cases diverge after 15-110
  tokens, in the reference itself as well.
• Cancel during generation: stopped 0.5 s after the request. Unchanged inputs: returned from the node cache in 0.65 s
  without loading the model.
• int8 single file: all five cases run; locked ids and boxes are kept.
• Whole generations for four cases: images written, flux3_caption in the metadata equals the node output, boxes
  land where drawn.

Timings, medians of three runs after a warm-up, no other program computing on the GPU, FLUX.3 Klein bf16 at
1024 x 1024 and 16 steps:

	seconds
Upsampler node, text prompt (300-token caption)	15.9
Upsampler node, one reference image (224 tokens)	13.6
Upsampler node, layout with two regions (649 tokens)	27.0
Upsampler node, int8 file, text prompt	34.5
Generation, text prompt, upsampler off / on	8.4 / 28.6
Generation, one reference image, upsampler off / on	15.6 / 33.7
Generation, text prompt, upsampler on, max_cache_ram_gb: 24 instead of 40	40.5

In the browser against that server: the switch toggles by keyboard and carries its help text as description; the
slot shows "None" while the switch is off; with the switch on and no model the slot is marked invalid with its
reason and Invoke shows one issue; picking a model unblocks it; a generation runs the upsampler and records caption
and model; switching off keeps the pick; "Reuse from image > All settings" turns the switch on for an image made with
the upsampler and off for one made without, restoring the typed prompt both times; Expand Prompt preselects the
upsampler and the FLUX.3 Prompt Upsampler system prompt and rewrites the prompt.
With the picked upsampler unregistered through the API, the switch on shows "Invoke · 1 issue" with
"Prompt upsampler ... is not installed" and a Clear button; switching off removes the issue and Invoke is enabled;
switching on brings it back.

To try it: install FLUX.3 Prompt Upsampler (or the int8 starter) and a FLUX.3 model, turn on Prompt
upsampler in Render, pick the model under Components, add a reference image or Regional Guidance layers, and
generate. The encoded caption is in the image's metadata as flux3_caption.

Not verified: Regional Guidance with the upsampler through the Canvas UI (covered by the canvas compile tests,
the backend-validated snapshot graphs and API runs with regions); the model-family round trip in the browser (the
test install has only FLUX.3 models); ROCm, MPS and CPU. Family E2E has no FLUX.3 manifest.

Review

Resolved:

• The model slot was first shown only while the switch was on. That needed special cases in the model-switch
  handling and in recall, and a kept pick that was no longer installed blocked Invoke in other model families. The
  slot is now always part of FLUX.3's Components and required only while the switch is on.
• The node takes its input device from the loaded model's compute_device instead of the first parameter.
• The caption, a user's prompt of up to about 15 KB, is logged at debug; info carries the mode and its length.
• Tests: the generate arguments must equal diffusers' apart from the stopping criterion; the node test file no longer
  spends its time copying a processor it does not test; restating and duplicate tests removed.
• Docs: RAM cache sizing with measured numbers, the Generate panel page, clearer wording on what is kept in a layout.
• A Prompt Upsampler pick that was no longer installed kept blocking Invoke after the switch was turned off. Slots
  behind a switch now say when their selection is in use, and the installed-model check skips a selection nothing
  loads: the prompt upsampler, the two PiD slots and Anima's inpaint adapter. Turning the switch on asks for the
  model again.

Remaining limits:

• The node moves the upsampler out of VRAM after its run. On cards that could hold all models this costs a re-stream
  of about 2.5 s per new prompt; on 24 GB it keeps the text encoder and transformer from sharing VRAM with it.
• The default RAM cache is capped at the GPU's VRAM minus the working memory (about 21 GB on a 24 GB card), which
  holds neither model set (32.1 GiB with the bf16 upsampler, 25.3 GiB with the int8 one), so models are read from
  disk again for each new prompt until max_cache_ram_gb is raised. Documented with the measured difference.
• The helpers come from diffusers' modular_pipelines.flux3.prompt_upsampler, which is not a public API; the tests
  use diffusers' upsample_prompt as their oracle and fail on a bump that changes it.
• Captions with reference images depend on the SDPA kernel order, so they differ from a stock-torch diffusers run.

Compatibility / Rollout

• One portable migration seeds the system prompt with a fixed id, insert-if-absent; an edited or deleted row stays
  as it is.
• New invocation flux3_prompt_upsampler; openapi.json and schema.ts are regenerated.
• Two new Generate settings, both off or empty by default; settings saved before this change load with those
  defaults.
• New metadata key flux3_prompt_upsampler; flux3_caption is now also written whenever the upsampler ran.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

---

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}

[Pfannkuchensack]