[Pfannkuchensack]

Summary

Tiny AutoEncoder previews fall back to linear previews whenever the TAE does not fit in VRAM next to the running model. On small GPUs with partially loaded models (e.g. a 12 GB card running a quantized model at ~60% residency) TAE previews therefore never appear, even with plenty of system RAM.

Each TAE now has a Decode previews on CPU switch in the model manager (cpu_only, the mechanism text encoders and VAEs already use). With it on, the TAE decodes on the CPU and needs no VRAM:

• the decoder is built in float32 on the CPU (CPU float16 is emulated; CPU bfloat16 is slow without native support and faults on some CPUs);
• CPU decodes are capped at a 512 px long edge (TAESDXL: 1.7 s at 1024² vs 0.3–0.4 s at 512² on a Ryzen 9 7900, fp32);
• frame pacing synchronizes the latents' device, so the GPU step still queued is not counted as the frame's cost, and a CPU TAE is paced from its first frame (lookup, load and lock excluded); the GPU path is unchanged;
• the "does not fit in VRAM" log line points to the switch.

Before / after on an RTX 4090 with max_cache_vram_gb: 3 (model only partly in VRAM), 1024×1024, 30 steps:

	Switch off (TAE refused → linear, 128 px)	Switch on (CPU TAE, 512 px)
SDXL + TAESDXL	23.4 s, linear every step	25.1 s (+7.4%), a TAE frame about every 5 s
Anima + TAEW2.1	35.3 s, linear every step	38.5 s (+9.0%), a TAE frame about every 12 s

Related Issues / Discussions

User question about running TAEs on the CPU/RAM when VRAM is too small for them next to the model.

QA Instructions

• pytest tests/backend/tiny_autoencoder/ tests/app/util/test_tiny_autoencoder_previews.py tests/backend/model_manager/load/test_tiny_autoencoder_loader.py tests/backend/model_manager/configs/test_tiny_autoencoder_config.py tests/app/services/model_manager/test_model_record_changes.py tests/app/services/progress_previews/test_signal_progress.py: 255 passed (CPU).
• Ruff check/format clean; webv2 lint:tsc, format:check and CpuOnlySetting.browser.test.tsx pass; webv2 production build passes.
• Browser: the switch appears on a TAE's model detail page with its own label and help text, saves at once (cpu_only: true via the API); toggling drops the cached TAE.
• GPU (RTX 4090, scratch root, enable_partial_loading: true, max_cache_vram_gb: 3, tiny_autoencoder_previews: true): SDXL + TAESDXL and Anima + TAEW2.1, 1024², 30 steps, switch off/on interleaved, 3 images each after a warm-up. Off: the log reports the TAE does not fit and previews are linear. On: the log shows the TAE loaded onto the CPU (cpu_only) and previews are 512 px TAE frames. Timings in the table above.
• Not measured: TAEW2.2 and TAELTX2.3 on the CPU, multi-GPU setups, CPUs other than a Ryzen 9 7900.

Review

Resolved: a CPU TAE's second frame went out unpaced because the first frame was never counted, which made up much of the overhead (Anima went from +11% to +9% after the fix); the working-memory helper now requires the decode device; the device comparison in pacing is normalized; a weakened video-cap test and a vacuous label assertion were fixed; pacing outcome tests were added for a CPU TAE with latents on another device and for the unchanged GPU first-frame rule; unsupported performance claims in the docs were replaced by measurements.

Remaining limits: a CPU decode runs synchronously between denoise steps, so the GPU waits for it (paced to a few percent of the denoise); with several GPUs it competes for CPU time with the other generation threads; its transient RAM (~0.3 GB TAESDXL at 512 px, ~1.2 GB for a TAEW2.1 two-frame window) is not counted by the model cache.

Compatibility / Rollout

New optional cpu_only field on Tiny AutoEncoder configs (default null); existing records load unchanged. openapi.json and schema.ts regenerated.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

🤖 Generated with Claude Code

---

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}