Skip to content

Make FFT3DFilter deterministic: build R2 from source with a patch (task #62) - #222

Merged
StuartCameronCode merged 1 commit into
mainfrom
stuart/62-temporaldegrain2-noise-reduction-gives-d
Nov 18, 2026
Merged

StuartCameronCode merged 1 commit into
mainfrom
stuart/62-temporaldegrain2-noise-reduction-gives-d

Conversation

@StuartCameronCode

@StuartCameronCode StuartCameronCode commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

Fixes task #62: TemporalDegrain2 (and every other FFT3DFilter user) gave a different output on every run, so a preview never matched its encode.

Cause

The fault is in FFT3DFilter itself (R2, and every other published build), not TemporalDegrain2 or MVTools. It has two bugs:

  • Left border mirrored before the line was copied (FramePlaneToCoverbuf). The left 10 luma / 6 chroma columns came from the previous frame's pixels or uninitialised memory. This is upstream regression da50417.
  • Per-frame Wiener sigma in a shared member of an fmParallel filter. An edge frame (btcur 1) could overwrite it while a middle frame was using it. This happens whenever bt > 1 with a single sigma: the FFT3D method, TD2's post pass, and QTGMC with NoiseTR > 0.

Change

  • New patch: Scripts/patches/fft3dfilter-r2-determinism.patch.
    • It copies the line before mirroring, in all three loops.
    • It makes sigmaSquaredNoiseNormed a local in Wiener3D and ApplyFilter.
  • FFT3DFilter is now built from source at tag R2 on every platform, with the patch applied. The build fails if the patch stops applying.
    • Linux: meson.
    • macOS: a direct compile against the bundle's own FFTW. This replaces yuygfgg's arm64 prebuilt.
    • Windows: a Zig cross-compile against FFTW 3.3.5, through an import library made from its .def. This replaces the R2 release DLL.
    • One build serves both x64 tiers, and the file names are unchanged.
  • Tests:
    • Push gate: vapoursynth_integration_test.dart adds "fft3dfilter is deterministic (task perf(arm64): route std.Expr through akarin's LLVM JIT #62)".
      • It runs six fresh graphs per argument set at 8 threads.
      • A second process must agree.
      • A flat picture must stay flat at the left edge.
      • It skips on bundles up to 1.17.x.
    • Heavy: new integration_fft3dfilter_determinism_test.dart. Two previews must be identical, and the preview must be identical to the FFV1 encode. It covers the FFT3D method and TD2 with its post pass.
  • Docs:
    • CLAUDE.md, BUILDING, ENGINEERING_NOTES and NOTICES.
    • A deps 1.18.0 row in the version history.
    • The report docs/reports/task-62-fft3dfilter-determinism.
  • bt=0 (Kalman) stays unreachable: NoiseReductionParameters::fft3d_effective_bt clamps to 1–5.

Testing

  • Heavy test, macOS arm64:
    • Against the published bundle it fails: previews of the same frame differ by up to 33 levels.
    • Against the patched build it passes: preview vs preview and preview vs encode both have a max diff of 0.
  • Windows build: cross-compiled locally.
    • The DLL exports VapourSynthPluginInit2.
    • It imports only libfftw3f-3.dll, KERNEL32 and the CRT.
    • It has not been loaded on Windows yet.
  • Re-run after review: vapoursynth_integration_test.dart and attribution_test.dart pass. The new test skips on the local 1.16 bundle.

Takes effect only in deps 1.18.0

The fix only reaches users through a new deps release (1.18.0), and deps-version.json is not bumped here. After the bundles are built, run the push gate on all four platforms with deps_run_id. That will be the first run of the Windows Zig build, and of the new test anywhere other than macOS arm64.

If any platform's second process disagrees, the fallback is measure=0 (FFTW_ESTIMATE). It goes at our call sites and as the plugin's default in the patch.

🤖 Generated with Claude Code

…(task #62)

TemporalDegrain2, the FFT3D Noise Reduction method and QTGMC's fft3dfilter
denoiser gave a different picture on every run, so the preview never matched
the encode. Both faults are in FFT3DFilter itself: the left border was
mirrored before the source line was copied (stale memory), and a per-frame
Wiener sigma lived in a shared member of an fmParallel filter.

Scripts/patches/fft3dfilter-r2-determinism.patch fixes both; every
download-deps-* script builds R2 from source with it (Windows via a Zig
cross-compile) and fails if it stops applying. Push-gate and heavy tests,
report docs/reports/task-62-fft3dfilter-determinism. Needs deps 1.18.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@StuartCameronCode
StuartCameronCode force-pushed the stuart/62-temporaldegrain2-noise-reduction-gives-d branch from 90380d5 to 4972554 Compare November 18, 2026 02:25
@StuartCameronCode
StuartCameronCode merged commit 7df4630 into main Nov 18, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant