Skip to content

E-6: render a slide as captured, and make its remaining controls honest - #790

Merged
marcinz606 merged 6 commits into
marcinz606:mainfrom
seanharding:e6-as-captured
Aug 10, 2026
Merged

E-6: render a slide as captured, and make its remaining controls honest#790
marcinz606 merged 6 commits into
marcinz606:mainfrom
seanharding:e6-as-captured

Conversation

@seanharding

Copy link
Copy Markdown
Contributor

Slides currently render nothing like the capture, and turning Normalize off doesn't fix it. This makes E-6 open faithfully, and cleans up several controls on that path that looked live but did nothing.

Measured against Photoshop, macOS Preview, Affinity and Darktable — which agree with each other, with NegPy the outlier.

The core change

With Normalize off, the Print stage is replaced by a transfer curve that is an exact identity at default settings, so the render is the capture:

  • The camera's own colour matrix is applied. RAW decodes output_color=raw, so the buffer is in camera primaries. The print path never needed the matrix because it derives colour from measured density and a paper model; the transfer path can't, so the decoder's XYZ→camera matrix is inverted against the working space and row-normalised (dcraw's cam_xyz_coeff construction). A source without a matrix — scanner TIFF, JPEG — passes through.
  • The exposure window is fixed, anchored to the decoder's white level rather than metered from the frame. This is what keeps a bracketed set rendering as a bracket: measured bounds make five exposures of one slide converge on the same render (spread 0.083), where a fixed window keeps them apart (0.544).
  • A standard display rendering is applied — a fixed +0.7 EV baseline (darktable's shipped default for raw) and Narkowicz's closed-form fit to the ACES RRT + sRGB ODT. This is not optional: a bare linear-to-gamma encode is the one thing no consumer converter does, and measured against a Lightroom export it left midtones 1.37 EV dark with highlights 2.2× short of white. With it, the residual is +0.11 EV at the median.

Both terms are fixed constants, never metered — metering is what would make a bracket converge again.

Normalize now defaults to off

It was added as a rescue path — 0.9.10's changelog calls it "an optional Normalize step that tries to save expired slides" — but shipped on, so every slide met it as the general-purpose render.

On faded film it's the right tool. On a correctly exposed slide it isn't: the density runs to Dmax but only the top ~1.5 decades carry picture, so a window metered across the whole range (3.57 decades on the test frame) puts the median at 13% of the scale, which the print curve renders near white. Measured on a well-exposed frame: median luma 0.88 and mean saturation 0.14, against 0.70 and 0.29 for the same frame as captured.

The algorithm is untouched, so the rescue case is unaffected and it stays one click away. Only files with no saved edit changee6_normalize is written into every record by to_dict, and the key shipped in the same release as E-6 mode, so no existing E-6 edit can be missing it.

Clamping the metered span was the alternative and was rejected: it fixes the look but costs the convergence Normalize exists to provide (bracket spread 0.083 → 0.136), and re-tuning one case degrades the other.

Controls that were lying

Three controls on this path moved a value that never reached the image:

  • Crosstalk now declares its film process. A matrix describes one dye set, so profiles carry process and only appear — and only apply — in the film they were made for. A C-41 matrix can no longer be applied to a slide, or to B&W. The in-app editor gains a Process control and stamps new matrices with the mode in use, so a matrix you build for slides is one you can then select.
  • Cast Removal is hidden outside C-41. The render already ignored it there — the solve needs the shadow and neutral-axis references, and both meters are gated to C-41 — so the slider was inert. It defeats the orange mask, a manufactured cast; a slide has no mask and its cast is the photograph.
  • The Global/R/G/B selector is hidden on the transfer path, along with the White/Black Point sliders it scopes, which a fixed window ignores.

Conversely, Shadows/Highlights Density now work on slides. They existed, had sliders and persisted, but apply_transfer_curve implemented none of it — leaving four tone controls where a print gets ten, and no mid-sparing one at all. Their geometry is derived from the print's by tonal position, not by copying its density numbers: the curves don't share a scale, so the raw 1.50 would sit 64% of the way to black on a print but 50% here. The lift also tapers to nothing at the bottom of the window — a print gets that bound from paper black, this curve has no paper, and without it a lift walks the black point away (14% of a frame below 0.03 became 0.1%).

Bug fixed along the way

Crosstalk was silently skipped on the GPU for E-6 with Normalize off. The shader applied the unmix on the print branch only, so the Strength slider did nothing on the engine the app actually renders with — CPU moved 196/255, GPU moved 0. Every parity test ran with crosstalk gated off, where both engines agree trivially, which is how it stayed hidden.

Verification

  • C-41 and B&W renders are unchanged, verified by SHA-256 fingerprints over float32 output buffers across a mode × paper × normalize sweep, on both CPU and GPU.
  • GPU parity for the new shader is asserted on real hardware with engagement guards — each test checks the term is doing something on both engines, not merely that they agree, since a shader ignoring a uniform lane agrees trivially.
  • 48 new tests in test_transparency_transfer.py and test_crosstalk_editor_process.py, plus additions to the sidebar suites. Full suite passes.

Behaviour changes for existing users

  • New E-6 files open with Normalize off. Saved edits keep whatever they were saved with.
  • B&W renders shift ~0.1%: it was receiving a Generic C41 unmix at the shipped default strength of 0.5, despite the sidebar having hidden the control there all along.
  • Cast Removal and the channel selector disappear where they were already inert.

Related to #789 (another PR is coming to close that out later).

Turning Normalize off did not stop NegPy transforming a slide, so a scan
opened looking nothing like the same file in Photoshop, Preview, Affinity
or Darktable. Two independent causes:

- Colour: RAW decodes output_color=raw, so the buffer is in the camera's
  primaries while the working space assumed Adobe RGB. The print path never
  needed the distinction (it derives colour from measured density and a
  paper model); a faithful render does.
- Tone: e6_normalize=False only swapped a measured ceiling for floor+3.0.
  The floor stayed the frame's own high percentile, so the stretch was
  still content-anchored, and the full paper H&D curve ran on top.

Normalize off now takes a transparency transfer path. Normalization applies
the decoder's camera matrix and a fixed density window anchored to the white
level, with no metering at all -- measured bounds are exactly what makes two
exposures of one slide converge, so a bracketed set stays a bracket (an 8x
scene range renders 7.96x, against 1.00x with Normalize on). The print curve
gives way to a transfer curve whose scene stage is an exact inverse of that
window, so nothing shapes the capture, followed by the standard rendering a
raw converter opens with: a fixed +0.7 EV baseline (darktable's default for
raw) and Narkowicz's fit to the ACES RRT + sRGB ODT. Both are fixed
constants, never metered, which is also how Lightroom preserves brackets at
Exposure 0. Against a Lightroom export of the same frame the residual is
+0.11 EV at the median, down from 1.37 EV with highlights 2.2x short.

The paper curve cannot serve this role: d_max floors the blacks whatever the
toe says, anchor_target_density places mid-grey at a print's mid-tone, and
the midtone snap and paper-white reference stay live at neutral settings.
Neutralizing it is not reachable from a paper profile.

Controls keep working: Density, Grade, Toe, Shoulder, both Width sliders and
their per-layer trims drive the transfer curve, each neutral at its shipped
default. The paper-specific controls hide, as does the normalization tuning
that only shapes a measured stretch.

Linear RAW and Narrowband are made inert here before being hidden -- both
are sticky and act outside the render config, so hiding a live one would
strand the user. Linear RAW's missing white balance is folded back into the
camera matrix (the row-normalized matrix assumes a balanced signal; a 2:1 raw
green-to-red ratio rendered as a green frame). Narrowband's implicit
RGBScan.icc is suppressed, since the camera matrix has already reached the
working space; an explicit Input ICC still wins. Narrowband capture cannot
satisfy this mode anyway -- it samples three isolated wavelengths, so the
inter-band overlap the eye integrates is never measured, and its real
payoffs belong to negatives.

C-41, B&W and E-6 with Normalize on are untouched, verified bit-for-bit:
86 render fingerprints (SHA-256 over the float32 output) across both engines
and the full control surface are identical to main.

(cherry picked from commit 92172a8)
A crosstalk matrix describes one dye set, and C-41 and E-6 do not share one.
Every bundled profile is a colour negative stock, yet the shipped default is
"Generic C41" at strength 0.5 — so every slide was quietly getting a negative's
dye unmix, and so was every B&W scan, despite the sidebar hiding the control
there since "nothing to unmix on one B&W emulsion".

Profiles now declare the film they were derived for:

    process = "E-6"    # or "C41", the default when absent

The value is baked into ProcessConfig at selection (matrices are baked too, so
the render never touches the disk) and both engines gate on it via
effective_crosstalk_matrix, mirroring effective_paper_profile. A mismatched
profile resolves to identity rather than mixing the wrong correction in. Legacy
configs carry no field and default to C-41, which is what every profile that
predates it actually is. The dropdown filters to the current mode, and the
controls hide when a mode has no matrices at all — keyed on the profile list
rather than the mode, so dropping a matching .toml into the crosstalk folder
brings them back on its own.

That gate is what makes it safe for the transparency transfer to honour
crosstalk instead of hard-skipping it: a rig-calibrated matrix is a capture
correction there, like Hue Trim, and it is inert at the shipped default.

Also fixes a GPU/CPU split introduced with that path: normalization.wgsl applied
the unmix on the print branch only, so an E-6 matrix moved the CPU render and
did nothing whatsoever on the GPU — the engine the app actually uses. Measured
strength 0->1 on a slide: CPU 196/255, GPU 0. The unmix now rides both branches,
which is also simpler, since the CPU packs identity rows when it does not apply
and there is nothing left to branch on.

Every GPU parity test ran with crosstalk gated off, where both engines agree
trivially, which is why the suite stayed green through that bug. There is now a
parity case with a matrix actually active, plus a guard asserting it moves the
image at all so it cannot pass for the wrong reason; it fails on the old shader.

No E-6 matrices are bundled. Three were derived from Tier 1 spectral dye-density
curves (Ektachrome E100, Provia 100F, Velvia 100F) and then dropped: for
reversal film the unmix is not a correction. A negative is an intermediate, so
removing the dyes' unwanted absorptions moves the render toward the scene; a
transparency is the finished image and those absorptions are part of how it
looks, so unmixing moves it away from the slide, toward the dye-amount image
behind it. That is a separation control, not fidelity, and it is not worth
shipping under names that imply parity with the negative profiles.

C-41 is unchanged, verified bit-for-bit (28/28 render fingerprints). B&W and E-6
Normalize-on do change, by design and by roughly 0.1%: they stop receiving a
correction derived for a film they are not.

(cherry picked from commit ff8109a)
…l selector

Gating the Crosstalk section on "does this process have any matrices" hid the
editor along with it, and the editor is the only way to make one — a slide user
had no way in. Keep the section for every process except B&W, and disable the
empty dropdown and its Strength slider instead, with a hint saying why.

Saving alone was not enough either: the editor wrote no `process` key, so a
matrix built while working on a slide was stored as C-41 and stayed invisible in
the mode it was made for. Profiles now carry the process the editor was opened
in, with a Process control to change it, and both the preview and Apply carry it
through to the config the render gates on.

The Global/R/G/B selector in Process is hidden on the transparency transfer. It
scopes the White/Black Point sliders, which that path already hides because it
normalizes with a fixed window — the selector was left behind with nothing to
scope. It resets to Global when hidden, as it does in B&W.

(cherry picked from commit 88ab723)
Normalize meters a per-frame stretch and prints it through the paper model. It
was added as a rescue path -- 0.9.10's changelog calls it "an optional Normalize
step that tries to save expired slides" -- but it shipped on, so every slide met
it as the general-purpose render.

On faded film it is the right tool. On a slide exposed as intended it is not: the
density runs to Dmax but only the top ~1.5 decades carry picture, so a window
metered across the measured range (3.57 decades on the test frame) puts the median
at 13% of the scale, which the print curve renders near white. Measured on a
well-exposed frame, median luma 0.88 and mean saturation 0.14, against 0.70 and
0.29 for the same frame as captured.

Nothing about the algorithm changes, so the rescue case is untouched and it stays
one click away. Clamping the metered span was the alternative and was rejected: it
fixes the look but costs the convergence Normalize exists to provide (bracket
spread 0.083 -> 0.136), and re-tuning one case degrades the other.

Only files with no saved edit move. e6_normalize is written by to_dict on every
record, and the key shipped in the same release as E-6 mode, so no E-6 edit can
be missing it. All 40 explicit mode/paper/normalize render fingerprints are
unchanged on CPU and GPU.

(cherry picked from commit 588cd93)
The render already ignored it on slides and B&W: the solve fits red and blue to
green's neutral axis, which needs the shadow and neutral-axis references, and both
meters are gated to C-41 (processor.py, and `needs_refs` in gpu_engine.py). With
no refs per_channel_curve_params falls through to one shared linear curve, so the
slider moved a persisted value that never entered the arithmetic. Verified across
0.0/0.5/1.0 on both engines: byte-identical output in E-6 either side of
Normalize, and in B&W.

The gating is right, so the fix is the UI. Cast Removal defeats the orange mask —
a cast the manufacturer built in, not part of the picture. A transparency has no
mask and its cast IS the photograph, so a neutral-axis solve would strip the light
it was shot in; B&W collapses to one density with no channels to balance.

Hidden rather than disabled because it is already inert, so nothing is left
running invisibly. The test asserts visibility against the render itself rather
than a repeated mode list, so the two cannot drift.

(cherry picked from commit 260e722)
Shadows/Highlights Density existed, had sliders, persisted and were documented,
but apply_transfer_curve implemented none of it, so they were hidden and inert on
slides -- leaving four tone controls where a print gets ten, and no mid-sparing
one at all. Grade pivots the whole scale and Toe compresses everything above its
knee, so opening shadows with either costs the midtones and highlights.

The term is the print path's own. Its geometry is derived by TONAL POSITION
rather than by copying the print's density numbers: the two curves do not share a
scale (a print runs d_min 0.06 to d_max 2.3, this runs 0 to
TRANSFER_DENSITY_RANGE), so carrying the raw 1.50 across would put the shadow
centre 64% of the way to black on a print but 50% here, and the slider would
reach into the midtones -- on a dusk frame that lifted a quarter of the picture
by 0.12. The sharpness scales with the range for the same reason.

The lift also tapers to nothing at the bottom of the window. A print bounds a
shadow burn at paper black; this curve has no paper, which is the point of it, so
without a bound a lift walks the black point up with it and the frame simply
stops having blacks -- 14% of a test frame below 0.03 became 0.1%. The taper
costs nothing: the lift is unchanged and the blacks survive.

Split Grade is deliberately not mirrored: it rotates zone-local contrast, and
this curve has no per-zone slope to rotate.

The GPU mirror grows two uniform lanes for the zone offsets and the taper. Parity
asserts each is doing something on BOTH engines rather than only that they agree
-- a shader ignoring a new lane agrees trivially, which is how the crosstalk
unmix stayed broken once already.
@marcinz606
marcinz606 merged commit 658e16e into marcinz606:main Aug 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants