E-6: render a slide as captured, and make its remaining controls honest - #790
Merged
Conversation
Turning Normalize off did not stop NegPy transforming a slide, so a scan opened looking nothing like the same file in Photoshop, Preview, Affinity or Darktable. Two independent causes: - Colour: RAW decodes output_color=raw, so the buffer is in the camera's primaries while the working space assumed Adobe RGB. The print path never needed the distinction (it derives colour from measured density and a paper model); a faithful render does. - Tone: e6_normalize=False only swapped a measured ceiling for floor+3.0. The floor stayed the frame's own high percentile, so the stretch was still content-anchored, and the full paper H&D curve ran on top. Normalize off now takes a transparency transfer path. Normalization applies the decoder's camera matrix and a fixed density window anchored to the white level, with no metering at all -- measured bounds are exactly what makes two exposures of one slide converge, so a bracketed set stays a bracket (an 8x scene range renders 7.96x, against 1.00x with Normalize on). The print curve gives way to a transfer curve whose scene stage is an exact inverse of that window, so nothing shapes the capture, followed by the standard rendering a raw converter opens with: a fixed +0.7 EV baseline (darktable's default for raw) and Narkowicz's fit to the ACES RRT + sRGB ODT. Both are fixed constants, never metered, which is also how Lightroom preserves brackets at Exposure 0. Against a Lightroom export of the same frame the residual is +0.11 EV at the median, down from 1.37 EV with highlights 2.2x short. The paper curve cannot serve this role: d_max floors the blacks whatever the toe says, anchor_target_density places mid-grey at a print's mid-tone, and the midtone snap and paper-white reference stay live at neutral settings. Neutralizing it is not reachable from a paper profile. Controls keep working: Density, Grade, Toe, Shoulder, both Width sliders and their per-layer trims drive the transfer curve, each neutral at its shipped default. The paper-specific controls hide, as does the normalization tuning that only shapes a measured stretch. Linear RAW and Narrowband are made inert here before being hidden -- both are sticky and act outside the render config, so hiding a live one would strand the user. Linear RAW's missing white balance is folded back into the camera matrix (the row-normalized matrix assumes a balanced signal; a 2:1 raw green-to-red ratio rendered as a green frame). Narrowband's implicit RGBScan.icc is suppressed, since the camera matrix has already reached the working space; an explicit Input ICC still wins. Narrowband capture cannot satisfy this mode anyway -- it samples three isolated wavelengths, so the inter-band overlap the eye integrates is never measured, and its real payoffs belong to negatives. C-41, B&W and E-6 with Normalize on are untouched, verified bit-for-bit: 86 render fingerprints (SHA-256 over the float32 output) across both engines and the full control surface are identical to main. (cherry picked from commit 92172a8)
A crosstalk matrix describes one dye set, and C-41 and E-6 do not share one.
Every bundled profile is a colour negative stock, yet the shipped default is
"Generic C41" at strength 0.5 — so every slide was quietly getting a negative's
dye unmix, and so was every B&W scan, despite the sidebar hiding the control
there since "nothing to unmix on one B&W emulsion".
Profiles now declare the film they were derived for:
process = "E-6" # or "C41", the default when absent
The value is baked into ProcessConfig at selection (matrices are baked too, so
the render never touches the disk) and both engines gate on it via
effective_crosstalk_matrix, mirroring effective_paper_profile. A mismatched
profile resolves to identity rather than mixing the wrong correction in. Legacy
configs carry no field and default to C-41, which is what every profile that
predates it actually is. The dropdown filters to the current mode, and the
controls hide when a mode has no matrices at all — keyed on the profile list
rather than the mode, so dropping a matching .toml into the crosstalk folder
brings them back on its own.
That gate is what makes it safe for the transparency transfer to honour
crosstalk instead of hard-skipping it: a rig-calibrated matrix is a capture
correction there, like Hue Trim, and it is inert at the shipped default.
Also fixes a GPU/CPU split introduced with that path: normalization.wgsl applied
the unmix on the print branch only, so an E-6 matrix moved the CPU render and
did nothing whatsoever on the GPU — the engine the app actually uses. Measured
strength 0->1 on a slide: CPU 196/255, GPU 0. The unmix now rides both branches,
which is also simpler, since the CPU packs identity rows when it does not apply
and there is nothing left to branch on.
Every GPU parity test ran with crosstalk gated off, where both engines agree
trivially, which is why the suite stayed green through that bug. There is now a
parity case with a matrix actually active, plus a guard asserting it moves the
image at all so it cannot pass for the wrong reason; it fails on the old shader.
No E-6 matrices are bundled. Three were derived from Tier 1 spectral dye-density
curves (Ektachrome E100, Provia 100F, Velvia 100F) and then dropped: for
reversal film the unmix is not a correction. A negative is an intermediate, so
removing the dyes' unwanted absorptions moves the render toward the scene; a
transparency is the finished image and those absorptions are part of how it
looks, so unmixing moves it away from the slide, toward the dye-amount image
behind it. That is a separation control, not fidelity, and it is not worth
shipping under names that imply parity with the negative profiles.
C-41 is unchanged, verified bit-for-bit (28/28 render fingerprints). B&W and E-6
Normalize-on do change, by design and by roughly 0.1%: they stop receiving a
correction derived for a film they are not.
(cherry picked from commit ff8109a)
…l selector Gating the Crosstalk section on "does this process have any matrices" hid the editor along with it, and the editor is the only way to make one — a slide user had no way in. Keep the section for every process except B&W, and disable the empty dropdown and its Strength slider instead, with a hint saying why. Saving alone was not enough either: the editor wrote no `process` key, so a matrix built while working on a slide was stored as C-41 and stayed invisible in the mode it was made for. Profiles now carry the process the editor was opened in, with a Process control to change it, and both the preview and Apply carry it through to the config the render gates on. The Global/R/G/B selector in Process is hidden on the transparency transfer. It scopes the White/Black Point sliders, which that path already hides because it normalizes with a fixed window — the selector was left behind with nothing to scope. It resets to Global when hidden, as it does in B&W. (cherry picked from commit 88ab723)
Normalize meters a per-frame stretch and prints it through the paper model. It was added as a rescue path -- 0.9.10's changelog calls it "an optional Normalize step that tries to save expired slides" -- but it shipped on, so every slide met it as the general-purpose render. On faded film it is the right tool. On a slide exposed as intended it is not: the density runs to Dmax but only the top ~1.5 decades carry picture, so a window metered across the measured range (3.57 decades on the test frame) puts the median at 13% of the scale, which the print curve renders near white. Measured on a well-exposed frame, median luma 0.88 and mean saturation 0.14, against 0.70 and 0.29 for the same frame as captured. Nothing about the algorithm changes, so the rescue case is untouched and it stays one click away. Clamping the metered span was the alternative and was rejected: it fixes the look but costs the convergence Normalize exists to provide (bracket spread 0.083 -> 0.136), and re-tuning one case degrades the other. Only files with no saved edit move. e6_normalize is written by to_dict on every record, and the key shipped in the same release as E-6 mode, so no E-6 edit can be missing it. All 40 explicit mode/paper/normalize render fingerprints are unchanged on CPU and GPU. (cherry picked from commit 588cd93)
The render already ignored it on slides and B&W: the solve fits red and blue to green's neutral axis, which needs the shadow and neutral-axis references, and both meters are gated to C-41 (processor.py, and `needs_refs` in gpu_engine.py). With no refs per_channel_curve_params falls through to one shared linear curve, so the slider moved a persisted value that never entered the arithmetic. Verified across 0.0/0.5/1.0 on both engines: byte-identical output in E-6 either side of Normalize, and in B&W. The gating is right, so the fix is the UI. Cast Removal defeats the orange mask — a cast the manufacturer built in, not part of the picture. A transparency has no mask and its cast IS the photograph, so a neutral-axis solve would strip the light it was shot in; B&W collapses to one density with no channels to balance. Hidden rather than disabled because it is already inert, so nothing is left running invisibly. The test asserts visibility against the render itself rather than a repeated mode list, so the two cannot drift. (cherry picked from commit 260e722)
Shadows/Highlights Density existed, had sliders, persisted and were documented, but apply_transfer_curve implemented none of it, so they were hidden and inert on slides -- leaving four tone controls where a print gets ten, and no mid-sparing one at all. Grade pivots the whole scale and Toe compresses everything above its knee, so opening shadows with either costs the midtones and highlights. The term is the print path's own. Its geometry is derived by TONAL POSITION rather than by copying the print's density numbers: the two curves do not share a scale (a print runs d_min 0.06 to d_max 2.3, this runs 0 to TRANSFER_DENSITY_RANGE), so carrying the raw 1.50 across would put the shadow centre 64% of the way to black on a print but 50% here, and the slider would reach into the midtones -- on a dusk frame that lifted a quarter of the picture by 0.12. The sharpness scales with the range for the same reason. The lift also tapers to nothing at the bottom of the window. A print bounds a shadow burn at paper black; this curve has no paper, which is the point of it, so without a bound a lift walks the black point up with it and the frame simply stops having blacks -- 14% of a test frame below 0.03 became 0.1%. The taper costs nothing: the lift is unchanged and the blacks survive. Split Grade is deliberately not mirrored: it rotates zone-local contrast, and this curve has no per-zone slope to rotate. The GPU mirror grows two uniform lanes for the zone offsets and the taper. Parity asserts each is doing something on BOTH engines rather than only that they agree -- a shader ignoring a new lane agrees trivially, which is how the crosstalk unmix stayed broken once already.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Slides currently render nothing like the capture, and turning Normalize off doesn't fix it. This makes E-6 open faithfully, and cleans up several controls on that path that looked live but did nothing.
Measured against Photoshop, macOS Preview, Affinity and Darktable — which agree with each other, with NegPy the outlier.
The core change
With Normalize off, the Print stage is replaced by a transfer curve that is an exact identity at default settings, so the render is the capture:
output_color=raw, so the buffer is in camera primaries. The print path never needed the matrix because it derives colour from measured density and a paper model; the transfer path can't, so the decoder's XYZ→camera matrix is inverted against the working space and row-normalised (dcraw'scam_xyz_coeffconstruction). A source without a matrix — scanner TIFF, JPEG — passes through.Both terms are fixed constants, never metered — metering is what would make a bracket converge again.
Normalize now defaults to off
It was added as a rescue path — 0.9.10's changelog calls it "an optional Normalize step that tries to save expired slides" — but shipped on, so every slide met it as the general-purpose render.
On faded film it's the right tool. On a correctly exposed slide it isn't: the density runs to Dmax but only the top ~1.5 decades carry picture, so a window metered across the whole range (3.57 decades on the test frame) puts the median at 13% of the scale, which the print curve renders near white. Measured on a well-exposed frame: median luma 0.88 and mean saturation 0.14, against 0.70 and 0.29 for the same frame as captured.
The algorithm is untouched, so the rescue case is unaffected and it stays one click away. Only files with no saved edit change —
e6_normalizeis written into every record byto_dict, and the key shipped in the same release as E-6 mode, so no existing E-6 edit can be missing it.Clamping the metered span was the alternative and was rejected: it fixes the look but costs the convergence Normalize exists to provide (bracket spread 0.083 → 0.136), and re-tuning one case degrades the other.
Controls that were lying
Three controls on this path moved a value that never reached the image:
processand only appear — and only apply — in the film they were made for. A C-41 matrix can no longer be applied to a slide, or to B&W. The in-app editor gains a Process control and stamps new matrices with the mode in use, so a matrix you build for slides is one you can then select.Conversely, Shadows/Highlights Density now work on slides. They existed, had sliders and persisted, but
apply_transfer_curveimplemented none of it — leaving four tone controls where a print gets ten, and no mid-sparing one at all. Their geometry is derived from the print's by tonal position, not by copying its density numbers: the curves don't share a scale, so the raw 1.50 would sit 64% of the way to black on a print but 50% here. The lift also tapers to nothing at the bottom of the window — a print gets that bound from paper black, this curve has no paper, and without it a lift walks the black point away (14% of a frame below 0.03 became 0.1%).Bug fixed along the way
Crosstalk was silently skipped on the GPU for E-6 with Normalize off. The shader applied the unmix on the print branch only, so the Strength slider did nothing on the engine the app actually renders with — CPU moved 196/255, GPU moved 0. Every parity test ran with crosstalk gated off, where both engines agree trivially, which is how it stayed hidden.
Verification
test_transparency_transfer.pyandtest_crosstalk_editor_process.py, plus additions to the sidebar suites. Full suite passes.Behaviour changes for existing users
Generic C41unmix at the shipped default strength of 0.5, despite the sidebar having hidden the control there all along.Related to #789 (another PR is coming to close that out later).