Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .claude/skills/verify/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
name: verify
description: Drive the real NegPy desktop app headlessly and capture rendered evidence — use when verifying pipeline/UI changes end-to-end (not via pytest).
---

# Verifying NegPy changes end-to-end

The surface is the PyQt6 GUI; the render pipeline's output is the canvas.

## Recipe that works

1. **Sandbox app state** so the run never touches `~/Documents/NegPy`:
`NEGPY_USER_DIR=<scratch>/negpy_home` (checked in `get_default_user_dir`).
Create the subdirs from `_bootstrap_environment` (or call it) before
`StorageRepository.initialize()`.
2. **Suppress the first-run tutorial** or every window grab is dimmed by the
overlay: `repo.save_global_setting("tutorial_seen", True)` before
constructing `MainWindow`.
3. **Headless display**: `xvfb-run -a -s "-screen 0 1920x1080x24" uv run python driver.py`.
wgpu/Vulkan works fine under Xvfb on this machine (real GPU).
4. **Driver skeleton**: build the app exactly like `negpy/desktop/main.py`
(`StorageRepository` → `DesktopSessionManager` → `AppController` →
`MainWindow`), then drive a QTimer state machine:
- `controller.load_file("samples/06.raw")` — same method the Files sidebar calls.
- advance steps on `controller.image_updated` (skip `state.last_metrics["splash"]`,
guard against multiple emissions per render, ~600 ms settle).
- change edits like the sidebars do: `session.update_config(replace(state.config, ...))`
+ `controller.request_render()`.
- **pixel evidence**: `window.canvas.grab()` (canvas only, overlay-free) →
numpy stats; `window.grab().save(...)` for reviewer screenshots.
- exit with a nonzero code on failed assertions; 120 s timeout guard.

A working driver from the dodge/burn + toning verification lives in the
session scratchpad as `drive_verify.py` — copy its `grab`/`set_cfg`/state-machine
parts rather than rewriting.

## Gotchas

- `samples/06.raw` is a dark graveyard scene — burn/dodge probes need the sky
band (top of frame in raw coords) to see roll-off; mid-frame is near black.
- Mask vertices are raw-image normalized coords, not canvas coords; analyze
screenshots diff-based (`before - after > threshold`) instead of guessing
where a mask lands on the canvas.
- Status bar process label may lag when configs are set programmatically —
don't treat it as ground truth for the active mode.
8 changes: 5 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,15 @@ Every feature under `negpy/features/<name>/` follows this structure:

### CPU pipeline (`negpy/services/rendering/engine.py`)

`DarkroomEngine.process()` runs stages in order: base (geometry + normalization) → exposure → retouch → lab → local (dodge/burn) → toning → crop → finish. The cached stages (base, exposure, retouch, lab, local) go through `_run_stage()`, which hashes the stage config and skips re-execution if the hash matches the cached `CacheEntry`; toning/crop/finish run unconditionally. Source image change clears the whole cache; a process-mode change invalidates only base/exposure/retouch/lab. Note `settings.geometry` drives both the early geometry transform and the late crop stage.
`DarkroomEngine.process()` runs stages in order: base (geometry + normalization) → exposure → retouch → lab → toning → crop → finish. The cached stages (base, exposure, retouch, lab) go through `_run_stage()`, which hashes the stage config and skips re-execution if the hash matches the cached `CacheEntry`; toning/crop/finish run unconditionally. Source image change clears the whole cache; a process-mode change invalidates only base/exposure/retouch/lab. Note `settings.geometry` drives both the early geometry transform and the late crop stage.

**Dodge/burn lives in the exposure stage.** Polygon masks (`negpy/features/local/`, `LocalAdjustmentsConfig`) rasterise to a per-pixel **EV map** (`compute_local_ev_map`, Σ strength·alpha) consumed by the print curve as a print-exposure offset next to the WB `cmy_offsets` (`local_ev_scale` = −log10(2)/stretch range per channel) — burns roll into paper black through the toe, dodges lift through the shoulder, like enlarger exposure. There is no separate local stage or shader: the exposure `_run_stage` is keyed on `(settings.exposure, settings.local)`, the GPU binds the EV map as a texture in the exposure pass (`ev_scale.w` gates a 1×1 dummy when no masks), and tiled export slices the full-res EV map per tile. The UI section sits on the Exposure ("tone") page.

**Flat-field profile (rig-level) + lens distortion.** Flat-field is a per-rig **profile** (DB `flatfield_profiles`: name, path, **k1**), not a per-image edit. `FlatFieldConfig` (`apply`, `reference_path`, `k1`) mirrors the active profile into the recipe but the profile is canonical — `reference_path` and `k1` are **re-seeded from the active profile on every file load** (`session.py`), so a roll shares them. The *photometric* correction (illumination falloff) is baked at **source-load** (`apply_flatfield`, before the pipeline → before any warp, so the flat stays registered to the raw sensor grid). The *geometric* correction is a single radial coefficient **k1** (poly3 / Brown-Conrady, half-diagonal–normalized so it's rotation/aspect invariant) folded into the **geometry transform** (`apply_radial_distortion` / `transform.wgsl`), gated by the same `apply` toggle (`k1_eff = k1 if apply else 0`). Being a geometry op, k1 invalidates the **base stage** (CPU `base_key`; GPU `_detect_invalidated_stage` checks `flatfield.apply`/`k1`) — **not** `source_hash`/`flatfield_token` (that would force a RAW re-decode per slider step). Scale-to-fill is baked into the same remap (`compute_distortion_scale`, no empty borders). The correction must stay consistent across the image warp **and** the coordinate mappers (`create_uv_grid` forward, `map_coords_to_geometry` inverse via `map_point_radial`) so crop/autocrop/retouch/dodge-burn placements stay aligned; CPU stages read the effective k1 from `context.metrics["distortion_k1"]`, GPU passes `k1_eff` directly. Tiled export applies it on the CPU `img_rot`/IR path (the shader geometry uniform is zeroed there).

**Scene-linear working space.** The pipeline is **scene-linear internally**: the exposure stage (`exposure/logic.py`) emits linear reflectance (`10^-D`, no OETF), and the creative stages **Lab, Local, Toning, Finish operate on linear light**. The working-space OETF — the **ProPhoto RGB (ROMM) TRC** (gamma `1.8` with a linear toe below `1/512`), `working_oetf_encode`/`working_oetf_decode` in `kernel/image/logic.py`, mirrored in WGSL `oetf_encode`/`oetf_decode` — is applied **only as the final engine step** (CPU: end of `process()`; GPU: the `output_encode` pass in `gpu_engine.process_to_texture`), so the encoded buffer composes correctly with the ProPhoto ICC at the display/export boundary (`WORKING_COLOR_SPACE`). **Retouch runs in the linear island** like the other creative stages: dust *detection* is perceptual, so on the CPU it computes the detection luma on a `working_oetf_encode`d copy while *healing* in linear (`retouch/logic.py`) — the engine no longer brackets it. The GPU keeps a single encoded perceptual region (exposure encodes → clahe/retouch operate encoded → lab's `load_lin` decodes back to linear), equivalent within parity tolerance. Lab/Toning compute CIELAB directly from linear (`rgb_to_lab_working`/`lab_to_rgb_working` use the **ProPhoto ROMM primaries at D50**, no transfer decode); CIELAB ops are encoding-invariant, while the light ops (glow/halation highlight mask, chemical-toning luma masks) compute their masks on a `working_oetf_encode`d copy so display-domain thresholds stay valid. The flat master keeps its own log encoding and is never OETF-encoded. When changing the working **TRC**, update all seven sites together: the CPU `working_oetf_*` helpers, the WGSL `oetf_*` in `exposure/output_encode/lab/toning/metrics` shaders, and the chart (`charts.py`); changing the **primaries/white point** additionally means the CPU `_PROPHOTO_TO_XYZ`/`_XYZ_TO_PROPHOTO`/`_D50_WHITE` and the inline matrices + white in `lab.wgsl`/`toning.wgsl`. The greyscale export path re-encodes luma from the working TRC to gamma 2.2 to match its `GrayGamma2.2` tag (`_apply_color_management_u16_greyscale`).
**Scene-linear working space.** The pipeline is **scene-linear internally**: the exposure stage (`exposure/logic.py`) emits linear reflectance (`10^-D`, no OETF), and the creative stages **Lab, Toning, Finish operate on linear light** (dodge/burn happens inside the exposure stage, pre-curve). The working-space OETF — the **ProPhoto RGB (ROMM) TRC** (gamma `1.8` with a linear toe below `1/512`), `working_oetf_encode`/`working_oetf_decode` in `kernel/image/logic.py`, mirrored in WGSL `oetf_encode`/`oetf_decode` — is applied **only as the final engine step** (CPU: end of `process()`; GPU: the `output_encode` pass in `gpu_engine.process_to_texture`), so the encoded buffer composes correctly with the ProPhoto ICC at the display/export boundary (`WORKING_COLOR_SPACE`). **Retouch runs in the linear island** like the other creative stages: dust *detection* is perceptual, so on the CPU it computes the detection luma on a `working_oetf_encode`d copy while *healing* in linear (`retouch/logic.py`) — the engine no longer brackets it. The GPU keeps a single encoded perceptual region (exposure encodes → clahe/retouch operate encoded → lab's `load_lin` decodes back to linear), equivalent within parity tolerance. Lab/Toning compute CIELAB directly from linear (`rgb_to_lab_working`/`lab_to_rgb_working` use the **ProPhoto ROMM primaries at D50**, no transfer decode); CIELAB ops are encoding-invariant; the glow highlight mask computes on a `working_oetf_encode`d copy so its display-domain threshold stays valid, and chemical toning (selenium/sepia) is **density-driven** — it reads print density `D = −log10(t)` straight off the linear buffer (`toning/logic.py` `TONING_CONSTANTS`), no OETF bracket (only the B&W black point keeps one). The flat master keeps its own log encoding and is never OETF-encoded. When changing the working **TRC**, update all six sites together: the CPU `working_oetf_*` helpers, the WGSL `oetf_*` in `exposure/output_encode/lab/metrics` shaders, and the chart (`charts.py`); changing the **primaries/white point** additionally means the CPU `_PROPHOTO_TO_XYZ`/`_XYZ_TO_PROPHOTO`/`_D50_WHITE` and the inline matrices + white in `lab.wgsl`/`toning.wgsl`. The greyscale export path re-encodes luma from the working TRC to gamma 2.2 to match its `GrayGamma2.2` tag (`_apply_color_management_u16_greyscale`).

**Flat ("for editing elsewhere") render intent.** When `settings.exposure.render_intent == RenderIntent.FLAT` (`negpy/features/exposure/models.py`), the Print stage uses `PhotometricProcessor._process_flat`, which calls `apply_flat_curve` (`negpy/features/exposure/logic.py`) — a true **log-video master**: the normalized log signal (`val`) is emitted **directly** as the code value, positive-oriented, `code = clip(lift + gain·(1 − val), 0, 1)` (Cineon-like `flat_log_gain`/`flat_log_lift`; `flat_curve_params` returns gain+lift). Crucially it does **not** apply `10^-D` or the working-space OETF — `10^-D` is the log→linear *decode* that would turn the signal back into a normal-contrast positive (that decode is why earlier "flat" attempts looked like ordinary photos, not log). The result is flat/milky and fully invertible for editing. It bypasses the asymmetric print kernel (`_apply_print_curve_kernel`) entirely. Ignores auto density/grade, cast removal, toe/shoulder, surround/flare (WB rides as a per-channel log shift). The engine **bypasses retouch/lab/local/toning/finish** (crop still runs). This is the digital-intermediate master path. `ImageProcessor` forces the **CPU engine** for flat renders (no WGSL flat shader — guarantees numerical exactness). The desktop exposes it as a hybrid output intent: `AppState.flat_output`/`flat_format` (persisted) drive export, `flat_peek` drives a preview-only peek; `flat_master_config()` / `flat_export_config()` in `negpy/domain/models.py` derive the flat config (full-res by default when Flat is enabled; Print/Pixels sizing honoured when explicitly selected; 16-bit TIFF or linear DNG; export colour space follows the user's selection). DNG export writes an uncompressed 16-bit LinearRaw DNG with `tifffile` via `ImageProcessor._encode_dng_bytes` → `write_dng_linear` (`negpy/services/scanning/writer.py`); `NewSubfileType=0` + the DNG version tags are required for LibRaw/rawpy to accept it.
**Flat ("for editing elsewhere") render intent.** When `settings.exposure.render_intent == RenderIntent.FLAT` (`negpy/features/exposure/models.py`), the Print stage uses `PhotometricProcessor._process_flat`, which calls `apply_flat_curve` (`negpy/features/exposure/logic.py`) — a true **log-video master**: the normalized log signal (`val`) is emitted **directly** as the code value, positive-oriented, `code = clip(lift + gain·(1 − val), 0, 1)` (Cineon-like `flat_log_gain`/`flat_log_lift`; `flat_curve_params` returns gain+lift). Crucially it does **not** apply `10^-D` or the working-space OETF — `10^-D` is the log→linear *decode* that would turn the signal back into a normal-contrast positive (that decode is why earlier "flat" attempts looked like ordinary photos, not log). The result is flat/milky and fully invertible for editing. It bypasses the asymmetric print kernel (`_apply_print_curve_kernel`) entirely. Ignores auto density/grade, cast removal, toe/shoulder, surround/flare, dodge/burn masks (WB rides as a per-channel log shift). The engine **bypasses retouch/lab/toning/finish** (crop still runs). This is the digital-intermediate master path. `ImageProcessor` forces the **CPU engine** for flat renders (no WGSL flat shader — guarantees numerical exactness). The desktop exposes it as a hybrid output intent: `AppState.flat_output`/`flat_format` (persisted) drive export, `flat_peek` drives a preview-only peek; `flat_master_config()` / `flat_export_config()` in `negpy/domain/models.py` derive the flat config (full-res by default when Flat is enabled; Print/Pixels sizing honoured when explicitly selected; 16-bit TIFF or linear DNG; export colour space follows the user's selection). DNG export writes an uncompressed 16-bit LinearRaw DNG with `tifffile` via `ImageProcessor._encode_dng_bytes` → `write_dng_linear` (`negpy/services/scanning/writer.py`); `NewSubfileType=0` + the DNG version tags are required for LibRaw/rawpy to accept it.

For **roll consistency**, flat masters only match across frames when the roll shares one normalization baseline; the Export panel shows a "Bake roll baseline" nudge (reusing `request_batch_normalization`) when flat output is on but bounds aren't locked. Flat masters use the **selected export colour space** like the print path — the pipeline's ProPhoto RGB working buffer is color-managed to that space at encode time (`_apply_color_management_u16_rgb`).

Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.33.0
0.34.0
1 change: 0 additions & 1 deletion build.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@
"--add-data=negpy/features/toning/shaders:negpy/features/toning/shaders",
"--add-data=negpy/features/retouch/shaders:negpy/features/retouch/shaders",
"--add-data=negpy/features/lab/shaders:negpy/features/lab/shaders",
"--add-data=negpy/features/local/shaders:negpy/features/local/shaders",
"--add-data=negpy/features/finish/shaders:negpy/features/finish/shaders",
"--add-data=negpy/desktop/view/styles:negpy/desktop/view/styles",
"--add-data=icc:icc",
Expand Down
4 changes: 3 additions & 1 deletion docs/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Change Log


## 0.33.1
## 0.34.0

- Change: **Dodge & Burn moved into the print exposure** — masks now adjust exposure before the paper curve instead of scaling the finished image, which is how dodging and burning physically work. Strong burns and dodges roll off through the paper's toe and shoulder instead of clipping flat. The section moved to the Exposure tab; existing dodge/burn edits will render slightly differently.
- Change: **Selenium and Sepia work on print density** — both toners now convert silver density instead of tinting by brightness, matching the real baths. Selenium acts on the densest areas (deeper blacks, cooler shadows); sepia acts on the thinnest (warmer highlights, shadows hold). Existing toned edits will render differently.
- Fix: **exporting all RGB-scan triplets no longer fails** with "Input/output error" on most frames. Batch export was reusing stale saved paths for each frame's green/blue exposures instead of the ones the triplet was actually built from, so it tried to read files that weren't there; it now uses each frame's own exposures, the same as exporting one at a time.


Expand Down
18 changes: 12 additions & 6 deletions negpy/desktop/view/sidebar/controls_panel.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,14 +204,20 @@ def _init_ui(self) -> None:
[self.geometry_section, self.flatfield_section],
["geometry_section", "flatfield_section"],
),
("tone", "fa5s.sun", "Exposure — Colour, Tone", [self.colour_section, self.tone_section], ["colour_section", "tone_section"]),
(
"tone",
"fa5s.sun",
"Exposure — Colour, Tone, Dodge & Burn",
[self.colour_section, self.tone_section, self.local_section],
["colour_section", "tone_section", "local_section"],
),
("color", "fa5s.palette", "Color — Lab, Toning", [self.lab_section, self.toning_section], ["lab_section", "toning_section"]),
(
"finish",
"fa5s.brush",
"Finish — Retouch, Dodge & Burn, Finishing",
[self.retouch_section, self.local_section, self.finish_section],
["retouch_section", "local_section", "finish_section"],
"Finish — Retouch, Finishing",
[self.retouch_section, self.finish_section],
["retouch_section", "finish_section"],
),
]

Expand Down Expand Up @@ -489,13 +495,13 @@ def apply_shortcut_tooltips(self) -> None:

ton.selenium_slider.setToolTip(
tooltip_with_shortcut(
"Simulates selenium toning — adds a cool blue-purple cast to shadows. B&W mode only",
"Simulates selenium toning — converts the densest silver first: deeper blacks, cool eggplant shadows. B&W mode only",
["selenium_inc", "selenium_dec"],
)
)
ton.sepia_slider.setToolTip(
tooltip_with_shortcut(
"Simulates sepia toning — adds a warm brown cast across the full tonal range. B&W mode only",
"Simulates sepia bleach-redevelop toning — warms the highlights first while shadows hold. B&W mode only",
["sepia_inc", "sepia_dec"],
)
)
Expand Down
7 changes: 2 additions & 5 deletions negpy/desktop/view/sidebar/export.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,8 +126,7 @@ def _add_presets_section(self) -> None:
self.manage_presets_btn.setIcon(qta.icon("fa5s.sliders-h", color=THEME.text_primary))
self.manage_presets_btn.setSizePolicy(QSizePolicy.Policy.Maximum, QSizePolicy.Policy.Fixed)
preset_scope_tooltip = (
"Export the selected filmstrip frames with every enabled preset. "
"Use the menu arrow for current frame or all visible frames."
"Export the selected filmstrip frames with every enabled preset. Use the menu arrow for current frame or all visible frames."
)
self.export_presets_group = QWidget()
export_presets_row = QHBoxLayout(self.export_presets_group)
Expand All @@ -145,9 +144,7 @@ def _add_presets_section(self) -> None:
export_current_action.setToolTip("Export the current file with every enabled preset")
export_current_action.triggered.connect(self.controller.request_preset_export)
export_all_presets_action = preset_menu.addAction("Export all visible frames…")
export_all_presets_action.setToolTip(
"Export every visible frame in the filmstrip with every enabled preset"
)
export_all_presets_action.setToolTip("Export every visible frame in the filmstrip with every enabled preset")
export_all_presets_action.triggered.connect(self.controller.request_preset_batch_export)

self.export_presets_menu_btn = QToolButton()
Expand Down
Loading
Loading