feat(LTX25-IC-LORA-REF-VIDEO): serve the IC-LoRA reference clip and its conditioning attention mask

[localai-org-maint-bot]

Gaps A15 and A16 of the LTX-2.5 completion scope, landed as one row. An
ic_lora render can now be conditioned on a reference clip through the public
video ABI, and an optional per-region mask attenuates that clip's attention.

WHY THEY ARE ONE ROW. Upstream applies
ConditioningItemAttentionStrengthWrapper at exactly one site
(iclora_utils.py:168-169) and its argument is always the
VideoConditionByReferenceLatent built two lines above it;
combined_image_conditionings never wraps. In this tree
Ltx2ConditionVideoByReference had zero production call sites, so A16 alone had
nowhere to attach that upstream also attaches to. Putting the mask on the
keyframe or image items that ARE reached would have invented behaviour upstream
does not have.

WHAT WAS ACTUALLY MISSING was smaller than the standing refusal claimed, and
settling that was the first thing this row did. Ltx2ReadFrameDirectory already
IS upstream's video_preprocess over a frame directory, and
Ltx2ConvVideoEncode already runs multi-frame for retake. What no code supplied
was the reference item's own geometry — the divisibility refusal,
height // reference_downscale_factor, and the temporal subsample that keeps
index 1 rather than stepping from 0 — plus the stage-1-only placement and the
whole mask half. So no media decoder, no checkpoint and no GPU lease were
needed.

THIS RETIRES A DEFECT THAT WAS ON main. Ltx2ModalityInput::attention_mask,
Ltx2PrepareSelfAttentionMask, self_bias on host and device and the additive
bias inside vt::AttentionCross were all built and all correct, and nothing in
src/ assigned either field: the only four assignments in the tree were in a
BuildModalities test helper in test_ltx2.cpp and test_ltx2_device.cpp.
That is the shape .agents/reachability.md names — a unit test that constructs
the type proves the class works, never that anything reaches it.
ltx2_video.cpp now makes that assignment on a production path.

ONE PREDICATE ROUTES AND REFUSES. serves_reference is bound once from the
recipe and read by the routing branch and by every refusal beside it, because a
refusal and its route predicate written as two expressions is how this campaign
shipped a silently wrong answer.

The oracle

Lightricks/LTX-2 @ fd4ded7f, EXECUTED. ltx_pipelines.iclora_utils cannot be
imported here — it reaches PyAV, which this project does not vendor — so
scripts/gen-ltx2-iclora-reference-goldens.py lifts its three functions BY AST
and compiles their own source text verbatim, and runs
append_ic_lora_reference_video_conditionings with only its frame decode, its
pixel preprocess and its VAE replaced by recording stubs, so every branch under
test is upstream's own control flow.

EVERY CASE CARRIES ITS REJECTED HYPOTHESIS AND THE GENERATOR REFUSES A CASE
WHOSE SEPARATION IS ZERO. That guard is not decorative: build_attention_mask
on a fixture with no PRIOR conditioning item is elementwise equal to the
plausible reading "cross on all existing rows" — measured 0 separating elements
— so such a fixture is a mute switch. The shipped fixture carries a prior item
and separates on 4.

A THIRD MUTE FIXTURE SURVIVED TO FRESH REVIEW, and the repair is in this branch.
mode="area" pools output index i over [floor(i*I/O), ceil((i+1)*I/O)), and
the port writes that general form. An integer-stride box filter
[i*(I//O), (i+1)*(I//O)) is elementwise EQUAL to it whenever O divides I —
and every fixture that reached the code was divisible. The only golden was
8 -> 2 and the reachability render reads a 64x64 mask and pools to 2x2, so
replacing the general form with the stride form survived BOTH suites entirely:
13/13 cases and 97/97 assertions on the value suite, 131/131 and 5225/5225 on
test_ltx2_video. The difference is not nothing — at 224 -> 3 the mutant is
~2300x further from torch than the port is. A 9 -> 2 golden now separates them
on 0.08158010244369507, the fixture asserts its own indivisibility so a later
edit cannot restore the mute switch, and the same mutation reds 2 assertions:
max |diff| 0.0815801 against the oracle and 5.96e-08 against the stride reading,
which is the mutant reproducing the rejected hypothesis exactly.

Two measurements shaped the fixture

The reachability render is 128x128, not the 64x64 every other case uses, because
the distilled recipe halves it for stage 1 and 64x64 gives a 1x1 spatial
reference grid where row-major and column-major patchify agree and a downscale
factor of 2 collapses onto 1.

The reference clips are CONSTANT frames, not ConditioningPpm's seeded ramp.
The VAE downsamples 32x, so two seeds of that ramp average to nearly the same
latent — measured absmax 0.539 against 0.464, with the two renders identical to
the byte. Constant frames survive it: 20 against 235 give 0.434 against 0.637.

The pixel claim is made on the skip-stage-2 arm, which is upstream's own
--skip-stage-2 (ic_lora.py:442-448) and not a test hook. MEASURED: stage 1's
latents for the two clips do differ (clean digests 7109293506366579875 against
12136535927875385707) and stage 2's inputs still differ, but stage 2 re-noises
to STAGE_2_DISTILLED_SIGMAS[0] = 0.909375 and on a 2-block random-weight DiT
the surviving difference falls below 1/255.

Mutations

Twelve, each confirmed APPLIED and COMPILED, each restored with an emptiness
assertion on git status. Ten DETECTED. Both survivors are identities,
proved rather than argued:

• M6 drops scale != 1 from the divisibility guard. For scale 1 the remaining
  conjunct is h % 1 || w % 1, which is 0 for every h and w, so the two
  predicates agree on every input and no fixture can separate them.
• M11 replaces serves_reference in the refusal with a string compare on
  pipeline_kind. Those are the same predicate over the recipe table as it
  stands, because ic_lora_reference is set by IcLoraRecipe alone. They stop
  being the same the moment a second recipe sets the flag, so this change adds
  the tripwire: test_ltx2_pipeline walks every kind against all four versions
  and requires exactly ONE resolved pair to carry it, naming
  ltx2_video.cpp's refusals as what must be re-read. The kind list is
  DERIVED, out of every pipeline_kind == "..." branch in
  ltx2_pipeline.cpp. It was a hand-copied eleven-entry array until fresh
  review, which is a list complete only until the next kind lands. Measured:
  adding a twelfth branch now reds the case and raises the assertion count by
  two, which is the new kind being walked; the hand-copied version stayed green.

M1, the reachability mutation, deletes the production vin.attention_mask
assignment: the reachability suite reds on 3 assertions — this said 4, and
fresh review re-measured it — while the value suite stays 97/97. M2 stops the
reference item being applied: the reachability suite throws while the value
suite stays 97/97.

WHICH ASSERTION DOES NOT RED UNDER M1 IS WORTH MORE THAN THE COUNT.
trace.ic_lora_attention_mask_rows == 16 still PASSES: the trace records that
the mask was built and how many rows it has, and the deleted line is the one
that hands it to the model, so the counter stays right about a mask nothing
consumes. Only the pixel compares see the drop. Gating this seam on the trace
alone would measure construction and not consumption.

The refusal fresh review found, and it was a silent one

ic_lora.py:230-233 refuses a conditioning_attention_strength outside
[0.0, 1.0] UNCONDITIONALLY. It sits in the method body, after
assert_resolution and outside every if args.conditioning_attention_mask is not None. This port guarded its copy on a non-empty mask directory, and the two
refusals below it did not close the hole either: 1.5 is not below 1.0, so the
scalar-arm refusal does not fire. The result was a knob that ACCEPTED 1.5,
refused nothing, and was then never read, because its only reader is inside the
mask branch. Accepted, ignored, silent, on every pipeline.

The guard is dropped and a gate rides with it. Without the fix the new subcase
does not merely mis-assert: the render is SERVED and the case fails on its own
FAIL, which is the shape of the defect rather than a restatement of it. The
refusal now reads:

ltx-2.5 video: 'conditioning_attention_strength' must be in [0.0, 1.0], got
1.500000. This is upstream's own refusal (ic_lora.py:230-233): the value
multiplies a mask in [0, 1] and the product becomes a log-space attention bias,
so a value above 1 would AMPLIFY attention rather than attenuate it.

Ordering is unchanged where it mattered: 0.5 with no mask still passes the range
check and still lands on the scalar-arm refusal.

ref_video_strength with no ref_video_dir is the same accept-and-ignore
shape, and it is recorded OWED rather than fixed. Upstream cannot express it —
--video-conditioning is nargs=2 — so there is no upstream reading to mirror,
and inventing a refusal would be a product decision.

Gates

Run on the MERGED head — origin/main was ten commits ahead and is merged in,
so what was measured is what would land. -j 2, CPU only, idle box.

THE WHOLE-TREE GATE IS NO LONGER PENDING. This body previously recorded it
as PENDING on CI, because a kill -9 during the row's scheduling corrupted
build/.ninja_deps, ninja rescheduled all 1338 edges from scratch, and under
-j 2 alongside four other compiling agents at load 20-33 that reached 76 of
1338 and stopped. It has now been run twice on an idle box and it passes:

• Fresh review, head 975bc0906: full build rc 0, ctest -j 2 100% tests
  passed, 0 tests failed out of 714, 7 skipped.

• This repair round, merged head: full build rc 0 over all 1337 edges, then
  ctest --test-dir build -j 2 --output-on-failure rc 0, 100% tests passed, 0
  tests failed out of 714, the same 7 skipped. Every skip is GPU-gated or
  checkpoint-gated and none is in this row's reach.

• test_ltx2_iclora_reference 102/102 over 14 cases, test_ltx2_video
  5237/5237 over 131, test_ltx2 4581/4581, test_ltx2_pipeline
  4726/4726, test_ltx2_retake 69/69

• scripts/agent-preflight.sh: rc 0, 155 ok and 5 SKIP, every skip a
  checker preflight cannot supply arguments for

• check-pr-size.py --base origin/main --head HEAD: rc 0 — run explicitly,
  because it is one of the five preflight skips

• agent-issue-index.py --refresh (316 open issues) then
  check-agent-record.py with the snapshot PRESENT: rc 0

Records

ltx25-ic-lora.md's two ## Owed rows are struck through with what discharged
them and what did not. Both cited #932 and #975, which 404 (tracked by
#2899); they now point at #3020, and so does the user-facing refusal text that
cited #932.

Refused by name and recorded owed: the EXR/HDR reference arm, tiled_encode for
a reference, reference IMAGES (upstream has no such flag, so this is the one
refusal in the file no later row can serve), the reference-AUDIO arm, the bf16
storage width of every buffer added here (the A24 campaign's standing
divergence), and upstream's scalar-only strength < 1.0 branch, which its own
CLI cannot reach either because the strength is assigned only alongside a mask.

FRESH REVIEW REPAIRED FIVE MORE RECORD DEFECTS, each of which read as a
measurement and was not one.

• A paragraph in ltx2_video.cpp said the reference arm "stays refused and LTX-2.5 reference video: the clip has no pixel path, and stage 2 must run UNFUSED — neither reason the refusal ever gave is still true #975
  stays open". This row SERVES it, LTX-2.5 reference video: the clip has no pixel path, and stage 2 must run UNFUSED — neither reason the refusal ever gave is still true #975 is one of 23 tracked files cite three issue numbers that 404 (#611, #1854, #2295), so those obligations cannot be traced #2899's five 404s, and the
  paragraph was orphaned besides — it sat between an unrelated local and the new
  section header, while the correct version of the same text already stood
  beside wants_reference_video. Deleted rather than corrected.
• docs/models/ltx-2-5.md named ltx-2.5-ic-lora-depth.safetensors, which the
  pinned checkout does not contain ANYWHERE. The doc's convention is real
  published filenames, so an invented one reads as a pin. It is now an
  unmistakable placeholder, with the adapters upstream actually publishes named
  beside it and the reason none fits: upstream's only 2.5 IC-LoRA is
  DFRPipeline's stage-2 detailing adapter, and every published
  reference-control adapter is 2.3-era or 19b. Verified against the pinned
  checkout at fd4ded7f alone; nothing was downloaded.
• The spec's separation table recorded 0.1890125870704651 and
  0.16791561245918274. The committed .inc — and a byte-identical
  regeneration against the pin — say 0.20678424835205078 and
  0.049749284982681274. Re-derived from the generated file rather than copied
  forward a third time.
• The spec cited the generator as tools/ltx2/gen_iclora_reference_goldens.py;
  that directory does not exist. It is
  scripts/gen-ltx2-iclora-reference-goldens.py.
• The spec named Ltx2ReadMaskFrameDirectory(dir, height, width) twice. grep
  finds it in those two lines and nowhere else: the tree composes the mask read
  at the call site out of Ltx2ReadFrameDirectory and
  Ltx2MaskVideoFromPixels. A function named in a spec and absent from the tree
  sends the next reader looking for a seam that was never built.

Issue #3020's records section named docs/USAGE.md, and this change does not
write it. STRUCK there rather than written: that file no longer carries
per-model commands — its own "Find a model recipe" section routes commands,
weights, settings and limits to docs/models/<family>.md, which is where these
knobs are — and it names no ltx2-gen flag at all. Writing them in both places
would put one fact in two documents.

Spec: .agents/specs/ltx25-ic-lora-ref-video.md, committed before any
implementation.

Closes #3020

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]