Skip to content

Preview the denoised estimate rather than the noisy sample - #34

Open
AlexanderIstomin wants to merge 1 commit into
antirez:mainfrom
AlexanderIstomin:upstream-preview-estimate
Open

Preview the denoised estimate rather than the noisy sample#34
AlexanderIstomin wants to merge 1 commit into
antirez:mainfrom
AlexanderIstomin:upstream-preview-estimate

Conversation

@AlexanderIstomin

Copy link
Copy Markdown

Denoising previews currently decode video_latent — the sample the
sampler is stepping through. At high sigma that sample genuinely is
mostly noise, so on a short schedule the preview shows near-identical
coloured latent mush for most of the run and only resolves on the final
step. That defeats the purpose: there is nothing to judge and nothing to
cancel on.

Decoding the model's current estimate of the finished frame instead
gives a blurry version of the result from the first step, which then
sharpens.

The change

x0 = x_t + sigma * v, written into a scratch buffer and handed to the
preview callback. That is the same expression denoise_res already uses
to build its denoised buffer — worth noting the sign, since the Euler
step moves the sample along +v as sigma falls, so this engine's
velocity points toward the clean image rather than away from it. I
initially wrote the textbook - and it would have made previews worse;
the existing sampler is what corrected me.

The sampler's own trajectory is untouched, so final output is unchanged:
verified bit-identical at SSIM 1.000000 against the same seed before and
after.

Before and after

Same generation, same seed, steps 1 / 4 / 8 of an 8-pass turbo run at
352 square. Before: two indistinguishable noise fields, then the result.
After: a soft shape, a recognisable cat, the finished cat.

Scope and what I could not test

denoise_euler_gpu still previews the sample. Its velocity lives in a
BF16 GPU tensor and would need a separate readback to combine, and that
path selects on M5-class hardware which I do not have. Rather than push
an untested change through code I cannot exercise, I left it and
recorded why in a comment — happy to do it if you would prefer, or to
drop the comment if you would rather it stayed silent.

Measured on an M1 Pro. The commit also frees the scratch buffer on both
exit paths.

Denoising previews decode the sample the sampler is stepping through.
At high sigma that sample genuinely is mostly noise, so on a short
schedule the preview shows near-identical coloured latent mush for most
of the run and only resolves on the last step — which defeats the point
of a preview, since there is nothing to judge and nothing to cancel on.

Decoding the model's current estimate of the finished frame instead
gives a blurry version of the result from the first step, which then
sharpens. The estimate is x0 = x_t + sigma * v, the same expression
denoise_res already uses to build its denoised buffer; note the sign,
because the Euler step moves the sample along +v as sigma falls, so this
engine's velocity points toward the clean image rather than away from
it.

The estimate is written to a scratch buffer, so the sampler's own
trajectory is untouched and final output is unchanged — verified
bit-identical at SSIM 1.000000 against the same seed before and after.

denoise_euler_gpu still previews the sample: its velocity lives in a
BF16 GPU tensor and would need a separate readback to combine, and that
path selects on M5-class hardware, which I could not test on. The
comment records that rather than leaving it silently inconsistent.

Measured on an M1 Pro with a turbo checkpoint at 8 passes: previews go
from unusable to recognisable at step 1.
LachlanStuart added a commit to LachlanStuart/h3.c that referenced this pull request Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant