Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion client/src/hooks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ grep -i "what you want to do" client/src/hooks/README.md
| `useYoutubeIngest` | One YouTube brain-ingest job slot: start/cancel + SSE progress + terminal-frame handling via `POST /api/brain/youtube/ingest`. Returns `{ active, percent, stage, start(body), cancel }` — `start` takes the whole payload (`{ url, captureTranscript, downloadVideo, ingestAudio, note, agentPrompt, tags }`), not a bare URL. `onComplete(ingest)` fires with the stored ingest record; non-fatal `warnings[]` on the terminal frame are toasted automatically. | Quick Capture's YouTube path, and any other surface that ingests a video into the brain. |
| `useYoutubeTrackImport` | One YouTube-audio-import job slot (#1945): start/cancel + SSE progress + terminal-frame handling via `POST /api/tracks/import/youtube`. Returns `{ active, percent, start(url, context), cancel }`; `onComplete(track, context)` fires with the finished Track. Call once per UI surface that can independently kick off an import — a shared slot would let one surface's kickoff orphan another's in-flight job. | Any picker that lets a user paste a YouTube URL to create a Track (Music Video's create form and track-change row). |
| `useVideoGenFieldState` | Mutable VideoGen field values, setters, and lifecycle refs initialized from URL handoff values. | Internal state slice composed by `useVideoGenForm`; keep model reconciliation and submission behavior in their focused units. |
| `useVideoGenForm` | `useVideoGenForm({ models, status, availableLoras, grokEnabled })` → every VideoGen form field + setter, the URL-param prefill paths (ImageGen handoff, Continue, Remix, `?lora=`), the mode/backend transitions that clear stale inputs, the derived model/keyframe/IC gates (`extendModeBlocked`, `a2vModeBlocked`, `icLoraModeBlocked`, `keyframesError`), `applyRemix` / `applyResumedParams`, and `buildGeneratePayload()`. | The /media/video form — the single client-side source of truth for the payload `server/routes/videoGen.js` validates. Reuse it for any second entry point that generates video. |
| `useVideoGenForm` | `useVideoGenForm({ models, modelContext, availableLoras, grokEnabled })` → every VideoGen form field + setter, the URL-param prefill paths (ImageGen handoff, Continue, Remix, `?lora=`), the mode/backend transitions that clear stale inputs, the derived model/keyframe/IC gates (`extendModeBlocked`, `a2vModeBlocked`, `icLoraModeBlocked`, `keyframesError`), `applyRemix` / `applyResumedParams`, and `buildGeneratePayload()`. | The /media/video form — the single client-side source of truth for the payload `server/routes/videoGen.js` validates. Reuse it for any second entry point that generates video. |
| `useVideoGenSubmitFlow` | Builds the current VideoGen wire payload and enveloped prompt from a form-state snapshot. | Internal submission slice composed by `useVideoGenForm`; all local, Grok, and federated request shaping remains in `videoGenSubmission`. |
| `useVideoGenValidation` | Derives keyframe errors and all VideoGen submit-blocking predicates from the current fields and model capabilities. | Internal validation slice composed by `useVideoGenForm`; also exports `validateVideoKeyframes` for focused tests. |
| `useVideoDownload` | One full-video-download job slot (#1946): start/cancel + SSE progress + terminal-frame handling via `POST /api/devtools/video-download`. Returns `{ active, percent, stage, context, start(url, context), cancel }`; `onComplete(video)` fires with the finished video-history entry. | The Dev Tools Video Downloader page. |
Expand Down
57 changes: 30 additions & 27 deletions client/src/hooks/useVideoGenForm.js
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,14 @@ const editableRemixModel = (models, defaultModelId) => {
* that clear now-irrelevant inputs, the derived model/keyframe/IC gates, and
* `buildGeneratePayload()` — the single client-side source of truth for the
* shape `server/routes/videoGen.js` validates. `VideoGen.jsx` keeps the
* fetching (status/models/history/gallery), the SSE run pipeline, the batch
* queue, and the rendering.
* fetching (status/model-context/history/gallery), the SSE run pipeline, the
* batch queue, and the rendering.
*
* The caller supplies the fetched context the form has to react to:
* - `models` / `status` — from `getVideoGenStatus()`; drive the model
* dropdown, the default-model seed, and the mode-compatibility fallback.
* - `models` / `modelContext` — from `getVideoGenModelContext()`; drive the
* model dropdown, the default-model seed, and the mode-compatibility
* fallback. Deliberately NOT `getVideoGenStatus()`: that route shells out
* to python, and the picker must not wait on the interpreter probe.
* - `availableLoras` — the installed LoRA library, for name resolution.
* - `grokEnabled` — the Settings → Image Gen toggle that reveals the
* Local/Grok backend switch.
Expand All @@ -69,7 +71,7 @@ const editableRemixModel = (models, defaultModelId) => {
* wire accepts — kept here rather than in the page so there stays exactly
* one builder for what `server/routes/videoGen.js` validates.
*/
export function useVideoGenForm({ models, status, availableLoras, grokEnabled, remoteSubmissionFields = null }) {
export function useVideoGenForm({ models, modelContext, availableLoras, grokEnabled, remoteSubmissionFields = null }) {
const [searchParams, setSearchParams] = useSearchParams();
const incomingSourceImage = searchParams.get('sourceImageFile');
const incomingAudioFilename = searchParams.get('audioFilename');
Expand Down Expand Up @@ -163,11 +165,11 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r
});
return () => { cancelled = true; };
}, [incomingAudioFilename, setSearchParams]);
// Seed the model dropdown from the server's default once /status lands,
// without clobbering a Remix/deep-link/user pick that already set it.
// Seed the model dropdown from the server's default once the model context
// lands, without clobbering a Remix/deep-link/user pick that already set it.
useEffect(() => {
if (status?.defaultModel) setModelId((prev) => prev || status.defaultModel);
}, [status?.defaultModel]);
if (modelContext?.defaultModel) setModelId((prev) => prev || modelContext.defaultModel);
}, [modelContext?.defaultModel]);

// Re-sync when ImageGen pipes a new image via ?sourceImageFile=...
useEffect(() => {
Expand Down Expand Up @@ -370,7 +372,8 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r
// server would 400 on submit; we proactively swap to a compatible model.
// a2v fallback preference: highest-memory model that fits this machine
// (leaving headroom for the OS + text encoder) > the largest if none fit.
// Other modes: status.defaultModel (if compatible) > first compatible model.
// Other modes: the context's defaultModel (if compatible) > first
// compatible model.
useEffect(() => {
if (!modelId || models.length === 0) return;
const current = models.find((m) => m.id === modelId);
Expand All @@ -385,40 +388,40 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r
// so the user can at least try, and the install banner / OOM surfaces
// the real constraint instead of a silent dropdown change.
const reserveGb = 16;
// typeof === 'number' (not `status?.systemMemoryGb ? ...`) so a server
// legitimately reporting a tiny number (0 GB after rounding on a
// sub-GB box) flows through the `fits` check and lands on the
// smallest model. The truthiness shortcut would collapse 0 with
// "absent" and pick the LARGEST model on a tiny machine.
const budget = typeof status?.systemMemoryGb === 'number'
? Math.max(0, status.systemMemoryGb - reserveGb)
// typeof === 'number' (not `modelContext?.systemMemoryGb ? ...`) so a
// server legitimately reporting a tiny number (0 GB after rounding on a
// sub-GB box) flows through the `fits` check and lands on the smallest
// model. The truthiness shortcut would collapse 0 with "absent" and pick
// the LARGEST model on a tiny machine.
const budget = typeof modelContext?.systemMemoryGb === 'number'
? Math.max(0, modelContext.systemMemoryGb - reserveGb)
: Number.POSITIVE_INFINITY;
const sortedDesc = [...visibleModels].sort(
(a, b) => videoModelMemoryGb(b) - videoModelMemoryGb(a),
);
const fits = sortedDesc.find((m) => videoModelMemoryGb(m) <= budget);
fallback = (fits || sortedDesc[sortedDesc.length - 1])?.id || '';
} else {
const defaultModel = models.find((m) => m.id === status?.defaultModel);
const defaultModel = models.find((m) => m.id === modelContext?.defaultModel);
if (defaultModel && isModelAllowedForMode(defaultModel, mode)) {
fallback = defaultModel.id;
} else {
fallback = visibleModels[0]?.id || status?.defaultModel || models[0]?.id || '';
fallback = visibleModels[0]?.id || modelContext?.defaultModel || models[0]?.id || '';
}
}
if (!fallback || fallback === modelId) return;
// Toast only for the stale-id case (model removed from catalog). The
// mode-incompatibility swap is expected behavior after a mode change —
// no need to surface it. Name the destination model so users on a2v
// don't think they landed on `status.defaultModel` (they may not have —
// don't think they landed on `modelContext.defaultModel` (they may not have —
// a2v picks the largest-fits model, which is often a dgrauet entry).
if (!current && staleModelToastRef.current !== modelId) {
staleModelToastRef.current = modelId;
const fallbackName = models.find((m) => m.id === fallback)?.name || fallback;
toast(`Original model "${modelId}" is no longer available — switched to "${fallbackName}"`);
}
applyModelSelection(fallback);
}, [modelId, models, status?.defaultModel, status?.systemMemoryGb, mode, visibleModels, applyModelSelection]);
}, [modelId, models, modelContext?.defaultModel, modelContext?.systemMemoryGb, mode, visibleModels, applyModelSelection]);

const currentModel = models.find((m) => m.id === modelId);

Expand All @@ -435,17 +438,17 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r
setNumFrames(frames);
}, [audioDurationSec, currentModel, fps, mode]);

// A source model can reach this hook either through a URL handoff before
// /status has populated `models`, or from the in-page gallery after it has.
// Resolve both cases here. The fallback is deliberately limited to models
// A source model can reach this hook either through a URL handoff before the
// model context has populated `models`, or from the in-page gallery after it
// has. Resolve both cases here. The fallback is deliberately limited to models
// that can run a text remix and expose all restored prompt/sampler controls;
// if no such model is installed we leave the source selected rather than
// silently changing a faithful re-render.
useEffect(() => {
if (!remixSourceModel || models.length === 0) return;
const source = models.find((model) => model.id === remixSourceModel.id);
if (source && !remixSourceModel.preserveConditioning && !hasEditableRemixControls(source)) {
const target = editableRemixModel(models, status?.defaultModel);
const target = editableRemixModel(models, modelContext?.defaultModel);
if (target) {
setModelId(target.id);
setRemixModelFallback({
Expand All @@ -459,7 +462,7 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r
setRemixModelFallback(null);
}
setRemixSourceModel(null);
}, [remixSourceModel, models, status?.defaultModel]);
}, [remixSourceModel, models, modelContext?.defaultModel]);

// Until the user deliberately chooses a size, model changes carry their own
// native default canvas. This is material for H3: the shared 768x512 default
Expand Down Expand Up @@ -712,7 +715,7 @@ export function useVideoGenForm({ models, status, availableLoras, grokEnabled, r
keyframesActive,
mode,
numFrames,
pixelBudget: status?.fflfLtx2PixelBudget,
pixelBudget: modelContext?.fflfLtx2PixelBudget,
sourceImageFile,
sourceImageUpload,
width,
Expand Down
34 changes: 17 additions & 17 deletions client/src/hooks/useVideoGenForm.test.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -67,13 +67,13 @@ const H3_REF2VA = {
defaultFrames: 124,
};
const MODELS = [MLX, LTX2];
const STATUS = { connected: true, defaultModel: MLX.id };
const MODEL_CONTEXT = { defaultModel: MLX.id };

const render = ({ models = MODELS, status = STATUS, availableLoras = [], grokEnabled = false, url = '/media/video' } = {}) => {
const render = ({ models = MODELS, modelContext = MODEL_CONTEXT, availableLoras = [], grokEnabled = false, url = '/media/video' } = {}) => {
const wrapper = ({ children }) => <MemoryRouter initialEntries={[url]}>{children}</MemoryRouter>;
return renderHook(
(props) => useVideoGenForm(props),
{ wrapper, initialProps: { models, status, availableLoras, grokEnabled } },
{ wrapper, initialProps: { models, modelContext, availableLoras, grokEnabled } },
);
};

Expand All @@ -94,7 +94,7 @@ describe('useVideoGenForm', () => {
vi.unstubAllGlobals();
});

it('seeds the model from status.defaultModel without clobbering a URL pick', async () => {
it('seeds the model from modelContext.defaultModel without clobbering a URL pick', async () => {
const { result } = render();
await waitFor(() => expect(result.current.modelId).toBe(MLX.id));

Expand Down Expand Up @@ -324,7 +324,7 @@ describe('useVideoGenForm', () => {
it('clears sampler overrides on an automatic mode-compatible model fallback', async () => {
const { result } = render({
models: [WAN_T2V, WAN_TI2V],
status: { connected: true, defaultModel: WAN_T2V.id },
modelContext: { defaultModel: WAN_T2V.id },
});
await waitFor(() => expect(result.current.modelId).toBe(WAN_T2V.id));
act(() => {
Expand All @@ -340,7 +340,7 @@ describe('useVideoGenForm', () => {
it('does not submit chunks for a T2V-only Wan profile', async () => {
const { result } = render({
models: [WAN_T2V],
status: { connected: true, defaultModel: WAN_T2V.id },
modelContext: { defaultModel: WAN_T2V.id },
});
await waitFor(() => expect(result.current.modelId).toBe(WAN_T2V.id));
act(() => {
Expand All @@ -353,7 +353,7 @@ describe('useVideoGenForm', () => {
it('normalizes MiniMax H3 to its fixed temporal and sampler contract', async () => {
const { result } = render({
models: [MLX, H3],
status: { connected: true, defaultModel: MLX.id },
modelContext: { defaultModel: MLX.id },
});
await waitFor(() => expect(result.current.modelId).toBe(MLX.id));
act(() => {
Expand Down Expand Up @@ -388,7 +388,7 @@ describe('useVideoGenForm', () => {
// Substitutable prompt conditioner (#4081).
describe('text encoder selection', () => {
const renderWithH3 = async () => {
const rendered = render({ models: [MLX, H3], status: { connected: true, defaultModel: MLX.id } });
const rendered = render({ models: [MLX, H3], modelContext: { defaultModel: MLX.id } });
await waitFor(() => expect(rendered.result.current.modelId).toBe(MLX.id));
act(() => rendered.result.current.handleModelChange(H3.id));
await waitFor(() => expect(rendered.result.current.modelId).toBe(H3.id));
Expand All @@ -398,7 +398,7 @@ describe('useVideoGenForm', () => {
// An empty list is what hides the picker; a model with substitutions
// exposes them straight off the server-decorated entry.
it('exposes only the selected model’s options', async () => {
const { result } = render({ models: [MLX, H3], status: { connected: true, defaultModel: MLX.id } });
const { result } = render({ models: [MLX, H3], modelContext: { defaultModel: MLX.id } });
await waitFor(() => expect(result.current.modelId).toBe(MLX.id));
expect(result.current.textEncoderOptions).toEqual([]);
act(() => result.current.handleModelChange(H3.id));
Expand Down Expand Up @@ -465,7 +465,7 @@ describe('useVideoGenForm', () => {
it('preserves H3 native 32px-grid geometry in the submitted payload', async () => {
const { result } = render({
models: [MLX, H3],
status: { connected: true, defaultModel: MLX.id },
modelContext: { defaultModel: MLX.id },
});
await waitFor(() => expect(result.current.modelId).toBe(MLX.id));
act(() => result.current.handleModelChange(H3.id));
Expand All @@ -482,7 +482,7 @@ describe('useVideoGenForm', () => {
it('offers MiniMax H3 image mode, chaining and a non-advisory last frame', async () => {
const { result } = render({
models: [MLX, H3],
status: { connected: true, defaultModel: MLX.id },
modelContext: { defaultModel: MLX.id },
});
await waitFor(() => expect(result.current.modelId).toBe(MLX.id));
act(() => {
Expand Down Expand Up @@ -791,7 +791,7 @@ describe('useVideoGenForm', () => {
it('moves a fixed-profile remix to an editable model while preserving its restored controls', async () => {
const { result } = render({
models: [MLX, H3],
status: { connected: true, defaultModel: MLX.id },
modelContext: { defaultModel: MLX.id },
});
await waitFor(() => expect(result.current.modelId).toBe(MLX.id));

Expand All @@ -818,7 +818,7 @@ describe('useVideoGenForm', () => {
it('uses the same editable-model fallback for a cross-page Remix handoff', async () => {
const { result } = render({
models: [MLX, H3],
status: { connected: true, defaultModel: MLX.id },
modelContext: { defaultModel: MLX.id },
url: `/media/video?modelId=${H3.id}&numFrames=124&steps=9&guidanceScale=0`,
});

Expand Down Expand Up @@ -938,10 +938,10 @@ describe('useVideoGenForm — i2v reference mode (#4874)', () => {
lastFrameAnchored: true, supportedModes: RUNTIME_MODES,
};
const LTX25_MODELS = [LTX25, LTX2];
const LTX25_STATUS = { connected: true, defaultModel: LTX25.id };
const LTX25_MODEL_CONTEXT = { defaultModel: LTX25.id };

const inImageMode = async (opts = {}) => {
const { result } = render({ models: LTX25_MODELS, status: LTX25_STATUS, ...opts });
const { result } = render({ models: LTX25_MODELS, modelContext: LTX25_MODEL_CONTEXT, ...opts });
await act(async () => { result.current.handleModeChange('image'); });
return result;
};
Expand Down Expand Up @@ -982,15 +982,15 @@ describe('useVideoGenForm — i2v reference mode (#4874)', () => {
(props) => useVideoGenForm(props),
{
wrapper: ({ children }) => <MemoryRouter initialEntries={['/media/video']}>{children}</MemoryRouter>,
initialProps: { models: [], status: LTX25_STATUS, availableLoras: [], grokEnabled: false },
initialProps: { models: [], modelContext: LTX25_MODEL_CONTEXT, availableLoras: [], grokEnabled: false },
},
);
await act(async () => {
result.current.applyResumedParams({ modelId: LTX25.id, mode: 'image', i2vReferenceMode: 'inspire' });
});
expect(result.current.i2vReferenceMode).toBe('inspire');

rerender({ models: LTX25_MODELS, status: LTX25_STATUS, availableLoras: [], grokEnabled: false });
rerender({ models: LTX25_MODELS, modelContext: LTX25_MODEL_CONTEXT, availableLoras: [], grokEnabled: false });
await waitFor(() => expect(result.current.currentModel?.id).toBe(LTX25.id));
expect(result.current.i2vReferenceMode).toBe('inspire');
});
Expand Down
Loading