Skip to content

Commit ad2a6eb

Browse files
committed
fix(editor): hold the "one busy phase" promise through the first-run download
The 253 MB model download happens inside the `stt:transcribe` IPC — i.e. inside a run the store has already marked `running` — so it correctly reads as one longer "Transcribing…" phase, with no separate step and nothing falsely clickable. The edges of that first run did not hold up, though: - `SttManager.init` cached a REJECTED `prepare()`, so one dropped connection during the download failed every later transcription in the session: the remaining assets in the queue flipped red in the same frame, and the retry the editor offers was a dead control until the app was restarted — reconnecting changed nothing. The slot is cleared on failure now. - A transient failure now stops the queue: the still-queued assets inherit the verdict instead of each spending a full retry budget and stacking an identical toast. It is the engine that failed, not their media. - The transcript pane's read-only state is scoped PER ASSET again. Widening it to the timeline-wide gate made every other clip's word stream swallow Backspace and hover-bin clicks for the whole background pass, with nothing on screen to say why — the exact "looks live, ignores you" failure mode. A block being rewritten now shows a spinner + "Transcribing…" and dims its stream, and the per-clip empty line no longer tells the user to regenerate an asset that is mid-run. - `mixToMono` hoists its channel arrays out of the sample loop. One WebIDL call per sample per channel (~57 M for a ten-minute stereo recording) froze the window — spinners included — for seconds, which was survivable while the pass was user-triggered and is not now that it is automatic. Found by an adversarial review of the first-run sequence; 4 new tests cover the init retry, the queue stop and the per-asset read-only scoping.
1 parent 9992005 commit ad2a6eb

11 files changed

Lines changed: 206 additions & 16 deletions

File tree

electron/stt/index.test.ts

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,26 @@ describe("SttManager", () => {
9494
expect(fakeWhisperServer.stop).toHaveBeenCalledOnce();
9595
});
9696

97+
it("retries setup after a failed one instead of caching the rejection", async () => {
98+
// First run downloads a 253 MB model. Caching a rejected `prepare()` meant
99+
// one dropped connection failed every later transcription in the session —
100+
// including the retry the editor offers — until the app was restarted.
101+
const { ensureModels } = await import("./modelManager");
102+
const mocked = vi.mocked(ensureModels);
103+
mocked.mockClear();
104+
mocked.mockRejectedValueOnce(new Error("Failed to download: network unreachable"));
105+
const mgr = new SttManager();
106+
107+
await expect(mgr.init({ modelsBaseDir: "/tmp/fake-stt-models" })).rejects.toThrow(
108+
"network unreachable",
109+
);
110+
// The network came back: the next attempt must actually attempt.
111+
await mgr.init({ modelsBaseDir: "/tmp/fake-stt-models" });
112+
113+
expect(mocked).toHaveBeenCalledTimes(2);
114+
expect(fakeWhisperServer.start).toHaveBeenCalledOnce();
115+
});
116+
97117
it("setStatusSink replaces the previous sink (last call wins)", () => {
98118
const mgr = new SttManager();
99119
const a = vi.fn();

electron/stt/index.ts

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,18 @@ export class SttManager {
5959
if (options.statusSink) this.statusSink = options.statusSink;
6060
if (options.modelsBaseDir) this.modelsBaseDir = options.modelsBaseDir;
6161
if (!this.initPromise) {
62-
this.initPromise = this.prepare();
62+
// A REJECTED init must not be cached. `prepare()` downloads a 253 MB
63+
// model on first run, and caching its rejection meant one dropped
64+
// connection poisoned the whole app session: every later transcription
65+
// — including the retry the UI offers, and every remaining asset in the
66+
// auto-transcription queue — awaited the same stale rejection and failed
67+
// in milliseconds, with no way back short of quitting the app.
68+
// Reconnecting the network changed nothing. Clearing the slot on failure
69+
// makes the next attempt a real attempt.
70+
this.initPromise = this.prepare().catch((error) => {
71+
this.initPromise = null;
72+
throw error;
73+
});
6374
}
6475
return this.initPromise;
6576
}

src/components/ai-edition/NewEditorShell.tsx

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import {
1515
import { type AxcutClip, documentSchema } from "@/lib/ai-edition/schema";
1616
import { useProjectStore } from "@/lib/ai-edition/store/projectStore";
1717
import {
18+
useAssetTranscriptions,
1819
useAutoTranscription,
1920
useTimelineTranscriptGate,
2021
useTranscriptionStore,
@@ -139,6 +140,17 @@ export function NewEditorShell() {
139140
// routinely silent, and keying the transcript pane off it made the pane claim
140141
// "no audio track" for a project whose actual footage was mid-transcription.
141142
const transcriptGate = useTimelineTranscriptGate();
143+
// Per asset, for the transcript pane: only the block whose transcript is
144+
// actually being rewritten goes read-only. The gate answers "may a
145+
// transcript-dependent ACTION run?", which is a different question.
146+
const transcriptions = useAssetTranscriptions();
147+
const busyAssetIds = useMemo(
148+
() =>
149+
Object.values(transcriptions)
150+
.filter((v) => v.status === "running" || v.status === "queued")
151+
.map((v) => v.assetId),
152+
[transcriptions],
153+
);
142154
const tl = useTimeline();
143155
useUndoRedoShortcuts(() => {
144156
// ponytail: placeholder, wire when undo stack merges with history
@@ -1002,7 +1014,7 @@ export function NewEditorShell() {
10021014
transcripts: document?.transcripts ?? [],
10031015
assets: document?.assets ?? [],
10041016
trimRanges: document?.timeline?.trimRanges ?? [],
1005-
busy: transcriptGate.state === "pending",
1017+
busyAssetIds,
10061018
onSeek: handleSeek,
10071019
onAddTrimRange: handleAddTrimRange,
10081020
onRemoveTrimRange: handleRemoveTrimRange,

src/components/ai-edition/RightPanes.tsx

Lines changed: 34 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ import {
99
FileText,
1010
HelpCircle,
1111
Layout as LayoutIcon,
12+
Loader2,
1213
MousePointerClick,
1314
Palette,
1415
Sliders,
@@ -524,7 +525,7 @@ export function TranscriptPane({
524525
transcripts,
525526
assets,
526527
trimRanges,
527-
busy,
528+
busyAssetIds,
528529
onSeek,
529530
onAddTrimRange,
530531
onRemoveTrimRange,
@@ -537,7 +538,12 @@ export function TranscriptPane({
537538
transcripts: AxcutTranscript[];
538539
assets: AxcutAsset[];
539540
trimRanges: AxcutTrimRange[];
540-
busy: boolean;
541+
/** Assets whose transcript is being (re)generated right now — their block is
542+
* read-only while the run is in flight, since it is about to be replaced.
543+
* PER ASSET on purpose: a timeline-wide flag made every other clip's word
544+
* stream silently swallow Backspace and hover-bin clicks for the whole
545+
* background pass, with nothing on screen to say why. */
546+
busyAssetIds: readonly string[];
541547
onSeek: (sec: number) => void;
542548
onAddTrimRange: (target: TrimTarget, startSec: number, endSec: number, reason: string) => void;
543549
onRemoveTrimRange: (trimId: string) => void;
@@ -646,7 +652,7 @@ export function TranscriptPane({
646652
key={section.clip.id}
647653
index={idx}
648654
section={section}
649-
busy={busy}
655+
busy={busyAssetIds.includes(section.clip.assetId)}
650656
cueWordId={cueWordId}
651657
onSeek={onSeek}
652658
onAddTrimRange={onAddTrimRange}
@@ -952,6 +958,23 @@ const TranscriptClipBlock = memo(function TranscriptClipBlock({
952958
{ts("transcript.clipLabel", { index: index + 1 })} · {sourceRangeLabel}
953959
</span>
954960
</span>
961+
{/* A block whose transcript is being regenerated is read-only — say it,
962+
rather than letting the word stream look live and drop the edits. */}
963+
{busy ? (
964+
<span
965+
style={{
966+
display: "inline-flex",
967+
alignItems: "center",
968+
gap: 5,
969+
flexShrink: 0,
970+
font: "500 11px/1 var(--font-body)",
971+
color: "var(--accent)",
972+
}}
973+
>
974+
<Loader2 size={12} className="animate-spin" />
975+
{ts("transcript.transcribing")}
976+
</span>
977+
) : null}
955978
</span>
956979
{words.length === 0 ? (
957980
<p
@@ -963,14 +986,16 @@ const TranscriptClipBlock = memo(function TranscriptClipBlock({
963986
fontStyle: "italic",
964987
}}
965988
>
966-
{ts("transcript.noClipTranscript")}
989+
{busy ? ts("transcript.transcribing") : ts("transcript.noClipTranscript")}
967990
</p>
968991
) : (
969992
<div
970993
ref={editorRef}
971994
role="textbox"
972995
tabIndex={0}
973996
contentEditable={!busy}
997+
aria-busy={busy}
998+
aria-readonly={busy}
974999
suppressContentEditableWarning
9751000
spellCheck={false}
9761001
aria-label={ts("transcript.editorAria", { filename })}
@@ -984,7 +1009,11 @@ const TranscriptClipBlock = memo(function TranscriptClipBlock({
9841009
font: "400 13px/1.65 var(--font-body)",
9851010
color: "var(--fg)",
9861011
textWrap: "pretty",
987-
cursor: "text",
1012+
// Read-only while its transcript is being regenerated: the cursor
1013+
// and the wash are what stop it from reading as an editor that
1014+
// ignores you (see the `busy` note on TranscriptPane).
1015+
cursor: busy ? "progress" : "text",
1016+
opacity: busy ? 0.6 : 1,
9881017
outline: "none",
9891018
// no overflow on the per-clip editor — the
9901019
// parent paneBody (already overflow-y: auto) is the

src/components/ai-edition/TranscriptPane.gating.test.tsx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ function renderPane(
5151
transcripts={[]}
5252
assets={[ASSET]}
5353
trimRanges={[]}
54-
busy={false}
54+
busyAssetIds={[]}
5555
onSeek={vi.fn()}
5656
onAddTrimRange={vi.fn()}
5757
onRemoveTrimRange={vi.fn()}

src/components/ai-edition/TranscriptPane.keyboardCut.test.tsx

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -67,15 +67,19 @@ const W2_TRIMMED: AxcutTrimRange = {
6767
reason: "",
6868
};
6969

70-
function renderPane(trimRanges: AxcutTrimRange[], onAddTrimRange = vi.fn()) {
70+
function renderPane(
71+
trimRanges: AxcutTrimRange[],
72+
onAddTrimRange = vi.fn(),
73+
busyAssetIds: string[] = [],
74+
) {
7175
const view = render(
7276
<I18nProvider>
7377
<TranscriptPane
7478
clips={[CLIP]}
7579
transcripts={[TRANSCRIPT]}
7680
assets={[ASSET]}
7781
trimRanges={trimRanges}
78-
busy={false}
82+
busyAssetIds={busyAssetIds}
7983
onSeek={vi.fn()}
8084
onAddTrimRange={onAddTrimRange}
8185
onRemoveTrimRange={vi.fn()}
@@ -128,6 +132,27 @@ describe("keyboard cut with the caret between words", () => {
128132
expect(cutRange(onAddTrimRange)).toEqual([2, 3]); // "trois"
129133
});
130134

135+
it("keeps cutting while ANOTHER asset is being transcribed", () => {
136+
// The background pass runs on its own now, so a run on some other media must
137+
// not quietly turn this block into an editor that ignores Backspace — the
138+
// read-only state is scoped to the asset whose transcript is being rewritten.
139+
const { editor, onAddTrimRange } = renderPane([], vi.fn(), ["asset_other"]);
140+
caretBeforeWordAt(editor, 3);
141+
fireEvent.keyDown(editor, { key: "Backspace" });
142+
expect(cutRange(onAddTrimRange)).toEqual([2, 3]);
143+
});
144+
145+
it("stops cutting, visibly, while THIS asset is being transcribed", () => {
146+
// Its transcript is about to be replaced, so the block is read-only — and it
147+
// says so, instead of swallowing the keystroke in silence.
148+
const { editor, onAddTrimRange, getByText } = renderPane([], vi.fn(), ["asset_1"]);
149+
caretBeforeWordAt(editor, 3);
150+
fireEvent.keyDown(editor, { key: "Backspace" });
151+
expect(cutRange(onAddTrimRange)).toBeNull();
152+
expect(editor).toHaveAttribute("aria-busy", "true");
153+
expect(getByText("Transcribing…")).toBeInTheDocument();
154+
});
155+
131156
it("Backspace skips over an already-trimmed word instead of doing nothing", () => {
132157
// Hold Backspace and you land here: "deux" is already struck through, so the word
133158
// immediately before the caret has nothing left to cut. The keystroke used to
@@ -184,7 +209,7 @@ describe("keyboard cut with the caret between words", () => {
184209
transcripts={[TRANSCRIPT]}
185210
assets={[ASSET]}
186211
trimRanges={trims}
187-
busy={false}
212+
busyAssetIds={[]}
188213
onSeek={vi.fn()}
189214
onAddTrimRange={(_target, startSec, endSec) =>
190215
setTrims((prev) => [

src/components/ai-edition/TranscriptPane.sharedMedia.test.tsx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ function renderPane(onSeek: (sec: number) => void = vi.fn()) {
7373
transcripts={[TRANSCRIPT]}
7474
assets={[ASSET]}
7575
trimRanges={[]}
76-
busy={false}
76+
busyAssetIds={[]}
7777
onSeek={onSeek}
7878
onAddTrimRange={vi.fn()}
7979
onRemoveTrimRange={vi.fn()}

src/lib/ai-edition/store/transcriptionStore.test.ts

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,23 @@ describe("useTranscriptionStore", () => {
211211
expect(toastMocks.error).toHaveBeenCalledTimes(1);
212212
});
213213

214+
it("stops the queue on an engine failure instead of failing each asset in turn", async () => {
215+
// The model download died / whisper-server didn't come up: that verdict is
216+
// about the engine, so the remaining assets inherit it rather than each
217+
// spending a full retry budget and stacking an identical toast.
218+
transcribeMocks.transcribeAsset.mockRejectedValue(new Error("whisper-server exited"));
219+
loadDocument(makeDoc(["asset_1", "asset_2", "asset_3"]));
220+
221+
useTranscriptionStore.getState().sync(useProjectStore.getState().document);
222+
await whenTranscriptionIdle();
223+
224+
expect(transcribeMocks.transcribeAsset).toHaveBeenCalledTimes(1);
225+
const jobs = useTranscriptionStore.getState().jobs;
226+
expect(Object.values(jobs).map((j) => j.status)).toEqual(["failed", "failed", "failed"]);
227+
expect(jobs.asset_3?.failure?.message).toBe("whisper-server exited");
228+
expect(toastMocks.error).toHaveBeenCalledTimes(1);
229+
});
230+
214231
it("request() re-runs a failed asset and clears the remembered verdict", async () => {
215232
transcribeMocks.transcribeAsset.mockRejectedValueOnce(
216233
new Error("No audio track found in this video."),

src/lib/ai-edition/store/transcriptionStore.ts

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -257,6 +257,29 @@ function patchJob(assetId: string, runId: number, patch: Partial<TranscriptionJo
257257
});
258258
}
259259

260+
/**
261+
* Give every still-queued job the verdict that just came back from the engine.
262+
* Waiters are flushed so a `requestTimelineTranscripts()` awaiting the batch
263+
* settles instead of hanging on runs that will never happen.
264+
*/
265+
function failRemainingQueue(projectId: string, failure: TranscriptionFailure): void {
266+
const queued = Object.entries(useTranscriptionStore.getState().jobs)
267+
.filter(([, job]) => job.status === "queued")
268+
.map(([assetId]) => assetId);
269+
if (queued.length === 0) return;
270+
useTranscriptionStore.setState((state) => {
271+
if (state.projectId !== projectId) return state;
272+
const jobs = { ...state.jobs };
273+
for (const assetId of queued) {
274+
const job = jobs[assetId];
275+
if (job?.status !== "queued") continue;
276+
jobs[assetId] = { ...job, status: "failed", phase: undefined, failure };
277+
}
278+
return { jobs };
279+
});
280+
for (const assetId of queued) flushSettleWaiters(assetId);
281+
}
282+
260283
/** True while `runId` is still the attempt the store is tracking for this asset. */
261284
function isCurrentRun(assetId: string, runId: number): boolean {
262285
return useTranscriptionStore.getState().jobs[assetId]?.runId === runId;
@@ -380,6 +403,13 @@ async function runJob(assetId: string, job: TranscriptionJob): Promise<void> {
380403
patchJob(assetId, runId, { status: "failed", phase: undefined, failure });
381404
flushSettleWaiters(assetId);
382405
await persistPermanentFailure(projectId, assetId, failure);
406+
// A transient failure is about the ENGINE, not about this media: the model
407+
// download died, whisper-server didn't come up. Marching the rest of the
408+
// queue into the same wall would spend a full retry budget per asset and
409+
// stack one identical toast per asset. Fail them with the same verdict
410+
// instead — the gate then reads "failed" (not "queued forever"), and one
411+
// manual retry re-runs them all once the engine is back.
412+
if (failure.kind === "error") failRemainingQueue(projectId, failure);
383413
// A silent recording is an expected outcome, not an incident: the media
384414
// card and every gated button already say so. Only surface the noisy
385415
// (retryable) failures, plus anything the user asked for by hand.

src/lib/captioning/extractMono16k.ts

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -57,13 +57,23 @@ async function loadSourceVideoFile(videoUrl: string, signal?: AbortSignal): Prom
5757

5858
function mixToMono(audioBuffer: AudioBuffer): Float32Array {
5959
const { length, numberOfChannels } = audioBuffer;
60+
if (numberOfChannels === 0) return new Float32Array(length);
61+
// `getChannelData` is a WebIDL call, so calling it INSIDE the sample loop cost
62+
// one call per sample per channel — ~57 M of them for a ten-minute stereo
63+
// recording, seconds of blocked main thread. That was survivable while
64+
// transcription only ran when the user asked for it; it now runs by itself when
65+
// a project opens, and a frozen window (spinners included) is exactly what the
66+
// automatic pass must not look like. Hoisting the channel arrays out of the loop
67+
// leaves plain typed-array indexing.
68+
const channels: Float32Array[] = [];
69+
for (let c = 0; c < numberOfChannels; c++) channels.push(audioBuffer.getChannelData(c));
70+
// Mono source: the mixdown is a copy. `slice` keeps the caller's contract of
71+
// owning its buffer (the AudioBuffer's own array is reused by the context).
72+
if (numberOfChannels === 1) return channels[0].slice();
6073
const out = new Float32Array(length);
61-
if (numberOfChannels === 0) return out;
6274
for (let i = 0; i < length; i++) {
6375
let sum = 0;
64-
for (let c = 0; c < numberOfChannels; c++) {
65-
sum += audioBuffer.getChannelData(c)[i];
66-
}
76+
for (let c = 0; c < numberOfChannels; c++) sum += channels[c][i];
6777
out[i] = sum / numberOfChannels;
6878
}
6979
return out;

0 commit comments

Comments
 (0)