Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 49 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -444,7 +444,37 @@ no editing logic in the adapter.
*before* the fit scale, so the preview, the still and the export all follow and the
inspector's sliders still have the last word. Clips already the delivery shape and
360-reframed clips are left out (that camera *is* the framing decision), and a pass
that changes nothing writes no revision. The **agent task queue** is a real `tasks` table (one row per `Task`,
that changes nothing writes no revision.
**One cut, every platform**: the same project can be delivered at several
frames in one pass (a 9:16 Reel, a 1:1 post and a 16:9 upload), which is
what exposed the tension in smart crop — its crop is baked into the
transform for *one* shape, and framing for a second overwrote the first. So
a clip carries **`Clip.framings`**, a crop per delivery shape (`Framing`,
keyed by the reduced ratio `Delivery::ratio`, `(9, 16)`) beside the
transform's, and **`Timeline::for_delivery(delivery)`** (pure +
unit-tested) is the render of the cut at another frame: a copy whose format
is that delivery and whose clips wear the crop they carry for its shape — the
same change-the-timeline-not-the-graph pattern as `for_render`, so the graph
builders never learned about it. A clip with no framing for the shape keeps
the crop it has (never throw away a hand-made crop), which is why the framing
pass writes an *identity* framing for a shot already that shape: a lookup
miss would otherwise leave a 16:9 shot cut 9:16 delivering at 16:9 as the
strip its 9:16 crop keeps. Generated captions are re-fit to the new aspect
(`fit_size` again); typed titles are left alone. The framing pass is the
smart-crop trio again for the *other* shapes — `framing_inputs(deliveries)`
under the lock (the project frame's own ratio excluded, duplicates
collapsed), the static `sample_framings` with it released (**one** salience
decode per clip, a crop per shape from it — the map is a property of the
shot, the crop of the frame), `apply_framings` under it as one `Frame for
9:16, 1:1` revision that a re-run leaves alone — and `engine::render_variants`
renders `ExportVariant`s (a `Delivery` + an output path; `ExportVariant::beside`
names each file by shape, `cut-9x16.mp4`, an `x` because `:` is not a Windows
filename character) **one after another**, each variant's `resolution` / `fit`
taken from its delivery, reporting a `VariantProgress` (which file of how many
plus the overall fraction). Sequential on purpose: an export takes every core
it is given and `cpu::lease` would serialize them anyway, and a cancel is
then clean — the file in flight is deleted, the finished ones kept.
The **agent task queue** is a real `tasks` table (one row per `Task`,
columns not JSON): `add_task` / `list_tasks` / `claim_next_task` / `complete_task`
/ `fail_task` / `resolve_task` / `remove_task` drive the `queued → working →
ready → done` (or `failed`) lifecycle in `model.rs`.
Expand Down Expand Up @@ -564,7 +594,15 @@ both writes the GUI picker makes, though the picker itself only re-reads at
launch, so a model an agent selects shows there on the next start.
`smart_crop` frames each shot for the delivery frame (the server `instructions`
pair it with `set_delivery_format`, since reshaping to 9:16 otherwise keeps
whatever was in the middle). `generate_captions` / `clear_captions` caption the
whatever was in the middle). `export_variants` is the one-call multi-format
delivery: `formats` are shape names (`9:16` / `1:1` / `4:5` / `16:9`, or
`WxH` — `Delivery::parse`), it runs the framing pass first unless
`smart_crop` is false (the one write it makes, `project-changed` only when a
clip actually changed), renders through `render_variants` with progress on
the client's token naming the file in flight, and reports each file with the
platforms it is `ready_for` and its non-tip issues — judged at *that* file's
frame via `cut_summary(Some(frame))`, so the agent does not run
`platform_check` per variant afterwards. `generate_captions` / `clear_captions` caption the
cut; its `style` picks `lines` or `word_punch` and the `instructions` say to
prefer the latter for a vertical cut, since nothing in a tool list tells an
agent that the subtitle shape is not what social captions look like. They also
Expand Down Expand Up @@ -871,7 +909,15 @@ loudness normalize, and a **Range: In → out** choice when marks are set. It **
the frame the project is cut for** (`initialExport`): the preset whose resolution is
that frame when one matches, else the default preset with its resolution cleared so
"Project frame" renders — otherwise a 9:16 project opened its export already
landscape and the readiness panel warned about the shape the user had just chosen. `MediaBin`'s
landscape and the readiness panel warned about the shape the user had just chosen.
Its **Deliver to** section is the multi-format export: shape chips (the
`DELIVERY_PRESETS` minus Source) that each add a file beside the chosen path
named by shape (`variantPath`, the bun-tested mirror of
`ExportVariant::beside`), a *Smart crop each shot for every shape* toggle, and a
per-file readiness line judged at that file's frame (`platformCheck([w, h])`),
in place of the single panel — with shapes picked, the Scaling rows hide (each
delivery brings its own resolution and fit) and the button reads `Export N
files`; `export-progress` then carries `variant` / `total`. `MediaBin`'s
**Transcript tab is an editing surface**: lines resolve to the clip carrying them,
click seeks, the playhead line highlights, and `×` cuts the sentence from the timeline
(`cut_clip_range`); cut lines render struck through. When it is *empty* it says which
Expand Down
71 changes: 71 additions & 0 deletions crates/kerf-app/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1374,6 +1374,76 @@ async fn export_timeline(
.await
}

/// Render the cut once per delivery frame — one file per shape beside
/// `output_path`, named by shape (`cut-9x16.mp4`). With `smart_crop`, every
/// shot is framed for every shape first (one revision, reused by later
/// exports); the project frame's own crop is never touched. Streams the same
/// `export-progress` event as a single export, with `variant` / `total` added.
#[tauri::command]
async fn export_variants(
app: AppHandle,
state: State<'_, AppState>,
output_path: String,
formats: Vec<Delivery>,
smart_crop: bool,
options: ExportOptions,
) -> CmdResult<Vec<String>> {
if formats.is_empty() {
return Err("pick at least one delivery frame".to_string());
}
let base = std::path::PathBuf::from(&output_path);
let mut deliveries: Vec<Delivery> = Vec::new();
for d in formats {
let d = Delivery::new(d.width, d.height, d.fit);
if !deliveries.contains(&d) {
deliveries.push(d);
}
}
let variants: Vec<kerf_core::ExportVariant> = deliveries
.iter()
.map(|d| kerf_core::ExportVariant::beside(&base, *d))
.collect();
let shared = state.project.clone();
let cancel = state.export_cancel.clone();
cancel.store(false, Ordering::SeqCst);

blocking(move || {
// Frame first — plan under the lock, sample without it, apply under it
// again — then snapshot and render with the lock released, like a
// single export.
if smart_crop {
let plan = lock_user(&shared).framing_inputs(&deliveries).map_err(|e| e.to_string())?;
if !plan.jobs.is_empty() {
let framings = Project::sample_framings(&plan).map_err(|e| e.to_string())?;
let framed = lock_user(&shared).apply_framings(&framings).map_err(|e| e.to_string())?;
if framed > 0 {
let _ = app.emit("project-changed", ());
}
}
}
let (timeline, assets) = {
let project = lock_user(&shared);
(
project.timeline().map_err(|e| e.to_string())?,
project.list_assets().map_err(|e| e.to_string())?,
)
};
let mut on_progress = |p: kerf_core::VariantProgress| {
let _ = app.emit("export-progress", p);
};
let (status, _) = kerf_core::render_variants(&timeline, &assets, &variants, &options, &mut on_progress, &|| {
cancel.load(Ordering::SeqCst)
})
.map_err(|e| e.to_string())?;
match status {
kerf_core::RenderStatus::Completed => Ok(variants.iter().map(|v| v.output.to_string_lossy().into_owned()).collect()),
// The variant in flight is already gone; the finished ones stay.
kerf_core::RenderStatus::Cancelled => Err("export cancelled".to_string()),
}
})
.await
}

/// Write the composited frame at `time_secs` to `output_path` as a **cover
/// image** — full delivery resolution, decoded from the original media rather
/// than a preview proxy. `format` follows the file extension when omitted.
Expand Down Expand Up @@ -1759,6 +1829,7 @@ pub fn run() {
remove_task,
hw_encoders,
export_timeline,
export_variants,
cancel_export,
cancel_analysis,
export_cover,
Expand Down
182 changes: 181 additions & 1 deletion crates/kerf-app/src/mcp.rs
Original file line number Diff line number Diff line change
Expand Up @@ -663,6 +663,32 @@ struct ExportParams {
options: Option<ExportOptions>,
}

#[derive(Debug, serde::Deserialize, schemars::JsonSchema)]
struct ExportVariantsParams {
#[schemars(
description = "Base output path; each delivery lands beside it with its shape in the name — \
`/renders/cut.mp4` at 9:16 and 1:1 writes `cut-9x16.mp4` and `cut-1x1.mp4`."
)]
output_path: String,
#[schemars(
description = "The delivery frames to render, one file each: \"9:16\" (1080x1920, Reels / Shorts / \
TikTok), \"1:1\" (1080x1080, feed), \"4:5\" (1080x1350, Instagram portrait), \"16:9\" \
(1920x1080, YouTube), or an explicit \"WxH\"."
)]
formats: Vec<String>,
#[schemars(
description = "Frame each shot for every shape first (default true): samples where each clip's \
content sits and keeps a crop per shape on the clip, so a 9:16 and a 1:1 delivery \
each keep the subject rather than the middle. The project frame's own crop is never \
touched. false renders whatever crop each clip already has."
)]
smart_crop: Option<bool>,
#[schemars(description = "Encode settings shared by every variant — the same fields as `export`. Its \
resolution and fit are replaced per variant by the delivery frame.")]
#[serde(default)]
options: Option<ExportOptions>,
}

#[derive(Debug, serde::Deserialize, schemars::JsonSchema)]
struct AddTaskParams {
#[schemars(description = "What the task should accomplish, in plain language")]
Expand Down Expand Up @@ -1819,6 +1845,156 @@ impl KerfMcp {
}
}

#[tool(
description = "Export the same cut at several delivery frames in one call — one file per shape, each shot \
framed for each: a 9:16 Reel, a 1:1 post and a 16:9 upload from one timeline. Shots are \
smart-cropped per shape first (unless smart_crop is false), which is recorded on the clips as \
one revision and reused by later exports; generated captions are re-fit to each frame. The \
project's own frame is untouched. Files land beside output_path named by shape \
(`cut-9x16.mp4`). Reports each file with the platforms it is ready for and any issue, so \
there is no need to run platform_check per variant afterwards. Progress and cancellation \
work as in `export`; cancelling keeps the files already finished and deletes the one in \
flight."
)]
async fn export_variants(
&self,
Parameters(p): Parameters<ExportVariantsParams>,
context: RequestContext<RoleServer>,
) -> Result<String, McpError> {
if p.formats.is_empty() {
return Err(McpError::invalid_params(
"formats must name at least one delivery frame",
None,
));
}
let mut deliveries: Vec<Delivery> = Vec::new();
for name in &p.formats {
let d = Delivery::parse(name).ok_or_else(|| {
McpError::invalid_params(
format!("unknown delivery format {name:?}: expected 9:16, 1:1, 4:5, 16:9 or WxH"),
None,
)
})?;
if !deliveries.contains(&d) {
deliveries.push(d);
}
}
let opts = p.options.unwrap_or_default();
let base = std::path::PathBuf::from(&p.output_path);
let variants: Vec<kerf_core::ExportVariant> = deliveries
.iter()
.map(|d| kerf_core::ExportVariant::beside(&base, *d))
.collect();
let project = self.project.clone();

// Same protocol plumbing as `export`: progress on the client's token,
// cancellation off the request's token.
let cancel = context.ct.clone();
let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel::<kerf_core::VariantProgress>();
let forward = {
let peer = context.peer.clone();
let token = context.meta.get_progress_token();
let labels: Vec<String> = deliveries.iter().map(Delivery::ratio_label).collect();
tauri::async_runtime::spawn(async move {
while let Some(progress) = rx.recv().await {
let Some(token) = token.clone() else { continue };
let param = ProgressNotificationParam::new(token, progress.fraction).with_total(1.0);
let mut message = format!(
"rendering {} of {} ({})",
progress.variant + 1,
progress.total,
labels.get(progress.variant).cloned().unwrap_or_default()
);
if let Some(eta) = progress.eta_secs {
message.push_str(&format!(", {} to go", fmt_ts(eta)));
}
let _ = peer.notify_progress(param.with_message(message)).await;
}
})
};

let smart_crop = p.smart_crop.unwrap_or(true);
let deliveries_for_plan = deliveries.clone();
let render_variants = variants.clone();
let result = blocking(move || {
// Frame first: plan under the lock, sample with it released (one
// short decode per clip, shared by every shape), apply under it again.
let mut framed = 0;
if smart_crop {
let plan = lock_agent(&project).framing_inputs(&deliveries_for_plan).map_err(core_err)?;
if !plan.jobs.is_empty() {
let framings = Project::sample_framings(&plan).map_err(core_err)?;
framed = lock_agent(&project).apply_framings(&framings).map_err(core_err)?;
}
}
let (timeline, assets) = {
let project = lock_agent(&project);
(
project.working_timeline().map_err(core_err)?,
project.list_assets().map_err(core_err)?,
)
};
let mut on_progress = |progress: kerf_core::VariantProgress| {
let _ = tx.send(progress);
};
let (status, done) =
kerf_core::render_variants(&timeline, &assets, &render_variants, &opts, &mut on_progress, &|| {
cancel.is_cancelled()
})
.map_err(core_err)?;
// Judged per file, at the frame that file actually is.
let mut outputs = Vec::new();
let project = lock_agent(&project);
for v in render_variants.iter().take(done) {
let summary = project
.cut_summary(Some((v.delivery.width, v.delivery.height)))
.map_err(core_err)?;
let checks = kerf_core::platform::check_all(&summary);
let is_tip = |i: &kerf_core::platform::DeliveryIssue| i.severity == kerf_core::platform::Severity::Tip;
let ready_for: Vec<&str> = checks
.iter()
.filter(|c| c.issues.iter().all(is_tip))
.map(|c| c.label.as_str())
.collect();
let issues: Vec<serde_json::Value> = checks
.iter()
.flat_map(|c| {
c.issues
.iter()
.filter(|i| !is_tip(i))
.map(move |i| serde_json::json!({ "target": c.label, "severity": i.severity, "message": i.message }))
})
.collect();
outputs.push(serde_json::json!({
"format": v.delivery.ratio_label(),
"width": v.delivery.width,
"height": v.delivery.height,
"output": v.output.to_string_lossy(),
"ready_for": ready_for,
"issues": issues,
}));
}
Ok((status, framed, outputs))
})
.await;
let _ = forward.await;
let (status, framed, outputs) = result?;
if framed > 0 {
self.changed();
}
match status {
kerf_core::RenderStatus::Completed => json(&serde_json::json!({ "clips_framed": framed, "outputs": outputs })),
kerf_core::RenderStatus::Cancelled => Err(McpError::internal_error(
format!(
"export cancelled after {} of {} files; the one in flight was removed",
outputs.len(),
variants.len()
),
None,
)),
}
}

#[tool(
description = "Check the assembled cut against each publishing target (Instagram Reels / YouTube Shorts / \
TikTok / Instagram feed / YouTube): length limits, frame shape, and the reach limits a platform \
Expand Down Expand Up @@ -2291,7 +2467,11 @@ impl ServerHandler for KerfMcp {
for it — reshaping 16:9 footage to 9:16 throws away most of the \
width, and without this the middle is what survives, subject or \
not. Look at the result with preview_timeline; the crop is an \
ordinary transform the user can adjust. \
ordinary transform the user can adjust. When the same cut is \
going to several places, export_variants renders it once per \
shape (9:16, 1:1, 4:5, 16:9) in one call, framing each shot for \
each shape and reporting what each file is ready for — prefer it \
to exporting three times by hand. \
Your task edits are STAGED, not applied: claiming a task opens a \
proposal, and every edit you make goes into it instead of changing \
the cut the user is looking at. Your own reads follow the proposal, \
Expand Down
Loading
Loading