Skip to content

PRODUCT.md

Latest commit

 

History

History
188 lines (147 loc) · 7.63 KB

PRODUCT.md

File metadata and controls

188 lines (147 loc) · 7.63 KB
Raw

vlogcut — Product Documentation

Status: Draft v0.1 · July 2026 (kept as design history; the shipped behavior is documented in USAGE.md) Platform: macOS (local-only, no cloud services)

1. Problem

Editing a talking-head vlog takes far longer than shooting it. A 10-minute video built from a few takes costs an hour or more in a timeline editor, and almost all of that time goes into four mechanical chores:

  1. Picking the best take out of several attempts
  2. Cutting out filler words ("ehm", "uh") and awkward pauses
  3. Adding styled captions in sync with speech
  4. Dropping in the occasional picture overlay

None of these require creative timeline work. They require knowing what was said and when — which speech recognition already provides.

2. Product idea

vlogcut is a local CLI that turns vlog editing into a text-editing task.

The core insight (the Descript model): once you have per-word timestamps for everything spoken, the video edit is a transcript edit. The user never scrubs a timeline. They delete lines of text they don't want, and the tool derives all cuts, caption timing, and overlay placement from the word-level transcript.

vlogcut ingest takes/*.mov        # transcribe → edit.md + words.json
$EDITOR edit.md                   # keep/delete takes as lines of text
vlogcut render -o final.mp4       # auto-cut + captions + overlays

Everything runs on-device: whisper.cpp (Metal-accelerated) for transcription, ffmpeg for cutting and rendering. No uploads, no subscriptions.

3. Target user

Solo vlog creators comfortable with a terminal and a text editor. Initially a single user (the author); design decisions favor scriptability and speed over GUI discoverability.

4. User journey

4.1 Record

The user films one or more takes with any camera/phone and drops the files into a folder. Multiple retakes of the same passage are expected and normal.

4.2 Ingest — vlogcut ingest takes/*.mov

For each input file the tool:

  1. Extracts mono 16 kHz audio (afconvert, built into macOS)
  2. Transcribes with whisper-cli using a shared ggml-large-v3 model — found in the machine's existing model caches when possible (never download a duplicate model; vlogcut setup downloads one only when none exists)
  3. Writes two artifacts into the project:
    • words.json — every word with start/end timestamps and source file
    • edit.md — the human-facing edit file: one sentence per line, prefixed with a timestamp and take/file reference

4.3 Select — edit edit.md

The user opens edit.md in any editor and deletes the lines (or whole takes) they don't want. What remains is the edit decision list.

Conventions:

  • A deleted line = cut that span entirely
  • Lines are re-orderable — the final video follows edit.md order, enabling restructuring across takes
  • Optional spoken marker support: saying a keyword on camera (e.g. "keep that") auto-preselects the preceding take at ingest time

This selection step is the only manual step in the pipeline, deliberately: a human has to choose which take was best. The goal is that it takes ~2 minutes, not 20.

4.4 Render — vlogcut render -o final.mp4

Within the kept ranges, the tool automatically:

  1. Removes filler words — a configurable blocklist ("ehm", "uh", "äh", standalone "like"…) matched against words.json, each removal using exact word boundaries
  2. Tightens pauses — inter-word gaps longer than a threshold (default 0.4 s) are shortened to a natural beat (default 0.15 s)
  3. Pads cuts by a few frames on each side so edits don't sound clipped
  4. Merges all cuts into a single cut list and renders one ffmpeg pass
  5. Burns in captions and composites overlays (below)

4.5 Captions

Generated from the same word timestamps, so they stay perfectly synced after cutting. Two tiers:

Tier Tech Look
v1 (default) Styled .ass subtitles burned in by ffmpeg Custom font/colors, per-word pop-in, karaoke-style highlight
v2 (later) Remotion (React/Node) render pass CapCut/TikTok-level animated captions: springs, emphasis colors, emoji

Caption style is defined in a preset file (captions.yaml): font, size, colors, position, highlight behavior, max words per line.

4.6 Picture overlays

Declared in a small manifest, overlays.yaml:

- at: "01:23"          # timestamp in final-cut time, or an anchor word/phrase
  img: assets/map.png
  duration: 4
  style: polaroid       # preset: plain | polaroid | shadow-card
  position: top-right

Anchoring to a word or phrase ("when I say 'Lisbon'") is preferred over raw timestamps because it survives re-cutting. v1 renders overlays with ffmpeg's overlay filter (static, with fade in/out); v2 moves them into Remotion for animated entrances.

5. Scope

v1 (MVP)

  • ingest: afconvert + whisper-cli → words.json + edit.md
  • render: transcript-driven cutting, filler-word removal, pause tightening, ASS caption burn-in, static image overlays
  • Config: vlogcut.yaml (thresholds, filler list), captions.yaml (style), overlays.yaml
  • Single ffmpeg render pass, H.264/AAC MP4 output

v2

  • Remotion caption/overlay renderer (animated, word-by-word)
  • Spoken take-markers ("keep that") for automatic preselection
  • Multi-resolution export (16:9 master + 9:16 vertical crop with re-positioned captions)

Non-goals

  • No GUI / timeline editing — text is the interface
  • No cloud transcription or rendering; everything stays local
  • No music/b-roll editing beyond static picture overlays (for now)
  • Not a general-purpose video editor — talking-head vlogs only

6. Technical foundation

Concern Tool Notes
Transcription whisper-cli (brew whisper-cpp) Metal-accelerated; shared ggml-large-v3 model resolved from config/env/model caches (see README); JSON output with word timestamps
Audio extraction afconvert Built into macOS, no dependency
Cutting / rendering / captions ffmpeg Not yet installed — brew install ffmpeg; select/aselect filters or segment-concat; subtitles= filter for ASS burn-in
CLI Python 3 Already installed; stdlib + PyYAML; ~200–300 LOC for v1
v2 captions/overlays Remotion (Node) Node already installed

Key data artifact: words.json

The single source of truth connecting transcription, editing, cutting, captions, and overlays:

{
  "source": "takes/take2.mov",
  "words": [
    {"w": "welcome", "start": 3.42, "end": 3.81},
    {"w": "ehm",     "start": 3.95, "end": 4.20, "filler": true}
  ]
}

All downstream timing (cut list, caption karaoke, overlay anchors) is computed from this file, so cuts can be re-derived at any time by re-running render — edits are non-destructive and reproducible.

7. Success criteria

  • A 10-minute vlog from 3–4 takes goes from raw files to finished MP4 in under 10 minutes of user time, of which the only manual step is editing edit.md
  • Filler-word removal catches ≥ 90 % of "ehm"/"uh" instances without clipping adjacent words
  • Captions never drift from speech (guaranteed by construction — same timestamps drive both)
  • Re-running render after tweaking edit.md or a style preset is a one-command, no-thought operation

8. Open questions

  • Pause-tightening threshold: fixed default vs. adaptive to speaking rhythm?
  • Should edit.md support inline overlay markers (e.g. ![map](assets/map.png) on its own line) instead of / in addition to overlays.yaml?
  • Vertical (9:16) export in v1 or defer entirely to v2?
  • Crossfade vs. hard cut at tightened pauses — does a 2-frame audio crossfade sound better at scale?