Skip to content

EchoMimicV3: per-scene generation, project.json generator field, presets #80

Description

@ConalMullan

Follow-up to #77, which shipped in #79. These three integration items were deliberately left out
of that PR so it could ship the correctness and documentation work; none of them block use of
tools/echomimic3.py.

1. Per-scene narrator generation

voiceover.py --scene-dir public/audio/scenes --json is the established pattern and
echomimic3.py should match it. This matters more here than for TTS: at 22.8-47.8x realtime a
3-minute narrator is 1.5-2.4 hours as a single render, so per-scene generation is the difference
between a workable loop and an overnight job. It also makes a bad take cheap to redo.

2. project.json narrator block hardcodes sadtalkerJobs

The field holds RunPod job IDs, which exist because SadTalker on RunPod is asynchronous and a
timed-out job can be recovered with --retrieve. Modal is synchronous — there are no job IDs to
store. Needs a generator field naming which tool produced the narrator instead, so a project
records what it used rather than implying SadTalker.

Only projects/pluribus-sprint/project.json currently carries the old shape.

3. Presets

SadTalker has five (default, natural, expressive, professional, fullbody).
echomimic3 has none. Worth having, but the parameter that would most obviously drive them is
unsettled — see below — so this is best done after that lands rather than baking in a guess.


Also still open from #77, and documented as open in docs/echomimic3.md:

  • --audio-guidance-scale is unsettled. Upstream suggests 1.8-2.0 for lip sync; the default
    here is 3.0 to match run_flash.sh. An earlier recommendation to lower it was retracted
    because it rested on the mouth-crop metric that EchoMimicV3 as SadTalker replacement — working spike, needs integration + a seam bug fixed #77 showed is not a quality score. Needs a
    human-reviewed A/B.
  • Full-length narration has never been run. Everything measured so far is 10-12s (4
    segments). A 3-minute narrator is ~62 segments; identity drift across that many re-anchorings
    is unknown.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions