You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow-up to #77, which shipped in #79. These three integration items were deliberately left out
of that PR so it could ship the correctness and documentation work; none of them block use of tools/echomimic3.py.
1. Per-scene narrator generation
voiceover.py --scene-dir public/audio/scenes --json is the established pattern and echomimic3.py should match it. This matters more here than for TTS: at 22.8-47.8x realtime a
3-minute narrator is 1.5-2.4 hours as a single render, so per-scene generation is the difference
between a workable loop and an overnight job. It also makes a bad take cheap to redo.
The field holds RunPod job IDs, which exist because SadTalker on RunPod is asynchronous and a
timed-out job can be recovered with --retrieve. Modal is synchronous — there are no job IDs to
store. Needs a generator field naming which tool produced the narrator instead, so a project
records what it used rather than implying SadTalker.
Only projects/pluribus-sprint/project.json currently carries the old shape.
3. Presets
SadTalker has five (default, natural, expressive, professional, fullbody). echomimic3 has none. Worth having, but the parameter that would most obviously drive them is
unsettled — see below — so this is best done after that lands rather than baking in a guess.
Also still open from #77, and documented as open in docs/echomimic3.md:
--audio-guidance-scale is unsettled. Upstream suggests 1.8-2.0 for lip sync; the default
here is 3.0 to match run_flash.sh. An earlier recommendation to lower it was retracted
because it rested on the mouth-crop metric that EchoMimicV3 as SadTalker replacement — working spike, needs integration + a seam bug fixed #77 showed is not a quality score. Needs a
human-reviewed A/B.
Full-length narration has never been run. Everything measured so far is 10-12s (4
segments). A 3-minute narrator is ~62 segments; identity drift across that many re-anchorings
is unknown.
Follow-up to #77, which shipped in #79. These three integration items were deliberately left out
of that PR so it could ship the correctness and documentation work; none of them block use of
tools/echomimic3.py.1. Per-scene narrator generation
voiceover.py --scene-dir public/audio/scenes --jsonis the established pattern andechomimic3.pyshould match it. This matters more here than for TTS: at 22.8-47.8x realtime a3-minute narrator is 1.5-2.4 hours as a single render, so per-scene generation is the difference
between a workable loop and an overnight job. It also makes a bad take cheap to redo.
2.
project.jsonnarrator block hardcodessadtalkerJobsThe field holds RunPod job IDs, which exist because SadTalker on RunPod is asynchronous and a
timed-out job can be recovered with
--retrieve. Modal is synchronous — there are no job IDs tostore. Needs a
generatorfield naming which tool produced the narrator instead, so a projectrecords what it used rather than implying SadTalker.
Only
projects/pluribus-sprint/project.jsoncurrently carries the old shape.3. Presets
SadTalker has five (
default,natural,expressive,professional,fullbody).echomimic3has none. Worth having, but the parameter that would most obviously drive them isunsettled — see below — so this is best done after that lands rather than baking in a guess.
Also still open from #77, and documented as open in
docs/echomimic3.md:--audio-guidance-scaleis unsettled. Upstream suggests 1.8-2.0 for lip sync; the defaulthere is 3.0 to match
run_flash.sh. An earlier recommendation to lower it was retractedbecause it rested on the mouth-crop metric that EchoMimicV3 as SadTalker replacement — working spike, needs integration + a seam bug fixed #77 showed is not a quality score. Needs a
human-reviewed A/B.
segments). A 3-minute narrator is ~62 segments; identity drift across that many re-anchorings
is unknown.