feat: stems option on text-to-music and video-to-music - #58
Merged
Conversation
stems: true (REST: stems, default false) also splits the generated track
into four separated stems — drums, bass, vocals, other. Free of charge.
On videoToMusic it splits the GENERATED music, never the source video's
own audio. It requires mode: "async" (the backend 400s it on the plain
stream), so only submit() sends it; stream()/generate() never do, and
videoToMusic.submit() auto-selects async, matching preserveSpeech and
ducking.
The task result gains two independent fields on MusicTaskResult: stems
(one StemsEntry — { stream_index, drums, bass, vocals, other }, each stem
an SfxMedia — per stream that separated successfully; looked up by
stream_index, never position, since the array can be shorter than audio)
and stems_error (why separation failed wholly or in part, or was skipped;
it can appear ALONGSIDE a partial stems array, so its presence never
means "no stems").
Separation runs after generation and typically adds 2-6 minutes to the
wait, giving up after 30 — so the CLI's --stems waits up to 40 minutes
(the generic 10 plus that ceiling), the way dubbing raises its own wait.
Both CLI commands write each arrived stem next to the main output with
the stem name inserted before the extension (per variant above
--variants 1), warn on stderr for a partial result, and exit non-zero
only when no stems came back at all.
Lightsage docs evalsWaiting for the staging docs URL before running evals. Lightsage will start the selected PR evals automatically when GitHub reports a successful docs deployment for this PR. This usually happens within 15 minutes. Commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the API's new
stemsoption to the core SDK and the CLI.stems: trueontextToMusic/videoToMusicasync submits returns the generated track split into four separated instrument tracks —drums,bass,vocals,other— as astemsarray alongside the cleanaudio. Free of charge; async-only (the SDK auto-selects async and rejects an explicit stream + stems client-side, same asducking). On video-to-music it splits the generated music, never the video's own audio.Result contract mirrored in the types:
stemsentries are looked up bystream_index, never position (a stream whose separation failed is absent), andstems_errorcan appear alongside a partial array — the CLI warns on a partial result and exits 0, and hard-fails only when--stemsproduced nothing, after the paid track is on disk. Separation adds 2-6 min typically with a 30-min ceiling, so the CLI uses a dedicated 40-min wait timeout for stems runs, following the dubbing precedent.Changeset bumps
sonilo0.15.0→0.16.0 andsonilo-cli0.14.0→0.15.0;sonilo-video-kitis not bumped (verified withchangeset status).Tests: 509 across the three workspaces, all green; lint + build clean.