perf(silero-native): profile and optimize the native engine (fixes #164) - #180
Merged
Conversation
…(issue #164) - Add StageTimings (frontend_text / homosolver / accentor / build_sequence / tts_main / istft / pqmf / wav_encode / concat_timestamps) collected in Engine::synthesize and summed over chunks in SileroNative::synthesize; exposed on SynthesisResult. - bench example: per-stage mean breakdown at 24k/48k/8k per run. - release profile gets debug = "line-tables-only" (crate-local builds only) so perf/flamegraph attribute to source lines; exact profiling commands documented in architecture.md "Debugging guide". - benchmarks.md: baseline breakdown — tts_main ~80%, istft ~16%, frontend < 0.5%, 48k PQMF path confirmed free.
…sis (issue #164) ORT defaults to one intra-op thread per logical core; for these graphs (chains of many small ops) the per-op fork/join sync across 24 threads costs more than the compute. Pinning with_intra_threads(8) takes the 24k full-pipeline bench mean from ~104 ms to ~37 ms; speedup vs the Python torch apply_tts reference goes from ~1.3x to ~4.0x (145.0 ms, same methodology). Engine load also drops 437 -> ~345 ms (fewer pool threads to spawn at session creation). Thread count is constrained by parity: changing it changes float reduction order, and 4/6 threads push the stress_marker parity case over the 1e-3 budget (1.5e-3 / 2.2e-3); 8 is the only reduced count keeping all 31 cases inside (worst 9.8e-4) and ties 6 for speed. benchmarks.md: new headline numbers + per-stage A/B record; allocation churn and bulk WAV encode measured and rejected (< 2.2% each). Gates: silero-native tests incl. bundle-gated parity green; exporter self-check 16/16; src-tauri tests + just lint green.
- StageTimings: full-literal AddAssign coverage test — adding a stage field now fails compilation instead of silently dropping it from chunk accumulation and the bench breakdown. - pqmf stage timer moved after input tensor construction, matching the other ORT stages' scope. - INTRA_OP_THREADS constant (single edit point for the parity- constrained value); comment now states the count is deliberately not derived from available_parallelism() — the reduction order must stay fixed for waveform parity. - StageTimings docstring: drop the stale ~100 ms reference.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
StageTimingsinEngine/SileroNative,bench breakdown at 24k/48k/8k) — closes the "no per-stage attribution" gap.
intra_op_num_threadsto 8: 24k full-pipeline mean 104 → 36.6 ms;warm full-pipeline speedup vs Python torch
apply_ttsgoes from ~1.3x to~4.0x (acceptance: ≥ 2x).
each); the remaining ~34 ms is ORT inference inside the exported graphs —
irreducible client-side without touching the graphs/exporter.
Why 8 threads (parity constraint)
Changing the thread count changes float reduction order. 4/6 threads push the
stress_markerparity case over the 1e-3 budget (1.5e-3 / 2.2e-3); 8 is theonly reduced count keeping all 31 cases inside (worst 9.8e-4) and ties 6 for
speed within noise. The count is deliberately not derived from
available_parallelism()— the reduction order must stay fixed for parity.Docs
silero-native/docs/benchmarks.md: per-stage breakdown (acceptance Try KugelAudio as an alternative TTS #1),thread A/B record, new headline numbers, honest Python comparison re-run
(145.0 ms, same methodology).
silero-native/docs/architecture.md: exact perf/flamegraph commands in thedebugging guide.
Gates
just lint: greenFixes #164