perf(silero-native): faster engine load (parallel sessions, lazy PQMF) - #173
Merged
Conversation
Add tracing timings for manifest verify, each ONNX session open, frontend load and the total in Engine::load, and surface them via RUST_LOG=silero_native=info. The bench example now reports load_ms alongside the synthesis stats, so engine load (previously excluded from the benchmark) is measurable. Baseline on Ryzen 9 7900: ~650 ms native load vs ~1680 ms ttsd spawn-to-ready (issue #165).
… lazy PQMF) - Sessions::open creates the four always-needed sessions concurrently on scoped threads (507 ms -> ~340 ms). - Manifest::verify hashes files concurrently (115 ms -> ~60 ms). - pqmf_24k/pqmf_8k sessions are lazy-opened on the first synthesis at that sample rate instead of at engine load. - Measured and rejected: ORT graph optimization level and the .ort compiled-model cache (no effect; model parse/arena init dominates). Engine load ~650 ms -> ~440 ms, ~3.8x faster than ttsd spawn-to-ready. Numbers and methodology recorded in silero-native/docs/benchmarks.md. Closes #165.
- Sessions::open assembles the struct by model name (tagged results) instead of spawn position — a reordered NAMES can no longer silently swap sessions. - Engine::load fails fast when a PQMF model file is missing; lazy open must not defer an incomplete-bundle error to the first synthesis. - Thread-panic payloads are included in the Bundle error message. - Pin verify's first-mismatch-in-manifest-order contract with tests (plus a multi-entry happy path exercising the hash fan-out). - Doc comments updated for lazy PQMF.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Cuts native engine load from ~650 ms to ~440 ms (~1.5x) — ~3.8x faster
than the Python ttsd spawn-to-ready (~1680 ms) on the same machine
(Ryzen 9 7900).
Closes #165.
Changes
ONNX session open, frontend, total) via
RUST_LOG=silero_native=info;the bench example now reports
load_ms.on scoped threads (507 → ~340 ms), assembled by model name.
pqmf_24k/pqmf_8kopen on the first synthesis atthat rate; a missing PQMF file still fails at load.
spawn_initial_warmupviathe engine switcher) — verified, no change needed.
Measured and rejected: ORT graph optimization level (Level3 ≈ Level1 ≈
Disable) and the
.ortcompiled-model cache — session creation isdominated by model parse/arena init, not the optimizers.
Baseline + final numbers and methodology: new "Engine load time"
section in
silero-native/docs/benchmarks.md.Verification
just test— green (Rust incl. the bundle-gated parity suite, TS, Python)just lint— green