Skip to content

QVAC-25109 feat[asr-ggml]: propagate Unified RNN-T cache-aware streaming - #4567

Open
freddy311082 wants to merge 3 commits into
mainfrom
QVAC-25109/asr-unified-cache-aware-streaming
Open

freddy311082 wants to merge 3 commits into
mainfrom
QVAC-25109/asr-unified-cache-aware-streaming

Conversation

@freddy311082

Copy link
Copy Markdown
Contributor

What

Propagates the Unified RNN-T cache-aware streaming that speech-cpp gained in
qvac-fabric-speech.cpp#258
up to @qvac/asr-ggml.

parakeet-unified-en-0.6b no longer re-encodes a sliding window while
streaming: the engine keeps per-layer attention and convolution caches across
steps, so the chunk cadence now selects a trained operating point instead of an
arbitrary window length.

Asana: QVAC-25109.

How

  • parakeet-unified-en-0.6b gets its own streaming chunk default of 560 ms, one
    of the trained chunks, instead of the generic 2000 ms that the engine would
    snap down to 1040 ms anyway. The documented default and the effective one now
    agree.
  • Out-of-set values keep being forwarded untouched, so validation and snapping
    stay in speech-cpp rather than being duplicated in the addon.
  • Documents the accepted values on the driver and the parakeet options: chunks
    of 80, 160, 560 or 1040 ms, right lookahead of 0, 80, 160, 240, 320, 560 or
    1040 ms, both snapped down to the nearest trained value.
  • Raises the speech-cpp floor to 2026-09-17.

Testing

New addon unit tests cover the default, the four trained chunks passing through
unchanged, that the default is itself a trained operating point, and that
out-of-set values are forwarded rather than coerced.

Full CI was run against the temporary registry and monorepo chain pinning the
ext-lib branch head, and passed end to end, including the ten prebuild
platforms and the GPU integration lanes:
https://github.com/tetherto/qvac/actions/runs/35274766020

Note for the reviewer

The speech-cpp floor in this PR only resolves once #258 merges and the
registry publishes 2026-09-17. Until then this branch's own CI will fail at
dependency resolution. The temporary vcpkg-configuration.json pointer used
for the run above is deliberately not part of this PR.

@freddy311082
freddy311082 requested review from a team as code owners September 17, 2026 21:31
@github-actions

Copy link
Copy Markdown
Contributor

Review Status

Current Status: ❌ PENDING
Approvals so far: none

Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member.

@github-actions

Copy link
Copy Markdown
Contributor

License compliance — clean

No new dependency license findings in this PR.

Warn-only (shadow) mode — this check does not block merges yet.

Updated automatically by the canonical license compliance workflow.

NOTICE presence (advisory)

Missing NOTICE (advisory, does not block):

  • ./docs/website
  • ./packages/fabric/test/integration
  • ./packages/llm-llamacpp/benchmarks/server
  • ./packages/llm-llamacpp/benchmarks/performance
  • ./packages/inference-addon-cpp/mobile
  • ./packages/asr-ggml/benchmarks/server
  • ./packages/embed-llamacpp/benchmarks/server
  • ./packages/embed-llamacpp/benchmarks/performance
  • ./packages/sdk/e2e
  • ./packages/vla-ggml/sim/server
  • ./.github/actions/release-merge-guard

@freddy311082
freddy311082 force-pushed the QVAC-25109/asr-unified-cache-aware-streaming branch from a718913 to 8e96fd0 Compare September 18, 2026 18:21
speech-cpp now drives parakeet-unified-en-0.6b through its native
cache-aware encoder instead of re-encoding a sliding window, so the chunk
cadence selects a trained operating point rather than an arbitrary window.

Give the model its own default of 560 ms, one of the trained chunks,
instead of the generic 2000 ms that the engine would snap down anyway, and
document the accepted chunk and right-lookahead values on the driver and
the parakeet options. The addon keeps forwarding out-of-set values
untouched so validation stays in speech-cpp.
…vision (2026-09-18#1)

The 2026-09-18#1 registry revision pins qvac-fabric-speech.cpp master at
6380471c, the first published version carrying the Unified RNN-T
cache-aware streaming that this package's defaults rely on.
@freddy311082
freddy311082 force-pushed the QVAC-25109/asr-unified-cache-aware-streaming branch from 8e96fd0 to db9c63e Compare September 18, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant