QVAC-25109 feat[asr-ggml]: propagate Unified RNN-T cache-aware streaming

packages/asr-ggml/CHANGELOG.md

@@ -14,12 +14,22 @@ restarts at `0.1.0`; the two pre-merge histories are preserved verbatim as
 
 ## [Unreleased]
 
+### Added
+
+- Cache-aware streaming for `parakeet-unified-en-0.6b`. The engine keeps
+  per-layer attention and convolution caches across steps instead of
+  re-encoding a sliding window, so `streamingChunkMs` now selects a trained
+  operating point: 80, 160, 560, or 1040 ms, with `streamingRightLookaheadMs`
+  at 0, 80, 160, 240, 320, 560, or 1040 ms. Values outside those sets snap
+  down to the nearest trained one. Omitting `streamingChunkMs` now yields
+  560 ms for this model instead of the generic 2000 ms.
+
 ### Changed
 
-- Raise the `speech-cpp` floor to `2026-09-16`, keeping the speech packages on
-  one engine stack. The pinned engine adds an optional Apple-only Core ML
-  sidecar for the Sortformer diarization encoder; the prebuilds keep it
-  disabled, so published behavior is unchanged.
+- Raise the `speech-cpp` floor to `2026-09-18#1`, keeping the speech packages on
+  one engine stack. The pinned engine adds cache-aware streaming for the
+  Unified RNN-T model, on top of the optional Apple-only Core ML sidecar for
+  the Sortformer diarization encoder that the prebuilds keep disabled.
 - Add Whisper `contextParams["main-gpu"]` / `contextParams.main_gpu` selection
   for raw ggml registry indices plus `dedicated` and `integrated` classes.
   The selector is mutually exclusive with `gpu_device`, does not enable GPU by

packages/asr-ggml/README.md

@@ -74,7 +74,7 @@ GGUF metadata** — there is no `modelType` to pass.
 |---------|-----------|---------|-------------:|-------|
 | **CTC** (`parakeet-ctc-0.6b`) | English | argmax CTC | ~700 MiB | Fast, no punctuation/capitalization |
 | **TDT** (`parakeet-tdt-0.6b-v3`) | ~25 | RNN-T greedy + duration | ~715 MiB | Recommended default; PnC + language auto-detect |
-| **Unified** (`parakeet-unified-en-0.6b`) | English | RNN-T | ~715 MiB | One checkpoint for batch and low-latency streaming; PnC |
+| **Unified** (`parakeet-unified-en-0.6b`) | English | RNN-T | ~715 MiB | One checkpoint for batch and cache-aware streaming at 80/160/560/1040 ms; PnC |
 | **EOU** (`parakeet-eou-120m-v1`) | English | RNN-T greedy + `<EOU>` | ~132 MiB | Streaming-trained; native end-of-turn token |
 | **Indic Conformer CTC** (`indic-conformer-ctc`) | Indic aggregate | argmax CTC + language mask | ~701 MiB | Multilingual Indic; set `parakeetConfig.language` (e.g. `"hi"`) |
 | **Sortformer v1** (`sortformer-4spk-v1`) | n/a | Diarization head (sliding history) | ~141 MiB | 4-speaker. Default for **offline** diarization |
@@ -95,7 +95,7 @@ language coverage, translation, and diarization.
 | If you need… | Use this model | Notes |
 | --- | --- | --- |
 | Default multilingual / English ASR (batch or duplex stream) | `parakeet-tdt-0.6b-v3` (q8_0 GGUF) | Recommended Parakeet default: ~25 languages, punctuation/capitalization, language auto-detect, low-latency streaming. |
-| English batch and low-latency streaming with one checkpoint | `parakeet-unified-en-0.6b` | Standard RNN-T with punctuation and capitalization; use when multilingual TDT or native EOU tokens are not required. |
+| English batch and low-latency streaming with one checkpoint | `parakeet-unified-en-0.6b` | Standard RNN-T with punctuation and capitalization; use when multilingual TDT or native EOU tokens are not required. Streaming uses the native cache-aware encoder: `streamingChunkMs` accepts 80, 160, 560, or 1040 and `streamingRightLookaheadMs` 0, 80, 160, 240, 320, 560, or 1040, both snapped down to the nearest trained value. Defaults to 560 ms. |
 | Native end-of-turn for conversational / duplex English | `parakeet-eou-120m-v1` | Emits `<EOU>`; smallest Parakeet (~132 MiB). Pair with TDT when you need broader language coverage *and* EOU. |
 | Fast English-only, no punctuation | `parakeet-ctc-0.6b` | Lowest decode cost in the Parakeet family; no PnC. |
 | Indic-language ASR (Hindi and other Indic ids) | `indic-conformer-ctc` | Pass `parakeetConfig.language` (e.g. `"hi"`). Same Parakeet engine; GGUF lives under `indic_conformer/` in the registry. |

packages/asr-ggml/addon/src/model-interface/parakeet/ParakeetConfig.hpp

@@ -9,6 +9,7 @@ namespace qvac::asrggml::parakeet {
 struct ParakeetConfig {
   static constexpr int DEFAULT_STREAMING_CHUNK_MS = 2000;
   static constexpr int DEFAULT_NEMOTRON_STREAMING_CHUNK_MS = 320;
+  static constexpr int DEFAULT_UNIFIED_STREAMING_CHUNK_MS = 560;
   static constexpr int DEFAULT_STREAMING_HISTORY_MS = 30000;
   static constexpr int DEFAULT_STREAMING_SPK_CACHE_LEN = 188;
   static constexpr int DEFAULT_STREAMING_FIFO_LEN = 188;

packages/asr-ggml/addon/src/model-interface/parakeet/ParakeetModel.cpp

@@ -421,6 +421,8 @@ int ParakeetModel::resolveStreamingChunkMs(
     return configuredChunkMs;
   if (modelType == ModelType::NEMOTRON)
     return ParakeetConfig::DEFAULT_NEMOTRON_STREAMING_CHUNK_MS;
+  if (modelType == ModelType::RNNT)
+    return ParakeetConfig::DEFAULT_UNIFIED_STREAMING_CHUNK_MS;
   return ParakeetConfig::DEFAULT_STREAMING_CHUNK_MS;
 }
 

packages/asr-ggml/addon/tests/parakeet_test_config_and_audio.cpp

@@ -1,3 +1,4 @@
+#include <algorithm>
 #include <cstdint>
 #include <vector>
 
@@ -56,6 +57,12 @@ TEST(ParakeetStreamingGetters, ResolveDefaultsByDetectedModelType) {
   EXPECT_EQ(
       nemotron.getStreamingChunkMs(),
       ParakeetConfig::DEFAULT_NEMOTRON_STREAMING_CHUNK_MS);
+
+  c.modelType = ModelType::RNNT;
+  ParakeetModel unified(c);
+  EXPECT_EQ(
+      unified.getStreamingChunkMs(),
+      ParakeetConfig::DEFAULT_UNIFIED_STREAMING_CHUNK_MS);
 }
 
 TEST(ParakeetStreamingGetters, HonourPositiveOverrides) {
@@ -79,6 +86,29 @@ TEST(ParakeetStreamingGetters, PreserveNemotronOperatingPointOverrides) {
       ParakeetModel::resolveStreamingChunkMs(ModelType::NEMOTRON, 2000), 2000);
 }
 
+TEST(ParakeetStreamingGetters, PreserveUnifiedOperatingPointOverrides) {
+  for (const int chunkMs : {80, 160, 560, 1040}) {
+    EXPECT_EQ(
+        ParakeetModel::resolveStreamingChunkMs(ModelType::RNNT, chunkMs),
+        chunkMs);
+  }
+
+  // speech-cpp snaps untrained values down to the nearest trained chunk;
+  // the addon must forward them untouched so that decision stays in one place.
+  EXPECT_EQ(
+      ParakeetModel::resolveStreamingChunkMs(ModelType::RNNT, 1000), 1000);
+  EXPECT_EQ(
+      ParakeetModel::resolveStreamingChunkMs(ModelType::RNNT, 2000), 2000);
+}
+
+TEST(ParakeetStreamingGetters, UnifiedDefaultIsATrainedOperatingPoint) {
+  const int resolved =
+      ParakeetModel::resolveStreamingChunkMs(ModelType::RNNT, 0);
+  EXPECT_EQ(resolved, ParakeetConfig::DEFAULT_UNIFIED_STREAMING_CHUNK_MS);
+  const std::vector<int> trained = {80, 160, 560, 1040};
+  EXPECT_NE(std::find(trained.begin(), trained.end(), resolved), trained.end());
+}
+
 TEST(ParakeetPreprocessAudio, S16LeHandlesRangeExtremes) {
   std::vector<uint8_t> raw = {0x00, 0x00, 0x00, 0x80, 0xFF, 0x7F};
   auto out = ParakeetModel::preprocessAudioData(raw, "s16le");

packages/asr-ggml/engines/parakeet/driver.d.ts

@@ -33,8 +33,10 @@ export interface ParakeetConfig {
      */
     streaming?: boolean;
     /**
-     * Streaming chunk cadence. Defaults to 320 ms for Nemotron and 2000 ms for
-     * existing models. Nemotron supports 80, 160, 320, 560, or 1120 ms.
+     * Streaming chunk cadence. Defaults to 320 ms for Nemotron, 560 ms for the
+     * Unified RNN-T model, and 2000 ms for existing models. Nemotron supports
+     * 80, 160, 320, 560, or 1120 ms; Unified RNN-T supports 80, 160, 560, or
+     * 1040 ms and snaps any other value down to the nearest trained chunk.
      */
     streamingChunkMs?: number;
     /** Sortformer rolling-history window in ms (default: 30000). */
@@ -45,7 +47,11 @@ export interface ParakeetConfig {
     streamingEnergyVad?: boolean;
     /** ASR encoder left-context window in milliseconds. */
     streamingLeftContextMs?: number;
-    /** ASR encoder right-lookahead window in milliseconds. */
+    /**
+     * ASR encoder right-lookahead window in milliseconds. Unified RNN-T
+     * cache-aware streaming supports 0, 80, 160, 240, 320, 560, or 1040 ms and
+     * snaps any other value down to the nearest trained right context.
+     */
     streamingRightLookaheadMs?: number;
     /** Enable v2.1 Sortformer AOSC speaker-cache streaming (default: true). */
     streamingSpkCacheEnable?: boolean;

packages/asr-ggml/engines/parakeet/parakeet.d.ts

@@ -15,7 +15,10 @@ export interface ParakeetConfigurationParams {
     /** Indic CTC language id or Nemotron locale alias; empty selects auto. */
     language?: string;
     streaming?: boolean;
-    /** Model-specific when omitted: Nemotron 320 ms, existing models 2000 ms. */
+    /**
+     * Model-specific when omitted: Nemotron 320 ms, Unified RNN-T 560 ms,
+     * existing models 2000 ms.
+     */
     streamingChunkMs?: number;
     streamingHistoryMs?: number;
     streamingEmitPartials?: boolean;

packages/asr-ggml/src/engines/parakeet/driver.ts

@@ -60,8 +60,10 @@ export interface ParakeetConfig {
    */
   streaming?: boolean;
   /**
-   * Streaming chunk cadence. Defaults to 320 ms for Nemotron and 2000 ms for
-   * existing models. Nemotron supports 80, 160, 320, 560, or 1120 ms.
+   * Streaming chunk cadence. Defaults to 320 ms for Nemotron, 560 ms for the
+   * Unified RNN-T model, and 2000 ms for existing models. Nemotron supports
+   * 80, 160, 320, 560, or 1120 ms; Unified RNN-T supports 80, 160, 560, or
+   * 1040 ms and snaps any other value down to the nearest trained chunk.
    */
   streamingChunkMs?: number;
   /** Sortformer rolling-history window in ms (default: 30000). */
@@ -72,7 +74,11 @@ export interface ParakeetConfig {
   streamingEnergyVad?: boolean;
   /** ASR encoder left-context window in milliseconds. */
   streamingLeftContextMs?: number;
-  /** ASR encoder right-lookahead window in milliseconds. */
+  /**
+   * ASR encoder right-lookahead window in milliseconds. Unified RNN-T
+   * cache-aware streaming supports 0, 80, 160, 240, 320, 560, or 1040 ms and
+   * snaps any other value down to the nearest trained right context.
+   */
   streamingRightLookaheadMs?: number;
   /** Enable v2.1 Sortformer AOSC speaker-cache streaming (default: true). */
   streamingSpkCacheEnable?: boolean;

packages/asr-ggml/src/engines/parakeet/parakeet.ts

@@ -37,7 +37,10 @@ export interface ParakeetConfigurationParams {
   /** Indic CTC language id or Nemotron locale alias; empty selects auto. */
   language?: string;
   streaming?: boolean;
-  /** Model-specific when omitted: Nemotron 320 ms, existing models 2000 ms. */
+  /**
+   * Model-specific when omitted: Nemotron 320 ms, Unified RNN-T 560 ms,
+   * existing models 2000 ms.
+   */
   streamingChunkMs?: number;
   streamingHistoryMs?: number;
   streamingEmitPartials?: boolean;

packages/asr-ggml/vcpkg.json

@@ -17,7 +17,7 @@
     },
     {
       "name": "speech-cpp",
-      "version>=": "2026-09-16",
+      "version>=": "2026-09-18#1",
       "default-features": false,
       "features": [
         "whisper",
@@ -29,7 +29,7 @@
     },
     {
       "name": "speech-cpp",
-      "version>=": "2026-09-16",
+      "version>=": "2026-09-18#1",
       "default-features": false,
       "features": [
         "whisper",
@@ -41,7 +41,7 @@
     },
     {
       "name": "speech-cpp",
-      "version>=": "2026-09-16",
+      "version>=": "2026-09-18#1",
       "default-features": false,
       "features": [
         "whisper",
@@ -66,7 +66,7 @@
       "dependencies": [
         {
           "name": "speech-cpp",
-          "version>=": "2026-09-16",
+          "version>=": "2026-09-18#1",
           "default-features": false,
           "features": [
             "cuda"