Skip to content

recipe(hubert-speech-emotion-russian): add CPU audio-classification configs#1211

Open
codykk wants to merge 1 commit into
mainfrom
yongyue/hubert-emotion-recognition
Open

recipe(hubert-speech-emotion-russian): add CPU audio-classification configs#1211
codykk wants to merge 1 commit into
mainfrom
yongyue/hubert-emotion-recognition

Conversation

@codykk

@codykk codykk commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Add verified cpu/cpu build recipes (fp32 + fp16) for xbgoose/hubert-large-speech-emotion-recognition-russian-dusha-finetuned, a HuBERT-Large model fine-tuned on Russian Dusha dataset for speech emotion recognition (audio-classification). Effort L0★ (recipe-only, no code changes); Goal L1 (perf) reached on both precisions.

Model metadata

What the model does: HuBERT-Large encoder fine-tuned on the Russian Dusha dataset for classifying speech audio into 5 emotion categories.

Primary user stories: Supply a Russian speech audio sample to obtain emotion class predictions for sentiment analysis or human-computer interaction applications.

Supported tasks: audio-classification — WINML-ONLY (no Optimum vendor coverage; all tasks added by winml).

Model architecture

HubertForSequenceClassification
├── HubertModel
│   ├── HubertFeatureEncoder (7× CNN layers)
│   ├── HubertFeatureProjection (1024)
│   └── HubertEncoderStableLayerNorm x 24
│       ├── HubertAttention (16 heads, 1024)
│       └── HubertFeedForward (1024 → 4096 → 1024, GELU)
└── ClassificationHead (ReduceMean → Gemm → 5 classes)
  • Source/confidence: pinned checkpoint config.json (architectures: ["HubertForSequenceClassification"], hidden_size: 1024, num_hidden_layers: 24); verified.

Validation and support evidence

Baseline

  • Main commit: 67169d45c40ed8326065e87ed29d19623b79cbb4
  • WinML version: 0.2.0
  • Baseline build: PASS — builds with auto-config (no recipe required), 107.9s, 771 optimized nodes, 1.2 GB
  • Optimum probe: WINML-ONLY (vendor=[], added_by_winml=[audio-classification, automatic-speech-recognition, feature-extraction])
  • Starting recipe: auto-generated via winml config

Goal

  • Effort: L0★ (recipe-only, no per-architecture code)
  • Goal ceiling: L1 (perf) — baseline passes L0, perf is the next reachable tier
  • Outcome: L0 (recipe JSONs shipped)

Outcome

  • Highest Goal reached: L1 PASS ✅
  • Coverage: full (all planned tuples pass)
  • Deferred tuples: none
  • Shipped artifacts:
    • examples/recipes/xbgoose_hubert-large-speech-emotion-recognition-russian-dusha-finetuned/cpu/cpu/audio-classification_fp32_config.json
    • examples/recipes/xbgoose_hubert-large-speech-emotion-recognition-russian-dusha-finetuned/cpu/cpu/audio-classification_fp16_config.json

Per-EP/device/precision results

Tier EP / Device Precision Verdict Mean p50 Throughput RAM Δ
L0 CPUExecutionProvider / cpu fp32 PASS
L0 CPUExecutionProvider / cpu fp16 PASS
L1 CPUExecutionProvider / cpu fp32 PASS 134.96 ms 134.94 ms 7.41 samples/s +1247.6 MB
L1 CPUExecutionProvider / cpu fp16 PASS 165.08 ms 163.81 ms 6.06 samples/s +1272.5 MB

fp16 on CPU is slower than fp32 — expected behavior (CPUExecutionProvider upcasts fp16→fp32 at runtime). Not a defect.

Delta

  • fp32 recipe: matches auto-config output identically (quant: null). Formalizes cpu/cpu as a checked-in verified-coverage claim.
  • fp16 recipe: adds quant.mode: "fp16" with fp16_keep_io_types: true relative to the auto-config baseline.
  • examples/recipes/README.md remains untouched.

Analyze summary — component level and op level

Static rule analysis completed (exit code 1 — no rule data for CPU/CUDA/MIGraphX/DML; NvTensorRTRTX/QNN/OpenVINO complete). This is compatibility analysis, not runtime execution.

Component-level summary

Artifact Architecture coverage Mapping Actionable EP findings
fp32 HubertFeatureEncoder (CNN); 24× HubertEncoderLayerStableLayerNorm (attention + FFN); ClassificationHead (ReduceMean + Gemm) 1150 export → 771 optimized, 100% HTP tagged, 0 unmapped None — full support on all EPs with rules

Op-level summary

Artifact Graph Dominant ops EP roll-up
fp32 771 ops / 13 types Reshape 244; Gemm 147; Transpose 111; LayerNorm 57; MatMul 48 NvTensorRTRTX/QNN/OpenVINO: 771 supported, 0 partial, 0 unsupported

Rule-less EPs (CPU, CUDA, MIGraphX, DML): all 771 ops unknown (no predefined rules; ops still run via fallback).

Reproduce commands

$OUT = 'temp/hubert-emotion-repro'
# fp32 build + perf
uv run winml build -m xbgoose/hubert-large-speech-emotion-recognition-russian-dusha-finetuned -c examples/recipes/xbgoose_hubert-large-speech-emotion-recognition-russian-dusha-finetuned/cpu/cpu/audio-classification_fp32_config.json -o $OUT/fp32 --ep cpu --device cpu
uv run winml perf --model $OUT/fp32/model.onnx --device cpu --ep cpu
# fp16 build + perf
uv run winml build -m xbgoose/hubert-large-speech-emotion-recognition-russian-dusha-finetuned -c examples/recipes/xbgoose_hubert-large-speech-emotion-recognition-russian-dusha-finetuned/cpu/cpu/audio-classification_fp16_config.json -o $OUT/fp16 --ep cpu --device cpu --precision fp16
uv run winml perf --model $OUT/fp16/model.onnx --device cpu --ep cpu
# analyze
uv run winml analyze --model $OUT/fp32/model.onnx --ep all --output $OUT/fp32/analyze_all.json

…n-dusha-finetuned

Add cpu/cpu recipe JSONs for fp32 and fp16 precisions for the HuBERT
speech emotion recognition model (audio-classification task).
@codykk codykk added the model-scale-by-skill Model support PR created or maintained by the adding-model-support skill label Jul 24, 2026
@codykk codykk changed the title Add recipe for xbgoose/hubert-large-speech-emotion-recognition-russian-dusha-finetuned Add cpu/cpu recipes for xbgoose/hubert-large-speech-emotion-recognition-russian-dusha-finetuned Jul 24, 2026
@codykk codykk changed the title Add cpu/cpu recipes for xbgoose/hubert-large-speech-emotion-recognition-russian-dusha-finetuned recipe(hubert-speech-emotion-russian): add CPU audio-classification configs Jul 24, 2026
@codykk
codykk marked this pull request as ready for review July 24, 2026 10:34
@codykk
codykk requested a review from a team as a code owner July 24, 2026 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model-scale-by-skill Model support PR created or maintained by the adding-model-support skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant