Skip to content

recipe(vitpose-base): add CPU keypoint detection recipes#1189

Draft
kujin66 wants to merge 1 commit into
mainfrom
kujin66/add-usyd-community-vitpose-base-recipe
Draft

recipe(vitpose-base): add CPU keypoint detection recipes#1189
kujin66 wants to merge 1 commit into
mainfrom
kujin66/add-usyd-community-vitpose-base-recipe

Conversation

@kujin66

@kujin66 kujin66 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds verified CPU recipes for usyd-community/vitpose-base keypoint detection. This is an L0 recipe-only contribution with CPU fp32/fp16 build, perf, L2 PyTorch-vs-ONNX numeric comparison, analyze evidence, and explicit L3 dataset blocker evidence on current PR head.

Model metadata

What the model does

usyd-community/vitpose-base is a ViTPose top-down human pose estimation model. It consumes an RGB image tensor and emits 17 COCO-style keypoint heatmaps.

Primary user stories

  • A user supplies a person crop/image tensor to obtain body keypoint heatmaps for pose estimation (verified from config and WinML export/build output).

Supported tasks

Task Support surface Evidence Confidence
keypoint-detection checkpoint, Transformers, WinML VitPoseForPoseEstimation; WinML build/perf/L2 commands below verified

Model architecture

VitPoseForPoseEstimation
|-- VitPoseBackbone
|   |-- patch embeddings (pixel_values -> patch tokens)
|   `-- encoder stack (ViT self-attention + MLP blocks)
`-- pose estimation head (heatmaps: [1, 17, 64, 48])

Validation and support evidence

Baseline

  • Base: origin/main at d9feefffccecfe47f08b7d2cfffb11fe57eb94df.
  • winml config -m usyd-community/vitpose-base --task keypoint-detection succeeded and auto-detected VitPoseForPoseEstimation / keypoint-detection with pixel_values [1, 3, 256, 192] and heatmaps output. Auto-config resolved device/EP to DML/GPU; this PR adds explicit CPU coverage recipes.

Goal

  • Effort: L0 recipe-only.
  • Goal ceiling reached here: L2 on CPU for fp32 and fp16.
  • Outcome: CPU recipe coverage for exact (cpu, cpu, fp32) and (cpu, cpu, fp16) tuples.

Outcome

  • Shipped recipes:
    • examples/recipes/usyd-community_vitpose-base/cpu/cpu/keypoint-detection_fp32_config.json
    • examples/recipes/usyd-community_vitpose-base/cpu/cpu/keypoint-detection_fp16_config.json
  • Coverage: partial by target breadth; CPU fp32/fp16 verified on this host. No NPU/GPU recipes are claimed.
  • L3 is blocked by missing local COCO keypoint dataset files, with exact CLI error recorded below.
  • Methodology friction: no additional skill methodology change required for this recipe-only PR.

Per-EP/device/precision results

Tier EP / Device Precision Verdict Mean p50 Throughput RAM delta Task metric
L0 CPUExecutionProvider / cpu fp32 PASS - - - - -
L0 CPUExecutionProvider / cpu fp16 PASS - - - - -
L1 CPUExecutionProvider / cpu fp32 PASS 84.93 ms 84.05 ms 11.77 samples/sec +381.8 MB total -
L1 CPUExecutionProvider / cpu fp16 PASS 104.74 ms 104.27 ms 9.55 samples/sec +392.8 MB total -
L2 CPUExecutionProvider / cpu fp32 PASS - - - - cosine 1.000000000, max_abs 0.000000276
L2 CPUExecutionProvider / cpu fp16 PASS - - - - cosine 0.999951883, max_abs 0.000411466
L3 CPUExecutionProvider / cpu fp32 CLI-BLOCKED-DATA - - - - Dataset ~/.cache/winml/datasets/coco_keypoints_val2017 not found
L3 CPUExecutionProvider / cpu fp16 CLI-BLOCKED-DATA - - - - Same dataset blocker as fp32; artifact already passed L0/L1/L2

Build evidence:

  • fp32: Build complete in 83.1s; final artifact model.onnx; input pixel_values [1, 3, 256, 192]; output heatmaps [1, 17, 64, 48].
  • fp16: Build complete in 64.6s; FP16 stage emitted 171.8 MB artifact; perf reported Model Precision: fp16.

L3 eval attempt evidence:

  • winml eval -m temp\verify_usyd_vitpose_base_fp32\model.onnx --model-id usyd-community/vitpose-base --task keypoint-detection --ep cpu --device cpu failed after schema-compatible setup because the default local dataset path had no data files: Couldn't find any data file at C:\Users\jinkun\.cache\winml\datasets\coco_keypoints_val2017.

Quality gates:

  • GitHub checks on PR head 3a2af091e8b831cc9d225c846c4681c4402cf981: lint, Analyze (Python), test (models), test (commands), test (analyze), test (optim), test (remaining), CodeQL, and license/cla all completed successfully.

Delta

  • Recipe-only change, copied from the established ViTPose recipe pattern and pinned to usyd-community/vitpose-base CPU coverage.
  • The production recipe README is untouched.

Analyze summary

Static rule analysis completed on both fp32 and fp16 artifacts; this is compatibility analysis, not runtime execution.

Component-level summary

Artifact Architecture coverage Mapping Actionable EP findings
fp32 ViTPose patch embeddings; encoder attention/MLP blocks; heatmap head 539/539 ONNX nodes tagged during export QNN partial: Add, Div, Erf, Mul; ConvTranspose unknown
fp16 ViTPose patch embeddings; encoder attention/MLP blocks; heatmap head 539/539 ONNX nodes tagged during export QNN partial: Add, Div, Erf, Mul; ConvTranspose unknown

Op-level summary

Artifact Graph Dominant ops EP roll-up
fp32 423 ops / 13 types Add 110; MatMul 96; Reshape 50; Transpose 50; Mul 48 TensorRT/OpenVINO 421 supported / 2 unknown; QNN 409 supported / 12 partial / 2 unknown
fp16 425 ops / 14 types Add 110; MatMul 96; Reshape 50; Transpose 50; Mul 48 TensorRT/OpenVINO 423 supported / 2 unknown; QNN 411 supported / 12 partial / 2 unknown

Rule-less EPs (CUDA, MIGraphX, DML): all operators unknown in static rules.

Reproduce commands

$OUT='temp/verify_usyd_vitpose_base_fp32'
uv run winml build -c examples\recipes\usyd-community_vitpose-base\cpu\cpu\keypoint-detection_fp32_config.json -m usyd-community/vitpose-base -o $OUT --ep cpu --device cpu --no-analyze --no-optimize --no-quant --no-compile --rebuild
uv run winml perf -m $OUT\model.onnx --ep cpu --device cpu

$OUT='temp/verify_usyd_vitpose_base_fp16'
uv run winml build -c examples\recipes\usyd-community_vitpose-base\cpu\cpu\keypoint-detection_fp16_config.json -m usyd-community/vitpose-base -o $OUT --ep cpu --device cpu --precision fp16 --no-analyze --no-optimize --no-compile --rebuild
uv run winml perf -m $OUT\model.onnx --ep cpu --device cpu

python temp\compare_usyd_l2.py
uv run winml eval -m temp\verify_usyd_vitpose_base_fp32\model.onnx --model-id usyd-community/vitpose-base --task keypoint-detection --ep cpu --device cpu

$env:WINMLCLI_RULES_DIR='<rules-dir>'
uv run winml analyze --model temp\verify_usyd_vitpose_base_fp32\model.onnx --ep all --output temp\verify_usyd_vitpose_base_fp32\analyze_all.json
uv run winml analyze --model temp\verify_usyd_vitpose_base_fp16\model.onnx --ep all --output temp\verify_usyd_vitpose_base_fp16\analyze_all.json
Remove-Item Env:WINMLCLI_RULES_DIR

@kujin66 kujin66 added the model-scale-by-skill Model support PR created or maintained by the adding-model-support skill label Jul 24, 2026
@kujin66

kujin66 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

APPROVE (skill reviewer verdict)

Reviewed PR #1189 against the adding-model-support reviewer checklist.

  • Shipment state: draft PR, model-scale-by-skill label present, head 3a2af091e8b831cc9d225c846c4681c4402cf981.
  • Diff scope: recipe-only Lane B, exactly examples/recipes/usyd-community_vitpose-base/cpu/cpu/keypoint-detection_fp32_config.json and _fp16_config.json; examples/recipes/README.md is untouched.
  • Required PR hierarchy: Summary, Model metadata, and Validation/support evidence are present with Baseline, Goal, Outcome, per-EP/device/precision results, Delta, Analyze summary, and Reproduce commands.
  • Verification evidence: CPU fp32/fp16 L0 build and L1 perf are recorded; L2 PyTorch-vs-ONNX comparison now recorded for both fp32 and fp16; L3 has an exact data blocker for missing local COCO keypoint dataset files.
  • Analyze evidence: fp32 and fp16 static analyze summaries are recorded with component-level and op-level rows.
  • GitHub checks on reviewed SHA: Analyze (Python), lint, test (analyze), test (commands), test (models), test (optim), test (remaining), CodeQL, and license/cla all completed successfully.
  • Review threads/comments: REST line comments empty; issue comments empty before this verdict; GraphQL reviewThreads empty.

No requested changes. Leave the PR in draft state per skill workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model-scale-by-skill Model support PR created or maintained by the adding-model-support skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant