recipe(rubentito/layoutlmv3-base-mpdocvqa): add verified CPU fp32/fp16 recipes

[codykk]

Summary

Add verified CPU fp32 and fp16 recipes for rubentito/layoutlmv3-base-mpdocvqa, a LayoutLMv3 model fine-tuned on MP-DocVQA for document question answering. Architecture support already exists on main (PR #1174); this contribution ships checkpoint-specific recipe configs with L1 (perf) verification on both precisions.

Model metadata

What the model does: LayoutLMv3 is a multimodal document understanding Transformer that jointly encodes text tokens (with bounding-box position embeddings) and document image patches to perform extractive question answering over visually-rich documents.

Primary user stories:

• Extract answers from scanned/digital documents given a natural-language question
• Automate document VQA pipelines for forms, invoices, and multi-page documents

Supported tasks: question-answering (extractive span prediction — start_logits / end_logits)

Model architecture:

LayoutLMv3ForQuestionAnswering
├── Embeddings
│   ├── Word embeddings (vocab 50265 × 768)
│   ├── Position embeddings (512 × 768)
│   ├── Spatial (bbox) embeddings (2D: x/y/w/h)
│   └── Patch embeddings (3×224×224 → 196 patches × 768)
├── Encoder stack × 12
│   ├── Self-attention (12 heads, 768 dim)
│   ├── Feed-forward (768 → 3072 → 768, GELU)
│   └── Residual + LayerNorm
└── QA span head (768 → 2: start/end logits)

• Source/confidence: pinned checkpoint config.json and LayoutLMv3ForQuestionAnswering class (verified).

Validation and support evidence

Baseline

On main (commit 67169d45, WinML CLI latest), rubentito/layoutlmv3-base-mpdocvqa builds successfully out-of-the-box for both fp32 (512.7 MB) and fp16 (260.3 MB) via auto-config. PR #1174 (merged 2026-07-23) registered LayoutLMv3IOConfig with full multi-modal input support. Optimum coverage: layoutlmv3 is a supported architecture in optimum exporters.

Goal

Axis	Level	Definition
Effort	L0 (recipe only)	No code changes; architecture infra already on main
Goal	L1 (perf)	Build + benchmark latency/throughput/memory
Outcome	L0 (recipe + report)	Verified recipes shipped

Outcome

Goal L1 reached on all required tuples. Coverage: full (all planned tuples PASS). No deferred tuples.

Per-EP/device/precision results

Tier	EP / Device	Precision	Verdict	Avg	P50	Throughput	RAM Δ
L0	CPUExecutionProvider / cpu	fp32	PASS	—	—	—	—
L1	CPUExecutionProvider / cpu	fp32	PASS	353.03 ms	352.95 ms	2.83 samples/s	+550.4 MB
L0	CPUExecutionProvider / cpu	fp16	PASS	—	—	—	—
L1	CPUExecutionProvider / cpu	fp16	PASS	354.06 ms	354.27 ms	2.82 samples/s	+550.0 MB

Eval: CLI-BLOCKED — no winml eval command available for task-metric verification.

Delta

Shipped recipes relative to auto-config baseline:

• examples/recipes/rubentito_layoutlmv3-base-mpdocvqa/cpu/cpu/question-answering_fp32_config.json — identical to auto-generated config (explicit checkpoint pin)
• examples/recipes/rubentito_layoutlmv3-base-mpdocvqa/cpu/cpu/question-answering_fp16_config.json — adds quant.mode: "fp16" with uint8 weight/activation quantization

examples/recipes/README.md untouched.

Analyze summary — component level and op level

CLI-BLOCKED — winml analyze requires runtime rule parquet files not present in this installation. This is an infrastructure gap, not a model issue. Static rule analysis was not executed.

Reproduce commands

# Build fp32
winml build -m rubentito/layoutlmv3-base-mpdocvqa -c examples/recipes/rubentito_layoutlmv3-base-mpdocvqa/cpu/cpu/question-answering_fp32_config.json --use-cache

# Build fp16
winml build -m rubentito/layoutlmv3-base-mpdocvqa -c examples/recipes/rubentito_layoutlmv3-base-mpdocvqa/cpu/cpu/question-answering_fp16_config.json --use-cache

# Perf fp32
winml perf -m rubentito/layoutlmv3-base-mpdocvqa -c examples/recipes/rubentito_layoutlmv3-base-mpdocvqa/cpu/cpu/question-answering_fp32_config.json

# Perf fp16
winml perf -m rubentito/layoutlmv3-base-mpdocvqa -c examples/recipes/rubentito_layoutlmv3-base-mpdocvqa/cpu/cpu/question-answering_fp16_config.json