Skip to content

recipe(multilingual-e5-large): add CPU fp32 and fp16 recipes#1210

Open
codykk wants to merge 1 commit into
mainfrom
yongyue/add-intfloat-multilingual-e5-large-recipe
Open

recipe(multilingual-e5-large): add CPU fp32 and fp16 recipes#1210
codykk wants to merge 1 commit into
mainfrom
yongyue/add-intfloat-multilingual-e5-large-recipe

Conversation

@codykk

@codykk codykk commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Add verified CPU execution provider coverage recipes for intfloat/multilingual-e5-large (feature-extraction, XLM-RoBERTa) with fp32 and fp16 precision. No source changes — current main already builds and runs this model recipe-free; these recipes record exact verified (cpu, cpu, fp32) and (cpu, cpu, fp16) coverage.

Model metadata

  • What the model does: Multilingual text embedding model producing dense 1024-dim vectors for semantic similarity, retrieval, and clustering across 100+ languages (confidence: verified — HuggingFace model card + architecture inspection)
  • Primary user stories: User supplies multilingual text queries/passages to obtain normalized embeddings for cross-lingual semantic search and sentence similarity
  • Supported tasks: feature-extraction (sentence-similarity via cosine; Optimum vendor-registered for xlm-roberta)
  • Model architecture:
    XLMRobertaModel (560M params)
    ├── XLMRobertaEmbeddings (word + position + type + LayerNorm)
    └── XLMRobertaEncoder
    └── XLMRobertaLayer ×24 (self-attention + intermediate + output)

Validation and support evidence

Baseline (origin/main 67169d45, winml 0.2.0):

  • winml build -m intfloat/multilingual-e5-large → ✅ PASS (82.7s)
  • winml perf → ✅ avg 411.65ms, 2.43 samples/s, +1309.1 MB RSS
  • winml eval --task feature-extraction → ✅ cosine_spearman 84.0374 (mteb/stsbenchmark-sts, 100 samples)
  • Goal floor: L3 (baseline already reaches task metric)

Committed tiers: Effort L0 / Goal L3 / Outcome L0

Per-tuple results:

EP Device Precision L0 (build) L1 (perf) L3 (eval)
cpu cpu fp32 ✅ 94.9s ✅ avg 506.25ms, 2.14 samples/s, +1308.7 MB RSS ✅ cosine_spearman 84.0374 (mteb/stsbenchmark-sts, 100 samples)
cpu cpu fp16 ✅ 94.3s ✅ avg 445.69ms, 2.01 samples/s, +1308.3 MB RSS ✅ cosine_spearman 84.0374 (mteb/stsbenchmark-sts, 100 samples)

Delta (recipe vs winml config --ep cpu --device cpu):

  • fp32: identical to auto-config (filed for verified cpu/cpu coverage)
  • fp16: adds quant block (mode: fp16, fp16_keep_io_types: true)

Analyze: CLI-BLOCKED — runtime rule parquet files not found on this host.

Reproduce commands:

# fp32
winml build -m intfloat/multilingual-e5-large \
-c examples/recipes/intfloat_multilingual-e5-large/cpu/cpu/feature-extraction_fp32_config.json \
-o out_fp32 --ep cpu --device cpu --rebuild
winml perf -m out_fp32/model.onnx --ep cpu --device cpu
winml eval -m out_fp32/model.onnx --model-id intfloat/multilingual-e5-large --task feature-extraction --ep cpu --device cpu

# fp16
winml build -m intfloat/multilingual-e5-large \
-c examples/recipes/intfloat_multilingual-e5-large/cpu/cpu/feature-extraction_fp16_config.json \
-o out_fp16 --ep cpu --device cpu --rebuild
winml perf -m out_fp16/model.onnx --ep cpu --device cpu
winml eval -m out_fp16/model.onnx --model-id intfloat/multilingual-e5-large --task feature-extraction --ep cpu --device cpu

Add feature-extraction recipes for intfloat/multilingual-e5-large on
CPU with fp32 and fp16 precision coverage.
@codykk
codykk marked this pull request as ready for review July 24, 2026 09:46
@codykk
codykk requested a review from a team as a code owner July 24, 2026 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant