Skip to content

Document phi4-mini latency on additional hardware in docs/benchmark.md #4

Description

@azurebeard

Context

docs/benchmark.md currently lists indicative latency for phi4-mini and a few alternative models, but the numbers assume 8 vCPU on commodity hardware. Operators sizing a deployment on smaller VMs (2 vCPU, 1 vCPU + GPU, ARM, Raspberry Pi 5, etc.) don't have datapoints to anchor against.

Scope

Run the benchmark recipe at the bottom of docs/benchmark.md § "Repeatable benchmark recipe" against a hardware shape not already documented (ideally one of):

  • 2 vCPU CPU-only VM
  • 8 vCPU + consumer-grade GPU (e.g. T4, RTX 3060)
  • ARM64 VM (AWS Graviton, Azure Ampere, or a Raspberry Pi 5)
  • Apple Silicon (M1 / M2 / M3) — note that this is for development only; production target is Linux

Record p50 and p95 classifier latency over a 10,000-event run, plus the steady-state cache hit rate.

Acceptance criteria

  • Extend the "Choosing a model" table in docs/benchmark.md with one new column (or one new row, depending on what's clearer) capturing the new hardware's numbers.
  • Add a short note on any practical gotchas (e.g. "Apple Silicon — Ollama runs fine; pyspark and python-evtx wheels not yet ARM-native") if relevant.

Why this is a good first issue

No code. Pure data collection plus a small docs PR. Useful for any operator sizing their first deployment.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationgood first issueGood for newcomers

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions