Add gfx11 (RDNA3/RDNA3.5) benchmark runner support

[mgehre-amd]

Motivation

Magpie picks a benchmark recipe by GPU architecture string. ImageSelector.arch_to_runner
only listed data-center parts, so get_runner_type("gfx1151") raised
No runner type found for gfx1151 and Magpie could not benchmark on any AMD client/edge
GPU at all.

There was also a latent mapping bug: gfx1100 was mapped to mi325x in two separate
places. gfx1100 is Navi31 (RX 7900) while MI325X is gfx942 silicon, so a consumer
RDNA3 card silently ran MI-tuned settings instead of erroring out.

What this does

Adds a shared vllm_gfx11.sh and maps the whole gfx11 family to a single gfx11 runner:

arch	part
gfx1100 / gfx1101 / gfx1102	Navi31 / Navi32 / Navi33 (discrete)
gfx1103	Phoenix APU
gfx1150	Strix Point APU
gfx1151	Strix Halo APU
gfx1152 / gfx1153	Krackan Point APU

The recipe is family-level rather than per-arch because the knobs that differ from the
Instinct scripts are architecture properties, not SKU properties. If one arch ever needs
different behaviour, adding vllm_gfx<n>.sh and repointing that one dict entry is cheap.

Impact

• New hardware support; no behaviour change for existing Instinct runners.
  gfx942 -> mi300x and gfx950 -> mi355x are asserted unchanged by tests.
• One existing assertion changes: tests/test_benchmark_unit_coverage.py had
  ("gfx1100", "mi325x") in its arch->runner table. That table enumerates the dict as it
  stood rather than asserting intent, so it codified the bug; it now expects "gfx11".
• benchmark_images.yaml gains a gfx1151 entry only.

Design notes

vllm_gfx11.sh vs vllm_mi300x.sh

Exactly three behavioural differences:

1. AITER defaults off (VLLM_ROCM_USE_AITER:-0; the MI scripts default it to 1).
   It ships kernels for gfx942/gfx950 only, so enabling it selects code paths with no
   kernels for this family. The base image's own build config confirms the split:

   $ docker image inspect rocm/vllm:rocm10.0.0_ubuntu24.04_py3.14_pytorch_2.12.0_vllm_0.27.0 \
       --format '{{range .Config.Env}}{{println .}}{{end}}' | grep ROCM_ARCH
   PYTORCH_ROCM_ARCH=gfx90a;gfx942;gfx950;gfx1100;gfx1101;gfx1102;gfx1103;gfx1150;gfx1151;gfx1152;gfx1153;gfx1200;gfx1201
   AITER_ROCM_ARCH=gfx942;gfx950
   

2. The MEC-firmware probe is dropped. It is an Instinct RCCL workaround. Its test is
   [[ "$version" == "" || $version -lt 177 ]], and gfx11 reports no MEC line at all, so
   on this family the empty-string branch fires for the wrong reason.
3. --gpu-memory-utilization honours $GPU_MEMORY_UTILIZATION. vllm_mi300x.sh
   defines that variable and then passes a hardcoded 0.95 anyway.

TraceLens

trace_arch_platform_from_runner returns None for gfx11 instead of deriving a
platform name. A family-level runner cannot identify a per-SKU TraceLens spec, and the
caller already treats None as "run without --gpu_arch_platform", which lets TraceLens
identify the part itself.

Eval-concurrency contract

vllm_gfx11.sh is added to LOCAL_EVAL_SCRIPTS in test_eval_concurrency_scripts.py.
That list is what pins every local script to the shared eval-concurrency contract, and a
script that is not listed can silently drift off it.

Validation

Unit tests

python -m pytest tests/ -q

400 passed. That includes 27 new gfx11 tests plus
test_local_eval_scripts_use_environment_for_concurrency[vllm_gfx11.sh], which asserts
the new script uses the same eval-concurrency contract as the MI scripts.

End to end on hardware

Environment: Strix Halo, gfx1151 / AMD Radeon 8060S, single GPU, Docker run mode,
image rocm/vllm:rocm10.0.0_ubuntu24.04_py3.14_pytorch_2.12.0_vllm_0.27.0
(vLLM 0.27.1.dev5) — the exact image this PR adds to benchmark_images.yaml.

Detection and selection resolve on their own. Nothing is pinned by hand except
gpu_arch, and gpu_selection runs at its default (auto: true):

$ python -c "from Magpie.utils.gpu import detect_gpu, get_gpu_count, find_idle_gpus; \
             print(detect_gpu(), get_gpu_count(), find_idle_gpus(count=1, min_free_memory_gb=8.0))"
(<GPUVendor.AMD: 1>, 'gfx1151') 1 [0]

Benchmark config used (benchmark_vllm_gfx11_e2e.yaml) — small model and request count,
since the point is to exercise the script path rather than produce perf numbers:

benchmark:
  framework: vllm
  model: Qwen/Qwen2.5-0.5B-Instruct
  precision: fp16
  gpu_arch: gfx1151

  envs:
    TP: 1
    CONC: 4
    ISL: 128
    OSL: 32
    RANDOM_RANGE_RATIO: 1
    NUM_PROMPTS: 8
    MAX_MODEL_LEN: 4096
    VLLM_LOGGING_LEVEL: "INFO"
    # run 2 only, to exercise the eval tail
    RUN_EVAL: "true"
    MAGPIE_EVAL_TASKS: "gsm8k"
    EVAL_LIMIT: 5

  profiler:
    torch_profiler: {enabled: false}
    system_profiler: {enabled: false}
    tracelens: {enabled: false}
  gap_analysis: {enabled: false}

  inferencex_path: "<path>/InferenceX"
  hf_cache_path: "<path>/hf_cache"
  timeout_seconds: 2400

Run 1 — throughput path (RUN_EVAL unset)

python -m Magpie benchmark \
  --benchmark-config benchmark_vllm_gfx11_e2e.yaml \
  --output-dir ./gfx11_e2e_out

Magpie routed itself correctly with no manual override:

Magpie.utils.gpu - INFO - Detected AMD GPU: gfx1151
Magpie.modes.benchmark.benchmarker - INFO - Selected benchmark script: benchmarks/vllm_gfx11.sh

Result:

Benchmark Result: VLLM
Model: Qwen/Qwen2.5-0.5B-Instruct
Status: SUCCESS

Throughput:
  Request throughput: 21.56 req/s
  Output throughput: 689.95 tok/s
  Total throughput: 3449.75 tok/s
  Completed requests: 8

Latency:
  TTFT (mean/p99): 23.04ms / 27.81ms
  TPOT (mean/p99): 5.21ms / 5.43ms
  ITL  (mean/p99): 5.21ms / 5.43ms
  E2EL (mean/p99): 184.40ms / 187.33ms

GPU Hardware Monitoring:
  GPU Clock: 2895 - 2900 MHz (avg: 2900)
  Power: 24.1 - 87.1 W (avg: 45.2)

setsid vllm serve Qwen/Qwen2.5-0.5B-Instruct --port 8888 --tensor-parallel-size=1 \
  --gpu-memory-utilization 0.7 --max-model-len 4096 --trust-remote-code

and echoed back by vLLM itself in server.log:

non-default args: {'model_tag': 'Qwen/Qwen2.5-0.5B-Instruct', 'port': 8888,
 'model': 'Qwen/Qwen2.5-0.5B-Instruct', 'trust_remote_code': True,
 'max_model_len': 4096, 'gpu_memory_utilization': 0.7}

Run 2 — accuracy path (RUN_EVAL=true)

Same command, with the eval envs above enabled, to exercise the eval tail
(magpie_run_eval_persisted). Also Status: SUCCESS, with the eval artifacts produced
and persisted:

$ ls gfx11_e2e_out/benchmark_vllm_*/
accuracy_report.json  benchmark_report.json  config.yaml
inferencex_result.json  lm_eval/  server.log  summary.txt

$ cat gfx11_e2e_out/benchmark_vllm_*/accuracy_report.json
{
  "status": "COMPLETED",
  "provider": "lm-eval",
  "task": "gsm8k",
  "metric": "exact_match,strict-match",
  "samples": 5,
  "score": 0.0,
  "error": null,
  "source_result": "lm_eval/results_2026-09-08T12-35-29.661085.json"
}

The score is meaningless at EVAL_LIMIT=5 on a 0.5B model — what is verified here is
that the path runs, persists lm_eval/, and writes a well-formed report. The
eval-concurrency contract is confirmed by lm_eval/meta_env.json, where
EVAL_CONCURRENT_REQUESTS resolved from CONC and the runner name propagated:

{ "conc": 4, "hw": "gfx11", "model": "Qwen/Qwen2.5-0.5B-Instruct", "isl": "128", "osl": "32" }

[haofrank]

Overall LGTM, just one comment.

[haofrank]

Hi @mgehre-amd I saw there is another PR #90 . Could you check this scope and leave a comment on that PR?

[mgehre-amd]

Hi @mgehre-amd I saw there is another PR #90 . Could you check this scope and leave a comment on that PR?

Thanks, I verified that one and left comments for the missing pieces.

[haofrank]

Hi @mgehre-amd Everything looks good overall. Do you have any other features to add? If not, we can go ahead and merge.

[mgehre-amd]

Hi @mgehre-amd Everything looks good overall. Do you have any other features to add? If not, we can go ahead and merge.

I just re-validated this PR and it works fine on my Strix Halo. We can merge.