Skip to content

Add reproducible Mac Studio M3 Ultra benchmark submission#28

Open
nabbilkhan wants to merge 1 commit intomaderix:mainfrom
nabbilkhan:contrib/m3-ultra-benchmark-package
Open

Add reproducible Mac Studio M3 Ultra benchmark submission#28
nabbilkhan wants to merge 1 commit intomaderix:mainfrom
nabbilkhan:contrib/m3-ultra-benchmark-package

Conversation

@nabbilkhan
Copy link
Contributor

@nabbilkhan nabbilkhan commented Mar 3, 2026

Summary

This PR adds a reproducible community benchmark submission package for Mac Studio (Apple M3 Ultra, 256 GB RAM), captured against commit 443194bca4491fae4400bae9dad2a0470692bdbf.

It is intended as a concrete contribution toward cross-chip coverage in #3.

What is included

  • benchmarks/README.md with a lightweight submission format
  • benchmarks/submissions/m3-ultra-mac-studio-2026-03-03/ containing:
    • README.md (environment + commands + key metrics)
    • metrics.json (machine-readable results)
    • commands.sh (exact repro commands)
    • raw/*.log + raw/system_info.txt + raw/upstream_commit.txt (auditable raw outputs)
  • Root README.md pointer to community benchmark submissions and this M3 Ultra entry

Key results (20-step training runs)

Pipeline Avg train ANE TFLOPS Total TFLOPS
train_large 81.2 ms/step 1.15 2.15
train_large_ane 71.4 ms/step 1.48 2.44
train_large_ane --no-ane-extras 123.8 ms/step 0.85 1.41
training_dynamic/train --scratch 115.4 ms/step n/a n/a

inmem_peak best observed: 8.08 TFLOPS (128x conv 512ch sp64).

Notes

  • Serial/UUID identifiers are redacted in raw/system_info.txt.

  • inmem_bench and sram_bench outputs are included as observed (FAIL(-1) for all rows on this clean setup), to keep the submission fully transparent.

  • Disclosure: I am a contributor to Open Claw.

dev-erik added a commit to dev-erik/ANE that referenced this pull request Mar 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant