fix: debug benchmarks by pfackeldey · Pull Request #3550 · scikit-hep/awkward

pfackeldey · 2025-06-16T19:20:11Z

DO NOT MERGE

github-actions · 2025-06-16T23:06:43Z

Benchmarks

🔹 ak.imag(Flat<4096,f64>)

Relative CPU Time Difference: 27.2% — 🟢 Improvement

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	4.357669e-02	3.425121e-02
`real_time` (ms)	4.358067e-02	3.425505e-02
`elements/s` (Hz)	9.40e+07	1.20e+08

🔹 ak.all(Jagged<4096,f64>, axis=None)

Relative CPU Time Difference: 188.9% — 🔴 Regression

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	1.525032e-01	4.406328e-01
`real_time` (ms)	1.525210e-01	1.005188e+02
`elements/s` (Hz)	2.69e+07	9.30e+06

🔹 ak.all(Jagged<4096,f64>, axis=0)

Relative CPU Time Difference: 89.5% — 🔴 Regression

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	4.470682e-01	8.474089e-01
`real_time` (ms)	4.471149e-01	1.009239e+02
`elements/s` (Hz)	9.16e+06	4.83e+06

🔹 ak.all(Jagged<4096,f64>, axis=1)

Relative CPU Time Difference: 132.6% — 🔴 Regression

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	2.584714e-01	6.011930e-01
`real_time` (ms)	2.584880e-01	1.006774e+02
`elements/s` (Hz)	1.58e+07	6.81e+06

🔹 ak.all(Jagged<65536,f64>, axis=None)

Relative CPU Time Difference: 112.9% — 🔴 Regression

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	2.849572e-01	6.066497e-01
`real_time` (ms)	2.849852e-01	1.006828e+02
`elements/s` (Hz)	2.30e+08	1.08e+08

🔹 ak.all(Jagged<65536,f64>, axis=0)

Relative CPU Time Difference: 42.0% — 🔴 Regression

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	1.723331e+00	2.447961e+00
`real_time` (ms)	1.723511e+00	1.025244e+02
`elements/s` (Hz)	3.80e+07	2.68e+07

🔹 ak.all(Jagged<65536,f64>, axis=1)

Relative CPU Time Difference: 99.0% — 🔴 Regression

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	4.014345e-01	7.988675e-01
`real_time` (ms)	4.014665e-01	1.008770e+02
`elements/s` (Hz)	1.63e+08	8.20e+07

🔹 ak.all(Jagged<1048576,f64>, axis=None)

Relative CPU Time Difference: 25.0% — 🔴 Regression

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	2.385330e+00	2.980776e+00
`real_time` (ms)	2.388469e+00	1.030591e+02
`elements/s` (Hz)	4.40e+08	3.52e+08

🔹 ak.all(Jagged<1048576,f64>, axis=1)

Relative CPU Time Difference: 25.6% — 🔴 Regression

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	2.610691e+00	3.277970e+00
`real_time` (ms)	2.610894e+00	1.033570e+02
`elements/s` (Hz)	4.02e+08	3.20e+08

🔹 ak.all(Flat<4096,f64>, axis=None)

Relative CPU Time Difference: 217.4% — 🔴 Regression

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	1.305395e-01	4.143288e-01
`real_time` (ms)	1.305505e-01	1.004906e+02
`elements/s` (Hz)	3.14e+07	9.89e+06

🔹 ak.all(Flat<65536,f64>, axis=None)

Relative CPU Time Difference: 121.9% — 🔴 Regression

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	2.656564e-01	5.894474e-01
`real_time` (ms)	2.656755e-01	1.006667e+02
`elements/s` (Hz)	2.47e+08	1.11e+08

🔹 ak.all(Flat<1048576,f64>, axis=None)

Relative CPU Time Difference: 28.3% — 🔴 Regression

Show full comparison

Metric	benchmark.py.json	.json
`cpu_time` (ms)	2.336778e+00	2.999138e+00
`real_time` (ms)	2.339054e+00	1.030775e+02
`elements/s` (Hz)	4.49e+08	3.50e+08

github-actions · 2025-06-17T13:38:26Z

Benchmarks

🔹 ak.imag(Flat<4096,f64>)

Relative CPU Time Difference: 25.2% — 🟢 Improvement

Show full comparison

Metric	benchmarks__1f8eae088ec5387828a7844dd5225a1c91c34087.json	debug_benchmarks_DONOTMERGE__96fb634a49b0d6bc94bc6f7d0e139aaab179031b.json
`cpu_time` (ms)	4.250441e-02	3.396040e-02
`real_time` (ms)	4.250705e-02	3.396317e-02
`elements/s` (Hz)	9.64e+07	1.21e+08

🔹 ak.round(Flat<1048576,f64>)

Relative CPU Time Difference: 11.8% — 🟢 Improvement

Show full comparison

Metric	benchmarks__1f8eae088ec5387828a7844dd5225a1c91c34087.json	debug_benchmarks_DONOTMERGE__96fb634a49b0d6bc94bc6f7d0e139aaab179031b.json
`cpu_time` (ms)	3.293008e-01	2.945439e-01
`real_time` (ms)	3.293405e-01	2.948629e-01
`elements/s` (Hz)	3.18e+09	3.56e+09

🔹 ak.all(Jagged<4096,f64>, axis=None)

Relative CPU Time Difference: 181.5% — 🔴 Regression

Show full comparison

Metric	benchmarks__1f8eae088ec5387828a7844dd5225a1c91c34087.json	debug_benchmarks_DONOTMERGE__96fb634a49b0d6bc94bc6f7d0e139aaab179031b.json
`cpu_time` (ms)	1.526265e-01	4.296427e-01
`real_time` (ms)	1.526306e-01	1.005057e+02
`elements/s` (Hz)	2.68e+07	9.53e+06

🔹 ak.all(Jagged<65536,f64>, axis=None)

Relative CPU Time Difference: 99.6% — 🔴 Regression

Show full comparison

Metric	benchmarks__1f8eae088ec5387828a7844dd5225a1c91c34087.json	debug_benchmarks_DONOTMERGE__96fb634a49b0d6bc94bc6f7d0e139aaab179031b.json
`cpu_time` (ms)	2.866951e-01	5.723715e-01
`real_time` (ms)	2.867163e-01	1.006485e+02
`elements/s` (Hz)	2.29e+08	1.14e+08

🔹 ak.all(Jagged<1048576,f64>, axis=None)

Relative CPU Time Difference: 23.5% — 🔴 Regression

Show full comparison

Metric	benchmarks__1f8eae088ec5387828a7844dd5225a1c91c34087.json	debug_benchmarks_DONOTMERGE__96fb634a49b0d6bc94bc6f7d0e139aaab179031b.json
`cpu_time` (ms)	2.383876e+00	2.945066e+00
`real_time` (ms)	2.384036e+00	1.030221e+02
`elements/s` (Hz)	4.40e+08	3.56e+08

🔹 ak.all(Flat<4096,f64>, axis=None)

Relative CPU Time Difference: 252.3% — 🔴 Regression

Show full comparison

Metric	benchmarks__1f8eae088ec5387828a7844dd5225a1c91c34087.json	debug_benchmarks_DONOTMERGE__96fb634a49b0d6bc94bc6f7d0e139aaab179031b.json
`cpu_time` (ms)	1.293830e-01	4.558587e-01
`real_time` (ms)	1.293936e-01	1.005335e+02
`elements/s` (Hz)	3.17e+07	8.99e+06

🔹 ak.all(Flat<65536,f64>, axis=None)

Relative CPU Time Difference: 130.6% — 🔴 Regression

Show full comparison

Metric	benchmarks__1f8eae088ec5387828a7844dd5225a1c91c34087.json	debug_benchmarks_DONOTMERGE__96fb634a49b0d6bc94bc6f7d0e139aaab179031b.json
`cpu_time` (ms)	2.619397e-01	6.040597e-01
`real_time` (ms)	2.619586e-01	1.006794e+02
`elements/s` (Hz)	2.50e+08	1.08e+08

🔹 ak.all(Flat<1048576,f64>, axis=None)

Relative CPU Time Difference: 29.3% — 🔴 Regression

Show full comparison

Metric	benchmarks__1f8eae088ec5387828a7844dd5225a1c91c34087.json	debug_benchmarks_DONOTMERGE__96fb634a49b0d6bc94bc6f7d0e139aaab179031b.json
`cpu_time` (ms)	2.336854e+00	3.021203e+00
`real_time` (ms)	2.336965e+00	1.030966e+02
`elements/s` (Hz)	4.49e+08	3.47e+08

🔹 ak.any(Flat<1048576,f64>, axis=None)

Relative CPU Time Difference: 10.3% — 🔴 Regression

Show full comparison

Metric	benchmarks__1f8eae088ec5387828a7844dd5225a1c91c34087.json	debug_benchmarks_DONOTMERGE__96fb634a49b0d6bc94bc6f7d0e139aaab179031b.json
`cpu_time` (ms)	2.329831e+00	2.568651e+00
`real_time` (ms)	2.329899e+00	2.568770e+00
`elements/s` (Hz)	4.50e+08	4.08e+08

github-actions · 2025-06-17T13:58:49Z

Benchmarks

🔹 ak.imag(Flat<4096,f64>)

Relative CPU Time Difference: 34.7% — 🟢 Improvement

Show full comparison

Metric	benchmarks__1f8eae088ec5387828a7844dd5225a1c91c34087.json	debug_benchmarks_DONOTMERGE__07f2f72617e292856e5478ed1091957cb9c60724.json
`cpu_time` (ms)	4.422925e-02	3.284300e-02
`real_time` (ms)	4.423173e-02	3.284535e-02
`elements/s` (Hz)	9.26e+07	1.25e+08

…cikit-hep/awkward into pfackeldey/debug_benchmarks_DONOTMERGE

github-actions · 2025-06-17T14:16:01Z

Benchmarks

🔹 ak.real(Flat<65536,f64>)

Relative CPU Time Difference: 10.9% — 🟢 Improvement

Show full comparison

Metric	branch: results/pfackeldey/benchmarks (sha: `1f8eae0`)	branch: results/pfackeldey/debug_benchmarks_DONOTMERGE (sha: `2a1731d`)
`cpu_time` (ms)	5.358981e-02	4.832270e-02
`real_time` (ms)	5.359375e-02	4.832410e-02
`elements/s` (Hz)	1.22e+09	1.36e+09

github-actions · 2025-06-17T14:45:07Z

Benchmarks

🔹 ak.imag(Flat<4096,f64>)

Relative CPU Time Difference: 35.6% — 🟢 Improvement

Show full comparison

Metric	branch: `pfackeldey/benchmarks` (sha: `1f8eae0`)	branch: `pfackeldey/debug_benchmarks_DONOTMERGE` (sha: `1a9e8cb`)
`cpu_time` (ms)	4.398644e-02	3.244064e-02
`real_time` (ms)	4.399161e-02	3.244424e-02
`elements/s` (Hz)	9.31e+07	1.26e+08

🔹 ak.mean(Flat<1048576,f64>, axis=None)

Relative CPU Time Difference: 11.1% — 🔴 Regression

Show full comparison

Metric	branch: `pfackeldey/benchmarks` (sha: `1f8eae0`)	branch: `pfackeldey/debug_benchmarks_DONOTMERGE` (sha: `1a9e8cb`)
`cpu_time` (ms)	7.632385e+00	8.479708e+00
`real_time` (ms)	7.633406e+00	8.480705e+00
`elements/s` (Hz)	1.37e+08	1.24e+08

…cikit-hep/awkward into pfackeldey/debug_benchmarks_DONOTMERGE

github-actions · 2025-06-17T15:22:06Z

Benchmarks

🔹 ak.imag(Jagged<1048576,f64>)

Relative CPU Time Difference: 20.6% — 🔴 Regression

Show full comparison

Metric	branch: `pfackeldey/benchmarks` (sha: `1f8eae0`)	branch: `pfackeldey/debug_benchmarks_DONOTMERGE` (sha: `738feca`)
`cpu_time` (ms)	1.467450e+00	1.769141e+00
`real_time` (ms)	1.477987e+00	1.782400e+00
`elements/s` (Hz)	7.15e+08	5.93e+08

🔹 ak.imag(Flat<1048576,f64>)

Relative CPU Time Difference: 21.3% — 🔴 Regression

Show full comparison

Metric	branch: `pfackeldey/benchmarks` (sha: `1f8eae0`)	branch: `pfackeldey/debug_benchmarks_DONOTMERGE` (sha: `738feca`)
`cpu_time` (ms)	1.434976e+00	1.740423e+00
`real_time` (ms)	1.444160e+00	1.748841e+00
`elements/s` (Hz)	7.31e+08	6.02e+08

debug test 1

608022f

pfackeldey temporarily deployed to docs June 16, 2025 19:30 — with GitHub Actions Inactive

fix output piping for benchmark comparisons

ca2dd79

pfackeldey temporarily deployed to docs June 16, 2025 19:48 — with GitHub Actions Inactive

try handling multiline outputs

b91d612

pfackeldey temporarily deployed to docs June 16, 2025 20:14 — with GitHub Actions Inactive

hopefully fix paths

16193e2

pfackeldey temporarily deployed to docs June 16, 2025 20:25 — with GitHub Actions Inactive

another try to pass multiline output

b59cc6c

pfackeldey temporarily deployed to docs June 16, 2025 23:07 — with GitHub Actions Inactive

try fix json file names

55fc422

pfackeldey temporarily deployed to docs June 17, 2025 12:49 — with GitHub Actions Inactive

pfackeldey added 2 commits June 17, 2025 09:19

fix directory creation for benchmark results

8bc22ec

please precommit

2cfc2b4

pfackeldey temporarily deployed to docs June 17, 2025 13:35 — with GitHub Actions Inactive

pfackeldey and others added 2 commits June 17, 2025 09:48

prettify table headers

844b928

style: pre-commit fixes

fd6b83f

pre-commit-ci Bot temporarily deployed to docs June 17, 2025 14:01 Inactive

pfackeldey added 2 commits June 17, 2025 10:02

go back to original commit for comparison

cc28045

Merge branch 'pfackeldey/debug_benchmarks_DONOTMERGE' of github.com:s…

28b0c28

…cikit-hep/awkward into pfackeldey/debug_benchmarks_DONOTMERGE

pfackeldey temporarily deployed to docs June 17, 2025 14:14 — with GitHub Actions Inactive

pfackeldey and others added 2 commits June 17, 2025 10:36

prettify branch name and SHA in table header

b75e54d

style: pre-commit fixes

a66496a

pre-commit-ci Bot temporarily deployed to docs June 17, 2025 14:47 Inactive

try self-hosted runner

1c817ef

pfackeldey added 3 commits June 17, 2025 10:56

Merge branch 'pfackeldey/debug_benchmarks_DONOTMERGE' of github.com:s…

b92f4d5

…cikit-hep/awkward into pfackeldey/debug_benchmarks_DONOTMERGE

another try with self-hosted runner

670dda0

another try

0ede375

pfackeldey deployed to docs June 17, 2025 15:25 — with GitHub Actions View deployment

pfackeldey marked this pull request as ready for review June 17, 2025 15:39

pfackeldey merged commit 3e2bf2d into pfackeldey/benchmarks Jun 17, 2025
43 checks passed

pfackeldey deleted the pfackeldey/debug_benchmarks_DONOTMERGE branch June 17, 2025 15:40

pfackeldey mentioned this pull request Jun 18, 2025

feat: add benchmarks for awkward #3548

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: debug benchmarks#3550

fix: debug benchmarks#3550
pfackeldey merged 18 commits into
pfackeldey/benchmarksfrom
pfackeldey/debug_benchmarks_DONOTMERGE

pfackeldey commented Jun 16, 2025

Uh oh!

github-actions Bot commented Jun 16, 2025

Uh oh!

github-actions Bot commented Jun 17, 2025

Uh oh!

github-actions Bot commented Jun 17, 2025

Uh oh!

github-actions Bot commented Jun 17, 2025

Uh oh!

github-actions Bot commented Jun 17, 2025

Uh oh!

github-actions Bot commented Jun 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

pfackeldey commented Jun 16, 2025

Uh oh!

github-actions Bot commented Jun 16, 2025

Benchmarks

🔹 ak.imag(Flat<4096,f64>)

🔹 ak.all(Jagged<4096,f64>, axis=None)

🔹 ak.all(Jagged<4096,f64>, axis=0)

🔹 ak.all(Jagged<4096,f64>, axis=1)

🔹 ak.all(Jagged<65536,f64>, axis=None)

🔹 ak.all(Jagged<65536,f64>, axis=0)

🔹 ak.all(Jagged<65536,f64>, axis=1)

🔹 ak.all(Jagged<1048576,f64>, axis=None)

🔹 ak.all(Jagged<1048576,f64>, axis=1)

🔹 ak.all(Flat<4096,f64>, axis=None)

🔹 ak.all(Flat<65536,f64>, axis=None)

🔹 ak.all(Flat<1048576,f64>, axis=None)

Uh oh!

github-actions Bot commented Jun 17, 2025

Benchmarks

🔹 ak.imag(Flat<4096,f64>)

🔹 ak.round(Flat<1048576,f64>)

🔹 ak.all(Jagged<4096,f64>, axis=None)

🔹 ak.all(Jagged<65536,f64>, axis=None)

🔹 ak.all(Jagged<1048576,f64>, axis=None)

🔹 ak.all(Flat<4096,f64>, axis=None)

🔹 ak.all(Flat<65536,f64>, axis=None)

🔹 ak.all(Flat<1048576,f64>, axis=None)

🔹 ak.any(Flat<1048576,f64>, axis=None)

Uh oh!

github-actions Bot commented Jun 17, 2025

Benchmarks

🔹 ak.imag(Flat<4096,f64>)

Uh oh!

github-actions Bot commented Jun 17, 2025

Benchmarks

🔹 ak.real(Flat<65536,f64>)

Uh oh!

github-actions Bot commented Jun 17, 2025

Benchmarks

🔹 ak.imag(Flat<4096,f64>)

🔹 ak.mean(Flat<1048576,f64>, axis=None)

Uh oh!

github-actions Bot commented Jun 17, 2025

Benchmarks

🔹 ak.imag(Jagged<1048576,f64>)

🔹 ak.imag(Flat<1048576,f64>)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant