Skip to content

Add basic JSONBench-based benchmark#8340

Open
AdamGS wants to merge 1 commit into
adamg/json-arrow-exportfrom
adamg/compress-json-bench
Open

Add basic JSONBench-based benchmark#8340
AdamGS wants to merge 1 commit into
adamg/json-arrow-exportfrom
adamg/compress-json-bench

Conversation

@AdamGS

@AdamGS AdamGS commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds a very naive "compress jsonbench as strings" benchmark. The use of vortex_json required some API changes to the Dataset API, but I think this is relatively straight forward.

This is a step towards more variant-focused benchmarks and functionality, trying to get the boilerplate out of the way before I try and land bigger changes.

This PR builds on top of #8339, which includes some missing functionality for vortes-json.

@AdamGS AdamGS requested review from mprammer and robert3005 June 10, 2026 16:51
@AdamGS AdamGS added the changelog/chore A trivial change label Jun 10, 2026
@AdamGS AdamGS changed the title bench stuff Add basic JSONBench-based benchmark Jun 10, 2026
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
@AdamGS AdamGS force-pushed the adamg/compress-json-bench branch from 6b912e1 to 1e1a170 Compare June 10, 2026 16:55
@codspeed-hq

codspeed-hq Bot commented Jun 10, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 10.62%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 2 improved benchmarks
❌ 6 regressed benchmarks
✅ 1519 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation chunked_bool_canonical_into[(1000, 10)] 31.9 µs 46.8 µs -31.82%
Simulation decompress_rd[f64, (100000, 0.0)] 845.8 µs 1,024.6 µs -17.45%
Simulation chunked_varbinview_into_canonical[(1000, 10)] 177.2 µs 213.3 µs -16.95%
Simulation decompress_rd[f32, (100000, 0.0)] 499.5 µs 587 µs -14.91%
WallTime cuda/bitpacked_u8/unpack/3bw[100M] 300 µs 349.9 µs -14.25%
Simulation chunked_varbinview_canonical_into[(100, 100)] 274.7 µs 309.5 µs -11.26%
Simulation decompress_rd[f64, (100000, 0.01)] 981.6 µs 846 µs +16.03%
Simulation decompress_rd[f64, (100000, 0.1)] 981.5 µs 846 µs +16.02%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing adamg/compress-json-bench (1e1a170) with adamg/json-arrow-export (5430d3a)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant