Sparse array programming frameworks, such as SciPy or pydata/sparse, are getting more advanced. Because the performance and optimization strategies for sparse frameworks depends heavily on the input sparsity patterns and programs, we need realistic applications to make informed design decisions. This benchmark suite consists of several applications written entirely using collective operations (such as +, *, sum, or reduce) over sparse arrays. The programs are adapted from real-world applications using a straightforward translation to an array-programming style. The standard form for our benchmark functions is vanilla python code using only Array-API functions, with minimal looping or control flow.
We take great inspiration from the great benchmarks in the database community, such as pandasbench, Join Order Benchmark, or TPC-H
Start with the policy doc, which describes the policies and processes by which you can contribute benchmarks to the repo.
Most importantly: Before implementing a benchmark, claim it! Select or create your own github issue describing which application you want to benchmark, with links to relevant source code, and assign yourself to the issue if possible so that others know you're working on that benchmark. You can only claim a benchmark for a maximum of 1 month, after which others can claim it.
Once you're on board, see CONTRIBUTING.md for software guidelines, development setup, and best practices.
SparseApplicationBenchmark uses poetry for packaging. To install for development, clone the repository and run:
poetry install --with testto install the current project and dev dependencies.