Skip to content

Reduce cloning in TestVariantIterator#683

Merged
MatthewMckee4 merged 1 commit intomainfrom
reduce-test-variant-clones
Apr 15, 2026
Merged

Reduce cloning in TestVariantIterator#683
MatthewMckee4 merged 1 commit intomainfrom
reduce-test-variant-clones

Conversation

@MatthewMckee4
Copy link
Copy Markdown
Member

Summary

TestVariantIterator sits on the hot path for every test invocation — once per parametrize variant, per fixture set, per test. It was doing more copying than it needed to.

The big change is ownership. Before, constructing the iterator rebuilt the whole DiscoveredTestFunction into an Rc just so each emitted variant could hold a shared pointer to it:

test: Rc::new(DiscoveredTestFunction {
    name: test.name.clone(),
    stmt_function_def: Rc::clone(&test.stmt_function_def),
    py_function: test.py_function.clone_ref(py),
    tags: test.tags.clone(),
}),

But the caller already owns the DiscoveredTestFunction inside module.test_functions(), and that outlives the iterator. So the iterator now borrows &'a DiscoveredTestFunction directly, and TestVariant<'a> carries that same borrow. The rebuild and its three clones (name, tags, py_function refcount bump) are gone.

The fixture lists are the other big win. A test with N parametrize variants and M fixtures used to allocate N fresh Vec<Rc<NormalizedFixture>> and bump N × M refcounts per next() call. Wrapping them in Rc<[Rc<NormalizedFixture>]> reduces each variant to a single refcount bump on the outer Rc, regardless of how many fixtures are in the list.

param_args is now consumed as a std::vec::IntoIter<ParametrizationArgs>. Each variant moves the owning values: HashMap<String, Arc<Py<PyAny>>> and tags: Tags out by value instead of cloning them. This has a follow-on effect in setup_test_fixtures: Arc::try_unwrap(value) used to always fall back to clone_ref(py) because the clone kept the refcount at ≥2, but with the move it hits the fast path and skips a Python refcount bump per parametrize argument.

With these changes, next() becomes the natural shape of an iterator, so the type now implements Iterator (plus size_hint/ExactSizeIterator from the underlying IntoIter), and the call site in package_runner uses a plain for variant in ... loop instead of while let Some(variant) = iterator.next_with_py(py).

Two small drive-by cleanups landed alongside the main change: a couple of match arms in package_runner that reshaped Err into FixtureCallError collapsed into map_err(|err| FixtureCallError { ... })?, and NormalizedFixture::call now only builds the kwargs PyDict when there are actually fixture arguments to pass.

Test Plan

  • just test — full suite passes (parametrize, fixtures, autouse, use_fixtures, snapshot coverage all exercise this code path)
  • cargo clippy -p karva_test_semantic --lib -- -D warnings — clean
  • uvx prek run -a

Rework `TestVariantIterator` so producing a variant is cheap. The
iterator now borrows `&DiscoveredTestFunction` from the surrounding
module instead of rebuilding it into an `Rc`, and shares fixture lists
across variants via `Rc<[Rc<NormalizedFixture>]>` so each variant costs
a handful of refcount bumps rather than a full `Vec` clone per fixture
set. `param_args` is consumed as an `IntoIter`, moving each
`ParametrizationArgs.values` (and `.tags`) into the emitted variant;
this also lets `Arc::try_unwrap` in `setup_test_fixtures` succeed,
skipping a Python refcount bump per parametrize argument.

Implements `Iterator` on `TestVariantIterator` so the call site in
`package_runner` can use a plain `for variant in ...` loop.

Also collapses a couple of `match` arms in `package_runner` into
`map_err(...)?` and skips building the kwargs `PyDict` in
`NormalizedFixture::call` when there are no fixture arguments.
@MatthewMckee4 MatthewMckee4 added internal An internal refactor or improvement performance Related to the performance of Karva test-running Related to running tests with karva rust Pull requests that update rust code and removed internal An internal refactor or improvement test-running Related to running tests with karva rust Pull requests that update rust code labels Apr 14, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Apr 14, 2026

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 1 untouched benchmark


Comparing reduce-test-variant-clones (d9d4ecd) with main (8a09fc2)

Open in CodSpeed

@MatthewMckee4 MatthewMckee4 merged commit 2f2b75d into main Apr 15, 2026
11 checks passed
@MatthewMckee4 MatthewMckee4 deleted the reduce-test-variant-clones branch April 15, 2026 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Related to the performance of Karva

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant