Add improvement plans and architecture decision docs

[blooop]

Summary

Adds a plans/ directory containing self-contained improvement plans and architecture decision documents for the bencher repository. Each plan is written so it can be executed independently by an AI agent or developer.

Tactical plans (plans/01–08):

• 01 Release & CI safety — gate PyPI auto-publish on CI passing
• 02 Inflight PR triage — 10 open PRs, several stale/conflicting
• 03 Repo hygiene — remove stale root-level plan files and dead packaging files
• 04 Dependencies & import time — diskcache CVE, numpy floor, extras split, eager imports
• 05 Test coverage gaps — ~27 untested results/ modules, no coverage threshold
• 06 Docs & onboarding — README, caching/over_time docs, missing docstrings
• 07 Low-risk core cleanup
• 08 Larger core refactors (needs owner sign-off)

Architecture decision docs (plans/architecture/):

• A1 Rendering backend unification (plugin registry; resolves Port visualizations from HoloViews/Panel to Plotly (~100x faster save) #830-vs-Plot plugin infrastructure (tier 0, additive) #932)
• A2 Plot selection redesign (ranked, serializable plot specs)
• A3 BenchData contract (one frozen, pickle-free data type) — keystone
• A4 Caching architecture (one storage interface; absorbs Replace diskcache with minimalkv (CVE-2025-69872) #760 and feat: netCDF-backed history_dir for self-benchmark tracking on RTD #799)

plans/README.md carries the plan index, recommended ordering, ground rules, and a state-of-the-repo review summary.

Test plan

Documentation-only change (markdown under plans/). No code paths affected; no tests required.

🤖 Generated with Claude Code

Summary by Sourcery

Add a structured planning system under a new plans/ directory, including tactical improvement plans and architecture decision documents for rendering, caching, data contracts, tests, docs, CI, and core refactors.

Documentation:

• Introduce multiple markdown-based improvement plans covering CI/release safety, PR triage, repo hygiene, dependencies/import time, test coverage, docs/onboarding, low-risk cleanup, and larger core refactors.
• Document four architecture decision proposals for rendering backend unification, plot selection redesign, a unified BenchData contract, and a caching architecture redesign, with phased migration plans and owner decision points.

[sourcery-ai]

Reviewer's Guide

Adds a new plans/ documentation tree containing tactical improvement plans and higher-level architecture decision documents for bencher, plus a README that indexes and contextualizes them. All changes are markdown-only and introduce no code or config modifications.

File-Level Changes

Change	Details	Files
Introduce tactical improvement plans covering CI/release safety, PR triage, repo hygiene, dependencies/import time, test coverage, docs, core cleanup, and larger refactors.	Add Plan 01 describing how to gate PyPI auto-publish on successful CI, add a coverage floor, adjust Codecov behavior, ignore test artifacts, and enforce uniqueness of generated example filenames. Add Plan 02 for triaging the current set of open PRs, including explicit owner decisions for conflicting rendering PRs and security-related dependency work. Add Plan 03 to archive stale root-level planning docs and remove legacy build tooling files, plus clarify root-level files in CLAUDE/AGENTS docs. Add Plan 04 to address dependency issues (diskcache CVE, numpy floor), document pickle boundaries, and lazily import heavy libraries. Add Plan 05 to systematically increase test coverage for results and core modules, deflake tests, and resolve or document long-skipped tests. Add Plan 06 to improve README onboarding, organize feature docs (result types, caching, over_time), and document missing public APIs. Add Plan 07 outlining low-risk core cleanups like dead-code deletion, logging/exception handling improvements, and documentation of config interactions. Add Plan 08 outlining higher-risk structural refactors for Bench, BenchCfg, and regression modules, with strict behavior-preserving and sequencing rules.	plans/01-release-safety.md plans/02-pr-triage.md plans/03-repo-hygiene.md plans/04-dependencies.md plans/05-test-coverage.md plans/06-docs-onboarding.md plans/07-core-cleanup.md plans/08-core-refactors.md
Add architecture decision documents defining target designs and phased migrations for rendering, plot selection, data contracts, and caching.	Add A1 rendering-backend plan that reconciles existing Plotly and plugin PRs into a unified plugin registry, phased fast-save integration, and backend coexistence strategy. Add A2 plot-selection plan that centralizes plugin-based matching/ranking, introduces a richer data signature, and replaces callable-based plot configuration with serializable specs. Add A3 BenchData contract plan that defines a single frozen, pickle-free representation for benchmark results, including a netCDF + JSON + artifacts directory format and a staged migration path. Add A4 caching-architecture plan that consolidates storage behind a BencherStore interface, centralizes key composition with code hashing, replaces path-convention media with manifests, removes pickle from cache values, and phases in history robustness.	plans/architecture/A1-rendering-backend-unification.md plans/architecture/A2-plot-selection-redesign.md plans/architecture/A3-benchdata-contract.md plans/architecture/A4-caching-architecture.md
Add a plans/README.md that explains how to use the new plans, their ordering, and the current state-of-repo assessment.	Describe global ground rules for running plans (pixi usage, CI requirements, branching, versioning, package naming). Provide an indexed table of the tactical plans with risk/effort and recommended execution order. Summarize the architecture plans, their dependencies, and sequencing guidance (with A3 as keystone). Document a concise state-of-the-repo review, highlighting strengths and prioritized problem areas tied back to specific plans.	plans/README.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've reviewed your changes and they look great!

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[github-actions]

Performance Report for 989fe22

Metric	Value
Total tests	1479
Total time	120.79s
Mean	0.0817s
Median	0.0020s

Top 10 slowest tests

Test	Time (s)
test.test_bench_examples.TestBenchExamples::test_example_meta	17.974
test.test_over_time_save_perf::test_save_faster_without_aggregated_tab	5.068
test.test_generated_examples::test_generated_example[regression/example_regression_tuning_drift.py]	4.466
test.test_generated_examples::test_generated_example[result_types/result_image/example_result_image_to_video.py]	3.511
test.test_generated_examples::test_generated_example[cartesian_animation/example_cartesian_animation.py]	3.065
test.test_split_render_examples::test_split_render_subprocess_media	2.996
test.test_split_render_examples::test_split_render_roundtrip[result_image/example_result_image_to_video.py]	2.775
test.test_hash_persistent.TestCrossProcessDeterminism::test_hash_stable_across_two_processes[ResultBool]	2.757
test.test_generated_examples::test_generated_example[regression/example_regression_tuning_step.py]	2.730
test.test_generated_examples::test_generated_example[regression/example_regression_tuning_noise.py]	2.655

Full report

Updated by Performance Tracking workflow