qe: one namespace — benchmark and audit fold into qe (0.7.0) - #53
Merged
Conversation
Implements the #43 consolidation: /benchmark:review-acceleration becomes /qe:benchmark, /audit:issues becomes /qe:audit-issues, and the two plugins retire. Engines and references move to namespaced subdirs (scripts/{benchmark,audit}/, references/{benchmark,audit}/) with every ${CLAUDE_PLUGIN_ROOT} path, relative link, and the worked examples' run_all.py engine lookup updated; the scoring engine reproduces all three committed scorecards from the new layout, and CI's reproduction step now runs from qe/. Changelogs fold into qe/CHANGELOG.md as historical sections; qe starts at 0.7.0, strictly above every retiring stream. marketplace.json drops to one plugin (catalogue 0.3.0), and README/CATALOG/using-skills/developing-skills/AGENTS/myst.yml and both tutorials are swept to the new names and paths. Migration for installed users is in the 0.7.0 changelog entry: uninstall benchmark@quantecon and audit@quantecon so the retired names don't linger; lecture repos drop benchmark@quantecon from enabledPlugins. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR consolidates the retired benchmark and audit plugins into the single qe plugin (v0.7.0), flattening the namespace so all skills are invoked as /qe:<name> while keeping the underlying benchmark/audit procedures unchanged.
Changes:
- Move and rename benchmark and audit skills/scripts/references under
qe/, updating${CLAUDE_PLUGIN_ROOT}paths and docs accordingly. - Retire the
benchmarkandauditplugins from the marketplace manifest; bump marketplace catalog version andqeplugin version. - Update CI/doc tooling (link checks, scorecard reproduction job working directory, docs site TOC) to the new layout.
Reviewed changes
Copilot reviewed 44 out of 72 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Updates install/usage docs to the one-plugin /qe:* namespace and removes benchmark/audit install steps. |
| qe/skills/workplan/SKILL.md | Updates cross-skill link from /audit:issues to /qe:audit-issues. |
| qe/skills/benchmark/SKILL.md | Renames skill to benchmark and updates all runtime paths/links to the new qe/ layout. |
| qe/skills/audit-issues/SKILL.md | Renames skill to audit-issues and updates invocation + reference links to qe/references/audit/* and qe/scripts/audit/*. |
| qe/scripts/README.md | Documents per-family deterministic script layout under qe/scripts/{benchmark,audit}/. |
| qe/scripts/benchmark/scoring/score.py | Updates CLI usage strings/paths for the benchmark scorer (one example path still needs correction). |
| qe/scripts/benchmark/scoring/rubric.py | Updates rubric doc pointer paths and example README references for the new directory structure. |
| qe/scripts/benchmark/scoring/EVIDENCE_TEMPLATE.json | Updates the “how to run” guidance to scripts/benchmark/scoring/score.py and new example paths. |
| qe/scripts/benchmark/scoring/env_stamp.py | Updates CLI usage text to the new scripts/benchmark/scoring/ path. |
| qe/scripts/benchmark/README.md | New benchmark scripts guide under the consolidated plugin layout. |
| qe/scripts/benchmark/calibration/bellman_bench.py | Adds the shared HIGH-efficiency calibration benchmark under qe/scripts/benchmark/calibration/. |
| qe/scripts/benchmark/calibration/bellman_bench.json | Commits calibration benchmark results for the shared anchor. |
| qe/scripts/audit/README.md | Updates audit fetcher docs and paths for the consolidated plugin layout. |
| qe/references/benchmark/README.md | Updates benchmark family guide to /qe:benchmark and new script/reference locations. |
| qe/references/benchmark/fixtures/rubric_v2/results/scorecard.json | Updates _note path strings to the new scorer location. |
| qe/references/benchmark/fixtures/rubric_v2/evidence.json | Adds the synthetic fixture evidence under the new references tree. |
| qe/references/benchmark/fixtures/README.md | Updates fixture regeneration instructions to the new scorer path. |
| qe/references/benchmark/examples/README.md | Updates reference-example intro and paths to /qe:benchmark and new scorer/calibration locations. |
| qe/references/benchmark/examples/markov_asset/scripts/static_metrics.py | Adds/relocates markov_asset static-metrics script under consolidated references. |
| qe/references/benchmark/examples/markov_asset/scripts/smoke_test.py | Adds/relocates markov_asset smoke test script under consolidated references. |
| qe/references/benchmark/examples/markov_asset/scripts/run_all.py | Updates shared-engine resolution and scorer/env-stamp invocation paths. |
| qe/references/benchmark/examples/markov_asset/scripts/model_old.py | Adds/relocates extracted baseline model for markov_asset example. |
| qe/references/benchmark/examples/markov_asset/scripts/model_new.py | Adds/relocates extracted candidate model for markov_asset example. |
| qe/references/benchmark/examples/markov_asset/scripts/check_equivalence.py | Adds/relocates markov_asset equivalence check script under consolidated references. |
| qe/references/benchmark/examples/markov_asset/scripts/benchmark.py | Adds/relocates markov_asset scaling benchmark script under consolidated references. |
| qe/references/benchmark/examples/markov_asset/scripts/as_used_total.py | Adds/relocates markov_asset as-used total benchmark script under consolidated references. |
| qe/references/benchmark/examples/markov_asset/results/static_metrics.json | Adds/relocates committed markov_asset static metrics results. |
| qe/references/benchmark/examples/markov_asset/results/scorecard.json | Updates _note path strings for consolidated scorer location. |
| qe/references/benchmark/examples/markov_asset/results/scaling.json | Adds/relocates committed scaling results. |
| qe/references/benchmark/examples/markov_asset/results/equivalence_x64_True.json | Adds/relocates committed x64 equivalence results. |
| qe/references/benchmark/examples/markov_asset/results/equivalence_x64_False.json | Adds/relocates committed default-dtype equivalence results. |
| qe/references/benchmark/examples/markov_asset/markov_asset_REPORT.md | Updates rubric-engine path references for consolidated scorer location. |
| qe/references/benchmark/examples/markov_asset/evidence.json | Updates _how run command to the consolidated scorer and example path. |
| qe/references/benchmark/examples/ge_arrow/scripts/sweep_bench.py | Adds/relocates ge_arrow sweep benchmark script under consolidated references. |
| qe/references/benchmark/examples/ge_arrow/scripts/static_metrics.py | Adds/relocates ge_arrow static metrics script under consolidated references. |
| qe/references/benchmark/examples/ge_arrow/scripts/run_all.py | Updates shared-engine resolution and scorer/env-stamp invocation paths. |
| qe/references/benchmark/examples/ge_arrow/scripts/model_old.py | Adds/relocates extracted baseline model for ge_arrow example. |
| qe/references/benchmark/examples/ge_arrow/scripts/model_new.py | Adds/relocates extracted candidate model for ge_arrow example. |
| qe/references/benchmark/examples/ge_arrow/scripts/cold_start.py | Adds/relocates ge_arrow cold-start measurement script under consolidated references. |
| qe/references/benchmark/examples/ge_arrow/scripts/check_equivalence.py | Adds/relocates ge_arrow equivalence check script (contains an x64 flag access issue). |
| qe/references/benchmark/examples/ge_arrow/scripts/benchmark.py | Adds/relocates ge_arrow benchmark script under consolidated references. |
| qe/references/benchmark/examples/ge_arrow/scripts/as_used_total.py | Adds/relocates ge_arrow as-used total benchmark script under consolidated references. |
| qe/references/benchmark/examples/ge_arrow/results/sweep.json | Adds/relocates committed sweep results. |
| qe/references/benchmark/examples/ge_arrow/results/static_metrics.json | Adds/relocates committed static metrics results. |
| qe/references/benchmark/examples/ge_arrow/results/scorecard.json | Updates _note path strings for consolidated scorer location. |
| qe/references/benchmark/examples/ge_arrow/results/equivalence.json | Adds/relocates committed equivalence results (summary metadata is incomplete). |
| qe/references/benchmark/examples/ge_arrow/results/benchmark.json | Adds/relocates committed benchmark results. |
| qe/references/benchmark/examples/ge_arrow/ge_arrow_REPORT.md | Updates rubric-engine path references for consolidated scorer location. |
| qe/references/benchmark/examples/ge_arrow/evidence.json | Updates _how run command to the consolidated scorer and example path. |
| qe/references/benchmark/EVALUATION_FRAMEWORK.md | Updates rubric-engine and workflow paths to qe/scripts/benchmark/... and new example locations. |
| qe/references/audit/README.md | New audit family guide under qe/references/audit/, reflecting consolidated plugin packaging. |
| qe/references/audit/quantecon-context.md | New consolidated location for audit org-context reference material. |
| qe/references/audit/doctrine.md | Updates audit doctrine references and invocation name to /qe:audit-issues. |
| qe/references/audit/deliverables.md | Updates audit deliverables doc to /qe:audit-issues. |
| qe/.claude-plugin/plugin.json | Bumps qe version to 0.7.0 and expands description to include benchmark/audit. |
| myst.yml | Updates docs site TOC to point at the new qe/references/{benchmark,audit}/ guides. |
| docs/using-skills.md | Updates setup/install/troubleshooting docs for the single qe plugin and renamed skills. |
| docs/tutorial-run-an-evaluation.md | Updates evaluation tutorial references and ${CLAUDE_PLUGIN_ROOT} paths to the consolidated layout. |
| docs/tutorial-run-an-audit.md | Updates audit tutorial references and install/invocation to /qe:audit-issues and qe@quantecon. |
| docs/developing-skills.md | Updates repository conventions to the one-plugin reality and adjusts dev loop examples accordingly. |
| CATALOG.md | Updates catalog framing to “one plugin, multiple families” and renames the benchmark/audit entries to /qe:*. |
| benchmark/scripts/README.md | Removes benchmark plugin scripts README (plugin retired; content moved under qe/scripts/benchmark/). |
| benchmark/CHANGELOG.md | Removes benchmark plugin changelog (folded into qe/CHANGELOG.md per PR description). |
| benchmark/.claude-plugin/plugin.json | Removes benchmark plugin manifest (plugin retired). |
| audit/README.md | Removes audit plugin README (content moved under qe/references/audit/). |
| audit/CHANGELOG.md | Removes audit plugin changelog (folded into qe/CHANGELOG.md per PR description). |
| audit/.claude-plugin/plugin.json | Removes audit plugin manifest (plugin retired). |
| AGENTS.md | Updates doc-map references and “numbers drift fastest” pointers to the new benchmark rubric locations. |
| .github/workflows/validate.yml | Updates scorecard reproduction job to run from qe/ and use new scorer paths. |
| .claude-plugin/marketplace.json | Removes benchmark/audit plugins, bumps marketplace version, and updates qe plugin version/description. |
Suppressed comments (1)
qe/scripts/benchmark/scoring/score.py:8
- The usage example in this docstring still points at the pre-consolidation path
references/examples/ge_arrow, but the examples now live underreferences/benchmark/examples/.... This example is likely to send users to a non-existent path from the plugin root.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The docs links CI walks reviews/ too, and three frozen run records still linked relatively into the benchmark/ and audit/ trees that #53 moves. These are dated records of the retired streams, so instead of repointing them at today's layout they now link to the trees they actually reviewed, pinned at benchmark--v0.4.0 and audit--v0.2.0. Every pinned path verified to exist at its tag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 25, 2026
mmcky
added a commit
that referenced
this pull request
Aug 25, 2026
…ion (#54) * Docs: last single-plugin tidy — intro, section heading, site description Three spots #53's sweep missed: README still framed the repo as multiple plugins (intro sentence, auto-install line, contributing line), using-skills still headed its skill table "The plugins", and myst.yml's site description still advertised style checks — which are planned (#3), not shipped. Repo-level files only; nothing under qe/ changes, so no version bump. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Docs: Contributing covers docs PRs too, and names the catalogue path Copilot's review of #54 caught the rewritten Contributing line reading as if only plugin-directory changes count — docs-only PRs are equally valid — and "its marketplace.json entry" not saying the catalogue lives at .claude-plugin/marketplace.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #43. One plugin, one namespace:
/benchmark:review-accelerationbecomes/qe:benchmark,/audit:issuesbecomes/qe:audit-issues, and thebenchmarkandauditplugins retire. Every invocation now visibly reads as a QuantEcon skill, and the slash menu is one flat five-item list. Both procedures are unchanged — this is a move and rename, not a rewrite.What moved where
benchmark/skills/review-acceleration/qe/skills/benchmark/(frontmattername: benchmark)audit/skills/issues/qe/skills/audit-issues/(frontmattername: audit-issues)benchmark/scripts/{scoring,calibration}/qe/scripts/benchmark/audit/scripts/qe/scripts/audit/benchmark/references/+benchmark/README.mdqe/references/benchmark/(the README becomes the family guide there)audit/references/+audit/README.mdqe/references/audit/(same)benchmark/CHANGELOG.md,audit/CHANGELOG.mdqe/CHANGELOG.md, entries frozen as shippedVerification
Every
${CLAUDE_PLUGIN_ROOT}path, relative link, and usage string is updated, including the worked examples'run_all.pyengine lookup and its repo-layout fallback (one level deeper now). The scoring engine reproduces all three committed scorecards from the new layout with identical numbers (only the_notepath string changed), CI's reproduction step now runs fromqe/, a link checker confirms every relative markdown link underqe/,docs/and the root resolves, andscripts/validate.py,claude plugin validate --strict(plugin and marketplace), and the version-bump guard all pass — the guard explicitly recognises the two plugins as removed-from-manifest.Versioning
qe0.7.0 starts strictly above every retiring stream (qe 0.6.0, benchmark 0.4.0, audit 0.2.0), so no number in the merged changelog ever names two trees. The marketplace catalogue's own version moves 0.2.0 → 0.3.0 (plugins removed). Old tags (benchmark--v0.4.0,audit--v0.2.0, …) remain valid archaeology.Migration (also in the 0.7.0 changelog entry)
Installed users run
claude plugin uninstall benchmark@quantecon audit@quanteconor the retired skills linger under their old names; lecture repos dropbenchmark@quanteconfromenabledPlugins. The one deliberate trade, weighed in #43: the plugin is the enable unit, so every consumer now gets the audit skills too — two extra read-only entries in a five-item menu.Docs swept
README.md (plugins table → skills table, install blocks, CI example), CATALOG.md (per-family rows), docs/using-skills.md, docs/developing-skills.md (naming convention updated to the one-plugin reality), AGENTS.md (doc map, rubric pointers), myst.yml (site TOC now renders the guides from their new homes), and both tutorials.
🤖 Generated with Claude Code