Skip to content

feat: test a fix before it merges, and publish pass badges - #9

Merged
chrikrah merged 1 commit into
mainfrom
feat/with-override-and-badges
Oct 11, 2026
Merged

chrikrah merged 1 commit into
mainfrom
feat/with-override-and-badges

Conversation

@chrikrah

Copy link
Copy Markdown
Contributor

What this changes

wiretruth run --with SPEC replaces one package of a subject with any pip requirement, such as the head of an upstream pull request, and records the commit it resolved to. wiretruth report --badges DIR writes a shields.io endpoint per library, and the nightly run publishes them under badges/. No issue exists for this change.

How it was tested

$ uv run pytest -m "not slow" -q
134 passed, 2 deselected
$ uv run pytest -m slow -q          # both controls end to end
2 passed, 134 deselected in 102.86s
$ uv run ruff check src tests && uv run ruff format --check src tests
All checks passed!
25 files already formatted

# --with against traceloop/openllmetry#4377, pinned 0.62.4 as the baseline
$ uv run wiretruth run --subject openllmetry --with "opentelemetry-instrumentation-anthropic @ git+https://github.com/traceloop/openllmetry@refs/pull/4377/head#subdirectory=packages/opentelemetry-instrumentation-anthropic"
anthropic/stream, async-stream, stream-cache-hit, stream-cache-cumulative: wrong -> pass
versions: opentelemetry-instrumentation-anthropic = git+https://github.com/traceloop/openllmetry@7c3385734b7d708053236dac22dc4e1682d1ab25#subdirectory=...

# badges from results/2026-10-10-pinned.json
otel-official 31/31 pass, openinference 26/31 pass, openllmetry 18/31 pass

Checklist

  • The title is a Conventional Commit, for example feat(subjects): add logfire.
  • A test covers the change, and it fails when the change is reverted.
  • control-honest passes, and control-faulty fails exactly its expected checks.
  • Every new red cell is explained: the line of code that causes it, or a note that I could not find it.

This change adds no scenario and no check, so it adds no red cell.

  • CHANGELOG.md is updated under Unreleased.

wiretruth run --with SPEC replaces one package of a subject with any pip
requirement, such as the head of a pull request, and records the commit
it resolved to. The environment is reinstalled on every run, because a
pull request head moves.

wiretruth report --badges DIR writes a shields.io endpoint document per
library. The nightly run publishes them under badges/ from the run
against the latest releases.
@chrikrah
chrikrah merged commit 5e02a1a into main Oct 11, 2026
8 checks passed
@chrikrah
chrikrah deleted the feat/with-override-and-badges branch October 11, 2026 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant