Fix two checks that appeared to run and didn't: inert complexity budget, CI-skipped real-SDK tracing test - #112
Merged
Conversation
`[tool.ruff.lint.mccabe] max-complexity = 10` in pyproject.toml was inert. Ruff only evaluates C901 when the rule is selected, and this project keeps Ruff's default rule set (E4, E7, E9, F), which excludes it -- so `ruff check .` never read the value. Verified by control: re-running with `--config "lint.mccabe.max-complexity = 1"` against report.py, which contains a complexity-20 function, still reports clean. The budget that actually governs is `complexity_gate.DEFAULT_THRESHOLD`, and `collect_complexities()` invokes Ruff with `--isolated` precisely so the base snapshot -- written to a temp dir with no pyproject.toml -- is measured under the same config as the working tree. The gate therefore *cannot* read pyproject.toml by design. Two declarations of "10", one of them decorative: tightening the obvious one would have been a silent no-op. Remove the inert block and document the real owner from both sides. test_project_config_and_ci_enforce_the_regression_gate asserted the decorative key, which is what made the lie load-bearing -- deleting it turned the test red and told you to put it back. It now asserts the invariant instead: a mccabe budget may be declared only when C901 is actually selected. That permits either design -- enforce through Ruff, or leave the gate as sole owner -- and forbids only the misleading middle state. Confirmed non-vacuous against the old state. No behaviour change: the gate's threshold is unmoved at 10. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lled tests/test_observability.py::test_real_sdk_export_is_sanitized is guarded on `lmnr` being importable, and the `gate` job installs no optional extras -- so it skipped on every push and pull request while CI reported green. It is the only test exercising the real Laminar SDK's export sanitization rather than a fake, and the code it guards took 35 of the last 46 commits, many corrective (sanitizing SDK-derived context, isolating SDK system exits, preserving interrupts). The most-churned path had its only real-SDK check dark in CI. This was invisible locally because a working `.venv` generally has the extra installed, so the test runs on a developer machine and skips in CI -- the one arrangement where nobody sees the skip. Add a second `tracing` job rather than adding the extra to `gate`: the default install is 13 packages with zero required runtime dependencies, which is the repo's strongest supply-chain property and worth keeping intact. The new lane is the single place the opt-in is actually taken. The lane asserts `lmnr` is importable before running pytest, so a rename or a lock change fails loudly instead of quietly restoring the same green skips. Verified both lanes: default install (no lmnr) 747 passed / 4 skipped; with the extra 748 passed / 3 skipped and test_observability.py:99 absent from the skip list -- the real-SDK test runs. mypy --strict is clean under the extra too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the two High findings from the 20260811 tech-debt assessment. Both are the same shape: a check that appeared to run and didn't. Neither changes runtime behaviour.
1 — The complexity budget had two declarations, and the authoritative-looking one was inert
pyproject.tomldeclared[tool.ruff.lint.mccabe] max-complexity = 10. Ruff only evaluates C901 when the rule is selected, and this project keeps Ruff's default rule set (E4, E7, E9, F), which excludes it — soruff check .never read the value.Verified by control: re-running against
report.py, which contains a complexity-20 function, with--config "lint.mccabe.max-complexity = 1"still reports clean.The budget that actually governs is
complexity_gate.DEFAULT_THRESHOLD. The gate cannot readpyproject.tomlby design —collect_complexities()invokes Ruff with--isolatedprecisely so the base snapshot, written to a temp dir with nopyproject.toml, is measured under the same config as the working tree. Two declarations of "10", one decorative; tightening the obvious one would have been a silent no-op.A test was pinning the decoration.
test_project_config_and_ci_enforce_the_regression_gateasserted the inert key — under that name. Deleting the dead config turned the suite red and told you to put it back, which is how it survived in a repo this disciplined.The test now asserts the invariant instead: a mccabe budget may be declared only when C901 is actually selected. That permits either design — enforce through Ruff, or leave the gate as sole owner — and forbids only the misleading middle state. Confirmed non-vacuous by evaluating the new assertion against the old config state (it fails, as it should).
Why not just enable the rule?
extend-select = ["C90"]was the alternative. It fails CI immediately on 8 pre-existing functions insrc/toolbench— topped byreport.py:623 _render_summary(20) andcomplex_runner.py:209 _assert_deps_base_safe(17), and includingshell_safety.py:154 read_escapes(14), the safety-critical bash tokenizer. That is a refactor decision, not a side effect of a config cleanup, so this PR leaves it open. The gate's threshold is unmoved at 10.The assessment filed those 8 separately as a Medium finding, with a suggestion to record them as a dated baseline so the ratchet has a visible starting line.
2 — The only real-SDK tracing test never ran in CI
tests/test_observability.py::test_real_sdk_export_is_sanitizedis guarded onlmnrbeing importable. Thegatejob installs no optional extras, so it skipped on every push and pull request while CI reported green. It is the only test exercising the real Laminar SDK's export sanitization rather than a fake.The code it guards took 35 of the last 46 commits, many corrective — sanitizing SDK-derived context, isolating SDK system exits, preserving interrupts, disabling inherited debugger context. The most-churned and most security-relevant path had its only real-SDK check dark.
Why nobody noticed: a working
.venvgenerally has the extra installed, so the test runs on a developer machine and skips only in CI — the one arrangement where the skip is invisible. Confirmed during review: after a plainuv syncpruned the local venv to CI's 13 packages, the skip appeared locally for the first time.Adds a second
tracingjob rather than adding the extra togate. The default install is 13 packages with zero required runtime dependencies — the repo's strongest supply-chain property, worth keeping intact. The new lane is the single place the opt-in is actually taken.The lane asserts
lmnris importable before running pytest, so a rename or lock change fails loudly instead of quietly restoring the same green skips. A new test also asserts the lane exists inci.yml, matching the repo's existing convention of testing its own CI config.Verification
gatedefault install (no extra,lmnrabsent)tracinglane (extra installed)test_observability.py:99absent from the skip list, i.e. it ranruff check .mypy --strict src/toolbench testscomplexity_gate --base mainci.ymlpython -cjoins to one valid lineBoth lanes were run against real synced environments, not the developer venv.
Reviewer notes
tracingjob uses a shallow checkout deliberately — unlikegate, it runs no Git-base comparison, sofetch-depth: 0is unnecessary.${{ }}interpolation is introduced into anyrun:command; the new job's commands are entirely static.🤖 Generated with Claude Code