Skip to content

feat(adr): forbid embedded raw benchmark/spike data in ADR bodies (T7, #1925) - #2020

Merged
FabioLeitao merged 8 commits into
mainfrom
feat/adr-1925-no-embedded-benchmark
Sep 29, 2026
Merged

FabioLeitao merged 8 commits into
mainfrom
feat/adr-1925-no-embedded-benchmark

Conversation

@FabioLeitao

@FabioLeitao FabioLeitao commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

What changes

  • ADR-0045 — new Decision item 9 (no embedded raw test/benchmark/spike data); Status history append-only line Amended for [P0][U1] adr(0045): proibir dado bruto de teste/benchmark no corpo de ADR — referenciar, nunca colar #1925.
  • ADR-0000 — one sentence in the Sections table pointing to item 9.
  • .cursor/rules/adr-trigger.mdc — bullet: link evidence, do not paste tables/logs/Operator decision (YYYY-MM-DD).
  • Detector T7 — tests/adr_governance_support.py (embedded_experimental_data_violations).
  • Tests — tests/test_adr_governance_phase1.py (T7 corpus, staged + lines, synthetic fixtures, auditor probe table).
  • Plan — T7 row in docs/plans/PLAN_ADR_GOVERNANCE_ENFORCEMENT.md.
  • docs/adr/INVENTORY.txt — regenerated via inv-adr.ps1 (body hash drift on ADR-0045).

What the detector flags

  • A line Operator decision (YYYY-MM-DD) with optional Markdown prefixes (bullet, blockquote, bold).
  • A Markdown table with three or more data rows whose cells look like measurement datasets (ms, µs/us, ×, x slower, req/s, MB/s).
  • A fenced code block containing test-runner output (PASSED, FAILED, or pytest-style N passed / passed in Ns).

What it does not flag

  • Prose numbers with a pinned artifact path or link (ADR-0078 style).
  • status: failed inside a YAML sample block (no case-insensitive bare failed match).
  • Option tables with % coverage splits or timeout tables with 5 s / 30 s (generic % / bare s are out of scope).

Transparency (required)

  • The detector was run against all 92 real docs/adr/ADR-*.md files.
  • ADR-0036 still contains a three-row microbenchmark table (µs/call) that the table heuristic would flag.
  • The full-corpus test intentionally runs with check_tables=False, so legacy Accepted ADRs are not retroactively charged; the table rule applies only to added lines in new or amended ADRs (pre-commit staged diff).
  • There is no grandfather.json.

Fixtures

  • Synthetic good/bad excerpts under tests/fixtures/adr_t7_* — no text copied from private repos.

Ratification

  • INVENTORY.txt was regenerated without -RatifiedBy; ADR-0045 SSHSIG ratification remains pending operator per ADR-0056.

Test plan

  • ./scripts/check-all.sh exit 0

Closes #1925


Note

Low Risk
Documentation and pre-commit governance tests only; no runtime, auth, or data-path changes.

Overview
Adds ADR governance T7 (#1925): ADRs must record decisions in prose and link versioned evidence instead of embedding raw spikes, benchmark tables, or test-runner logs.

ADR-0045 gains Decision item 9 (forbidden shapes + allowed pinned paths) with a Status history amendment; ADR-0000 and .cursor/rules/adr-trigger.mdc tell agents to follow the same rule when materializing ADRs. embedded_experimental_data_violations in tests/adr_governance_support.py detects Operator decision (YYYY-MM-DD) lines, fenced pytest-style output, and (on staged + lines only) markdown tables with three or more measurement-like rows; the full-corpus check uses check_tables=False so legacy Accepted ADRs are not retroactively blocked. Phase 1 tests and synthetic good/bad fixtures exercise the detector; the enforcement plan and regenerated INVENTORY.txt reflect the ADR body edits.

Reviewed by Cursor Bugbot for commit 07e020b. Bugbot is set up for automated code reviews on this repo. Configure here.

Incident-shaped heuristics, synthetic good/bad fixtures, staged + lines scan, corpus must stay clean.
Incident-shaped heuristics, synthetic good/bad fixtures, staged + lines scan, corpus must stay clean.
Markdown-prefixed operator decision lines; case-sensitive runner tokens; tighten measurement cells; probe table parametrized.
ADR-0000 pointer, adr-trigger bullet, PLAN_ADR_GOVERNANCE_ENFORCEMENT, inventory regen; clarify staged table scope.
@FabioLeitao

Copy link
Copy Markdown
Collaborator Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 07e020b. Configure here.

measurement_rows = sum(
1 for row in data_rows if BENCHMARK_TABLE_CELL_RE.search(row)
)
return measurement_rows >= 3

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Staged table scan drops data rows

Medium Severity

_table_data_rows always discards the first two pipe-lines as header and separator. The staged T7 gate then runs that parser on concatenated + lines only, so adding or replacing three measurement rows on an existing table never meets the 3-row threshold and is not flagged.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 07e020b. Configure here.

r"\d[\d.,]*\s*(?:ms|µs|us|sec|×|x\s*slower|req/s|MB/s)"
r"|(?:\d+\.\d+|\d+)\s*×",
re.IGNORECASE,
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unit regex matches English prefixes

Low Severity

BENCHMARK_TABLE_CELL_RE matches sec, ms, and us without a trailing boundary, so ordinary words such as seconds, messages, or users after a number count as measurement cells. A timeout or options table written in English can then fail the T7 gate.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 07e020b. Configure here.

Regenerate via inv-adr.ps1 after item 9 staged-scope edit (no -RatifiedBy).
Only skip header+separator when line 2 is a --- row; staged + chunks may be data-only.
Require digit before measurement units; (?:ms|µs|us|sec) not followed by letters; extend parametrized probes.
@FabioLeitao

Copy link
Copy Markdown
Collaborator Author

Corrigidos: inventário do ADR-0045 (5dd0d3a), tabelas parciais nas linhas adicionadas (646c4a3, bugbot média) e fronteira das unidades da regex (af58fae, bugbot baixa).

@FabioLeitao
FabioLeitao merged commit 9987dbb into main Sep 29, 2026
28 checks passed
@FabioLeitao
FabioLeitao deleted the feat/adr-1925-no-embedded-benchmark branch September 29, 2026 01:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P0][U1] adr(0045): proibir dado bruto de teste/benchmark no corpo de ADR — referenciar, nunca colar

1 participant