Skip to content

feat(public-surface): add rule codes, a suggested bump, and gateable findings - #498

Merged
clay-good merged 5 commits into
mainfrom
feat/public-surface-rule-codes
Sep 13, 2026
Merged

clay-good merged 5 commits into
mainfrom
feat/public-surface-rule-codes

Conversation

@clay-good

@clay-good clay-good commented Sep 13, 2026

Copy link
Copy Markdown
Owner

Status

LGTM. CI is green on 91291e0ca, including Windows unit tests and Windows smoke. Four parallel adversarial reviews and three verification rounds found no gate bypass; every confirmed finding is fixed. The last verification round found no defect in the main behavior, and its two low-severity gaps (extra code extensions, an untested Bicep exclusion) are fixed with tests.

What was missing

  • certify_public_surface classified breaking changes with prose reasons only. There was no stable rule identity, so enforcement.policy could not gate "block export removals but not type narrowings", and the tool emitted no registered governance finding at all.
  • The verdict said breaking but not what version bump that implies.
  • The consumer disclosure claimed "Under federation, indexed sibling repos are also checked." The handler takes no federation input and never looks at sibling repositories.

What it does

Piece Behavior
Rule codes Every breaking or potentially-breaking change, and every added export, carries ruleCodes from a closed set: export-removed, export-renamed, export-visibility-reduced, export-added, param-removed, param-required-added, param-became-required, param-type-narrowed, return-type-narrowed, signature-unprovable.
Suggested bump suggestedBump: major when any change is breaking; withheld (null, with suggestedBumpWithheld) when any change is potentially-breaking or no signature-classifiable file was assessed; otherwise minor when an export was added, else patch.
Findings The eight breaking-classed codes (severity error) and signature-unprovable (severity warning) are registered in FINDING_CODE_REGISTRY (source public-surface, default advisory). publicSurfaceFindings emits one finding per such code per changed export in findings[], so the caller that runs the tool can gate a rule.
Honesty signature-unprovable is never breaking-classed. A bump is never offered on unproven evidence. The disclosure now says consumers are in-repo only, including under federation.
CLI openlore certify-public-surface --base <ref> prints the bump and each change's rule codes.

Round-1 review fixes

Finding Fix
suggestedBump: patch/minor on unproven evidence (potentially-breaking, or changed Go/Rust/Java/… files the classifier never reads) Bump withheld (null) with a reason
Removing a type annotation hid a narrowing from a policy (potentially-breaking emitted no finding) signature-unprovable is a registered warning finding
$$/$& in a path corrupted the remediation (String.replace patterns) Function replacement; test with a Remix-style $$ route
Findings repeated every reason a third time in the response The message names the rule; reasons stay on the change
A rename's finding pointed at the pre-change file Location uses the rename target file
Tests passed with codes missing (mutation run) Exact codes for every signature-unprovable path, export-renamed, name-level exports, parameter codes through the handler, a mixed breaking+unprovable change, and an exact finding snapshot
Spec and docs said every change carries a code; docs implied enforce gates it Wording matches the code; "the caller that runs the tool (not openlore enforce)"
Deferred scope was not discoverable Archived proposal notes the narrowing; new open change add-public-surface-acceptance-baseline in To-build

Round-2 verification fixes

Finding Fix
The handler only reads TS/JS/Python files, so a diff that changed only Go/Rust/… code reached the classifier empty and suggested patch Changed code files in an unclassified language withhold the bump with a reason; docs and config files do not; handler-level tests for a .go change and a docs-only change
A signature-unprovable finding said it "breaks" a rule It "triggers" the rule; breaking codes still "break"

Round-3 verification fixes

Finding Fix
Vue, Svelte, Astro, shell, .hxx, and .pyi changes still suggested patch Unassessed code is derived from the canonical language map (not TS/JS/Python) plus extensions it lacks; tests, Terraform, Bicep, and config files do not count
Renaming lib.go to lib.txt hid removed Go code A rename counts its old path too
The test-file exclusion and the untracked-file count had no test Tests for _test.go, Terraform, package.json, and an untracked .go file in a real git repository
Spec and docs described the older withhold rule Wording updated, plus a scenario for mixed diffs

Round-4 verification fixes

Finding Fix
36 code extensions the language map lacks (Cython .pyx/.pxd, .inl, CUDA, Groovy, Razor, ERB, Perl, PowerShell, …) still suggested patch Added to the extra code extensions; test for .pyx
The Bicep exclusion had no test Test that a changed .bicep file does not withhold the bump

Proof

  • src/core/analyzer/public-surface.test.ts: each signature rule fires its code; a potentially-breaking change never carries a breaking code; suggestedBump for breaking, additive, and potentially-breaking-only diffs.
  • src/core/services/mcp-handlers/public-surface.test.ts: codes for removed, visibility-reduced, added, and narrowed changes on file-content fixtures; a minor bump and no findings for an additive diff, and a withheld bump with one signature-unprovable warning finding for a potentially-breaking diff; one registered finding per breaking code, and per-rule gating (export-removed → blocking leaves param-type-narrowed advisory); the disclosure no longer claims sibling repos are checked.
  • Spec archived: 2026-09-13-refine-public-surface-certification (ADD PublicSurfaceRuleCodesAndSuggestedBump). openspec validate --specs --strict: 16/16.

Notes

  • Known classifier gaps that already exist on main (removed overloads, generator parameters, narrowed type aliases, Go parameter changes, and the return-type rule being backwards for callers) are filed as a separate task; the bump is withheld where the classifier cannot assess a change.

  • Narrowed from the proposal. Deferred: the --accept baseline with required justifications and decision anchoring, the breaking-consumed / breaking-unconsumed-in-index split with the federation consumer union, and running the tool inside openlore enforce (it needs a base ref).

  • The MCP tool description and tools/list payload are unchanged; the response grows by ruleCodes per change, suggestedBump, and findings.

🤖 Generated with Claude Code

clay-good and others added 5 commits September 13, 2026 05:45
…findings

certify_public_surface diff verdicts now carry stable rule codes per
change (export-removed, param-type-narrowed, ..., signature-unprovable,
export-added) and a suggestedBump (major/minor/patch). The eight
breaking-classed codes are registered findings (source public-surface)
emitted in findings[], so an enforcement.policy can gate individual
rules; signature-unprovable is never breaking-classed. The false
'sibling repos are also checked' disclosure now says in-repo only.
The accept baseline and the consumed/unconsumed split are deferred.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round-1 review fixes:
- withhold suggestedBump (null, with a reason) when any change is
  potentially-breaking or no classifiable-language file was assessed
- register signature-unprovable as a warning finding, so removing a
  type annotation cannot hide a narrowing from a policy
- remediation uses a function replacement ($$ paths stay intact);
  finding messages name the rule instead of repeating every reason
- a rename's finding points at the post-change file
- tests: exact codes per rule, export-renamed, name-level exports,
  parameter codes through the handler, mixed breaking+unprovable
- spec/docs match the code; CLI types findings; archived proposal notes
  the narrowing; deferred parts move to add-public-surface-acceptance-baseline

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ified

Round-2 verification fixes:
- the handler only reads TS/JS/Python files, so a diff that changed only
  Go/Rust/Java/... code reached the classifier empty and suggested patch;
  changed code files in an unclassified language now withhold the bump
  (docs and config files do not)
- a signature-unprovable finding says it triggers a rule, not breaks it

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round-3 verification fixes:
- a changed code file counts as unassessed when the canonical language
  map knows its language and it is not TS/JS/Python (Vue, Svelte, shell,
  PHP, ...), plus extensions the map lacks (.pyi, .hxx, .mm, ...);
  tests, Terraform, Bicep, and config files do not count
- a rename counts its old path too (lib.go -> lib.txt)
- tests for Vue, .pyi, shell, rename, _test.go, Terraform, config, and an
  untracked .go file; spec and docs wording plus a scenario

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Round-4 verification fixes: Cython (.pyx/.pxd), C++ template and CUDA
headers, Groovy, Razor, ERB, CoffeeScript, Perl, PowerShell, Zsh/Fish,
batch, Solidity, and other code extensions the language map lacks now
withhold the bump; tests for .pyx and the Bicep exclusion.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@clay-good
clay-good merged commit 92cad32 into main Sep 13, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant