Skip to content

Package metric regression gates in CI #8

Description

@jflowers

Summary

Add CI gates that detect regression in package-level coupling metrics between the base branch and the PR, failing the build when structural quality degrades beyond tolerance.

RFC: https://github.com/orgs/unbound-force/discussions/483 — Group 2d
Phase: P2 (Accessibility)

Context

Once vibe-check analyze produces package-level metrics, we need CI enforcement that prevents PRs from degrading those metrics. This adds regression detection as a first-class vibe-check capability.

Implementation

Regression Detection

  • Run vibe-check analyze on both the base branch and the PR branch
  • Compare per-package metrics: Instability (I), Distance (D), LCOM, Circular Deps
  • Fail if any metric regresses beyond a configurable tolerance

CLI Flags

vibe-check analyze ./... \
  --regression-base=origin/main \
  --max-instability-delta=0.05 \
  --max-distance-delta=0.05 \
  --no-new-circular-deps

Output

  • Table of packages with metric deltas
  • Clear indication of which packages regressed and by how much
  • JSON output with regression section for programmatic consumption

Acceptance Criteria

  • --regression-base flag compares metrics between branches
  • Configurable delta thresholds for instability, distance, cohesion
  • --no-new-circular-deps flag detects newly introduced cycles
  • Clear CI output showing regression details
  • JSON output includes regression section conforming to metrics.ModuleGraph schema
  • Non-zero exit code on regression beyond tolerance
  • Integrated into CI workflow

Dependencies

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High priority — Phase 1–2

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions