Skip to content

Trained ranker's total_churn/convention_bug_fix_count share burst_score's ratchet risk #182

Description

@stephenc222

Problem

The audit in #172 (docs/cumulative-feature-ratchet-audit.md, PR #178) found that while total_churn and convention_bug_fix_count don't feed the default live composite score (hotspots-core/src/scoring.rs::compute_activity_risk), they do reach a live score through a path the original ratchet investigation didn't cover: hotspots-cli/src/cmd/analyze.rs::apply_trained_ranker overwrites activity_risk on every analyze run once a user has run hotspots train (i.e. .hotspots/ranker.json exists). Both features are monotonically non-decreasing over a function's lifetime — the same shape that made burst_score a one-way ratchet in the live formula (see burst-score-remove-from-live-score in the hotspots-research promotion tracker, PR #126/v1.33.2).

The trained-ranker case is murkier than the direct-formula case was: whether this actually manifests as a ratchet in practice depends on what the fitted model (XGBoost/RF) learned to do with those features, not just their raw monotonicity — a model could in principle learn to discount or saturate them.

Proposal

Investigate whether total_churn and convention_bug_fix_count cause the same "permanently elevated risk" behavior in the trained-ranker path that burst_score caused in the live formula. If confirmed, decide on a fix — likely bumping model_version, removing/bounding the features in the ranker's live feature vector (leaving cold-start/training-eval usage untouched, mirroring the burst_score precedent), and handling invalidation of existing trained models (.hotspots/ranker.json).

Why this wasn't done in #172/PR #178

Per CLAUDE.md, removing features from the trained ranker is a non-trivial, ambiguous change — it touches model_version, trainer.rs's feature vector, and many call sites, and the actual ratchet behavior needs to be empirically confirmed against the fitted model rather than assumed from feature shape alone. That audit documented this as a blocker rather than guessing at a fix.

Context

Follow-up from #172 (audit findings: docs/cumulative-feature-ratchet-audit.md, PR #178), which itself followed from the risk-stability review in #165 (docs/RISK_STABILITY.md).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions