Problem
The audit in #172 (docs/cumulative-feature-ratchet-audit.md, PR #178) found that while total_churn and convention_bug_fix_count don't feed the default live composite score (hotspots-core/src/scoring.rs::compute_activity_risk), they do reach a live score through a path the original ratchet investigation didn't cover: hotspots-cli/src/cmd/analyze.rs::apply_trained_ranker overwrites activity_risk on every analyze run once a user has run hotspots train (i.e. .hotspots/ranker.json exists). Both features are monotonically non-decreasing over a function's lifetime — the same shape that made burst_score a one-way ratchet in the live formula (see burst-score-remove-from-live-score in the hotspots-research promotion tracker, PR #126/v1.33.2).
The trained-ranker case is murkier than the direct-formula case was: whether this actually manifests as a ratchet in practice depends on what the fitted model (XGBoost/RF) learned to do with those features, not just their raw monotonicity — a model could in principle learn to discount or saturate them.
Proposal
Investigate whether total_churn and convention_bug_fix_count cause the same "permanently elevated risk" behavior in the trained-ranker path that burst_score caused in the live formula. If confirmed, decide on a fix — likely bumping model_version, removing/bounding the features in the ranker's live feature vector (leaving cold-start/training-eval usage untouched, mirroring the burst_score precedent), and handling invalidation of existing trained models (.hotspots/ranker.json).
Why this wasn't done in #172/PR #178
Per CLAUDE.md, removing features from the trained ranker is a non-trivial, ambiguous change — it touches model_version, trainer.rs's feature vector, and many call sites, and the actual ratchet behavior needs to be empirically confirmed against the fitted model rather than assumed from feature shape alone. That audit documented this as a blocker rather than guessing at a fix.
Context
Follow-up from #172 (audit findings: docs/cumulative-feature-ratchet-audit.md, PR #178), which itself followed from the risk-stability review in #165 (docs/RISK_STABILITY.md).
Problem
The audit in #172 (docs/cumulative-feature-ratchet-audit.md, PR #178) found that while
total_churnandconvention_bug_fix_countdon't feed the default live composite score (hotspots-core/src/scoring.rs::compute_activity_risk), they do reach a live score through a path the original ratchet investigation didn't cover:hotspots-cli/src/cmd/analyze.rs::apply_trained_rankeroverwritesactivity_riskon everyanalyzerun once a user has runhotspots train(i.e..hotspots/ranker.jsonexists). Both features are monotonically non-decreasing over a function's lifetime — the same shape that madeburst_scorea one-way ratchet in the live formula (seeburst-score-remove-from-live-scorein the hotspots-research promotion tracker, PR #126/v1.33.2).The trained-ranker case is murkier than the direct-formula case was: whether this actually manifests as a ratchet in practice depends on what the fitted model (XGBoost/RF) learned to do with those features, not just their raw monotonicity — a model could in principle learn to discount or saturate them.
Proposal
Investigate whether
total_churnandconvention_bug_fix_countcause the same "permanently elevated risk" behavior in the trained-ranker path thatburst_scorecaused in the live formula. If confirmed, decide on a fix — likely bumpingmodel_version, removing/bounding the features in the ranker's live feature vector (leaving cold-start/training-eval usage untouched, mirroring theburst_scoreprecedent), and handling invalidation of existing trained models (.hotspots/ranker.json).Why this wasn't done in #172/PR #178
Per
CLAUDE.md, removing features from the trained ranker is a non-trivial, ambiguous change — it touchesmodel_version,trainer.rs's feature vector, and many call sites, and the actual ratchet behavior needs to be empirically confirmed against the fitted model rather than assumed from feature shape alone. That audit documented this as a blocker rather than guessing at a fix.Context
Follow-up from #172 (audit findings: docs/cumulative-feature-ratchet-audit.md, PR #178), which itself followed from the risk-stability review in #165 (docs/RISK_STABILITY.md).