Skip to content

fix(migration-keeper): preserve an active migration snapshot under alternating candidate rates - #705

Open
ZacLou wants to merge 1 commit into
drydocs:mainfrom
ZacLou:fix/migration-keeper-snapshot-starvation-699
Open

fix(migration-keeper): preserve an active migration snapshot under alternating candidate rates#705
ZacLou wants to merge 1 commit into
drydocs:mainfrom
ZacLou:fix/migration-keeper-snapshot-starvation-699

Conversation

@ZacLou

@ZacLou ZacLou commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

The migration keeper re-derives the best-rate candidate from scratch on every scheduled run with no memory of an already-begun migration. When the top candidate flips between two adapters across consecutive runs, each run overwrites the prior begin_migration snapshot and resets the ledger-gap cooldown, so a migration can in principle never reach migrate_adapter despite a real, sustained improvement opportunity existing throughout.

Closes #699

Problem

  1. Run 1: candidate A clears minImprovementBps over the current adapter's rate. Keeper calls begin_migration(A), snapshotting A.
  2. Before run 2, rates shift so candidate B is now best.
  3. Run 2 reads get_migration_snapshot(), sees adapter = A != best.adapterId = B, and calls begin_migration(B), overwriting A's snapshot and resetting the cooldown.
  4. If rates keep alternating between A and B across runs, no single candidate's snapshot survives long enough to reach MIN_LEDGER_GAP with a matching "best" on a later run.

Fix

Before falling through to begin_migration for the freshly-derived best candidate, check whether an active migration snapshot already exists for a different adapter. If it does and that adapter still clears minImprovementBps against the current rate, override best to the snapshotted adapter so the existing migrate_adapter path completes it instead of resetting the cooldown. Only let the snapshot lapse (and pick a new candidate) once the snapshotted adapter's rate genuinely stops clearing the threshold.

Key design decisions

  • Snapshot rate re-verified on each run: the snapshot's adapter rate is queried fresh (not cached from the begin_migration run), so a stale snapshot whose adapter has genuinely decayed is correctly replaced.
  • Concurrent rate lookups: the snapshot adapter's rate and the current rate are queried via Promise.all + withKeeperRetry, matching the existing candidate evaluation pattern.
  • Graceful fallback: if the snapshot's rate lookup fails transiently, the keeper falls through to begin_migration for the fresh best rather than blocking the migration entirely — strictly better for liveness.
  • No change to fund safety: migrate_adapter's contract-side slippage/stability checks are unchanged; this only affects which candidate the keeper tries to complete.

Testing

Two new test cases covering both halves of the fix:

  1. "preserves an active migration snapshot whose adapter still clears the improvement threshold (Migration keeper can starve a completable migration under alternating candidate rates #699)" — snapshot is for adapter B (rate 650), best is adapter A (rate 700), current is 500. The keeper migrates to B (not A), preserving the existing cooldown.
  2. "replaces a stale migration snapshot whose adapter no longer clears the threshold (Migration keeper can starve a completable migration under alternating candidate rates #699)" — snapshot is for adapter B (rate 510, below threshold), best is adapter A (rate 700). The keeper submits begin_migration for A, correctly replacing the stale snapshot.

All 54 existing + new tests pass.

Scope

Field Value
Area SDK (migration keeper)
Protocol affected None
Network Both
Breaking change? No

@vercel

vercel Bot commented Sep 3, 2026

Copy link
Copy Markdown

@ZacLou is attempting to deploy a commit to the Collins' projects Team on Vercel.

A member of the Team first needs to authorize it.

The migration keeper re-derives the best-rate candidate from scratch on
every scheduled run with no memory of an already-begun migration. When
the top candidate flips between two adapters across consecutive runs,
each run overwrites the prior begin_migration snapshot and resets the
ledger-gap cooldown, so a migration can in principle never reach
migrate_adapter despite a real, sustained improvement opportunity.

Fix: before falling through to begin_migration for the freshly-derived
best candidate, check whether an active migration snapshot already
exists for a different adapter. If it does and that adapter still
clears minImprovementBps against the current rate, override best to
the snapshotted adapter so the existing migrate_adapter path completes
it instead of resetting the cooldown. Only let the snapshot lapse when
its adapter's rate genuinely stops clearing the threshold.

Two new tests cover both halves: preservation when the snapshot still
qualifies, and replacement when it has genuinely decayed.
@ZacLou
ZacLou force-pushed the fix/migration-keeper-snapshot-starvation-699 branch from 52ffa12 to ccfd342 Compare September 3, 2026 08:21
// migrate_adapter. Only let the snapshot lapse (and pick a new
// candidate) once the snapshotted adapter's rate genuinely stops
// clearing the threshold.
if (

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The #699 fix is bolted on as a second, separate re-evaluation pass after findBestCandidate already returned, rather than making findBestCandidate itself snapshot-aware, e.g. by passing the existing snapshot's adapter in so it's evaluated alongside the other candidates in the same concurrent batch. As written, every future case needing to reason about "the current best vs. an existing on-chain commitment" will be tempted to bolt on another special-cased post-pass with its own duplicated rate fetch and threshold check (see the two comments below) instead of reusing one generalized candidate-evaluation path.

// of this check. Each is retried individually via
// withKeeperRetry, same as findBestCandidate's own candidate
// evaluation.
const [snapshotRateResult, currentRateResult] = await Promise.all([

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currentRateResult refetches the vault's current-adapter rate via a second retried rateSource/withKeeperRetry call, even though findBestCandidate already fetched and validated that exact rate moments earlier in the same run. On every run where an active snapshot exists for a different adapter than the freshly-derived best, the common steady-state case this PR fixes, this adds an extra full-retry RPC round trip purely to re-derive a rate the run already has, without changing the outcome when the rate hasn't moved.

if (
isUsableRate(snapshotRate) &&
isUsableRate(currentRate) &&
snapshotRate - currentRate >= config.minImprovementBps

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This improvement-threshold comparison duplicates the identical comparison already implemented inside findBestCandidate's candidate loop instead of extracting a shared helper. A future change to the threshold semantics, e.g. switching to a percentage-based or rounding-aware comparison, only updates one of the two inline copies, silently making the snapshot-preservation path and the fresh-candidate path disagree about what counts as a clearing improvement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migration keeper can starve a completable migration under alternating candidate rates

2 participants