Skip to content

fix(#456): cover remaining legacy backing migration edge cases - #458

Merged
dcccrypto merged 1 commit into
dcccrypto:mainfrom
Bayyan16:fix/456-postmerge-ledger-migration-edge-cases
Sep 4, 2026
Merged

fix(#456): cover remaining legacy backing migration edge cases#458
dcccrypto merged 1 commit into
dcccrypto:mainfrom
Bayyan16:fix/456-postmerge-ledger-migration-edge-cases

Conversation

@Bayyan16

@Bayyan16 Bayyan16 commented Sep 3, 2026

Copy link
Copy Markdown

Summary

Follow-up to #456.

This PR covers two remaining legacy backing-ledger migration edge cases identified during post-merge validation of #456.

It does not replace, reopen, or redesign the main #433 / #456 fix. The canonical backing-ledger creation and reconciliation path introduced by #456 remains intact.

This follow-up is intentionally limited to two migration cases:

  1. a legacy ledgerless backing domain may already contain outstanding utilization_fee_earnings when its canonical BackingDomainLedger is first materialized;
  2. a fully wound-down Resolved market may still require a zero-capital reconciliation pass to materialize a missing canonical backing ledger before the remaining legacy backing can be withdrawn.

Both cases were reproduced against the clean merged #456 baseline before applying this follow-up.


Context

PR #456 added migration-specific reconciliation for backing domains that were funded before their canonical BackingDomainLedger existed.

That migration correctly reconstructs historical principal and impairment from the existing backing bucket while preserving the generic zero-principal semantics of:

read_or_new_backing_domain_ledger(...)

for unrelated callers such as LP-vault accounting.

Post-merge validation exposed two remaining lifecycle boundaries:

  • outstanding backing earnings that existed before ledger creation were not represented in the reconstructed ledger baseline;
  • the zero-capital migration path could no longer be reached after a market had entered the fully wound-down Resolved state.

These are narrow follow-up cases around the migration lifecycle introduced by #456, not changes to its core accounting model.


1. Preserve Outstanding Pre-Ledger Backing Earnings

Problem

A legacy ledgerless backing bucket may already contain non-zero:

utilization_fee_earnings

before its canonical ledger exists.

The #456 migration reconstructed historical principal and unavailable principal, but an already-existing earnings balance was not part of that reconstructed baseline.

At the same time, the newly materialized ledger could begin with its earnings observation watermark already aligned to the bucket's current earnings value.

That could produce a state equivalent to:

bucket.utilization_fee_earnings > 0

ledger.total_earnings_atoms = 0
ledger.last_observed_bucket_earnings_atoms = bucket.utilization_fee_earnings

A subsequent normal sync would then observe no new earnings delta.

The pre-ledger earnings would remain present in the bucket, but would never have been represented in total_earnings_atoms.

Fix

The migration-specific baseline now snapshots outstanding provider earnings from the same bucket state used to reconstruct the rest of the legacy accounting baseline:

ledger.total_earnings_atoms = bucket.utilization_fee_earnings;
ledger.last_observed_bucket_earnings_atoms =
    bucket.utilization_fee_earnings;

This behavior is deliberately contained inside:

seed_legacy_backing_domain_ledger(...)

rather than changing the generic semantics of:

read_or_new_backing_domain_ledger(...)

The reconstructed baseline therefore starts from one coherent snapshot containing:

historical principal
historical unavailable principal
historical outstanding backing earnings
matching observation watermarks

This prevents outstanding pre-ledger earnings from being skipped by an already-advanced watermark and prevents the first normal sync from re-booking the migrated snapshot.


2. Keep Zero-Capital Migration Reachable After Full Resolution

Problem

The migration path introduced by #456 still passed through the normal Live-only TopUpBackingBucket lifecycle gate.

A legacy domain could therefore reach:

existing legacy backing
+ missing canonical BackingDomainLedger
+ market already Resolved

with no remaining path to materialize the canonical ledger.

This matters because the canonical backing ledger is now authoritative for the backing-withdrawal path.

A fully wound-down market could still contain legitimate backing that predates its ledger, while reconciliation itself was no longer reachable.

The required operation in this state is not another backing deposit.

It is an accounting-only migration with:

amount == 0

Fix

TopUpBackingBucket now distinguishes normal backing funding from the narrow migration-only terminal case.

For normal Live markets, existing behavior remains unchanged and continues through:

require_domain_accepts_live_topup_view(...)

For a Resolved market, zero-capital reconciliation is allowed only when all of the following are true:

mode == Resolved
amount == 0
materialized_portfolio_count == 0
c_tot == 0

This represents an already wound-down market with no remaining materialized user portfolios or user capital.

The amount == 0 requirement makes the operation accounting-only.

Non-zero backing deposits after resolution remain rejected.

In particular:

Resolved + amount > 0

still fails closed.


Safety Boundaries

The resolved migration exception is intentionally narrow.

It does not:

  • reopen normal backing deposits after resolution;
  • introduce new provider capital;
  • increase the engine backing bucket through the migration exception;
  • recreate user exposure;
  • reopen trading;
  • alter backing-withdrawal authorization;
  • change LP-vault accounting semantics;
  • change generic read_or_new_backing_domain_ledger(...) behavior.

Migration is allowed only when:

amount == 0

and the market is already fully wound down:

materialized_portfolio_count == 0
c_tot == 0

Its only purpose is to materialize canonical accounting for backing that already existed before the ledger became mandatory.


Accounting Invariants

The follow-up keeps the accounting rules established by #456 intact.

Principal and impairment remain seeded together

Historical gross principal is reconstructed from:

fresh_unliened
+ valid_liened
+ consumed_liened
+ impaired_liened

while historical unavailable principal is reflected in the migration loss baseline.

This preserves:

available = principal - (loss - recovery)

and avoids making previously consumed or impaired backing appear newly withdrawable.

Migration watermarks remain pinned to the reconstructed snapshot

The unavailable-principal watermark remains aligned with the same migration snapshot so the first normal sync does not re-book historical impairment as a newly observed loss.

The earnings watermark now follows the same rule for outstanding pre-ledger earnings.

Refill remains distinct from recovery

The #456 re-baselining behavior after top-up remains intact.

A top-up that satisfies provider receivable may reduce consumed backing, but that new capital is not treated as recovery of historical impairment.

Generic ledger initialization remains unchanged

The migration-specific reconstruction remains isolated from callers for which an uninitialized backing ledger legitimately means the vault has never funded the domain.


Regression Coverage

Two targeted regressions cover the remaining post-merge cases.

Outstanding backing earnings

v16_bpf_legacy_ledgerless_migration_seeds_outstanding_backing_earnings

The fixture recreates a historically funded ledgerless domain with:

historical principal      = 100
historical unavailable    = 40
outstanding earnings      = 30
canonical ledger          = absent

The test verifies that migration materializes the canonical ledger with the expected historical accounting baseline, including the pre-existing earnings balance and matching earnings watermark.

It then exercises an earnings withdrawal and verifies that the ledger, backing bucket, watermark, vault balance, and withdrawn accounting remain coherent.


Fully resolved migration

v16_bpf_legacy_ledgerless_resolved_zero_topup_reconciles_without_reopening_deposits

The fixture recreates:

legacy backing            = 100
canonical ledger          = absent
market                     = Resolved
materialized portfolios   = 0
c_tot                      = 0

The regression first exercises the negative control:

Resolved + non-zero top-up

and verifies that it remains rejected.

The rejected attempt does not:

  • consume source tokens;
  • modify the vault balance;
  • leave a canonical ledger PDA behind.

The test then executes:

Resolved + amount == 0

and verifies that canonical-ledger migration succeeds without introducing new capital.

The resulting ledger reflects the historical backing baseline rather than a newly fabricated deposit.

Existing legacy backing can then proceed through the canonical withdrawal path.


Baseline Reproduction

Both new regressions were reproduced against the clean merged #456 baseline before the follow-up was applied.

Baseline:

be8c8dd2

Before the follow-up:

v16_bpf_legacy_ledgerless_migration_seeds_outstanding_backing_earnings
FAIL

v16_bpf_legacy_ledgerless_resolved_zero_topup_reconciles_without_reopening_deposits
FAIL

With this follow-up:

v16_bpf_legacy_ledgerless_migration_seeds_outstanding_backing_earnings
PASS

v16_bpf_legacy_ledgerless_resolved_zero_topup_reconciles_without_reopening_deposits
PASS

Existing #456 migration regressions remain green, including:

v16_bpf_backing_topup_then_withdraw_works_without_an_lp_vault
PASS

v16_bpf_legacy_ledgerless_backing_zero_topup_reconciles_before_withdraw
PASS

v16_bpf_legacy_ledgerless_nonzero_topup_does_not_book_refill_as_recovery
PASS

Full v16_cu Verification

Full v16_cu on the follow-up branch:

107 passed
1 failed

The remaining failure is:

v16_bpf_stale_full_14_leg_tradenocpi_is_under_tx_limit

The failure reaches the existing transaction compute-unit ceiling:

ProgramFailedToComplete
exceeded CUs meter at BPF instruction
compute_units_consumed: 1400000

For comparison, the same test was rebuilt and run independently against the clean merged #456 baseline (be8c8dd2).

It fails there with the same failure class and the same 1,400,000-CU ceiling.

The remaining failure is therefore baseline-equivalent and is not introduced by this follow-up.

No changes to the unrelated TradeNoCPI / compute-budget path are included here.


Relationship to #456

This PR is intentionally a post-merge follow-up.

#456 remains the primary reconciliation fix.

This follow-up only completes two remaining boundaries around it:

#456
  |
  +-- create/reconcile canonical backing ledger
  |
  +-- reconstruct historical principal and impairment
  |
  +-- preserve refill vs. recovery semantics
  |
  +-- follow-up:
        |
        +-- preserve outstanding pre-ledger earnings
        |
        +-- keep zero-capital migration reachable
            after full resolution

The core #456 design remains unchanged.


Scope

Production code:

src/v16_program.rs

Regression coverage:

tests/v16_cu.rs

Final diff against upstream/main:

src/v16_program.rs | 36
tests/v16_cu.rs    | 308

2 files changed
341 insertions(+)
3 deletions(-)

git diff --check upstream/main...HEAD is clean.

No unrelated source files are included.


No ABI / Layout Change

This follow-up does not change:

  • instruction tags;
  • instruction account ordering;
  • PDA derivation;
  • BackingDomainLedger layout;
  • wrapper or engine account layouts;
  • token-program wiring;
  • WithdrawBackingBucket ABI;
  • LP-vault accounting semantics;
  • matcher paths;
  • trade paths;
  • generic read_or_new_backing_domain_ledger(...) semantics;
  • compute-unit limits.

The production change is limited to migration-baseline initialization and the migration-specific lifecycle gate.


Branch State

Branch:

fix/456-postmerge-ledger-migration-edge-cases

Head:

b3541b6

Commit:

fix(#456): cover legacy backing migration edge cases

The working tree is clean and the branch contains only:

src/v16_program.rs
tests/v16_cu.rs

relative to upstream/main.


Result

After this follow-up:

legacy ledgerless backing
+ outstanding historical earnings
    -> migration preserves the earnings baseline

legacy ledgerless backing
+ fully wound-down Resolved market
    -> zero-capital reconciliation remains reachable

Resolved market
+ non-zero backing deposit
    -> still rejected

This completes the remaining post-merge migration edge cases discovered while validating #456 without widening normal backing-deposit behavior or changing the core reconciliation design.

@coderabbitai

coderabbitai Bot commented Sep 3, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 36c69e92-d023-4861-ac6a-8e846abb2273


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@dcccrypto

Copy link
Copy Markdown
Owner

Your PR is not the cause of this red check. Clearing that up before anything else, since the failure looks damning and isn't yours.

build + test reports nine tag87_* failures. Your diff against the merge base is src/v16_program.rs (+36/−3) and tests/v16_cu.rs (+308) — it touches neither tests/v16_fee_split.rs, where those nine live, nor ci/deployed-refs.env, nor tests/KNOWN_FAILING.txt.

main fails on the same nine, and has since 15eb8b0c on 2026-08-31 — three days and fourteen commits before yours. Your merge base be8c8dd2 is itself red.

The cause is in CI, not in anyone's PR: devnet isn't a default feature, and without it tag 87 compiles to an unconditional return Err(StakeProgramNotPinned)Custom(60). CI built the wrapper --no-default-features, so those tests failed before reaching anything they assert. Fix is in #459, which is one line of scripts/ci-test.sh.

Once #459 lands, please rebase (or I'll re-run it) and the check should reflect your actual change.

On the substance of #458 itself: it's a program src/ change, which is a hard stop here for a human's explicit go regardless of how clean the review is, so it'll wait on that — not on anything you need to fix. Thanks for the follow-up work on the #456 edge cases, and sorry for the misleading signal.

dcccrypto added a commit that referenced this pull request Sep 3, 2026
… red since 15eb8b0 (#459)

`main` has failed CI for three days, from 15eb8b0 (2026-08-31 13:31) through
be8c8dd, on nine tag87_* tests. Fourteen commits have landed on a red trunk, and
contributor PR #458 inherited the red and looks like it caused it.

It did not. Neither did any of the fourteen.

ROOT CAUSE

`devnet` is not a default feature — `default = ["anchor-v2"]`. Without it, tag 87
(WithdrawInsuranceReserveToStake) compiles to an unconditional

    #[cfg(not(feature = "devnet"))]
    { return Err(PercolatorError::StakeProgramNotPinned.into()); }   // Custom(60)

because a non-devnet build has no pinned percolator-stake id to send tokens to.
v17 percolator-stake has no mainnet deployment, so this is correct fail-closed
behaviour of the program — not a bug in it.

CI built the wrapper `--no-default-features`. So every tag87_* test failed at
Custom(60) before reaching anything it asserts about, regardless of the stake pin.

#441 removed those nine from tests/KNOWN_FAILING.txt as "they now pass". That was
true where it was measured: a local `cargo build-sbf` takes the default features,
and anyone testing tag 87 builds devnet. It could never be true in CI, and those
nine had never once passed here.

Note this is NOT the failure deployed-refs.env predicts for a wrapper/stake pin
skew ("Custom(9) at tag 87"). It is Custom(60), and it is independent of the pins:
the pins were advanced correctly by 78c2005 and main stayed red.

WHY devnet IS THE RIGHT BUILD, not an allowlist restore

The suite's sibling pins are all deployed-DEVNET refs, and the deployed devnet
wrapper is itself built `--features devnet` — that is what reproduces DhSkE7u
byte-for-byte. A non-default build tests a configuration deployed nowhere.

It also meant the entire tag-87 security surface was never exercised, including
`tag87_blocks_the_creator_forged_stake_pool_exploit` and
`tag87_owner_pin_fires_before_any_pool_byte_is_read` — a test that exists precisely
to prove the owner pin fires before any pool byte is read. Restoring the allowlist
entries would have gone green while keeping that blind.

MEASURED on be8c8dd, siblings at the pinned refs (stake d0c6ecb, nft 215842e,
matcher d4d4f1c):

    --no-default-features   passed=596 failed=30   9 NEW failures vs the allowlist
    --features devnet       passed=603 failed=21   exact match: 0 new, 0 stale

`./scripts/ci-test.sh` now exits 0 with its own verdict:
"OK: failing set matches the allowlist exactly".

Upstream aeyakovenko/percolator-prog carries neither scripts/ci-test.sh nor
tests/KNOWN_FAILING.txt — both are ours, so there is nothing to coordinate.

No test is edited, skipped or re-scoped, and no program source changes.


Claude-Session: https://claude.ai/code/session_01NgoNgagkvw7i5SSRC3FJ8D

Co-authored-by: dcccrypto <dcccrypto@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@Bayyan16
Bayyan16 force-pushed the fix/456-postmerge-ledger-migration-edge-cases branch from b3541b6 to 9ef25f9 Compare September 3, 2026 11:38
@Bayyan16

Bayyan16 commented Sep 3, 2026

Copy link
Copy Markdown
Author

Rebased onto main after #459 and force-pushed the updated branch.

Local verification after rebuilding the wrapper with --features devnet:

The previous CU-bound failure no longer reproduces with the correctly built devnet wrapper.

No source changes were made as part of the rebase beyond the existing #458 follow-up.

@dcccrypto
dcccrypto merged commit 6d6dc7f into dcccrypto:main Sep 4, 2026
3 checks passed
@dcccrypto

Copy link
Copy Markdown
Owner

Merged — thank you, this was careful work and the writeup made it reviewable.

Both changes were verified against source rather than accepted from the description, and each got an independent negative control run on current main (not your base):

  • removing the two earnings-seeding lines → ..._seeds_outstanding_backing_earnings fails, the other three stay green
  • restoring group.header.mode != 0 to the outer gate → ..._resolved_zero_topup_reconciles_without_reopening_deposits fails, the other three stay green

Two notes for the record:

Your change 1 is more severe than the PR describes. You framed the missing earnings baseline as a reconstruction gap. total_earnings_atoms is also an input to lp_vault_nav_atoms, alongside total_earnings_withdrawn_atoms — which the withdrawal path does increment. An understated earnings total against a growing withdrawn total walks the NAV computation into EngineCounterUnderflow, so the failure mode isn't just a wrong number in the ledger, it's a vault read that fails closed. Worth knowing if you touch this area again.

What made change 2 acceptable wasn't the new match arms on their own — it's that the amount != 0 block independently re-checks mode != 0 and re-calls require_domain_accepts_live_topup_view inside itself, so "non-zero deposits stay closed after resolution" holds in two places rather than one. And amount == 0 still requires the domain's backing_bucket_authority to sign, so there's no path for a stranger to pre-create the ledger PDA under their own key. Both of those were load-bearing for the verdict.

main had moved six commits under you (#459, #460, #461, #462, #463, #464), so your green CI was against a base that no longer existed. Test-merged onto 9cd12d0f: clean auto-merge, then cargo build-sbf -- --features devnet + the full suite — 609 passed, 21 failed all matching the pinned-NFT allowlist exactly.

One habit worth changing: the PR body measures its diff against upstream/main (aeyakovenko) while the PR targets dcccrypto/main. The numbers happened to agree here, but that's how base confusion starts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants