feat(aid_escrow): index packages by recipient for O(1) queries - #445
Merged
kilodesodiq-arch merged 1 commit intoAug 20, 2026
Merged
Conversation
Maintain a per-recipient secondary index (rcnt counter + rpidx (recipient, seq) -> package_id entries) written atomically on every package-creation path, and back get_recipient_package_count and list_recipient_packages with it instead of a linear scan of the global package-ID space. get_recipient_package_count is now O(1), list_recipient_packages pages over the recipient's own index (contiguous matches, continuation cursor, no skipped matches for sparse IDs) and clamps limit to a documented MAX_RECIPIENT_PAGE_SIZE. The index uses instance storage: persistent writes metered ~3x higher in Soroban SDK 23 and pushed the 200-package batch past the budget, while instance storage keeps it within limits (see GAS_PROFILING_REPORT.md).
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #424
Adds a per-recipient secondary index to
aid_escrowand rewrites the two recipient query entrypoints to use it:get_recipient_package_countis now an O(1) counter read, andlist_recipient_packagespages over the recipient's own index — contiguous matches, a continuation cursor, no skipped matches for sparse ID spaces — instead of scanning0..KEY_PKG_COUNTER. The single most important design decision: the index lives in instance storage ((rpidx, recipient, seq) -> package_idplus anrcntcounter map), because persistent-storage writes meter ~3x higher in Soroban SDK 23 and pushed the existing 200-package batch test past the read budget; instance storage keeps every existing test passing unchanged.Why
Before this change, the only recipient-oriented reads on the contract were linear scans over the entire package-ID space:
get_recipient_package_countdid one persistentgetper ID up toKEY_PKG_COUNTER, which never shrinks — read cost grows linearly with campaign size until it blows the per-call budget.list_recipient_packagesadvanced its cursor over the ID space, not over matches: a recipient owning packages 2 and 200 could not enumerate both in any bounded number of pages, and pages could be empty while matches remained.limitwas au32accepted verbatim, so a caller could request a scan window guaranteed to exhaust the read budget.The obvious shortcut — capping
limitand keeping the loop — leaves both the O(counter) scan and the broken cursor semantics in place. The fix is a storage-index design (as the issue's "Why this is architecturally hard" section requires), with the index written atomically on every package-creation path so a failed creation reverts the index write with it.What was built
app/onchain/contracts/aid_escrow/src/lib.rs:KEY_RECIPIENT_COUNT(rcnt)Map<Address, u64>— packages per recipient; powersget_recipient_package_countin O(1).KEY_RECIPIENT_IDX(rpidx)(rpidx, recipient, seq) -> package_id, one per (recipient, creation-ordinal); powers paginated enumeration.index_recipient_packagecreate_packageandbatch_create_packages(batch hoists the count map into memory and persists it once after the loop, avoiding O(n²) map re-serialization).get_recipient_package_countlist_recipient_packagesRecipientPackagesPage { ids, next_cursor }, iterates(rpidx, recipient, seq)fromcursor, clampslimittoMAX_RECIPIENT_PAGE_SIZE, and pinsnext_cursorat the count when exhausted.RecipientPackagesPage#[contracttype]page struct — the return contract for indexers.MAX_RECIPIENT_PAGE_SIZEpub const= 100; the documented clamp so a single read cannot request an unbounded scan window.app/onchain/contracts/aid_escrow/tests/recipient_index.rs(new, 5 tests, each with a matching snapshot):count_is_independent_of_large_id_gapspagination_enumerates_every_match_across_sparse_idsnext_cursorat the count; other recipients unaffected.batch_creation_maintains_the_indexbatch_create_packageswith mixed recipients — counts and full enumeration correct, batch IDs in order.limit_is_clamped_to_max_page_sizelimit = u32::MAXreturns 100 +next_cursor; remainder reachable on the next page.individual_and_batch_ids_share_one_indexIntegration changes outside the module
app/onchain/README.md— documents the new pagination contract (cursor semantics, return shape,MAX_RECIPIENT_PAGE_SIZE = 100) and adds the two query rows.docs/onchain/api.md— updates the two query rows to the new return shape and cap.app/onchain/contracts/aid_escrow/GAS_PROFILING_REPORT.md— adds a measured section for the index's write cost and the instance-storage decision (per-package CPU/memory for batch sizes 10–200).The public ABI changes:
list_recipient_packagesnow returnsRecipientPackagesPageinstead ofVec<u64>. This is a breaking change for any external caller, but the only callers are in the contract test suites — no backend or frontend code references either function (verified by search) — and the issue explicitly proposes this new return contract. The deployed testnet contract (CDSBJ27PKTNFTRW6OKPCVXDRUSSRUIQUG6DW5PUTKLDXTDT23NQIS6JG) needs redeploy to pick this up; no storage migration is possible or needed since the index is only written by new creates.Acceptance criteria coverage
Contract
get_recipient_package_countdoes not loop over0..KEY_PKG_COUNTER; its cost is independent of the global counter. (Rewritten as an O(1)rcntmap read; pinned bycount_is_independent_of_large_id_gaps.)list_recipient_packagesreturns every matching package across cursor pages (no skipped matches) and yields a continuation cursor. (Index-backed iteration +next_cursor; pinned bypagination_enumerates_every_match_across_sparse_idsandlimit_is_clamped_to_max_page_size.)limitis clamped to a documented maximum and largelimitvalues cannot exhaust the read budget. (MAX_RECIPIENT_PAGE_SIZE = 100, enforced bylimit_is_clamped_to_max_page_size.)Tests
pagination_enumerates_every_match_across_sparse_ids,count_is_independent_of_large_id_gaps.)batch_create_packages. (batch_creation_maintains_the_index,individual_and_batch_ids_share_one_index.)Documentation
app/onchain/README.mddocuments the new pagination contract (cursor type, return shape) for indexers. (New "Recipient pagination contract" section.)Test plan
cd app/onchain && cargo test --package aid_escrow— 186/186 passing across all suites (5 new tests intests/recipient_index.rs); 0 failurescargo fmt --all -- --check— cleancargo clippy --tests --target x86_64-unknown-linux-gnu -- -D warnings— cleancargo clippy --target wasm32-unknown-unknown -- -D warnings— clean (matches contract-ci.yml)cargo check --locked— succeeds--test gas_profiling) — 11/11 passing, including the 200-package batch (94M CPU / 18.4MB, within the 100M/40MB test budget)Env vars / Notes
No new env vars or config keys. Notes for reviewers/operators:
Address-keyed persistent write ~2–3x higher than instance storage and the existing 200-package gas test exceeded budget (175 packages also failed). Moving the index entries to instance storage kept every existing test green at the original batch sizes. This is documented inGAS_PROFILING_REPORT.mdwith measured numbers.get_recipient_package_count/list_recipient_packageson a migrated deployment reflect only post-migration packages. The issue scoped this as a redeploy decision; the testnet contract requires redeploy.test_snapshots/*.jsonare regenerated test artifacts (not verified by contract-ci.yml); only the snapshots for the changed/new tests are included here.