Skip to content

feat(aid_escrow): index packages by recipient for O(1) queries - #445

Merged
kilodesodiq-arch merged 1 commit into
ChainForgee:mainfrom
Xhristin3:fix/issue-424-recipient-index
Aug 20, 2026
Merged

feat(aid_escrow): index packages by recipient for O(1) queries#445
kilodesodiq-arch merged 1 commit into
ChainForgee:mainfrom
Xhristin3:fix/issue-424-recipient-index

Conversation

@Xhristin3

Copy link
Copy Markdown
Contributor

Summary

Closes #424

Adds a per-recipient secondary index to aid_escrow and rewrites the two recipient query entrypoints to use it: get_recipient_package_count is now an O(1) counter read, and list_recipient_packages pages over the recipient's own index — contiguous matches, a continuation cursor, no skipped matches for sparse ID spaces — instead of scanning 0..KEY_PKG_COUNTER. The single most important design decision: the index lives in instance storage ((rpidx, recipient, seq) -> package_id plus an rcnt counter map), because persistent-storage writes meter ~3x higher in Soroban SDK 23 and pushed the existing 200-package batch test past the read budget; instance storage keeps every existing test passing unchanged.

Why

Before this change, the only recipient-oriented reads on the contract were linear scans over the entire package-ID space:

  • get_recipient_package_count did one persistent get per ID up to KEY_PKG_COUNTER, which never shrinks — read cost grows linearly with campaign size until it blows the per-call budget.
  • list_recipient_packages advanced its cursor over the ID space, not over matches: a recipient owning packages 2 and 200 could not enumerate both in any bounded number of pages, and pages could be empty while matches remained.
  • limit was a u32 accepted verbatim, so a caller could request a scan window guaranteed to exhaust the read budget.

The obvious shortcut — capping limit and keeping the loop — leaves both the O(counter) scan and the broken cursor semantics in place. The fix is a storage-index design (as the issue's "Why this is architecturally hard" section requires), with the index written atomically on every package-creation path so a failed creation reverts the index write with it.

What was built

app/onchain/contracts/aid_escrow/src/lib.rs:

Change What it contains
KEY_RECIPIENT_COUNT (rcnt) Instance Map<Address, u64> — packages per recipient; powers get_recipient_package_count in O(1).
KEY_RECIPIENT_IDX (rpidx) Instance entries (rpidx, recipient, seq) -> package_id, one per (recipient, creation-ordinal); powers paginated enumeration.
index_recipient_package Private helper appending a package to its recipient's index; called from create_package and batch_create_packages (batch hoists the count map into memory and persists it once after the loop, avoiding O(n²) map re-serialization).
get_recipient_package_count Rewritten: single instance-map read, independent of the global counter.
list_recipient_packages Rewritten: returns RecipientPackagesPage { ids, next_cursor }, iterates (rpidx, recipient, seq) from cursor, clamps limit to MAX_RECIPIENT_PAGE_SIZE, and pins next_cursor at the count when exhausted.
RecipientPackagesPage New #[contracttype] page struct — the return contract for indexers.
MAX_RECIPIENT_PAGE_SIZE pub const = 100; the documented clamp so a single read cannot request an unbounded scan window.

app/onchain/contracts/aid_escrow/tests/recipient_index.rs (new, 5 tests, each with a matching snapshot):

Test Covers
count_is_independent_of_large_id_gaps Count reflects only the recipient's own packages even with a 5000-ID gap; unknown recipients read 0.
pagination_enumerates_every_match_across_sparse_ids Recipient with ids 2 and 200: both found across 2 one-item pages with no empty page between; exhaustion pins next_cursor at the count; other recipients unaffected.
batch_creation_maintains_the_index batch_create_packages with mixed recipients — counts and full enumeration correct, batch IDs in order.
limit_is_clamped_to_max_page_size 105 packages, limit = u32::MAX returns 100 + next_cursor; remainder reachable on the next page.
individual_and_batch_ids_share_one_index Individual create + batch create on the same recipient share one ordinal sequence.

Integration changes outside the module

  • app/onchain/README.md — documents the new pagination contract (cursor semantics, return shape, MAX_RECIPIENT_PAGE_SIZE = 100) and adds the two query rows.
  • docs/onchain/api.md — updates the two query rows to the new return shape and cap.
  • app/onchain/contracts/aid_escrow/GAS_PROFILING_REPORT.md — adds a measured section for the index's write cost and the instance-storage decision (per-package CPU/memory for batch sizes 10–200).

The public ABI changes: list_recipient_packages now returns RecipientPackagesPage instead of Vec<u64>. This is a breaking change for any external caller, but the only callers are in the contract test suites — no backend or frontend code references either function (verified by search) — and the issue explicitly proposes this new return contract. The deployed testnet contract (CDSBJ27PKTNFTRW6OKPCVXDRUSSRUIQUG6DW5PUTKLDXTDT23NQIS6JG) needs redeploy to pick this up; no storage migration is possible or needed since the index is only written by new creates.

Acceptance criteria coverage

Contract

  • get_recipient_package_count does not loop over 0..KEY_PKG_COUNTER; its cost is independent of the global counter. (Rewritten as an O(1) rcnt map read; pinned by count_is_independent_of_large_id_gaps.)
  • list_recipient_packages returns every matching package across cursor pages (no skipped matches) and yields a continuation cursor. (Index-backed iteration + next_cursor; pinned by pagination_enumerates_every_match_across_sparse_ids and limit_is_clamped_to_max_page_size.)
  • limit is clamped to a documented maximum and large limit values cannot exhaust the read budget. (MAX_RECIPIENT_PAGE_SIZE = 100, enforced by limit_is_clamped_to_max_page_size.)

Tests

  • A test creates packages for two recipients with a large ID gap and asserts both recipients are fully enumerated across pages. (pagination_enumerates_every_match_across_sparse_ids, count_is_independent_of_large_id_gaps.)
  • A test asserts package creation maintains the index, including via batch_create_packages. (batch_creation_maintains_the_index, individual_and_batch_ids_share_one_index.)

Documentation

  • app/onchain/README.md documents the new pagination contract (cursor type, return shape) for indexers. (New "Recipient pagination contract" section.)

Test plan

  • cd app/onchain && cargo test --package aid_escrow — 186/186 passing across all suites (5 new tests in tests/recipient_index.rs); 0 failures
  • cargo fmt --all -- --check — clean
  • cargo clippy --tests --target x86_64-unknown-linux-gnu -- -D warnings — clean
  • cargo clippy --target wasm32-unknown-unknown -- -D warnings — clean (matches contract-ci.yml)
  • cargo check --locked — succeeds
  • Gas profiling suite (--test gas_profiling) — 11/11 passing, including the 200-package batch (94M CPU / 18.4MB, within the 100M/40MB test budget)

Env vars / Notes

No new env vars or config keys. Notes for reviewers/operators:

  • Instance vs persistent storage. The index was first implemented on persistent storage; SDK 23 metered each Address-keyed persistent write ~2–3x higher than instance storage and the existing 200-package gas test exceeded budget (175 packages also failed). Moving the index entries to instance storage kept every existing test green at the original batch sizes. This is documented in GAS_PROFILING_REPORT.md with measured numbers.
  • Deploy ordering. The new index is only populated by package creations after deploy. Packages created before this change are not in the index, so get_recipient_package_count/list_recipient_packages on a migrated deployment reflect only post-migration packages. The issue scoped this as a redeploy decision; the testnet contract requires redeploy.
  • test_snapshots/*.json are regenerated test artifacts (not verified by contract-ci.yml); only the snapshots for the changed/new tests are included here.

Maintain a per-recipient secondary index (rcnt counter + rpidx
(recipient, seq) -> package_id entries) written atomically on every
package-creation path, and back get_recipient_package_count and
list_recipient_packages with it instead of a linear scan of the global
package-ID space.

get_recipient_package_count is now O(1), list_recipient_packages pages
over the recipient's own index (contiguous matches, continuation
cursor, no skipped matches for sparse IDs) and clamps limit to a
documented MAX_RECIPIENT_PAGE_SIZE. The index uses instance storage:
persistent writes metered ~3x higher in Soroban SDK 23 and pushed the
200-package batch past the budget, while instance storage keeps it
within limits (see GAS_PROFILING_REPORT.md).

@kilodesodiq-arch kilodesodiq-arch left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kilodesodiq-arch
kilodesodiq-arch merged commit 7609aa3 into ChainForgee:main Aug 20, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

get_recipient_package_count scans every package ID: unbounded gas and wrong pagination for sparse recipients

2 participants