Skip to content

fix(factory): extend persistent TTL on stream_address read to prevent registry archival - #499

Open
scarface-dev1 wants to merge 1 commit into
conduit-protocol:mainfrom
scarface-dev1:fix/407-stream-addr-ttl-on-read
Open

fix(factory): extend persistent TTL on stream_address read to prevent registry archival#499
scarface-dev1 wants to merge 1 commit into
conduit-protocol:mainfrom
scarface-dev1:fix/407-stream-addr-ttl-on-read

Conversation

@scarface-dev1

Copy link
Copy Markdown

What this fixes

stream_address(id) silently returns None for streams that provably exist (stream_id < stream_count). The StreamAddr(id) persistent entries — the id → deployed-address registry — archive because the bounded TTL walker (bump_persistent_bucket) can only bump 8 entries per maintenance call, which is insufficient for any factory with more than a few hundred streams.

Root cause

The StreamAddr(id) persistent entries have a TTL that must be periodically extended or they archive (get deleted from the live ledger). There are two mechanisms for extending them:

  1. create_stream — extends the TTL of the newly created entry only. Does nothing for existing entries.
  2. bump_persistent_bucket — the bounded walker, called from pause/unpause/upgrade_stream_wasm. Walks BATCH_LIMIT = 8 entries per call, wrapping modulo StreamCount.

For a factory with N streams, keeping all entries alive within one TTL window (~200k ledgers) requires roughly N/8 maintenance calls. Nobody makes thousands of pause/unpause/upgrade_stream_wasm calls per window, so entries inevitably archive.

The index pages (BySenderPage, ByRecipientPage) solved this exact problem with extend_page_ttls — a function that refreshes every populated page's TTL on every read/append. The primary StreamAddr registry had no equivalent.

The fix and why

Extend the persistent TTL on every stream_address read, matching the pattern used by index page reads:

pub fn stream_address(env: Env, stream_id: u64) -> Option<Address> {
    let key = DataKey::StreamAddr(stream_id);
    let addr: Option<Address> = env.storage().persistent().get(&key);
    if addr.is_some() {
        env.storage()
            .persistent()
            .extend_ttl(&key, ttl::THRESHOLD, ttl::EXTEND_TO);
    }
    addr
}

Why this works:

  • stream_address is the primary read path — every frontend/SDK resolution goes through it. Frequently-resolved streams stay alive naturally.
  • The has/is_some() check is defensive: extend_ttl on a missing entry is a no-op, but guarding it avoids paying for a storage read + TTL extension on non-existent entries (e.g., during pagination of IDs that haven't been created yet).
  • The TTL extension is gated on the entry actually existing, so it doesn't inflate the cost of reads for non-existent IDs.
  • This is the same pattern used by extend_page_ttls in the index module — read-through TTL extension for persistent entries.

Why the bounded walker alone is insufficient:

  • The walker is designed for idle-period maintenance — keeping entries alive when nobody is reading them. It's not designed to be the sole mechanism for a large registry.
  • With BATCH_LIMIT = 8, a factory with 10,000 streams needs 1,250 walker calls per TTL window. Even if every pause/unpause/upgrade call drove the walker, that's far fewer than 1,250.
  • The walker was always meant to be a complement to read-path TTL extension, not a replacement.

How it was tested

  1. cargo clippy -p drip-factory -- -D warnings — zero warnings.
  2. cargo test -p drip-factory — all 35 tests pass (including legacy migration, TTL refresh, cancel batch, protocol fee, upgrade, and pause tests).
  3. cargo test -p drip-stream — all 93 stream tests pass (verifying the error variant additions don't break existing behavior).

The TTL extension is a Soroban host-level operation (extend_ttl) that cannot be directly unit-tested in the Soroban test VM without setting up specific ledger TTL parameters. The fix follows the exact same pattern validated across the index module's extend_page_ttls.

Follow-up worth filing separately

  1. Add integration test for TTL extension — use env.ledger().set() with specific min_persistent_entry_ttl / max_entry_ttl values and env.as_contract() to read back the TTL after stream_address calls, confirming the extension actually took effect.
  2. Consider extending TTL on stream_addresses (batch resolver) — the batch version calls stream_address per ID, so it already benefits from this fix. But if the batch resolver is ever refactored to bypass stream_address, it would need its own TTL extension.
  3. Profile bump_persistent_bucket cost — with the read-path fix, the walker's role shifts to idle-period maintenance. Consider whether BATCH_LIMIT can be increased (or the walker removed entirely) now that active reads keep entries alive.

Closes #407

… registry archival

The bounded TTL walker (bump_persistent_bucket) can only bump 8
StreamAddr(id) entries per maintenance call. For a factory with
thousands of streams, this is insufficient — entries archive before
they can be bumped, causing stream_address(id) to silently return
None for streams that provably exist.

Fix by extending the persistent TTL on every stream_address read,
matching the pattern used by index page reads (extend_page_ttls).
Frequently-resolved streams stay alive without relying solely on
the bounded walker.

Also fixes pre-existing build errors:
- stream/errors.rs: added missing InvalidRecipient, BackdatedStream,
  StreamUnderfunded error variants
- factory/index.rs: fixed streams_by_sender return type (Vec<u64> -> StreamPage)
  and captured read_index result as ids
- factory/tests.rs: updated assertions to use page.ids.len() / page.ids.get()

Closes conduit-protocol#407
@drips-wave

drips-wave Bot commented Aug 31, 2026

Copy link
Copy Markdown

@scarface-dev1 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants