Skip to content

fix(reticulum): PN deposit link timeouts and local PN official parity - #832

Merged
rinchen merged 3 commits into
mainfrom
fix/pn-delivery-local-parity
Aug 9, 2026
Merged

fix(reticulum): PN deposit link timeouts and local PN official parity#832
rinchen merged 3 commits into
mainfrom
fix/pn-delivery-local-parity

Conversation

@rinchen

@rinchen rinchen commented Aug 9, 2026

Copy link
Copy Markdown
Member

Summary

Fixes flaky / broken Remote Prefer PN deposits (not limited to local-prop-only setups) by advancing the Prefer cascade on Propagated link-establishment timeouts, hosting local-prop deposits in-process, and keeping sync↔deposit Link serialization so Prefer PN sync and outbound deposits stop fighting the same Link.

Also includes chore: pnpm 11.21.0 + Flatpak archive sha sync.

Problem

Remote Prefer PN deposits were flaky or broken even when users were not on a local-prop-only configuration.

Field evidence (PN delivery / w0rmt + Joey diagnosis):

  • Joey Prefer thunderhost pn-9f3f189e logged repeated link-timeout … skip dest=9f3f189e. With local-prop off, the cascade could fail visibly instead of quietly masking the miss.
  • w0rmt Prefer pn-deadbeef saw syncTimedOut, never got a durable lastPropagationSyncAt, and accumulated heavy linkDeliveryTimeouts on Prefer PN / Ratspeak / Direct paths.
  • Direct chat often still worked. The failure mode was concentrated on Propagated packed Link delivery to PN destinations plus PN sync establish contention on the same Link.

Why it looked OK sometimes:

  • Local-prop Completes had been masking last-resort Prefer cascade failures (local host “succeeded” while Prefer remotes never truly deposited).
  • Docs/UI incorrectly framed local prop as outbox-only, which hid that a hosted local PN should behave like a real propagation node.

Root causes:

  1. On Propagated link establishment timeout, outbound requeued the same Prefer PN until the attempt budget instead of marking it tried and advancing the cascade.
  2. Deposit vs sync fought the same PN Link (has_pending_to / propagation_sync_target), producing deferral loops and syncTimedOut.
  3. Local cascade could self-Link timeout instead of accepting the stamped blob in-process when this client hosts local prop.
  4. Tests never proved outbound → local deposit → drain, or deposit → peer /offer inventory visibility.

Fix

  • on_propagated_link_failure: if other cascade candidates remain, mark the timed-out Prefer PN as tried and advance; only requeue the same PN when the cascade is exhausted / last-resort retry budget applies.
  • Local-prop cascade: when hosting (set_local_prop_node), pack and accept_stamped_propagated_blob in-process; emit stored_locally; skip self-Link and skip sync-busy defer for that path; pin local prop identity while serving.
  • Keep sync↔deposit serialization: deposit defers, then advances after PN_DEPOSIT_DEFER_ADVANCE_AFTER; sync returns PROPAGATION_SYNC_OUTBOUND_BUSY when deposit owns the Link.
  • Official local PN parity language (docs/i18n): full PN (announce / admit / peer / get); clients need not Prefer you; house badge means deposited on the hosted node, not a dead-end outbox.
  • Chore: pnpm 11.21.0 + Flatpak archive sha sync.

Test plan

Automated coverage added/extended for the diagnosis paths:

  • T1 — outbound DeliverPropagated → local accept → stored_locally → drain
  • T2 — Prefer link-timeout advance / exhaust→failed (fixtures for 9f3f / deadbeef)
  • T3 — sync-busy defer then advance
  • T5 — deposit appears in peer /offer inventory
  • T6 — autopeer cost / depth / static_peers

Manual (as before):

  • Prefer a remote PN; send Propagated traffic; confirm deposit Completes without repeated link-timeout … skip dest=… on the same Prefer id until budget burn
  • With local-prop hosting on: confirm in-process stored_locally (no self-Link timeout) and house badge semantics
  • With local-prop off: confirm cascade advances across Prefer remotes and fails clearly when exhausted
  • Overlap Prefer sync + outbound deposit: sync should see PROPAGATION_SYNC_OUTBOUND_BUSY / defer rather than syncTimedOut loops; deposit should advance after defer window
  • Peer offer path shows deposited material in inventory after local/remote deposit
  • Spot-check Direct chat still works alongside Propagated Prefer PN

Note: PR #831 PATH_UNKNOWN fail-fast improves sync error surfacing, but Prefer PN deposit reliability required this cascade / Link ownership fix.

Advance Prefer PN cascade on link-establishment timeout instead of
hammering the same hash; deposit into hosted local-prop in-process;
cover outbound→drain and peer-offer round-trips with tests and docs.
@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@rinchen, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 49 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: QUIET

Plan: Pro Plus

Run ID: d364cd80-2644-4e86-9b9a-80fffe647243

📥 Commits

Reviewing files that changed from the base of the PR and between 0af0d2c and 712c8ea.

📒 Files selected for processing (6)
  • docs/agents/reticulum.md
  • docs/reticulum.md
  • org.coloradomesh.MeshClient.yml
  • package.json
  • reticulum-sidecar/src/stack/lxmf_outbound.rs
  • reticulum-sidecar/src/stack/pn_hosting_apply.rs
📝 Walkthrough

Walkthrough

Local propagation now deposits messages directly into the hosted PropagationNode. The outbound driver reports hosted-PN completion, supports peer synchronization and retrieval, and advances remote cascades after link failures, timeouts, or deferrals.

Changes

Hosted PN delivery

Layer / File(s) Summary
Local PN wiring and hosting policy
reticulum-sidecar/src/stack/live.rs, reticulum-sidecar/src/stack/lxmf_outbound.rs, reticulum-sidecar/src/stack/pn_cascade.rs, reticulum-sidecar/src/stack/pn_hosting_apply.rs
The live stack registers or clears the local PropagationNode. The outbound driver stores the node. Hosting-policy tests cover peer acceptance and autopeer limits.
In-process local deposit flow
reticulum-sidecar/src/stack/lxmf_outbound.rs
Local cascade messages are validated and accepted by the hosted PN before Link delivery. Successful deposits report stored_locally; failed deposits use Link fallback. Tests cover local retrieval and peer synchronization visibility.
Cascade failure and timeout advancement
reticulum-sidecar/src/stack/lxmf_outbound.rs
Propagated failures, link-establishment timeouts, and repeated deferrals advance to remaining PN candidates. Exhausted cascades emit terminal failure.
Hosted-PN semantics and operational documentation
docs/agents/reticulum.md, docs/reticulum-sidecar-ipc.md, docs/reticulum.md, docs/troubleshooting.md, reticulum-sidecar/src/stack/propagation_bridge.rs
Documentation defines local propagation as a hosted, peer-synchronizable PN deposit with stored_locally completion and cross-node retrieval. Test guidance references the outbound round-trip coverage.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant LxmfOutboundDriver
  participant PropagationNode
  participant PNPeer
  participant LXMFClient
  LxmfOutboundDriver->>PropagationNode: accept local propagation deposit
  PropagationNode->>PNPeer: synchronize through /offer
  LXMFClient->>PropagationNode: request message through /get
  PropagationNode-->>LXMFClient: return stored message
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the two main changes: PN deposit link-timeout handling and local PN behavior parity.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/pn-delivery-local-parity

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Keep Flatpak standalone pnpm archive URLs/sha256 in sync with the pin.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments (4)
docs/agents/reticulum.md-19-19 (1)

19-19: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Use the in-process local-prop drain name.

Line 19 describes local-prop as “client /get draining”. The subsystem guidance distinguishes remote PropagationClient /get retrieval from local-prop PropagationBridge::drain_local_inbox() replay. Update this sentence to name the local path and describe remote /get separately.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/agents/reticulum.md` at line 19, Update the local-prop description in
the Reticulum outbound documentation to identify the in-process drain as
PropagationBridge::drain_local_inbox(). Distinguish this local replay path from
remote PropagationClient /get retrieval, while preserving the existing host-peer
/offer synchronization description.
docs/reticulum.md-41-41 (1)

41-41: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Correct the Propagation row’s cascade summary.

Line 41 still says Manual falls back to the “local inbox”. When local hosting is enabled, the candidate is the hosted local-prop PN and the terminal status is stored_locally. The row also conflates the Auto sync target with outbound deposit order: outbound deposits try configured remotes before Discovered PNs and local-prop last. Update this summary to match Lines 35 and 314-321.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/reticulum.md` at line 41, Update the Propagation row to describe Manual
fallback as the hosted local-prop PN with terminal status stored_locally, and
distinguish Auto’s best Discovered PN sync target from outbound deposit
ordering. State that outbound deposits try configured remotes first, then
Discovered PNs, with local-prop last, while preserving the existing mode
guidance.
reticulum-sidecar/src/stack/lxmf_outbound.rs-743-762 (1)

743-762: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Return false when the message has no hash so the Link fallback still runs.

The function returns true after the warning. The caller then returns from deliver_propagated without starting packed Link delivery, without requeueing, and without emitting a status. A message that has neither hash nor message_id is dropped silently. Every other failure path in this function returns false and falls back to Link delivery.

🐛 Proposed fix
         let msg_hash = message.hash.or(message.message_id);
-        if let Some(hash) = msg_hash {
-            self.pending_pn_deposits
-                .insert(hash, (prop_hash, last_tid.or(message.transient_id)));
-            self.handle_delivery_result(
-                router,
-                event_tx,
-                DeliveryResult::Complete {
-                    link_id: prop_hash,
-                    msg_hash: Some(hash),
-                },
-            );
-        } else {
+        let Some(hash) = msg_hash else {
             tracing::warn!(
                 target: "propagation-deposit",
                 pn_hash = %prop_hex,
-                "local-prop in-process deposit missing message hash"
+                "local-prop in-process deposit missing message hash — falling back to Link"
             );
-        }
+            return false;
+        };
+        self.pending_pn_deposits
+            .insert(hash, (prop_hash, last_tid.or(message.transient_id)));
+        self.handle_delivery_result(
+            router,
+            event_tx,
+            DeliveryResult::Complete {
+                link_id: prop_hash,
+                msg_hash: Some(hash),
+            },
+        );
         true
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@reticulum-sidecar/src/stack/lxmf_outbound.rs` around lines 743 - 762, Update
the hash-missing branch in the outbound delivery function containing the
pending_pn_deposits insertion and propagation-deposit warning to return false
after logging the warning, while retaining the true return for successfully
hashed messages. This must allow the caller’s Link fallback, requeue, and status
handling to run when both message.hash and message.message_id are absent.
reticulum-sidecar/src/stack/lxmf_outbound.rs-589-595 (1)

589-595: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Avoid blocking the outbound tick on the shared PropagationNode mutex.

process_tick now holds the outbound driver and router locks while reading local_fl oor and accepting packed entries. Other tasks can hold the same PropagationNode mutex, so an accept_stamped_propagated_blob path can delay every local-propagation cascade while the maintenance loop is blocked. Keep the local-node access short and avoid long work, I/O, or CPU-heavy validation while that mutex is held.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@reticulum-sidecar/src/stack/lxmf_outbound.rs` around lines 589 - 595, Update
process_tick’s local propagation cost and packed-entry acceptance flow to avoid
blocking on the shared PropagationNode mutex while outbound driver/router locks
are held. Read the needed local floor with a short, non-blocking access (or
defer/skip when the mutex is unavailable), and ensure
accept_stamped_propagated_blob and any validation or other lengthy work occur
after releasing the PropagationNode guard.
🧹 Nitpick comments (1)
reticulum-sidecar/src/stack/pn_hosting_apply.rs (1)

141-164: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert that declined candidates are not added to router.peers.

The test checks only the boolean return of autopeer. A regression that returns false but still inserts the peer would pass. Add the negative membership assertions for deep and costly, matching the positive assertion at Line 177.

♻️ Proposed test hardening
             metadata: None,
             hops: Some(5),
         }));
+        assert!(!router.peers.contains_key(&deep));
 
         // Peering cost above max declined.
         assert!(!router.autopeer(AutopeerCandidate {
@@
             metadata: None,
             hops: Some(1),
         }));
+        assert!(!router.peers.contains_key(&costly));
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@reticulum-sidecar/src/stack/pn_hosting_apply.rs` around lines 141 - 164,
Extend the test around the declined AutopeerCandidate cases to assert that
neither deep nor costly is present in router.peers after autopeer returns false,
matching the existing positive membership assertion near the later
accepted-candidate case.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Other comments:
In `@docs/agents/reticulum.md`:
- Line 19: Update the local-prop description in the Reticulum outbound
documentation to identify the in-process drain as
PropagationBridge::drain_local_inbox(). Distinguish this local replay path from
remote PropagationClient /get retrieval, while preserving the existing host-peer
/offer synchronization description.

In `@docs/reticulum.md`:
- Line 41: Update the Propagation row to describe Manual fallback as the hosted
local-prop PN with terminal status stored_locally, and distinguish Auto’s best
Discovered PN sync target from outbound deposit ordering. State that outbound
deposits try configured remotes first, then Discovered PNs, with local-prop
last, while preserving the existing mode guidance.

In `@reticulum-sidecar/src/stack/lxmf_outbound.rs`:
- Around line 743-762: Update the hash-missing branch in the outbound delivery
function containing the pending_pn_deposits insertion and propagation-deposit
warning to return false after logging the warning, while retaining the true
return for successfully hashed messages. This must allow the caller’s Link
fallback, requeue, and status handling to run when both message.hash and
message.message_id are absent.
- Around line 589-595: Update process_tick’s local propagation cost and
packed-entry acceptance flow to avoid blocking on the shared PropagationNode
mutex while outbound driver/router locks are held. Read the needed local floor
with a short, non-blocking access (or defer/skip when the mutex is unavailable),
and ensure accept_stamped_propagated_blob and any validation or other lengthy
work occur after releasing the PropagationNode guard.

---

Nitpick comments:
In `@reticulum-sidecar/src/stack/pn_hosting_apply.rs`:
- Around line 141-164: Extend the test around the declined AutopeerCandidate
cases to assert that neither deep nor costly is present in router.peers after
autopeer returns false, matching the existing positive membership assertion near
the later accepted-candidate case.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: QUIET

Plan: Pro Plus

Run ID: 43a16c38-9192-4553-98fd-754705f456c6

📥 Commits

Reviewing files that changed from the base of the PR and between 6b99225 and 0af0d2c.

⛔ Files ignored due to path filters (1)
  • src/renderer/locales/en/translation.json is excluded by !src/renderer/locales/**
📒 Files selected for processing (9)
  • docs/agents/reticulum.md
  • docs/reticulum-sidecar-ipc.md
  • docs/reticulum.md
  • docs/troubleshooting.md
  • reticulum-sidecar/src/stack/live.rs
  • reticulum-sidecar/src/stack/lxmf_outbound.rs
  • reticulum-sidecar/src/stack/pn_cascade.rs
  • reticulum-sidecar/src/stack/pn_hosting_apply.rs
  • reticulum-sidecar/src/stack/propagation_bridge.rs

…lback)

Clarify local drain vs remote /get; distinguish Auto sync vs deposit order;
try_lock + validate outside PropagationNode; return false without hash for Link
fallback; assert declined autopeers are absent from router.peers.
@rinchen
rinchen merged commit 0a80001 into main Aug 9, 2026
21 checks passed
@rinchen
rinchen deleted the fix/pn-delivery-local-parity branch August 9, 2026 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant