Fix STUN busy loop in controlled agent's HandleBindingRequest#907
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #907 +/- ##
==========================================
+ Coverage 88.25% 88.29% +0.03%
==========================================
Files 45 45
Lines 5740 5741 +1
==========================================
+ Hits 5066 5069 +3
+ Misses 462 461 -1
+ Partials 212 211 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
JoTurk
left a comment
There was a problem hiding this comment.
Good catch, Thank you so much.
Do you mind adding a regression test so we don't break this again? and fix the lint.
Thanks :)
Once a candidate pair reaches Succeeded state with a selected pair, controlledSelector.HandleBindingRequest no longer sends a triggered check (PingCandidate). Previously, every inbound Binding Request unconditionally called PingCandidate, creating a ping-pong loop where each Response triggered a new Request at 1/RTT speed. After connection is established, consent freshness is maintained by checkKeepalive() on a timer, so triggered checks are not needed. Added regression tests verifying: - No triggered check for succeeded+selected pairs - Triggered check still sent during ICE checking phase
15c8173 to
d71dcf7
Compare
|
Thanks for the review! I've addressed your feedback:
|
|
@Star-ho, great catch! |
|
@erivni Our lite implementation isn't yet complete, other than this startup ping behavior in Lite mode that we should fix, we don't support lite controlling agent [1], so if Pion is both controlling and lite we fall-back to full ICE, it's in my todo-list to implement this, but any PRs are welcome :)
|
|
You're both right — re-reading RFC 8445 §2.5 ("Lite agents … do not generate connectivity checks") together with §6.1.1 makes it clear that guarding My setup only exercises the full-to-full path, so I couldn't reproduce the handshake loop locally. If either of you picks this up — or @erivni can share a minimal repro / pcap from the AWS Kinesis case — I'm happy to help review. |
|
please see: #909 |
Summary
The controlled selector's
HandleBindingRequestunconditionally callsPingCandidate(triggered check) for every inbound Binding Request, even after ICE connectivity is fully established. This creates a STUN ping-pong busy loop between the controlled and controlling agents that repeats at 1/RTT speed.The Problem
When the controlling agent sends keepalive Binding Requests to the controlled agent:
PingCandidate(unnecessary after connection)This loop runs continuously at network RTT speed:
With 200 concurrent WebRTC sessions, total STUN traffic reaches ~57,600 req/s instead of the expected ~100 req/s, causing kernel softirq CPU spikes (up to 70%) on the media server.
The Fix
Skip the triggered check when the candidate pair has already succeeded and a selected pair exists. During ICE checking phase, triggered checks are still sent per RFC 8445 §7.3.1.4. After connection, consent freshness is handled by
checkKeepalive()on a timer.Observed Impact
How to Reproduce
RequestsSentfromGetCandidatePairsStats()— the rate scales with 1/RTT instead of staying at the keepalive intervalNotes
165d04e(May 2019) when STUN Binding Indications were replaced with Binding Requests for RFC 7675 consent freshnessHandleBindingRequestdoes not have this issue — it does not callPingCandidate