Skip to content

fix(nntp): add configurable connection idle timeout + cap prefetch at HTTP Range end#444

Open
loambit wants to merge 1 commit into
nzbdav-dev:mainfrom
loambit:upstream-pr/idle-timeout-and-prefetch-cap
Open

fix(nntp): add configurable connection idle timeout + cap prefetch at HTTP Range end#444
loambit wants to merge 1 commit into
nzbdav-dev:mainfrom
loambit:upstream-pr/idle-timeout-and-prefetch-cap

Conversation

@loambit
Copy link
Copy Markdown
Contributor

@loambit loambit commented May 25, 2026

This change improves stuttering/stalling during playback with two changes:

  • Raises the default connection idle timeout from 30 seconds to 300 seconds (and makes it configurable). The previous setting was too aggressive, connections in the pool were being disposed during inter-chunk gaps, forcing reconnect costs (TCP + TLS + AUTH) on every chunk request and stalling playback while nzbdav waited for new connections to come up.
  • Caps prefetch at the requested HTTP Range end + 4 articles. Without the cap, nzbdav was fetching up to usenet.article-buffer-size extra articles past what rclone requested (default 40, ~28 MB per chunk) then discarding them on stream close. For chunk fetches this means ~20% wasted RX at upstream defaults (more at higher buffer sizes); for small Range requests like Plex's Background Media Analyzer (which can't be disabled) probes the waste was much larger, sometimes hundreds of MB per probe.

@xbmc4lyfe
Copy link
Copy Markdown

The connection timeout is not really presented as the reason a nzb fails as well.

This proxy: https://github.com/mjc/nntp-proxy

Does a much better job of handling multiple servers then nzbdav, but it causes massive timeouts on nzbdav.

Things that would take minutes to repair / process would be seconds with use of nntp proxy.

I have figured out that you have to basically give first tier the absolute best NNTP servers lowest latency (and lowest REAL latency) which that nntp-proxy can give you metrics for, but nzbdav doesn't handle slower servers well. And when you get into a cascading failure where multiple backends are missing articles but one has it it can slow down the stream massively.

I have had to add a very large amount of logic including handling fall back stream candidates where multiple NZB's are submitted to nzbdav then the stream player falls back when nzbdav starts to give out bytes slower then the player needs it switches over to a known good stream that is exactly the same (but a diff nzb). It does this by checking random bytes in the alternative nzb files and ensurings its the same as well making sure the length of the file inside is the same verifying its the same file just reuploaded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants