Reduce Redis fetch time on news browse page (single giant request per load)

[kevin8181]

Problem

The news browse page loads slowly. The likely cause is that fetching posts pulls the full set of cached posts across every feed from Redis on every request, regardless of how many are actually shown or which filters are active — likely one Redis request per feed (see FeedManager.getAllPosts() / feeds/feedManager.ts), merged and sorted at request time.

Scope

Two concrete angles to investigate, in order:

1. Fetch only what's needed: only pull as many posts as will actually be rendered on the page (pagination-aware fetching) instead of the full cached set per feed.
2. Reduce request count: look into whether Upstash Redis supports batching/pipelining multiple feed lookups into a single round trip instead of issuing one request per feed.

Notes

Not yet investigated: whether the Redis client/Upstash REST API supports pipelining, and what the actual current data volume per feed is. Start with profiling to confirm the bottleneck before changing the fetch strategy.

[taksh1507]

Hi! @kevin8181 I'd like to work on this issue.

From the description, it looks like the news browse page may be spending most of its time fetching and merging cached posts from every feed on each request, even though only a small subset is ultimately rendered.

My plan would be to:

Profile the current request path to confirm where the time is being spent (Redis fetches, merge/sort, or something else).
Investigate FeedManager.getAllPosts() / feeds/feedManager.ts to measure how many posts are fetched per feed and how many Redis requests are made.
Implement pagination-aware fetching so we only retrieve the number of posts needed for the current page, rather than the full cached dataset.
Explore whether the Upstash Redis client or REST API supports pipelining/batching to reduce the number of network round trips when fetching multiple feeds.
Benchmark the changes to verify the performance improvement and ensure the existing filtering and sorting behavior remains correct.

I'll start with profiling to validate the bottleneck before making any optimization changes. could you please assign this issue to me? Thanks!