[Enhancement] Search Query Optimization & SearXNG Rate Limit Handling

[Smilin01]

Problem

SearXNG public instances get rate-limited under load, causing silent failures or empty results.
Query rewriting sometimes produces overly broad or redundant queries that waste fetch cycles.

Expected Behavior

• Smarter query deduplication before firing concurrent searches
• Exponential backoff + retry logic when a SearXNG node fails
• Health-check pool: track which instances are alive and rotate only healthy ones
• Query intent scoring to skip search entirely for greetings/chitchat

Current Behavior

• Falls back to DuckDuckGo Lite but with no user feedback
• Redundant queries increase latency without improving result quality

Suggested Approach

• Maintain an in-memory instance health map with TTL
• Add a query classifier (simple regex or tiny model) to gate search calls
• Log which instances succeed/fail for debugging

Labels

enhancement search good first issue