Skip to content

REF-048.md

Latest commit

 

History

History
81 lines (65 loc) · 3.37 KB

REF-048.md

File metadata and controls

81 lines (65 loc) · 3.37 KB
Raw
id REF-048
workflow github
github_issue emulebb/emulebb#101
title Reduce queue and tracking scan cost without policy drift
status OPEN
priority Minor
category refactor
labels
queues
kad
download-queue
client-list
performance
post-0.7.3
milestone post-0.7.3
created 2026-05-27
source senior C++ performance and I/O review of up/down clients and queues

Workflow status is tracked in GitHub: emulebb/emulebb#101. This local document is retained as an engineering spec/evidence record.

REF-048 - Reduce queue and tracking scan cost without policy drift

Summary

Several active queue and tracking paths still pay avoidable linear-scan or repeated-selection costs. The safe opportunity is to reduce scan work while preserving the existing queue policy, network timing, packet order, and stale entry semantics.

This is a small refactor lane, not a queue-policy rewrite.

Current Evidence

  • CDownloadQueue local server source-request selection repeatedly searches for the next best request, effectively O(15 * N) for the current batch.
  • Kad packet tracking and callback tracking lists mostly clean expired tail entries during insert paths, so idle periods can leave stale records that are still scanned on later queries.
  • Client duplicate/source lookup paths still contain broad linear scans, but the mutable key set makes secondary indexes risky without diagnostics.
  • CDownloadQueue::SortByPriority still uses CList::FindIndex during heap sorting. This is lower priority because tie-order regressions are easy.

Intended Shape

  1. Factor stale-tail cleanup into Kad packet-tracking and callback-query paths before scans, only removing entries that are already expired by current rules.
  2. Replace repeated local server source-request selection with a one-pass top-N selection that preserves the same priority and wait-order ties.
  3. Add targeted counters or debug-only assertions around candidate counts and selected request order before attempting riskier client lookup indexes.
  4. Consider vector-backed stable priority sorting only after measuring priority resort cost on large download lists.

Scope Constraints

  • Do not change source admission, upload/download scheduling, queue caps, request cadence, retry timing, or server packet format.
  • Do not add secondary client indexes until all key mutation points are known and debug-verified.
  • Do not snapshot raw client pointers across locks without an explicit lifetime guard.
  • Keep any stale-entry cleanup equivalent to entries the current code would already ignore.

Acceptance Criteria

  • Kad and callback query paths remove expired tail entries before scanning.
  • Local server source-request batching chooses the same top 15 files as the current repeated-selection logic for golden queue fixtures.
  • OP_GETSOURCES frame content and ordering remain unchanged for tested fixtures.
  • Metrics or debug assertions document candidate counts and selected order.
  • Client secondary indexes remain deferred unless diagnostics proves a clear hotspot and every key update point is covered.

Validation

  • python -m emule_workspace validate
  • focused native queue-selection golden tests
  • Kad packet-tracking expiry tests
  • source-request packet golden comparison
  • x64 Debug and Release app builds before commit