perf: 精排 P99 延迟优化 (180ms → ≤50ms)#201
Open
JamesSunXX wants to merge 11 commits into
Open
Conversation
added 11 commits
June 2, 2026 11:53
- Add WorkerPoolSize, RateLimitQPS, JitterMaxMs, DropOnBackpressure to CallBackConfig - Refactor CallBackControllerHandler with configurable pool/buffer size - Add backpressure detection: drop on full channel instead of blocking - Add rate limiter (golang.org/x/time/rate) before callback EAS calls - Add random jitter delay to spread callback load - Expose CallbackPending() for metrics
…cate slices - AlgoDataGenerator.AddFeatures: store userFeatures reference instead of copying per item - AlgoDataGenerator.GeneratorAlgoData: merge userFeatures at generation time - EasyrecAlgoDataGenerator: preallocate contextFeatures slice capacity to 128
…rator GeneratorAlgoData and GeneratorAlgoDataDebugWithLevel now transfer requestItem slice ownership directly instead of allocating + copying, since the generator resets immediately after.
Reduces the number of parallel EAS invocations per rank request. Still configurable via RankConfig.BatchCount for per-scene tuning.
Use proto.Buffer from sync.Pool to avoid allocating a new byte slice on every proto.Marshal call. Buffer is returned after BytesPredict.
Wire CallbackPending() into metrics via function pointer to avoid import cycles. Gauge reports current channel backlog for alerting.
High: Wire CallBackConfig.WorkerPoolSize and DropOnBackpressure into runtime via InitHandler() called from configloader. Without config, falls back to 20 workers + blocking (no drop). Medium: Rate limiter burst clamped to min 1, so sub-1 QPS values (e.g. 0.5) now correctly throttle instead of silently passing through. Low: Pending counter incremented BEFORE enqueue and decremented AFTER processing, eliminating transient negative values under high concurrency.
P1: limiter.Wait now has 2s timeout — prevents indefinite goroutine blocking when QPS is very low. On timeout, skips the EAS call entirely (callback is best-effort). P3: proto.Buffer pool caps at 1MB — oversized buffers from outlier requests are discarded instead of retained in the pool.
1. Config: aggregate max WorkerPoolSize across all scenes instead of taking random first map entry. DropOnBackpressure is OR'd. 2. Init race: InitHandler and Send share one sync.Once (initOnce). Whoever calls first wins. If InitHandler is called with real config before any Send, config takes effect. If traffic arrives first, defaults are used (same Once prevents config from overriding later). 3. Pool: oversized buffers (>1MB) are simply not returned to pool, letting GC reclaim them. No unnecessary allocation of replacement.
1. Replace limiter.Wait (blocking) with limiter.Allow() (non-blocking). Callback is best-effort; no reason to block shared worker pool. Exceeding rate limit simply skips the EAS call. 2. Remove data race: eliminate bare nil check on callBackControllerHandler outside sync.Once. Now initOnce.Do is always called in Send path. 3. Support config hot reload: getCallbackLimiter calls SetLimit/SetBurst on cached limiter when QPS config changes.
|
sch seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
精排 P99 延迟从峰刺时 180ms 优化至目标 ≤ 50ms。核心问题:Callback EAS 调用与主请求在服务端排队竞争 + 高 alloc rate 触发 GC assist 征用精排 goroutine。
变更内容
Phase 1: Callback 隔离 + 内存预分配
RateLimitQPS配置 +limiter.Allow()非阻塞限速DropOnBackpressure)Phase 2: 减少 Copy + Batch 优化
Phase 3: Proto 优化 + Metrics
callback_channel_pendingprometheus gauge 监控背压新增配置字段 (CallBackConfig)
{ "WorkerPoolSize": 10, "RateLimitQPS": 200, "JitterMaxMs": 500, "DropOnBackpressure": true }测试
go build ./...✅go vet通过(改动包)go test通过(改动包);EAS/sort/bloomfilter 失败为 pre-existing灰度建议