Skip to content

perf: 精排 P99 延迟优化 (180ms → ≤50ms)#201

Open
JamesSunXX wants to merge 11 commits into
alibaba:masterfrom
JamesSunXX:perf/rank-p99-optimization
Open

perf: 精排 P99 延迟优化 (180ms → ≤50ms)#201
JamesSunXX wants to merge 11 commits into
alibaba:masterfrom
JamesSunXX:perf/rank-p99-optimization

Conversation

@JamesSunXX

Copy link
Copy Markdown

概述

精排 P99 延迟从峰刺时 180ms 优化至目标 ≤ 50ms。核心问题:Callback EAS 调用与主请求在服务端排队竞争 + 高 alloc rate 触发 GC assist 征用精排 goroutine。

变更内容

Phase 1: Callback 隔离 + 内存预分配

  • Callback 限流: RateLimitQPS 配置 + limiter.Allow() 非阻塞限速
  • 背压丢弃: channel 满时丢弃 callback(可配置 DropOnBackpressure
  • AlgoDataGenerator: 共享 userFeatures 引用,contextFeatures slice 预分配

Phase 2: 减少 Copy + Batch 优化

  • EasyrecAlgoDataGenerator: 直接转移 slice 所有权替代 copy
  • batchCount 默认 100 → 200(仍可配置覆盖)

Phase 3: Proto 优化 + Metrics

  • proto.Buffer sync.Pool 复用(>1MB 不回池)
  • callback_channel_pending prometheus gauge 监控背压
  • limiter 支持配置热更新(SetLimit/SetBurst)

新增配置字段 (CallBackConfig)

{
  "WorkerPoolSize": 10,
  "RateLimitQPS": 200,
  "JitterMaxMs": 500,
  "DropOnBackpressure": true
}

测试

  • go build ./...
  • go vet 通过(改动包)
  • go test 通过(改动包);EAS/sort/bloomfilter 失败为 pre-existing

灰度建议

  1. Phase 1: 配置 RateLimitQPS=200, JitterMaxMs=500,观察 P99
  2. Phase 2: 监控 EAS 单次调用 P99 在 batch=200 时 ≤ 30ms
  3. Phase 3: 观察 alloc rate 下降至 ≤ 2GB/s

sch added 11 commits June 2, 2026 11:53
- Add WorkerPoolSize, RateLimitQPS, JitterMaxMs, DropOnBackpressure to CallBackConfig
- Refactor CallBackControllerHandler with configurable pool/buffer size
- Add backpressure detection: drop on full channel instead of blocking
- Add rate limiter (golang.org/x/time/rate) before callback EAS calls
- Add random jitter delay to spread callback load
- Expose CallbackPending() for metrics
…cate slices

- AlgoDataGenerator.AddFeatures: store userFeatures reference instead of copying per item
- AlgoDataGenerator.GeneratorAlgoData: merge userFeatures at generation time
- EasyrecAlgoDataGenerator: preallocate contextFeatures slice capacity to 128
…rator

GeneratorAlgoData and GeneratorAlgoDataDebugWithLevel now transfer
requestItem slice ownership directly instead of allocating + copying,
since the generator resets immediately after.
Reduces the number of parallel EAS invocations per rank request.
Still configurable via RankConfig.BatchCount for per-scene tuning.
Use proto.Buffer from sync.Pool to avoid allocating a new byte slice
on every proto.Marshal call. Buffer is returned after BytesPredict.
Wire CallbackPending() into metrics via function pointer to avoid
import cycles. Gauge reports current channel backlog for alerting.
High: Wire CallBackConfig.WorkerPoolSize and DropOnBackpressure into
runtime via InitHandler() called from configloader. Without config,
falls back to 20 workers + blocking (no drop).

Medium: Rate limiter burst clamped to min 1, so sub-1 QPS values
(e.g. 0.5) now correctly throttle instead of silently passing through.

Low: Pending counter incremented BEFORE enqueue and decremented AFTER
processing, eliminating transient negative values under high concurrency.
P1: limiter.Wait now has 2s timeout — prevents indefinite goroutine
blocking when QPS is very low. On timeout, skips the EAS call entirely
(callback is best-effort).

P3: proto.Buffer pool caps at 1MB — oversized buffers from outlier
requests are discarded instead of retained in the pool.
1. Config: aggregate max WorkerPoolSize across all scenes instead of
   taking random first map entry. DropOnBackpressure is OR'd.

2. Init race: InitHandler and Send share one sync.Once (initOnce).
   Whoever calls first wins. If InitHandler is called with real config
   before any Send, config takes effect. If traffic arrives first,
   defaults are used (same Once prevents config from overriding later).

3. Pool: oversized buffers (>1MB) are simply not returned to pool,
   letting GC reclaim them. No unnecessary allocation of replacement.
1. Replace limiter.Wait (blocking) with limiter.Allow() (non-blocking).
   Callback is best-effort; no reason to block shared worker pool.
   Exceeding rate limit simply skips the EAS call.

2. Remove data race: eliminate bare nil check on callBackControllerHandler
   outside sync.Once. Now initOnce.Do is always called in Send path.

3. Support config hot reload: getCallbackLimiter calls SetLimit/SetBurst
   on cached limiter when QPS config changes.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


sch seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants