Skip to content

feat: version rollout weight updates - #47

Closed
0z5a wants to merge 2 commits into
ViperEkura:mainfrom
0z5a:codex/version-rollout-weight-updates-0z5a
Closed

feat: version rollout weight updates#47
0z5a wants to merge 2 commits into
ViperEkura:mainfrom
0z5a:codex/version-rollout-weight-updates-0z5a

Conversation

@0z5a

@0z5a 0z5a commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • track a monotonic policy version from optimizer steps through the inference scheduler and rollout results
  • serialize synchronous generation with weight acknowledgements and invalidate reusable prefix KV pages on version changes
  • preserve the behavior-policy version on cached rollout samples and initialize it correctly when resuming training
  • document the weight-update and rollout attribution lifecycle

Control-plane benchmark

Exact PR commit 09bdc83, Python 3.12.2 and PyTorch 2.11.0+cu128 on the L20 server. Timings isolate InferenceScheduler.update_weights() after the scheduler is stopped and queues are empty, as required by the API.

version transition state median p99 / range
monotonic acknowledgement no cached prefix pages, 10,000 updates 1.48 us p99 2.41 us
acknowledgement + invalidation 131,072 cached tokens, page size 64, 2,048 pages 2.43 ms 2.30–2.50 ms

The normal no-cache acknowledgement is microsecond-scale. Clearing a fully cached 2,048-page radix/allocator state takes about 2.4 ms and occurs only at the explicit weight-update barrier; it is not added to the decode hot path.

Validation

  • PYTHONPATH=$PWD CUDA_VISIBLE_DEVICES=5 bash scripts/pre_commit.sh --skip-deps
  • 643 tests passed
  • ruff format and import checks passed

The full GPU suite used the existing compiled CUDA extension modules from the L20 checkout, linked into the isolated test worktree.

0z5a and others added 2 commits September 2, 2026 12:56
Track a monotonic policy version across optimizer steps, scheduler updates, and rollout results. Serialize synchronous generation with weight acknowledgements and invalidate reusable prefix KV entries so cached samples remain attributable to the behavior policy that generated them.
@ViperEkura

Copy link
Copy Markdown
Owner

landed as e58a728

@ViperEkura ViperEkura closed this Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants