Skip to content

test(llmd): make pressure qualification deterministic - #122

Merged
nerdalert merged 1 commit into
praxis-proxy:mainfrom
nerdalert:test/llmd-deterministic-pressure
Sep 4, 2026
Merged

test(llmd): make pressure qualification deterministic#122
nerdalert merged 1 commit into
praxis-proxy:mainfrom
nerdalert:test/llmd-deterministic-pressure

Conversation

@nerdalert

@nerdalert nerdalert commented Sep 4, 2026

Copy link
Copy Markdown
Member

Summary

Makes the llm-d pool-metrics pressure and recovery qualification deterministic. This uses llm-d sim to set the metric and then kick the pod to show pressure. VCR is great but getting pressure generation without flakes was too unstable for IT since it's also hardware dependent.

  • Replace the request-flood pressure generator with upstream llm-d-inference-sim.
  • Persist simulator metrics in ConfigMaps and roll both simulator Deployments for each phase.
  • Drive the queue-depth path through waiting-requests: 0 -> 9 -> 0.
  • Drive the optional KV-cache path independently through kv-cache-usage: 0.0 -> 0.95 -> 0.0.
  • Continue using real llm-d EPP scraping, Grid scoring/ranking, overlay publication, overlay-sync, Praxis reload, and attributed inference requests.
  • Verify the requested metrics are visible through EPP before evaluating placement.
  • Record the requested queue and KV-cache values for each phase in timeline.jsonl.
  • Remove the obsolete pressure-generator Deployment.

Why

The previous qualification tried to create pressure with concurrent inference traffic. Startup timing and request completion made the EPP signal nondeterministic, so pressure and recovery could pass availability checks without reliably proving the expected pool preference changes.

The simulator makes only the input metric deterministic. The production integration path from EPP through Grid and Praxis remains under test.

Validation

Two fresh queue-depth Kind qualifications passed before this final review:

  • baseline: Pool A preferred at metric 0
  • pressure: Pool A metric 9, preference changed to Pool B
  • recovery: Pool A metric 0, preference returned to Pool A
  • 4/4 proofs passed with automatic teardown in both runs

Additional validation after rebasing onto current main (including #121):

  • 41 focused llm-d xtask tests passed
  • cargo clippy -p xtask --all-targets -- -D warnings
  • nightly formatting check
  • git diff --check

The focused tests cover independent queue-depth and KV-cache simulator configurations, required EPP metric parsing, scoring-mode selection, materialization, and recovery predicates. The optional KV-cache mode was not rerun in Kind as part of this patch.

Generated resolved Forge files and runtime evidence are excluded from the commit. No production Grid, AI, Praxis, EPP, or simulator source code is changed.

Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
@nerdalert
nerdalert merged commit 6436813 into praxis-proxy:main Sep 4, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant