test(llmd): make pressure qualification deterministic - #122
Merged
nerdalert merged 1 commit intoSep 4, 2026
Merged
Conversation
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the llm-d pool-metrics pressure and recovery qualification deterministic. This uses llm-d sim to set the metric and then kick the pod to show pressure. VCR is great but getting pressure generation without flakes was too unstable for IT since it's also hardware dependent.
llm-d-inference-sim.waiting-requests: 0 -> 9 -> 0.kv-cache-usage: 0.0 -> 0.95 -> 0.0.timeline.jsonl.Why
The previous qualification tried to create pressure with concurrent inference traffic. Startup timing and request completion made the EPP signal nondeterministic, so pressure and recovery could pass availability checks without reliably proving the expected pool preference changes.
The simulator makes only the input metric deterministic. The production integration path from EPP through Grid and Praxis remains under test.
Validation
Two fresh queue-depth Kind qualifications passed before this final review:
Additional validation after rebasing onto current
main(including #121):cargo clippy -p xtask --all-targets -- -D warningsgit diff --checkThe focused tests cover independent queue-depth and KV-cache simulator configurations, required EPP metric parsing, scoring-mode selection, materialization, and recovery predicates. The optional KV-cache mode was not rerun in Kind as part of this patch.
Generated resolved Forge files and runtime evidence are excluded from the commit. No production Grid, AI, Praxis, EPP, or simulator source code is changed.