load-testing: add MLflow tracing structure, validation + bottleneck examples - #251
Open
RamVegiraju wants to merge 1 commit into
Open
load-testing: add MLflow tracing structure, validation + bottleneck examples#251RamVegiraju wants to merge 1 commit into
RamVegiraju wants to merge 1 commit into
Conversation
…xamples Additive update to the load-testing skill (source of truth + synced template copies): - Note that templates are served by the MLflow AgentServer (the /invocations SSE contract the scripts hit, and the source of the per-request traces). - Step 6 "Validate & Diagnose with MLflow Tracing": reliability from trace state, latency over successful traces, and bottleneck diagnosis via the span waterfall (CHAT_MODEL vs TOOL split, per-tool p95). - examples/locustfile.py (streaming, TTFT, M2M OAuth, ramp shape) and examples/validate_with_mlflow.py (reliability + latency + span breakdown). - Resources: Locust, MLflow tracing/search-traces/entities API, AgentServer, Databricks load-test-agent-app docs. - description gains an MLflow-tracing trigger. Also add load-testing to the .gitignore source-skill allowlist (!.claude/skills/load-testing/) so its example files are versioned and sync-skills.py can reproduce the template copies from a clean clone. Synced to all templates via .scripts/sync-skills.py.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/invocationsSSE contract the scripts hit, and the source of the per-request traces).state, latency over successful traces, and bottleneck diagnosis via the span waterfall (CHAT_MODELvsTOOLsplit, per-tool p95). Locust measures the system from the outside, MLflow traces from the inside — together they turn a QPS number into an explanation.examples/locustfile.py(streaming, TTFT, M2M OAuth, ramp shape) andexamples/validate_with_mlflow.py(reliability + latency + span breakdown).description.load-testingto the.gitignoresource-skill allowlist so its example files are versioned andsync-skills.pycan reproduce the template copies from a clean clone.All changes are additive (existing wording untouched aside from the extended
description) and synced to all templates via.scripts/sync-skills.py.Test plan
validate_with_mlflow.pycorrectly surfaced the bottleneck tool (get_stock_pricep95 ≈ 897ms vsget_weather≈ 200ms) and the model-vs-tool split.events.request.fire/LoadTestShape.tick; MLflowsearch_traces(locations=…),trace.data.spans,span_type/start_time_ns/end_time_ns.examples/*.pycompile in source and all synced template copies.CHAT_MODEL/TOOL/CHAIN, leaf-span caveat).