fix: add guards against ZeroDivisionError across cloud-edge LLM metrics and backends#547
fix: add guards against ZeroDivisionError across cloud-edge LLM metrics and backends#547Fahmid-Arman wants to merge 1 commit into
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Fahmid-Arman The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Code Review
This pull request improves the robustness of LLM inference and evaluation scripts by adding checks to prevent division-by-zero errors when calculating token latency, throughput, and other metrics. The feedback recommends adhering to PEP 8 guidelines by using implicit boolean evaluation for empty sequences instead of checking their length with len() > 0.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
…cs and backends Signed-off-by: Fahmid Arman <fahmid.brac@gmail.com>
52e6e60 to
83b52ff
Compare
|
Feedback addressed:
/assign @MooreZheng |
What type of PR is this?
/kind bug
What this PR does / why we need it:$\le1$ token (e.g., immediate EOS).
The
cloud-edge-collaborative-inference-for-llmbenchmark previously crashed with aZeroDivisionErrorwhen executed against an empty dataset or when a model generatedThis PR implements comprehensive, zero-safe division guards across the 5 metric modules and 4 backend wrappers to ensure the framework returns
0.0gracefully under these edge cases, preserving the execution of the broader benchmark suite.Files patched:
Metrics:
testenv/accuracy.pytestenv/throughput.pytestenv/internal_token_latency.pytestenv/time_to_first_token.pytestenv/edge_ratio.pyBackends:
models/huggingface_llm.pymodels/vllm_llm.pymodels/eagle_llm.pymodels/api_llm.py(Identified and patched proactively)Which issue(s) this PR fixes:
Fixes #337