Skip to content

fix(libsy): retain model call spans through completion - #915

Merged
ayushag-nv merged 1 commit into
mainfrom
nachiketb/call-model-span
Oct 6, 2026
Merged

ayushag-nv merged 1 commit into
mainfrom
nachiketb/call-model-span

Conversation

@nachiketb-nvidia

@nachiketb-nvidia nachiketb-nvidia commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Why

Cancelling an algorithm while its model call is outstanding can leave libsy.llm_call without an outcome. The outcome was recorded by the waiting algorithm, which may never resume.

How

  • Retain the originating span in CallModel, matching CallDecision.
  • Record the outcome when the host responds, fails, or drops the call. Record buffered token usage on successful responses.
  • Keep the existing exactly-once metrics and streamed-response behavior.

Validation

Extended one existing cancellation test with an outcome assertion. It fails before the fix (None instead of error) and passes afterward. All 18 tests in the LLM client observability integration suite pass.

Based directly on main; independent of the Jev configuration MR.

Summary by CodeRabbit

  • Bug Fixes
    • Model-call outcomes are now recorded as success or error when calls complete, fail, or are interrupted, including when a run is dropped while a call is still outstanding.
    • Token-usage telemetry is recorded for completed responses with aggregated usage. Streamed responses without aggregate usage do not produce token-usage records.

Signed-off-by: nachiketb <nachiketb@nvidia.com>
@nachiketb-nvidia
nachiketb-nvidia requested a review from a team as a code owner October 6, 2026 00:21
@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1

🚀 View preview at
https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-915/

Built to branch gh-pages at 2026-10-06 00:22 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@coderabbitai

coderabbitai Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA-NeMo/Switchyard/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: eb2282be-cd27-4b51-a7da-53d5eb6b9ea1
📥 Commits

Reviewing files that changed from the base of the PR and between db0a6bc and 092f29a.

📒 Files selected for processing (3)
  • crates/libsy-llm-client/tests/observability.rs
  • crates/libsy/src/core/algorithm.rs
  • crates/libsy/src/observability.rs

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.


Walkthrough

CallModel retains the originating tracing span. It records response usage and call outcomes on that span. The observability test checks that an abandoned run records an error outcome for the selected model.

Changes

Call Span Telemetry

Layer / File(s) Summary
Record telemetry on the originating span
crates/libsy/src/observability.rs, crates/libsy/src/core/algorithm.rs
record_llm_response records usage from aggregated responses. CallModel retains the span, records response usage and outcome, and the driver passes the current span into the call.
Check the abandoned-call outcome
crates/libsy-llm-client/tests/observability.rs
The test checks that the selected model's call span has outcome error after the run is abandoned.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 092f2

The change records telemetry for completed and abandoned model calls. No actionable merge-blocking risk was identified.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: retaining model call spans through completion.
Docstring Coverage ✅ Passed Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 3 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

I’m a rabbit, ears held high,
I watched the call span pass on by.
It logged the usage, marked the end,
And errors too, when calls suspend.
I twitch my nose and hop with cheer!

Comment @coderabbitai help to get the list of available commands.

@ayushag-nv ayushag-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ayushag-nv
ayushag-nv merged commit b9e7ccc into main Oct 6, 2026
19 checks passed
@ayushag-nv
ayushag-nv deleted the nachiketb/call-model-span branch October 6, 2026 00:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants