Skip to content

OTelTracer/StepSpan never attach span context, so every run/step span is an orphaned root #89

Description

@goanpeca

Summary

The live OpenTelemetry path (OTelTracer + StepSpan) starts spans but never makes
them current and never passes a parent context=. As a result every run span and
step span lands in its own separate trace with no parent, so the run/step hierarchy
is lost. Traces are mis-structured, not data-affecting, hence P3.

Affected files

  • libs/core/genblaze_core/observability/tracer.py:186 (on_run_start -> start_span), :204 (on_step_start -> start_span): spans are stored in dicts but never made current and never given a parent context=
  • libs/core/genblaze_core/observability/span.py:35 (StepSpan.__enter__ -> tracer.start_span(self.name)), used at libs/core/genblaze_core/providers/base.py:1636 (invoke) and :1808 (ainvoke)
  • Confirmed absent (grep returned zero hits): start_as_current_span, trace.use_span, context.attach, set_span_in_context, any context= argument

Evidence

Runtime demo with InMemorySpanExporter:

Sequential run() with OTelTracer (2 steps): 5 spans captured
  distinct trace_ids = 5     (expected 1)
  root spans (parent=None) = 5   (expected 1: the pipeline span)
  ROOTS: 'pipeline:h2-sequential', 'step:mock/m1', 'mock/m1', 'step:mock/m2', 'mock/m2'
  step 'step:mock/m1' child-of-pipeline? False
abatch_run(3 inputs, max_concurrency=4): 9 spans -> 9 distinct traces -> 9 orphan roots

Two independent span sources double-instrument each step (step:mock/m1 from
OTelTracer, mock/m1 from BaseProvider.invoke's StepSpan), and neither is
nested under the other or under the run span, even single-threaded. This is distinct
from issue #37 (which covers storage/_tracer.py as dead code; that @traced
decorator has zero call sites, so there are no storage spans to orphan). The
existing tests/unit/test_tracer.py asserts no parent/child relationship.

Why it matters

Genblaze advertises OTel tracing as a first-class observability backend. In any real
collector (Jaeger, Tempo, Honeycomb), users expect one trace per pipeline run with
step spans nested under it. Instead every span is its own root trace, so run/step
hierarchy, per-run latency roll-ups, and step-within-run correlation are all lost.
No data-correctness impact.

Suggested fix

In OTelTracer, hold the run span's Context (trace.set_span_in_context(run_span))
and start step spans with start_span(name, context=run_ctx). For spans that should
be current during execution, use start_as_current_span / trace.use_span(...) with
a token detached on exit. For StepSpan, accept and attach the parent context so the
provider-level span nests under the tracer's step span. Add an in-memory-exporter
test asserting a single trace id and correct parentage.

Acceptance criteria

  • A pipeline run with an OTelTracer produces one trace whose step spans are nested
    under the run span.
  • A regression test with an in-memory exporter asserts the parent/child structure.

Metadata

Metadata

Assignees

No one assigned

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions