Skip to content

[043][Phase 8][US6] Trace ingestion: providers, mappers, extractors #756

@jwesleye

Description

@jwesleye

Scope

US6 — ingest agent traces from external observability backends: TraceProvider trait + always-on in-memory provider + feature-gated OTLP/Langfuse/OpenSearch/CloudWatch providers + three session mappers (OpenInference, LangChain, OTel GenAI) + trace extractors (including swarm/graph adapters for specs 040/039).

Priority: P2

Tasks

Core trace surface (feature trace-ingest)

  • T119 [P] [US6] Tests in eval/tests/trace_ingest_test.rs — in-memory round-trip, missing attribute → structured error, partial session → SessionInProgress
  • T120 [US6] TraceProvider trait + TraceProviderError + RawSession
  • T121 [US6] OtelInMemoryTraceProvider wrapping opentelemetry-sdk's InMemorySpanExporter

Session mappers

  • T122 [US6] SessionMapper trait + MappingError
  • T123 [P] [US6] OpenInferenceSessionMapper
  • T124 [P] [US6] LangChainSessionMapper
  • T125 [US6] OtelGenAiSessionMapper + GenAIConventionVersion enum (V1_27, V1_30, Experimental) with per-version attribute mapping tables

Per-backend providers

  • T126 [P] [US6] OtlpHttpTraceProvider (feature trace-otlp) with wiremock-backed test
  • T127 [P] [US6] LangfuseTraceProvider (feature trace-langfuse) with test
  • T128 [P] [US6] OpenSearchTraceProvider (feature trace-opensearch) with test
  • T129 [P] [US6] CloudWatchTraceProvider (feature trace-cloudwatch) with test
  • T130 [US6] Feature-gate error — disabled-feature access yields clear compile/construction error (US6 scenario 4); add doc-test

Extractors

  • T131 [US6] EvaluationLevel enum + TraceExtractor trait
  • T132 [P] [US6] SwarmExtractor consuming spec-040 result types
  • T133 [P] [US6] GraphExtractor consuming spec-039 result types
  • T134 [US6] eval/tests/us6_end_to_end_test.rs — record → re-load → score; bit-identical scores for deterministic evaluators (SC-008)

Acceptance

  • OtelInMemoryTraceProvider round-trips a recorded in-process run back into a scorable Invocation.
  • All four backend providers compile + pass their wiremock tests under their respective feature flags.
  • OtelGenAiSessionMapper handles V1_27 and V1_30 attribute conventions without code changes beyond flipping the enum value.
  • Missing attributes surface as MappingError::MissingAttribute { name } — never panic.
  • Disabled-feature backend access produces a clear error (not vague runtime failure).

References

  • Spec FR-031 through FR-034
  • Success criterion SC-008
  • Research R-005

Depends on

#747 (foundational types).

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestevalin-progressAutomated agent is working on thisspecSpec-driven implementation task

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions