Add OTLP/HTTP support to MockTracerAgent and migrate OTLP AspNetCore tests off Docker - #9097
Draft
chojomok wants to merge 8 commits into
Draft
Add OTLP/HTTP support to MockTracerAgent and migrate OTLP AspNetCore tests off Docker#9097chojomok wants to merge 8 commits into
chojomok wants to merge 8 commits into
Conversation
Documents the plan to add OTLP/HTTP (JSON + protobuf) trace decoding to MockTracerAgent per the RFC, and to migrate the OtlpAspNetCore* test suites off the Docker ddapm-test-agent onto it as end-to-end validation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
MockTracerAgent (in TestHelpers) needs the generated OTLP trace proto types to decode OTLP/HTTP payloads, but TestHelpers can't depend on Datadog.Trace.Tests where they previously lived. Relocate the generated bindings and their vendored .proto sources so both projects can use them; Datadog.Trace.Tests already project-references TestHelpers. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Routes /v1/traces to a new decoder that parses both OTLP/HTTP JSON and
protobuf into the official ExportTraceServiceRequest type via
Google.Protobuf, then maps it into a typed MockOtlp* DTO tree
(MockOtlpTraceRequest -> MockOtlpResourceSpans -> MockOtlpScopeSpans ->
MockOtlpSpan) plus a flattened OtlpSpans view, mirroring the existing
Spans/TraceRequestHeaders/WaitForSpansAsync conventions. /v1/metrics
and /v1/logs are captured raw (no decode) so they never reach the
MessagePack decoder. Responses are protocol-correct: {} for JSON,
zero-byte body for protobuf.
The tracer's OTLP/JSON exporter encodes trace/span/parent IDs as hex
(not the standard base64 protobuf-JSON mapping), so JSON requests get
a pre-pass converting those fields to base64 before Google.Protobuf's
JsonParser runs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Covers protobuf decode, JSON decode with hex-to-base64 ID normalization, gzip-compressed bodies, protocol-correct response shapes, unsupported content-type errors, /v1/metrics and /v1/logs raw capture (isolation from trace decoding), coexistence of Datadog and OTLP requests on the same agent, and WaitForOtlpSpansAsync. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Points the OTLP AspNetCore suites at the fixture's existing in-process MockTracerAgent instead of the Docker ddapm-test-agent, using the new OTLP/HTTP decoding support: ConfigureOtlpExport now takes the mock agent's own /v1/traces URL, set once the agent (and its port) exists via a new onAgentCreated hook on AspNetCoreTestFixture.TryStartApp. Test-case isolation mirrors the existing non-OTLP flow exactly: MockTracerAgent.WaitForOtlpSpansAsync filters by a minDateTime captured before each request, rather than clearing any shared session state. Added OtlpSpanFilters (mirroring SpanFilters) so the warm-up alive-check request's span can be excluded the same way the Datadog protocol path already excludes it. The snapshot pipeline reuses OtlpSnapshotHelper unchanged, bridging each captured MockOtlpTraceRequest back to the OTLP JSON wire shape via Google.Protobuf's JsonFormatter (always camelCase/base64, regardless of which wire protocol the request arrived over) and OtlpFieldNames.For (isJson: true). All existing snapshots pass unmodified. OpenTelemetrySdkTests/OpenTelemetryWebRequestTests/OpenTelemetryHttpClientTests are untouched and remain on the Docker test agent (gRPC/metrics/logs coverage, and traces-only suites left as optional follow-up). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
global:: isn't used elsewhere in the codebase outside a handful of unrelated System.* attribute usages; alias the OTLP proto types instead, matching the existing OtlpSpan/OtlpStatusCode convention from OtlpTracesProtobufSerializerTests.cs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (9097) and master. ✅ No regressions detected |
BenchmarksBenchmark execution time: 2026-08-21 22:07:18 Comparing candidate commit 7eb68bf in PR branch Found 0 performance improvements and 1 performance regressions! Performance is the same for 71 metrics, 0 unstable metrics, 68 known flaky benchmarks, 58 flaky benchmarks without significant changes.
|
- MockHeaders.GetValue: use TryGetValue instead of GetValues, which throws InvalidOperationException when a header is absent -- a request to /v1/traces with no Content-Type would crash instead of getting the intended 400 response. - Add MockOtlpAttributeValueKind.Empty and handle both a null KeyValue.Value and AnyValue.ValueOneofCase.None, instead of throwing NRE/NotSupportedException for legal-but-empty OTLP attribute values. - MockOtlpJsonIdNormalizer: throw a clear FormatException for a malformed hex ID instead of silently leaving it unconverted, which previously surfaced as a confusing "invalid base64" error from Google.Protobuf's JsonParser further downstream. - Use names.SpanId instead of a hardcoded "spanId" literal in the AspNetCore snapshot-bridge span filter, matching every other lookup in the same method. - Normalize is null/is not null usage in the new OTLP code paths. - Extract the WaitForSpansAsync/WaitForOtlpSpansAsync polling loop (deadline, 16ms clock-skew tolerance, operationName filtering) into a shared private WaitForSpansCoreAsync<TSpan>, removing the near-duplicate implementation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
chojomok
added
AI Generated
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
MockTracerAgent:/v1/tracesdecodes into a typedMockOtlpSpan/MockOtlpTraceRequestDTO tree via the officialExportTraceServiceRequestprotobuf type;/v1/metrics//v1/logsare captured raw (no decode), so they never reach the MessagePack decoder.Datadog.Trace.TestsintoDatadog.Trace.TestHelperssoMockTracerAgentcan use them.WaitForOtlpSpansAsync,OtlpSpanFilters, and protocol-correct responses ({}for JSON, zero-byte body for protobuf), following the existingWaitForSpansAsync/SpanFilters/TraceRequestHeadersconventions.OtlpAspNetCoreMvc21Tests/OtlpAspNetCoreMvc31Tests/OtlpAspNetCoreMinimalApisTestssuites off the Dockerddapm-test-agentonto this new in-process infrastructure.Reason for change
HTTP OTLP integration tests currently send payloads to the Docker
ddapm-test-agentand normalize/snapshot raw JSON. The in-processMockTracerAgentalready receives Datadog traces, stores test-friendly DTOs, and provides async wait helpers. Supporting OTLP/HTTP in the same mock agent gives tests typed spans instead of transport-specific snapshots, and removes a Docker dependency from the OTLP AspNetCore HTTP suites.Implementation details
Design doc:
docs/superpowers/specs/2026-08-21-otlp-http-mocktracer-design.mdHandlePotentialOtlpTracesdecodesContent-Type: application/x-protobufviaExportTraceServiceRequest.Parser.ParseFrom, andapplication/jsonvia a hex-to-base64 ID pre-pass (the tracer's OTLP/JSON exporter encodes IDs as hex, but the standard OTLP JSON mapping expects base64) followed byGoogle.Protobuf'sJsonParser.OtlpAspNetCoreTestBasenow points the sample app's OTLP export at the fixture's existingMockTracerAgentinstance (via a newonAgentCreatedhook onAspNetCoreTestFixture.TryStartApp, since the agent's port isn't known until the agent is created) instead of the Dockerddapm-test-agent. Test-case isolation mirrors the existing non-OTLP flow:WaitForOtlpSpansAsyncfilters by aminDateTimecaptured before each request, and a newOtlpSpanFilterslist (mirroringSpanFilters) excludes the warm-up/alive-checkspan the same way the Datadog-protocol path already does.OtlpSnapshotHelperunchanged, bridging each capturedMockOtlpTraceRequestback to the OTLP JSON wire shape viaGoogle.Protobuf'sJsonFormatter(always camelCase/base64, regardless of which wire protocol the request arrived over) andOtlpFieldNames.For(isJson: true).OpenTelemetrySdkTests/OpenTelemetryWebRequestTests/OpenTelemetryHttpClientTestsare untouched and remain on the Docker test agent (gRPC/metrics/logs coverage, and traces-only suites left as optional follow-up).Test coverage
Datadog.Trace.Tests/Agent/MockOtlpTraceDecodingTests.cs: protobuf decode, JSON decode with hex→base64 ID normalization, gzip, response shapes, unsupported content-type, metrics/logs isolation, Datadog+OTLP coexistence,WaitForOtlpSpansAsync.OtlpAspNetCoreMvc31Tests(54 cases, all feature-flag/OTel-semantics variants) — all pass against the existing, unmodified Verify snapshots.OtlpAspNetCoreMinimalApisTests(54 cases) — all pass against existing snapshots.OtlpAspNetCoreMvc21Tests— not run locally (sample targetsnetcoreapp2.1, not installed on this dev machine); should be verified in CI.Other details
This PR was developed with heavy AI assistance (Claude Code): the design was brainstormed and written up as a spec, the implementation (decoding pipeline, test migration, and code-review fixes) was written and iterated by the AI agent, with human review and direction throughout. Please review accordingly.