Fix OTel tracing of HTTP headers#709
Merged
Merged
Conversation
OTel tracing of HTTP headers was broken, as we were using the LabelerFromContext which actually labels metrics, not adds tracing attributes. I fixed headers tracing, introducing an exclusion list similar to the one we use in logging. I also added tests for OTel tracing: I had to move these under server/internal/ because since both Jaeger and OTel tracing set up a global state, they need to run in different test packages. I took a slightly different approach for excluded headers: instead of just silently skipping them, I've added a trace attribute saying they were present in the request: that will likely make debugging easier. Ref: #707 Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
There was a problem hiding this comment.
Pull Request Overview
This PR fixes the OpenTelemetry tracing of HTTP headers by correctly using tracing attributes and excluding specific headers, and adds dedicated tests to validate the changes.
- Updated server configuration to support tracing header options.
- Refactored middleware to use a new tracer implementation and updated header-exclusion handling.
- Added/updated test files under server/internal for both OpenTelemetry and OpenTracing.
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| server/server.go | Added config flags and updated tracer usage for header tracing. |
| server/internal/oteltest/server_otel_test.go | Added tests for OTel tracing including header exclusion behavior. |
| server/internal/opentracingtest/server_opentracing_test.go | Refactored package name and fixed references to server instance. |
| middleware/logging.go | Renamed the default excluded headers variable and updated dump function. |
| middleware/http_tracing.go | Introduced NewTracer and revised tracing middleware to add header attributes. |
| CHANGELOG.md | Added a bugfix entry for HTTP headers in OTel tracing. |
|
|
||
| if t.traceHeaders { | ||
| headers := maps.Keys(r.Header) | ||
| const maxHeadersToAddAsSpanAttributes = 100 |
There was a problem hiding this comment.
Consider moving the constant 'maxHeadersToAddAsSpanAttributes' to a package-level declaration to avoid using a magic number and facilitate future adjustments.
Suggested change
| const maxHeadersToAddAsSpanAttributes = 100 |
tcard
approved these changes
Jun 5, 2025
tcard
left a comment
tcard
left a comment
Contributor
There was a problem hiding this comment.
LGTM (FWIW), just a few nitpicks and micro-optimization suggestions.
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
tcard
approved these changes
Jun 5, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OTel tracing of HTTP headers was broken, as we were using the LabelerFromContext which actually labels metrics, not adds tracing attributes.
I fixed headers tracing, introducing an exclusion list similar to the one we use in logging.
I also added tests for OTel tracing: I had to move these under server/internal/ because since both Jaeger and OTel tracing set up a global state, they need to run in different test packages.
I took a slightly different approach for excluded headers: instead of just silently skipping them, I've added a trace attribute saying they were present in the request: that will likely make debugging easier.
Ref: #707
What this PR does:
Which issue(s) this PR fixes:
Fixes #
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]