Skip to content

Refactored span events to avoid not needed JSON deserialization for v… #11855

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AlexeyKuznetsov-DD wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Refactored span events to avoid not needed JSON deserialization for v… #11855

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .../agent-otel/otel-shim/src/main/java/datadog/opentelemetry/shim/trace/OtelConventions.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
import static datadog.trace.api.DDTags.ERROR_MSG;
import static datadog.trace.api.DDTags.ERROR_STACK;
import static datadog.trace.api.DDTags.ERROR_TYPE;
import static datadog.trace.api.DDTags.SPAN_EVENTS;
import static datadog.trace.bootstrap.instrumentation.api.Tags.SPAN_KIND;
import static datadog.trace.bootstrap.instrumentation.api.Tags.SPAN_KIND_CLIENT;
import static datadog.trace.bootstrap.instrumentation.api.Tags.SPAN_KIND_CONSUMER;
Expand Down Expand Up @@ -136,11 +135,12 @@ public static void applyNamingConvention(AgentSpan span) {
}
}

public static void setEventsAsTag(AgentSpan span, List<OtelSpanEvent> events) {
public static void recordSpanEvents(AgentSpan span, List<OtelSpanEvent> events) {
if (events == null || events.isEmpty()) {
return;
}
span.setTag(SPAN_EVENTS, OtelSpanEvent.toTag(events));

span.addSpanEvents(events);

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve events for pre-serialization span observers

When OpenTelemetry spans record events, this now only stores them in DDSpan.spanEvents; the legacy events tag is not materialized until processTagsAndBaggage runs during payload mapping. Trace interceptors and test/custom writers such as ListWriter inspect span.getTags() before that mapping step, so spans with addEvent/recordException now appear to have no events there, whereas the previous setTag(SPAN_EVENTS, ...) made them visible immediately. Please keep the tag populated or otherwise materialize structured events before pre-serialization observers read the span tags.

Useful? React with 👍 / 👎.

}

public static void applySpanEventExceptionAttributesAsTags(
Expand Down
4 changes: 2 additions & 2 deletions ...a-agent/agent-otel/otel-shim/src/main/java/datadog/opentelemetry/shim/trace/OtelSpan.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import static datadog.opentelemetry.shim.trace.OtelConventions.applyNamingConvention;
import static datadog.opentelemetry.shim.trace.OtelConventions.applyReservedAttribute;
import static datadog.opentelemetry.shim.trace.OtelConventions.applySpanEventExceptionAttributesAsTags;
import static datadog.opentelemetry.shim.trace.OtelConventions.setEventsAsTag;
import static datadog.opentelemetry.shim.trace.OtelConventions.recordSpanEvents;
import static datadog.opentelemetry.shim.trace.OtelSpanEvent.EXCEPTION_SPAN_EVENT_NAME;
import static datadog.opentelemetry.shim.trace.OtelSpanEvent.initializeExceptionAttributes;
import static datadog.trace.bootstrap.instrumentation.api.AgentTracer.activateSpan;
Expand Down Expand Up @@ -179,7 +179,7 @@ public AgentSpan asAgentSpan() {
@Override
public void onSpanFinished() {
applyNamingConvention(this.delegate);
setEventsAsTag(this.delegate, this.events);
recordSpanEvents(this.delegate, this.events);
}

private static class NoopSpan implements Span {
Expand Down
86 changes: 57 additions & 29 deletions ...nt/agent-otel/otel-shim/src/main/java/datadog/opentelemetry/shim/trace/OtelSpanEvent.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,20 @@

import datadog.trace.api.time.SystemTimeSource;
import datadog.trace.api.time.TimeSource;
import datadog.trace.bootstrap.instrumentation.api.AgentSpanEvent;
import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.common.AttributesBuilder;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import javax.annotation.Nonnull;

public class OtelSpanEvent {
public class OtelSpanEvent implements AgentSpanEvent {
public static final String EXCEPTION_SPAN_EVENT_NAME = "exception";
public static final AttributeKey<String> EXCEPTION_MESSAGE_ATTRIBUTE_KEY =
AttributeKey.stringKey("exception.message");
Expand All @@ -26,33 +28,69 @@ public class OtelSpanEvent {
private static TimeSource timeSource = SystemTimeSource.INSTANCE;

private final String name;
private final String attributes;
private final Attributes attributes;

/** Event timestamp in nanoseconds. */
private final long timestamp;

public OtelSpanEvent(String name, Attributes attributes) {
this.name = name;
this.attributes = AttributesJsonParser.toJson(attributes);
this.timestamp = OtelSpanEvent.timeSource.getCurrentTimeNanos();
this(name, attributes, OtelSpanEvent.timeSource.getCurrentTimeNanos());
}

public OtelSpanEvent(String name, Attributes attributes, long timestamp, TimeUnit unit) {
this(name, attributes, unit.toNanos(timestamp));
}

private OtelSpanEvent(String name, Attributes attributes, long timestampNanos) {
this.name = name;
this.attributes = AttributesJsonParser.toJson(attributes);
this.timestamp = unit.toNanos(timestamp);
this.attributes = attributes;
this.timestamp = timestampNanos;
}

@Nonnull
public static String toTag(List<OtelSpanEvent> events) {
StringBuilder builder = new StringBuilder("[");
for (OtelSpanEvent event : events) {
if (builder.length() > 1) {
builder.append(',');
}
builder.append(event.toJson());
@Override
public String name() {
return this.name;
}

@Override
public long timeNanos() {
return this.timestamp;
}

/**
* Exposes the event attributes as typed values for native (V1) encoding. OpenTelemetry attribute
* values are already {@link String}, {@link Boolean}, {@link Long}, {@link Double} or a {@link
* List} of those, so they are passed through unchanged.
*/
@Override
public Map<String, Object> attributes() {
if (this.attributes == null || this.attributes.isEmpty()) {
return Collections.emptyMap();
}
return builder.append(']').toString();
Map<String, Object> map = new LinkedHashMap<>(this.attributes.size());

@mcculls mcculls Jul 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is order of entries important? If not then this could be a HashMap which uses less space

this.attributes.forEach((key, value) -> map.put(key.getKey(), value));
return map;
}

@Override
public CharSequence toJson() {
StringBuilder builder = new StringBuilder(128);
builder
.append('{')
.append("\"time_unix_nano\":")
.append(this.timestamp)
.append(',')
.append("\"name\":")
.append('"')
.append(this.name)
.append('"');
Comment on lines +83 to +86

String attributesJson = AttributesJsonParser.toJson(this.attributes);
if (!attributesJson.isEmpty()) {
builder.append(",\"attributes\":").append(attributesJson);
}

return builder.append('}').toString();
}

/**
Expand Down Expand Up @@ -170,24 +208,14 @@ public static void setTimeSource(TimeSource newTimeSource) {
timeSource = newTimeSource;
}

public String toJson() {
StringBuilder builder =
new StringBuilder(
"{\"time_unix_nano\":" + this.timestamp + ",\"name\":\"" + this.name + "\"");
if (!this.attributes.isEmpty()) {
builder.append(",\"attributes\":").append(this.attributes);
}
return builder.append('}').toString();
}

@Override
public String toString() {
return "OtelSpanEvent{timestamp="
+ this.timestamp
+ ", name='"
+ this.name
+ "', attributes='"
+ "', attributes="
+ this.attributes
+ "'}";
+ '}';
}
}
5 changes: 5 additions & 0 deletions ...rumentation-testing/src/main/groovy/datadog/trace/agent/test/TrackingSpanDecorator.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import datadog.trace.api.interceptor.MutableSpan
import datadog.trace.bootstrap.CallDepthThreadLocalMap
import datadog.trace.bootstrap.instrumentation.api.AgentSpan
import datadog.trace.bootstrap.instrumentation.api.AgentSpanContext
import datadog.trace.bootstrap.instrumentation.api.AgentSpanEvent
import datadog.trace.bootstrap.instrumentation.api.AgentSpanLink
import datadog.trace.core.DDSpan

Expand Down Expand Up @@ -405,6 +406,10 @@ class TrackingSpanDecorator implements AgentSpan {
delegate.addLink(link)
}

void addSpanEvents(List<? extends AgentSpanEvent> events) {
delegate.addSpanEvents(events)
}

@Override
AgentSpan setMetaStruct(String field, Object value) {
return delegate.setMetaStruct(field, value)
Expand Down
73 changes: 14 additions & 59 deletions dd-trace-core/src/main/java/datadog/trace/common/writer/ddagent/TraceMapperV1.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
import datadog.trace.api.ProcessTags;
import datadog.trace.api.TagMap;
import datadog.trace.api.sampling.SamplingMechanism;
import datadog.trace.bootstrap.instrumentation.api.AgentSpanEvent;
import datadog.trace.bootstrap.instrumentation.api.AgentSpanLink;
import datadog.trace.bootstrap.instrumentation.api.InstrumentationTags;
import datadog.trace.bootstrap.instrumentation.api.Tags;
Expand Down Expand Up @@ -89,7 +90,7 @@ public void map(List<? extends CoreSpan<?>> trace, Writable writable) {
}

CoreSpan<?> firstSpan = trace.get(0);
firstSpan.processTagsAndBaggageWithStructuredLinks(spanMetadata);
firstSpan.processTagsAndBaggageWithStructuredLinksAndEvents(spanMetadata);
Metadata firstSpanMeta = spanMetadata.metadata;

// encoded fields: 1..7, but skipping #5, as not required by tracers and set by the agent.
Expand Down Expand Up @@ -128,7 +129,7 @@ private void encodeSpans(Writable writable, int fieldId, List<? extends CoreSpan
Metadata meta = spanMetadata.metadata;
for (CoreSpan<?> span : spans) {
if (meta == null) {
span.processTagsAndBaggageWithStructuredLinks(spanMetadata);
span.processTagsAndBaggageWithStructuredLinksAndEvents(spanMetadata);
meta = spanMetadata.metadata;
}
TagMap tags = meta.getTags();
Expand Down Expand Up @@ -162,7 +163,7 @@ private void encodeSpans(Writable writable, int fieldId, List<? extends CoreSpan
// links = 11, a collection of links to other spans
encodeSpanLinks(writable, 11, meta.getSpanLinks());
// events = 12, a collection of events that occurred during this span
encodeSpanEvents(writable, 12, tags.getObject(DDTags.SPAN_EVENTS));
encodeSpanEvents(writable, 12, meta.getSpanEvents());

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 3, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve legacy events tags in V1 payloads

For spans that still provide span events through the existing events tag as a List<Map<...>> (for example custom instrumentation or a trace interceptor calling setTag("events", eventPayload)), the V1 mapper now reads only Metadata.getSpanEvents(), so field 12 is emitted empty and the legacy tag is flattened as a normal span attribute instead of native events. The previous mapper explicitly encoded tags.getObject(DDTags.SPAN_EVENTS), so please keep that fallback when the structured event list is empty or migrate all producers before dropping it.

Useful? React with 👍 / 👎.

// env = 13, the optional string environment of this span
encodeString(writable, 13, tags.getString(Tags.ENV));
// version = 14, the optional string version of this span
Expand Down Expand Up @@ -200,48 +201,21 @@ private void encodeSpanLinks(
}
}

private void encodeSpanEvents(Writable writable, int fieldId, Object eventsObject) {
private void encodeSpanEvents(
Writable writable, int fieldId, List<? extends AgentSpanEvent> events) {
writable.writeInt(fieldId);
if (!(eventsObject instanceof List) || ((List<?>) eventsObject).isEmpty()) {
if (events == null || events.isEmpty()) {
writable.startArray(0);
return;
}

List<?> events = (List<?>) eventsObject;
int encodableCount = 0;
for (Object event : events) {
if (isEncodableSpanEvent(event)) {
encodableCount++;
}
}
writable.startArray(encodableCount);
for (Object event : events) {
if (!(event instanceof Map)) {
continue;
}
Map<?, ?> eventMap = (Map<?, ?>) event;
Long timeUnixNano = asLong(eventMap.get("time_unix_nano"));
Object nameObject = eventMap.get("name");
if (timeUnixNano == null || nameObject == null) {
continue;
}

Map<?, ?> attributes =
eventMap.get("attributes") instanceof Map ? (Map<?, ?>) eventMap.get("attributes") : null;

writable.startArray(events.size());
for (AgentSpanEvent event : events) {
writable.startMap(3);
encodeLong(writable, 1, timeUnixNano);
encodeString(writable, 2, String.valueOf(nameObject));
encodeEventAttributes(writable, 3, attributes);
}
}

private boolean isEncodableSpanEvent(Object event) {
if (!(event instanceof Map)) {
return false;
encodeLong(writable, 1, event.timeNanos());
encodeString(writable, 2, event.name());
encodeEventAttributes(writable, 3, event.attributes());
}
Map<?, ?> eventMap = (Map<?, ?>) event;
return eventMap.get("name") != null && asLong(eventMap.get("time_unix_nano")) != null;
}

private void encodeEventAttributes(Writable writable, int fieldId, Map<?, ?> attrs) {
Expand Down Expand Up @@ -340,20 +314,6 @@ private boolean isIntegralNumber(Number number) {
return !(number instanceof Float || number instanceof Double);
}

private Long asLong(Object value) {
if (value instanceof Number) {
return ((Number) value).longValue();
}
if (value instanceof CharSequence) {
try {
return Long.parseLong(value.toString());
} catch (NumberFormatException ignored) {
return null;
}
}
return null;
}

private void encodeSpanAttributes(
Writable writable, int fieldId, Metadata meta, Map<String, Object> metaStruct) {
TagMap tags = meta.getTags();
Expand All @@ -364,9 +324,7 @@ private void encodeSpanAttributes(
boolean writeTopLevel = meta.topLevel();
int tagCount = 0;
for (TagMap.EntryReader entry : tags) {
if (!DDTags.SPAN_EVENTS.equals(entry.tag())) {

@mcculls mcculls Jul 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a user sets a DDTags.SPAN_EVENTS tag this will now write them as as a plain tag - is that acceptable?

tagCount += getFlatAttributeCount(entry);
}
tagCount += getFlatAttributeCount(entry);
}

writable.writeInt(fieldId);
Expand All @@ -387,9 +345,6 @@ private void encodeSpanAttributes(
}

for (TagMap.EntryReader entry : tags) {
if (DDTags.SPAN_EVENTS.equals(entry.tag())) {
continue;
}
writeFlattenedTagAttribute(writable, entry);
}
if (writeHttpStatus) {
Expand Down Expand Up @@ -487,7 +442,7 @@ private void writeAttribute(Writable writable, String key, Object value) {
return;
}

if (!(value instanceof String) && value != null) {
if (!(value instanceof CharSequence) && value != null) {
log.debug("Not a string value for key: {}, value: {}", key, value);
}
writable.writeInt(VALUE_TYPE_STRING);
Expand Down
6 changes: 3 additions & 3 deletions dd-trace-core/src/main/java/datadog/trace/core/CoreSpan.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,13 +100,13 @@ default String getSpanKindString() {

/**
* Variant of {@link #processTagsAndBaggage(MetadataConsumer)} for protocols that serialize span
* links as first-class structured data rather than tags. Baggage tag injection still follows the
* tracer configuration.
* links and span events as first-class structured data rather than flattening them into tags.
* Baggage tag injection still follows the tracer configuration.
*
* <p>To simplify tests, by default delegating to {@link
* #processTagsAndBaggage(MetadataConsumer)}.
*/
default void processTagsAndBaggageWithStructuredLinks(MetadataConsumer consumer) {
default void processTagsAndBaggageWithStructuredLinksAndEvents(MetadataConsumer consumer) {
processTagsAndBaggage(consumer);
}

Expand Down
Loading
Loading