Skip to content

Commit 8f6b8d0

Browse files
OmarAlJarrahclaude
andcommitted
feat(pipeline): complete the async HttpTracer lifecycle with AsyncOperationTracingPolicy
The async pipeline already emitted attempt-level HttpTracer events (AsyncRetryPolicy's attempt_started/attempt_failed and AsyncRedirectPolicy's request_url_resolved) but never the operation lifecycle, so an async operator saw attempts and redirect hops with no operation_started bracket and no final operation_succeeded/operation_failed. The per-operation lifecycle had been extracted into the sync-only OperationTracingPolicy without an async counterpart, leaving the async tracer stream permanently incomplete. The HttpTracer callbacks are synchronous and transport-agnostic by contract, so there is no async barrier to emitting them — only a missing policy. - Add AsyncOperationTracingPolicy (async twin of OperationTracingPolicy) at the outermost Stage.OPERATION; it awaits only the downstream send. - Wire it as the outermost policy in default_async_pipeline and export it from pipeline.policies. - Cover it: the lifecycle fires exactly once and reflects the final outcome across async retry (success-after-failure and exhaustion) and redirect (success and late-hop failure), the tracing-disabled path, and the default async stack wiring. - Regenerate the public API surface baseline. - Correct the docs that described the async stack as carrying no tracing (CHANGELOG, README, CLAUDE.md, tracing module docstring): only the per-attempt OpenTelemetry span policy and structured logging remain sync-only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent bf090db commit 8f6b8d0

10 files changed

Lines changed: 312 additions & 22 deletions

File tree

CHANGELOG.md

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -41,13 +41,16 @@ Changed).
4141
- **Client-identity policy** (`pipeline.policies.client_identity`, plus its
4242
async twin). Sets a consistent `User-Agent` / client-identity header derived
4343
from the configured application id and SDK version.
44-
- **Per-operation tracing policy** (`OperationTracingPolicy` in
45-
`pipeline.policies.tracing_policy`, with a new outermost `Stage.OPERATION`).
46-
Emits the per-operation `HttpTracer` lifecycle (`operation_started`, then
47-
exactly one `operation_succeeded` / `operation_failed`) from outside the retry
48-
and redirect wrappers, so the reported outcome reflects the final result of
49-
the whole call rather than a single attempt or hop. Sync-only, in line with
50-
the rest of the tracing stack; the async pipeline carries no tracing.
44+
- **Per-operation tracing policy** (`OperationTracingPolicy` and its async twin
45+
`AsyncOperationTracingPolicy`, with a new outermost `Stage.OPERATION`). Emits
46+
the per-operation `HttpTracer` lifecycle (`operation_started`, then exactly
47+
one `operation_succeeded` / `operation_failed`) from outside the retry and
48+
redirect wrappers, so the reported outcome reflects the final result of the
49+
whole call rather than a single attempt or hop. Both `default_pipeline` and
50+
`default_async_pipeline` wire it, so the async stack now reports the same
51+
lifecycle alongside the attempt-level events its retry / redirect policies
52+
already emit. Only `TracingPolicy`'s per-attempt OpenTelemetry span policy
53+
remains sync-only.
5154
- **HTTP tracer** (`instrumentation.http_tracer`). An adapter-style tracer base
5255
whose per-event methods default to no-ops, so a subclass overrides only the
5356
events it cares about. Wired through the tracing policy for span emission.
@@ -153,9 +156,11 @@ The following were intentionally left out of this round and are **not** included
153156
errors themselves.
154157
- **`sendfile` fast-path** — file bodies are streamed via the existing
155158
`iter_bytes` path; no zero-copy `sendfile` transport optimisation was added.
156-
- **Async tracing / logging** — the tracing and logging policies (including the
157-
new `OperationTracingPolicy`) ship sync-only; `default_async_pipeline` carries
158-
no tracing, and async callers handle per-operation observability themselves.
159+
- **Async OpenTelemetry spans / logging** — the per-attempt span policy
160+
(`TracingPolicy`) and `LoggingPolicy` ship sync-only, so
161+
`default_async_pipeline` emits the per-operation `HttpTracer` lifecycle and
162+
attempt-level events but no OpenTelemetry spans or structured request /
163+
response logs.
159164
- **MCP support** — no Model Context Protocol integration is included.
160165
- **Java SDK items** — the Java counterpart lives in a separate repository and
161166
was out of scope here.

CLAUDE.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,7 @@ python-sdk/
132132
│ │ │ │
133133
│ │ │ ├── policies/ # redirect, idempotency, retry, set_date,
134134
│ │ │ │ # client_identity, logging, tracing
135-
│ │ │ │ # (async twins only for the first five)
135+
│ │ │ │ # (async twins for all but logging and per-attempt tracing)
136136
│ │ │ └── step/ # PipelineStep, StepMetadata
137137
│ │ ├── client/ # HttpClient + AsyncHttpClient Protocols
138138
│ │ ├── config/ # Configuration
@@ -207,13 +207,14 @@ Layered, bottom-up:
207207
`Pipeline` / `AsyncPipeline` run an ordered set of policies grouped into
208208
`Stage`s via `StagedPipelineBuilder`. Shipped policies: redirect,
209209
idempotency, retry, set-date, client-identity, logging, tracing, and
210-
operation-tracing. Async twins under `pipeline/policies/` exist only for
211-
redirect, idempotency, retry, set-date, and client-identity; logging,
212-
tracing, and operation-tracing are sync-only.
210+
operation-tracing. Async twins exist for redirect, idempotency, retry,
211+
set-date, client-identity, and operation-tracing; logging and the
212+
per-attempt tracing policy are sync-only.
213213
`default_pipeline()` / `default_async_pipeline()` assemble the standard
214214
stack in the order operation-tracing → redirect → idempotency → retry →
215215
set-date → client-identity → [auth] → logging → tracing (the async
216-
pipeline omits the tracing and logging policies). The lower-level
216+
pipeline keeps operation-tracing but omits logging and the per-attempt
217+
tracing span). The lower-level
217218
`pipeline/step/PipelineStep` Protocol (`(input, context) -> output`) plus
218219
`StepMetadata` remain for custom composition.
219220
5. **`client/HttpClient`** — single-method Protocol

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -176,7 +176,7 @@ Bottom-up, the layers are:
176176
| `http.webhooks` | `WebhookVerifier`, `InvalidWebhookSignatureError` — HMAC signature verification with timestamp tolerance |
177177
| `pagination` | `Page`, `Paginator` / `AsyncPaginator`, `PaginationStrategy` (`CursorStrategy`, `PageNumberStrategy`, `LinkHeaderStrategy`) |
178178
| `pipeline` | `Pipeline`, `AsyncPipeline`, `Policy` ABC, `Stage` enum, `StagedPipelineBuilder`, `default_pipeline()` |
179-
| `pipeline.policies` | `RedirectPolicy`, `IdempotencyPolicy`, `RetryPolicy`, `SetDatePolicy`, `ClientIdentityPolicy`, `LoggingPolicy`, `OperationTracingPolicy`, `TracingPolicy` (async twins for all but logging/tracing) |
179+
| `pipeline.policies` | `RedirectPolicy`, `IdempotencyPolicy`, `RetryPolicy`, `SetDatePolicy`, `ClientIdentityPolicy`, `LoggingPolicy`, `OperationTracingPolicy`, `TracingPolicy` (async twins for all but `LoggingPolicy` and `TracingPolicy`) |
180180
| `client` | `HttpClient` and `AsyncHttpClient` Protocols |
181181
| `serde` | `Serde`, `Serializer`, `Deserializer` Protocols + `JsonSerde` reference impl |
182182
| `instrumentation` | `ClientLogger`, `UrlRedactor`, `Tracer`, `Span`, `InstrumentationContext`, `contextvars` correlation helpers, noop singletons |

packages/dexpace-sdk-core/src/dexpace/sdk/core/pipeline/defaults.py

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
from .policies.async_redirect import AsyncRedirectPolicy
1414
from .policies.async_retry import AsyncRetryPolicy
1515
from .policies.async_set_date import AsyncSetDatePolicy
16+
from .policies.async_tracing_policy import AsyncOperationTracingPolicy
1617
from .policies.client_identity import ClientIdentityPolicy
1718
from .policies.idempotency import IdempotencyPolicy
1819
from .policies.logging_policy import LoggingPolicy
@@ -106,11 +107,19 @@ def default_async_pipeline(
106107
) -> AsyncStagedPipelineBuilder:
107108
"""Async twin of `default_pipeline`.
108109
109-
Mirrors the sync version's stack minus logging/tracing, which currently
110-
only ship as sync policies. Async-side observability lives on the caller's
111-
side until async versions land.
110+
Mirrors the sync version's stack. `AsyncOperationTracingPolicy` brackets
111+
the whole operation from the outermost stage so the per-operation
112+
``HttpTracer`` lifecycle (``operation_started`` / ``operation_succeeded`` /
113+
``operation_failed``) fires once and reflects the final outcome — completing
114+
the attempt-level events the async retry and redirect policies already emit
115+
through the same per-operation tracer. The per-attempt OpenTelemetry span
116+
policy (`TracingPolicy`) and `LoggingPolicy` ship sync-only, so the async
117+
stack omits those two.
112118
"""
113119
builder = AsyncStagedPipelineBuilder(client)
120+
# Sorts to Stage.OPERATION (outermost), bracketing every hop / attempt so
121+
# the per-operation lifecycle fires once on the final outcome.
122+
builder.append(AsyncOperationTracingPolicy())
114123
builder.append(redirect or AsyncRedirectPolicy())
115124
builder.append(idempotency or AsyncIdempotencyPolicy())
116125
builder.append(retry or AsyncRetryPolicy())

packages/dexpace-sdk-core/src/dexpace/sdk/core/pipeline/policies/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
from .async_redirect import AsyncRedirectPolicy
1212
from .async_retry import AsyncRetryPolicy
1313
from .async_set_date import AsyncSetDatePolicy
14+
from .async_tracing_policy import AsyncOperationTracingPolicy
1415
from .client_identity import ClientIdentityPolicy, default_user_agent
1516
from .idempotency import IdempotencyPolicy
1617
from .logging_policy import LoggingPolicy
@@ -22,6 +23,7 @@
2223
__all__ = [
2324
"AsyncClientIdentityPolicy",
2425
"AsyncIdempotencyPolicy",
26+
"AsyncOperationTracingPolicy",
2527
"AsyncRedirectPolicy",
2628
"AsyncRetryPolicy",
2729
"AsyncSetDatePolicy",
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# Copyright (c) 2026 dexpace and Omar Aljarrah.
2+
# Licensed under the MIT License. See LICENSE.md in the repository root for details.
3+
4+
"""Async twin of `OperationTracingPolicy`."""
5+
6+
from __future__ import annotations
7+
8+
from typing import TYPE_CHECKING, ClassVar, Literal
9+
10+
from ..async_policy import AsyncPolicy
11+
from ..stage import Stage
12+
from .redirect import resolve_http_tracer
13+
14+
if TYPE_CHECKING:
15+
from ...http.request.request import Request
16+
from ...http.response.async_response import AsyncResponse
17+
from ..context import PipelineContext
18+
19+
20+
class AsyncOperationTracingPolicy(AsyncPolicy):
21+
"""Async variant of `OperationTracingPolicy`.
22+
23+
Emits the per-operation ``HttpTracer`` lifecycle around the whole async
24+
call. Placed at `Stage.OPERATION`, outside the redirect and retry wrappers,
25+
so its single ``send`` brackets every hop and attempt: it emits
26+
``operation_started`` before dispatching the chain and exactly one of
27+
``operation_succeeded`` / ``operation_failed`` once the chain unwinds, so
28+
the operation outcome reflects what the caller observes rather than the
29+
result of the first attempt.
30+
31+
This completes the async ``HttpTracer`` lifecycle. `AsyncRetryPolicy` and
32+
`AsyncRedirectPolicy` already emit the attempt-level events and
33+
``request_url_resolved`` through the same per-operation tracer (resolved
34+
via ``resolve_http_tracer`` and cached in ``ctx.data``), so without this
35+
policy the async stack reports attempts but never the operation outcome.
36+
The tracer callbacks are synchronous, so the body matches the sync twin
37+
apart from the ``await`` on the downstream send.
38+
39+
Disable per-call by setting ``ctx.options["tracing_enabled"] = False``.
40+
41+
Attributes:
42+
STAGE: Pinned to `Stage.OPERATION` at the type level so mis-slotting is
43+
caught by ``mypy``.
44+
"""
45+
46+
STAGE: ClassVar[Literal[Stage.OPERATION]] = Stage.OPERATION
47+
__slots__ = ()
48+
49+
async def send(self, request: Request, ctx: PipelineContext) -> AsyncResponse:
50+
"""Bracket the downstream chain with the per-operation lifecycle.
51+
52+
Args:
53+
request: Outgoing request.
54+
ctx: Pipeline context, forwarded unchanged.
55+
56+
Returns:
57+
The response from the downstream chain.
58+
"""
59+
if not ctx.options.get("tracing_enabled", True):
60+
return await self.next.send(request, ctx)
61+
http_tracer = resolve_http_tracer(ctx)
62+
http_tracer.operation_started()
63+
try:
64+
response = await self.next.send(request, ctx)
65+
except BaseException as err:
66+
http_tracer.operation_failed(err)
67+
raise
68+
http_tracer.operation_succeeded()
69+
return response
70+
71+
72+
__all__ = ["AsyncOperationTracingPolicy"]

packages/dexpace-sdk-core/src/dexpace/sdk/core/pipeline/policies/tracing_policy.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,10 @@
3030
the sentinel trace ids. Disable either per-call by setting
3131
``ctx.options["tracing_enabled"] = False``.
3232
33-
Both policies are sync-only, in line with the rest of the tracing and logging
34-
stack; ``default_async_pipeline`` carries no tracing, so async callers own
35-
per-operation observability on their side.
33+
`OperationTracingPolicy` has an async twin, `AsyncOperationTracingPolicy`,
34+
wired into ``default_async_pipeline`` so the async stack reports the same
35+
per-operation lifecycle. `TracingPolicy`'s per-attempt OpenTelemetry span
36+
machinery is sync-only and has no async counterpart.
3637
"""
3738

3839
from __future__ import annotations

packages/dexpace-sdk-core/tests/pipeline/test_defaults.py

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,3 +126,14 @@ def test_default_async_pipeline_returns_builder() -> None:
126126
def test_default_async_pipeline_builds_async_pipeline() -> None:
127127
pipeline = default_async_pipeline(_AsyncStubTransport()).build()
128128
assert isinstance(pipeline, AsyncPipeline)
129+
130+
131+
def test_default_async_pipeline_wires_operation_tracing() -> None:
132+
from dexpace.sdk.core.pipeline.policies import AsyncOperationTracingPolicy
133+
134+
builder = default_async_pipeline(_AsyncStubTransport())
135+
# Mirrors the sync default: the per-operation lifecycle policy occupies the
136+
# outermost OPERATION pillar, so async callers get the same
137+
# operation_started / operation_succeeded / operation_failed bracket around
138+
# the attempt-level events the retry/redirect policies already emit.
139+
assert isinstance(builder._pillars[Stage.OPERATION], AsyncOperationTracingPolicy)

0 commit comments

Comments
 (0)