[Bug]: gen_ai.response.model not reliably populated for LangChain Responses API (useResponsesApi: true)

[JacksonWeber]

Component

genai / langchain

Description

For the LangChain Responses API code path (ChatOpenAI with useResponsesApi: true, including Foundry deployments), gen_ai.response.model is not reliably populated on emitted spans, and there is no test coverage for the RAPI shape.

This is the third scenario from #128. PRs #127 and #98 closed the two Chat Completions API (CAPI) scenarios; this issue tracks the RAPI-specific work.

Root cause

getResponseModel in src/genai/instrumentations/langchain/utils.ts (lines 462–475) looks up response_metadata.model_name (v0 + v1) and llmOutput.{model_name,model}, but never response_metadata.model.

LangChain's RAPI converter (langchain-ai/langchainjs → libs/providers/langchain-openai/src/converters/responses.ts) currently sets both:

const response_metadata = {
  model_provider: "openai",
  model: response.model,
  // for backwards compat with chat completion calls
  model_name: response.model,
};

So today the model_name fallback works incidentally. The risks:

1. The model_name alias is explicitly labeled "backwards compat" — LangChain can drop it at any minor version.
2. Other RAPI-capable providers (e.g. @langchain/perplexity) and downstream Foundry-specific patches may set only model.
3. Because there is no test fixture for the RAPI run shape, a future regression would be silent.

Scope note (per PengF (@fpfp100) on #128)

"gen_ai.response.model is not in A365 Observability schema. … For #3, not required for A365 Observability schema, it is a no op, but might need for Azure Foundry scenario."

The distro is vendor-neutral and is consumed via OTLP / Azure Monitor / Foundry / console exporters. Spans must be correct at emission; A365 ingestion-side filtering is a separate concern and a parallel A365 schema-extension request is needed to surface the attribute in the A365 portal.

Expected behavior

For an RAPI run (ChatOpenAI or AzureChatOpenAI with useResponsesApi: true):

• gen_ai.request.model = the deployment alias / model kwarg (already correct).
• gen_ai.response.model = response.model from the Responses-API result, regardless of whether LangChain exposes it via response_metadata.model, response_metadata.model_name, or both.
• gen_ai.response.id = response.id (already covered by PR fix(langchain): split request/response model and emit gen_ai.response.id #127; needs an RAPI-shape regression test).
• gen_ai.usage.input_tokens / gen_ai.usage.output_tokens populated from the RAPI usage_metadata shape.
• gen_ai.provider.name = "openai" (from response_metadata.model_provider).

Proposed fix

1. Broaden getResponseModel to read response_metadata.model before response_metadata.model_name on both v0 (message.kwargs.response_metadata) and v1 (message.response_metadata) paths. Keep the model_name and llmOutput.* lookups as fallbacks.
2. Add unit tests in test/internal/unit/genai/langchain/utils.test.ts:
   • RAPI run with response_metadata.model only (no model_name).
   • RAPI run with both model and model_name (current LangChain shape).
   • RAPI run with v0 kwargs.response_metadata.model only.
   • RAPI run with response_metadata.id → asserts gen_ai.response.id.
   • RAPI run with usage_metadata → asserts input/output token attributes.
3. Add a tracer-level integration test in test/internal/unit/genai/langchain/tracer.test.ts using a Run fixture mirroring the shape langchain-openai/converters/responses.ts produces, driven through LangChainTracer. Assert gen_ai.operation.name, gen_ai.request.model, gen_ai.response.model, gen_ai.response.id, token attrs, and provider name.
4. (Optional) Add samples/langchain-responses-api/ for customer-facing repro and smoke validation.

Validation

Use the console exporter (enableConsoleExporters: true) or an OTLP exporter — not the A365 portal, which currently drops gen_ai.response.model at ingestion.

Out of scope

• GenAI semconv audit (gen_ai.response.finish_reasons, gen_ai.output.type, request params).
• Broader invoke_agent detection (AgentExecutor, createReactAgent, tag-based).
• A365 ingestion schema extension to surface gen_ai.response.model / gen_ai.response.id in the A365 portal — separate workstream.
• Retiring the isAzureChatOpenAIRun workaround in setModelAttribute, which is tied to upstream langchain-ai/langchainjs#10886 ([Bug]: LangChain instrumentation mis-attributes response model into gen_ai.request.model; gen_ai.response.model never set #128 scenario 1).

Environment

• Package: @microsoft/opentelemetry-distro-javascript@1.0.2
• Node.js: 20.x
• LangChain: @langchain/openai with ChatOpenAI({ useResponsesApi: true })

References

• Parent issue: [Bug]: LangChain instrumentation mis-attributes response model into gen_ai.request.model; gen_ai.response.model never set #128
• Prior fix (CAPI scenarios): fix(langchain): split request/response model and emit gen_ai.response.id #127
• Upstream RAPI converter: langchain-ai/langchainjs/libs/providers/langchain-openai/src/converters/responses.ts

[github-actions]

cc Hector Hernandez (@hectorhdzg) Jackson Weber (@JacksonWeber) Leighton Chen (@lzchen) Nev (@MSNev) Radhika Gupta (@rads-1996) Jeremy Voss (@jeremydvoss)

[JacksonWeber]

Required work items are complete.