[Feature]: Add proper token usage extraction for LangChain tracer to match langchain-azure's normalization #158

Component

Langchain

Problem Statement

The LangChain tracer in opentelemetry-distro-python currently does not robustly extract or normalize token usage information from LLM outputs the way langchain-azure-ai does. As a result, token counts (input/output/total) in traces may be missing, incomplete, or only present when certain providers report metadata directly. By contrast, langchain-azure-ai performs detailed normalization in its on_llm_end callback, extracting, canonicalizing, and attaching token usage from many provider formats and propagating them to agent spans and OTEL metrics. This leaves a feature/robustness gap and reduces the utility of traces and metrics for langchain/langgraph agents monitored via the distro.

Proposed Solution

Match or exceed the normalization logic from langchain-azure-ai tracer in the distro's LangChain tracer:

• In the callback or utility responsible for recording LLM/agent/chain usage, call a normalization routine similar to langchain-azure-ai’s _resolve_usage_from_llm_output, passing the llm_output and generation objects.
• Look for token usage fields in all locations currently covered by langchain-azure-ai (llm_output['token_usage'], llm_output['usage'], message.usage_metadata, generation_info, etc.), matching input/output/total/other provider fields as langchain-azure.
• If found, attach gen_ai.usage.input_tokens and gen_ai.usage.output_tokens to the span, and consider propagating counts up to parent agent spans as done in langchain-azure-ai’s _accumulate_usage_to_agent_spans.
• Optionally, consider supporting total_tokens internally even if not exposed as an OTel attribute, for feature parity.
• Consider aligning SDK metrics reporting to the behavior in langchain-azure-ai as well.

References:

• https://github.com/langchain-ai/langchain-azure/blob/main/libs/azure-ai/langchain_azure_ai/callbacks/tracers/inference_tracing.py
• https://github.com/langchain-ai/langchain-azure/blob/main/libs/azure-ai/langchain_azure_ai/chat_models/inference.py
• See detailed flow/pseudocode in this issue for additional extraction/normalization scenarios.

Alternatives Considered

• Accept missing tokens for non-OpenAI providers (current behavior, but leads to incomplete observability).

Additional Context

langchain-azure tracer flow summary:

The tracer does not count tokens itself. It reads token usage reported by the model/provider via LangChain’s LLMResult, normalizes it, writes input/output counts onto the OpenTelemetry span, records token usage metrics, and propagates the counts to parent agent spans.

Relevant logic and code flow (langchain-azure):

• inference_tracing.py L2274-L2314, L2337-L2338: Token usage extraction and span attribute setting
• _resolve_usage_from_llm_output, _extract_usage_tokens for normalization
• Reference: https://github.com/langchain-ai/langchain-azure/blob/main/libs/azure-ai/langchain_azure_ai/callbacks/tracers/inference_tracing.py

Current distro implementation only extracts tokens from a small set of output locations and does not cover all variants, nor does it propagate/normalize across agent hierarchies or metric sinks. Improving parity here will produce more useful and robust observability for all instrumented models.