Skip to content

Endpoint aggregation drops cross-file-resolved SDK calls with sdk:// placeholder URLs (A5 cross-file factory invisible to benchmark) #141

Description

@AndresL230

Summary

The scanner detects an A5 cross-file factory call at the raw local.apiCalls layer, but it is dropped during endpoint aggregation, so it never appears in result.endpoints — the array the benchmark runner (and the webview) consume. The capability works; it's just invisible downstream. This surfaced while adding the A3/A5 benchmark fixtures (recost-dev/extension-benchmark#7): the factory-cross-file fixture had to be dropped because no must_detect endpoint could ever match.

Repro

node dist/cli/scan.js src/test/fixtures/a5/factory-direct --format json | jq '{apiCalls: .local.apiCalls, endpoints: (.endpoints|length)}'

(Run a warm-up scan first — the first invocation in a fresh shell returns empty due to a wasm parser cold-start.)

Observed:

  • local.apiCalls: 1consumer.ts:6, provider: "openai", methodSignature: "client.chat.completions.create", url: "sdk://openai/client.chat.completions.create", crossFileOriginclient-factory.ts.
  • endpoints: 0 (empty).

Contrast the same-file factory (extension-benchmark/factory-in-file), which emits an endpoint because its call resolves to a concrete url: "https://api.openai.com/v1/chat/completions".

Root cause

In src/scan-results.ts, mergeRemoteAndLocalEndpoints skips any call that fails shouldIncludeSynthetic:

  • scan-results.ts:476if (!shouldIncludeSynthetic(call)) continue;
  • shouldIncludeSynthetic (:403) gates on isHighConfidenceEndpointUrl(call.url).
  • isHighConfidenceEndpointUrl accepts only https?://, root-relative /…, or specific <dynamic:…> tokens. An sdk://… placeholder falls through to return false.

A cross-file factory call keeps the sdk://<provider>/<methodSignature> placeholder because the factory body (return new OpenAI()) contains no actual .create call to resolve a concrete provider URL from. So a call with a confidently known provider and methodSignature is discarded purely because its URL is a placeholder.

Why barrels survive but factory doesn't: a barrel fixture's leaf call (e.g. api.ts:6) lives in the same file as the SDK client, resolves to a concrete https:// URL, and anchors a synthetic endpoint; the cross-file consumer call then joins that group. The cross-file factory has no such concrete-URL anchor — its only call is the sdk:// one — so nothing is emitted.

Impact

  • A5 cross-file factory attribution — proven at the scanFiles layer by src/test/a5-factory-di-aliased.test.ts ("A5.audit.factory") — is not measurable through the benchmark and not surfaced in the endpoints view.
  • Blocks the factory-cross-file fixture in [Measurement] Add barrel + factory fixtures to benchmark corpus extension-benchmark#7 (only 6 of 7 shapes could ship; factory-in-file stands in as a same-file proxy).
  • In real scans, genuine cross-file factory/DI client usage that can't fold to a concrete URL is silently omitted from endpoints, undercounting cost/usage.

Proposed fix direction

Retain provider+methodSignature-confident calls even when the URL is an sdk:// placeholder. Options:

  1. Treat sdk:// URLs as high-confidence in shouldIncludeSynthetic/isHighConfidenceEndpointUrl when provider and methodSignature are both present, and key the synthetic endpoint on methodSignature (the runner's methodsEquivalent already matches on the SDK chain), or
  2. Resolve the sdk:// placeholder to the canonical provider URL during cross-file resolution (mirror what the same-file path produces).

Either keeps precision intact — these are confident SDK detections, not speculative URL guesses.

Acceptance criteria

  • node dist/cli/scan.js src/test/fixtures/a5/factory-direct --format json emits one endpoint: consumer.ts:6, provider: openai, methodSignature client.chat.completions.create.
  • No regression to existing benchmark precision/recall (npm run benchmark).
  • Follow-up: add the factory-cross-file fixture to extension-benchmark and include it in the baseline ([Measurement] Add barrel + factory fixtures to benchmark corpus extension-benchmark#7).

Links

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions