You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The scanner detects an A5 cross-file factory call at the raw local.apiCalls layer, but it is dropped during endpoint aggregation, so it never appears in result.endpoints — the array the benchmark runner (and the webview) consume. The capability works; it's just invisible downstream. This surfaced while adding the A3/A5 benchmark fixtures (recost-dev/extension-benchmark#7): the factory-cross-file fixture had to be dropped because no must_detect endpoint could ever match.
Contrast the same-file factory (extension-benchmark/factory-in-file), which emits an endpoint because its call resolves to a concrete url: "https://api.openai.com/v1/chat/completions".
Root cause
In src/scan-results.ts, mergeRemoteAndLocalEndpoints skips any call that fails shouldIncludeSynthetic:
scan-results.ts:476 — if (!shouldIncludeSynthetic(call)) continue;
shouldIncludeSynthetic (:403) gates on isHighConfidenceEndpointUrl(call.url).
isHighConfidenceEndpointUrl accepts only https?://, root-relative /…, or specific <dynamic:…> tokens. An sdk://… placeholder falls through to return false.
A cross-file factory call keeps the sdk://<provider>/<methodSignature> placeholder because the factory body (return new OpenAI()) contains no actual .create call to resolve a concrete provider URL from. So a call with a confidently known provider and methodSignature is discarded purely because its URL is a placeholder.
Why barrels survive but factory doesn't: a barrel fixture's leaf call (e.g. api.ts:6) lives in the same file as the SDK client, resolves to a concrete https:// URL, and anchors a synthetic endpoint; the cross-file consumer call then joins that group. The cross-file factory has no such concrete-URL anchor — its only call is the sdk:// one — so nothing is emitted.
Impact
A5 cross-file factory attribution — proven at the scanFiles layer by src/test/a5-factory-di-aliased.test.ts ("A5.audit.factory") — is not measurable through the benchmark and not surfaced in the endpoints view.
In real scans, genuine cross-file factory/DI client usage that can't fold to a concrete URL is silently omitted from endpoints, undercounting cost/usage.
Proposed fix direction
Retain provider+methodSignature-confident calls even when the URL is an sdk:// placeholder. Options:
Treat sdk:// URLs as high-confidence in shouldIncludeSynthetic/isHighConfidenceEndpointUrl when provider and methodSignature are both present, and key the synthetic endpoint on methodSignature (the runner's methodsEquivalent already matches on the SDK chain), or
Resolve the sdk:// placeholder to the canonical provider URL during cross-file resolution (mirror what the same-file path produces).
Either keeps precision intact — these are confident SDK detections, not speculative URL guesses.
Summary
The scanner detects an A5 cross-file factory call at the raw
local.apiCallslayer, but it is dropped during endpoint aggregation, so it never appears inresult.endpoints— the array the benchmark runner (and the webview) consume. The capability works; it's just invisible downstream. This surfaced while adding the A3/A5 benchmark fixtures (recost-dev/extension-benchmark#7): thefactory-cross-filefixture had to be dropped because nomust_detectendpoint could ever match.Repro
(Run a warm-up scan first — the first invocation in a fresh shell returns empty due to a wasm parser cold-start.)
Observed:
local.apiCalls: 1 —consumer.ts:6,provider: "openai",methodSignature: "client.chat.completions.create",url: "sdk://openai/client.chat.completions.create",crossFileOrigin→client-factory.ts.endpoints: 0 (empty).Contrast the same-file factory (
extension-benchmark/factory-in-file), which emits an endpoint because its call resolves to a concreteurl: "https://api.openai.com/v1/chat/completions".Root cause
In
src/scan-results.ts,mergeRemoteAndLocalEndpointsskips any call that failsshouldIncludeSynthetic:scan-results.ts:476—if (!shouldIncludeSynthetic(call)) continue;shouldIncludeSynthetic(:403) gates onisHighConfidenceEndpointUrl(call.url).isHighConfidenceEndpointUrlaccepts onlyhttps?://, root-relative/…, or specific<dynamic:…>tokens. Ansdk://…placeholder falls through toreturn false.A cross-file factory call keeps the
sdk://<provider>/<methodSignature>placeholder because the factory body (return new OpenAI()) contains no actual.createcall to resolve a concrete provider URL from. So a call with a confidently known provider and methodSignature is discarded purely because its URL is a placeholder.Why barrels survive but factory doesn't: a barrel fixture's leaf call (e.g.
api.ts:6) lives in the same file as the SDK client, resolves to a concretehttps://URL, and anchors a synthetic endpoint; the cross-file consumer call then joins that group. The cross-file factory has no such concrete-URL anchor — its only call is thesdk://one — so nothing is emitted.Impact
scanFileslayer bysrc/test/a5-factory-di-aliased.test.ts("A5.audit.factory") — is not measurable through the benchmark and not surfaced in the endpoints view.factory-cross-filefixture in [Measurement] Add barrel + factory fixtures to benchmark corpus extension-benchmark#7 (only 6 of 7 shapes could ship;factory-in-filestands in as a same-file proxy).Proposed fix direction
Retain provider+methodSignature-confident calls even when the URL is an
sdk://placeholder. Options:sdk://URLs as high-confidence inshouldIncludeSynthetic/isHighConfidenceEndpointUrlwhenproviderandmethodSignatureare both present, and key the synthetic endpoint onmethodSignature(the runner'smethodsEquivalentalready matches on the SDK chain), orsdk://placeholder to the canonical provider URL during cross-file resolution (mirror what the same-file path produces).Either keeps precision intact — these are confident SDK detections, not speculative URL guesses.
Acceptance criteria
node dist/cli/scan.js src/test/fixtures/a5/factory-direct --format jsonemits one endpoint:consumer.ts:6,provider: openai, methodSignatureclient.chat.completions.create.npm run benchmark).factory-cross-filefixture toextension-benchmarkand include it in the baseline ([Measurement] Add barrel + factory fixtures to benchmark corpus extension-benchmark#7).Links
factory-cross-filedrop is documented there)src/test/a5-factory-di-aliased.test.ts("A5.audit.factory")src/scan-results.ts:476(shouldIncludeSynthetic) +:403+isHighConfidenceEndpointUrl