[corpus] Fixture: agent-frameworks (pydantic-ai .run() + LangChain .invoke(), provider from model string)

[AndresL230]

Part of #10. Validates recost-dev/extension#144.

Goal

A fixture exercising user-level agent-framework calls where the provider is hidden in a model string, not an import — pydantic-ai Agent("google-gla:…").run() and LangChain ChatOpenAI(...).invoke(). (Distinct from the existing langchain-openai fixture, which tests LangChain's internal leaf openai SDK calls — this is the caller's code.)

Branch

fixture/agent-frameworks

Files to create

agent-frameworks/src/pydantic_agent.py

from pydantic_ai import Agent

classifier_agent = Agent("google-gla:gemini-2.5-flash", system_prompt="Classify the input.")


async def classify(text: str) -> str:
    result = await classifier_agent.run(text)
    return result.output


tutor_agent = Agent("anthropic:claude-3-5-sonnet-latest")


async def tutor(question: str) -> str:
    result = await tutor_agent.run(question)
    return result.output

agent-frameworks/src/langchain_user.py

from langchain_openai import ChatOpenAI
from langchain_google_genai import ChatGoogleGenerativeAI

llm = ChatOpenAI(model="gpt-4o-mini")
gemini_llm = ChatGoogleGenerativeAI(model="gemini-2.5-flash")


def ask_openai(prompt: str) -> str:
    return llm.invoke(prompt).content


def ask_gemini(prompt: str) -> str:
    return gemini_llm.invoke(prompt).content

agent-frameworks/expected.json

{
  "schemaVersion": 1,
  "fixtureSlug": "agent-frameworks",
  "endpoints": [
    { "file": "src/pydantic_agent.py", "function": "classify", "line": 7,  "provider": "gemini",    "method": "run",    "must_detect": true, "notes": "pydantic-ai Agent; provider from model string 'google-gla:gemini-2.5-flash'." },
    { "file": "src/pydantic_agent.py", "function": "tutor",    "line": 15, "provider": "anthropic", "method": "run",    "must_detect": true, "notes": "pydantic-ai Agent; provider from model string 'anthropic:claude-...'." },
    { "file": "src/langchain_user.py", "function": "ask_openai","line": 9, "provider": "openai",    "method": "invoke", "must_detect": true, "notes": "LangChain ChatOpenAI(...).invoke — ctor class → openai." },
    { "file": "src/langchain_user.py", "function": "ask_gemini","line": 13,"provider": "gemini",    "method": "invoke", "must_detect": true, "notes": "LangChain ChatGoogleGenerativeAI(...).invoke — ctor class → gemini." }
  ],
  "findings": []
}

FIXTURE.md

Synthetic fixture authored for the ReCost benchmark (no upstream source). Scope: caller-side agent/LLM construction where the billed provider is named in the model string (pydantic-ai) or the ctor class (LangChain ChatOpenAI / ChatGoogleGenerativeAI), invoked via .run() / .invoke(). License: n/a (synthetic).

Current behaviour (the gap)

• agent = Agent(...) binds to package pydantic_ai (not a registered provider) → agent.run() dropped at core-scanner.ts:269. The model-string provider is never read.
• ChatOpenAI / ChatGoogleGenerativeAI are not in CLASS_TO_PACKAGE, so llm.invoke() is dropped too.
• Expected current result: 0/4 detected.

Flips green when

recost-dev/extension#144 lands (recognize agent/LLM construction, parse the model string / ctor class → provider, treat .run()/.invoke() as metered).

Acceptance criteria

• Fixture added with both src files + expected.json + FIXTURE.md.
• Runner output records the current 0/4 recall for agent-frameworks.