Skip to content

[corpus] Fixture: gemini-python-genai (google.genai SDK — module/function/with-scope clients) #11

Description

@AndresL230

Part of #10. Validates recost-dev/extension#143.

Goal

A fixture exercising the Python google.genai SDK (genai.Client().models.generate_content), including a client constructed in a function scope and one in a with block. The extension detects none of these today (provider resolves to "google" with no fingerprint, and inner-scope client vars aren't tracked).

Branch

fixture/gemini-python-genai

Files to create

gemini-python-genai/src/generate.py

import os

from google import genai

client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])


def summarize(text: str) -> str:
    resp = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=text,
    )
    return resp.text


def embed(text: str) -> list[float]:
    resp = client.models.embed_content(
        model="text-embedding-004",
        contents=text,
    )
    return resp.embedding


def classify(text: str) -> str:
    local_client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])
    resp = local_client.models.generate_content(
        model="gemini-2.5-flash-lite",
        contents=f"Classify: {text}",
    )
    return resp.text

gemini-python-genai/src/with_scope.py

import os

from google import genai


def transcribe(prompt: str) -> str:
    with genai.Client(api_key=os.environ["GEMINI_API_KEY"]) as c:
        resp = c.models.generate_content(
            model="gemini-2.5-flash",
            contents=prompt,
        )
    return resp.text

gemini-python-genai/expected.json

{
  "schemaVersion": 1,
  "fixtureSlug": "gemini-python-genai",
  "endpoints": [
    { "file": "src/generate.py",   "function": "summarize", "line": 9,  "provider": "gemini", "method": "models.generate_content", "must_detect": true, "notes": "google.genai Client, module-level client var." },
    { "file": "src/generate.py",   "function": "embed",     "line": 17, "provider": "gemini", "method": "models.embed_content",    "must_detect": true, "notes": "Embeddings via the same client." },
    { "file": "src/generate.py",   "function": "classify",  "line": 26, "provider": "gemini", "method": "models.generate_content", "must_detect": true, "notes": "Client constructed inside the function body (not module-level)." },
    { "file": "src/with_scope.py", "function": "transcribe","line": 8,  "provider": "gemini", "method": "models.generate_content", "must_detect": true, "notes": "Client constructed in a `with` block; alias `c`." }
  ],
  "findings": []
}

FIXTURE.md

Synthetic fixture authored for the ReCost benchmark (no upstream source). Scope: the from google import genaigenai.Client().models.* SDK shape, covering module-level, function-scope, and with-block client construction. License: n/a (synthetic).

Current behaviour (the gap)

  • processPythonAssignment (import-resolver.ts:497) maps client = genai.Client() → package "google" (import segment); "google" is not in PACKAGE_TO_PROVIDER, so resolveProvider yields provider "google", no fingerprint, dropped at core-scanner.ts:269.
  • local_client / with … as c are inner-scope and aren't bound at all.
  • Expected current result: 0/4 detected → detectionRecall miss on this fixture.

Flips green when

recost-dev/extension#143 lands (map google.genai/google.generativeaigemini, track inner-scope client vars, register the models.generate_content/models.embed_content chains).

Acceptance criteria

  • Fixture added with the two src files + expected.json + FIXTURE.md.
  • Runner output records the current 0/4 recall for gemini-python-genai.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/measurementdocs/accuracy/measurement.md — benchmark corpus and CI gatesenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions