Skip to content

Add Google Vertex AI provider #29

@vzegnameta

Description

@vzegnameta

Add Google Vertex AI provider

Summary

Add a new google-vertex provider that authenticates via Application Default Credentials (ADC) instead of a Gemini API key. This enables running the MCP server on Google Cloud (e.g., Cloud Run with an attached service account) and locally with a downloaded service account JSON key file.

Motivation

The current Google provider (google) uses the direct Gemini API with an API key. Many teams run workloads on Google Cloud where Vertex AI is the standard way to access Gemini models. Vertex AI uses ADC for authentication, which means:

  • On Cloud Run / GKE / GCE: auth is automatic via the attached service account — no key file or API key needed.
  • Locally: auth uses a downloaded service account JSON key file pointed to by GOOGLE_APPLICATION_CREDENTIALS.

This is a common enterprise deployment pattern and doesn't require managing or rotating API keys.

Proposed changes

1. New dependency

@ai-sdk/google-vertex

2. New provider: src/providers/google-vertex.ts

A new ProviderConfig that mirrors the existing google.ts provider but:

  • createModel: Uses createVertex({ project, location }) from @ai-sdk/google-vertex instead of createGoogleGenerativeAI({ apiKey }).
  • preparePdf: Uses new GoogleGenAI({ vertexai: true, project, location }) instead of new GoogleGenAI({ apiKey }). The Vertex AI File API uses GCS-backed storage, so the service account needs the Storage Admin (or equivalent) role in addition to Vertex AI User.
  • apiKey parameter: Ignored — ADC handles authentication.
  • id: Should remain "google" so that cached URI logic (Gemini File API URIs) continues to work.

Everything else (chunking, rolling findings, structured output, caching) stays the same since it's abstracted behind the ProviderConfig interface.

3. Cloud vs local detection

function isCloudRun(): boolean {
  return !!process.env.K_SERVICE;
}
  • Cloud Run: ADC picks up the attached service account automatically.
  • Local: The setup flow stores the path to the SA JSON key file. At runtime, set GOOGLE_APPLICATION_CREDENTIALS to that path before creating clients.

4. Setup flow changes

When the user selects "Google Vertex AI" during --setup:

  1. Skip the API key prompt — Vertex AI doesn't use API keys.
  2. Prompt for GCP Project ID (required).
  3. Prompt for Region/Location (required, e.g., us-east5, us-central1).
  4. Prompt for SA key file path (optional — only needed for local development, not on Cloud Run).

Store project, location, and key file path in the keychain using the existing credential storage mechanism.

5. Registry and exports

  • Register vertexProvider in src/providers/registry.ts under the key "google-vertex".
  • Export vertexProvider from src/providers/index.ts.

6. Required IAM roles for the service account

Role Purpose
Vertex AI User Call Gemini models via Vertex AI
Storage Admin (or storage.objects.create) Upload PDFs via the Vertex AI File API (GCS-backed)

Environment variables

Variable Required Description
VERTEX_PROJECT Yes (or set during --setup) GCP project ID
VERTEX_LOCATION Yes (or set during --setup) GCP region
GOOGLE_APPLICATION_CREDENTIALS Local only Path to SA JSON key file
K_SERVICE Auto-set by Cloud Run Used to detect Cloud Run environment

Non-goals

  • This does not replace the existing google provider — it's additive.
  • No changes to the MCP server, transport layer, or analyzePdf() service function.
  • No changes to other providers (Anthropic, OpenAI).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions