Add support for custom providers and local models

[dundalek]

Resolves #8
Alternative implementation to #36

By providing options to specify custom endpoint, we can cover both uses of using local models and using different providers. See updated README.md for details.

AI disclamer: Claude Code used

[vercel]

@dundalek is attempting to deploy a commit to the Yury Selivanov's projects Team on Vercel.

A member of the Team first needs to authorize it.

[lars20070]

I would prefer the custom apiBase approach in this PR to #36. IMO, it is more future-proof.

[lars20070]

Why remove this check? Can it not simply stay as is?

[dundalek]

This is to allow connecting to local llama-server without authentication.

[lars20070]

Maybe check more carefully. Number() can produce NaN for invalid strings. Suggested code below.

function customProvider(config: LatConfig): EmbeddingProvider | null {
  const base = process.env.LAT_LLM_BASE ?? config.llm_base;
  if (!base) return null;
  return {
    name: 'custom',
    apiBase: base,
    model: process.env.LAT_LLM_MODEL ?? config.llm_model ?? 'default',
    dimensions: (() => {
      const envDimensions = process.env.LAT_LLM_DIMENSIONS;
      if (envDimensions != null) {
        const parsed = parseInt(envDimensions, 10);
        if (isNaN(parsed)) throw new Error(`Invalid LAT_LLM_DIMENSIONS: "${envDimensions}"`);
        return parsed;
      }
      return config.llm_dimensions ?? 1536;
    })(),
    headers: (k) => {
      const h: Record<string, string> = {
        'Content-Type': 'application/json',
      };
      if (k) h['Authorization'] = `Bearer ${k}`;
      return h;
    },
  };
}

[dundalek]

good point, probably want to extract it as variable instead of IIFE

[dundalek]

As a note since creating the PR I discovered draft of llm uri scheme spec.

That would make setting it more convenient, instead of needing 4 env vars like:

LAT_LLM_BASE=https://openrouter.ai/api/v1
LAT_LLM_MODEL=qwen/qwen3-embedding-8b
LAT_LLM_DIMENSIONS=4096
LAT_LLM_KEY=sk-or-v1-xyz

We could use a single var like:

LAT_LLM_KEY=llm://:sk-or-v1-xyz@https://openrouter.ai/api/v1/qwen%2Fqwen3-embedding-8b?dimensions=4096

But I will wait on maintainer feedback before spending more time on this.