LLMrest

A unified TypeScript library for querying multiple LLM providers concurrently — OpenAI, Anthropic Claude, Google Gemini, and Perplexity — from a single client.

Features

• Concurrent requests — fan out to multiple models in parallel, results arrive as each completes
• Unified interface — one client, four providers
• Per-model messages — send different prompts to different models in a single call
• Streaming callbacks — responses delivered via onResponse as each model finishes
• Content moderation — optional OpenAI moderation check before sending
• Input size validation — optional byte-limit enforcement before sending
• Full TypeScript — complete type definitions included
• Extensible — register custom providers via ModelRegistry

Requirements

• Node.js >= 18.0.0

Installation

npm install @armagank/llmrest

Quick Start

import { LLMClient } from '@armagank/llmrest';

const client = new LLMClient({
  apiKeys: {
    openai: process.env.OPENAI_API_KEY,
    claude: process.env.ANTHROPIC_API_KEY,
    gemini: process.env.GEMINI_API_KEY,
    perplexity: process.env.PERPLEXITY_API_KEY,
  },
});

// CommonJS
const { createAIClient } = require('@armagank/llmrest');
const client = createAIClient({ apiKeys: { ... } });

You only need to provide API keys for the providers you intend to use.

API

createChat(params)

Send the same messages to multiple models concurrently.

const result = await client.createChat({
  models: ['gpt-4o', 'claude-sonnet-4-6', 'gemini-2.5-flash'],
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is the capital of France?' },
  ],
  maxInput: 100_000,  // bytes (optional)
  maxOutput: 1_000,   // tokens (optional)
  moderationEnabled: false,
});

// result: Record<modelId, string | { error: string }>
console.log(result['gpt-4o']);            // "The capital of France is Paris."
console.log(result['claude-sonnet-4-6']); // "Paris is the capital of France."

Parameter	Type	Required	Description
models	string[]	Yes	Model IDs to query
messages	Message[]	Yes	Chat messages
maxInput	number	No	Max input size in bytes
maxOutput	number	No	Max output tokens
moderationEnabled	boolean	No	Run OpenAI content moderation first

createChatMessages(params)

Send different messages to different models in one call.

const result = await client.createChatMessages({
  models: ['gpt-4o', 'claude-sonnet-4-6'],
  messages: {
    'gpt-4o': [
      { role: 'system', content: 'You are a coding assistant.' },
      { role: 'user', content: 'Write a TypeScript hello world.' },
    ],
    'claude-sonnet-4-6': [
      { role: 'system', content: 'You are a poet.' },
      { role: 'user', content: 'Write a haiku about TypeScript.' },
    ],
  },
  maxOutput: 500,
});

createChatStreaming(params)

Dispatch requests concurrently; receive each result via callback as it arrives rather than waiting for all to finish.

await client.createChatStreaming({
  models: ['gpt-4o', 'claude-sonnet-4-6', 'gemini-2.5-flash'],
  messages: [{ role: 'user', content: 'Tell me a joke.' }],
  maxOutput: 300,
  onResponse: (response) => {
    if (response.status === 'success') {
      console.log(`[${response.model}] ${response.data}`);
    } else {
      console.error(`[${response.model}] Error: ${response.error}`);
    }
  },
});

onResponse receives a StreamingResponse:

interface StreamingResponse {
  model: string;
  status: 'success' | 'error';
  data?: string;   // present on success
  error?: string;  // present on error
  timestamp: string;
}

Model-specific messages also work with streaming:

await client.createChatStreaming({
  models: ['gpt-4o', 'claude-sonnet-4-6'],
  messages: {
    'gpt-4o': [{ role: 'user', content: 'Explain recursion.' }],
    'claude-sonnet-4-6': [{ role: 'user', content: 'Explain iteration.' }],
  },
  onResponse: (r) => console.log(r),
});

Supported Models

OpenAI

gpt-5, gpt-5-mini, gpt-5-nano, gpt-5.1, gpt-5.2, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano, gpt-4.1, gpt-4.1-mini, gpt-4.1-nano, gpt-4o, gpt-4o-mini, gpt-4-turbo, o1, o1-mini, o3, o3-mini, o4-mini

Anthropic Claude

claude-opus-4-7, claude-sonnet-4-6, claude-haiku-4-5, claude-haiku-4-5-20251001, claude-opus-4-6, claude-sonnet-4-5, claude-opus-4-5, claude-opus-4-1

Google Gemini

gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-lite, gemini-3.1-flash-lite, gemini-3-flash-preview

Perplexity

sonar, sonar-pro, sonar-reasoning-pro, sonar-deep-research

Custom Providers

Extend BaseProvider to add any model not in the built-in list:

import { BaseProvider, ModelRegistry } from '@armagank/llmrest';
import type { ProviderChatOptions } from '@armagank/llmrest';

class MyProvider extends BaseProvider {
  async chat({ messages, maxOutput }: ProviderChatOptions): Promise<string> {
    // call your API here
    return 'response text';
  }
}

const registry = new ModelRegistry();
registry.registerMany(['my-model-v1', 'my-model-v2'], new MyProvider());

Error Handling

Failed models never throw — they return an error object in the result map. Only pre-flight errors (bad config, moderation rejection, input too large) throw.

const result = await client.createChat({
  models: ['gpt-4o', 'claude-sonnet-4-6'],
  messages: [{ role: 'user', content: 'Hello' }],
});

for (const [model, response] of Object.entries(result)) {
  if (typeof response === 'string') {
    console.log(`${model}: ${response}`);
  } else {
    console.error(`${model} failed: ${response.error}`);
  }
}

License

ISC