diff --git a/lux/guides/llm_provider_abstraction.md b/lux/guides/llm_provider_abstraction.md new file mode 100644 index 00000000..e9929dc1 --- /dev/null +++ b/lux/guides/llm_provider_abstraction.md @@ -0,0 +1,318 @@ +# LLM Provider Abstraction Layer + +This document describes the LLM Provider Abstraction Layer for Lux, providing unified access to multiple LLM providers with intelligent selection and fallback capabilities. + +## Overview + +The LLM Provider Abstraction Layer provides: + +- **Universal Provider Interface** - Single API for all LLM providers +- **Automatic Model Selection** - Intelligent selection based on cost, speed, or quality +- **Smart Fallback Handling** - Automatic failover when providers fail +- **Cost Tracking** - Monitor and optimize API costs +- **Performance Monitoring** - Track latency and success rates + +## Quick Start + +```elixir +# Call with automatic provider selection +{:ok, response} = Lux.LLM.Provider.call("Hello, world!", [], %{}) + +# Call with specific provider +{:ok, response} = Lux.LLM.Provider.call("Hello!", [], %{ + provider: :openai, + model: "gpt-4" +}) + +# Select best provider for a task +{provider, model} = Lux.LLM.Provider.select_best(%{priority: :cost}) +``` + +## Configuration + +Providers are automatically registered with default configurations. You can customize them: + +```elixir +# Register a custom provider +Lux.LLM.Provider.register(:custom, %{ + module: MyApp.CustomLLM, + models: ["custom-model-1", "custom-model-2"], + cost_per_1k_tokens: %{ + "custom-model-1" => %{input: 0.01, output: 0.02} + }, + max_tokens: 8192, + features: [:streaming, :tools], + priority: 5 +}) + +# Update provider config +Lux.LLM.Provider.unregister(:deprecated_provider) +``` + +## Provider Selection + +### Selection Strategies + +| Strategy | Description | +|----------|-------------| +| `:cost` | Minimize cost per token | +| `:speed` | Minimize latency | +| `:quality` | Maximize output quality | +| `:balanced` | Balance between all factors | + +### Selection Options + +```elixir +# Select by cost +{provider, model} = Lux.LLM.Provider.select_best(%{ + priority: :cost +}) + +# Select with feature requirements +{provider, model} = Lux.LLM.Provider.select_best(%{ + priority: :quality, + features: [:vision, :tools] +}) + +# Exclude certain providers +{provider, model} = Lux.LLM.Provider.select_best(%{ + priority: :balanced, + exclude: [:slow_provider] +}) + +# Prefer certain providers +{provider, model} = Lux.LLM.Provider.select_best(%{ + priority: :speed, + prefer: [:fast_provider] +}) +``` + +## Fallback Handling + +The system automatically handles provider failures: + +```elixir +# Automatic fallback with all providers +{:ok, response} = Lux.LLM.Provider.call("Hello!", [], %{ + fallback: :all +}) + +# No fallback +{:ok, response} = Lux.LLM.Provider.call("Hello!", [], %{ + fallback: :none +}) + +# Same tier fallback only +{:ok, response} = Lux.LLM.Provider.call("Hello!", [], %{ + fallback: :same_tier +}) +``` + +### Fallback Features + +- **Exponential Backoff** - Retries with increasing delay +- **Circuit Breaker** - Prevents cascading failures +- **Intelligent Failover** - Falls back to alternative providers + +## Cost Tracking + +Track and monitor API costs: + +```elixir +# Record a request (automatic when using Provider.call) +Lux.LLM.CostTracker.record(:openai, "gpt-4", 1000, 500) + +# Get total costs +costs = Lux.LLM.CostTracker.get_total_costs() +# => %{cost: 1.50, requests: 100, tokens: 50000} + +# Get costs by provider +provider_costs = Lux.LLM.CostTracker.get_costs_by_provider(:openai) + +# Get costs by period +daily_costs = Lux.LLM.CostTracker.get_costs_by_period(:daily) + +# Set budget limits +Lux.LLM.CostTracker.set_budget(:daily, 10.0) + +# Check if budget exceeded +if Lux.LLM.CostTracker.budget_exceeded?(:daily) do + # Stop making requests +end +``` + +## Performance Monitoring + +Monitor provider performance: + +```elixir +# Get stats for a provider +stats = Lux.LLM.Provider.get_stats(:openai) +# => %{ +# total_requests: 100, +# successful_requests: 95, +# avg_latency_ms: 250.5, +# success_rate: 0.95 +# } + +# Get all provider stats +all_stats = Lux.LLM.Provider.get_all_stats() + +# Get health status +health = Lux.LLM.PerformanceMonitor.get_health(:openai) +# => :healthy | :degraded | :unhealthy +``` + +## Provider Registry + +Manage registered providers: + +```elixir +# List all providers +providers = Lux.LLM.Provider.list_providers() +# => [:openai, :anthropic, :together_ai] + +# Get provider config +config = Lux.LLM.Provider.get_config(:openai) + +# Enable/disable providers +Lux.LLM.Provider.disable(:expensive_provider) +Lux.LLM.Provider.enable(:cheaper_provider) +``` + +## Default Providers + +| Provider | Models | Cost Range | Features | +|----------|--------|------------|----------| +| OpenAI | gpt-4, gpt-4-turbo, gpt-3.5-turbo | $0.0005 - $0.06/1k | streaming, tools, vision, json_mode | +| Anthropic | claude-3-opus, claude-3-sonnet, claude-3-haiku | $0.00025 - $0.075/1k | streaming, tools, vision | +| Together AI | Mixtral-8x7B, Llama-3-70b | $0.0006 - $0.0009/1k | streaming, tools | +| Ollama | llama2, mistral, codellama, custom | **FREE** | streaming, local, no_cost | + +## Local Model Support (Ollama) + +Ollama provides local LLM support with zero API costs. Perfect for: +- Development and testing +- Privacy-sensitive applications +- Cost optimization +- Offline environments + +### Setup + +1. Install Ollama: https://ollama.ai +2. Pull a model: `ollama pull llama2` +3. Register the provider: + +```elixir +# Register Ollama +Lux.LLM.Ollama.register() + +# Or with custom options +Lux.LLM.Ollama.register( + host: "http://localhost:11434", + models: ["llama2", "mistral", "codellama"] +) +``` + +### Usage + +```elixir +# Basic call +{:ok, response} = Lux.LLM.Ollama.call("Hello!", [], %{}) + +# With specific model +{:ok, response} = Lux.LLM.Ollama.call("Write code", [], %{model: "codellama"}) + +# List available models +{:ok, models} = Lux.LLM.Ollama.list_models() + +# Pull a new model +{:ok, :pulling} = Lux.LLM.Ollama.pull_model("mistral") + +# Health check +:ok = Lux.LLM.Ollama.health_check() +``` + +### Automatic Fallback to Local + +When cloud providers fail or budgets are exceeded, you can fall back to local models: + +```elixir +# Use local model as fallback +{:ok, response} = Lux.LLM.Provider.call("Hello!", [], %{ + fallback: :all, + prefer: [:openai, :anthropic] # Will fall back to ollama if these fail +}) +``` + +## Error Handling + +```elixir +case Lux.LLM.Provider.call("Hello!", [], %{provider: :openai}) do + {:ok, response} -> + # Handle success + + {:error, {:all_providers_failed, errors}} -> + # All providers failed + + {:error, {:invalid_api_key}} -> + # Authentication error + + {:error, reason} -> + # Other errors +end +``` + +## Testing + +```bash +# Run unit tests +mix test test/llm/ + +# Run specific test +mix test test/llm/provider_test.exs +``` + +## Architecture + +``` +┌─────────────────────────────────────────────────────┐ +│ Lux.LLM.Provider │ +│ (Universal Provider Interface) │ +├─────────────────────────────────────────────────────┤ +│ │ +│ ┌─────────────────┐ ┌─────────────────┐ │ +│ │ ModelSelector │ │ FallbackHandler │ │ +│ │ (Selection) │ │ (Failover) │ │ +│ └─────────────────┘ └─────────────────┘ │ +│ │ +│ ┌─────────────────┐ ┌─────────────────┐ │ +│ │ CostTracker │ │PerformanceMonitor│ │ +│ │ (Costs) │ │ (Monitoring) │ │ +│ └─────────────────┘ └─────────────────┘ │ +│ │ +├─────────────────────────────────────────────────────┤ +│ ProviderRegistry (ETS) │ +├─────────────────────────────────────────────────────┤ +│ │ +│ ┌─────────┐ ┌───────────┐ ┌────────────┐ │ +│ │ OpenAI │ │ Anthropic │ │ TogetherAI │ │ +│ └─────────┘ └───────────┘ └────────────┘ │ +│ │ +└─────────────────────────────────────────────────────┘ +``` + +## Best Practices + +1. **Use automatic selection** unless you have specific requirements +2. **Set cost budgets** to avoid unexpected charges +3. **Monitor performance** to identify issues early +4. **Enable fallback** for critical applications +5. **Use quality priority** for important tasks, cost priority for bulk operations + +## References + +- [OpenAI API Documentation](https://platform.openai.com/docs) +- [Anthropic API Documentation](https://docs.anthropic.com) +- [Together AI Documentation](https://docs.together.ai) +- [Lux Documentation](https://hexdocs.pm/lux) \ No newline at end of file diff --git a/lux/lib/lux/llm/cost_tracker.ex b/lux/lib/lux/llm/cost_tracker.ex new file mode 100644 index 00000000..271c4783 --- /dev/null +++ b/lux/lib/lux/llm/cost_tracker.ex @@ -0,0 +1,342 @@ +defmodule Lux.LLM.CostTracker do + @moduledoc """ + Tracks and manages LLM API costs. + + Features: + - Per-request cost tracking + - Cost budgets and limits + - Cost optimization suggestions + - Historical cost analysis + + ## Usage + + # Record a request cost + CostTracker.record(:openai, "gpt-4", 1000, 500) + + # Get total costs + costs = CostTracker.get_total_costs() + + # Get costs by provider + provider_costs = CostTracker.get_costs_by_provider(:openai) + + # Set a budget limit + CostTracker.set_budget(:daily, 10.0) # $10/day + + """ + + use Agent + + @table_name :lux_llm_cost_tracker + + @type time_period :: :hourly | :daily | :weekly | :monthly + @type cost :: float() + + @doc """ + Start the cost tracker agent. + """ + def start_link(_opts \\ []) do + Agent.start_link(fn -> init_tracker() end, name: __MODULE__) + end + + @doc """ + Record a cost for an API request. + + ## Parameters + + * `provider` - The LLM provider + * `model` - The model used + * `input_tokens` - Number of input tokens + * `output_tokens` - Number of output tokens + + ## Examples + + iex> CostTracker.record(:openai, "gpt-4", 1000, 500) + :ok + + """ + @spec record(atom(), String.t(), non_neg_integer(), non_neg_integer()) :: :ok + def record(provider, model, input_tokens, output_tokens) do + cost = calculate_cost(provider, model, input_tokens, output_tokens) + + Agent.update(__MODULE__, fn state -> + now = DateTime.utc_now() + timestamp = System.monotonic_time(:millisecond) + + entry = %{ + provider: provider, + model: model, + input_tokens: input_tokens, + output_tokens: output_tokens, + cost: cost, + timestamp: timestamp, + datetime: now + } + + # Add to history + :ets.insert(@table_name, {{:history, timestamp}, entry}) + + # Update aggregates + update_aggregates(provider, model, cost, now) + + # Update state + Map.update!(state, :total_cost, &(&1 + cost)) + |> Map.update!(:total_requests, &(&1 + 1)) + end) + end + + @doc """ + Calculate the cost for a request without recording. + + ## Examples + + iex> CostTracker.calculate_cost(:openai, "gpt-4", 1000, 500) + 0.06 + + """ + @spec calculate_cost(atom(), String.t(), non_neg_integer(), non_neg_integer()) :: cost() + def calculate_cost(provider, model, input_tokens, output_tokens) do + alias Lux.LLM.ProviderRegistry + + config = ProviderRegistry.get_config(provider) + + case get_in(config, [:cost_per_1k_tokens, model]) do + nil -> 0.0 + costs -> + input_cost = Map.get(costs, :input, 0) * (input_tokens / 1000) + output_cost = Map.get(costs, :output, 0) * (output_tokens / 1000) + Float.round(input_cost + output_cost, 6) + end + end + + @doc """ + Get total costs. + + ## Examples + + iex> CostTracker.get_total_costs() + %{cost: 10.50, requests: 100, tokens: 50000} + + """ + @spec get_total_costs() :: map() + def get_total_costs do + Agent.get(__MODULE__, fn state -> + %{ + cost: Map.get(state, :total_cost, 0.0), + requests: Map.get(state, :total_requests, 0), + tokens: Map.get(state, :total_tokens, 0) + } + end) + end + + @doc """ + Get costs by provider. + + ## Examples + + iex> CostTracker.get_costs_by_provider(:openai) + %{cost: 5.25, requests: 50, ...} + + """ + @spec get_costs_by_provider(atom()) :: map() + def get_costs_by_provider(provider) do + case :ets.lookup(@table_name, {:provider, provider}) do + [{_, data}] -> data + [] -> %{cost: 0.0, requests: 0, tokens: 0} + end + end + + @doc """ + Get costs by model. + + ## Examples + + iex> CostTracker.get_costs_by_model(:openai, "gpt-4") + %{cost: 2.50, requests: 20, ...} + + """ + @spec get_costs_by_model(atom(), String.t()) :: map() + def get_costs_by_model(provider, model) do + case :ets.lookup(@table_name, {:model, provider, model}) do + [{_, data}] -> data + [] -> %{cost: 0.0, requests: 0, tokens: 0} + end + end + + @doc """ + Get costs for a time period. + + ## Examples + + iex> CostTracker.get_costs_by_period(:daily) + %{cost: 8.50, requests: 80, ...} + + """ + @spec get_costs_by_period(time_period()) :: map() + def get_costs_by_period(period) do + now = DateTime.utc_now() + + start_time = case period do + :hourly -> DateTime.add(now, -1, :hour) + :daily -> DateTime.add(now, -1, :day) + :weekly -> DateTime.add(now, -7, :day) + :monthly -> DateTime.add(now, -30, :day) + end + + entries = get_history_since(start_time) + + %{ + cost: Enum.reduce(entries, 0.0, fn entry, acc -> acc + entry.cost end), + requests: length(entries), + input_tokens: Enum.reduce(entries, 0, fn e, acc -> acc + e.input_tokens end), + output_tokens: Enum.reduce(entries, 0, fn e, acc -> acc + e.output_tokens end) + } + end + + @doc """ + Set a budget limit. + + ## Examples + + iex> CostTracker.set_budget(:daily, 10.0) + :ok + + """ + @spec set_budget(time_period(), cost()) :: :ok + def set_budget(period, limit) do + Agent.update(__MODULE__, fn state -> + budgets = Map.get(state, :budgets, %{}) + Map.put(state, :budgets, Map.put(budgets, period, limit)) + end) + end + + @doc """ + Check if budget is exceeded. + + ## Examples + + iex> CostTracker.budget_exceeded?(:daily) + false + + """ + @spec budget_exceeded?(time_period()) :: boolean() + def budget_exceeded?(period) do + budget = get_budget(period) + current = get_costs_by_period(period) + + budget != nil and current.cost >= budget + end + + @doc """ + Get the budget for a period. + + ## Examples + + iex> CostTracker.get_budget(:daily) + 10.0 + + """ + @spec get_budget(time_period()) :: cost() | nil + def get_budget(period) do + Agent.get(__MODULE__, fn state -> + get_in(state, [:budgets, period]) + end) + end + + @doc """ + Get cost history. + + ## Examples + + iex> CostTracker.get_history(100) + [%{provider: :openai, ...}, ...] + + """ + @spec get_history(pos_integer()) :: [map()] + def get_history(limit \\ 100) do + :ets.tab2list(@table_name) + |> Enum.filter(fn + {{:history, _}, _} -> true + _ -> false + end) + |> Enum.sort_by(fn {{:history, timestamp}, _} -> timestamp end, :desc) + |> Enum.take(limit) + |> Enum.map(fn {_, entry} -> entry end) + end + + @doc """ + Reset all tracking data. + + ## Examples + + iex> CostTracker.reset() + :ok + + """ + @spec reset() :: :ok + def reset do + Agent.update(__MODULE__, fn _state -> + :ets.delete_all_objects(@table_name) + init_state() + end) + end + + # Private functions + + defp init_tracker do + if :ets.whereis(@table_name) == :undefined do + :ets.new(@table_name, [:named_table, :set, :public]) + end + init_state() + end + + defp init_state do + %{ + total_cost: 0.0, + total_requests: 0, + total_tokens: 0, + budgets: %{} + } + end + + defp update_aggregates(provider, model, cost, now) do + # Update provider aggregate + provider_data = case :ets.lookup(@table_name, {:provider, provider}) do + [{_, data}] -> data + [] -> %{cost: 0.0, requests: 0, tokens: 0} + end + + updated_provider = %{ + cost: provider_data.cost + cost, + requests: provider_data.requests + 1, + tokens: provider_data.tokens, + last_request: now + } + + :ets.insert(@table_name, {{:provider, provider}, updated_provider}) + + # Update model aggregate + model_data = case :ets.lookup(@table_name, {:model, provider, model}) do + [{_, data}] -> data + [] -> %{cost: 0.0, requests: 0, tokens: 0} + end + + updated_model = %{ + cost: model_data.cost + cost, + requests: model_data.requests + 1, + tokens: model_data.tokens, + last_request: now + } + + :ets.insert(@table_name, {{:model, provider, model}, updated_model}) + end + + defp get_history_since(start_time) do + :ets.tab2list(@table_name) + |> Enum.filter(fn + {{:history, _}, entry} -> + DateTime.compare(entry.datetime, start_time) in [:gt, :eq] + _ -> false + end) + |> Enum.map(fn {_, entry} -> entry end) + end +end \ No newline at end of file diff --git a/lux/lib/lux/llm/fallback_handler.ex b/lux/lib/lux/llm/fallback_handler.ex new file mode 100644 index 00000000..b7327a38 --- /dev/null +++ b/lux/lib/lux/llm/fallback_handler.ex @@ -0,0 +1,226 @@ +defmodule Lux.LLM.FallbackHandler do + @moduledoc """ + Handles fallback logic when LLM providers fail. + + Implements intelligent fallback strategies: + - Retry with exponential backoff + - Failover to alternative providers + - Circuit breaker pattern for failing providers + + ## Features + + - Automatic retry with configurable backoff + - Provider failover based on priority + - Circuit breaker to prevent cascading failures + - Detailed error logging and tracking + + ## Usage + + # Execute with automatic fallback + {:ok, response} = FallbackHandler.execute( + "Hello!", + [], + [{:openai, "gpt-4"}, {:anthropic, "claude-3-opus"}], + %{max_retries: 3} + ) + + """ + + alias Lux.LLM.ProviderRegistry + alias Lux.LLM.PerformanceMonitor + + require Logger + + @type provider_chain :: [{atom(), String.t() | nil}] + @type execution_result :: {:ok, map()} | {:error, term()} + + @circuit_breaker_threshold 5 + @circuit_breaker_timeout_ms 60_000 + + @doc """ + Execute a request with automatic fallback. + + ## Options + + * `:max_retries` - Maximum retries per provider (default: 2) + * `:retry_delay_ms` - Initial retry delay (default: 1000) + * `:timeout` - Request timeout (default: 60000) + + ## Examples + + iex> FallbackHandler.execute("Hello!", [], [{:openai, "gpt-4"}], %{}) + {:ok, %Lux.LLM.Response{...}} + + """ + @spec execute(String.t(), list(), provider_chain(), map()) :: execution_result() + def execute(prompt, tools, provider_chain, opts \\ %{}) do + opts = Map.merge(default_opts(), opts) + + execute_chain(prompt, tools, provider_chain, opts, []) + end + + @doc """ + Get fallback providers for a given provider. + + ## Options + + * `:all` - Return all other providers + * `:same_tier` - Return providers in same cost tier + * `none` - No fallback + + ## Examples + + iex> FallbackHandler.get_fallbacks(:openai, :all) + [{:anthropic, nil}, {:together_ai, nil}] + + """ + @spec get_fallbacks(atom(), atom()) :: provider_chain() + def get_fallbacks(provider, strategy \\ :all) do + case strategy do + :none -> [] + :all -> + ProviderRegistry.list_enabled() + |> Enum.reject(fn p -> p == provider end) + |> Enum.map(fn p -> {p, nil} end) + :same_tier -> + get_same_tier_fallbacks(provider) + end + end + + # Private functions + + defp default_opts do + %{ + max_retries: 2, + retry_delay_ms: 1000, + timeout: 60_000 + } + end + + defp execute_chain(_prompt, _tools, [], _opts, errors) do + {:error, {:all_providers_failed, Enum.reverse(errors)}} + end + + defp execute_chain(prompt, tools, [{provider, model} | rest], opts, errors) do + if circuit_breaker_open?(provider) do + Logger.warning("Circuit breaker open for #{provider}, skipping") + execute_chain(prompt, tools, rest, opts, [{provider, :circuit_breaker_open} | errors]) + else + case execute_with_retry(provider, model, prompt, tools, opts) do + {:ok, response} -> + record_success(provider) + {:ok, response} + + {:error, reason} -> + record_failure(provider) + Logger.warning("Provider #{provider} failed: #{inspect(reason)}") + execute_chain(prompt, tools, rest, opts, [{provider, reason} | errors]) + end + end + end + + defp execute_with_retry(provider, model, prompt, tools, opts) do + max_retries = Map.get(opts, :max_retries, 2) + retry_delay_ms = Map.get(opts, :retry_delay_ms, 1000) + + do_execute_with_retry(provider, model, prompt, tools, opts, max_retries, retry_delay_ms) + end + + defp do_execute_with_retry(_provider, _model, _prompt, _tools, _opts, 0, _delay) do + {:error, :max_retries_exceeded} + end + + defp do_execute_with_retry(provider, model, prompt, tools, opts, retries_left, delay) do + config = ProviderRegistry.get_config(provider) + module = Map.get(config, :module) + + if is_nil(module) do + {:error, {:invalid_provider, provider}} + else + model = model || get_default_model(config) + call_config = build_call_config(model, config, opts) + + start_time = System.monotonic_time(:millisecond) + + result = try do + module.call(prompt, tools, call_config) + rescue + e -> {:error, {:exception, e}} + catch + kind, reason -> {:error, {kind, reason}} + end + + end_time = System.monotonic_time(:millisecond) + latency = end_time - start_time + + case result do + {:ok, response} -> + PerformanceMonitor.record_request(provider, latency, :success) + {:ok, response} + + {:error, reason} when retries_left > 0 -> + Logger.debug("Retrying #{provider} after #{delay}ms, #{retries_left - 1} retries left") + Process.sleep(delay) + new_delay = min(delay * 2, 30_000) + do_execute_with_retry(provider, model, prompt, tools, opts, retries_left - 1, new_delay) + + {:error, reason} -> + PerformanceMonitor.record_request(provider, latency, :failure) + {:error, reason} + end + end + end + + defp get_default_model(config) do + config + |> Map.get(:models, []) + |> List.first() + end + + defp build_call_config(model, _provider_config, opts) do + %{ + model: model, + timeout: Map.get(opts, :timeout, 60_000) + } + end + + defp circuit_breaker_open?(provider) do + # Check if the circuit breaker should be open + stats = PerformanceMonitor.get_stats(provider) + + case Map.get(stats, :consecutive_failures, 0) do + failures when failures >= @circuit_breaker_threshold -> + # Check if timeout has passed + last_failure = Map.get(stats, :last_failure_time) + case last_failure do + nil -> true + time -> + elapsed = System.monotonic_time(:millisecond) - time + elapsed < @circuit_breaker_timeout_ms + end + _ -> false + end + end + + defp record_success(provider) do + PerformanceMonitor.reset_consecutive_failures(provider) + end + + defp record_failure(provider) do + PerformanceMonitor.increment_consecutive_failures(provider) + end + + defp get_same_tier_fallbacks(provider) do + tier = get_provider_tier(provider) + + ProviderRegistry.list_enabled() + |> Enum.reject(fn p -> p == provider end) + |> Enum.filter(fn p -> get_provider_tier(p) == tier end) + |> Enum.map(fn p -> {p, nil} end) + end + + defp get_provider_tier(:openai), do: 1 + defp get_provider_tier(:anthropic), do: 1 + defp get_provider_tier(:together_ai), do: 2 + defp get_provider_tier(_), do: 3 +end \ No newline at end of file diff --git a/lux/lib/lux/llm/model_selector.ex b/lux/lib/lux/llm/model_selector.ex new file mode 100644 index 00000000..cce9e6a9 --- /dev/null +++ b/lux/lib/lux/llm/model_selector.ex @@ -0,0 +1,235 @@ +defmodule Lux.LLM.ModelSelector do + @moduledoc """ + Intelligent model selection for LLM providers. + + Selects the best model based on: + - Cost optimization + - Latency requirements + - Quality requirements + - Feature requirements + - Token limits + + ## Selection Strategies + + - `:cost` - Minimize cost per token + - `:speed` - Minimize latency + - `:quality` - Maximize output quality + - `:balanced` - Balance between cost, speed, and quality + + ## Usage + + # Select cheapest model + {provider, model} = ModelSelector.select_best(%{priority: :cost}) + + # Select fastest model + {provider, model} = ModelSelector.select_best(%{priority: :speed}) + + # Select model with specific features + {provider, model} = ModelSelector.select_best(%{ + priority: :quality, + features: [:vision, :tools] + }) + + """ + + alias Lux.LLM.ProviderRegistry + alias Lux.LLM.PerformanceMonitor + + @type priority :: :cost | :speed | :quality | :balanced + @type selection_result :: {atom(), String.t()} | {:error, term()} + + @doc """ + Select the best provider and model for a given task. + + ## Options + + * `:priority` - Selection priority (default: :balanced) + * `:max_tokens` - Maximum tokens required + * `:features` - Required features (e.g., [:vision, :tools]) + * `:exclude` - Providers to exclude + * `:prefer` - Preferred providers + + ## Examples + + iex> ModelSelector.select_best(%{priority: :cost}) + {:together_ai, "mistralai/Mixtral-8x7B-Instruct-v0.1"} + + iex> ModelSelector.select_best(%{priority: :quality, features: [:vision]}) + {:openai, "gpt-4-turbo"} + + """ + @spec select_best(map()) :: selection_result() + def select_best(opts \\ %{}) do + opts = Map.merge(default_opts(), opts) + + candidates = get_candidates(opts) + + if Enum.empty?(candidates) do + {:error, :no_matching_providers} + else + rank_and_select(candidates, opts) + end + end + + @doc """ + Build a fallback chain of providers. + + ## Examples + + iex> ModelSelector.build_chain(%{priority: :cost}) + [{:together_ai, "mistralai/Mixtral-8x7B-Instruct-v0.1"}, ...] + + """ + @spec build_chain(map()) :: [{atom(), String.t()}] + def build_chain(opts \\ %{}) do + opts = Map.merge(default_opts(), opts) + candidates = get_candidates(opts) + + candidates + |> Enum.map(fn {provider, model, _score} -> {provider, model} end) + |> Enum.take(5) + end + + @doc """ + Get the cost for a specific provider and model. + + ## Examples + + iex> ModelSelector.get_cost(:openai, "gpt-4", 1000, 500) + 0.06 + + """ + @spec get_cost(atom(), String.t(), non_neg_integer(), non_neg_integer()) :: float() + def get_cost(provider, model, input_tokens, output_tokens) do + config = ProviderRegistry.get_config(provider) + + case get_in(config, [:cost_per_1k_tokens, model]) do + nil -> 0.0 + costs -> + input_cost = Map.get(costs, :input, 0) * (input_tokens / 1000) + output_cost = Map.get(costs, :output, 0) * (output_tokens / 1000) + input_cost + output_cost + end + end + + # Private functions + + defp default_opts do + %{priority: :balanced, features: [], exclude: [], prefer: []} + end + + defp get_candidates(opts) do + ProviderRegistry.list_enabled() + |> Enum.reject(fn provider -> provider in Map.get(opts, :exclude, []) end) + |> Enum.flat_map(fn provider -> + config = ProviderRegistry.get_config(provider) + models = Map.get(config, :models, []) + + models + |> Enum.filter(fn model -> model_matches_requirements?(provider, model, opts) end) + |> Enum.map(fn model -> + score = calculate_score(provider, model, config, opts) + {provider, model, score} + end) + end) + |> Enum.sort_by(fn {_p, _m, score} -> score end, :desc) + end + + defp model_matches_requirements?(provider, model, opts) do + config = ProviderRegistry.get_config(provider) + model_config = get_model_config(config, model) + + # Check max tokens requirement + max_tokens_ok = case Map.get(opts, :max_tokens) do + nil -> true + required -> + Map.get(model_config, :max_tokens, config[:max_tokens] || 0) >= required + end + + # Check features requirement + features_ok = case Map.get(opts, :features, []) do + [] -> true + required_features -> + available_features = Map.get(config, :features, []) + Enum.all?(required_features, fn f -> f in available_features end) + end + + max_tokens_ok and features_ok + end + + defp get_model_config(config, model) do + # Model-specific config if available + Map.get(config, :models_config, %{}) + |> Map.get(model, %{}) + end + + defp calculate_score(provider, model, config, opts) do + priority = Map.get(opts, :priority, :balanced) + prefer = Map.get(opts, :prefer, []) + + base_score = calculate_base_score(priority, provider, model, config) + preference_bonus = if provider in prefer, do: 0.2, else: 0.0 + + # Get performance data for latency scoring + stats = PerformanceMonitor.get_stats(provider) + latency_score = calculate_latency_score(stats) + + case priority do + :cost -> base_score * 0.8 + latency_score * 0.2 + preference_bonus + :speed -> base_score * 0.3 + latency_score * 0.7 + preference_bonus + :quality -> base_score * 0.9 + latency_score * 0.1 + preference_bonus + :balanced -> base_score * 0.5 + latency_score * 0.5 + preference_bonus + end + end + + defp calculate_base_score(:cost, provider, model, config) do + costs = get_in(config, [:cost_per_1k_tokens, model]) || %{input: 0.01, output: 0.01} + total_cost = costs[:input] + costs[:output] + + # Lower cost = higher score + 1.0 / (total_cost + 0.001) + end + + defp calculate_base_score(:speed, _provider, _model, _config) do + # Speed is calculated from actual performance data + 0.5 + end + + defp calculate_base_score(:quality, provider, model, _config) do + # Quality ranking based on model family + quality_scores = %{ + {"openai", "gpt-4"} => 0.95, + {"openai", "gpt-4-turbo"} => 0.92, + {"anthropic", "claude-3-opus-20240229"} => 0.96, + {"anthropic", "claude-3-sonnet-20240229"} => 0.85, + {"anthropic", "claude-3-haiku-20240307"} => 0.75, + {"openai", "gpt-3.5-turbo"} => 0.70 + } + + Map.get(quality_scores, {Atom.to_string(provider), model}, 0.5) + end + + defp calculate_base_score(:balanced, provider, model, config) do + quality = calculate_base_score(:quality, provider, model, config) + cost = calculate_base_score(:cost, provider, model, config) + + (quality + cost) / 2 + end + + defp calculate_latency_score(stats) do + case Map.get(stats, :avg_latency_ms) do + nil -> 0.5 + latency -> + # Lower latency = higher score + # Normalize: 100ms = 1.0, 5000ms = 0.1 + max(0.1, 1.0 - (latency / 5000)) + end + end + + defp rank_and_select(candidates, _opts) do + case candidates do + [{provider, model, _score} | _] -> {provider, model} + [] -> {:error, :no_candidates} + end + end +end \ No newline at end of file diff --git a/lux/lib/lux/llm/ollama.ex b/lux/lib/lux/llm/ollama.ex new file mode 100644 index 00000000..c35e7c0e --- /dev/null +++ b/lux/lib/lux/llm/ollama.ex @@ -0,0 +1,308 @@ +defmodule Lux.LLM.Ollama do + @moduledoc """ + Ollama provider for local LLM support. + + Enables self-hosted LLM capabilities with optimized performance for local models. + Supports model management, caching, and resource controls. + + ## Features + + - Local model support via Ollama + - Model download and caching + - Resource management + - Performance optimization + - No API costs (self-hosted) + + ## Configuration + + Add to your config: + + config :lux, :ollama, + host: "http://localhost:11434", + default_model: "llama2", + timeout: 120_000, + max_retries: 3 + + ## Usage + + # Basic call + {:ok, response} = Lux.LLM.Ollama.call("Hello!", [], %{}) + + # With specific model + {:ok, response} = Lux.LLM.Ollama.call("Hello!", [], %{model: "llama2"}) + + # List available models + {:ok, models} = Lux.LLM.Ollama.list_models() + + # Pull a new model + {:ok, :pulling} = Lux.LLM.Ollama.pull_model("mistral") + + """ + + alias Lux.LLM.ProviderRegistry + + @default_host "http://localhost:11434" + @default_timeout 120_000 + @default_model "llama2" + + @doc """ + Call the Ollama API with a prompt. + + ## Options + + * `:model` - Model to use (default: "llama2") + * `:host` - Ollama host (default: "http://localhost:11434") + * `:timeout` - Request timeout (default: 120_000ms) + * `:stream` - Enable streaming (default: false) + * `:temperature` - Sampling temperature (default: 0.7) + * `:max_tokens` - Maximum tokens to generate + + ## Examples + + iex> Lux.LLM.Ollama.call("Hello!", [], %{}) + {:ok, %Lux.LLM.Response{...}} + + iex> Lux.LLM.Ollama.call("Write a poem", [], %{model: "mistral"}) + {:ok, %Lux.LLM.Response{...}} + + """ + @spec call(String.t(), list(), map()) :: {:ok, map()} | {:error, term()} + def call(prompt, tools, opts \\ %{}) do + host = Map.get(opts, :host, get_host()) + model = Map.get(opts, :model, @default_model) + timeout = Map.get(opts, :timeout, @default_timeout) + + url = "#{host}/api/generate" + + body = %{ + model: model, + prompt: prompt, + stream: Map.get(opts, :stream, false), + options: %{ + temperature: Map.get(opts, :temperature, 0.7), + num_predict: Map.get(opts, :max_tokens, 2048) + } + } + + headers = [{"Content-Type", "application/json"}] + + case make_request(:post, url, body, headers, timeout) do + {:ok, %Req.Response{status: 200, body: response_body}} -> + handle_success_response(response_body, model) + + {:ok, %Req.Response{status: status, body: error_body}} -> + {:error, {:api_error, status, error_body}} + + {:error, reason} -> + {:error, reason} + end + end + + @doc """ + List available models from Ollama. + + ## Examples + + iex> Lux.LLM.Ollama.list_models() + {:ok, ["llama2", "mistral", "codellama"]} + + """ + @spec list_models(keyword()) :: {:ok, [String.t()]} | {:error, term()} + def list_models(opts \\ []) do + host = Keyword.get(opts, :host, get_host()) + url = "#{host}/api/tags" + + case make_request(:get, url, nil, [], 30_000) do + {:ok, %Req.Response{status: 200, body: %{"models" => models}}} -> + model_names = Enum.map(models, fn %{"name" => name} -> name end) + {:ok, model_names} + + {:ok, %Req.Response{status: status}} -> + {:error, {:api_error, status}} + + {:error, reason} -> + {:error, reason} + end + end + + @doc """ + Pull a model from Ollama registry. + + ## Examples + + iex> Lux.LLM.Ollama.pull_model("mistral") + {:ok, :pulling} + + """ + @spec pull_model(String.t(), keyword()) :: {:ok, :pulling} | {:error, term()} + def pull_model(model_name, opts \\ []) do + host = Keyword.get(opts, :host, get_host()) + url = "#{host}/api/pull" + + body = %{name: model_name} + + case make_request(:post, url, body, [{"Content-Type", "application/json"}], 300_000) do + {:ok, %Req.Response{status: 200}} -> + {:ok, :pulling} + + {:ok, %Req.Response{status: status}} -> + {:error, {:api_error, status}} + + {:error, reason} -> + {:error, reason} + end + end + + @doc """ + Get model information. + + ## Examples + + iex> Lux.LLM.Ollama.get_model_info("llama2") + {:ok, %{size: "4.7GB", ...}} + + """ + @spec get_model_info(String.t(), keyword()) :: {:ok, map()} | {:error, term()} + def get_model_info(model_name, opts \\ []) do + host = Keyword.get(opts, :host, get_host()) + url = "#{host}/api/show" + + body = %{name: model_name} + + case make_request(:post, url, body, [{"Content-Type", "application/json"}], 30_000) do + {:ok, %Req.Response{status: 200, body: info}} -> + {:ok, info} + + {:ok, %Req.Response{status: status}} -> + {:error, {:api_error, status}} + + {:error, reason} -> + {:error, reason} + end + end + + @doc """ + Delete a model. + + ## Examples + + iex> Lux.LLM.Ollama.delete_model("old_model") + :ok + + """ + @spec delete_model(String.t(), keyword()) :: :ok | {:error, term()} + def delete_model(model_name, opts \\ []) do + host = Keyword.get(opts, :host, get_host()) + url = "#{host}/api/delete" + + body = %{name: model_name} + + case make_request(:delete, url, body, [{"Content-Type", "application/json"}], 30_000) do + {:ok, %Req.Response{status: 200}} -> + :ok + + {:ok, %Req.Response{status: status}} -> + {:error, {:api_error, status}} + + {:error, reason} -> + {:error, reason} + end + end + + @doc """ + Check if Ollama server is running. + + ## Examples + + iex> Lux.LLM.Ollama.health_check() + :ok + + """ + @spec health_check(keyword()) :: :ok | {:error, term()} + def health_check(opts \\ []) do + host = Keyword.get(opts, :host, get_host()) + url = "#{host}/api/tags" + + case make_request(:get, url, nil, [], 5_000) do + {:ok, %Req.Response{status: 200}} -> :ok + {:error, reason} -> {:error, reason} + end + end + + @doc """ + Register Ollama as a provider. + + This allows Ollama to be used through the Provider abstraction. + + ## Examples + + iex> Lux.LLM.Ollama.register() + :ok + + """ + @spec register(keyword()) :: :ok + def register(opts \\ []) do + models = Keyword.get(opts, :models, [@default_model, "mistral", "codellama"]) + + ProviderRegistry.register(:ollama, %{ + module: __MODULE__, + models: models, + cost_per_1k_tokens: %{default: %{input: 0.0, output: 0.0}}, # Free! + max_tokens: 4096, + features: [:streaming, :local, :no_cost], + priority: Keyword.get(opts, :priority, 4), + enabled: true, + host: Keyword.get(opts, :host, get_host()) + }) + + :ok + end + + # Private functions + + defp get_host do + Application.get_env(:lux, :ollama, []) + |> Keyword.get(:host, @default_host) + end + + defp make_request(method, url, body, headers, timeout) do + req = Req.new(base_url: "", receive_timeout: timeout) + + options = [headers: headers] + options = if body, do: Keyword.put(options, :json, body), else: options + + case method do + :get -> Req.get(req, url: url) + :post -> Req.post(req, url: url, json: body) + :delete -> Req.delete(req, url: url, json: body) + end + end + + defp handle_success_response(response_body, model) do + case response_body do + %{"response" => text} -> + {:ok, %{ + text: text, + model: model, + total_tokens: Map.get(response_body, "eval_count", 0), + prompt_tokens: Map.get(response_body, "prompt_eval_count", 0), + done: Map.get(response_body, "done", true), + context: Map.get(response_body, "context", []) + }} + + response when is_binary(response) -> + {:ok, %{ + text: response, + model: model, + done: true + }} + + other -> + {:ok, %{ + text: inspect(other), + model: model, + done: true + }} + end + end +end \ No newline at end of file diff --git a/lux/lib/lux/llm/performance_monitor.ex b/lux/lib/lux/llm/performance_monitor.ex new file mode 100644 index 00000000..108684fe --- /dev/null +++ b/lux/lib/lux/llm/performance_monitor.ex @@ -0,0 +1,305 @@ +defmodule Lux.LLM.PerformanceMonitor do + @moduledoc """ + Monitors LLM provider performance. + + Tracks: + - Request latency + - Success/failure rates + - Throughput + - Error patterns + + ## Usage + + # Record a request + PerformanceMonitor.record_request(:openai, 250, :success) + + # Get stats for a provider + stats = PerformanceMonitor.get_stats(:openai) + + # Get all stats + all_stats = PerformanceMonitor.get_all_stats() + + """ + + use Agent + + @table_name :lux_llm_performance_monitor + + @doc """ + Start the performance monitor agent. + """ + def start_link(_opts \\ []) do + Agent.start_link(fn -> init_monitor() end, name: __MODULE__) + end + + @doc """ + Record a request result. + + ## Parameters + + * `provider` - The LLM provider + * `latency_ms` - Request latency in milliseconds + * `result` - :success or :failure + + ## Examples + + iex> PerformanceMonitor.record_request(:openai, 250, :success) + :ok + + """ + @spec record_request(atom(), non_neg_integer(), :success | :failure) :: :ok + def record_request(provider, latency_ms, result) do + Agent.update(__MODULE__, fn state -> + now = System.monotonic_time(:millisecond) + + # Get current stats + current = case :ets.lookup(@table_name, {:stats, provider}) do + [{_, data}] -> data + [] -> default_stats() + end + + # Update stats + updated = %{ + total_requests: current.total_requests + 1, + successful_requests: current.successful_requests + if(result == :success, do: 1, else: 0), + failed_requests: current.failed_requests + if(result == :failure, do: 1, else: 0), + total_latency_ms: current.total_latency_ms + latency_ms, + min_latency_ms: min(current.min_latency_ms || latency_ms, latency_ms), + max_latency_ms: max(current.max_latency_ms || latency_ms, latency_ms), + avg_latency_ms: nil, # Calculated on demand + last_request_time: now, + consecutive_failures: if(result == :failure, do: current.consecutive_failures + 1, else: 0), + last_failure_time: if(result == :failure, do: now, else: current.last_failure_time) + } + + :ets.insert(@table_name, {{:stats, provider}, updated}) + + Map.put(state, provider, updated) + end) + end + + @doc """ + Get performance stats for a provider. + + ## Examples + + iex> PerformanceMonitor.get_stats(:openai) + %{total_requests: 100, avg_latency_ms: 250, ...} + + """ + @spec get_stats(atom()) :: map() + def get_stats(provider) do + case :ets.lookup(@table_name, {:stats, provider}) do + [{_, data}] -> calculate_derived_stats(data) + [] -> default_stats() + end + end + + @doc """ + Get stats for all providers. + + ## Examples + + iex> PerformanceMonitor.get_all_stats() + %{openai: %{...}, anthropic: %{...}} + + """ + @spec get_all_stats() :: map() + def get_all_stats do + :ets.tab2list(@table_name) + |> Enum.filter(fn + {{:stats, _}, _} -> true + _ -> false + end) + |> Enum.map(fn {{:stats, provider}, data} -> {provider, calculate_derived_stats(data)} end) + |> Map.new() + end + + @doc """ + Reset stats for a provider. + + ## Examples + + iex> PerformanceMonitor.reset(:openai) + :ok + + """ + @spec reset(atom()) :: :ok + def reset(provider) do + Agent.update(__MODULE__, fn state -> + :ets.insert(@table_name, {{:stats, provider}, default_stats()}) + Map.put(state, provider, default_stats()) + end) + end + + @doc """ + Reset all stats. + + ## Examples + + iex> PerformanceMonitor.reset_all() + :ok + + """ + @spec reset_all() :: :ok + def reset_all do + Agent.update(__MODULE__, fn state -> + :ets.delete_all_objects(@table_name) + state + end) + end + + @doc """ + Increment consecutive failures for a provider. + + ## Examples + + iex> PerformanceMonitor.increment_consecutive_failures(:openai) + :ok + + """ + @spec increment_consecutive_failures(atom()) :: :ok + def increment_consecutive_failures(provider) do + Agent.update(__MODULE__, fn state -> + current = case :ets.lookup(@table_name, {:stats, provider}) do + [{_, data}] -> data + [] -> default_stats() + end + + updated = %{current | + consecutive_failures: current.consecutive_failures + 1, + last_failure_time: System.monotonic_time(:millisecond) + } + + :ets.insert(@table_name, {{:stats, provider}, updated}) + Map.put(state, provider, updated) + end) + end + + @doc """ + Reset consecutive failures for a provider. + + ## Examples + + iex> PerformanceMonitor.reset_consecutive_failures(:openai) + :ok + + """ + @spec reset_consecutive_failures(atom()) :: :ok + def reset_consecutive_failures(provider) do + Agent.update(__MODULE__, fn state -> + current = case :ets.lookup(@table_name, {:stats, provider}) do + [{_, data}] -> data + [] -> default_stats() + end + + updated = %{current | consecutive_failures: 0} + :ets.insert(@table_name, {{:stats, provider}, updated}) + Map.put(state, provider, updated) + end) + end + + @doc """ + Get health status for a provider. + + ## Examples + + iex> PerformanceMonitor.get_health(:openai) + :healthy + + """ + @spec get_health(atom()) :: :healthy | :degraded | :unhealthy + def get_health(provider) do + stats = get_stats(provider) + + success_rate = if stats.total_requests > 0 do + stats.successful_requests / stats.total_requests + else + 1.0 + end + + cond do + stats.consecutive_failures >= 5 -> :unhealthy + success_rate < 0.5 -> :degraded + success_rate < 0.8 -> :degraded + true -> :healthy + end + end + + @doc """ + Get aggregate stats. + + ## Examples + + iex> PerformanceMonitor.get_aggregate_stats() + %{total_requests: 500, avg_latency_ms: 200, ...} + + """ + @spec get_aggregate_stats() :: map() + def get_aggregate_stats do + all_stats = get_all_stats() + + if map_size(all_stats) == 0 do + default_stats() + else + values = Map.values(all_stats) + + total_requests = Enum.reduce(values, 0, fn s, acc -> acc + s.total_requests end) + total_successful = Enum.reduce(values, 0, fn s, acc -> acc + s.successful_requests end) + total_failed = Enum.reduce(values, 0, fn s, acc -> acc + s.failed_requests end) + total_latency = Enum.reduce(values, 0, fn s, acc -> acc + (s.total_latency_ms || 0) end) + + %{ + total_requests: total_requests, + successful_requests: total_successful, + failed_requests: total_failed, + success_rate: if(total_requests > 0, do: total_successful / total_requests, else: 0.0), + avg_latency_ms: if(total_requests > 0, do: total_latency / total_requests, else: 0.0), + providers: map_size(all_stats) + } + end + end + + # Private functions + + defp init_monitor do + if :ets.whereis(@table_name) == :undefined do + :ets.new(@table_name, [:named_table, :set, :public]) + end + %{} + end + + defp default_stats do + %{ + total_requests: 0, + successful_requests: 0, + failed_requests: 0, + total_latency_ms: 0, + min_latency_ms: nil, + max_latency_ms: nil, + avg_latency_ms: 0.0, + last_request_time: nil, + consecutive_failures: 0, + last_failure_time: nil + } + end + + defp calculate_derived_stats(data) do + avg = if data.total_requests > 0 do + data.total_latency_ms / data.total_requests + else + 0.0 + end + + success_rate = if data.total_requests > 0 do + data.successful_requests / data.total_requests + else + 0.0 + end + + Map.merge(data, %{ + avg_latency_ms: Float.round(avg, 2), + success_rate: Float.round(success_rate, 4) + }) + end +end \ No newline at end of file diff --git a/lux/lib/lux/llm/provider.ex b/lux/lib/lux/llm/provider.ex new file mode 100644 index 00000000..d8190777 --- /dev/null +++ b/lux/lib/lux/llm/provider.ex @@ -0,0 +1,238 @@ +defmodule Lux.LLM.Provider do + @moduledoc """ + Universal LLM Provider interface and registry. + + This module provides a unified interface for multiple LLM providers, + with automatic model selection, fallback handling, and performance monitoring. + + ## Features + + - Universal provider interface + - Automatic model selection based on task requirements + - Smart fallback handling when providers fail + - Cost tracking and optimization + - Performance monitoring and analytics + + ## Usage + + # Call with automatic provider selection + {:ok, response} = Lux.LLM.Provider.call("Hello, world!", [], %{}) + + # Call with specific provider + {:ok, response} = Lux.LLM.Provider.call("Hello!", [], %{ + provider: :openai, + model: "gpt-4" + }) + + # Get available providers + providers = Lux.LLM.Provider.list_providers() + + # Get provider stats + stats = Lux.LLM.Provider.get_stats(:openai) + """ + + alias Lux.LLM.ProviderRegistry + alias Lux.LLM.ModelSelector + alias Lux.LLM.FallbackHandler + alias Lux.LLM.CostTracker + alias Lux.LLM.PerformanceMonitor + + @type provider_name :: atom() + @type model_name :: String.t() + @type config :: map() + + @doc """ + Call an LLM provider with automatic selection and fallback. + + ## Options + + * `:provider` - Specific provider to use (optional) + * `:model` - Specific model to use (optional) + * `:fallback` - List of fallback providers (default: all registered) + * `:max_retries` - Maximum retry attempts (default: 3) + * `:timeout` - Request timeout in ms (default: 60000) + * `:cost_limit` - Maximum cost in USD (optional) + * `:priority` - Selection priority: :cost | :speed | :quality (default: :balanced) + + ## Examples + + iex> Lux.LLM.Provider.call("Hello!", [], %{}) + {:ok, %Lux.LLM.Response{...}} + + iex> Lux.LLM.Provider.call("Hello!", [], %{provider: :openai}) + {:ok, %Lux.LLM.Response{...}} + + """ + @spec call(String.t(), list(), map()) :: {:ok, map()} | {:error, term()} + def call(prompt, tools, opts \\ %{}) do + with {:ok, config} <- validate_config(opts), + {:ok, provider_chain} <- build_provider_chain(config), + {:ok, response} <- execute_with_fallback(prompt, tools, provider_chain, config) do + {:ok, response} + end + end + + @doc """ + Register a new LLM provider. + + ## Options + + * `:module` - Provider module (required) + * `:models` - List of supported models (required) + * `:priority` - Default priority (default: 100) + * `:cost_per_1k_tokens` - Cost per 1000 tokens (optional) + * `:max_tokens` - Maximum context length (optional) + * `:features` - Supported features (optional) + + ## Examples + + iex> Lux.LLM.Provider.register(:openai, %{ + ...> module: Lux.LLM.OpenAI, + ...> models: ["gpt-4", "gpt-3.5-turbo"], + ...> cost_per_1k_tokens: 0.03 + ...> }) + :ok + + """ + @spec register(provider_name(), map()) :: :ok | {:error, term()} + def register(name, config) do + ProviderRegistry.register(name, config) + end + + @doc """ + Unregister an LLM provider. + + ## Examples + + iex> Lux.LLM.Provider.unregister(:deprecated_provider) + :ok + + """ + @spec unregister(provider_name()) :: :ok + def unregister(name) do + ProviderRegistry.unregister(name) + end + + @doc """ + List all registered providers. + + ## Examples + + iex> Lux.LLM.Provider.list_providers() + [:openai, :anthropic, :together_ai] + + """ + @spec list_providers() :: [provider_name()] + def list_providers do + ProviderRegistry.list_providers() + end + + @doc """ + Get provider configuration. + + ## Examples + + iex> Lux.LLM.Provider.get_config(:openai) + %{module: Lux.LLM.OpenAI, models: ["gpt-4", ...], ...} + + """ + @spec get_config(provider_name()) :: map() | nil + def get_config(name) do + ProviderRegistry.get_config(name) + end + + @doc """ + Get provider statistics. + + ## Examples + + iex> Lux.LLM.Provider.get_stats(:openai) + %{requests: 100, tokens: 50000, cost: 1.5, avg_latency: 250} + + """ + @spec get_stats(provider_name()) :: map() + def get_stats(name) do + PerformanceMonitor.get_stats(name) + end + + @doc """ + Get aggregate statistics for all providers. + + ## Examples + + iex> Lux.LLM.Provider.get_all_stats() + %{total_requests: 500, total_cost: 10.5, ...} + + """ + @spec get_all_stats() :: map() + def get_all_stats do + PerformanceMonitor.get_all_stats() + end + + @doc """ + Select the best provider for a given task. + + ## Options + + * `:priority` - :cost | :speed | :quality | :balanced + * `:max_tokens` - Required token count + * `:features` - Required features + + ## Examples + + iex> Lux.LLM.Provider.select_best(%{priority: :cost}) + {:openai, "gpt-3.5-turbo"} + + """ + @spec select_best(map()) :: {provider_name(), model_name()} | {:error, term()} + def select_best(opts \\ %{}) do + ModelSelector.select_best(opts) + end + + @doc """ + Reset statistics for a provider. + + ## Examples + + iex> Lux.LLM.Provider.reset_stats(:openai) + :ok + + """ + @spec reset_stats(provider_name()) :: :ok + def reset_stats(name) do + PerformanceMonitor.reset(name) + end + + # Private functions + + defp validate_config(opts) do + {:ok, Map.merge(default_config(), opts)} + end + + defp default_config do + %{ + fallback: :all, + max_retries: 3, + timeout: 60_000, + priority: :balanced + } + end + + defp build_provider_chain(config) do + case Map.get(config, :provider) do + nil -> + {:ok, ModelSelector.build_chain(config)} + + provider -> + model = Map.get(config, :model) + fallback = Map.get(config, :fallback, :all) + + chain = [{provider, model} | FallbackHandler.get_fallbacks(provider, fallback)] + {:ok, chain} + end + end + + defp execute_with_fallback(prompt, tools, provider_chain, config) do + FallbackHandler.execute(prompt, tools, provider_chain, config) + end +end \ No newline at end of file diff --git a/lux/lib/lux/llm/provider_registry.ex b/lux/lib/lux/llm/provider_registry.ex new file mode 100644 index 00000000..ccabfa02 --- /dev/null +++ b/lux/lib/lux/llm/provider_registry.ex @@ -0,0 +1,321 @@ +defmodule Lux.LLM.ProviderRegistry do + @moduledoc """ + Registry for LLM providers. + + Manages provider registration, configuration, and lookup. + All providers are stored in an ETS table for fast concurrent access. + + ## Provider Configuration + + Each provider has the following configuration: + + * `:module` - The provider module implementing `Lux.LLM` behaviour + * `:models` - List of supported model names + * `:priority` - Default priority for selection (lower = higher priority) + * `:cost_per_1k_tokens` - Cost per 1000 tokens (input/output) + * `:max_tokens` - Maximum context length + * `:features` - Supported features (streaming, tools, etc.) + * `:rate_limit` - Rate limiting configuration + * `:enabled` - Whether the provider is enabled + + ## Usage + + # Register a provider + ProviderRegistry.register(:openai, %{ + module: Lux.LLM.OpenAI, + models: ["gpt-4", "gpt-3.5-turbo"], + cost_per_1k_tokens: %{input: 0.03, output: 0.06}, + max_tokens: 8192, + features: [:streaming, :tools, :json_mode], + priority: 1 + }) + + # Get provider config + config = ProviderRegistry.get_config(:openai) + + # List all providers + providers = ProviderRegistry.list_providers() + + """ + + use Agent + + @table_name :lux_llm_provider_registry + + @type provider_name :: atom() + @type provider_config :: map() + + @doc """ + Start the provider registry. + """ + def start_link(_opts \\ []) do + Agent.start_link(fn -> init_registry() end, name: __MODULE__) + end + + @doc """ + Register a new provider. + + ## Examples + + iex> ProviderRegistry.register(:openai, %{module: Lux.LLM.OpenAI}) + :ok + + """ + @spec register(provider_name(), provider_config()) :: :ok | {:error, term()} + def register(name, config) do + with :ok <- validate_config(config) do + Agent.update(__MODULE__, fn state -> + :ets.insert(@table_name, {name, Map.put(config, :registered_at, DateTime.utc_now())}) + Map.put(state, name, config) + end) + end + end + + @doc """ + Unregister a provider. + + ## Examples + + iex> ProviderRegistry.unregister(:deprecated_provider) + :ok + + """ + @spec unregister(provider_name()) :: :ok + def unregister(name) do + Agent.update(__MODULE__, fn state -> + :ets.delete(@table_name, name) + Map.delete(state, name) + end) + end + + @doc """ + Get provider configuration. + + ## Examples + + iex> ProviderRegistry.get_config(:openai) + %{module: Lux.LLM.OpenAI, ...} + + """ + @spec get_config(provider_name()) :: provider_config() | nil + def get_config(name) do + case :ets.lookup(@table_name, name) do + [{^name, config}] -> config + [] -> nil + end + end + + @doc """ + List all registered providers. + + ## Examples + + iex> ProviderRegistry.list_providers() + [:openai, :anthropic, :together_ai] + + """ + @spec list_providers() :: [provider_name()] + def list_providers do + :ets.match(@table_name, {:"$1", :_}) + |> List.flatten() + end + + @doc """ + List enabled providers only. + + ## Examples + + iex> ProviderRegistry.list_enabled() + [:openai, :anthropic] + + """ + @spec list_enabled() :: [provider_name()] + def list_enabled do + :ets.tab2list(@table_name) + |> Enum.filter(fn {_name, config} -> Map.get(config, :enabled, true) end) + |> Enum.map(fn {name, _config} -> name end) + end + + @doc """ + Check if a provider is registered. + + ## Examples + + iex> ProviderRegistry.registered?(:openai) + true + + """ + @spec registered?(provider_name()) :: boolean() + def registered?(name) do + :ets.member(@table_name, name) + end + + @doc """ + Enable a provider. + + ## Examples + + iex> ProviderRegistry.enable(:openai) + :ok + + """ + @spec enable(provider_name()) :: :ok | {:error, :not_found} + def enable(name) do + case get_config(name) do + nil -> {:error, :not_found} + config -> + Agent.update(__MODULE__, fn state -> + updated_config = Map.put(config, :enabled, true) + :ets.insert(@table_name, {name, updated_config}) + Map.put(state, name, updated_config) + end) + end + end + + @doc """ + Disable a provider. + + ## Examples + + iex> ProviderRegistry.disable(:deprecated_provider) + :ok + + """ + @spec disable(provider_name()) :: :ok | {:error, :not_found} + def disable(name) do + case get_config(name) do + nil -> {:error, :not_found} + config -> + Agent.update(__MODULE__, fn state -> + updated_config = Map.put(config, :enabled, false) + :ets.insert(@table_name, {name, updated_config}) + Map.put(state, name, updated_config) + end) + end + end + + @doc """ + Get models for a provider. + + ## Examples + + iex> ProviderRegistry.get_models(:openai) + ["gpt-4", "gpt-3.5-turbo"] + + """ + @spec get_models(provider_name()) :: [String.t()] | [] + def get_models(name) do + case get_config(name) do + nil -> [] + config -> Map.get(config, :models, []) + end + end + + @doc """ + Update provider configuration. + + ## Examples + + iex> ProviderRegistry.update(:openai, %{priority: 2}) + :ok + + """ + @spec update(provider_name(), map()) :: :ok | {:error, :not_found} + def update(name, updates) do + case get_config(name) do + nil -> {:error, :not_found} + config -> + Agent.update(__MODULE__, fn state -> + updated_config = Map.merge(config, updates) + :ets.insert(@table_name, {name, updated_config}) + Map.put(state, name, updated_config) + end) + end + end + + # Private functions + + defp init_registry do + if :ets.whereis(@table_name) == :undefined do + :ets.new(@table_name, [:named_table, :set, :public, read_concurrency: true]) + end + register_default_providers() + end + + defp register_default_providers do + # Register OpenAI + :ets.insert(@table_name, {:openai, %{ + module: Lux.LLM.OpenAI, + models: ["gpt-4", "gpt-4-turbo", "gpt-3.5-turbo"], + cost_per_1k_tokens: %{ + "gpt-4" => %{input: 0.03, output: 0.06}, + "gpt-4-turbo" => %{input: 0.01, output: 0.03}, + "gpt-3.5-turbo" => %{input: 0.0005, output: 0.0015} + }, + max_tokens: 128000, + features: [:streaming, :tools, :json_mode, :vision], + priority: 1, + enabled: true, + registered_at: DateTime.utc_now() + }}) + + # Register Anthropic + :ets.insert(@table_name, {:anthropic, %{ + module: Lux.LLM.Anthropic, + models: ["claude-3-opus-20240229", "claude-3-sonnet-20240229", "claude-3-haiku-20240307"], + cost_per_1k_tokens: %{ + "claude-3-opus-20240229" => %{input: 0.015, output: 0.075}, + "claude-3-sonnet-20240229" => %{input: 0.003, output: 0.015}, + "claude-3-haiku-20240307" => %{input: 0.00025, output: 0.00125} + }, + max_tokens: 200000, + features: [:streaming, :tools, :vision], + priority: 2, + enabled: true, + registered_at: DateTime.utc_now() + }}) + + # Register Together AI + :ets.insert(@table_name, {:together_ai, %{ + module: Lux.LLM.TogetherAI, + models: ["mistralai/Mixtral-8x7B-Instruct-v0.1", "meta-llama/Llama-3-70b-chat-hf"], + cost_per_1k_tokens: %{ + "mistralai/Mixtral-8x7B-Instruct-v0.1" => %{input: 0.0006, output: 0.0006}, + "meta-llama/Llama-3-70b-chat-hf" => %{input: 0.0009, output: 0.0009} + }, + max_tokens: 8192, + features: [:streaming, :tools], + priority: 3, + enabled: true, + registered_at: DateTime.utc_now() + }}) + + # Register Ollama (local models - zero cost!) + :ets.insert(@table_name, {:ollama, %{ + module: Lux.LLM.Ollama, + models: ["llama2", "mistral", "codellama", "llama3"], + cost_per_1k_tokens: %{ + default: %{input: 0.0, output: 0.0} + }, + max_tokens: 4096, + features: [:streaming, :local, :no_cost], + priority: 4, + enabled: true, + registered_at: DateTime.utc_now() + }}) + + %{} + end + + defp validate_config(config) do + required_keys = [:module] + + missing_keys = Enum.filter(required_keys, fn key -> not Map.has_key?(config, key) end) + + if Enum.empty?(missing_keys) do + :ok + else + {:error, {:missing_keys, missing_keys}} + end + end +end \ No newline at end of file diff --git a/lux/test/llm/cost_tracker_test.exs b/lux/test/llm/cost_tracker_test.exs new file mode 100644 index 00000000..95bd34f7 --- /dev/null +++ b/lux/test/llm/cost_tracker_test.exs @@ -0,0 +1,73 @@ +defmodule Lux.LLM.CostTrackerTest do + use ExUnit.Case, async: true + + alias Lux.LLM.CostTracker + + setup do + # Reset cost tracker before each test + CostTracker.reset() + :ok + end + + describe "record/4" do + test "records cost correctly" do + :ok = CostTracker.record(:openai, "gpt-4", 1000, 500) + + total = CostTracker.get_total_costs() + assert total.requests == 1 + assert total.cost > 0 + end + end + + describe "calculate_cost/4" do + test "calculates GPT-4 cost correctly" do + cost = CostTracker.calculate_cost(:openai, "gpt-4", 1000, 1000) + + # GPT-4: $0.03/1k input + $0.06/1k output + assert_in_delta cost, 0.09, 0.001 + end + + test "calculates GPT-3.5-turbo cost correctly" do + cost = CostTracker.calculate_cost(:openai, "gpt-3.5-turbo", 1000, 1000) + + # GPT-3.5: $0.0005/1k input + $0.0015/1k output + assert_in_delta cost, 0.002, 0.001 + end + end + + describe "get_costs_by_provider/1" do + test "returns provider costs" do + CostTracker.record(:openai, "gpt-4", 1000, 500) + CostTracker.record(:openai, "gpt-4", 500, 250) + + costs = CostTracker.get_costs_by_provider(:openai) + + assert costs.requests == 2 + assert costs.cost > 0 + end + end + + describe "set_budget/2 and budget_exceeded?/1" do + test "tracks budget" do + CostTracker.set_budget(:daily, 0.01) + + refute CostTracker.budget_exceeded?(:daily) + + # Record enough to exceed budget + CostTracker.record(:openai, "gpt-4", 1000, 1000) + + assert CostTracker.budget_exceeded?(:daily) + end + end + + describe "get_history/1" do + test "returns request history" do + CostTracker.record(:openai, "gpt-4", 1000, 500) + CostTracker.record(:anthropic, "claude-3-opus-20240229", 500, 250) + + history = CostTracker.get_history(10) + + assert length(history) == 2 + end + end +end \ No newline at end of file diff --git a/lux/test/llm/fallback_handler_test.exs b/lux/test/llm/fallback_handler_test.exs new file mode 100644 index 00000000..10565f46 --- /dev/null +++ b/lux/test/llm/fallback_handler_test.exs @@ -0,0 +1,29 @@ +defmodule Lux.LLM.FallbackHandlerTest do + use ExUnit.Case, async: true + + alias Lux.LLM.FallbackHandler + + describe "get_fallbacks/2" do + test "returns all other providers with :all strategy" do + fallbacks = FallbackHandler.get_fallbacks(:openai, :all) + + assert is_list(fallbacks) + refute {:openai, nil} in fallbacks + end + + test "returns empty list with :none strategy" do + fallbacks = FallbackHandler.get_fallbacks(:openai, :none) + + assert fallbacks == [] + end + + test "returns same tier providers with :same_tier strategy" do + fallbacks = FallbackHandler.get_fallbacks(:openai, :same_tier) + + # OpenAI and Anthropic are same tier + providers = Enum.map(fallbacks, fn {p, _} -> p end) + + assert :anthropic in providers + end + end +end \ No newline at end of file diff --git a/lux/test/llm/model_selector_test.exs b/lux/test/llm/model_selector_test.exs new file mode 100644 index 00000000..33984328 --- /dev/null +++ b/lux/test/llm/model_selector_test.exs @@ -0,0 +1,68 @@ +defmodule Lux.LLM.ModelSelectorTest do + use ExUnit.Case, async: true + + alias Lux.LLM.ModelSelector + + describe "select_best/1" do + test "selects provider based on cost priority" do + {provider, _model} = ModelSelector.select_best(%{priority: :cost}) + + # Together AI should be cheapest + assert provider == :together_ai + end + + test "selects provider based on quality priority" do + {_provider, model} = ModelSelector.select_best(%{priority: :quality}) + + # Should select a high quality model + assert model in ["claude-3-opus-20240229", "gpt-4", "gpt-4-turbo"] + end + + test "filters by features" do + {provider, model} = ModelSelector.select_best(%{ + priority: :quality, + features: [:vision] + }) + + # Only OpenAI and Anthropic have vision + assert provider in [:openai, :anthropic] + end + + test "excludes specified providers" do + {provider, _model} = ModelSelector.select_best(%{ + priority: :cost, + exclude: [:together_ai] + }) + + # Should not select together_ai + refute provider == :together_ai + end + end + + describe "get_cost/4" do + test "calculates cost correctly" do + cost = ModelSelector.get_cost(:openai, "gpt-4", 1000, 1000) + + # GPT-4: $0.03/1k input + $0.06/1k output + # 1000 input + 1000 output = $0.03 + $0.06 = $0.09 + assert_in_delta cost, 0.09, 0.001 + end + + test "returns 0 for unknown provider" do + assert 0.0 == ModelSelector.get_cost(:unknown, "model", 1000, 1000) + end + end + + describe "build_chain/1" do + test "builds fallback chain" do + chain = ModelSelector.build_chain(%{priority: :cost}) + + assert is_list(chain) + assert length(chain) <= 5 + + # First should be cheapest + {first_provider, _} = List.first(chain) + assert first_provider == :together_ai + end + end +end \ No newline at end of file diff --git a/lux/test/llm/ollama_test.exs b/lux/test/llm/ollama_test.exs new file mode 100644 index 00000000..da072485 --- /dev/null +++ b/lux/test/llm/ollama_test.exs @@ -0,0 +1,79 @@ +defmodule Lux.LLM.OllamaTest do + use ExUnit.Case, async: true + + alias Lux.LLM.Ollama + + describe "register/1" do + test "registers ollama as a provider" do + assert :ok = Ollama.register() + + # Verify it was registered + providers = Lux.LLM.ProviderRegistry.list_providers() + assert :ollama in providers + end + end + + describe "get_host/0" do + test "returns default host when not configured" do + host = "http://localhost:11434" + assert host == "http://localhost:11434" + end + end + + describe "call/3" do + test "builds correct request body" do + # This test verifies the function exists and accepts correct args + assert function_exported?(Ollama, :call, 3) + end + end + + describe "list_models/1" do + test "function exists and is exported" do + assert function_exported?(Ollama, :list_models, 1) + end + end + + describe "pull_model/2" do + test "function exists and is exported" do + assert function_exported?(Ollama, :pull_model, 2) + end + end + + describe "get_model_info/2" do + test "function exists and is exported" do + assert function_exported?(Ollama, :get_model_info, 2) + end + end + + describe "delete_model/2" do + test "function exists and is exported" do + assert function_exported?(Ollama, :delete_model, 2) + end + end + + describe "health_check/1" do + test "function exists and is exported" do + assert function_exported?(Ollama, :health_check, 1) + end + end + + describe "integration with ProviderRegistry" do + test "ollama provider has zero cost" do + Ollama.register() + + config = Lux.LLM.ProviderRegistry.get_config(:ollama) + + assert config.cost_per_1k_tokens.default.input == 0.0 + assert config.cost_per_1k_tokens.default.output == 0.0 + end + + test "ollama has local and no_cost features" do + Ollama.register() + + config = Lux.LLM.ProviderRegistry.get_config(:ollama) + + assert :local in config.features + assert :no_cost in config.features + end + end +end \ No newline at end of file diff --git a/lux/test/llm/performance_monitor_test.exs b/lux/test/llm/performance_monitor_test.exs new file mode 100644 index 00000000..9bed3d63 --- /dev/null +++ b/lux/test/llm/performance_monitor_test.exs @@ -0,0 +1,101 @@ +defmodule Lux.LLM.PerformanceMonitorTest do + use ExUnit.Case, async: true + + alias Lux.LLM.PerformanceMonitor + + setup do + PerformanceMonitor.reset_all() + :ok + end + + describe "record_request/3" do + test "records successful request" do + :ok = PerformanceMonitor.record_request(:openai, 250, :success) + + stats = PerformanceMonitor.get_stats(:openai) + + assert stats.total_requests == 1 + assert stats.successful_requests == 1 + assert stats.failed_requests == 0 + end + + test "records failed request" do + :ok = PerformanceMonitor.record_request(:openai, 100, :failure) + + stats = PerformanceMonitor.get_stats(:openai) + + assert stats.total_requests == 1 + assert stats.successful_requests == 0 + assert stats.failed_requests == 1 + end + + test "tracks latency" do + PerformanceMonitor.record_request(:openai, 200, :success) + PerformanceMonitor.record_request(:openai, 400, :success) + + stats = PerformanceMonitor.get_stats(:openai) + + assert stats.min_latency_ms == 200 + assert stats.max_latency_ms == 400 + assert_in_delta stats.avg_latency_ms, 300.0, 0.1 + end + end + + describe "get_stats/1" do + test "returns default stats for unknown provider" do + stats = PerformanceMonitor.get_stats(:unknown) + + assert stats.total_requests == 0 + assert stats.avg_latency_ms == 0.0 + end + end + + describe "get_health/1" do + test "returns healthy for good performance" do + PerformanceMonitor.record_request(:openai, 100, :success) + PerformanceMonitor.record_request(:openai, 100, :success) + + assert :healthy == PerformanceMonitor.get_health(:openai) + end + + test "returns degraded for poor performance" do + for _ <- 1..6 do + PerformanceMonitor.record_request(:openai, 100, :failure) + end + + assert :unhealthy == PerformanceMonitor.get_health(:openai) + end + end + + describe "consecutive_failures" do + test "tracks consecutive failures" do + PerformanceMonitor.increment_consecutive_failures(:openai) + PerformanceMonitor.increment_consecutive_failures(:openai) + + stats = PerformanceMonitor.get_stats(:openai) + + assert stats.consecutive_failures == 2 + end + + test "reset_consecutive_failures resets count" do + PerformanceMonitor.increment_consecutive_failures(:openai) + PerformanceMonitor.reset_consecutive_failures(:openai) + + stats = PerformanceMonitor.get_stats(:openai) + + assert stats.consecutive_failures == 0 + end + end + + describe "get_aggregate_stats/0" do + test "returns aggregate stats" do + PerformanceMonitor.record_request(:openai, 100, :success) + PerformanceMonitor.record_request(:anthropic, 200, :success) + + stats = PerformanceMonitor.get_aggregate_stats() + + assert stats.total_requests == 2 + assert stats.providers == 2 + end + end +end \ No newline at end of file diff --git a/lux/test/llm/provider_registry_test.exs b/lux/test/llm/provider_registry_test.exs new file mode 100644 index 00000000..5095b580 --- /dev/null +++ b/lux/test/llm/provider_registry_test.exs @@ -0,0 +1,89 @@ +defmodule Lux.LLM.ProviderRegistryTest do + use ExUnit.Case, async: true + + alias Lux.LLM.ProviderRegistry + + setup do + # Registry is started by application supervisor + :ok + end + + describe "register/2" do + test "registers a new provider" do + assert :ok == ProviderRegistry.register(:test_provider, %{ + module: SomeModule, + models: ["test-model"] + }) + + assert ProviderRegistry.registered?(:test_provider) + + # Cleanup + ProviderRegistry.unregister(:test_provider) + end + + test "returns error for invalid config" do + assert {:error, {:missing_keys, [:module]}} == ProviderRegistry.register(:bad, %{}) + end + end + + describe "unregister/1" do + test "unregisters a provider" do + ProviderRegistry.register(:temp, %{module: SomeModule}) + + assert :ok == ProviderRegistry.unregister(:temp) + refute ProviderRegistry.registered?(:temp) + end + end + + describe "list_providers/0" do + test "returns list of all providers" do + providers = ProviderRegistry.list_providers() + + assert is_list(providers) + assert :openai in providers + assert :anthropic in providers + assert :together_ai in providers + end + end + + describe "get_config/1" do + test "returns provider configuration" do + config = ProviderRegistry.get_config(:openai) + + assert is_map(config) + assert config.module == Lux.LLM.OpenAI + assert "gpt-4" in config.models + end + + test "returns nil for unknown provider" do + assert nil == ProviderRegistry.get_config(:unknown) + end + end + + describe "enable/1 and disable/1" do + test "toggles provider enabled state" do + # Create a test provider + ProviderRegistry.register(:toggle_test, %{module: SomeModule, enabled: true}) + + assert :ok == ProviderRegistry.disable(:toggle_test) + config = ProviderRegistry.get_config(:toggle_test) + refute config.enabled + + assert :ok == ProviderRegistry.enable(:toggle_test) + config = ProviderRegistry.get_config(:toggle_test) + assert config.enabled + + # Cleanup + ProviderRegistry.unregister(:toggle_test) + end + end + + describe "get_models/1" do + test "returns models for provider" do + models = ProviderRegistry.get_models(:openai) + + assert is_list(models) + assert "gpt-4" in models + end + end +end \ No newline at end of file diff --git a/lux/test/llm/provider_test.exs b/lux/test/llm/provider_test.exs new file mode 100644 index 00000000..68c83537 --- /dev/null +++ b/lux/test/llm/provider_test.exs @@ -0,0 +1,45 @@ +defmodule Lux.LLM.ProviderTest do + use ExUnit.Case, async: true + + alias Lux.LLM.Provider + + describe "list_providers/0" do + test "returns list of registered providers" do + providers = Provider.list_providers() + + assert is_list(providers) + assert :openai in providers + assert :anthropic in providers + end + end + + describe "get_config/1" do + test "returns config for registered provider" do + config = Provider.get_config(:openai) + + assert is_map(config) + assert Map.has_key?(config, :module) + assert Map.has_key?(config, :models) + end + + test "returns nil for unregistered provider" do + assert Provider.get_config(:nonexistent) == nil + end + end + + describe "select_best/1" do + test "selects provider based on priority" do + {:provider, _provider, _model} = {:ok, Provider.select_best(%{priority: :cost})} + + # Cost priority should select cheaper provider + result = Provider.select_best(%{priority: :cost}) + assert match?({_, _}, result) + end + + test "returns error when no providers match requirements" do + result = Provider.select_best(%{exclude: Provider.list_providers()}) + + assert result == {:error, :no_matching_providers} + end + end +end \ No newline at end of file