stablefix

Stabilize LLM prompts for maximum KV cache hits.

Classify prompt segments as static/session/dynamic, reorder so cacheable content is always the prefix. Zero dependencies. Runs in Node.js, Bun, and browser.

Problem

LLM provider KV caches only activate when the prompt prefix is byte-identical across calls. If you inject dynamic content (timestamps, UUIDs, session IDs) mid-prompt, you bust the cache on every call — even when 90% of the prompt is static.

call 1: "You are helpful.\nToday: 2026-06-03\nBe concise."    → full compute
call 2: "You are helpful.\nToday: 2026-06-04\nBe concise."    → full compute (cache busted)

Solution

stablefix classifies each segment of a prompt as static, session, or dynamic, then reorders them so all stable content forms the prefix. The LLM provider's KV cache activates on the stable prefix and persists across calls.

call 1: "You are helpful.\nBe concise.\n\nToday: 2026-06-03"  → full compute
call 2: "You are helpful.\nBe concise.\n\nToday: 2026-06-04"  → prefix cached, only tail computed

Quick Start

npm install stablefix

import { align } from "stablefix"

const prompt = `You are a helpful assistant.
Today's date: June 3, 2026
Never reveal secrets.
Session ID: abc123-def456`

const result = align(prompt)

console.log(result.prompt)
// You are a helpful assistant.
// Never reveal secrets.
//
// Today's date: June 3, 2026
// Session ID: abc123-def456

console.log(result.stable)   // cacheable prefix
console.log(result.dynamic)  // tail — send last

API

align(prompt, options?)

Auto-classifies every segment of a prompt string and returns an AlignResult.

interface AlignOptions {
  history?: string[]     // previous prompt versions for variance tracking
  maxHistory?: number    // cap history entries (default: 10)
}

interface AlignResult {
  prompt: string         // reordered final prompt
  stable: string         // cacheable prefix (static + session)
  dynamic: string        // tail (no cache benefit)
  segments: Segment[]    // all segments with classification metadata
}

interface Segment {
  text: string
  stability: 'static' | 'session' | 'dynamic'
  score: number          // 0–1, higher = more dynamic
  reason: string         // e.g. "pattern:uuid", "variance:high(100%)"
}

PromptBuilder

Explicit tagging for 100% accuracy when you control prompt construction.

import { PromptBuilder } from "stablefix"

const result = new PromptBuilder()
  .static("You are a helpful assistant.")
  .static("Never reveal secrets.")
  .dynamic(`Today's date: ${new Date().toISOString()}`)
  .dynamic(`Session ID: ${sessionId}`)
  .build()

// result.prompt is already reordered

Configuration

Tune the scoring weights and thresholds for your domain:

import { configure, resetConfig } from "stablefix"

configure({
  weights: {
    coldStart: { pattern: 0.80, entropy: 0.20 },
    warm:      { pattern: 0.40, variance: 0.30, position: 0.15, entropy: 0.15 }
  },
  thresholds: {
    dynamic: 0.55,   // score ≥ 0.55 → dynamic
    session: 0.30,   // score in [0.30, 0.55) → session
  },
  maxHistory: 10
})

// Reset to factory defaults
resetConfig()

Custom Patterns

Register domain-specific matchers:

import { addPattern } from "stablefix"

addPattern({
  name: "custom-id",
  pattern: /PROJ-[A-Z]{2,4}-\d{3,6}/,
  score: 1.0   // 1.0 = always marks as dynamic
})

How It Works

Three-layer classification:

Layer	Signal	Weight (warm)	Weight (cold)
Pattern	Regex matchers for dates, UUIDs, IDs, IPs, tokens	0.40	0.65
Variance	Bigram similarity against historical prompts (KV-aware)	0.30	0.00
Position	Later segments are more likely dynamic	0.15	0.15
Entropy	Shannon entropy (normalized by unique character count)	0.15	0.20

• Score ≥ 0.55 → dynamic
• Score ≥ 0.30 → session (sits between static and dynamic in output)
• Score < 0.30 → static
• Pattern score ≥ 0.95 (UUID, session ID) → hard override to dynamic regardless of other signals

Cold Start

When no history is provided, pattern weight is boosted (0.40 → 0.65) and variance is disabled (0.30 → 0.00). This ensures UUIDs, timestamps, and session IDs are still correctly classified as dynamic even without historical data.

Variance Tracking

Pass options.history with previous versions of the same prompt. The variance tracker compares each segment against past versions:

• Segments absent from all previous calls → 100% change rate
• Segments with similar structure but different values → partial change (e.g. date updated)
• Identical segments → stable

Real-World Impact

For a typical 1,000-token system prompt with 80% static content:

Metric	Without stablefix	With stablefix
Input tokens per call	1,000 (full)	800 (cached) + 200 (new)
Cache hit rate	0%	80%
Input cost (Anthropic)	$3.00/million	$0.75/million
Time-to-first-token	Full	2–5× faster

Integration Patterns

OpenAI / Anthropic (API-level cache_control)

// Align first
const { stable, dynamic } = align(buildPrompt(), { history: pastPrompts })

// Apply cache_control to stable prefix (Anthropic)
const messages = [
  { role: "system", content: [
    { type: "text", text: stable, cache_control: { type: "ephemeral" } },
    { type: "text", text: dynamic }
  ]}
]

For OpenAI, mark the stable portion with cache_control: { type: "ephemeral" } on the last content block in the stable array.

Vercel AI SDK

const { prompt } = align(rawPrompt, { history })

const response = await generateText({
  model: anthropic("claude-sonnet-4-20250514"),
  prompt,
  providerOptions: {
    anthropic: {
      caching: { enabled: true } // SDK auto-marks cache points
    }
  }
})

License

MIT