Retrieval finds candidates. ContextOS decides what deserves the context window.
ContextOS is an open-source context orchestration layer for LLM applications.
Instead of passing the top-K retrieved documents directly to a model, ContextOS treats the context window as a constrained budget. It retrieves candidate context, combines retrieval signals, reranks candidates, and selects a final context within a configurable token budget.
ContextOS is LLM-agnostic: build the context with ContextOS, then pass the result to whichever model or framework your application uses.
ContextOS ships with a rich, interactive local dashboard.
View more dashboard screenshots
Instantly compare Naive Dense Retrieval vs ContextOS Budgeting.
Watch the token planner replay how the context budget is allocated.
Every candidate memory evaluates multiple dimensions—dense rank, BM25 rank, RRF score, and reranker score.
top-K retrieval isn't context selection.
Retrieve → Fuse → Rerank → Plan
- Memory Management (
contextos.add_item): When a user speaks, or a document is indexed, ContextOS persists it and generates a vector embedding automatically. - Context Building (
contextos.build_context): When a query occurs, ContextOS retrieves candidate memories. - The Planner: Candidates are scored and dynamically packed into the remaining token budget.
- Execution: ContextOS returns a
ContextResultcontaining the budgeted prompt and a detailed trace of its decisions.
- LLM-Agnostic: ContextOS handles context selection while your application handles generation. Use the result with OpenAI, Anthropic, local models, or your own stack.
- Token-Budgeted Context Selection: Configure a strict token budget and select context items based on relevance, recency, importance, and available budget.
- Hybrid Retrieval: Combine vector similarity with lexical/BM25 retrieval before downstream ranking.
- Execution Traces: Inspect why candidates were selected or rejected and examine the retrieval, ranking, and planning stages.
- PostgreSQL + pgvector: Use standard PostgreSQL infrastructure rather than requiring a specialized vector database.
A .env.example is provided for immediate local development.
cp .env.example .envContextOS relies on PostgreSQL with the pgvector extension.
docker compose up -dalembic upgrade head# Install the library
pip install -e .Check out examples/1_minimal_quickstart.py to see ContextOS in action using a local embedding model.
export EMBEDDING_PROVIDER=local
python examples/1_minimal_quickstart.pyfrom contextos import ContextOS
# Initialize ContextOS connected to your database
context = ContextOS(
database_url="postgresql+asyncpg://postgres:postgres@localhost:5433/contextos",
token_budget=4000
)
# Add context items
# await context.add_item(...)
# Retrieve, fuse, rerank, and plan a context window
result = await context.build_context(
session_id=session_id,
query="Why is the database timing out?"
)
# Pass the budgeted context to your LLM of choice!
print(result.context)
print(f"Tokens Used: {result.tokens_used}")See examples/2_custom_llm_integration.py for a complete example of injecting the result into an OpenAI client.
ContextOS includes a reproducible evaluation harness for comparing retrieval and context-selection strategies.
The evaluation measures metrics including candidate recall, planner recall, MRR, NDCG, and latency.
See Evaluation for methodology and reproduction instructions.
Dive deeper into the architecture and concepts in the docs/ directory:
We welcome contributions! Please read our Contributing Guide to learn how to run tests, add features, and submit pull requests.
ContextOS is open-source software licensed under the MIT License.