retrieval-hub

One MCP server for every retrieval corpus across an enterprise. Clinical guidelines, internal documentation, codebases, policy libraries, regulatory filings -- any corpus that exists to be searched, referenced, and cited. Agents connect to one server, call retrieve, and get governed, domain-adapted results with provenance. They never know or care whether they're querying a vector store, a SQL database, or a knowledge graph.

Data owners publish sources with a per-source semantic layer they control: entity definitions, vocabulary mappings, relationship hints, metric definitions, and refinement strategies. The platform uses this metadata to rewrite queries into domain-specific terminology, rerank results, and expand context -- all invisible to the agent. The data owner's domain expertise is encoded once and reused by every agent that consumes the source.

retrieval-hub is a retrieval platform, not an agent framework. It provides six MCP tools (list_sources, describe_source, retrieve, refine, write, request_access) that any agent architecture can compose -- including agentic RAG implementations. Query rewriting, reranking, elicitation, and semantic layer lookups are internal mechanisms that make retrieve and refine better. The agent calls one tool and gets the best result the platform can produce.

Status: early development. Design documentation and a scaffolded core library are in place. See docs/SYSTEMS.md for the build order and the status of every subsystem.

See it

The fastest way to understand what retrieval-hub does is to run the UI mockup. It uses static data (no backend required) but shows the full catalog experience: source cards, detail views with eval scores and rewriter config, a query playground, and an admin dashboard.

Prerequisites: Node.js 18+.

git clone https://github.com/rdwj/retrieval-hub.git
cd retrieval-hub/retrieval-hub-ui/frontend
npm install
npm run dev

Open http://localhost:5173. The landing page has a guided tour that walks through the key features.

Understand why

Why platform-managed retrieval makes the case for treating retrieval as a platform concern rather than a per-team bespoke effort. It covers governance, access control, compliance, observability, forensic reconstruction, quality transparency, and the "accidental platform" problem.

Provenance posture explains how retrieval-hub produces provenance-aware responses that a trust framework can verify, aligned to the Trust Bricks PTC specification. This is the security differentiator: most retrieval systems return bare chunks with no basis for trust. retrieval-hub returns chunks with content hashes, source classifications, ingestion lineage, and optional cryptographic signatures.

Understand how

MCP tool surface describes the six tools agents use: list_sources, describe_source, retrieve, refine, write, and request_access. The design principle is that agents speak in intent ("get me data relevant to this query") and the source adapter translates intent into mechanism (vector search, text-to-SQL, graph traversal) based on the source's family.

Architecture is the full system overview: components, data flows, deployment topology, integration points. Start here for how the pieces fit together.

Catalog data model specifies sources, recipes, physical indexes, rewriter metadata, eval results, and agent write policies. This is the heart of the platform.

Understand where it's going

Vision and roadmap is the full positioning document: the organizational case, provenance posture, MCP design, data residency, source onboarding, and phased build plan. Start here for the big picture.

Systems index lists every subsystem with its current status (Implemented, Skeleton, Design, or TBD) and links to the relevant design doc.

Try it with Claude Code

The deployed MCP server is live and authenticated via Google OAuth. Any Red Hat employee can connect with their Google identity.

# Create a test directory
mkdir test-retrieval-hub && cd test-retrieval-hub

# Tell Claude to use the MCP server
echo "For all questions in this project, use the retrieval-hub MCP server. Do not answer any questions from training data. If the answer cannot be found by using Retrieval Hub, say you don't know." > CLAUDE.md

# Register the MCP server
claude mcp add --transport http retrieval-hub \
  https://retrieval-hub-mcp-retrieval-hub.apps.cluster-z9hbt.z9hbt.sandbox1495.opentlc.com/mcp

# Start Claude
claude

On first use, type /mcp, select retrieval-hub, and authenticate with your Red Hat Google account. Then ask a question that exercises the clinical guidelines corpus:

Tell me what you can about how to treat hypertension for a patient who presented with 140/85 for the 2nd time in a row. Pt is male 85 yo.

Without the MCP server, Claude answers from its training data. With it, Claude calls retrieve against the VA Clinical Practice Guidelines and returns a sourced, guideline-grounded answer. Note the datasets it cites in the response.

Quick start (full demo)

End-to-end: start databases, ingest a corpus, query it, and run the MCP server. Requires Python 3.11+, Podman, and Ansible (for the local dev playbooks).

# 1. Clone and set up
git clone https://github.com/rdwj/retrieval-hub.git
cd retrieval-hub
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev,ingest]"
pip install -e retrieval-hub-mcp/

# 2. Start local Postgres (catalog on :5434, pgvector on :5433)
scripts/step4_local_up.sh

# 3. Apply catalog migrations
make migrate

# 4. Ingest the VA CPG clinical guidelines corpus
python scripts/ingest_va_cpg.py

# 5. Seed query-rewriter metadata (vocabulary mappings, sample queries)
python scripts/seed_va_cpg_rewriter_metadata.py

# 5b. Seed semantic layer (entity definitions, metrics, abbreviations)
python scripts/seed_va_cpg_semantic_context.py

# 6. Query the corpus
python scripts/query_va_cpg_demo.py "what does the VA CPG recommend for PTSD treatment"

# 7. Test the query rewriter against gpt-oss-120b
python scripts/test_rewriter.py --query "high blood sugar after a meal"

# 8. Start the MCP server (streamable-http on :8000)
python -m retrieval_hub_mcp

Step 4 requires the VA CPG corpus files in a sibling retrieval-hub-data-sources/ directory. If you don't have the corpus, skip steps 4-6 and ingest the code source instead:

python scripts/ingest_code_repo.py --repo rdwj/retrieval-hub
python scripts/query_code_demo.py "how does the retrieval adapter work"

Quick start (core library only)

python3 -m venv .venv
source .venv/bin/activate

make install      # install the core library + dev tooling
make test         # run the unit test suite
make migrate      # apply alembic migrations against $RETRIEVAL_HUB_DB_URL

The default database URL points at a local Postgres for development; override it via the RETRIEVAL_HUB_DB_URL environment variable.

Layout

retrieval-hub/
├── src/retrieval_hub/        # core library (models, schemas, adapters, ingestion, rewriter, semantic layer)
├── retrieval-hub-mcp/        # MCP server (list_sources, describe_source, retrieve)
├── retrieval-hub-ui/         # PatternFly catalog UI (React + Vite)
├── retrieval-hub-bff/        # backend-for-frontend (query playground)
├── retrieval-hub-auth/       # auth service (OAuth 2.1, JWT)
├── prompts/                  # YAML prompt templates (rewriter, etc.)
├── scripts/                  # ingestion, query demos, rewriter smoke test
├── alembic/                  # database migrations
├── tests/                    # unit tests for the core library
├── docs/                     # architecture, subsystem designs, roadmap
├── pyproject.toml
├── Makefile
└── Containerfile             # core-library image (UBI9 Python 3.11)

Contributing

See CONTRIBUTING.md for development setup, coding conventions, and the PR workflow.

License

Copyright 2026 Red Hat, Inc. Licensed under the Apache License, Version 2.0.

Security

To report a vulnerability, see SECURITY.md. Do not file public issues for security concerns.

Code of Conduct

This project follows the Contributor Covenant v2.1.