A FastMCP server that turns structured lore documents (wiki pages, chapter summaries, character essays) into a taxonomy-aware RAG corpus.
Built for LLMs to use directly — Claude can ingest a new fandom, knowledge base, or domain-specific text and query it with primary-key filtering, without the developer writing RAG pipeline code.
Generic chunkers (LangChain's SentenceSplitter, LlamaIndex's
SemanticSplitter) produce flat chunks with minimal metadata. lore-builder
is opinionated about three things:
- Size-band + section-aware chunking — 200–400 word passages that never straddle section boundaries or split mid-sentence.
- Taxonomy-aware tagging — every chunk carries a primary value (character, author, era, …) for exact-match filtered retrieval, plus secondary tags (life-themes, topics) for post-retrieval ranking. An LLM assigns the secondary tags from a controlled vocabulary you define.
- Human-in-the-loop review tools — a
sample_for_reviewtool that surfaces N stratified chunks per primary-value for spot-checking before you commit to a tagging run. Most tools skip this step.
pip install lore-builder
# Or, with ChromaDB for local vector storage:
pip install 'lore-builder[chroma]'Connect to Claude Desktop:
claude mcp add lore-builder -- python -m lore_builder.server| Tool | Purpose |
|---|---|
chunk_document |
Split a text document into size-banded, section-aware chunks |
tag_chunks |
Assign controlled-vocabulary tags to chunks via an LLM |
build_taxonomic_corpus |
End-to-end: chunk + tag + ingest into a queryable collection |
query_taxonomic_corpus |
Retrieve top-k chunks with optional primary-value filtering |
sample_for_review |
Stratified random sample as a Markdown review doc |
from lore_builder.client import LoreBuilderClient
client = LoreBuilderClient()
handle = await client.build_taxonomic_corpus(
sources=[
{"kind": "wikitext", "content": luna_wiki, "primary_value": "luna-lovegood",
"source_url": "https://harrypotter.fandom.com/wiki/Luna_Lovegood",
"license": "CC-BY-SA-3.0"},
# ... more characters
],
taxonomy={
"primary_key": "character",
"primary_values": ["luna-lovegood", "severus-snape", ...],
"secondary_tags": ["grief", "identity", "loyalty", "sacrifice", ...],
},
)
# Later...
chunks = await client.query_taxonomic_corpus(
handle=handle,
query="How did Luna deal with being different?",
primary_filter="luna-lovegood",
top_k=5,
)v0.1 — alpha. Core chunker + tagger + MCP server wired. Not yet on PyPI. API may change.
See examples/ for worked examples (Harry Potter characters, Lord of the Rings fellowship, Memory Alpha Star Trek).
MIT