feat(ts/examples): add FAQ bot example (RAG over local corpus) - #214
Open
yasumorishima wants to merge 5 commits into
Open
feat(ts/examples): add FAQ bot example (RAG over local corpus)#214yasumorishima wants to merge 5 commits into
yasumorishima wants to merge 5 commits into
Conversation
Addresses open-chat-labs#158. Adds ts/examples/faq/, a minimal command bot that answers /ask <question> by embedding the query, retrieving the top-k chunks from a sqlite-vec index built from a local markdown FAQ, and composing a grounded answer via a configurable LLM provider. - Same Express + JWT middleware skeleton as ts/examples/openai. - RAG module (src/rag/{embed,llm,store}.ts) with env-swappable providers: embedding defaults to Hugging Face Inference API; chat supports Hugging Face and Groq. - Ingest script (scripts/ingest.ts) rebuilds the vector index from any markdown source, so the FAQ corpus is easy to replace. - README documents setup, corpus replacement, and open questions for maintainers (data source, deployment shape, preferred provider default).
- ask handler: validate question argument before sending the placeholder message so we never double-send to Express. - ask handler: guard against a missing vector index at startup with a clear "run npm run ingest first" error, instead of silently creating an empty DB and failing on every query. - rag/embed: assert the Hugging Face feature-extraction response is a sentence embedding and reject token-level outputs with a helpful error. - rag/store: use the canonical sqlite-vec `MATCH ? AND k = ?` form so the engine returns top-k instead of scanning all rows and truncating. - scripts/ingest: drop chunks that are only markdown headings so the index is not polluted with low-information vectors. - package.json: drop unused `node-fetch` dependency (Node 18+ global fetch is sufficient). - README: clarify the setup cd steps.
Actionable: - ask handler: validate TOP_K env as a positive integer, fall back to 4 for missing or malformed values. - ask handler: short-circuit with a grounded "no results" reply when the vector search returns zero chunks, avoiding an unnecessary LLM call and potential hallucination. - ask handler: cover the placeholder send with the same try/catch as the rest of the async flow, and fall back to a 500 when the response has not been sent yet. - rag/store: insert the text row and the vec row inside a single better-sqlite3 transaction so the index is never left inconsistent. - README: capitalise "Markdown" for consistency. Nitpicks: - .gitignore: also ignore SQLite WAL/SHM/journal sidecars. - package.json: point the `main` field at `dist/server.js` (the actual entry used by `start`). - rag/llm: throw when the chat response is missing `choices[0].message.content` instead of silently returning an empty string. - rag/llm + rag/embed: add an `AbortSignal.timeout` to every outbound fetch so slow or stalled providers cannot hang ingestion or requests indefinitely. - scripts/ingest: wrap the embedding loop in try/finally so the database handle is closed even when embed() or insertChunk() throws.
yasumorishima
requested review from
hpeebles,
ivan-jukic,
julianjelfs and
megrogan
as code owners
April 14, 2026 14:39
Adds a small eval set (data/eval.jsonl) of (query, expected_keyword) pairs and an scripts/eval.ts runner that reports top-1 / top-k hit rate and retrieval latency (p50 / p95 / avg) against the sqlite-vec index. Wired up as npm run eval. Hit rate is measured by case-insensitive keyword containment in the retrieved chunk text, which avoids depending on chunk-id stability across re-ingests. The script exits non-zero on any miss so it can serve as a CI smoke test.
There was a problem hiding this comment.
Pull request overview
Adds a new TypeScript example bot under ts/examples/faq that demonstrates a local-corpus RAG workflow for OpenChat command bots, addressing issue #158 by embedding FAQ content into a sqlite-vec index and answering /ask queries with retrieved context plus a configurable LLM.
Changes:
- Adds a new Express-based FAQ command bot with
/ask, JWT-based bot client setup, and retrieval/generation logic. - Adds ingest and evaluation scripts for building and measuring a local
sqlite-vecFAQ index from Markdown/JSONL inputs. - Adds example data, environment template, package configuration, and README documentation for setup and provider selection.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
ts/examples/faq/tsconfig.json |
TypeScript build config for app and scripts. |
ts/examples/faq/src/types.ts |
Shared Express request type carrying the bot client. |
ts/examples/faq/src/server.ts |
Server entrypoint that boots the Express app. |
ts/examples/faq/src/rag/store.ts |
SQLite vector-store initialization, insert, and search helpers. |
ts/examples/faq/src/rag/llm.ts |
Provider-swappable chat completion client for HF/Groq. |
ts/examples/faq/src/rag/embed.ts |
Embedding client and HF response normalization logic. |
ts/examples/faq/src/middleware/botclient.ts |
Middleware that creates a BotClient from the OpenChat JWT header. |
ts/examples/faq/src/handlers/success.ts |
Small response helper for command outputs. |
ts/examples/faq/src/handlers/schema.ts |
Bot definition exposing the /ask command. |
ts/examples/faq/src/handlers/executeCommand.ts |
Command dispatcher routing requests to the handler. |
ts/examples/faq/src/handlers/ask.ts |
Main RAG command flow: validate input, search, prompt LLM, send response. |
ts/examples/faq/src/factory.ts |
Bot client factory wiring from environment variables. |
ts/examples/faq/src/app.ts |
Express route registration for command execution and bot schema. |
ts/examples/faq/scripts/ingest.ts |
Offline index builder from Markdown FAQ content. |
ts/examples/faq/scripts/eval.ts |
Smoke evaluation script for retrieval quality and latency. |
ts/examples/faq/package.json |
Example package metadata, dependencies, and runnable scripts. |
ts/examples/faq/data/faq.md |
Seed FAQ corpus used for local indexing. |
ts/examples/faq/data/eval.jsonl |
Example retrieval eval cases for smoke testing. |
ts/examples/faq/README.md |
Setup, usage, corpus replacement, provider, and evaluation docs. |
ts/examples/faq/.gitignore |
Ignores generated SQLite database artifacts. |
ts/examples/faq/.env.example |
Environment template for bot, index, and provider configuration. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+4
to
+7
| "main": "dist/server.js", | ||
| "scripts": { | ||
| "build": "tsc", | ||
| "start": "node dist/server.js", |
Comment on lines
+30
to
+33
| const chunks = raw | ||
| .split(/\n\s*\n/) | ||
| .map((c) => c.trim()) | ||
| .filter((c) => c.length > 0 && !isHeadingOnly(c)); |
Comment on lines
+44
to
+47
| if (fs.existsSync(indexPath)) { | ||
| fs.unlinkSync(indexPath); | ||
| } | ||
| const db = initIndex(indexPath, dim); |
Comment on lines
+61
to
+64
| const start = Date.now(); | ||
| const queryVec = await embed(c.query); | ||
| const results = search(db, queryVec, k); | ||
| latenciesMs.push(Date.now() - start); |
Comment on lines
+92
to
+95
| | Env var | Values | Notes | | ||
| | --- | --- | --- | | ||
| | `EMBEDDING_PROVIDER` | `hf` | Hugging Face Inference API, default model `sentence-transformers/all-MiniLM-L6-v2` | | ||
| | `LLM_PROVIDER` | `hf`, `groq` | `hf` uses the Inference API chat endpoint; `groq` uses the Groq OpenAI-compatible API | |
…ty, heading merge, latency split) - tsconfig.json: set rootDir to src and drop scripts from include so dist/server.js lands at the path package.json main/start expect instead of dist/src/server.js. - scripts/ingest.ts: stop dropping heading-only chunks. Merge each heading with the following content chunk so question wording stays in the embedded text (significantly improves retrieval for queries that match heading phrasing). - scripts/ingest.ts: write the index to FAQ_INDEX.tmp and rename atomically on success. A mid-build embed failure no longer leaves a partial faq.db in place of the previous working index. - scripts/eval.ts: split timing into embed-call and sqlite-vec search latencies and report them separately. The previous combined number was described as retrieval latency but included the remote embedding round trip.
yasumorishima
force-pushed
the
feat/faq-bot
branch
from
May 6, 2026 23:57
82bfa9f to
9a057c4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #158.
Summary
Adds
ts/examples/faq/, a TypeScript Command-bot example implementing/ask <question>:sqlite-vecindex → compose a grounded answer via a configurable LLM provider.scripts/ingest.ts) builds the vector index from any Markdown source, so the FAQ corpus is trivial to replace.ts/examples/openai(same Express + JWT middleware pattern, sameBotDefinitionshape).Provider choices
sentence-transformers/all-MiniLM-L6-v2) — free tier.LLM_PROVIDER=groq.src/rag/{embed,llm}.ts, and new providers only require aswitchbranch.Open questions
Noted in
ts/examples/faq/README.md, repeated here for visibility:data/faq.mdis placeholder content. Happy to swap for an authoritative source (docs excerpt, curated Q&A, etc.) on request.ts/examples. An on-chain canister variant would be a separate, larger change — let me know if that's preferred instead.Test plan
npm installints/librarythents/examples/faqsucceedstsc --strictbuilds cleanly (no errors, no warnings)npm run ingestwith a realHF_TOKENproducesdata/faq.db;npm run dev+/register_boton a local OpenChat answers an/askcall — I can run this against a localopen-chatcheckout on requestNotes on the review pass
This branch already went through a self-review cycle (fork-internal CI + CodeRabbit). 12 items were raised; 10 were applied, 2 were intentionally skipped to stay consistent with
ts/examples/openai(x-oc-jwtheader normalisation and the 500-response sanitisation) — happy to align those if the project has a preference either way.