feat: self-hosted deployment overhaul (docker, demo removal, namespaces, retrieval UX) - #140
Open
gdccyuen wants to merge 46 commits into
Open
feat: self-hosted deployment overhaul (docker, demo removal, namespaces, retrieval UX)#140gdccyuen wants to merge 46 commits into
gdccyuen wants to merge 46 commits into
Conversation
Add a second chat backend alongside the Vercel AI Gateway: when CHAT_BASE_URL is set, resolve a LanguageModelV3 via @ai-sdk/openai-compatible using CHAT_API_KEY, mandating CHAT_MODEL. Call sites in prompt.ts and diagram.ts use getChatModel()/isChatConfigured()/getChatModelLabel() instead of hard-requiring AI_GATEWAY_API_KEY. Pin @ai-sdk/openai-compatible@2.x (provider spec V3) to match ai@6; the 3.x line targets @ai-sdk/provider@4 and is incompatible. ADR 0007 records the decision.
Add commands, architecture, key conventions, domain language, UI/design, and testing-quirks sections so future sessions ramp up without re-deriving them. Includes the chat-provider convention and a note that the test:integration script glob is stale.
loadChunkPageForSource called readCachedChunkPage (Vercel Blob get) with no BLOB_READ_WRITE_TOKEN, throwing 'No token found' and crashing the chunks route (500, empty body). Local/self-hosted dev has no Blob store, so the inspect/chunks view 500'd on every workspace source. Gate the chunk-page cache on BLOB_READ_WRITE_TOKEN (an explicitly injected cacheStore bypasses the gate, so tests are unaffected) and treat any cache read failure as a miss. With no Blob configured the route now serves chunks straight from Knowhere. Verified: /api/sources/<id>/chunks now returns 200 and the /inspect/<docId>/chunks page renders with no client errors.
next/font/google (Geist) failed to reach fonts.googleapis.com in airgapped/local dev, repeating 'error while requesting resource' warnings and falling back to a system font. The geist npm package ships the same typeface as next/font/local with the same --font-geist-sans / --font-geist-mono variables, so globals.css is unchanged and no Google Fonts calls are made.
Multi-stage Dockerfile (node:22-alpine): deps -> builder (pnpm build) -> runner copying .next/standalone + .next/static + public, runs the traced standalone server as non-root. Enable output: 'standalone' so next build emits a self-contained server. .dockerignore excludes node_modules/.next/ .git/.env.* etc. Verified against the host's self-hosted Knowhere stack: container reaches Knowhere API + Postgres via host.docker.internal, serves sources/chunks, and uses local geist fonts (no Google Fonts calls).
README: add a Deployment (Docker) section with build/run, env-file flow, host.docker.internal note, and the Blob-optional behavior. AGENTS.md: add Docker build/run and the corrected `db:push --force` + inline DATABASE_URL commands; add conventions for Vercel Blob optionality (chunk cache degrades to direct Knowhere fetch) and the local geist font requirement.
…on click The chunk section tree was fully expanded by default. Now it shows only root + 1 level; clicking an internal (section) node toggles between expand-one-level and collapse-the-subtree. End (chunk) nodes and assistant-pane reference links keep their existing behavior. Root is always expanded and not toggleable. ChevronRight/ChevronDown indicators on toggleable sections. Tests updated to expand before asserting deep items.
The chunks panel is no longer a permanent middle panel. It renders as a fixed inset-0 z-50 overlay shown only when: - clicking the tree icon on a source row (now a button, not a Link), or - clicking a citation reference in the chat panel. The Parsed/Original toggle is replaced with a Close button. The middle desktop/mobile panel shows a placeholder when the overlay is closed. Changes: - workspace-shell.tsx: isChunksOverlayVisible state; citation click opens overlay - workspace-shell-layout.tsx: conditional placeholder vs overlay; ChunksPlaceholder - sources-panel.tsx: onOpenChunksOverlay prop; tree icon as button callback - source-row.tsx: onTreeClick replaces chunkTreeHref Link - chunks-panel.tsx: onClose prop + Close button; removed Parsed/Original toggle - 4 Original-view tests removed (feature dropped); workspace-shell tests updated to query chunks-panel testid (overlay) instead of desktop-chunks-panel
Drop the 3-panel desktop layout (sources | chunks | chat) in favor of a 2-panel layout (sources | chat). Chunks are already a full-screen overlay (triggered by source tree button or citation click); library is now also a full-screen overlay. - workspace-shell-state.ts: 2-panel width math (sources | chat, one gutter) - workspace-desktop-panels.ts: track 2 panels, one resize handle - workspace-shell-layout.tsx: chat panel is the grow panel; no middle panel - mobile-tab-bar.tsx: 2 tabs (Sources | Chat), remove Content tab - PanelId type: "sources" | "chat" (no "content") - Library renders as fixed inset-0 overlay (like chunks) - Tests: updated panel-width assertions, removed obsolete chunks-panel tests
…library AGENTS.md: desktop layout convention note; README: new Desktop Layout section.
|
@gdccyuen is attempting to deploy a commit to the Ontos AI Team on Vercel. A member of the Team first needs to authorize it. |
…ger localization - Remove demo catalog, guest mode, and Official Library panel entirely - Simplify SourceKind to "workspace" | "remote" (no "demo") - Drop demo_source_visibilities table, demo_key columns from schema - Remove all demo plumbing: demoApi deps, fetchCatalog, hideDemoSource, materializeDemoSources, demo chat seeding, demo asset hardening - Proxy no longer allows anonymous reads; redirects to login - Replace Official Library panel with namespace dropdown in sources header - Add GET /api/namespaces + POST /api/namespaces/[namespace]/localize - Add listKnowhereNamespaces calling GET /v1/documents/namespaces directly - Eagerly localize compatible-namespace docs on every source list load - Pre-filter existing DB rows to avoid redundant upsert writes - Update AGENTS.md, CONTEXT.md, add ADR 0008 84 files changed, +597/-6089 lines
…ummary Table chunks have their HTML served via chunk.assetUrl (text/html), not in chunk.content (which holds a summary string). TableChunkCard was only checking chunk.content, always falling back to the icon+summary view — duplicating the summary already shown above. Now fetches HTML from chunk.assetUrl, sanitizes via getSanitizedTableHtml, and renders via dangerouslySetInnerHTML — mirroring how ImageChunkCard uses chunk.assetUrl.
The Knowhere listChunks endpoint doesn't return assetUrl for table/image chunks (returns null even with includeAsset_urls=true). The getChunk single-chunk endpoint does return it. After listChunks, for each table/image chunk missing assetUrl, call getChunk(includeAssetUrls=true) to fetch the asset URL. The URL points to LocalStack S3 which is reachable from the browser. The browser-side useTableAssetHtml hook then fetches the HTML from the populated assetUrl, sanitizes it, and renders the actual table.
The table HTML assetUrl (LocalStack S3) is unreachable from the browser (CORS) and from inside the Docker container (localhost.localstack.cloud resolves to 127.0.0.1 inside the container). Fix: fetch table HTML server-side during chunk page loading and set it as chunk.content. The existing getSanitizedTableHtml(chunk.content) path in TableChunkCard then works as originally designed. Requires --add-host localhost.localstack.cloud:host-gateway in docker run so the container can reach LocalStack on the host.
- AGENTS.md: update docker run command with --add-host flag and port 3001, add table chunk enrichment convention - CONTEXT.md: update Parsed Chunk definition to mention server-side HTML enrichment from assetUrl
…etrieval trace Backend: - Enable LLM reranking in retrieval queries (rerank: true, internalRecallK: 30) to offset BM25's weaker ranking - Add retrieval guidance to the harness system prompt: keyword-based (BM25) query crafting, query expansion with synonyms/domain terms, and multiple focused retrieve calls for multi-part questions - Build a transient RetrievalTraceView from retrieval responses and return it with the chat answer (never persisted to the DB) UI: - New ChatRetrievalTrace component showing each issued query, its namespace, hit count, cited chunk count, and top scores - Render the trace under the sources section of assistant messages - Add the citation score to the citation chip tooltip - Show 'Searching sources…' as the live status while an answer is in flight - Remove now-unused client param from enrichChunksWithAssetUrls Tests: update query-param expectations, add trace rendering tests, a multi-query trace test, and the searching status assertion.
…etrieval tuning controls - Sources and Retrieval blocks in assistant messages are collapsed by default via a shared CollapsibleSection (Base UI Collapsible) with a chevron trigger and count badge - Composer Create button becomes an icon-only WandSparkles trigger with a Templates tooltip; canned prompts moved to public/data/chat-prompt-templates.json, fetched client-side by usePromptTemplates (cache-busted) so self-hosted deployments can override them by bind-mounting a JSON over the container path - Composer gains retrieval tuning controls: a Rerank switch, and Recall K (5-50) and Top K (1-12) sliders. Values travel as optional retrievalParams in the chat request body (validated + clamped server-side) and override the hardcoded defaults and harness-chosen topK via RetrievalOverrides - Installs @base-ui/react and the shadcn collapsible/switch/slider primitives; adds tests for parsing, overrides, controls, and folded sections; updates AGENTS.md + CONTEXT.md
…trols - Wand button tooltip/aria-label becomes 'Prompts / Chart' to cover both the canned prompts and the diagram action - Retrieval controls (Rerank switch, Recall K / Top K sliders) move from their own row into the composer bottom row, right of the Wand button
Before the expanded query list, the Retrieval block now shows a stats
row: wall-clock time to answer (seconds, 1 decimal), LLM call count, and
input/output token split.
- Harness: accumulate response.steps.length and
response.totalUsage.{input,output}Tokens across the agent loop and
revision attempts; expose as llmCallCount/inputTokens/outputTokens on
HarnessTrace (optional fields)
- answerQuestionWithRetrieval captures wall-clock time and threads the
harness usage into RetrievalTraceView (durationSeconds, llmCallCount,
inputTokens, outputTokens — all optional, transient)
- ChatRetrievalTrace renders the stats row above the query list only
when stat fields are present
…aul) Records the tagged state ae514fe (tag: checkpoint/single-user-workable-pre-overhaul) as the known-good single-user baseline before the multi-domain + Notebook-owned auth overhaul. Captures what the checkpoint guarantees, how to return to it, and the deferred alternatives it keeps open.
gdccyuen
changed the title
Phase 1 of the multi-domain overhaul:
- knowhere-keys.ts: server-side key reader for
config/knowhere-keys.json ({ label, apiKey }[]), mtime-cached so edits
take effect without a restart; falls back to KNOWHERE_API_KEY env as a
single 'default' key. Exposes masked labels for UI display.
- knowhere-api-key.ts: edge-safe dev-mode check now also honors
KNOWHERE_KEYS_FILE presence (proxy/auth short-circuits must not
redirect when file-backed keys exist).
- Schema: workspaces drops unique on userId and namespace; adds
knowhere_key_label; unique index on (userId, knowhereKeyLabel,
namespace) — one workspace per (user, domain, namespace).
- Repository: findAllByUserIdEffect, findByIdEffect,
findByIdAndUserIdEffect, findByUserIdAndLabelAndNamespaceEffect,
insertForUserLabelNamespaceEffect.
- Service: ensureWorkspace resolves the active workspace from the
notebook-ws cookie (falls back to first workspace, then creates a
legacy default); ensureWorkspaceForLabelAndNamespace creates the
workspace bound to a specific (keyLabel, namespace) pair.
- Credential resolver: ensureApiKeyForWorkspace looks up the workspace
row, resolves its knowhereKeyLabel from the key source, then falls
back to the default key, then the env override, then Dashboard JWT.
- SSR initial state: exposes all workspaces for the user + masked key
labels for the domain switcher UI.
- GET /api/knowhere-keys: masked key labels for the domain switcher - GET /api/knowhere-keys/[label]/namespaces: namespaces visible to a specific key, for the new-workspace picker - POST /api/workspaces/activate: validates ownership and sets the notebook-ws cookie - POST /api/workspaces: creates the workspace for a (keyLabel, namespace) pair and sets the notebook-ws cookie - Route tests for all four endpoints
- WorkspaceSwitcher at the top of the sources panel: lists workspaces grouped by domain (API key label), activates on click (sets notebook-ws cookie + router.refresh()), and a New workspace dialog that picks a domain key, fetches its namespaces, then creates the workspace for the chosen namespace - workspaceClient: fetchKnowhereKeys, fetchKnowhereKeyNamespaces, activateWorkspace, createWorkspace - Sources panel + shell + layout thread activeWorkspace/workspaces/ knowhereKeyLabels through from the SSR initial state - Dockerfile: create /app/config owned by nextjs for the keys-file bind mount - AGENTS.md: keys-file format + mount + multi-domain workspace convention; CONTEXT.md: Workspace + Knowhere Key Label definitions; ADR 0009
Phase 2 of the auth overhaul: - Schema: users, account_links (modular providers, passwordHash lives here), sessions (DB-backed, revocable) tables - src/lib/password.ts: @node-rs/argon2 hash/verify (Argon2id, interactive cost tuned for login) - src/infrastructure/auth/session.ts: notebook-session cookie (HttpOnly, SameSite=Lax, Secure in prod, 30-day TTL), createSession/deleteSession against the sessions table, expired-session sweep - src/infrastructure/auth/index.ts rewritten: getCurrentUser resolves the session cookie → sessions × users join; dev-mode KNOWHERE_API_KEY bootstrap short-circuit kept (P2-10); requireUser redirects to the local /login with callbackURL; extractUser accepts the new plain user shape - Proxy: cheap presence check now tests the notebook-session cookie (edge constant shared, no DB import in the edge bundle); anonymous redirect goes to local /login - Repositories: users, account-links, sessions (Drizzle + Effect) - Auth + proxy tests rewritten for the DB session flow
- src/app/login: real email+password form (Server Action loginAction verifies via account_links passwordHash with argon2, creates a DB session, redirects to /); login test updated - src/app/auth/logout: logoutAction deletes the session row + cookie - TopNav: replaces the 'Open Dashboard' link with a Sign out button (form posting the logout action); removes the dashboardUrl prop - initial-state: removes resolveDashboardUrl/dashboardUrl; shell + layout drop the prop plumbing - posthog: removes the notebook_dashboard_link_clicked tracker - scripts/create-user.ts: admin-provisioned user CLI (email, password, --name) with argon2 hashing + users + account_links(password) insert; tsconfig.scripts.json maps server-only to the test stub for tsx runs - shadcn label primitive added for the login form
…docs - New src/integrations/knowhere-credentials.ts: ensureApiKeyForWorkspace (workspace key label → keys file → env) + isAuthError; the Dashboard JWT path is gone - Delete src/integrations/dashboard/ (api-key-service, orpc-request), src/infrastructure/auth/urls.ts, session-cookie-names.ts - Rewire consumers: request-context, sources route-dependencies (signature drops cookieHeader), route-listing/retry/upload, both namespaces routes, chat route-answer + diagram route (isAuthError import) - requireUser redirects to local /login; proxy + login page already local (previous commit) - Env: .env.local.example drops DASHBOARD_ORIGIN/SESSION_COOKIE_NAMES, documents KNOWHERE_KEYS_FILE + admin CLI - ADR 0010 (Notebook-owned auth); AGENTS.md/CONTEXT.md/README updated - Tests: port ensureApiKeyForWorkspace/isAuthError tests to the new module; update route/deps mocks for the 1-arg signature
- password.ts: fix @node-rs/argon2 verify argument order (hash first, password second) — correct passwords now verify; add password tests - app/page.tsx: anonymous visitors redirect to /login (NEXT_REDIRECT via Suspense streaming); page test covers the redirect - login page: LoginForm extracted to a client component (useActionState cannot live in a Server Component — fixed the Turbopack build error) - WorkspaceSwitcher: legacy (null keyLabel) workspaces display as '<username> / default' instead of the raw notebook-<uuid> namespace; userName threaded layout → sources panel → switcher; test added
- Schema: knowhere_api_keys table (workspace FK, label, cipherBlob, cipherNonce, soft delete, unique label per workspace) + workspaces.active_knowhere_api_key_id - src/lib/secret-crypto.ts: AES-256-GCM encryption (key from KNOWHERE_KEY_ENCRYPTION_KEY env, 32-byte base64 or hashed passphrase); nonce + auth tag; unit tests incl. tamper rejection - knowhere-api-keys-repository: create/list/find/soft-delete/set-active/ get-active/decrypt (Drizzle + Effect) - ensureApiKeyForWorkspace (knowhere-credentials): resolves the active DB key first, then the workspace-label DB key, then file/env fallback — decrypts on demand, never logs the key - API routes: GET/POST /api/workspaces/[workspaceId]/api-keys, PATCH/DELETE /api/workspaces/[workspaceId]/api-keys/[apiKeyId] (ownership-checked); route tests - UI: WorkspaceApiKeysDialog (list, add with label+key, activate, delete, masked never-shown-after-save) reachable from the workspace switcher's 'API keys…' item - P3-5: removed knowhereApiKeyOverride entirely — the dev-user identity bootstrap is gone (real login always required); the env/file credential fallback stays; proxy + auth tests updated
…pty-state setup
UX overhaul of workspace switching:
- ensureWorkspace returns null when the user has no workspace (no legacy
notebook-<uuid> auto-create); all route callers return a clear
'No workspace yet' error; getOptionalAuthenticated returns null without
a workspace; SSR initial-state shows an empty setup state
- Workspaces are key-agnostic: identity is (user, namespace); the
credential is the mutable workspaces.active_knowhere_api_key_id
- knowhere_api_keys become user-scoped (user_id FK, key_mask stored at
save) — the API keys dialog is user-scoped, validates new keys against
Knowhere (422 on invalid, nothing stored), and auto-creates the
(user, 'default') home workspace with the new key active
- Combined WorkspaceSwitcher dropdown replaces WorkspaceSwitcher +
NamespaceDropdown: lists each user key's namespaces (fetched per key),
marks existing workspaces, picks a namespace to create/switch +
eager-localize (blocking spinner); 'Add API key' trigger when empty
- POST /api/workspaces {keyId, namespace} creates (user, namespace),
sets the active key, and eagerly localizes the namespace's documents
- getCompatibleNamespaces returns only [workspace.namespace]; uploads and
retries target the workspace's own namespace (no more hardcoded
'default')
- Removed: NamespaceDropdown component, localizeNamespace client method,
POST /api/namespaces/[ns]/localize + /api/knowhere-keys routes,
onSourcesLocalized prop chain, handleSourcesLocalized workflow method
- New routes: /api/api-keys (GET/POST), /api/api-keys/[keyId]
(PATCH/DELETE), /api/api-keys/[keyId]/namespaces
- ensureApiKeyForWorkspace: active key → first user key → file/env
fallback; key deletion sweeps workspace active pointers
- Legacy notebook-% workspaces deleted from the DB; tests updated
The API keys dialog refreshed only its own list — the combined dropdown's SSR-fed knowhereKeyLabels stayed stale after adding a key, so the new key's namespaces never appeared. The dialog now calls an onKeysChanged callback after a successful add/delete; the switcher wires it to router.refresh(), which re-runs SSR state so the dropdown picks up the new key and refetches its namespaces.
…ace on key add 'Chat thread not found' appeared after adding a 2nd API key because the client kept the previous workspace's threadId: the API keys dialog's router.refresh() re-rendered SSR with a new workspace, but WorkspaceShell did not remount, so chat.threadId stayed stale and the server rejected it. - WorkspaceShell now keys WorkspaceShellContent by workspace.id, so switching workspaces (namespace pick or key-add auto-create) remounts the shell and resets chat/source state with a fresh thread - POST /api/api-keys now sets the notebook-ws cookie to the home (user, default) workspace on key add, so the next SSR load lands on it with fresh state - New api-keys route tests (list/add-valid+invalid/duplicate/delete) and a shell remount test verifying stale chat messages are cleared
Phase 4, P4-1/P4-2. Env-configured provider registry (OAUTH_GOOGLE_CLIENT_ID/_SECRET, OAUTH_GITHUB_CLIENT_ID/_SECRET); a provider is only offered when its env pair is present. Authorization-code flow with PKCE + state cookie: - GET /api/auth/[provider]/start returns the provider authorize URL (JSON; client navigates) - GET /api/auth/[provider]/callback verifies state + PKCE, exchanges the code, fetches userinfo, finds-or-creates the user + account_link (provider-agnostic, no password), creates a session, redirects to / - login page renders 'Continue with <provider>' buttons when configured, plus password form below a separator Userinfo handles Google (sub/email/name) and GitHub (id; private emails fetched from /user/emails). Users created via OAuth get email or a provider-scoped fallback handle.
Phase 4, P4-3. workspace_members(userId, workspaceId) rows grant non-owner access to a workspace's sources/chats. Owner remains workspaces.user_id; members are invited by email (existing Notebook users only — users are admin-provisioned). - findAllByUserIdEffect now returns owned + member workspaces, so the switcher and SSR show shared workspaces - findByIdAndUserIdEffect allows members; all route guards therefore inherit membership access automatically - API: GET/POST /api/workspaces/:id/members (list, invite by email), DELETE /api/workspaces/:id/members/:userId (owner only; owner cannot be removed) - Workspace switcher gains a Members… item opening a dialog with invite-by-email input, member list, remove buttons - Re-inviting a removed member revives the soft-deleted row (plain unique index + onConflictDoUpdate) Verified end-to-end: owner invites ada@example.com; ada sees the shared workspace in her switcher, lists its 9 sources, and can chat in it.
Phase 4 addendum. When DASHBOARD_ORIGIN is set, the login page offers 'SSO (Dashboard)': the browser's Dashboard Better Auth session cookie is host-scoped (not port-scoped), so it reaches the notebook on another port. GET /api/auth/dashboard/start forwards the full cookie jar to the Dashboard's public users.getCurrentUser oRPC endpoint (empty JSON body, 3s timeout) and logs the user in via find-or-create: - links by (dashboard, providerUserId) first - email collision: adopts an existing user only when they have no password (pristine or OAuth-created); password-protected accounts are refused with a 409 so nobody can take over an admin account - no-dashboard-session surfaces as 401 with an inline error on /login Fixes a Next.js 16 cacheComponents pitfall: GET route handlers are prerendered at build time, and the route's early provider-404 was baked into a year-long static cache. cookies() (a dynamic API) is now read before any early return, deferring to request-time rendering. Same pattern keeps the Google/GitHub start/callback routes dynamic (they read request.url first). Proxy now whitelists /api/auth so anonymous login flows are not redirected to /login.
…tions, table enrichment, working upload Six user-facing fixes: 1. Section-tree wheel: zoom only on Ctrl/Cmd+wheel (trackpad pinch); plain wheel now scrolls the outer ScrollArea so expanding sections past the pane height scrolls instead of zooming (chunks-panel.tsx) 2. Focused chunks: getChunksWithFocusedFirst now shows ONLY the focused chunk (citation click or tree leaf click) instead of reordering the whole document around it; a 'Show all chunks' header button clears the focus (chunks-panel-state.ts, chunks-panel-workflow.ts, chunks-panel.tsx) 3. Tree button: verified the source-row ListTree button opens the overlay in tree mode; fixes 1+5 make the tree scrollable and content-complete 4. Inline citations: [Source N: ...] markers in the answer text are now converted to superscript [n] links (two-pass token replacement) that open the same chunk-pane focus flow as the citation chips; chips remain below the answer (chat-message-list.tsx) 5. Load-all enrichment: loadChunksForSource now uses visible mode (includeAssetUrls + enrichChunksWithAssetUrls), so citation-focus and tree loads render real table/image HTML instead of summaries (domains/chunks/server.ts) 6. Upload: root cause was the client always using the Vercel Blob staged path, which requires BLOB_READ_WRITE_TOKEN (unset in self-hosted). New direct multipart upload to POST /api/sources (postFormData in route-client) used when Blob is unconfigured (SSR isBlobConfigured flag). Upload dialog gains a namespace dropdown listing the same per-key namespaces as the workspace switcher (default = active workspace); the server resolves the target workspace via findWorkspaceByIdAndUserId when a workspaceId is supplied (route-upload, route-upload-request, upload-request, source-upload-dialog, initial-state) Verified end-to-end: direct multipart upload returns 201 and creates a Knowhere document without BLOB_READ_WRITE_TOKEN; 591 tests pass.
… parsing to ready Root cause: source reconciliation only ran through the Upstash workflow (QSTASH_TOKEN). Self-hosted Docker has no QStash, so startBackground- Reconciliation logged 'skipping' and every uploaded document stayed in 'parsing' forever — even after reload, since the SSR re-trigger hit the same dead end. When QSTASH_TOKEN is unset, background-reconcile now spawns an in-process poll loop: pollSourceReconciliation with 3s->30s backoff (25 attempts, mirroring the Upstash poll-and-ready workflow) and markSourceReadyAfterReconciliation once the Knowhere job is done. A per-source active-poller guard prevents duplicate loops from concurrent triggers (upload route + SSR re-trigger). Verified end-to-end: uploaded a document into the default workspace while adobe was active; within seconds the local poller marked it ready, and a previously forever-parsing source (SSA statistics PDF) also resolved to ready on the next reconciliation pass.
…2 chunkId citations, M3 dataType 7)
…null-answer guard Follow-ups from the Knowhere page-snippets fix (PR #245): M1 — never render literal null: answers that are empty or exactly 'null'/'undefined' fall back to the existing NO_RESULTS_ANSWER; a real answer containing the word is untouched. M2 — citation resolution by parser chunkId: retrieval results and referenced chunks now carry chunkId through the evidence ledger, chat citations, and persisted CitationView JSON. resolveCitationChunk tiers become chunkId (against parserChunkId) → content excerpt → unique section path, so snippet-window citations still highlight the correct page chunk (verified live: the Telephone Directory page citation resolves by node_59c79a94-...). M3 — page-targeted retrieval: 'page' joins AgenticRetrievalTargetContent → dataType 7, TargetModality/tool schema accept it, and the harness system prompt directs directory-style lookups (people, phone numbers, addresses) to use retrieve modalities ['page'] (verified live: dataType 7 reached the API and both Gordon entries came back grounded).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Self-hosted deployment overhaul for knowhere-notebook: Docker packaging, demo/guest removal, namespace-driven document localization, table-chunk rendering, BM25 retrieval improvements, and chat UX upgrades.
What's included
Self-hosted deployment
Dockerfile, non-root, port 3000), optional Vercel Blob cache, localgeistfonts (no Google Fonts dependency)CHAT_BASE_URL+CHAT_API_KEY+CHAT_MODEL) alongside the AI Gateway providerLayout
Demo/guest removal
SourceKindis now"workspace" | "remote"onlyChunks
assetUrl(avoids browser CORS with LocalStack S3 URLs), rendered as sanitized HTML tablesChat / retrieval
rerank: true,internalRecallK: 30), harness prompt guidance for keyword queries, query expansion, and multiple focusedretrievecallsDocs
Notes
KNOWHERE_API_KEY) skips Dashboard auth with a deterministic local user; production Dashboard path unchangedpnpm typecheck,pnpm lint,pnpm test(543+ passing), Docker runtime verified against a self-hosted Knowhere instance