Mode taxonomy + inspector bundling + content-negotiation unification by castor-agent · Pull Request #323 · markmhendrickson/neotoma

castor-agent · 2026-05-20T11:15:49Z

Summary

Three feature units in dependency order. Inspector now serves at the server root via Accept-header dispatch; the existing REST API is unchanged for every JSON consumer. Tracked as plan entities in Neotoma: ent_6e2080f7f2a961e49876c718, ent_c1d65039242aa2a920d805c7, ent_1f176dbbe9a39e6bbad27f1f.

1. Mode taxonomy (`ent_6e2080f7f2a961e49876c718`)

Splits the legacy authenticated verdict so an installed end-user instance is distinct from a hosted multi-tenant deployment.

resolveSandboxMode() returns local | production | local_sandbox | hosted_sandbox | refuse. local is the new default for a single-user installed app; production is reserved for hosted multi-tenant.
New NEOTOMA_FORCE_MODE env var (honored only outside production; production env hard-rejects at boot with exit 1).
getResolvedServerMode() exposes the boot-time verdict. /me surfaces it unconditionally.
Inspector typing: UserInfo.sandbox_mode is now the ServerMode enum.
Resolver tests expanded from 22 → 29 cases.

2. Inspector bundling (`ent_c1d65039242aa2a920d805c7`)

npm run build now chains build:inspector, so end users get a working bundled inspector by default. Previously it only built for prepublishOnly / pack:local.

Originally planned to mount the inspector at /. During execution I discovered the API and Inspector client-route namespaces overlap by design (/entities/:id is both an API endpoint and an inspector page) — the /inspector/* prefix was load-bearing. Solved via content negotiation (next FU) rather than the heavier API-namespace migration originally drafted.

3. Content-negotiation unification (`ent_1f176dbbe9a39e6bbad27f1f`)

Inspector SPA and REST API share the same URLs, dispatched on Accept:

acceptPrefersHtml(accept) parses Accept with quality factors. Missing-header and bare */* default to JSON (preserves the agent contract: curl <origin>/entities/foo still returns JSON).
isApiOnlyPath(pathname) deny-list for /me, /server-info, /sandbox/session, /.well-known/*, /api/*, /mcp/*, /oauth/*, /sync/*, /admin/*. These never serve HTML even with Accept: text/html.
installInspectorSpaFallback(app, env, logger) registered last. GET/HEAD only; mutations never dispatched. Vary: Accept on every HTML response (cache safety).
installInspectorRootStaticAssets(app, env, logger) serves /assets/* and /favicon.svg from dist/inspector at the server root with fallthrough: true.
installInspectorLegacyRedirect(app, logger) defined but not yet invoked. Staged for future cutover once known link sources (MCP instructions, conversation summary format) migrate off the /inspector/* prefix.
build_inspector.js defaults VITE_PUBLIC_BASE_PATH=/. Asset URLs now resolve at /assets/*.
inspector/src/api/client.ts (submodule): every fetch declares Accept explicitly so a JSON call can never accidentally receive the SPA shell.
Legacy /inspector/* mount remains active for back-compat — MCP instruction links and conversation summary URLs continue to work without breakage.

Design invariant (B — chosen explicitly over a stricter alternative): the same canonical entity is the source of truth for both surfaces. Any data displayed in the Inspector at URL X is fetchable as JSON from URL X. The reverse is NOT required — JSON may expose fields the HTML does not render (debug data, internal timestamps, derived aggregates).

What changed

src/services/sandbox_mode.ts — new local / production verdicts; forceMode input; resolveForceMode() helper.
src/services/inspector_mount.ts — acceptPrefersHtml, isApiOnlyPath, installInspectorSpaFallback, installInspectorRootStaticAssets, installInspectorLegacyRedirect.
src/actions.ts — wires the new resolver inputs into the boot path; surfaces resolved mode in /me; installs the SPA fallback after all API routes; installs root static assets.
scripts/build_inspector.js — default base path now /.
package.json — build chains build:inspector; dev scripts default to base path /.
tests/security/sandbox_mode_resolver.test.ts — expanded resolver tests.
tests/integration/inspector_content_negotiation.test.ts — new file, 20 cases.
Inspector submodule (a4dc87b → 48b9d42): API client Accept invariant; mode badge for non-local modes; ServerMode enum on UserInfo.

Test plan

Not in this PR

Hosted-sandbox funnel UX (ent_adb92fee6b3c0cfd171a6e0b) — unblocked but is a 4–6 week effort; deferred to its own session.
neotoma_user entity type (ent_8d0ec258955475a701f902f7) — independent.
API namespace migration (ent_462837191284c8b21450ba70) — superseded by the content-negotiation approach.

🤖 Generated with Claude Code

…ntent (#267) Add Intent-triggered task creation rule to [TASKS & COMMITMENTS] section of MCP instructions. When user messages contain trigger phrases ("I need to", "remind me", "follow up", "I should", "don't let me forget", "make sure I", "I have to", "I want to", "I must", "don't forget", "remember to"), agents MUST create a task entity with entity_type: "task" and status: "pending" in the user-phase store (Step 2) before composing the reply. Explicit FORBIDDEN clauses prevent deferring task creation or skipping it when trigger phrases are present. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

… creation (#253) Add [RELATIONSHIP CREATION] section to the MCP fenced instruction block with five rules: - Pre-store candidate discovery: check for logically related entities before completing a store; FORBIDDEN to skip this consideration - Relationship-in-same-store: prefer the store relationships array over separate create_relationship calls; use create_relationships only as follow-up when target id was unknown at store time - Canonical relationship examples: 8 typed REFERS_TO patterns (person→org, task→conversation, activity→source conversation, issue→plan/spec, note→subject, event→place, task→person, entity→source artifact) - retrieve_related_entities for traversal guidance - Relationship direction convention for REFERS_TO and PART_OF Add pointer-only entry in cli_agent_instructions.md per canonical-first sync rules (no duplicating full instruction body). Update Design rationale section inventory to include [RELATIONSHIP CREATION]. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@claude

When the diff-size gate evaluates to substantial=false, a new step posts a PR comment stating the file/line count and the thresholds that would trigger a review, plus the @claude review escape hatch. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Update inspector to entity history, timeline layers, and design showcase. Refine lexical search so multi-word titles containing registered type names still match (e.g. plan titles ending in "Strategy"). Add shadcn audit/rules and FU-2026-05-003 plan. Docs hierarchy Playwright spec lands separately once GET /docs?format=json is wired.

Adds `src/services/skills/seed_schema.ts` — an idempotent boot-time schema seeder for the `skill` entity type, following the same pattern as plans and issues. Wires `seedSkillSchema()` into `actions.ts` so the global skill schema (fields: name, description, triggers, content, slug, user_invocable, enabled, version, supported_harnesses, harness_config, synced_at) is guaranteed to exist on every server start, surviving fresh DB installs without requiring manual MCP calls. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Adds `docs/developer/skills_guide.md` covering skill entity storage, MCP/CLI retrieval, mirror-to-disk layout, slash-command palette activation via `user_invocable`, and the full field reference. Also adds the Feature guides section to `getting_started.md` (in line with PR #314) with skills listed alongside plans, issues, mirror, and subscriptions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…versation (#258) Add scrape_chatgpt_workout skill to .claude/skills/ with explicit provenance wiring between stored workout_session entities and the source conversation entity. Key additions over the pre-fix state: - Phase 3 preamble requires conversation_entity_id to be set before the store loop; directs agent to run Phase 4 first on a fresh capture - Step 3.2 explicitly collects every entity_id from store responses into session_entity_ids for use in Step 3.3 - Step 3.3 calls create_relationships to batch-create REFERS_TO edges from each workout_session entity_id to conversation_entity_id - Step 4.2 directs agent to store the returned entity_id as conversation_entity_id for use in Step 3.3 - Constraints section adds two MUST rules: run Phase 4 before Phase 3 on fresh capture; collect all session entity_ids before create_relationships Closes #258 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…on unregistered entity_type (#269) When update_schema_incremental is called for an entity_type with no registered or code-defined schema, it previously threw an opaque McpError(-32603). Now returns a structured non-throwing response with error_code ERR_NO_SCHEMA_FOR_ENTITY_TYPE, no_schema_for_entity_type: true, and an actionable hint pointing to register_schema and analyze_schema_candidates. When the entity_type has an existing schema that lacks canonical_name_fields and identity_opt_out (R2 enforcement), the R2 error is caught and surfaced as a structured response with error_code ERR_SCHEMA_MISSING_IDENTITY_CONFIG and a hint to call register_schema with a full schema_definition including canonical_name_fields. Adds regression tests covering both cold-start cases. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…248) - Add docs/subsystems/entity_field_semantics.md: canonical definitions for source (system slug), source_url (URL), source_ref (upstream external ID), and data_source (audit string); includes canonical slug table and forbidden examples - Update docs/developer/mcp/instructions.md fenced block: add source field rule — slug only, never a URL or person name, use source_url/source_ref for those; link to entity_field_semantics.md - Update docs/subsystems/record_types.md: tighten dataset source description to reference entity_field_semantics.md - Update src/services/schema_definitions.ts: add inline comments on income, note, and agent_task source field declarations pointing to canonical doc - Update docs/doc_dependencies.yaml: register entity_field_semantics.md as upstream for schema_definitions.ts, instructions.md, and record_types.md Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

#195) On first issue-filing encounter with no reporting_mode configured, the agent now checks for a stored `preference` entity with `title: "issue_filing_consent"` before prompting. If a preference entity exists, its value (`always`/`ask`/`never`) drives the resolved mode without re-asking the user. If no entity exists, the agent asks once with clearer always/ask/never framing, then persists the choice two ways: as a `preference` entity (for cross-session memory) and via `neotoma issues config --mode` (for the runtime config flag). The QA-driven issue filing section is updated to reference the same preference-entity lookup flow rather than the old prompt-only path. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…an ent_b4958d038bd41e8694fe0aef) Implements all 6 phases of the sandbox-mode / topology-aware auth plan, closing the v0.11.1 advisory regression class for self-hosted deployments that lack explicit auth configuration. Phase 1 — Resolver + install fingerprint (sandbox_mode.ts) - `resolveSandboxMode()`: pure-function resolver classifies boot topology into authenticated | local_sandbox | hosted_sandbox | refuse. - `resolveRefusePolicy()`: reads NEOTOMA_REFUSE_MODE (default "warn"). - `getOrCreateInstallFingerprint()`: stable 16-hex ID stored at <dataDir>/.install_fingerprint; drives per-install sandbox principal. - `sandboxPrincipalIdFromFingerprint()`: deterministic UUID-shaped user ID. Phase 2 — Wire resolver into actions.ts startup - Boot-time call to `resolveSandboxMode()` + `emitSandboxBootBanner()`. - `refuse` mode: logs advisory banner with three remediation options; `NEOTOMA_REFUSE_MODE=enforce` makes it fatal (default: warn, no breakage for existing self-hosters). - Precedence: auth configured → authenticated; NEOTOMA_SANDBOX_MODE=1 → hosted_sandbox; loopback bind + non-prod → local_sandbox; else → refuse. Phase 3 — ensureLocalSandboxUser() (local_auth.ts) - Materialises a deterministic per-install `local_sandbox` user on every boot; replaces the silent all-zeros LOCAL_DEV_USER_ID fallback so two co-located installs resolve to distinct principals. Phase 4 — sandbox_allowed column in protected_routes_manifest.json - sync_protected_routes_manifest.js emits `sandbox_allowed` for every route: auth-required routes default to "none", open routes to "hosted_ok". - auth_topology_matrix.test.ts: two new sanity tests assert every route declares a valid sandbox_allowed value and that auth-required routes default to "none". - Manifest regenerated (108 routes, all with sandbox_allowed). Phase 5 — UI auth bootstrap (already shipped; plan confirmed no-op delta) Phase 6 — Advisory cross-reference - docs/security/advisories/2026-05-11-inspector-auth-bypass.md extended with remediation guidance and links to the new resolver. Also: fix typo in package.json copy:env script (colpy → copy). Also: add "root": true to .eslintrc.json to stop ESLint traversing up to the main repo's eslintrc when running in a git worktree. Tests: 38 pass, 1 skipped (HTTP probe, off by default). Type-check: 0 errors. Format: clean. Lint: 0 errors. Security manifest: in sync (108 routes). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

PageShell.headerRight → PageShell.meta; QueryRefreshIndicator.query prop removed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…rride NEOTOMA_DATA_DIR Previously hydrateDataDirFromUserEnvConfig() ran first, setting NEOTOMA_DATA_DIR from ~/.config/neotoma/.env before the project .env was loaded. dotenv override:false then had no effect. Swap order: load project .env with override:true first, then call hydrateDataDirFromUserEnvConfig() which already bails if the var is set. A worktree .env with NEOTOMA_DATA_DIR now wins over the user global config. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…incipal When resolveSandboxMode() returns local_sandbox at boot, set the module-level _localSandboxActive flag and provision the per-install fingerprinted principal via ensureLocalSandboxUser(). Both auth middleware paths (local storage fast path and encryption-off fallback) now check the flag and stamp the sandbox principal instead of the shared nil-UUID LOCAL_DEV_USER_ID. This closes the silent fallback that made every local unauthenticated request resolve to 00000000-0000-0000-0000-000000000000, causing all browsers and installs on the same machine to share a single user. Each install now gets a distinct, stable principal derived from its data dir's install fingerprint. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ation Three feature units in dependency order. All complete; stored as plan entities in Neotoma (ent_6e2080f7f2a961e49876c718, ent_c1d65039242aa2a920d805c7, ent_1f176dbbe9a39e6bbad27f1f). ## Mode taxonomy (ent_6e2080f7f2a961e49876c718) Split the legacy `authenticated` verdict so installed-end-user instances have a distinct mode from hosted multi-tenant. Adds a NEOTOMA_FORCE_MODE dev override (honored only outside production; hard-rejected at boot in production with exit 1). - resolveSandboxMode() returns local | production | local_sandbox | hosted_sandbox | refuse. `local` is the new default verdict for the installed end-user app. `production` is reserved for hosted multi-tenant. - resolveForceMode() helper reads NEOTOMA_FORCE_MODE; null on unrecognized values (typo-safe). - getResolvedServerMode() exposes the boot-time verdict. /me now surfaces the resolved mode unconditionally. - Tests: resolver expanded from 22 to 29 cases. ## Inspector bundling (ent_c1d65039242aa2a920d805c7) npm run build now chains build:inspector so end users get a working bundled inspector by default. Originally planned to mount inspector at /; discovered that API route names overlap with Inspector client routes by design. Solved via content negotiation in the next FU. ## Content-negotiation unification (ent_1f176dbbe9a39e6bbad27f1f) Inspector and REST API share the same URL namespace, dispatched on Accept. Browsers get the SPA shell; agents/curl/fetch with Accept: application/json get the existing API unchanged. - acceptPrefersHtml(accept) parses Accept with quality factors. Wildcard and missing-header default to JSON, preserving the agent contract. - isApiOnlyPath(pathname) deny-list for /me, /server-info, /sandbox/session, /.well-known/*, /api/*, /mcp/*, /oauth/*, /sync/*, /admin/*. Never serve HTML. - installInspectorSpaFallback registered last. GET/HEAD only; mutations never dispatched. Vary: Accept on every HTML response. - installInspectorRootStaticAssets serves /assets/* and /favicon.svg at server root with fallthrough. - installInspectorLegacyRedirect defined, not yet invoked. Staged for future cutover. - build_inspector.js default VITE_PUBLIC_BASE_PATH=/. Asset URLs now at /assets/* (was /inspector/assets/*). - Inspector api/client.ts (submodule): every fetch declares Accept. - Legacy /inspector/* mount stays active for back-compat. ## Tests - Resolver: 29 cases pass. - Inspector bundled mount: 33 cases pass. - New inspector_content_negotiation.test.ts: 20 cases. - Security topology matrix: 18 pass, 1 pre-existing skip. - Total in touched suites: 100/101 passing. - Type-check clean. Lint: 0 errors in touched files. ## Not in this PR - Hosted-sandbox funnel UX (ent_adb92fee6b3c0cfd171a6e0b) — unblocked, 4-6 week effort, deferred. - neotoma_user entity type (ent_8d0ec258955475a701f902f7) — independent. - API namespace migration (ent_462837191284c8b21450ba70) — superseded. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

castor-agent · 2026-05-20T14:18:04Z

@claude review

github-actions · 2026-05-20T14:18:36Z

Claude finished @castor-agent's task —— View job

PR Review In Progress

Read docs/developer/pr_review_reading_list.md
Read always-read docs
Read conditional docs based on changed paths
Review PR diff against documented invariants
Post review feedback
· Branch: claude/flamboyant-shannon-845053

castor-agent · 2026-05-21T08:35:44Z

/create_pr skill audit

Automated audit against the /create_pr skill (Neotoma plan ent_8cde942847555a83253dfe5b), which defines the standard PR description structure (Problems / Solutions / UX improvements / Documentation / Test plan / Breaking changes), the pre-PR checklist from .claude/rules/change_guardrails_rules.md, and the functional-change documentation gate (docs entry + docs/site/site_doc_manifest.yaml + docs server surfacing).

Verdict: does not fully meet standards. Gaps below; recommended changes follow each item.

Findings

Title is missing a conventional-commit prefix (feat(scope):, fix(scope):, etc.).
Missing required sections: Problems, Solutions, UX improvements, Documentation, Breaking changes.
Breaking changes section MUST be present, even for non-breaking PRs (write No breaking changes.).
PR body does not confirm the doc is reachable via the docs server (src/services/docs/) or list the specific docs/...md path under the Documentation section.
No Related section and no Fixes #N / Closes #N reference — link the plan and/or issue(s) so the PR closes them on merge.

Recommended changes

Update the PR description to match the skill template exactly:

## Problems
- <Concrete pain point or gap.>

## Solutions
- <Concrete change made.>

## UX improvements
- <User-visible behavior change, or `No user-visible change.`>

## Documentation
- <docs/... path(s) added or updated; parameters/outputs/examples/error modes covered.>
- <`docs/site/site_doc_manifest.yaml` entry added/updated; docs service tests pass.>
- <Or: `No functional change; no user-facing docs required.`>

## Test plan
- [ ] `npm run type-check`
- [ ] `npm test`
- [ ] `npm test -- src/services/docs` (if docs changed)
- [ ] Manual verification: <steps>

## Breaking changes
No breaking changes.

## Related
- Plan: <Neotoma plan entity_id or docs/ path>
- Issue(s): <#N>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Functional surfaces detected from the diff: MCP, Schema. Per the skill, the Documentation section must point at a real docs/...md path covering parameters, outputs, at least one example, and error modes, and the doc must be listed in docs/site/site_doc_manifest.yaml so it is surfaced by the docs server and docs site.

Posted by /create_pr skill audit run. See .claude/skills/create_pr/SKILL.md for the full template.

Catalog was stale after new test files were added by this branch. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

castor-agent and others added 15 commits May 19, 2026 16:14

fix(inspector): bump submodule to fix usage page TS errors

5689503

PageShell.headerRight → PageShell.meta; QueryRefreshIndicator.query prop removed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

castor-agent requested a review from markmhendrickson as a code owner May 20, 2026 11:15

This was referenced May 20, 2026

chore: consolidate neotoma-inspector into neotoma (subtree import) #324

Open

chore: combined working stack — inspector consolidation + mode taxonomy/bundling/content-negotiation [DO NOT REVIEW] #327

Draft

chore(test-catalog): regenerate automated test catalog

fce8e97

Catalog was stale after new test files were added by this branch. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mode taxonomy + inspector bundling + content-negotiation unification#323

Mode taxonomy + inspector bundling + content-negotiation unification#323
castor-agent wants to merge 16 commits into
mainfrom
claude/flamboyant-shannon-845053

castor-agent commented May 20, 2026

Uh oh!

castor-agent commented May 20, 2026

Uh oh!

github-actions Bot commented May 20, 2026 •

edited

Loading

Uh oh!

castor-agent commented May 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

castor-agent commented May 20, 2026

Summary

1. Mode taxonomy (ent_6e2080f7f2a961e49876c718)

2. Inspector bundling (ent_c1d65039242aa2a920d805c7)

3. Content-negotiation unification (ent_1f176dbbe9a39e6bbad27f1f)

What changed

Test plan

Not in this PR

Uh oh!

castor-agent commented May 20, 2026

Uh oh!

github-actions Bot commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review In Progress

Uh oh!

castor-agent commented May 21, 2026

/create_pr skill audit

Findings

Recommended changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

1. Mode taxonomy (`ent_6e2080f7f2a961e49876c718`)

2. Inspector bundling (`ent_c1d65039242aa2a920d805c7`)

3. Content-negotiation unification (`ent_1f176dbbe9a39e6bbad27f1f`)

github-actions Bot commented May 20, 2026 •

edited

Loading