Skip to content

chore: consolidate neotoma-inspector into neotoma (subtree import)#324

Open
castor-agent wants to merge 26 commits into
mainfrom
claude/consolidate-inspector
Open

chore: consolidate neotoma-inspector into neotoma (subtree import)#324
castor-agent wants to merge 26 commits into
mainfrom
claude/consolidate-inspector

Conversation

@castor-agent
Copy link
Copy Markdown
Collaborator

Summary

Consolidates the standalone neotoma-inspector GitHub repo into neotoma as a regular subdirectory under inspector/, preserving full commit history via git subtree. Tracked as plan entity ent_96dbd62a85216fe0bbf76276.

Why

The Inspector is no longer a third-party tool that talks to Neotoma — it's the product UI, served by the same binary, on the same URLs (after PR #323's content-negotiation unification), sharing the same canonical data. The submodule split implies more separation than actually exists. Every cross-boundary change requires two coordinated PRs (e.g. PR #323 needed inspector commit 48b9d42 + parent commit 7fb9ada4e). Consolidating removes that friction.

What's in the PR

Three commits:

  1. chore: deregister inspector submodule (prep for subtree consolidation)

    • Removes the [submodule "inspector"] entry from .gitmodules.
    • Removes the inspector submodule registration.
  2. Add 'inspector/' from commit 'a4dc87bd4ad33b535bce3d5dbdd06a9cba0f92c2' (auto-generated by git subtree add)

    • Imports the inspector's origin/main HEAD as a real subtree under inspector/.
    • Every previous inspector commit is reachable from HEAD (e.g. git log -- inspector/ shows c50ada70c, 33c0840f9, 1f3c48713, etc. as real commits in this repo's history).
    • No history loss: --squash was NOT used.
  3. chore: update docs + build script after inspector consolidation

    • scripts/build_inspector.js: error message no longer suggests git submodule update --init inspector.
    • docs/plans/neotoma-harness-plan.md, docs/plans/neotoma-harness-landing-page-net-plan.md: parenthetical refers to inspector/ in this repo with an archive note.
    • docs/testing/ROUTE_COVERAGE_MATRIX.md, docs/testing/WHY_PARTIAL_ROUTE_COVERAGE.md: 2026-04-15 notes updated.
    • frontend/src/components/illustrations/InspectorPreviewIllustration.tsx: external CTA URL points at the inspector directory in this repo.

Test plan

  • Verified inspector commit history is preserved (git log --oneline -- inspector/ | head -20 shows real inspector commits, not just the merge).
  • Verified inspector/.git does NOT exist (would indicate residual submodule registration).
  • npm install && node scripts/build_inspector.js produces dist/inspector/index.html.
  • npx tsc --noEmit clean.
  • 33/33 tests/integration/inspector_bundled_mount.test.ts pass.

What's NOT in this PR

What stays as submodules

foundation, mcp/web-scraper, writ, frontend/src/shared, docs/private. They have genuine independent existence (foundation is shared across repos; web-scraper is published as an MCP server independently; writ is a separate project).

Reviewer focus

  • Spot-check that an inspector commit (e.g. a4dc87bd4) is reachable from this branch's HEAD.
  • Spot-check that inspector/ is a normal directory (no .git, no symlink).
  • Verify the build pipeline still produces a working dist/inspector/.
  • Verify nothing important still references the standalone repo URL.

🤖 Generated with Claude Code

markmhendrickson and others added 25 commits April 7, 2026 18:15
- Vite + React + TypeScript admin for entities, sources, timeline, graph
- API client for Neotoma HTTP endpoints
- Radix UI components and Tailwind styling
- Dev scripts for local/prod API targeting

Made-with: Cursor
Expand the inspector into a fuller operator UI with agent attribution, recent activity and conversations, richer entity and source detail views, and shared components for browsing provenance-heavy records.

Also update local inspector DX by wiring the matching API environment and keeping lint focused on source files instead of generated build output.
Publish the inspector as a static GitHub Pages shell so users can point it at their own Neotoma API, while documenting the HTTPS and CORS requirements that make that setup work.

Also add the project-base routing and SPA fallback needed for deep links to work on Pages.
GitHub Pages project URLs use /<repo>/ as the asset base, but a root
custom domain (e.g. inspector.neotoma.io) serves assets at /. Read
optional Actions variable VITE_PUBLIC_BASE_PATH; when unset, keep
format('/{repo}/') for default github.io deployments.
Hosted static builds (e.g. GitHub Pages) no longer default the API base to
loopback. Production bundles without VITE_NEOTOMA_API_URL use an empty
default; isApiUrlConfigured gates TanStack queries until Settings saves a
URL. requireApiBase guards fetch helpers; dashboard shows a Settings CTA.

Add optional sandbox mode (VITE_NEOTOMA_SANDBOX_UI): banner, /sandbox page,
session + sandbox API helpers, Settings lock/hide destructive UI when enabled.

Also: header API target copy when unset; search/sidebar/source queries
respect API readiness.
Preconfigure GitHub Pages builds for inspector.neotoma.io to use
https://sandbox.neotoma.io as the production API base unless a repo
variable overrides it for another hosted deployment.
Adapt the Inspector for the bundled-at-/inspector architecture:
- Remove DEFAULT_SANDBOX_BASE_URL (same-origin, relative URLs only)
- Remove fallbackApiBase from consumeSandboxSessionHandoff
- Use relative /sandbox/session/* URLs in banner and redeem
- Delete deploy-pages.yml (no standalone GitHub Pages deployment)
- Update README for bundled architecture
Add the inspector UI, API clients, and shared components needed by the agent compliance and attribution work staged in the parent repo.
Add feedback admin unlock, sync, and local store UI needed for the v0.11.0 inspector release work.
Add inspector screens and API clients for issues, access policies, schema admission grants, and agent/entity attribution views. Replace the feedback-admin UI with the issue-oriented workflow so the bundled inspector matches the current Neotoma issue pipeline.
Use a const-typed verified-via config so badge lookups stay type-safe while preserving the existing fallback behavior for unknown verification sources.
Add peer and subscription API clients, hooks, navigation entries, and pages so the bundled Inspector can display sync peer and substrate subscription state.

Enhance dashboard, entity detail, issue detail, access policy, and timeline views to surface nervous-system entities, external actor attribution, and local development proxy behavior used by the host Neotoma branch.
Pin dev:vite, build:vite, build:watch, and preview scripts to
vite.config.ts so CI contract test passes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add home/docs/FAQ routes, bundled docs navigation, and pinned primitive
shortcuts in the sidebar. Refactor header search, entity detail segments,
source preview, and relationship panels. Collapse the sidebar More section
by default with a persisted toggle and auto-expand on active routes.
…-2026-05-003)

Wires the new GET /conversations/:id/turn-index endpoint into the conversation
detail page. Adds:

- ConversationTurnIndex, ConversationTurnIndexTurn, ConversationTurnEntityRef
  types in src/types/api.ts mirroring the backend response schema.
- getConversationTurnIndex API client in api/endpoints/recent_conversations.
- useConversationTurnIndex React Query hook.
- TurnTimelineSidebar component — scrollable list of turns with role icon,
  stored/retrieved/issue counts, and deep-link to #msg-N anchor.
- TurnAnchorSections component — one card per turn with deterministic anchor
  IDs (#msg-N, #stored-N, #retrieved-N, #issues-N). Issue consent card renders
  inline under #issues-N when the turn has pending issues.

The conversation detail page now renders a two-column layout (sidebar + anchor
sections) when the turn-index loads; falls back to the existing flat message
list when the index errors (e.g. before the backend ships).

Depends on the GET /conversations/:id/turn-index endpoint shipped in the main
repo via FU-2026-05-003 backend PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…026-05-003)

Dismissible banner shown on the conversation page that tells users about the
inline turn-summary widget available in MCP clients that load
`@neotoma/ext-apps-widget-host`. Renders above the turn anchor sections.

Dismissal persists in localStorage (`neotoma_inspector_widget_host_prompt_dismissed`)
so the banner does not nag once acknowledged.

The prompt is purely informational — the widget host is loaded by MCP clients
themselves, not from inside Inspector. The prompt surfaces that the option
exists.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a Usage page to the Inspector that surfaces aggregate statistics
about the user's own Neotoma activity, computed from local data only.

- Add UsageStats interface to src/types/api.ts
- Add GET /usage API endpoint wrapper in src/api/endpoints/usage.ts
- Add useUsage() TanStack Query hook in src/hooks/use_usage.ts
- Add UsagePage with 4 stat cards, entities-by-type bar chart, and
  observations-by-source bar chart in src/pages/usage.tsx
- Wire UsagePage route in App.tsx
- Add Usage nav item (TrendingUp icon) to sidebar analytics section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extract graph layout utilities, tighten entity ID handling in graph explorer,
and extend humanize/relationship panel grouping for relationship views.
Per-turn anchor sections, turn timeline sidebar, and ext-apps widget host
mode discovery. Resolve api.ts conflict by keeping UsageStats and turn-index types.
…wcase

Add dedicated entity history and design showcase routes, shared timeline
components (world-time events, layer headers, history sections), and refine
graph explorer, entity detail, and timeline pages for clearer navigation.
…eshIndicator

PageShell was refactored to use `meta`/`actions` instead of `headerRight`.
QueryRefreshIndicator no longer accepts a `query` prop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removes the [submodule "inspector"] entry from .gitmodules and the
inspector/ submodule registration. Next commit reimports the inspector
source as a regular subtree, preserving history. See plan
ent_96dbd62a85216fe0bbf76276.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
git-subtree-dir: inspector
git-subtree-mainline: 7b2024d
git-subtree-split: a4dc87b
Follow-up to the subtree import (previous commit). Updates references
that point at the standalone neotoma-inspector repo to instead point
at inspector/ in this repo.

- scripts/build_inspector.js: error message no longer suggests
  `git submodule update --init inspector`; inspector source is now
  part of the repo.
- docs/plans/neotoma-harness-plan.md,
  docs/plans/neotoma-harness-landing-page-net-plan.md: parenthetical
  refers to `inspector/` in this repo with archive note.
- docs/testing/ROUTE_COVERAGE_MATRIX.md,
  docs/testing/WHY_PARTIAL_ROUTE_COVERAGE.md: 2026-04-15 notes updated
  to point at the consolidated location.
- frontend/src/components/illustrations/InspectorPreviewIllustration.tsx:
  external CTA URL now points at the inspector directory in this repo.

Plan: ent_96dbd62a85216fe0bbf76276

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@castor-agent
Copy link
Copy Markdown
Collaborator Author

@claude review

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 20, 2026

Claude finished @castor-agent's task —— View job


Claude PR Review

Reviewed against docs/developer/pr_review_reading_list.md (always-read invariants + conditional docs for touched paths).

Scope sanity-check. Setting aside the ~38k-line subtree import, the substantive diff is tiny and well-contained: .gitmodules (−3), scripts/build_inspector.js (±5), frontend/.../InspectorPreviewIllustration.tsx (±8), four docs (±10). No src/, no openapi.yaml, no tests/contract/, no auth surface, no error envelope. Therefore the cross-cutting guardrails in docs/architecture/change_guardrails_rules.mdc (idempotency keys, openapi-first, proxy trust, manifest, breaking changes) are not in scope here. Subtree-imported inspector/ code was authored under the standalone repo's own controls and is being moved, not modified — this is correctly out of scope for invariant checks.

History preservation confirmed: git log -- inspector/ shows real inspector commits, no inspector/.git, no submodule pointer left over.

Blocking

  1. CI workflow still references the deleted submodule. .github/workflows/ci_test_lanes.yml:34-35 runs git submodule update --init --depth 1 inspector. After this PR, there is no inspector entry in .gitmodules, so the command will exit non-zero ("No submodule mapping found in .gitmodules for path 'inspector'") and break the ci_test_lanes job on every PR. This step needs to be removed in the same PR. Note your bot can't edit .github/workflows/, so a human committer must land that change — but the PR should not be merged before the workflow is fixed, or main CI will go red on the merge commit.

Non-blocking (should fix in this PR or a follow-up)

  1. Stale "submodule" wording in production code. src/services/inspector_mount.ts:46,53,54 — JSDoc and an internal variable name (submoduleDistDir) still talk about "the submodule". Behavior is unchanged (path is inspector/dist either way), so this is a comment/symbol drift, not a logic bug. Worth renaming for accuracy.

  2. Stale wording in scripts/build_inspector.js:4. Top-of-file docstring still says "Build the Inspector submodule…". The error message in the same file was updated (commit 5588ce6cb), but the docstring wasn't.

  3. Stale references in non-historical docs:

    • docs/coverage_manifest.md:179,181,261 — describes inspector/ as "git submodule" / "external submodule".
    • docs/subsystems/plans.md:84 — "A separate PR against the inspector submodule will…".
    • docs/developer/scripts_reference.md:75 — "packages the inspector submodule for distribution".
    • inspector/README.md:121 — "git submodule add … inspector" instructions (this is the inspector's own README; arguably out of scope of the move, but it now lives in this repo and is reachable). Consider patching to reflect the new layout.

    Historical release supplements under docs/releases/in_progress/v0.4.1, v0.6.0, v0.7.0, v0.12.0, v0.12.1 etc. are fine to leave alone — supplement immutability rule (change_guardrails_rules.mdc MUST Feature unit FU-110 testing #7).

  4. inspector/package.json still names the package neotoma-inspector. Not a defect per se — it just means the published name (if Inspector were ever published independently) is still neotoma-inspector. Likely intentional but worth a conscious choice given the consolidation narrative.

Things I verified and is fine

  • No openapi.yaml / src/shared/contract* / handler / contract-test changes → no openapi-first or idempotency_key obligations triggered (change_guardrails_rules.mdc MUST Feature unit FU-110 completion #1, Feature unit FU-112 execution #11).
  • No new Express routes / auth middleware / LOCAL_DEV_USER_ID / req.socket.remoteAddress / X-Forwarded-For touched → no protected_routes_manifest.json regeneration needed; threat-model surface unchanged.
  • No PII risk introduced (no logging/metrics/event changes).
  • Determinism: not affected (no reducer / store / event logic touched).
  • InspectorPreviewIllustration.tsx URL change is purely a marketing link redirect; analytics sendCtaClick call unchanged.
  • Scope creep: none — diff outside the subtree is exclusively rename-aware reference updates.

Release-supplement implications

A consolidation like this is normally worth a one-liner in the next supplement's notes (developer-facing: "Inspector source consolidated into the main repo; submodule no longer required"). Not a "Breaking changes" entry since no published API / CLI / MCP surface tightens — just a heads-up for anyone with stale git submodule update --init muscle memory.

Summary

The subtree mechanics look clean and history is preserved. The blocking issue is CI: removing .gitmodules's inspector entry without removing the matching ci_test_lanes.yml step will break CI on merge. Everything else is cosmetic / drift.

  • Read docs/developer/pr_review_reading_list.md
  • Read always-read invariants (change_guardrails_rules.mdc)
  • Scoped diff outside the subtree; confirmed no src/, openapi.yaml, tests/contract/, or auth-surface changes (no further conditional docs triggered)
  • Reviewed PR against constraints
  • Posted review
    · Branch: claude/consolidate-inspector

Inspector is now in-tree via subtree import. The submodule no longer
exists in .gitmodules so `git submodule update --init --depth 1 inspector`
would exit non-zero and break ci_test_lanes on every PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@castor-agent
Copy link
Copy Markdown
Collaborator Author

/create_pr skill audit

Automated audit against the /create_pr skill (Neotoma plan ent_8cde942847555a83253dfe5b), which defines the standard PR description structure (Problems / Solutions / UX improvements / Documentation / Test plan / Breaking changes), the pre-PR checklist from .claude/rules/change_guardrails_rules.md, and the functional-change documentation gate (docs entry + docs/site/site_doc_manifest.yaml + docs server surfacing).

Verdict: does not fully meet standards. Gaps below; recommended changes follow each item.

Findings

  • Missing required sections: Problems, Solutions, UX improvements, Documentation, Breaking changes.
  • Breaking changes section MUST be present, even for non-breaking PRs (write No breaking changes.).
  • No Related section and no Fixes #N / Closes #N reference — link the plan and/or issue(s) so the PR closes them on merge.

Recommended changes

Update the PR description to match the skill template exactly:

## Problems
- <Concrete pain point or gap.>

## Solutions
- <Concrete change made.>

## UX improvements
- <User-visible behavior change, or `No user-visible change.`>

## Documentation
- <docs/... path(s) added or updated; parameters/outputs/examples/error modes covered.>
- <`docs/site/site_doc_manifest.yaml` entry added/updated; docs service tests pass.>
- <Or: `No functional change; no user-facing docs required.`>

## Test plan
- [ ] `npm run type-check`
- [ ] `npm test`
- [ ] `npm test -- src/services/docs` (if docs changed)
- [ ] Manual verification: <steps>

## Breaking changes
No breaking changes.

## Related
- Plan: <Neotoma plan entity_id or docs/ path>
- Issue(s): <#N>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Posted by /create_pr skill audit run. See .claude/skills/create_pr/SKILL.md for the full template.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants