Skip to content

feat(test): add --watch mode, re-running whenever the target config changes#284

Closed
albatrossflyon-coder wants to merge 1 commit into
KryptosAI:mainfrom
albatrossflyon-coder:feat/test-watch-mode
Closed

feat(test): add --watch mode, re-running whenever the target config changes#284
albatrossflyon-coder wants to merge 1 commit into
KryptosAI:mainfrom
albatrossflyon-coder:feat/test-watch-mode

Conversation

@albatrossflyon-coder

Copy link
Copy Markdown
Contributor

Summary

Fixes #254.

Adds --watch to test: after the initial check, watches the target config file and re-runs a check+diff whenever it changes, until Ctrl+C.

Design choices

  • fs.watch over chokidar — the issue offered either; chokidar isn't a dependency here and watching a single file doesn't need glob support, so no new dependency.
  • Reused watch.ts's runWatchOneShot for the re-run/render/diff logic on each change, rather than re-running test's own much richer flow (interactive enforce prompts, CI-conversion offers, SARIF writing) — that flow makes sense once, at the start, not on every file save. The initial check still gets test's full behavior unchanged; only the watch-triggered re-runs use the lighter watch-style rendering.
  • 300ms debouncefs.watch commonly fires multiple events for a single file write; without debouncing this would re-run 2-3x per save.

A real ambiguity in the issue, and how I resolved it

The issue's own expected-usage example is:

mcp-observatory test --watch npx -y @modelcontextprotocol/server-memory

That's a raw command, not --target <file> — there's no config file to actually watch in that invocation. I made --watch require --target and error clearly when it's missing, rather than silently doing nothing:

✗ --watch requires --target <config.json> — there's no file to watch for a raw command target.

A real bug this surfaced (fixed as part of this PR)

Testing the issue's own example command (test <raw-command> --watch) initially did nothing — no error, --watch just silently had no effect. Root cause: test uses passThroughOptions() with a variadic [command...] argument, so flags placed after a raw command get swallowed into the command-args array meant for the server invocation, instead of being parsed as test's own options. This repo already has a known, existing workaround for exactly this (extractTrailingConversionFlags, which manually re-extracts --setup-ci/--enforce/--auto-enforce/etc. out of the trailing args) — --watch just wasn't added to that list. Fixed, with a regression test covering the exact scenario.

Verified live

Initial run + watch message:

$ mcp-observatory test --target config.json --watch
  ✓ filesystem   14 tools, 0 prompts, 0 resources
  ...
  Watching for changes... (config.json, Ctrl+C to stop)

Edited the config file → detected, debounced, re-ran, rendered via watch.ts's diff renderer:

--- 6:52:46 PM ---
secure-filesystem-server 0.2.0 — 77/100 (C) — pass
✓ No changes
Artifact: .../filesystem.json

No---target error path also confirmed (exit code 1, clear message) — including with the issue's exact raw-command example, after the extractTrailingConversionFlags fix above.

Testing

…hanges

Add --watch to the test command: after the initial check, watches the
target config file (fs.watch, no new dependency needed for a single
file) and re-runs a one-shot check+diff on each change, reusing
watch.ts's existing runWatchOneShot for the re-run/render/diff logic
rather than test.ts's own richer interactive flow (enforce prompts,
CI conversion) -- that flow only makes sense once, not in a loop.
300ms debounce since a single file write commonly fires multiple
fs.watch events. Ctrl+C closes the watcher and exits cleanly.

The issue's own example usage (`test --watch npx -y <server>`, a raw
command with no --target) has no config file to actually watch --
--watch now errors clearly in that case rather than doing nothing,
telling the user --target is required.

Also fixes a real bug this surfaced: --watch (and --enforce,
--auto-enforce, etc.) placed after a raw server command
(`test npx -y <server> --watch`) was being silently swallowed into
the variadic command-args array instead of being parsed as a test
option, because of how passThroughOptions() interacts with a
variadic positional argument. This repo already has a known
workaround for exactly this (extractTrailingConversionFlags,
handling --setup-ci/--enforce/etc. the same way) -- --watch just
wasn't added to it. Fixed and covered with a regression test; this
exact scenario (the issue's own example command) now works.

Fixes KryptosAI#254
KryptosAI added a commit that referenced this pull request Jul 15, 2026
Co-authored-by: albatrossflyon-coder <albatrossflyon-coder@users.noreply.github.com>
@KryptosAI

Copy link
Copy Markdown
Owner

Merged! #281 and #283 have file conflicts — please rebase #283 on top of #281. Great work on the watch mode!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add --watch flag to test command for continuous monitoring

2 participants