Skip to content

kaizen: fleet refresh CLI と dogfood script を追加し、登録済み workspace の sync/setup/verify readi (#78)#82

Closed
s-hiraoku wants to merge 4 commits into
mainfrom
kaizen/issue-78-monitor-verify-fleet-refresh-before-returning-to
Closed

kaizen: fleet refresh CLI と dogfood script を追加し、登録済み workspace の sync/setup/verify readi (#78)#82
s-hiraoku wants to merge 4 commits into
mainfrom
kaizen/issue-78-monitor-verify-fleet-refresh-before-returning-to

Conversation

@s-hiraoku

@s-hiraoku s-hiraoku commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Closes #78

Summary

fleet refresh CLI と dogfood script を追加し、登録済み workspace の sync/setup/verify readiness を検証できるようにしました。
失敗時は JSON/終了コードで可視化し、回帰テストと CLI ドキュメントも追加しました。

Builder notes

検証: npm test, npm run typecheck, npm run build, shared skill file checks passed. Commit: 623bbce kaizen: add fleet refresh verification (#78)

Verification

  • npm test
  • npm run typecheck
  • npm run build
  • test -f skills/gh-link-issue-pr/SKILL.md && test -f skills/kaizen-bug-router/SKILL.md && test -f skills/pr-guardian/SKILL.md

Kaizen risk policy

Verifier cleared PR: Open PR with 0 should_fix item(s); risk is low.

Changed files: 8
Changed lines: 380

Summary by CodeRabbit

  • New Features

    • Added kaizen fleet refresh to refresh and validate all registered workspaces in one command.
    • Supports --project targeting, --sync behavior, and --json output.
  • Bug Fixes

    • Improved step/result reporting across config, sync, setup, and verify, including consistent failure handling and correct exit code.
    • Safely prevents refresh into unsafe workspace paths and skips work when a run lock is already active.
  • Documentation

    • Documented fleet refresh command behavior, verification flow, and exit conditions.
  • Tests

    • Added fleet refresh test coverage and improved CLI test isolation using per-run temporary directories.

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: kaizen-agents-org/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: bd34cce4-cb91-48d4-b7bb-2e55923916a7

📥 Commits

Reviewing files that changed from the base of the PR and between 989ae0c and 4a74814.

📒 Files selected for processing (2)
  • src/commands/fleet.ts
  • test/fleet.test.ts

📝 Walkthrough

Walkthrough

This PR adds kaizen fleet refresh, which discovers registered fleet workspaces, optionally syncs them, runs configured setup and verification commands, documents the command behavior, adds a dogfood wrapper, and updates tests and CLI temp-directory handling.

Changes

Fleet refresh command

Layer / File(s) Summary
Shared result shapes
src/workspace/manager.ts, src/commands/fleet.ts
WorkspaceCommandResult is introduced, and the fleet step/result types and workspace method signatures use the shared command result shape.
Fleet refresh orchestration
src/commands/fleet.ts
refreshFleet resolves registry targets, refreshes each project, loads workspace config, handles run locks, runs setup/verify, and normalizes non-URL remotes.
CLI command and dogfood entrypoint
src/cli.ts, docs/02-cli-spec.md, package.json, scripts/dogfood-fleet-refresh.sh
kaizen fleet refresh is wired into the CLI, documented with its options and exit code, and exposed through the dogfood build-and-run script.
Fleet refresh validation
test/fleet.test.ts, test/report.test.ts
Tests cover synced refresh, verify failure, no-sync behavior, missing workspaces, active locks, unsafe workspace paths, and the CLI test helper now uses a per-invocation temp directory.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • kaizen-agents-org/kaizen-loop#1: Direct overlap with WorkspaceManager result-shape changes that are consumed by the new fleet refresh command.
  • kaizen-agents-org/kaizen-loop#29: The shared CLI test harness temp-directory behavior in test/report.test.ts is modified in the same area as this PR’s CLI temp handling.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is specific and clearly describes the fleet refresh CLI and dogfood script addition.
Linked Issues check ✅ Passed Implements fleet discovery, optional sync, setup/verify execution, and clear failure signaling via the CLI and dogfood script.
Out of Scope Changes check ✅ Passed The changes stay focused on fleet refresh behavior, docs, scripts, and tests, with no obvious unrelated additions.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch kaizen/issue-78-monitor-verify-fleet-refresh-before-returning-to

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 623bbcee4c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/commands/fleet.ts Outdated
Comment thread src/commands/fleet.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@package.json`:
- Line 20: The npm script entry for dogfood:fleet-refresh currently invokes
scripts/dogfood-fleet-refresh.sh directly, which depends on the executable bit
being set. Update that package.json script to run the shell script through sh
instead, keeping the same script name but changing the command so it works
regardless of file mode.

In `@src/commands/fleet.ts`:
- Around line 80-83: Validate the registry workspace path before calling
workspace.ensure() in fleet refresh so a malformed registry entry cannot trigger
destructive repairs on an arbitrary local directory during --sync. Add a guard
around the workspace!.ensure() call in the sync path to confirm
project.workspacePath points to the expected dedicated Kaizen workspace before
any clone/repair logic runs, and skip or fail the step if it does not.

In `@test/report.test.ts`:
- Around line 278-288: The temporary directory setup in the test helper is
hardcoded to "/tmp", which makes the suite non-portable. Update the temp dir
creation in the helper that uses fs.mkdtemp and execFileAsync to build the
prefix from os.tmpdir() instead, and ensure node:os is imported in this test
file so the isolated temp directory works across platforms.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: kaizen-agents-org/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a019da42-bdaa-4bca-a8e3-68493ab890ae

📥 Commits

Reviewing files that changed from the base of the PR and between 53d9d5d and 623bbce.

📒 Files selected for processing (8)
  • docs/02-cli-spec.md
  • package.json
  • scripts/dogfood-fleet-refresh.sh
  • src/cli.ts
  • src/commands/fleet.ts
  • src/workspace/manager.ts
  • test/fleet.test.ts
  • test/report.test.ts

Comment thread package.json Outdated
Comment thread src/commands/fleet.ts Outdated
Comment thread test/report.test.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/commands/fleet.ts`:
- Around line 89-92: The RunLock.isActiveError branch in fleet command handling
is missing the sync result when a locked run blocks a synced refresh. Update the
error path in the fleet command flow to also append a failed sync step alongside
workspace, setup, and verify, using the same “skipped because run is already
active” message so the JSON output stays complete for --sync runs.
- Around line 82-86: The `fleet` command currently makes
`GitClient.remoteUrl('origin')` a hard failure when `--sync` is used, so a
missing or invalid local checkout stops refresh before falling back to the
registered repo. Update the `remoteUrl` selection in `src/commands/fleet.ts` to
catch failures from the `origin` lookup and use `project.repo` as the fallback,
then pass that resolved value into `WorkspaceManager` so workspace repair can
continue even when the local checkout is broken.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: kaizen-agents-org/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 975443ad-b75c-4810-b65a-cfcbdcbaaad9

📥 Commits

Reviewing files that changed from the base of the PR and between 623bbce and 03ae2a2.

📒 Files selected for processing (2)
  • src/commands/fleet.ts
  • test/fleet.test.ts

Comment thread src/commands/fleet.ts
Comment thread src/commands/fleet.ts
@s-hiraoku

Copy link
Copy Markdown
Collaborator Author

PR Guardian final status for this pass:

Mergeability: not mergeable yet. mergeStateStatus=BLOCKED, reviewDecision=CHANGES_REQUESTED. The GitHub Actions verify check is passing, but the CodeRabbit status context is still pending (Review queued / Review in progress) after the retry window.

Watched CI runs:

  • CI 28220160213: passed (initial head 623bbce)
  • CI 28220521558: passed after 03ae2a2
  • CI 28220663533: passed after 989ae0c
  • CI 28220812399: passed after 4a74814

Fixes pushed:

  • 03ae2a2 fix: respect fleet workspace origins and run locks
  • 989ae0c fix: address fleet refresh review feedback
  • 4a74814 fix: complete fleet refresh blocked states

Feedback addressed:

  • Codex: preserved registered checkout origin for --sync workspace cloning.
  • Codex: respected per-project RunLock before workspace ensure/sync/setup/verify.
  • CodeRabbit: invoked dogfood script through sh.
  • CodeRabbit: validated managed fleet workspace paths before WorkspaceManager.ensure().
  • CodeRabbit: replaced hardcoded CLI test temp prefix with os.tmpdir() while keeping tsx IPC paths short in long workspaces.
  • CodeRabbit: made checkout origin lookup best-effort with fallback to registered repo URL.
  • CodeRabbit: recorded a skipped sync step when a run lock blocks --sync.

Review feedback state: paginated GraphQL review-thread and PR-comment reads completed with hasNextPage=false; all 7 review threads are resolved. No unresolved actionable review feedback remains.

Remaining blocker: external CodeRabbit status context remains pending, so GitHub still reports the PR as blocked. No merge was attempted.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/commands/fleet.ts (1)

121-135: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Validate the workspace path before no-sync setup/verify too.

The guard currently runs only in the sync branch. With --no-sync, a malformed registry entry pointing workspacePath at any existing Git checkout passes the .git check and then setup/verify commands execute there.

🛡️ Proposed fix
   } else {
     workspaceReady = await runStep(steps, 'workspace', async () => {
+      assertSafeWorkspacePath(slug, project.workspacePath);
       await fs.access(path.join(project.workspacePath, '.git'));
     });
   }

As per coding guidelines, “Do not modify secrets, credentials, billing, destructive data changes, or production infrastructure without explicit human approval.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/commands/fleet.ts` around lines 121 - 135, The workspace path validation
is only applied in the sync branch, so no-sync flows can still operate on an
unsafe workspacePath. Update the fleet command’s workspaceReady logic in the
runStep workflow to call assertSafeWorkspacePath(slug, project.workspacePath)
before both the sync and no-sync checks, keeping the guard in the workspace
verification path so setup/verify cannot run against a malformed registry entry.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/commands/fleet.ts`:
- Around line 208-212: The workspace safety check in assertSafeWorkspacePath is
relying on workspaceDir(slug) after the slug has already been accepted, so
unsafe registry keys can still escape the intended workspaces root. Add slug
validation at the registry boundary in resolveProject and upsertProject, or
harden assertSafeWorkspacePath to reject slugs containing path separators, dot
segments, or other unsafe characters before calling workspaceDir(slug). Keep the
fix centered around slugFromRepo, resolveProject, upsertProject, and
assertSafeWorkspacePath so the workspace path can never be derived from an
unsafe slug.

---

Outside diff comments:
In `@src/commands/fleet.ts`:
- Around line 121-135: The workspace path validation is only applied in the sync
branch, so no-sync flows can still operate on an unsafe workspacePath. Update
the fleet command’s workspaceReady logic in the runStep workflow to call
assertSafeWorkspacePath(slug, project.workspacePath) before both the sync and
no-sync checks, keeping the guard in the workspace verification path so
setup/verify cannot run against a malformed registry entry.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: kaizen-agents-org/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 990454a6-30f1-4d92-8ecf-cdc0ddf5a299

📥 Commits

Reviewing files that changed from the base of the PR and between 03ae2a2 and 989ae0c.

📒 Files selected for processing (4)
  • package.json
  • src/commands/fleet.ts
  • test/fleet.test.ts
  • test/report.test.ts

Comment thread src/commands/fleet.ts
Comment on lines +208 to +212
function assertSafeWorkspacePath(slug: string, projectWorkspacePath: string): void {
if (path.resolve(projectWorkspacePath) !== path.resolve(workspaceDir(slug))) {
throw new Error(`Refusing to refresh unsafe workspace path for ${slug}: ${projectWorkspacePath}`);
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Expect to find validation that rejects '/', '\', '.', and '..' in registry slugs.
rg -n -C4 'loadRegistry|resolveProject|saveRegistry|RegistryProject|slug|workspaceDir' src/config src/utils src/commands/fleet.ts

Repository: kaizen-agents-org/kaizen-loop

Length of output: 11269


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the registry schema and any slug validation used before registry entries are created.
sed -n '1,220p' src/config/schema.ts
printf '\n---\n'
sed -n '1,220p' src/utils/slug.ts
printf '\n---\n'
rg -n -C3 'upsertProject\(|registry\.projects|projectSlug|slugify\(|slugFromRepo\(|registrySchema|registryProjectSchema' src

Repository: kaizen-agents-org/kaizen-loop

Length of output: 23341


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the registry schema and any slug validation used before registry entries are created.
sed -n '1,220p' src/config/schema.ts
printf '\n---\n'
sed -n '1,220p' src/utils/slug.ts
printf '\n---\n'
rg -n -C3 'upsertProject\(|registry\.projects|projectSlug|slugify\(|slugFromRepo\(|registrySchema|registryProjectSchema' src

Repository: kaizen-agents-org/kaizen-loop

Length of output: 23341


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Look for any explicit slug validation or normalization beyond slugFromRepo.
rg -n -C2 'slug.*regex|regex\(/.*slug|slugFromRepo\(|projectSlug|registry\.projects\[|workspaceDir\(' src/config src/init src/commands src/utils src/workspace

Repository: kaizen-agents-org/kaizen-loop

Length of output: 3103


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Look for any explicit slug validation or normalization beyond slugFromRepo.
rg -n -C2 'slug.*regex|regex\(/.*slug|slugFromRepo\(|projectSlug|registry\.projects\[|workspaceDir\(' src/config src/init src/commands src/utils src/workspace

Repository: kaizen-agents-org/kaizen-loop

Length of output: 3103


Enforce registry slug validation before deriving workspace paths.

resolveProject and upsertProject accept arbitrary string keys, and slugFromRepo() only strips /, so a registry slug containing .., \, or other separators can still push workspaceDir(slug) outside workspaces. Reject unsafe slugs at the registry boundary, or guard assertSafeWorkspacePath before it uses workspaceDir(slug).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/commands/fleet.ts` around lines 208 - 212, The workspace safety check in
assertSafeWorkspacePath is relying on workspaceDir(slug) after the slug has
already been accepted, so unsafe registry keys can still escape the intended
workspaces root. Add slug validation at the registry boundary in resolveProject
and upsertProject, or harden assertSafeWorkspacePath to reject slugs containing
path separators, dot segments, or other unsafe characters before calling
workspaceDir(slug). Keep the fix centered around slugFromRepo, resolveProject,
upsertProject, and assertSafeWorkspacePath so the workspace path can never be
derived from an unsafe slug.

@s-hiraoku

Copy link
Copy Markdown
Collaborator Author

Closing as superseded by PR #77, which already merged the fleet refresh verification work for issue #78 into main. This branch now conflicts with the merged implementation and should not be merged separately.

@s-hiraoku s-hiraoku closed this Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[monitor] Verify fleet refresh before returning to scheduled checks

1 participant