Skip to content

feat: add fleet refresh verification#77

Merged
s-hiraoku merged 7 commits into
mainfrom
codex/scheduler-provider-docs
Jun 26, 2026
Merged

feat: add fleet refresh verification#77
s-hiraoku merged 7 commits into
mainfrom
codex/scheduler-provider-docs

Conversation

@s-hiraoku

@s-hiraoku s-hiraoku commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add kaizen fleet --verify so local runner refresh can sync each fleet workspace to the default branch and run configured setup/verify commands.
  • Add dogfood:verify for the standard post-merge dogfood refresh path.
  • Keep the existing fleet sync command, scheduler migration, and sync-PR backpressure handling on a ready-for-review branch.

Verification

  • npm test -- test/fleet.test.ts
  • npm run typecheck
  • npm test
  • npm run build
  • node dist/cli.js fleet --root .. --owner kaizen-agents-org --prune --verify --dry-run --json

Closes #78

Refs #76

Summary by CodeRabbit

  • New Features

    • Added kaizen fleet to rebuild/synchronize local registry, workspaces, labels, and scheduler jobs across multiple repos, with --dry-run, --verify, --prune, and component-level disable options.
    • Added dogfood npm scripts for fleet planning, syncing, and verification.
    • Added run.maxOpenPullRequests to cap automated PR selection (sync PRs are excluded from the limit).
  • Bug Fixes

    • Improved remote URL parsing for dotted repo names.
    • Refined PR-limit counting to report the counted non-sync open PR total.
  • Documentation

    • Updated README and CLI/config/pipeline/scheduler docs for kaizen fleet usage and run.maxOpenPullRequests behavior.
  • Tests

    • Added/expanded unit and integration coverage for fleet, PR-limit behavior, and text helpers.

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: kaizen-agents-org/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ae321c92-2d87-4da0-a43c-a8cc55f0bfdb

📥 Commits

Reviewing files that changed from the base of the PR and between 9288c42 and a7e16c8.

📒 Files selected for processing (2)
  • src/commands/fleet.ts
  • test/fleet.test.ts

📝 Walkthrough

Walkthrough

Adds kaizen fleet, updates scheduled open-PR counting to ignore sync branches, expands doctor builder checks, removes a Codex approval flag, broadens GitHub remote parsing, and adds shared text tail helpers.

Changes

Fleet command and sync workflow

Layer / File(s) Summary
CLI wiring and usage docs
src/cli.ts, package.json, README.md, docs/02-cli-spec.md, docs/12-scheduler-providers.md
Registers kaizen fleet, adds dogfood scripts, and updates the command reference, quickstart, and scheduler-provider guidance.
Discovery and scheduler migration
src/commands/fleet.ts, test/fleet.test.ts
Defines fleet project discovery, config loading, legacy scheduler conversion, registry shaping, and the matching helper and migration coverage.
Project sync and registry updates
src/commands/fleet.ts, test/fleet.test.ts
Runs per-project sync work, repairs stale locks, updates registry state, prunes missing projects, and covers the end-to-end fleet flow.

Open PR backpressure

Layer / File(s) Summary
Limit config and docs
.kaizen/config.yml, docs/03-config-spec.md, docs/04-nightly-pipeline.md
Sets run.maxOpenPullRequests to 3 and documents the automated-run counting rule.
Scheduled limit filtering
src/orchestrator/run.ts, test/integration/dry-run.test.ts
Filters sync PRs out of the open-PR count before scheduled issue selection applies the limit, with dry-run coverage.

Shared text truncation helpers

Layer / File(s) Summary
Tail text utilities
src/utils/text.ts, src/orchestrator/run.ts, test/text.test.ts
Adds shared tail helpers, switches verification failure truncation to line-based output, and covers both helpers.

Doctor runtime checks

Layer / File(s) Summary
Builder smoke test and helper export
src/commands/doctor.ts, test/doctor.test.ts
Splits builder-agent validation into availability and runtime checks, exports requiredLabels, and updates the smoke-test coverage.

Codex exec arguments

Layer / File(s) Summary
Command arguments
src/agents/codex.ts, test/codex.test.ts
Removes --ask-for-approval never from the Codex exec arguments and updates the argument assertion.

GitHub remote slug parsing

Layer / File(s) Summary
Remote parsing
src/utils/slug.ts, test/slug.test.ts
Updates GitHub remote parsing to accept repository names containing dots and covers dot-prefixed repo slugs.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related issues

Possibly related PRs

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The PR also bundles ancillary doctor, codex, slug, text, and scheduler/backpressure changes that go beyond fleet verification. Split those ancillary changes into separate PRs, or document why they are required for fleet verification.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately reflects the main change: adding fleet refresh verification.
Linked Issues check ✅ Passed The PR adds kaizen fleet --verify and dogfood:verify, syncs workspaces, runs setup/verify, and fails on unhealthy fleets.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/scheduler-provider-docs

Comment @coderabbitai help to get the list of available commands.

@s-hiraoku s-hiraoku force-pushed the codex/scheduler-provider-docs branch from 6af9899 to 1537d94 Compare June 26, 2026 00:25

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6af9899d4c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/commands/fleet.ts
Comment thread src/commands/fleet.ts
@s-hiraoku s-hiraoku force-pushed the codex/scheduler-provider-docs branch from 1537d94 to 41106df Compare June 26, 2026 00:29
coderabbitai[bot]
coderabbitai Bot previously requested changes Jun 26, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
README.md (1)

93-104: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

npm run dogfood:* don't belong in the generic target-repository quickstart.

This block is headed "For a target repository", but dogfood:sync / dogfood:verify are package scripts defined only in this repo's package.json and hardcode --owner kaizen-agents-org / --root ... A user following this in an arbitrary target repo will hit "missing script" or sync the wrong owner. These steps belong in a kaizen-loop self-dogfooding section, not the generic target-repo flow.

As per path instructions: "docs that disagree with the implemented CLI behavior [are] actionable review findings."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@README.md` around lines 93 - 104, The “For a target repository” quickstart
includes repo-specific dogfooding commands that only exist in this project and
hardcode this repo’s owner/root, so they should be removed from the generic
flow. Update the README section around the target-repository examples to keep
only commands that work in an arbitrary repo, and move the `npm run
dogfood:sync` / `npm run dogfood:verify` steps into a separate kaizen-loop
self-dogfooding section. Use the surrounding quickstart text and the `dogfood:*`
script references to locate the affected block.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/cli.ts`:
- Around line 111-129: The CLI action for `fleet` currently always resolves
after `syncFleet`, so failures in `syncFleetProject` and `verify` are never
reflected in the process exit code. Update the `.action` handler in `src/cli.ts`
to inspect the returned `result.projects` for any `error` entries or
`verifyPassed === false`, and set `process.exitCode = 1` before printing when
any project failed; keep the existing `print` and successful `0` behavior
unchanged for all-pass runs.

In `@src/commands/doctor.ts`:
- Around line 116-119: The tail(text, maxLength) helper in doctor.ts duplicates
the same logic used in run.ts, so extract it into a shared utility and update
both call sites to use that common helper. Locate the existing tail function in
doctor.ts and the matching helper in orchestrator/run.ts, then move the
implementation to a reusable shared module and import it from both places to
keep behavior consistent and avoid divergence.
- Around line 41-62: The builder agent smoke test is running against the
developer checkout instead of the isolated workspace. In doctor.ts, update the
`BuilderAgentAdapter.run()` call inside the `check('builder agent runtime',
...)` block to use `resolved.project.workspacePath` for `workspaceDir` so `cwd`
and `KAIZEN_BUILD_RESULT_PATH` stay confined to the sandbox. Keep the rest of
the smoke test flow unchanged, including the config loading, preferred backend
selection, and result handling.

In `@src/orchestrator/run.ts`:
- Around line 267-271: The open PR limit check in run.ts is using a capped
sample of openPullRequests, so filtering sync PRs afterward can undercount
non-sync PRs and bypass run.maxOpenPullRequests. Update runKaizen() and the
helper logic around openCount/remaining to fetch enough PRs before applying
isSyncPullRequest filtering, either by paging until all open PRs are seen or by
continuing until at least limit + 1 non-sync PRs have been collected. Keep the
skipSelectedForOpenPrLimit flow and the open pull request limit reached message,
but ensure the count is based on a complete enough set from
options.openPullRequests.

---

Outside diff comments:
In `@README.md`:
- Around line 93-104: The “For a target repository” quickstart includes
repo-specific dogfooding commands that only exist in this project and hardcode
this repo’s owner/root, so they should be removed from the generic flow. Update
the README section around the target-repository examples to keep only commands
that work in an arbitrary repo, and move the `npm run dogfood:sync` / `npm run
dogfood:verify` steps into a separate kaizen-loop self-dogfooding section. Use
the surrounding quickstart text and the `dogfood:*` script references to locate
the affected block.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: kaizen-agents-org/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0524f83d-9413-406c-913c-1cca8c24752c

📥 Commits

Reviewing files that changed from the base of the PR and between 53d9d5d and 41106df.

📒 Files selected for processing (18)
  • .kaizen/config.yml
  • README.md
  • docs/02-cli-spec.md
  • docs/03-config-spec.md
  • docs/04-nightly-pipeline.md
  • docs/12-scheduler-providers.md
  • package.json
  • src/agents/codex.ts
  • src/cli.ts
  • src/commands/doctor.ts
  • src/commands/fleet.ts
  • src/orchestrator/run.ts
  • src/utils/slug.ts
  • test/codex.test.ts
  • test/doctor.test.ts
  • test/fleet.test.ts
  • test/integration/dry-run.test.ts
  • test/slug.test.ts
💤 Files with no reviewable changes (1)
  • src/agents/codex.ts

Comment thread src/cli.ts
Comment thread src/commands/doctor.ts
Comment thread src/commands/doctor.ts Outdated
Comment thread src/orchestrator/run.ts
@s-hiraoku

Copy link
Copy Markdown
Collaborator Author

PR guardian update

Addressed the latest CodeRabbit findings on the current head:

  • kaizen fleet now sets a non-zero process exit code when any project reports an error or verifyPassed === false.
  • doctor now runs the builder smoke test inside the configured workspace path and does not create a missing workspace while checking it.
  • Shared text tail helpers now live in src/utils/text.ts and are used by doctor/run code.
  • Automatic open PR limit checks now request a wider PR list before filtering fixed-branch sync PRs, avoiding undercounts when sync PRs fill the initial sample.
  • README now separates generic target-repository commands from this repository's own npm run dogfood:* commands.

Verification after the changes:

  • npm test -- test/fleet.test.ts test/doctor.test.ts test/integration/dry-run.test.ts
  • npm run typecheck
  • npm test
  • npm run build
  • git diff --check

Current state: fixes are pushed; waiting for the new CI and CodeRabbit pass on the updated head.

coderabbitai[bot]
coderabbitai Bot previously requested changes Jun 26, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/commands/fleet.ts`:
- Around line 81-82: The fleet failure check in fleetHasFailures currently
treats project.error as a truthy value, so an empty-string error is missed and
can let kaizen fleet succeed incorrectly. Update the predicate to check for the
presence of the error field rather than its truthiness, while keeping the
verifyPassed === false check intact, so FleetSyncResult projects with error: ''
are counted as failures.

In `@src/utils/text.ts`:
- Around line 6-7: The tailLines helper currently counts the empty segment
created by a trailing newline as a real line, so the retry context can lose one
actual output line. Update tailLines in text.ts to ignore terminal newline-only
segments before slicing, so it returns the last meaningful lines from command
output. Use the existing tailLines symbol as the fix point and keep the behavior
correct for both trailing-newline and non-trailing-newline input.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: kaizen-agents-org/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 7ed70310-0ef7-48a8-9aba-86ced0b1507b

📥 Commits

Reviewing files that changed from the base of the PR and between 41106df and 2095811.

📒 Files selected for processing (9)
  • README.md
  • src/cli.ts
  • src/commands/doctor.ts
  • src/commands/fleet.ts
  • src/orchestrator/run.ts
  • src/utils/text.ts
  • test/doctor.test.ts
  • test/fleet.test.ts
  • test/integration/dry-run.test.ts

Comment thread src/commands/fleet.ts Outdated
Comment thread src/utils/text.ts Outdated
@s-hiraoku

Copy link
Copy Markdown
Collaborator Author

PR guardian update

Addressed the latest CodeRabbit edge-case findings:

  • fleetHasFailures now treats the presence of the error field as a failure, including error: ''.
  • tailLines now ignores terminal newline-only segments before slicing, preserving the last meaningful output lines.
  • Added focused coverage in test/fleet.test.ts and test/text.test.ts.

Verification:

  • npm test -- test/fleet.test.ts test/text.test.ts
  • npm run typecheck
  • git diff --check
  • npm test
  • npm run build

Current state: changes are pushed; waiting for CI and CodeRabbit on the updated head.

@s-hiraoku

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai[bot]
coderabbitai Bot previously requested changes Jun 26, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/utils/text.ts`:
- Around line 6-9: The exported tailLines function currently treats non-positive
line counts incorrectly, since lines === 0 falls through to slice behavior that
returns the full input. Update tailLines to explicitly handle lines <= 0 by
returning an empty string before splitting or slicing, while preserving the
existing behavior for positive counts and the trailing-empty-line trimming
logic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: kaizen-agents-org/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 358a60e6-76cc-4c4a-afcb-3b7dcef63301

📥 Commits

Reviewing files that changed from the base of the PR and between 2095811 and ea7957c.

📒 Files selected for processing (4)
  • src/commands/fleet.ts
  • src/utils/text.ts
  • test/fleet.test.ts
  • test/text.test.ts

Comment thread src/utils/text.ts
@s-hiraoku

Copy link
Copy Markdown
Collaborator Author

PR guardian update

Addressed the latest CodeRabbit edge case:

  • tailLines now returns an empty string for lines <= 0 before splitting/slicing output.
  • Added coverage for zero and negative line counts in test/text.test.ts.

Verification:

  • npm test -- test/text.test.ts
  • npm run typecheck
  • git diff --check
  • npm test
  • npm run build

Current state: change is pushed; waiting for CI and CodeRabbit on the updated head.

@s-hiraoku s-hiraoku dismissed stale reviews from coderabbitai[bot], coderabbitai[bot], and coderabbitai[bot] June 26, 2026 09:17

Dismissed stale CodeRabbit requested-changes review after follow-up commits addressed the comments and checks passed.

@s-hiraoku s-hiraoku merged commit b1b351c into main Jun 26, 2026
1 of 2 checks passed
@s-hiraoku s-hiraoku deleted the codex/scheduler-provider-docs branch June 26, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[monitor] Verify fleet refresh before returning to scheduled checks

1 participant