Skip to content

fix(verifier): suppress native caches while preserving unknown files - #294

Draft
dstkwll wants to merge 6 commits into
All-The-Vibes:mainfrom
dstkwll:codex/issue-140-verifier-caches
Draft

dstkwll wants to merge 6 commits into
All-The-Vibes:mainfrom
dstkwll:codex/issue-140-verifier-caches

Conversation

@dstkwll

@dstkwll dstkwll commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Python verification could leave bytecode caches that forced preservation of an otherwise clean worktree. Apply native cache controls only to verifier children: direct Python gets -B and its environment setting; wrappers and Node compile caches have explicit typed opt-ins. Keep the conservative file-preservation gate and record requested controls separately from observed source state.

Closes #140.

Scope and provenance

Behavior

Explicit controls require the selected runner's available, global verifier-cache-suppression-v1 capability. The server enforces this during scheduling and current-epoch sends, including resumes and durable recovery. Unsupported launch consumes no attempt. Offline direct-plan creation and the exact legacy readiness response remain available for policies without explicit controls.

Automatic suppression for omitted controls needs an updated runner. Environment-only wrappers can discard settings; explicit file writes remain possible. Requested suppression is not proof of zero caches or ownership. Unknown ignored files remain preserved. Node policy covers its native compile cache only.

Validation

  • All six repository gates passed on the current PR head: migration integrity (41), formatting, all-target Clippy, workspace tests, web build and web lint. Default Rust suite: 523 passed, 0 failed, 341 explicitly ignored.
  • Five new real-PostgreSQL admission regressions passed locally: legacy recovery and resume reject before preparation, capable-runner controls reach the secret/dependency/artifact canaries, and recovery failure evidence remains stable across repeated dispatcher sweeps. CI now runs these opt-in tests. The new Windows-gated launcher policy test passed in native Windows CI (216 runner tests passed, 4 ignored). The new real-SQL CI step also passed.
  • Earlier source-bound feature validation: all five opt-in runner cases passed, covering actual Python imports under normal/-E/-I/-IE, Node cache disabling, clean linked-worktree removal and byte-preserving ignored-file retention.
  • Earlier browser/server/runner validation: two browser-launched missions completed with six persisted passing checks; clean worktree removed and ignored-file worktree preserved. A legacy-capability WebSocket fixture received no assignment; launch left the inspected task at zero attempts/runs. Deterministic provider fixtures, no vendor inference.
  • GitHub Copilot Astra/high design and independent reviews completed; findings were verified locally. All six hosted checks passed at the tested implementation revision debbc2138c27b7e7cc6e9c212719be3072421c15 (CI run). All six hosted checks also passed for the current documentation-only head 63006e57ae0808f125978037b6fe492bb7c53d7e (CI run); all six local gates passed again.

Feature evidence and reproduction. Earlier runtime evidence is source-bound to the revisions recorded there; the current-main reconciliation was checked with the six local gates.

Integration and limits

A separately agreed local-only experiment combines this PR with the published #305 and #307 heads. Its changes are not part of this PR; results are reported in the discussion. Full recovery/resume lifecycle permutations and native Windows acceptance are not claimed by the macOS evidence. Owner review and the other PRs' acceptance gates remain separate.

This PR remains draft. No merge, auto-merge, deployment or automatic Factory intake has been enabled.

Copilot AI balanced review requested due to automatic review settings September 16, 2026 18:04

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Explicit suppression policies can be silently ignored by older selected runners without capability enforcement.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds verifier-scoped cache suppression while retaining conservative workspace preservation.

Changes:

  • Adds typed Python and Node cache controls.
  • Records suppression and workspace-disposition evidence.
  • Adds focused regressions and supporting documentation.
File summaries
File Description
apps/web/src/App.tsx Extends frontend verifier policy types.
crates/crony-domain/src/lib.rs Defines typed cache-suppression policies.
crates/crony-runner/src/main.rs Updates verifier test construction.
crates/crony-runner/src/retained_provider_receipt_tests.rs Updates receipt test policies.
crates/crony-runner/src/source_checkpoint.rs Updates checkpoint test policies.
crates/crony-runner/src/verifier.rs Applies and records native cache controls.
crates/crony-runner/src/workspace.rs Adds NUL-safe dirty-file accounting and regressions.
crates/crony-server/src/planning.rs Updates planning and policy validation patterns.
crates/crony-store/src/contract_revision.rs Supports the extended verifier variants.
docs/ARCHITECTURE.md Documents behavior and compatibility limits.
docs/EVALS.md Records the focused regression matrix.
docs/evidence/2026-09-16-verifier-cache-suppression.md Provides implementation and acceptance evidence.
Review details
  • Files reviewed: 12/12 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread crates/crony-domain/src/lib.rs
Comment thread crates/crony-runner/src/main.rs Outdated
@afateen

afateen commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

@dstkwll, thanks for the offer to coordinate. This is a compatibility and landing-order request for PR #305 / issue #297 and PR #307 / issue #79, not a confirmed defect report or a request for a broad rewrite.

Rationale and integration checks

#294 and #305 both change runner capability admission and current-epoch dispatch, including resume/recovery. #307 also changes shared planning/store surfaces. Please check that integration preserves all of the following:

  • When an assignment requires both features, verifier-cache-suppression-v1 and verified-dependency-files-v1 are both enforced. Neither change should overwrite the other's checks; legacy behavior must remain intact where a capability is not required.
  • Resume/recovery keeps its epoch fencing, idempotency and command-queue behavior, including progress for unrelated compatible commands.
  • Verified research notes/probes remain ordinary source/reference files. Preserve fix(verifier): suppress native caches while preserving unknown files #294's conservative handling of unknown files; no cache-deletion or file-preservation exception is requested.
  • Shared planning/store integration retains fix: reject incompatible Factory cost budgets before claims #307's early strategy-cost rejection, no-side-effect regressions for invalid new claims, and connection/source-selector guards for explicit legacy reconciliation.

Please confirm your preferred landing order and flag any minimal adjustment needed, with rationale. If no code adjustment is needed, a compatibility confirmation is useful. A sanitized description of the pinned toolchain and approved validation setup used for your passing gates would also help; no credentials or IT-policy workaround are requested.

We will need fresh evidence on the exact integrated candidate. Separate PR test results do not prove the combination. Rajesh's requested review on #305 and Shyam's on #307 remain separate; this coordination does not replace owner review or outstanding acceptance gates.

No merge, auto-merge or deployment is requested.

@dstkwll

dstkwll commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator Author

@afateen, thanks for the clear coordination request and for separating compatibility work from the outstanding owner reviews and acceptance gates. I agree with the integration requirements you listed.

Proposed order: #294 looks like the first candidate based on its completed local validation and passing hosted CI at 32e072ed1b84659a78bd5388147c1104ca1b3df3, subject to owner review. Does that order work for you? I do not see a demonstrated hard dependency between #307 and #305; their subsequent order can follow readiness and your preference.

There is a concrete integration seam: #294 changes the shared sender to a typed result, while #305 adds its own typed delivery path behind the boolean sender. I suggest reconciling these into one final admission path that preserves both capability checks, the current-epoch guard, and each caller's handling of failure. This is not a standalone defect claim against either PR.

One behavior needs explicit agreement: #294 currently fails an unsupported cache-policy recovery before dispatch, whereas #305 retains dependency-incompatible recovery and lets unrelated compatible commands progress. For the initial experiment, I propose preserving those existing semantics: missing cache support rejects recovery; with cache support satisfied, missing dependency support retains it. Neither condition sends unsupported work. If you prefer a unified temporary-incompatibility policy, let's decide that separately rather than changing recovery semantics implicitly during conflict resolution.

Bounded experiment, if that works for you: create a separate local integration branch from the exact agreed PR heads, make only the necessary integration adjustments, and test:

  • Both required capabilities on the same runner, including neither/either/both supported and legacy assignments requiring neither.
  • Start/resume/recovery epoch fencing, idempotency, and unrelated-command progress, including a queue extending beyond one blocked page.
  • Preservation of unknown files and ordinary research notes/probes, with no new deletion exception.
  • fix: reject incompatible Factory cost budgets before claims #307's early cost rejection, no-side-effect assertions, and legacy reconciliation selector guards.
  • Focused regressions and applicable full contributor gates on the combined candidate.

I will post the exact source hashes, minimal adjustments, commands/results and limitations, including failures. The temporary branch is an experiment, not a replacement PR or approval of either contribution. Your remaining native/SQL/browser acceptance and Rajesh's/Shyam's requested reviews remain separate.

Sanitized setup: personal Apple Silicon Mac, fully local isolated worktrees outside cloud-backed storage, owned disposable PostgreSQL 17 fixtures and candidate server/runner binaries. Currently verified tools are Homebrew Rust 1.98.1, Node 22.23.1 and pnpm 11.19.0; the repository specifies Rust minimum 1.94, not an exact Rust pin. Tests use the committed lockfiles (--locked --offline for Cargo) and source-bound evidence. Test credentials/database state remain private; no employer IT-policy changes or workarounds are involved. I will record the actual versions again for the experiment.

No merge, auto-merge, deployment or Factory intake is proposed. Please confirm the provisional order and recovery behavior, or suggest a small adjustment.

@afateen

afateen commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

@dstkwll, yes: your proposed #294-first order works as a provisional integration order, subject to its owner review and conflict resolution. #307 and #305 can follow independently according to their remaining gates.

For the bounded experiment, preserve the existing recovery behavior you described:

  • Missing verifier-cache-suppression-v1 rejects cache-policy recovery before dispatch.
  • Once cache support is satisfied, missing verified-dependency-files-v1 retains dependency-incompatible recovery while unrelated compatible commands can progress.
  • Neither path sends unsupported work. A unified temporary-incompatibility policy would be a separate behavioral decision, not an implicit conflict-resolution change.

Your proposed single final admission path should retain both capability checks, current-epoch fencing, caller-specific failure handling and queue progress beyond one blocked page. Preserve unknown files and ordinary research notes/probes; no deletion exception is requested. Retain #307's early rejection, no-side-effect and selector-fence regressions when including it.

The current snapshot still shows #294 at 32e072ed1b84659a78bd5388147c1104ca1b3df3, draft and conflicting, with green checks. Local source main is now 7eecd39e43720851512225844bf5d8158e8a944f; our unchanged heads remain #305 ca15684b84710dbc6450c20b9117f62f89d38887 and #307 05e302c957b62611dcfde5583f11b659835f93cf. Please record the new exact heads/base after your conflict resolution before testing a combination.

This confirms the scope of your separate local experiment, not replacement PRs, combined acceptance, native-provider execution, Factory intake, publication, merge or deployment. Rajesh's/Shyam's reviews and the remaining #305 native/SQL/browser/adversarial gates stay separate. Thank you for the concrete proposal and sanitized setup details.

Copilot AI review requested due to automatic review settings September 17, 2026 15:23

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical recovery, resume, and Windows-specific admission behavior lacks direct regression coverage.

Get a fresh assessment by requesting another Copilot review.

Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread crates/crony-runner/src/verifier.rs
Comment thread crates/crony-server/src/main.rs
Comment thread crates/crony-server/src/main.rs
@dstkwll

dstkwll commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator Author

@afateen, thanks for confirming the scope. The local experiment is complete, with useful results and two concrete integration/test findings.

Exact inputs and disposition

The integration retains one typed final sender, both available/global capability checks on the same runner, epoch fencing, cache rejection before preparation, dependency-incompatible retention, and the existing caller-specific outcomes. VerifyRun still carries no dependency files. Mechanical adjustments retain both planner tests/store fields and initialize the new research command's optional cache policy to None.

Verified results

  • All six contributor gates passed on the combined candidate: migration integrity (41), format, all-target Clippy, workspace tests (559 passed, 0 failed, 347 explicitly ignored), web build and lint. Rust --locked --offline; Rust 1.98.1, Node 22.23.1, pnpm 11.19.0, owned PostgreSQL 17 fixtures on Apple Silicon macOS.
  • Focused runs for issue79_, issue297_, integration_, and issue223_: 24 passed, including opt-in SQL cases. Five opt-in Python/Node cache tests also passed.
  • Combined admission: 44 start/resume/verify/stop cases; 144 single-runner selection combinations plus split-runner checks. Existing epoch and dependency-file preservation tests also pass.
  • Real CLI/server/PostgreSQL cost canary: 14 invalid CLI cases, four invalid server cases, two valid boundary cases, six legacy selector denials; audited reconciliation passed, with zero Project mutations and zero missions/tasks/runs.
  • Real server/runner cache canary: two completed runs, six passing persisted checks; clean worktree removed, ignored valuable file preserved.
  • Research mission: all three native fixture runs completed with passing verification and preserved worktrees. Read-only re-verification through signed artifact APIs confirmed synthesis consumed both notes and both probes, with exact byte/digest equality, distinct worktrees and correct parent-before-child ordering. No replacement mission was launched for that re-verification.

Findings to carry forward

  1. Mixed-page queue progress needs more than blocked-run exclusion. The existing if dispatched || batch_len < 100 ends the sweep after any successful send, even with later compatible work behind blocked entries. Copilot's independent review identified this; an actual dispatcher/SQL regression reproduced the general pagination limit: 100 of 202 unacknowledged commands delivered. The local correction excludes both visited command IDs and blocked run IDs before LIMIT 100, then continues until a short/empty page. The regression now delivers 202 once per sweep, repeats the same set on a later sweep, and leaves all commands pending. Separate SQL tests preserve blocked rows across page boundaries. This is finite-backlog, per-invocation evidence; it is not globally exactly-once delivery or a completed mixed-recovery lifecycle test. I suggest carrying this bounded correction and the mixed-recovery regression into fix: deliver verified research files to synthesis #305's integration work.

  2. The research driver overstates mission_launch. verifyOutcome requires mission_launch === true for both roots. The store emits that flag only when the mission was ready; the second explicitly actor-launched root correctly has false. The unmodified driver therefore failed after successful execution. A separate read-only replay corrected the assertion to exactly one root transition and passed the remaining content checks. The original failure and database/artifacts are retained. The parent fixture must also wait for successful read-only mission preview: connection registration precedes reconciliation readiness, and the initial early attempt returned HTTP 400 without creating a mission.

Remaining limits

The inherited startup fixture passed its rejection checks but failed the production TLS-storage contact assertion on this Mac. A focused diagnostic still observed zero storage requests after five seconds and artifact staging discovery deferred; this is unresolved, not a passing result or a demonstrated regression attributable to these PRs. All corresponding evidence is retained.

Copilot Astra/high reviewed the integration and pagination correction; I independently checked the findings and executed the tests. This experiment does not replace #305's remaining native/SQL/browser/adversarial gates, vendor-provider acceptance, or Rajesh's/Shyam's reviews. A live mixed recovery/approval queue test remains desirable before claiming that entire lifecycle. No merge, auto-merge, deployment, or automatic Factory intake was enabled.

My recommendation remains the agreed provisional #294-first order, subject to owner review, with #307/#305 following their own readiness. The queue and driver findings above are concrete follow-ups for the later integration.

Copilot AI review requested due to automatic review settings September 17, 2026 16:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The source-bound evidence contains stale fingerprints and contradicts the stated current-head revision.

Get a fresh assessment by requesting another Copilot review.

Review details
  • Files reviewed: 16/16 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread docs/evidence/2026-09-16-verifier-cache-suppression.md
Comment thread docs/evidence/2026-09-16-verifier-cache-suppression.md Outdated
Copilot AI review requested due to automatic review settings September 17, 2026 17:04

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-service dispatch, recovery, and workspace lifecycle changes warrant final human review while the PR remains draft.

Review details
  • Files reviewed: 16/16 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Suppress runner-owned verifier caches without deleting unknown ignored files

3 participants