Fix oversized revalidation prompt metadata

[pai-scaffolde]

Summary

• Compact revalidation finding history and feature metadata before provider submission
• Preserve omitted counts for truncated metadata so providers know context was bounded
• Hard-cap revalidation metadata independently of user-configured review file limits
• Add a regression covering very large feature metadata staying below the provider input ceiling

Maintainer repair

• Rebased onto current main
• Clamped owned, context, and test metadata lists to 50 entries even when maxOwnedFiles or maxContextFiles is raised
• Added the unreleased changelog entry while preserving contributor credit

Verification

• Regression before repair: elevated file limits produced a 1,098,118-character prompt, exceeding the 1,048,576-character ceiling
• Regression after repair: pnpm test src/prompt.test.ts — 15 passed
• Full suite: pnpm test — 871 passed, 1 skipped
• pnpm typecheck
• pnpm lint
• pnpm format:check
• pnpm build
• Built live provider proof: oversized metadata (2,500 context refs, 200 test refs, 40 history entries, configured limits 10,000) produced 162,761 characters / 162,765 bytes, then the real Codex revalidation provider returned a valid structured outcome
• Autoreview: /Users/steipete/Projects/agent-skills/skills/autoreview/scripts/autoreview --mode branch --base origin/main — clean; no accepted/actionable findings

Public Model Identifier Gate: PASS — the candidate adds no model identifiers; the only provider-bearing fixture value is the existing public provider name with a null model.

[clawsweeper]

Codex review: needs real behavior proof before merge. Reviewed June 30, 2026, 4:15 PM ET / 20:15 UTC.

Summary
The PR compacts revalidation prompt finding/history and feature metadata before provider submission and adds a regression test for large revalidation metadata.

Reproducibility: yes. source-reproducible: current main embeds full finding and feature JSON in the revalidation prompt, and the PR regression shows the oversized metadata shape. I did not run the test because this review had to keep the checkout read-only.

Review metrics: 1 noteworthy metric.

• Changed surface: 2 files changed, +111/-2. The patch is focused on one prompt builder and its adjacent regression test, so the review risk is concentrated in provider prompt construction.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🦪 silver shellfish
Result: blocked until real behavior proof is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

• [P1] Add redacted terminal output or logs from an oversized clawpatch revalidate run showing the prompt stays below the provider ceiling.
• Hard-cap revalidation file-reference metadata independently of configured review file limits and cover that with a regression test.

Proof guidance:

• [P1] Needs real behavior proof before merge: The PR body lists verification commands but does not include redacted terminal output, logs, recording, or another artifact proving the after-fix revalidation behavior; updating the PR body with proof should trigger re-review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

• [P1] The PR body lists commands but does not include real terminal output, logs, screenshots, recordings, or a linked artifact proving the changed revalidation path after the fix.
• [P1] With high configured maxContextFiles or maxOwnedFiles, the new helper can still serialize thousands of file-reference metadata entries before the file-content budget applies.

Maintainer options:

1. Decide the mitigation before merge
   Keep the compact revalidation evidence approach, but hard-cap metadata list sizes independent of user file-context config and include redacted real terminal/log proof from an oversized revalidation run.
2. Pause or close
   Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

• [P1] Contributor action is needed for real behavior proof, and the remaining code issue is a review finding rather than a safe ClawSweeper repair marker because automation cannot provide the contributor's runtime proof.

Security
Cleared: No concrete security or supply-chain concern was found; the diff only changes TypeScript prompt shaping and adjacent tests.

Review findings

• [P2] Cap revalidation metadata independently of config — src/prompt.ts:466-468

Review details

Best possible solution:

Keep the compact revalidation evidence approach, but hard-cap metadata list sizes independent of user file-context config and include redacted real terminal/log proof from an oversized revalidation run.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: current main embeds full finding and feature JSON in the revalidation prompt, and the PR regression shows the oversized metadata shape. I did not run the test because this review had to keep the checkout read-only.

Is this the best way to solve the issue?

No, not yet: compacting metadata is the right direction, but file-reference metadata should be hard-capped independently of configurable review limits and the PR needs after-fix runtime proof.

Full review comments:

• [P2] Cap revalidation metadata independently of config — src/prompt.ts:466-468
config.review.maxContextFiles and maxOwnedFiles are user-configurable positive integers with no upper bound, but the new revalidation evidence helper uses them directly for ownedFiles, contextFiles, and tests. If a project raises those limits, this patch can still serialize thousands of long file-reference records before the 120k file-content budget applies, so the Codex hard input ceiling can still be exceeded. Clamp these metadata lists with a revalidation-specific hard cap and add a regression with elevated config limits.
  Confidence: 0.86

Overall correctness: patch is incorrect
Overall confidence: 0.78

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 24763d26cdff.

Label changes

Label changes:

• add P2: The PR addresses a normal-priority provider prompt-size bug with limited blast radius, but it still needs a bounded code repair and real behavior proof before merge.
• add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
• add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body lists verification commands but does not include redacted terminal output, logs, recording, or another artifact proving the after-fix revalidation behavior; updating the PR body with proof should trigger re-review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

• P2: The PR addresses a normal-priority provider prompt-size bug with limited blast radius, but it still needs a bounded code repair and real behavior proof before merge.
• rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦪 silver shellfish.
• status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body lists verification commands but does not include redacted terminal output, logs, recording, or another artifact proving the after-fix revalidation behavior; updating the PR body with proof should trigger re-review, or a maintainer can comment @clawsweeper re-review. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence reviewed

What I checked:

• AGENTS.md policy read: The full 55-line AGENTS.md was read; its guidance supports focused TypeScript changes with adjacent Vitest coverage and no generated output edits. (AGENTS.md:1, 24763d26cdff)
• Current main behavior: Current main serializes the full finding and feature records into buildRevalidatePrompt, so large metadata can be sent directly to the provider. (src/prompt.ts:434, 24763d26cdff)
• PR implementation: The PR replaces raw finding/feature JSON in the revalidation prompt with compact helper views and adds a large-metadata regression test. (src/prompt.ts:435, c136c25f2152)
• Config limit gap: maxContextFiles and maxOwnedFiles are positive integers with no upper bound, but the PR uses them directly for revalidation metadata list sizes. (src/config.ts:140, 24763d26cdff)
• Real behavior proof check: The PR body lists verification commands, but no terminal output, logs, recording, or linked artifact showing an after-fix real run was provided. (c136c25f2152)
• History provenance: git blame attributes the current buildRevalidatePrompt implementation on main to the v0.7.0 release commit; git log -S also shows revalidation prompt history in earlier lifecycle work. (src/prompt.ts:396, 0cd24d07a262)

Likely related people:

• Peter Steinberger: git blame and git log -S tie the current revalidation prompt implementation and earlier lifecycle/revalidation work to these commits. (role: introduced behavior and recent area contributor; confidence: high; commits: 72da7a401a2a, 0cd24d07a262; files: src/prompt.ts, src/revalidate.ts, src/types.ts)
• Vincent Koc: Recent prompt provenance and capped prompt evidence commits touched the same prompt/test area that this PR extends. (role: adjacent prompt provenance contributor; confidence: medium; commits: 43df8aab17a6, 1745e208bed9, 87ef57fdefb9; files: src/prompt.ts, src/prompt.test.ts, src/review-validation.ts)
• Hunter Sadler: Prior prompt-bound evidence and normalized fix-evidence work changed nearby prompt assembly behavior and tests. (role: adjacent prompt behavior contributor; confidence: medium; commits: b11499a9892d, 27cf2488a1a9; files: src/prompt.ts, src/prompt.test.ts, src/review-validation.ts)

What the crustacean ranks mean

• 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
• 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
• 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
• 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
• 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
• 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
• 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c136c25f21

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Clamp revalidation metadata to a hard cap

When users raise review.maxContextFiles or review.maxOwnedFiles (the config schema only requires positive integers at src/types.ts:141-142), these values let the new evidence view serialize thousands of file refs and long reasons again. For example, maxContextFiles=2500 with 300-byte context reasons yields roughly 960 KB in the Feature JSON alone; adding the 120 KB file-context budget pushes the revalidation prompt back over the 1 MiB Codex ceiling this change is meant to avoid. Clamp these to a hard revalidation metadata cap instead of trusting review prompt limits.

Useful? React with 👍 / 👎.

[steipete]

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b9895f5302

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Fix the impossible context-path assertion

In this fixture the 2,500 docs/context-*.md files are never written, so rawFileBlock emits a short [unreadable] block for each; buildRevalidatePrompt still takes paths from fixPromptPaths using the raised maxContextFiles and only stops at the 120k current-file budget. Those unreadable docs plus the 200 test refs are only about 100k characters, so docs/context-2499.md remains in the Relevant current files section and this new test fails deterministically; either assert against the compact Feature JSON specifically or make the fixture exceed/avoid the file-context path budget.

Useful? React with 👍 / 👎.

[steipete]

Landed in 1ffaf19.

Proof on exact PR head 9698057:

• Regression: src/prompt.test.ts covers 2,500 context refs, 200 test refs, 40 feature histories, 40 finding histories, and configured metadata limits of 10,000. The original patch exceeded the 1 MiB ceiling (1,098,118 characters); the repaired prompt stays bounded.
• Focused/full gates: pnpm vitest run src/prompt.test.ts; pnpm typecheck; pnpm lint; pnpm format:check; pnpm build; pnpm test (871 passed, 1 skipped).
• Built runtime: dist/prompt.js produced a 162,765-byte prompt and the real built Codex provider returned a valid structured false-positive revalidation result.
• Public Model Identifier Gate: pass; no model identifiers added or changed.
• Autoreview: clean on the final branch diff.
• CI: CI, CodeQL, and Secret Scan all succeeded.

Thanks @pai-scaffolde for the contribution.