Skip to content

add deployment related files (DO NOT MERGE) - #18

Closed
tarunpandey23 wants to merge 56 commits into
mainfrom
feat/deployment
Closed

tarunpandey23 wants to merge 56 commits into
mainfrom
feat/deployment

Conversation

@tarunpandey23

Copy link
Copy Markdown
Collaborator

No description provided.

safayavatsal and others added 30 commits March 17, 2026 17:49
… Feature Spec)

Tech stack v1.1 — updated with latest stable versions as of 2026-03-17.
- pnpm workspaces (core, cli, action) with tsdown bundling
- TypeScript 5.9 strict config (Node16 module resolution)
- Vitest for testing, ESLint 10 flat config, Prettier
- CI workflow (Node 20/22/24), release workflow (npm publish)
- .env.example with full configuration template
- README, CONTRIBUTING, CODE_OF_CONDUCT, MIT LICENSE
pnpm/action-setup@v4 reads version from package.json packageManager
field — specifying it again in the workflow causes a conflict error.
- Fix eslint.config.js import order (eslint-plugin-import-x before typescript-eslint)
- Drop Node 24 from CI matrix (not yet available on GitHub runners), use 20+22
- Set release workflow to Node 22
- Add "type": "module" to root package.json to eliminate ESM parse warning
- Create all core/src/ module directories (review, github, chat, learnings,
  sandbox, config, server, llm, trace)
- Create cli/src/commands/ directory
- Add SKILL.md agent skill definition (Claude Code, Cursor, Gemini CLI)
- Add REVIEW.md with project review rules for dogfooding
…ader)

- core/src/config/env.ts: typed OpenReviewConfig with dotenv, validates
  LLM API keys, parses booleans/globs/ints/floats, expands ~ in paths
- core/src/config/instructions.ts: discovers REVIEW.md, AGENTS.md,
  CLAUDE.md, .cursorrules, .windsurfrules with hierarchical scoping,
  priority-ordered concatenation, 10k token cap
- 29 unit tests (16 env + 13 instructions)
- Added dotenv, tinyglobby, @types/node dependencies
Completes Phase 1 Week 1 — all 78 foundation tasks done.

- core/github/client.ts: GitHubClient with PR fetch, file fetch, pagination,
  rate limit interceptor, 5xx retry with exponential backoff, 30s timeout,
  PR URL parser
- core/github/diff.ts: unified diff parser, copy/move detection via Jaccard
  similarity, include/exclude glob filtering
- core/github/comments.ts: CommentPoster with batched review posting,
  replace-not-duplicate summary comments, inline formatting with severity
  badges and suggested fix blocks
- core/llm/router.ts: LangChain model router (OpenAI/Anthropic/Google),
  streaming support, custom OpenAI-compatible endpoint support
- 65 tests passing across 6 test files
- Dependencies: axios, @langchain/core, @langchain/openai, @langchain/anthropic,
  @langchain/google-genai, @langchain/langgraph
feat: GitHub API client, diff parser, comment poster & LLM router
…nt guide

- Add review data model (ReviewFinding, Citation, PRContext, severity/category/source types)
- Add fast review engine with LLM prompt construction, response parsing, and citation validation
- Add parallel linter orchestration (ESLint, Ruff, Semgrep, ShellCheck, Gitleaks) with 30s timeouts
- Add finding deduplication (AI + linter overlap detection by file + line range)
- Add summary and inline comment formatters with GitHub suggestion syntax
- Migrate comments.ts to canonical review types (single source of truth)
- Add SETUP.md deployment guide (prerequisites, CLI/Action quickstart, config reference, troubleshooting)
- Harden all linter parsers against malformed JSON input
- 189 tests passing, lint clean, typecheck clean
feat: fast mode review engine, linter orchestration, and deployment guide
Implements executeSandboxed() with strict permission sandboxing (no network, no write, no env), 30s hard timeout via AbortController, globals injection, and environment variable stripping to prevent secret leakage. Includes verifyDenoInstallation() to check for Deno 2.7+ at startup.
Implements SnapshotBuilder class with lazy file fetching from GitHub API, in-memory caching, per-file and total byte caps, binary file detection, and full file tree listing. Pre-loads diff file paths for fast access during review.
Implements the Reason-Loop-Measure review engine using LangGraph StateGraph with 5 nodes (reason, code_writer, sandbox, observe, finalize). Supports iteration and LLM call limits, finish_review signal, file fetch requests, event streaming, and grounded finding generation from sandbox observations.
Implements TraceLogger class that records fast review entries, RLM iterations, findings, and session metadata to ~/.openreview/traces/. Includes automatic secret scrubbing for API keys, GitHub tokens, and other credentials before writing to disk.
Implements handleChatMention() for @openreview PR comment interactions with thread history loading, streaming LLM response, citation validation against snapshot, and bot loop prevention. Adds generateSuggestions() using the sub LLM to produce concise follow-up questions appended to chat replies.
Implements LearningsStore class for persistent per-repo learning storage at ~/.openreview/learnings/. Supports add, list, delete, usage tracking, and automatic pruning at 50 learnings capacity. Includes trigger phrase detection for 'false positive', 'ignore this', etc. and formatLearningsForPrompt() with 2,000 token cap for injection into review prompts.
1. Fix postChatReply to use correct GitHub API endpoint (POST /issues/{prNumber}/comments instead of invalid /issues/comments/{id})
2. Add explicit Deno permission flags (--allow-read, --deny-net, --deny-write, --deny-env, --deny-run) for proper sandbox isolation
3. Replace private client['api'] access in chat-handler with public getIssueComments() method on GitHubClient
4. Fix snapshot.ts to use ParsedDiff.file instead of non-existent newPath property
5. Fix deno-runner.test.ts type casts for ChildProcess return type
1. Clarify exitCode extraction in deno-runner — handle string vs number error.code
2. Add null-safety to getIssueComments — filter deleted users, normalize null bodies
3. Fix snapshot concurrent access race condition — track in-flight fetches to prevent double-fetching and inflated byte counts
4. Fix floating promise in learnings-store test — assertions were never executing
5. Fix broken error-path test in suggestions — LLM mock now throws via flag variable
6. Guard against empty LLM response in chat-handler — don't post empty replies
7. Add try/catch to TraceLogger.close() — trace failures don't crash reviews
8. Add 4 new tests: concurrent snapshot access, empty answer guard, getIssueComments with null users, perPage parameter
New test files covering previously untested areas:
- CommentPoster: postReview, postSummaryComment (create + update), postChatReply, postAcknowledgement, summary marker pagination
- Linter orchestration: runLinters with enabled/disabled linters, graceful failure handling, combined findings
- RLM deep mode: code block extraction, FETCH_FILE requests, finding parsing, edge conditions, citation handling
- LLM router: createLLM for all 3 providers, baseURL passthrough, createMainLLM/createSubLLM, streamChat chunking, provider detection edge cases
- Fast review integration: runFastReview end-to-end with citation validation, deduplication, severity sorting, summary construction
- Trace logger edge cases: multi-entry accumulation, all secret formats, special chars, idempotent close
- Sandbox edge cases: empty code, large globals, exit code 2, string error codes, Deno version boundary tests
Move all 22 test files from core/src/**/*.test.ts to tests/core/**/\*.test.ts,
mirroring the source module structure. Update all internal import paths to
use ../../../core/src/ prefix. Add axios and @langchain/* as root-level
devDependencies for test module resolution. Update vitest.config.ts include
pattern to tests/**/*.test.ts.
…nd todo list

Adds Phase 1 (Tree-sitter static import/dependency graph, CLI integration,
terminal + JSON output, review finding enrichment) and Phase 2 (LLM data-flow
analysis, screenshot diffing, live preview, GitHub PR comment, HTML dashboard)
impact analysis feature across all planning documents and REVIEW.md architecture rules.
feat: Phase 1 Week 3 — RLM deep mode, chat, learnings, sandbox, trace
* docs: extract Impact Analysis as standalone product spec

Consolidates all Impact Analysis content from PRD, Feature Spec,
Milestones, and TodoList into a single shareable document for PM review.

* fix: resolve 5 user-reported issues from manual testing

1. Suppress dotenv v17 noisy logging (quiet: true)
2. Improve no-token error with scope guidance and token creation link
3. Add structured LLM output via Zod + withStructuredOutput() with
   resilient parser fallback for models that don't support it
4. Set temperature=0 for review calls (deterministic output),
   temperature=0.3 for chat/suggestions
5. Fix RLM recursion limit (maxIterations * 5 + 10) that caused
   LangGraph GRAPH_RECURSION_LIMIT errors

Additional fixes from testing:
- Auth scheme detection: classic PATs (ghp_) use "token" scheme,
  fine-grained PATs use "Bearer"
- Auth error interceptor with actionable 401/403/404 messages
- Resilient LLM response parser handles alternate field names
  (type→category, message→title, location→file:line)
- Citation validation includes diff context lines (not just added)
- Sandbox timeout reduced 30s→15s for faster RLM iterations
- RLM convergence hints and iteration budget awareness in prompts
- SUB_MODEL default set to gpt-4o-mini (gpt-3.5-turbo too weak
  for structured schema compliance)
- MAX_ITERATIONS 20→12, MAX_LLM_CALLS 25→35

Tested end-to-end against open-metadata/docs-om#138.
All 320 tests passing.

* fix: resolve lint and typecheck errors in router.ts

Move zod, @langchain/core/runnables, @langchain/core/messages imports
to top of file in correct order (import-x/order). Fix TypeScript
error in createStructuredLLM return type with explicit cast.

* fix: resolve lint and typecheck errors in router.ts

Remove unused BaseMessageLike import, fix Runnable return type,
add explicit type annotation for structured output result.

* fix: resolve all 5 fast review issues from manual testing

1. Severity normalization map — maps "low"→"non-severe", "high"→"severe",
   "critical"→"severe", "medium"→"non-severe", etc. for raw parser fallback
2. Category normalization — maps "Documentation"→"flag", "Security"→"bug",
   etc. with keyword matching
3. Structured output empty retry — if structured path returns 0 findings,
   retries with raw LLM + resilient parser as second chance
4. Snap-to-nearest citation validation — finds nearest diff line within
   ±10 lines (like PR-Agent) instead of silently dropping findings
5. ReviewTrace diagnostic object — tracks raw→validated→final counts,
   dropped findings with reasons, empty reason classification, LLM path

Additional fixes:
- Handle description-only findings (no separate title/explanation)
- Infer file from changedFiles when LLM omits file field
- Default startLine=1 when file known but line missing (context-aware)
- Strengthen prompt: config/YAML correctness section, explicit enum
  enforcement with CRITICAL instruction, better example
- Add raw error handling in fallback path

Tested E2E: SQL injection test PR → 3 findings caught correctly.
All 320 tests passing, lint clean, typecheck clean.

* fix: RLM graceful sandbox degradation and stderr in events

1. Check Deno availability at RLM startup via verifyDenoInstallation()
2. When Deno is not installed, switch to reasoning-only mode:
   - System prompt tells model "Do NOT write code blocks"
   - Model goes straight to reasoning + finish_review
   - Reduces wasted iterations from 5+ to 1
3. Include stderr in sandbox event message when exit code != 0
   for better error diagnostics

Before: 5 iterations, 20s, sandbox fails silently every time
After:  1 iteration, 10s, clean reasoning-only mode with 3 findings

* feat: file-type-aware prompting for config/docs/k8s PRs

Detects dominant file type in the diff (code/config/docs/k8s) and:
- Switches reviewer persona (code reviewer → K8s auditor / config
  specialist / documentation reviewer)
- Injects file-type-specific review checklist (K8s security, config
  structure, docs accuracy)
- Adds anti-empty instruction for non-code PRs forcing at least one
  informational finding
- Adds focused retry (Path D) with stronger file-type-aware re-prompt
  when initial review returns empty on non-code files

Before: docs-om PR #138 → 0 findings (3 runs)
After:  docs-om PR #138 → 3 findings consistently (3 runs)

Inspired by CodeRabbit's path_instructions and KubeGuard's
file-type-specific persona approach.

* fix: rewrite sandbox to use deno run instead of deno eval

deno eval has implicit all-permissions and doesn't support --deny-*
flags. Changed to deno run with temp files:
- Write script to temp file in /tmp/openreview-sandbox/
- Run with --allow-read=<script> --deny-net --deny-env --deny-run
- Clean up temp file after execution

Tested with Deno 2.7.7:
- Basic execution: works (exit=0)
- GLOBALS injection: works (reads file content correctly)
- Network access: correctly denied (exit=1)
- File write: correctly denied (exit=1)
- Timeout: correctly fires (exit=124)
- RLM with real sandbox: 2 iterations, 14s, 3/3 findings

* feat: implement CLI commands (review, ask, serve, traces) and formatter

Section 10 of Week 4 tasks:

10.1 CLI Entry Point (cli/src/main.ts)
- commander v14 setup with version, help, error handling
- Registers all 4 subcommands

10.2 Review Command (cli/src/commands/review.ts)
- openreview review --url <PR-URL> [--mode fast|rlm] [--output text|markdown|json] [--model <id>] [--expert] [--quiet]
- Fetches PR via GitHub API, runs fast or RLM review
- --expert adds SOLID/security/quality instructions
- Progress output to stderr, results to stdout

10.3 Ask Command (cli/src/commands/ask.ts)
- openreview ask [--repo <path>] [--url <PR-URL>]
- Interactive REPL with readline
- Commands: reset, history, files, exit
- Streams LLM responses with citations

10.4 Serve Command (cli/src/commands/serve.ts)
- openreview serve [--port <n>] [--host <host>]
- Express.js server with /health endpoint

10.5 Traces Command (cli/src/commands/traces.ts)
- openreview traces --pr <url> | --list | --open <file>
- Lists, filters, and pretty-prints trace files

10.6 Output Formatter (cli/src/formatter.ts)
- formatText() — plain text with severity icons
- formatMarkdown() — severity-grouped markdown with badges
- formatJSON() — raw JSON

Also fixes:
- core/package.json: main/types point to actual .mjs/.d.mts files
- Router tests: rewritten without ESM mocking issues

* feat: implement GitHub Action (action.yml, pr-handler, comment-handler)

Section 11 of Week 4 tasks:

11.1 Action Definition (action/action.yml)
- All inputs: github-token, openai/anthropic/gemini-api-key,
  main-model, sub-model, max-files, review-drafts
- runs: using node24, main: dist/index.mjs
- Branding: eye icon, blue color

11.2 Action Entry Point (action/src/index.ts)
- Reads event type from github.context
- Routes pull_request → pr-handler (opened/synchronize/reopened/ready_for_review)
- Routes pull_request_review_comment/issue_comment → comment-handler
- Wraps in try/catch with core.setFailed()

11.3 PR Handler (action/src/pr-handler.ts)
- Extracts PR from payload, skips drafts and "openreview: skip"
- Posts "Review started..." acknowledgement
- Runs fast review, posts batch review + summary comment
- Injects action inputs to process.env for core config
- Posts error comment on failure

11.4 Comment Handler (action/src/comment-handler.ts)
- Bot loop prevention (skips [bot] authors)
- @openreview rlm → RLM deep review with progress events
- @openreview review → fresh fast review
- @openreview list learnings → post learnings list
- @openreview forget: <desc> → delete matching learning
- @openreview <question> → chat handler with snapshot context
- Learnings trigger detection (ignore this, false positive, etc.)

* docs: add Codex example and API key verification to SKILL.md

Complete Section 12 checklist:
- Added Codex example for agent ecosystem
- Added API key verification section with shell commands
- All 6 SKILL.md tasks complete

Section 13 (README, REVIEW.md, CONTRIBUTING.md) already complete
from previous commits.

* test: add tests for auth scheme detection, factory methods, structured LLM

Section 14 — Testing & QA:
- Auth scheme tests: classic PAT (ghp_) uses "token", fine-grained uses "Bearer"
- Factory method tests: fromPRUrl creates client correctly, throws on invalid URL
- Structured LLM tests: createStructuredLLM with Zod schema and custom temperature
- Added zod as root devDependency for test access

321 tests passing, 0 lint errors, typecheck clean.

* chore: add coverage/ to gitignore and remove from tracking

* test: add 10-PR QA validation suite (Section 14)

Tests 10 PRs with known bugs across 5 language categories:
- 2x TypeScript (off-by-one, assignment-in-condition, SQL injection, eval)
- 2x Python (mutable default, is-vs-==, hardcoded API key)
- 2x Shell (unquoted vars, rm -rf, eval injection, chmod 777)
- 2x Terraform (public S3 ACL, overly permissive IAM wildcard)
- 2x Multi-file (type regression on move, string-vs-number mismatch)

Result: 9/10 bugs caught (target: >= 8) — PASS
Only miss: subtle type safety regression in moved code (test 9)

* test: complete Section 14 QA — CLI integration + lint fixes

Section 14 Testing & QA checklist:
- [x] 321 unit tests (>80% coverage on core modules)
- [x] 10-PR QA suite: 9/10 bugs caught (target: >=8) — PASS
- [x] CLI integration: review (fast + RLM), text/json/markdown output
- [x] Fast mode <60s on all 10 PRs (all under 8s)
- [x] RLM with real Deno sandbox (4 iterations, exit=0)
- [ ] Create 10 test PRs on GitHub (deferred — tested via synthetic diffs)
- [ ] Manual @openreview commands (requires live Action deployment)
- [ ] Learnings CRUD via live comment (requires live Action deployment)

Lint: 0 errors, 37 warnings (console.log in CLI — expected)
Typecheck: all 3 packages pass

* docs: update progress docs to reflect Week 4 completion

- Milestones.md: Week 3 + Week 4 marked complete (v1.2)
- TodoList.md: Sections 10-14 all marked complete (v1.2)
- PRD.md: status updated to Phase 1 Week 4 complete (2026-03-24)

Phase 1 at 97.5% — 78/80 tasks done.
Remaining: 2 live Action tests + 6 launch checklist items.

* fix: add pnpm build step before typecheck in CI

CLI and Action packages resolve @openreview/core types from
dist/index.d.mts which only exists after build. Without the
build step, typecheck fails with 'Cannot find module' errors.

* fix: resolve 4 issues from fresh user testing

1. Adaptive prompt length (CRITICAL):
   - Small PRs (≤5 files, ≤3000 char diff) use compact prompt
   - Large PRs use comprehensive prompt with full category checklist
   - Fixes 0-findings on real code PRs (skyflo#133: 0→2 findings)
   - Compact prompt is more focused, preventing model conservatism
   - Consistent across 3 runs (2 findings every time)

2. Clean help output when no command:
   - Shows help and exits 0 instead of Error: (outputHelp)

3. Better 404 error message:
   - Now says "PR/repo does not exist" as first possibility
   - Then mentions private repo access as second possibility

4. Consistency improvement:
   - Compact prompt produces more deterministic results
   - 3/3 runs on skyflo#133 → 2 findings each time

Tested on:
- skyflo-ai/skyflo#133 (small React PR): 2 findings consistently
- #6 (large 12-file PR): 11 findings

* feat: chunked diff review for large PRs + file filtering

Large PRs (627K+ char diffs) were sending the entire diff to the
LLM in a single call, exceeding context limits and returning 0 findings.

Changes:
- Skip non-reviewable files: lock files (yarn.lock, pnpm-lock.yaml,
  package-lock.json), minified files, source maps, images, fonts,
  dist/, vendor/, generated files
- Chunk large diffs by file boundaries (~40K chars per chunk)
- Review each chunk independently through the full pipeline
  (structured output → raw fallback → focused retry)
- Merge findings from all chunks with deduplication
- Single files exceeding chunk limit are truncated with notice

Before: skyflo#135 (45 files, 627K diff) → 0 findings, 3.5 min
After:  skyflo#135 → 44 findings (2 severe, 12 investigate), 7 min
Small PRs unchanged: skyflo#133 → 2 findings, 11s
README.md:
- Added full CLI Commands section (review, ask, traces, serve)
- Added --expert mode documentation with coverage details
- Added smart prompting explanation (compact vs comprehensive)
- Added diff chunking and file filtering description
- Added severity table with emoji badges
- Added RLM reasoning-only mode note
- Added OPENAI_BASE_URL and MAX_LLM_CALLS to config table
- Updated Development section with git clone URL and test count
- Marked Phase 1 as complete in roadmap

SETUP.md:
- Fixed GitHub Action reference: openreview/action@v1 → deuex-solutions/OpenReview@v1
- Fixed clone URL to correct org

REVIEW.md:
- Removed references to unimplemented Impact Analysis module
- Added diff chunking and file filtering conventions
- Added structured output preference guideline

SKILL.md:
- Added traces and serve commands to usage examples
- Fixed ask command to use --url instead of --repo

CONTRIBUTING.md:
- Added Project Guidance Files section linking CLAUDE.md, REVIEW.md, SETUP.md

.env.example:
- Added missing OPENAI_BASE_URL with description and example
New GETTING_STARTED.md with three onboarding paths:
- Path A: Try it now (30 seconds, npx, no install)
- Path B: GitHub Action (5 minutes, automated PR reviews)
- Path C: Local development (10 minutes, full setup)

Includes:
- OS-specific prerequisites (macOS, Linux, Windows/WSL2)
- Step-by-step API key setup with links
- 6-point verification checklist
- How It Works section (Fast mode pipeline, RLM agentic loop)
- Customizing Reviews (instruction files, team learnings)
- 7 troubleshooting scenarios with fixes
- Configuration reference table

Inspired by OpenMetadata's verification-first approach and
AsyncReview's instant-action README pattern.
From competitive analysis with AsyncReview:

Phase 1 (pre-launch):
- --submit flag: post CLI review findings as GitHub PR comment
  Added to: TodoList §14.5.1, Milestones Week 4, PRD §4.1, Feature Spec §15

Phase 2:
- Local directory review (--path): review local code without GitHub PR
  Added to: TodoList Phase 2 §11, Milestones §2.1.1, PRD §7, Feature Spec §15
- GitHub Issue review: support /issues/ URLs alongside /pull/
  Added to: TodoList Phase 2 §12, Milestones §2.1.2, PRD §7, Feature Spec §15
When --submit is set:
1. Review runs normally (findings printed to stdout)
2. CommentPoster.postReview() posts inline comments on specific lines
3. CommentPoster.postSummaryComment() posts severity table summary
4. Confirmation: "Review posted on PR #X (N findings)"

Skips posting inline comments if 0 findings (still posts summary).
Requires GITHUB_TOKEN or GITHUB_PAT with write access.

Closes gap with AsyncReview's --submit flag functionality.
Progress docs:
- TodoList: §14.5.1 --submit marked complete with E2E results
- Milestones: Week 4 updated with --submit, chunked diff, adaptive prompts,
  competitive analysis, GETTING_STARTED.md, and doc audit items

User-facing docs:
- README.md: added --submit to CLI examples and options table
- SETUP.md: added --submit examples including combined flags
- SKILL.md: added --submit to usage examples
- GETTING_STARTED.md: added --submit to Step 5 review commands
- REVIEW.md: added CLI conventions section (stderr/stdout separation,
  --submit behavior, replace-not-duplicate, auth error guidance),
  updated error handling with fallback chain, updated security with
  deno run flags and auth scheme detection
- CONTRIBUTING.md: added Manual E2E Testing section with --submit
  and QA suite commands
safayavatsal and others added 26 commits March 24, 2026 20:06
- TodoList: mark sections 6-9 (RLM, Trace, Chat, Learnings) as complete
  based on codebase audit confirming full implementations
- Remove Impact Analysis from all progress docs (moved to separate product)
- Update status lines across Milestones, PRD, Feature Spec, TodoList
Introduces a new `@openreview/service` workspace package that runs OpenReview
as a long-lived self-hosted service instead of a CLI/Action invocation.
Architecture:
- `web` process: Express HTTP server that verifies the X-Hub-Signature-256
  HMAC, routes pull_request / issue_comment / pull_request_review_comment
  events to handlers, and enqueues jobs onto a BullMQ queue. Responds to
  GitHub in <100ms.
- `worker` process: BullMQ worker that consumes jobs and dispatches to
  per-job processors (fast-review, rlm-review, chat, learnings-list,
  learnings-forget) which delegate to @openreview/core.
- Redis backs the queue for retries, backoff, idempotency, and crash safety.
Highlights:
- HMAC verification (timing-safe), raw-body capture for signature checks.
- Idempotent enqueue keyed on GitHub delivery id + repo + PR.
- Zod-validated env config, layered loading (service/.env then repo-root .env).
- Pino structured logging.
- /health and /ready endpoints for liveness/readiness probes.
- Multi-stage Dockerfile builds both web and worker images.
- 21 unit tests for webhook verification + event routing.
Also bundled (required to keep `pnpm typecheck` green):
- fix(cli): import FindingSeverity and use it as the Map key type in
  cli/src/formatter.ts so grouping by severity typechecks.
- chore: ignore local .pnpm-store/ directory.
- chore(vitest): include service/**/*.test.ts in the test glob.
- chore(workspace): register the new service package in pnpm-workspace.yaml.
Verification (all green on Node 23 locally; CI matrix is Node 20 & 22):
- pnpm install --frozen-lockfile: ok
- pnpm lint: 0 errors
- pnpm -r typecheck: ok
- pnpm -r build: ok
- pnpm test: 342 / 342 passing
feat(service): add self-hosted GitHub webhook service for PR reviews
feat: add coverage and unit test case service
OpenReview now drives the coverage-service end-to-end: it registers the
repo, kicks off coverage analysis, polls for completion, and opens a
stacked PR carrying the LLM-generated unit tests against the original
feature branch — plus a coverage-delta summary comment on the source PR.
Both webhook-driven and curl-driven entrypoints are supported, with
retry-resume so a worker restart doesn't re-trigger an in-flight run.

Highlights:
- New COVERAGE_SERVICE_* config (URL, optional API key, branch prefix,
  poll cadence, repo defaults) replacing the prior TEST_WRITER_* stubs.
- New coverage-analysis job kind + processor. Persists prRunId via
  job.updateData so retries resume polling instead of restarting work.
- New service/src/dispatch/coverage/ — typed Zod client and pure
  markdown formatters for the PR comment and stacked-PR body
  (includes a per-file test/target/status table).
- New service/src/github/pr-author.ts — blob/tree/commit/ref/PR
  helper used to open the stacked PR programmatically.
- New POST /coverage-runs/trigger endpoint (service/src/routes/
  coverage-trigger.ts) — curl-driven entrypoint that lets us run the
  pipeline on repos without a webhook configured.
- Webhook handler enqueues the coverage-analysis job alongside
  review-fast when COVERAGE_SERVICE_ENABLED=true.
- BullMQ job IDs no longer use ':' (forbidden by recent BullMQ); we now
  use '~' as the separator and '-' inside delivery IDs.
- coverage-service: POST /repositories is now idempotent (upsert) so
  re-registering the same githubRepo returns the existing row instead
  of failing with a Prisma unique-constraint 500.
- Three pre-existing shellQuote functions in coverage-service/worker
  had no-useless-escape lint errors blocking the build; fixed.
- README rewrite of the Coverage Service section: architecture diagram,
  prerequisites with GitHub PAT scope guidance, full curl sequence,
  webhook alternative, response-field reference, and a troubleshooting
  table covering every failure mode we hit during integration.

Verified end-to-end on Kenil27/band#3#4.

Test/lint/build status:
- 376/376 tests passing (30 suites, including new tests for client,
  summary, processor, trigger route, and pr-author).
- Typecheck clean across all 7 packages.
- Lint: 0 new errors. The 167 errors reported on this branch are all
  pre-existing on main in coverage-service (NestJS DI requires the
  runtime-class imports that import-x/consistent-type-imports wants to
  strip; needs its own cleanup PR).

Co-authored-by: Cursor <cursoragent@cursor.com>
…egration

feat(service): wire coverage-service for automated stacked test PRs
feat: enhance coverage-service integration with database generation s…
feat: enhance PRAuthor to manage branch existence
feat: implement support for OpenReview stacked test PRs with skip marker
feat: add test file preparation and path resolution utilities
@tarunpandey23
tarunpandey23 requested a review from Kenil27 June 23, 2026 11:07
@tarunpandey23 tarunpandey23 changed the title add deployment related files add deployment related files (DO NOT MERGE) Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants