Skip to content

review(bandcamp): add deterministic task verifiers - #79

Open
reacher-z wants to merge 2 commits into
aiming-lab:mainfrom
reacher-z:review/bandcamp-verifiers
Open

reacher-z wants to merge 2 commits into
aiming-lab:mainfrom
reacher-z:review/bandcamp-verifiers

Conversation

@reacher-z

Copy link
Copy Markdown
Contributor

Reviewer contribution

This builds on #36 while preserving XuanRui LI's original Bandcamp commit and authorship. The reviewer changes are isolated in a second commit.

What changed

  • rebased the Bandcamp mirror onto current main, retaining Merriam-Webster and assigning Bandcamp the next available port 40016
  • added deterministic verifiers for all 18 tasks, with required-navigation checks to reject prior-knowledge shortcuts
  • added SQLite before/after verification for wishlist, cart, checkout, and profile-update tasks
  • supplied exact seeded credentials for each standalone authenticated task
  • added explicit judge_rubric and verifier_path contracts without leaking an answer field
  • corrected site registration, Docker exposure, startup/health counts, and README port documentation

Verification

  • python3 -m unittest discover -s sites/bandcamp/verify -p 'test_*.py' -v: 7 test groups pass
  • positive, no-op, wrong-answer, and prior-knowledge cases cover all 18 tasks
  • all four stateful false-claim cases fail without the corresponding database transition
  • every verifier wrapper was exercised for both pass and fail exit codes
  • py_compile, JSONL parsing, shell syntax, git diff --check, and asset checks pass
  • Docker runtime health reports 17/17 mirrors alive
  • wishlist, cart, checkout, and profile flows were executed through real HTTP and passed live SQLite before/after checks
  • browser QA covered the homepage, release/merch pages, login, cart, checkout, orders, and profile editing

Asset dependency

The companion asset submission remains ChilleD/WebHarbor discussion #26. It should be merged/pinned before this environment is treated as self-contained.

This branch independently uses the next free port on current main. If the Cookpad reviewer PR #78 lands first, I will rebase this branch and move Bandcamp to 40017 so both environments remain conflict-free.

@TabsPhasers

Copy link
Copy Markdown
Contributor

🔒 Review lock(补登记):我将基于当前 upstream main 继续审核并修复本 PR,保留 #36 原作者与 #79 既有 reviewer 的贡献归属;按当前 18 个 Task 逐项执行 asset、build/health/reset、真实浏览器、verifier/rubric 与独立纯视觉轨迹审计,并在后续 Review PR 中关联结果。lock 有效期间请避免重复 Review。

@TabsPhasers

TabsPhasers commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

后续 Review PR 已创建:#104https://github.com/aiming-lab/WebHarbor/pull/104)。

该 PR 保留 @reacher-z 的贡献归属,基于当前 main 接管原贡献,并在那里跟踪剩余 review/asset blocker:

后续 Review 请转到 #104

jackjin1997 pushed a commit to jackjin1997/WebHarbor that referenced this pull request Sep 14, 2026
Points .assets-revision at ChilleD/WebHarbor discussions/79 so this branch
fetches and builds without waiting for that PR to merge. The commit is
upstream's current pin plus phet_simulations.tar.gz; every other archive is
byte-identical, verified by comparing the file OIDs of both trees.

Temporary. Repin to the merge commit once aiming-lab#79 lands.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jackjin1997 pushed a commit to jackjin1997/WebHarbor that referenced this pull request Sep 14, 2026
…sets

Upstream merged fedex, webmd_doctor, healthline, kaggle and nvidia while this
branch was in review, so take the next free slot.

.assets-revision points at ChilleD/WebHarbor discussions/86 so this branch
fetches and builds without waiting for that PR to merge. That commit is
upstream's current pin b7e605c plus phet_simulations.tar.gz, with every other
archive byte-identical - checked by comparing the file OIDs of both trees, which
matters because the earlier asset PR aiming-lab#79 predated the NVIDIA repin and would
have regressed those images. Temporary; repin to the merge commit once aiming-lab#86 lands.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants