Review: validate IMDb mirror, sourced seed and task grading - #89
Merged
Merged
Conversation
Adds sites/imdb/ — a Flask mirror of imdb.com built per the
.claude/skills/ six-phase contributor pipeline (clone-website,
design-tasks, evolve-env, harden-env, seed-database).
Catalog (scraped from imdb.com via Playwright, gitignored in
sites/imdb/scraped_data/; shipped as instance_seed/imdb.db +
static/images/ via HF dataset):
- 392 titles (Top 250 + Top TV 250 + Most Popular + Box Office)
- 3796 persons (cast / crew with real bios + headshots)
- 4280 real posters + headshots
- 19 canonical IMDb genres
Coverage:
- 22 routes: homepage / title detail / fullcredits / reviews /
rate / watchlist toggle / person detail / scored search /
advanced search (genre / year / rating / sort) /
4 charts (Top 250 / Top TV / Most Popular / Box Office) /
genre browse / news / auth (login / register / account)
- scored token-overlap search (not strict-AND), 18 stop words
- 4 benchmark users alice.j / bob.c / carol.d / david.k
@ test.com (password TestPass123!) — pre-seeded with
4-item watchlists, 3-5 ratings, one written review each
- 12 seeded news items + 12 featured reviews (high
helpful-count) across major titles
- 18 benchmark tasks in tasks.jsonl across 6 functional areas:
chart browse, title detail, person filmography, advanced
search, genre browse, user state — including 4 hard tasks
(>= 5 steps) and 2 disambiguation tasks (bob's SF/fantasy
watchlist subset; carol's crime film subset)
Data integrity:
- canonical-URL guard skips 65 IMDb-redirected tt_ids
(some unknown tt_ids returned the wrong page)
- garbage filter skips 1854 names that hit IMDb's 403/
error fallback during concurrent scraping
- html.unescape on all string fields; hero h1 preferred
over ld.name (which is sometimes original-language)
- lowercase substring match for box-office data-testid keys
HF asset PR:
https://huggingface.co/datasets/ChilleD/WebHarbor/discussions/23
Contributor
Author
Independent blind execution review
This was an execution-completion judgment only. The blind reviewer did not inspect or certify source fidelity, deterministic verifier correctness, runner independence, repository history, PR discussion, or prior reviews, and did not replay the runs live. The frozen run records predate the latest presentation-only candidate changes; their reuse basis and the current-candidate guided/deterministic validation are tracked separately and were intentionally outside this blind pass. The structured first-pass result was frozen and hashed before comparison. The Review PR remains Draft; reconciliation and final readiness have not started. |
Refresh review-reports/PR-33-IMDB.md to the fixed candidate (code 1c8a1eb, HF PR aiming-lab#57 revision f9ddfd25): drop the superseded HF pin and Owner-pending text, record the formal 20-task blind review and reconciliation, the fixed-candidate engineering and guided impact regression, Owner visual acceptance, known limitations and the maintainer handoff order. Add public summaries and current-candidate screenshots under review-reports/assets/pr33-imdb/ and mark the historical probe summaries as superseded. Docs-only: no application, task, verifier, seed or asset bytes change.
jackjin1997
marked this pull request as ready for review
September 12, 2026 15:53
…ex 30 / 40030 after aiming-lab#116 took index 29 Conflict resolution against current main (aiming-lab#116 merged: nvidia index 28, UC Berkeley index 29, 30 sites): - websyn_start.sh, control_server.py: keep main's 30-site order and append imdb as index 30 (container port 40030). - Dockerfile: keep main's build steps unchanged (the PR adds no build step for imdb, whose seed ships in the HF archive) and set the canonical header/EXPOSE lines: "31 Flask mirror sites" / "EXPOSE 8101 40000-40030". The merged file differs from main's by exactly those two lines. - sites/imdb/tasks.jsonl: 20 web URLs 40024 -> 40030. No task semantics changed (ques/judge_rubric/verifier_path untouched); sites/imdb/app.py keeps its PORT-env default of 5000, so no other file in the site referenced 40024. - nvidia (40028) and berkeley (40029) task URLs are unchanged from main. - .assets-revision: kept as the PR's value (revision: f9ddfd2596229f2610418d57fc88c3051e1056bb, the head commit of HF dataset PR aiming-lab#57). This pin is NOT usable after this merge: refs/pr/57 carries 26 archives and has no archive for fedex, webmd_doctor, healthline, kaggle, nvidia or berkeley, and c32018ca (the pin main uses) has no imdb.tar.gz. The pin will be replaced with the merged HF dataset commit once HF PR aiming-lab#57 is merged; this commit does not change it. - Documentation (README.md, AGENTS.md, CLAUDE.md, CONTRIBUTING.md) took main's side here; the 31-site / 40000-40030 statements follow in the next commit.
With IMDb appended as index 30 the registry has 31 sites. Current-state statements in README.md, AGENTS.md, CLAUDE.md, CONTRIBUTING.md, agent_demo/README.md and the five .claude/skills files now say 31 sites, container ports 40000-40030 and alt ports 41000-41030. README.md's review-candidate section gains the IMDb row (index 30, container port 40030, local review host port 48030) and the mirror list. review-reports/** is left untouched as historical record (it documents the earlier 25-site registry, which is correct for its own revision).
… of flat purple blocks
…ex 33 / 40033 after aiming-lab#124 appended bh_photo, accuweather and gov_uk
Contributor
|
Thanks for your contribution! @jackjin1997 @hqhq1025 |
Raibows
added a commit
to jackjin1997/WebHarbor
that referenced
this pull request
Sep 17, 2026
…x 34 / 40034 after aiming-lab#89 added imdb main now ends with imdb at index 33 / port 40033 (PR aiming-lab#89 merged), so the NBA mirror is appended as the 35th entry at index 34 / container port 40034. The asset revision keeps main's merged dataset-main pin (it carries imdb, gov_uk and every pre-existing archive) and adds site.nba for the reviewed NBA candidate commit from HF dataset PR aiming-lab#88, which is still open; accuweather keeps its existing per-site pin.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This reviewer branch preserves @hqhq1025's IMDb contribution and commit history from #33, corrects sourced profile/title metadata and interactive flows, adds deterministic grading for 20 tasks, and expands the homepage and featured editorial path from sourced captures. It integrates
mainat36004932bdf82afbe36dc14e00f66841eccf9946; IMDb is site 25 at port 40024.Ready for maintainer review — nothing is merged. Implementation fixed point: code
1c8a1eba19c084e202ecbab5977280701b9068c2. The current headfba88dd6385ee4029cd49851a86f642da6b3d774is a docs-only commit on top of it (onlyreview-reports/changes; application, task, verifier, registry, Dockerfile and.assets-revisiontrees are identical). Assets: open HF PR #57 revisionf9ddfd2596229f2610418d57fc88c3051e1056bb;imdb.tar.gzis 39,111,545 bytes, SHA2560663d30fe90ed2a0659dfe78ecf7ab1718970f565a64c6855a2c6aaadc606513; seed SHA2569d843c5388ecbc0d5265ac316cdaee915f8b203121b5a5330d207af5fb6206b0. Onlyimdb.tar.gzchanges in that PR; it reports no conflicting files.What was verified on the fixed candidate
/feature/featured-today-1HTTP 200, no horizontal overflow, 17/17 sourced entries with local media, 17/17 detail routes 200.claude-opus-5, fresh restricted session, no network) over a hash-verified 826-file packet covering all 20 sealed canonical runs: 20 PASS / 0 FAIL (comment). Reconciliation against the deterministic results: 0 differences, 0 substantive findings; common-PASS spot checks of tasks 9/15/16/17 against frozen DOM and full-table DB diffs confirmed the exact single-row changes.ebe92f0/HF4d5709e; the rest on r250bcce5/HFe70f49d) and are disclosed as such, not relabeled. Reuse basis:tasks.jsonlandsites/imdb/verifyare byte-identical since the expansion freeze, all ten business tables are logically identical between the execution seed and the current seed (onlyhome_featureswas added), and the later code changes are confined to homepage/feature/news/session routes.Retained rulings: Task 19 is graded from displayed amounts (
30.1M / 52.0M→ 57.9%; raw values would give 57.8% with the same winner). Task 26 completes its registration/queue/sign-out/re-login flow with exactly one new user and two Watchlist rows, but its run-start and seal-time provenance metadata disagree, so it is not claimed as a reliable independent-exploration attestation.Self-contained report: review-reports/PR-33-IMDB.md at
fba88ddwith public summaries (formal-blind-review-1c8a1eb-summary.json,reconciliation-1c8a1eb-summary.json,candidate-regression-1c8a1eb-summary.json) and current-candidate screenshots underreview-reports/assets/pr33-imdb/.Maintainer handoff — directly mergeable now: no. The code is MERGEABLE/CLEAN against
main, but.assets-revisionpins the open HF PR revision rather than a merged asset-main commit. Suggested order: (1) review and merge HF #57 (adds onlyimdb.tar.gz; asset main has since gained two unrelated archives and the PR reports no conflicts); (2) confirm the merged commit still servesimdb.tar.gzwith the SHA256 above; (3) update.assets-revisionon this branch to that commit, or confirm the immutable PR revision is acceptable — a pin-only change with identical bytes needs onlyfetch_assets.shplus the hash check, not new task runs or blind review; (4) merge this PR (supersedes #33) and close #33 and HF #23 with references so the old seed is not reintroduced. The reviewer will not merge either PR.