Skip to content

Correct evaluator and canonical replay contracts - #4

Closed
lanyue-llk wants to merge 6 commits into
lex-mainfrom
codex/evaluator-replay-contracts
Closed

lanyue-llk wants to merge 6 commits into
lex-mainfrom
codex/evaluator-replay-contracts

Conversation

@lanyue-llk

Copy link
Copy Markdown

Summary

  • fix network-event evaluation for dynamic placeholders, ordered candidate streams, explicit last-event semantics, and typed array schemas
  • correct canonical WebArena task contracts that previously described the wrong endpoint, payload, navigation mode, group set, repository, or route direction
  • add positive and negative behavior controls, including trailing wrong requests and forward versus reverse route checks

Ownership

These changes belong in the lexmount/webarena-verified fork because they correct reusable evaluator and canonical dataset contracts. Browser replay execution, Rainbond deployment bindings, and run evidence remain in browser-eval.

Verification

  • evaluator, migration, API, and dataset integrity: 1,862 passed
  • complete locally runnable suite: 5,739 passed, 6,391 skipped, 2 deselected
  • Ruff, ty, and diff checks passed

External-service and Docker-dependent tests were excluded from the locally runnable suite.

@lanyue-llk

Copy link
Copy Markdown
Author

已将本 PR 的完整变更以普通 merge commit 收敛到 #3;#3 是唯一保留的 fork PR。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant