Fix final-page event evaluation and five verified task contracts - #1
Merged
Merged
Conversation
lanyue-llk
commented
Sep 17, 2026
lanyue-llk
commented
Sep 17, 2026
lanyue-llk
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
本 PR 面向
lex-main修复网络事件选取和 5 个任务契约，保持 WebArena-Verified 独立于部署工具。它不更新 browser-eval 的已发布分数。网络事件检查默认保留“最后一次文档导航”的规则。任务 676 显式使用
navigation_only=false，只检查最终页面加载后的订单列表 XHR，并通过event_query_params.namespace排除共用接口的通知请求。last_event_only保持开启，清除筛选、离开页面或重新加载后不能沿用旧请求通过验收。任务契约修正：
/a11yproject/a11yproject.com/-/issues?state=opened&label_name[]=bug；列表地址允许有或没有结尾斜线，拒绝具体 issue 详情地址，保留 GraphQL 条件检查。project[initialize_with_readme]。接受明确的[null, "0"]两种形式，拒绝"1"；比较器只在契约显式声明 null 备选值时允许缺值。website_ids[0]=1、customer_group_ids[0]=1。修正字段名，保留促销名称、对象、折扣类型和金额要求；错误客户组仍失败。原 PR 中 600、605、609、625 的 Reddit 路由扩展已撤回：复查的固定环境 HAR 使用
/submit/consoles或/submit，不足以证明新增 gaming/technology 路由与指定论坛 ID 等价。hard 数据集不再改动。验证：通过公开
WebArenaVerified.evaluate_taskAPI 测试有效轨迹和相邻反例；新增导航/表单/null 回归 20 项全部通过。数据集完整性、核心比较器和类型测试共 2308 项通过、4 项原有跳过；修改文件 Ruff 检查通过。未执行 Hugging Face 远程数据集测试或 Docker 集成测试。旧版直接删除导航筛选的实现已被反例证实会产生误判，本 PR 已改为显式、有限的 XHR 模式。上游重复 PR ServiceNow/webarena-verified#46 已关闭。