docs(site): introduce LHTB tasks and lead application benchmarks with the study - #4705
Conversation
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
huangruiteng
left a comment
There was a problem hiding this comment.
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
Reviewed head: cff201ec9f05144619f3b5c477b202b5ab247263.
动机
应用场景文章需要先说明 LHTB 考什么,再解释五臂结果;研究页需要给出类别、代表工作与评分方式,静态发布也必须能直接访问研究页。#4703 已合入,本次完整差异为 7 个文件、116 行新增、9 行删除,覆盖这一阅读路径。
改动思路
复用现有 LhtbBrief、双语 copy、表格样式与 copyPublicSiteRoutes。没有引入新页面框架、运行状态或执行策略。只加链接无法解决类别解释与静态路由缺失,当前改动落在已有组件和导出器中,范围适当。
入口仍是应用场景文章 → /benchmarks/lhtb/?lang=zh#categories → main.tsx 的现有页面选择 → LhtbBrief。公开聚合数据和双语内容负责展示;不参与 Agent 调度、权限或实验评分。
具体改动
- 两个语言版本的文章都以 LHTB 为首项,明确五臂平均 Reward 与严格通过数不同;新版平均分最高,但与 Plain、旧版同为 7/46。保留替代运行、预算及归因限制,并保留另外三项研究。
LhtbBrief在现有 benchmark section 中通过categories.map渲染九类任务表,包含可直达的categories锚点、官方来源、UNISON 产物验收例子以及历史评测设置说明。lhtb-copy.json提供中英文内容,CSS 处理窄屏换行和锚点留白。copyPublicSiteRoutes将benchmarks/lhtb加入既有目录复制循环;现有 bundle smoke 增加相应文件断言。路由选择仍由原有main.tsx负责。
正向路径已检查:文章链接进入研究页、九行类别展示、中英文切换、桌面和 390px 布局、直接 fragment 定位均正常。
反向检查针对最容易漏掉的问题:开发服务器的 SPA 回退可能掩盖静态站点缺页。在同一站点输入下实际运行主干版本导出器和当前 head 导出器,两者均构建成功;独立静态路由断言在旧导出器输出上失败(exit 1),在当前输出上通过(exit 0),证明修复覆盖真实发布边界。
对主干的风险
未发现阻塞问题。影响限于公开内容展示与静态导出,不修改实验数据、评分、权限或控制面。两种语言、既有路由及相对链接由完整 bundle smoke 覆盖。
验证包括完整 frontstage share-bundle smoke(构建、路由、引用与公开/私有边界检查)、bilingual blog smoke、五臂 46 题均分和通过数复核,以及上述浏览器与基线/当前导出比较。此前 3fa0a7982 的完整验证可以复用:本次同步主干仅增加八个无关 RFC 文档,产品代码及调用方未变;当前 cff201ec9 又完成实际导出与 TypeScript/Vite 构建。git diff --check 通过。该组合覆盖本次改动的风险,作为仓库要求的等效风险验证集。
没有必需的本地检查失败或跳过,没有手动 hold。按 owner 明确指令,不查询或等待远端 CI。剩余限制是历史实验没有重复 seed,替代运行和成本口径不同;正文已披露,不能据此宣称普遍收益或同预算效率优势。生产站点部署不在本次验证范围内。回滚只需撤回本次展示与导出改动,无数据迁移。
我的整体评价
APPROVE。需求在现有页面与导出边界中完整实现,复杂度与收益相称。已按 policy revision 6 完成当前 head 的证据记录并通过 pr-review --check-result 一致性检查;该检查仅验证记录一致性,实际证据来自上述源码、构建、输出与页面检查。Owner 已明确授权快速自合并、不等 CI,可在当前提交的 merge-readiness 通过后执行。
English verdict: APPROVE - cff201e. No blocking findings: the bilingual LHTB introduction and static reading path are complete. Full bundle and bilingual smokes, aggregate verification, browser checks, and baseline/head export regression checks passed. CI waiting is waived by the owner; no scoring or runtime authority changes.
The application-scenarios article now leads with LHTB’s 46-task study and explains the benchmark before interpreting the results. The bilingual research page adds nine task categories, representative work, an artifact-grading example, and the distinction between official evaluation settings and this five-arm study.
The research brief dependency #4703 has merged. This PR adds the bilingual article integration and task introduction, and materializes
/benchmarks/lhtb/in the static export with a check in the existing bundle smoke.Mean reward stays separate from strict solves: New Heartbeat has the highest mean, while Plain, Legacy, and New Heartbeat each solve 7/46. Replacement-run, budget, and attribution limits remain explicit. The three existing supporting studies retain their coverage. No scoring or experiment data changed.
Validation: recomputed all five mean rewards and solve counts from the public 46-task aggregate; bilingual blog smoke; complete frontstage share-bundle smoke; desktop and 390px browser checks in both languages, category fragment entry and article-to-study navigation. After syncing main, both the baseline exporter and final-head exporter were executed with the same site fixture: the direct LHTB route assertion fails on the old exporter and passes on the final head. The final head includes successful TypeScript/Vite builds. CI waiting is waived by the owner for this merge.