Skip to content

docs(site): introduce LHTB tasks and lead application benchmarks with the study - #4705

Merged
huangruiteng merged 4 commits into
mainfrom
codex/lhtb-application-scenarios-20260918
Sep 18, 2026
Merged

huangruiteng merged 4 commits into
mainfrom
codex/lhtb-application-scenarios-20260918

Conversation

@huangruiteng

@huangruiteng huangruiteng commented Sep 18, 2026 •

Copy link
Copy Markdown
Collaborator

The application-scenarios article now leads with LHTB’s 46-task study and explains the benchmark before interpreting the results. The bilingual research page adds nine task categories, representative work, an artifact-grading example, and the distinction between official evaluation settings and this five-arm study.

The research brief dependency #4703 has merged. This PR adds the bilingual article integration and task introduction, and materializes /benchmarks/lhtb/ in the static export with a check in the existing bundle smoke.

Mean reward stays separate from strict solves: New Heartbeat has the highest mean, while Plain, Legacy, and New Heartbeat each solve 7/46. Replacement-run, budget, and attribution limits remain explicit. The three existing supporting studies retain their coverage. No scoring or experiment data changed.

Validation: recomputed all five mean rewards and solve counts from the public 46-task aggregate; bilingual blog smoke; complete frontstage share-bundle smoke; desktop and 390px browser checks in both languages, category fragment entry and article-to-study navigation. After syncing main, both the baseline exporter and final-head exporter were executed with the same site fixture: the direct LHTB route assertion fails on the old exporter and passes on the final head. The final head includes successful TypeScript/Vite builds. CI waiting is waived by the owner for this merge.

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
@huangruiteng
huangruiteng marked this pull request as ready for review September 18, 2026 16:06

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Reviewed head: cff201ec9f05144619f3b5c477b202b5ab247263.

动机

应用场景文章需要先说明 LHTB 考什么,再解释五臂结果;研究页需要给出类别、代表工作与评分方式,静态发布也必须能直接访问研究页。#4703 已合入,本次完整差异为 7 个文件、116 行新增、9 行删除,覆盖这一阅读路径。

改动思路

复用现有 LhtbBrief、双语 copy、表格样式与 copyPublicSiteRoutes。没有引入新页面框架、运行状态或执行策略。只加链接无法解决类别解释与静态路由缺失,当前改动落在已有组件和导出器中,范围适当。

入口仍是应用场景文章 → /benchmarks/lhtb/?lang=zh#categories → main.tsx 的现有页面选择 → LhtbBrief。公开聚合数据和双语内容负责展示;不参与 Agent 调度、权限或实验评分。

具体改动

  • 两个语言版本的文章都以 LHTB 为首项,明确五臂平均 Reward 与严格通过数不同;新版平均分最高,但与 Plain、旧版同为 7/46。保留替代运行、预算及归因限制,并保留另外三项研究。
  • LhtbBrief 在现有 benchmark section 中通过 categories.map 渲染九类任务表,包含可直达的 categories 锚点、官方来源、UNISON 产物验收例子以及历史评测设置说明。lhtb-copy.json 提供中英文内容,CSS 处理窄屏换行和锚点留白。
  • copyPublicSiteRoutes 将 benchmarks/lhtb 加入既有目录复制循环;现有 bundle smoke 增加相应文件断言。路由选择仍由原有 main.tsx 负责。

正向路径已检查:文章链接进入研究页、九行类别展示、中英文切换、桌面和 390px 布局、直接 fragment 定位均正常。

反向检查针对最容易漏掉的问题:开发服务器的 SPA 回退可能掩盖静态站点缺页。在同一站点输入下实际运行主干版本导出器和当前 head 导出器,两者均构建成功;独立静态路由断言在旧导出器输出上失败(exit 1),在当前输出上通过(exit 0),证明修复覆盖真实发布边界。

对主干的风险

未发现阻塞问题。影响限于公开内容展示与静态导出,不修改实验数据、评分、权限或控制面。两种语言、既有路由及相对链接由完整 bundle smoke 覆盖。

验证包括完整 frontstage share-bundle smoke(构建、路由、引用与公开/私有边界检查)、bilingual blog smoke、五臂 46 题均分和通过数复核,以及上述浏览器与基线/当前导出比较。此前 3fa0a7982 的完整验证可以复用:本次同步主干仅增加八个无关 RFC 文档,产品代码及调用方未变;当前 cff201ec9 又完成实际导出与 TypeScript/Vite 构建。git diff --check 通过。该组合覆盖本次改动的风险,作为仓库要求的等效风险验证集。

没有必需的本地检查失败或跳过,没有手动 hold。按 owner 明确指令,不查询或等待远端 CI。剩余限制是历史实验没有重复 seed,替代运行和成本口径不同;正文已披露,不能据此宣称普遍收益或同预算效率优势。生产站点部署不在本次验证范围内。回滚只需撤回本次展示与导出改动,无数据迁移。

我的整体评价

APPROVE。需求在现有页面与导出边界中完整实现,复杂度与收益相称。已按 policy revision 6 完成当前 head 的证据记录并通过 pr-review --check-result 一致性检查;该检查仅验证记录一致性,实际证据来自上述源码、构建、输出与页面检查。Owner 已明确授权快速自合并、不等 CI,可在当前提交的 merge-readiness 通过后执行。

English verdict: APPROVE - cff201e. No blocking findings: the bilingual LHTB introduction and static reading path are complete. Full bundle and bilingual smokes, aggregate verification, browser checks, and baseline/head export regression checks passed. CI waiting is waived by the owner; no scoring or runtime authority changes.

@huangruiteng
huangruiteng merged commit 45abe1d into main Sep 18, 2026
18 of 19 checks passed
@huangruiteng
huangruiteng deleted the codex/lhtb-application-scenarios-20260918 branch September 18, 2026 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant