The studies below observe continuation, delivery, and validation behavior under different tasks, model settings, and measurement rules. Each needs to be read on its own terms. The current evidence does not establish a universal LoopX gain.
+Start with recovery and regression across long-horizon domains, then use three software-engineering studies to examine continuation, delivery, and validation. The four studies differ in tasks, model settings, and measurement rules. Each needs to be read on its own terms; the current evidence does not establish a universal LoopX gain.
+ +GPT-5.6 Sol / max · 46 matched tasks, five execution mechanisms · one effective run per task-arm cell
+What LHTB tests: Long-Horizon Terminal-Bench places an agent in a stateful container and asks it to sustain work across hundreds of dependent terminal actions. Its official introduction uses nine categories: software and reverse engineering; scientific computing and simulation; earth, climate, and energy; multimodal and imaging analysis; research reproduction and ML; systems, performance, and security; interactive games; APEX professional workflows; and logic and constraint puzzles.
+Tasks include migrating an old framework, recovering data from scientific figures, reproducing paper experiments, handling investment-banking or legal matters, playing 2048 turn by turn, and searching for puzzle solutions. Nine categories and representative tasks →
+How it scores: hidden verifiers check final artifacts or replayable outcomes and award continuous reward from 0 to 1. This study counts reward ≥ 0.95 as solved, keeping partial progress distinct from full acceptance.
+| Execution mechanism | Mean reward | Solved / 46 |
|---|---|---|
| Plain | 0.4218 | 7 |
| Native Goal | 0.4475 | 4 |
| LoopX SSH-Goal | 0.4678 | 6 |
| Legacy Heartbeat | 0.4794 | 7 |
| New Heartbeat | 0.4948 | 7 |
Observation: New Heartbeat has the highest mean reward, 0.0154 above Legacy Heartbeat. Its strict solve count matches Plain and Legacy at 7/46. Higher average progress has not translated into more fully solved tasks.
+Insight: each new Heartbeat wake starts a fresh executor, recovering progress from the workspace, registry, and Todos. The brief reports a consulting task resumed at S27 after interruption, alongside a DuckDB regression where later optimization broke correctness. The next questions are whether externalized state reliably enables recovery and whether checkpoints and rollback preserve working results.
+Boundary: transport, session lifetime, LoopX version, and replanning change together. Effective runs include replacement trials, some with longer time limits. There are no repeated seeds, and cost telemetry differs, so this does not isolate a mechanism or establish an equal-budget efficiency advantage.
+Full study: five mechanisms, gains and losses, and the 46-task matrix →
+Full study: requirement coverage, behavior cases, and the long-duration slice →
The current signal is that useful continuation, reliable delivery, and counterexample-driven repair deserve further study.
-The studies offer local positive observations while exposing cost, failure, and attribution problems. Matched budgets and repeated experiments are needed to determine which gains reproduce reliably.
+Recover progress, preserve working results, and check whether continuation closes acceptance gaps.
+The four studies offer local positive observations while exposing regressions, cost, and attribution problems. Matched budgets and repeated experiments should examine recovery, rollback, delivery, and counterexample-driven repair separately.
Withdrawn SSH Goal and Codex CLI scores and conclusions from SWE-Marathon and DeepSWE × Sol are not used in these comparisons.
The technical content is derived only from public repository material. Source and data links are pinned to the revisions read for this article; historical experiments retain their own versions. This page introduces no new experimental results.
三项研究分别观察续跑、交付和验证行为。任务、模型设置与统计口径不同，应分别理解；现有结果还不足以概括 LoopX 的普遍增益。
+先看跨领域长程任务中的恢复与回退，再用三项软件工程研究补充续跑、交付和验证行为。四项研究的任务、模型设置与统计口径不同，应分别理解；现有结果还不足以概括 LoopX 的普遍增益。
+ +GPT-5.6 Sol / max · 46 个匹配任务、五种执行机制 · 每个任务 × 实验臂一条有效运行
+LHTB 考什么：Long-Horizon Terminal-Bench 把 Agent 放进有状态的容器环境，要求它跨数百个相互依赖的终端动作完成工作。官方介绍页分为九类：软件与逆向工程、科学计算与仿真、地球/气候/能源、多模态与图像分析、研究复现与机器学习、系统/性能/安全、交互式游戏、APEX 专业工作流、逻辑与约束谜题。
+具体会遇到旧版框架迁移、从科学图还原数据、复现论文实验、投行或法律事项，以及逐步玩 2048、搜索谜题解。九类任务与代表例子 →
+怎样评分：隐藏验证器检查最终产物或可重放结果，给出 0–1 的连续 Reward；本研究以 Reward ≥ 0.95 计为通过。因此，部分进展与完整验收可以分开观察。
+| 执行机制 | 平均 Reward | 通过 / 46 |
|---|---|---|
| Plain | 0.4218 | 7 |
| 原生 Goal | 0.4475 | 4 |
| LoopX SSH-Goal | 0.4678 | 6 |
| 旧版 Heartbeat | 0.4794 | 7 |
| 新版 Heartbeat | 0.4948 | 7 |
观察：新版 Heartbeat 平均 Reward 最高，比旧版高 0.0154；严格通过数与 Plain、旧版相同，均为 7/46。平均进展增加，尚未转化为更多完全通过的任务。
+Insight：新版每次唤醒启动新的执行器，依靠工作区、Registry 与 Todo 恢复进度。简报记录了咨询任务中断后从 S27 接续的案例，也记录了 DuckDB 后续优化破坏正确性的退步。值得继续验证的是：外部化状态能否稳定支持恢复，以及检查点和回滚能否保住已有成果。
+范围：新旧同时改变了传输、会话生命周期、LoopX 版本和 replan 策略；有效运行包含替代 trial，部分替代运行时限更长。没有重复 seed，成本记录口径也不同，不能据此归因单个机制或宣称同预算效率优势。
+ +当前的信号是：有效续跑、可靠交付和反例驱动的修复值得继续研究。
-三项研究提供了局部正向观察，也暴露了成本、失败与归因问题。下一步需要在匹配预算和重复实验中，判断哪些收益能够稳定复现。
+让进度可恢复，让已有成果可保留，再检查续跑是否补上了验收缺口。
+四项研究提供了局部正向观察，也暴露了回退、成本和归因问题。下一步需要在匹配预算和重复实验中，分别验证恢复、回滚、交付与反例驱动修复的收益。
SWE-Marathon 与 DeepSWE × Sol 已撤回的 SSH Goal、Codex CLI 成绩和结论均不用于这里的比较。
技术内容仅据公开仓库材料整理。源码与数据引用固定到本次读取修订；历史实验使用其自身版本。本页没有新增实验结果。
{c.categoriesBody} {c.categoriesSource} · {c.categoriesExamplesSource}
+| {label} | )}|
|---|---|
| {category} | {examples} |
{c.taskExample}
+{c.benchmarkSetupNote}
+