Skip to content

docs: prioritize steward journeys and small-team golden-query evaluation - #5038

Merged
huangruiteng merged 2 commits into
mainfrom
codex/steward-attention-evaluation
Sep 25, 2026
Merged

huangruiteng merged 2 commits into
mainfrom
codex/steward-attention-evaluation

Conversation

@huangruiteng

@huangruiteng huangruiteng commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Goal And Delivered Outcome

Steward planning prioritized answer presentation before proving that an ordinary request reaches a responsible worker and returns a usable result. This proposal makes creation/connection, responsible dispatch and real small-team coordination explicit P0 outcomes, with portfolio management and distributed scale following at P1/P2.

A public-safe pack defines 17 natural query families, the required initial context, independent acceptance observations, recovery/correction variants and paired outcome/attention/cost measures. The overall roadmap and manager/presentation RFCs now point to the same evaluation targets in both language versions. It reuses R1–R3, G1–G4 and M1–M3/A1–A24; intended base: main.

Broad owner discovery is a P0 default: the registered local inventory and already-authorized connected sources remain searchable beyond the first page. Discovery, evidence access, delegation grants and execution readiness are separate; a narrow recipient allowlist must not imply that no matching Agent exists. Shared audiences retain their own visibility boundary. These are target defaults, not a claim of shipped runtime behavior.

Scope And Continuation

Planning/evaluation scope is complete. Runtime implementation and live golden-query acceptance remain open: no passing product score, new protocol, capability, runner or model call is claimed. Implementation continues through existing creation/Chat, directory/binding, collaboration/artifact and return owners. Small-team acceptance includes parallel joins, peer help, disagreement, independent review and receiver adoption across two real cycles.

Future-facing pass: replace the report-first dependency and stale #4978 proposed status; keep one case/measurement pack rather than duplicate acceptance rules across RFCs. Canonical task planning is maintained outside the public repository. Private conversations and operating state are excluded.

Validation

  • Tested revision: 5c200fa
  • Run state: finished
  • Input classes: public_fixture, authorized_private_read_only
Check kind Result Public-safe evidence / limitation
static passed uv run --extra test python examples/docs-governance-smoke.py; documentation placement and links
static passed uv run --extra test python examples/docs-asset-integrity-smoke.py; 7 existing assets verified
static passed git diff --check and explicit changed-path public-boundary scan; zero public errors. Two unrelated existing Goal-state warnings were not altered
manual passed Reviewed bilingual priority/ownership parity, natural query wording, positive/negative outcomes, public/private separation and existing milestone reuse
static passed Exact-scope change-quality receipt cqr_1505ef9092f41e6de3c0, 9 files, fingerprint 1505ef9092f41e6de3c0fef02d3c4de1784b72033521dd00ed671427e6bc3161; safe fix permitted/not applied, zero repair passes, zero blockers/warnings/advisories
integration prior revision only On prior head 943ab8b, risk-selected premerge: 19 selected checks passed. The latest docs-only refinement reran the focused documentation/boundary checks above; the full selection was not repeated. Initial vocabulary check could not load missing worktree Node dependencies; installed with npm ci --ignore-scripts, then reran the full selection successfully. No source changes were needed
real_entrypoint not_run All live query cases remain unqualified. Private read-only observations informed generalized cases; they do not certify current runtime causes or successful repairs

Coverage: docs-only diff; runtime/frontend/backend execution is unchanged. The proposed 30% reduction in avoidable coordination is a future evaluation target, not a measured improvement. Later cross-host/scale cases require their own frozen fixture packs and authorized resources.

Frontend / Visual Evidence

  • UI impact: none. Markdown evaluation and RFC content only; no product chrome, primary navigation, rendering code or packaged assets changed.
  • Source data: public_fixture. No private screenshots or message bodies are attached.

Type of Change

  • Documentation update

LoopX Area

  • Public docs or presentation surface (README, protocols, dashboard)

Technical Direction

Operator surface and IM integration; R1–R3/G1 supported path first, R3/R4 portfolio management next, R6/G3 then R7/G4 separately qualified.

Shared-authority RFC fixture impact

N/A: no provider, transaction, compatibility or promotion change.

Boundary Checklist

  • No private state, credentials, raw traces, internal links or local machine paths in the diff or PR.
  • No duplicated maintainer-owned benchmark work or live model experiment.
  • Scope limited to the product evaluation plan and owning RFCs.
  • UI impact is none.
  • Every commit includes a DCO sign-off.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

动机

用户需要把“创建、找负责人、真实小团队协作、结果回到原入口”排到同一产品目标里。旧文档缺少一套简短请求和可独立检查的结果标准,容易让目录、收件箱回执或漂亮的回答被误当成完整交付。本 PR 将这些目标明确为 P0,同时保留后续能力的层次。

改动思路

以一份公开安全的 golden-query 集作为评测入口,再让整体 roadmap、管家 RFC、共用对话界面 RFC 双语引用它。它复用既有 R/G/M/A 里程碑和 Todo owner,不创建新协议或状态机。正向路径是冻结场景、运行基线与候选、检查负责人实际执行和原对话回传;失败路径明确保留未验收状态。

具体改动

  • docs/product/use-cases/steward/golden-queries.md 定义 17 类自然请求、P0/P1/P2 分层、两轮协作和独立质量/注意力度量;创建、负责人发现、采用、纠偏、停止、恢复都有反例。
  • 中英 roadmap 把入口与负责人路由、2–3 人真实协作列为 P0;中英管家 RFC 补足宽范围发现、委派准入及真实回传;展示 RFC 强调成果、普通进度、必要决策和首次请求保留。Steward use-case 索引提供直接入口。

关键内容讲解

GQ03 把“找 PR review 负责人”验收到因果检查、实际处理和回传,明确基线 CI 失败不是自动批准。GQ05/GQ11–GQ13 要求接收方使用修订版本、独立复核、处理分歧并完成两轮;一条 inbox ACK 不算协作。宽范围发现与证据读取、委派、执行就绪分别判断,外部受众不能继承主人的可见范围。

对主干的风险

主要风险是把规划误读为现有能力。文件明确标明所有真实案例尚未验收、30% 注意力下降只是预设目标,跨主机和规模场景需要另行冻结资源。完整 diff 无运行时代码和状态迁移;本地文档治理、资源完整性、空白和签署检查通过。未跑真实 Agent,因此不能据此作产品发布结论。

我的整体评价

对长期协作和用户体验的规划都有正向增量:它让后续 PR 必须证明实际执行、采用和回传,也避免为了减少消息数牺牲结果质量。没有阻塞发现。文档切片完整且可独立撤回;残余风险是评测尚未实施,需由既有 R2/R3/A24 owner 用真实入口继续验收。

English verdict: APPROVE - exact head 5c200fa30835fd6f7699a82e848fea58cb5161bb; docs-only P0 evaluation contract, focused documentation checks passed, live golden-query acceptance remains open.

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
@huangruiteng
huangruiteng force-pushed the codex/steward-attention-evaluation branch from 5c200fa to 2b27e16 Compare September 25, 2026 08:13

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

动机

用户需要把“创建、找负责人、真实小团队协作、结果回到原入口”排到同一产品目标里。旧文档缺少一套简短请求和可独立检查的结果标准,容易让目录、收件箱回执或漂亮的回答被误当成完整交付。本 PR 将这些目标明确为 P0,同时保留后续能力的层次。

改动思路

以一份公开安全的 golden-query 集作为评测入口,再让整体 roadmap、管家 RFC、共用对话界面 RFC 双语引用它。它复用既有 R/G/M/A 里程碑和 Todo owner,不创建新协议或状态机。正向路径是冻结场景、运行基线与候选、检查负责人实际执行和原对话回传;失败路径明确保留未验收状态。

具体改动

  • docs/product/use-cases/steward/golden-queries.md 定义 17 类自然请求、P0/P1/P2 分层、两轮协作和独立质量/注意力度量;创建、负责人发现、采用、纠偏、停止、恢复都有反例。
  • 中英 roadmap 把入口与负责人路由、2–3 人真实协作列为 P0;中英管家 RFC 补足宽范围发现、委派准入及真实回传;展示 RFC 强调成果、普通进度、必要决策和首次请求保留。Steward use-case 索引提供直接入口。

关键内容讲解

GQ03 把“找 PR review 负责人”验收到因果检查、实际处理和回传,明确基线 CI 失败不是自动批准。GQ05/GQ11–GQ13 要求接收方使用修订版本、独立复核、处理分歧并完成两轮;一条 inbox ACK 不算协作。宽范围发现与证据读取、委派、执行就绪分别判断,外部受众不能继承主人的可见范围。

对主干的风险

主要风险是把规划误读为现有能力。文件明确标明所有真实案例尚未验收、30% 注意力下降只是预设目标,跨主机和规模场景需要另行冻结资源。完整 diff 无运行时代码和状态迁移;本地文档治理、资源完整性、空白和签署检查通过;新基线的 19 项风险选择合并前检查全部通过,质量凭据 cqr_dc60b1f7885721fc7044 有效,无失败或跳过。未跑真实 Agent,因此不能据此作产品发布结论。

我的整体评价

对长期协作和用户体验的规划都有正向增量:它让后续 PR 必须证明实际执行、采用和回传,也避免为了减少消息数牺牲结果质量。没有阻塞发现。文档切片完整且可独立撤回;残余风险是评测尚未实施,需由既有 R2/R3/A24 owner 用真实入口继续验收。

English verdict: APPROVE - exact head 2b27e1618fd68fb5c4f1809e023f141482b200b3; docs-only P0 evaluation contract, focused documentation checks passed, live golden-query acceptance remains open.

@huangruiteng
huangruiteng merged commit 19358f1 into main Sep 25, 2026
18 checks passed
@huangruiteng
huangruiteng deleted the codex/steward-attention-evaluation branch September 25, 2026 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant