Skip to content

fix(scheduler): avoid secret-shaped transport chunks - #4139

Merged
huangruiteng merged 2 commits into
loopx-project:mainfrom
Duang777:codex/fix-scheduler-hint-safe-encoding
Sep 11, 2026
Merged

huangruiteng merged 2 commits into
loopx-project:mainfrom
Duang777:codex/fix-scheduler-hint-safe-encoding

Conversation

@Duang777

@Duang777 Duang777 commented Sep 9, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • encode scheduler host-fact chunks with the standard Base64 alphabet
  • let the native TypeScript decoder accept both old URL-safe and new standard Base64 chunks
  • remove the now-unreachable dash-prefixed argv workaround
  • prevent opaque transport bytes from accidentally matching the public credential detector

Root cause

URL-safe Base64 can emit hyphen and underscore. A generated chunk containing -ak- or -sk- is then mistaken for an access/secret-key marker, causing unrelated model-behavior tests and PR merge gates to fail. Standard Base64 avoids those marker shapes, while the native decoder keeps accepting both alphabets for compatibility.

Validation

  • regression test failed before the implementation change and passes after it
  • 45 focused Python scheduler/native-launcher/portfolio tests passed
  • 878 TypeScript control-plane tests passed, 1 skipped
  • TypeScript typecheck passed
  • Ruff and git diff --check passed
  • LoopX premerge gate passed: 17/17 selected checks, 0 failures, 0 warnings, 0 manual holds

Signed-off-by: duanjialing.777 duanjialing.777@bytedance.com

@Duang777

Duang777 commented Sep 9, 2026 •

Copy link
Copy Markdown
Collaborator Author

@huangruiteng 当前 head 4ad1d58 已同步 main,并带入同一已验证的 Node 26 fence-error 稳定化修复。验证:Python 6/6、TypeScript 29/29、Ruff 与 diff check 通过;新 CI 已触发,请按此 head 复审。

@Duang777
Duang777 force-pushed the codex/fix-scheduler-hint-safe-encoding branch 4 times, most recently from 03de8a7 to 21b0beb Compare September 9, 2026 22:48
Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
@Duang777
Duang777 force-pushed the codex/fix-scheduler-hint-safe-encoding branch from 21b0beb to b1ed13a Compare September 10, 2026 06:10
Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

评审绑定 exact head:4ad1d582ebeeec0737d9d7b164a16f1d92c3e3a0。

动机

这个 PR 处理的是 scheduler host-facts 传输与 public-safety 的偶发碰撞。旧 producer 使用 URL-safe Base64,字符集包含 -/_;压缩字节随机形成 -ak-<suffix> 或 -sk-<suffix> 时,会被 credential-like regex 当成 AK/SK,从而让本来只是公开安全、压缩后的 host facts 阻断 hint 或 merge gate。它不是 credential 处理问题,而是 derived transport alphabet 与扫描规则不相容。

改动思路

现有边界分工保持不变:Python scheduler_hint.py 负责生成 bounded argv,TypeScript heartbeat_followup_cli.ts 负责解码、校验 facts identity 并提交 scheduler receipt。producer 改用无 padding 的标准 Base64,避免 -/_;consumer 的 alphabet guard 同时允许旧 URL-safe 与新标准字符,Buffer.from(..., "base64url") 继续完成 rolling compatibility。这样不需要放宽 credential detector,也不改变 scheduler 状态机。

具体改动

关键代码讲解

  1. _scheduler_host_followup_transport_args 从 urlsafe_b64encode 切到 b64encode,并删除只有 chunk 以 - 开头才需要的 --flag=value 特例;size bound、zlib、JSON shape 和 chunk size 均不变。
  2. decodeSchedulerHostFactsChunks 将 lexical guard 扩为 URL-safe 与 standard Base64 的并集,随后仍执行 encoded-size guard、bounded inflate、JSON object 校验与 exact identity matching。
  3. Python transport tests 与 TypeScript native tests同步使用新 producer,覆盖 ACK/failure、JSON/Markdown、malformed、encoded/inflated overflow 和 launcher route。当前 head 还带有一笔 writer-fence error-path normalization,但相同语义已经随 #4095 进入当前 main,合成 merge 时不形成新增 delta。

有一个非阻塞的测试精度建议:test_native_facts_chunks_do_not_look_like_credentials 当前构造的旧文本只有 -ak-,而真实 regex 要求 ak- 后至少 10 个字符,所以它没有直接证明“旧值会命中 detector”。我用 -ak-AAAAAAAAAAAA 做了同字节 counterfactual:旧值确实命中,新标准编码 +ak+AAAAAAAAAAAA 不命中。建议把 fixture 加长并加入 old-match 断言。

对主干的风险

风险集中在 rolling compatibility 与 native dispatch。exact head 上,scheduler transport/native launcher/CLI 相关 Python suite 为 34 passed,native TypeScript 为 7 passed;Ruff 与 diff check 通过,远端 required checks 全绿。当前 origin/main merge-tree 干净,合成 merge 上相同 34+7 tests 也通过。旧 URL-safe underscore payload 能进入 decoder 并到达 bounded-inflate rejection,standard producer 的真实 ACK/failure path也已执行。

没有 live Codex App cadence mutation,但验证穿过了真实 scripts/loopx native launcher、独立 fixture runtime 与 scheduler commit receipt,足以覆盖本 PR 的 transport surface。剩余风险主要是上述 fixture 没有在仓库测试内直接锁住旧 regex 命中;实现本身已由独立 counterfactual 和 alphabet contract验证。

我的整体评价

结论是 APPROVE(附 P2 测试加强建议)。这是一个比例合适的修复:不去削弱 credential gate,而是在 transport owner 消除冲突字符,并让 consumer 保持向后兼容;删除的 argparse 特例也让后续维护更简单。没有 state、authority、cadence、quota 或 guidance/obligation 漂移,当前主干集成证据完整。

English verdict: APPROVE — exact head 4ad1d582ebeeec0737d9d7b164a16f1d92c3e3a0 safely moves scheduler host-fact emission to standard Base64 while retaining URL-safe decode compatibility; 34 Python and 7 native TypeScript tests pass on both the exact head and current-main integration, with a P2 request to make the committed fixture reproduce the actual old credential-regex hit.

@huangruiteng
huangruiteng merged commit b8e7be8 into loopx-project:main Sep 11, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants