fix(sqlite): retain fast COUNT and add D2 qualification checks - #4328
Conversation
Signed-off-by: Lihua <1017343802@qq.com>
Signed-off-by: Lihua <1017343802@qq.com>
Signed-off-by: Lihua <1017343802@qq.com>
Signed-off-by: Lihua <1017343802@qq.com>
Signed-off-by: Lihua <1017343802@qq.com>
huangruiteng
left a comment
There was a problem hiding this comment.
动机
评审目标是确认本 PR 在精确 head 3400fab4231adf51c3b8137f7049940d29fd9b25 上,是否真正解决 SQLite WAL-reset 风险、保持 authority 的原子性,并让容量报告不会把不完整的 D2 证据误报为通过。作者同时把 SQLite 资格 runtime、容量 profile 和 CI 分层写清楚;这比仅在旧 helper 中增加一个版本字符串更完整,但也扩大了示例与验证面。
改动思路
sqlite_runtime.ts 成为 SQLite runtime admission 的唯一决策者:先在 :memory: 中读取实际 sqlite_version()/sqlite_source_id(),验证已知修复版本线和同步 statement finalization,失败时在创建 authority 目录前返回协议失败。SqliteAuthorityStore 继续拥有 schema、identity、WAL、head/receipt 和事务不变量;只调用新的 admission helper,不改变 File provider。
容量路径由 sqlite-capacity.ts 驱动一次性 synthetic database,sqlite-capacity-report.ts 负责 typed passed/failed/missing ledger。默认命令改为小型 rehearsal,正式 matched-64k 明确运行 10k/100k 两条轴;CI 将 public-minimum Node 与 qualified SQLite Node 22.22.3 分开。文档说明了未覆盖的写放大、锁等待、OS/恢复、长 soak 和 promotion hold。
具体改动
sqliteAuthorityRuntime严格解析 3.44.6+/3.50.7+/3.51.3+ 修复线并验证关闭后的 prepared statement;SqliteAuthorityStore.open在任何 mkdir/open 前调用它。current将 scalarCOUNT(*)的 cast 移到子查询外,保留 SQLiteCountfast opcode,同时继续检查min=1,max=count=head的连续性。capacityLedger只有完整 64 KiB、10k/100k、规定样本数且 cleanup 成功时才允许 formal pass;其余预算和 D2 维度保持missing,失败保持failed。measureAxis使用真实 authority store、local selector、legacy fence、CLI 及共享 fixture,记录 runtime/source identity、分位延迟、请求字节、文件大小、RSS 与 cleanup,并在失败时保留 phase/非零退出。- workflow、英文/中文 authority 文档、进程 crash/CAS/scan/容量测试和 todo continuation fixture 同步更新;没有改变默认 File provider 或持久 schema。
对主干的风险
我未发现阻塞性问题。风险主要是运行成本和环境边界:当前机器的 Node 25.5.0 内置 SQLite 3.51.2,按新契约被正确拒绝,因此本地无法直接跑完整 SQLite suite;远端 required CI 已在 qualified Node 22.22.3 lane 通过。正式 matched-64k/CLI profile 仍是昂贵的维护者操作,且报告明确不等价于十天 soak、OS ENOSPC、restore 或 promotion 证据。未来 SQLite minor 通过 minor > 51 规则被接受,若上游修复承诺发生变化,应更新版本表和测试。没有新增 scheduler/quota/authority 外部权限。
我的整体评价
APPROVE。该 PR 的 runtime gate、authority 复用、typed missing ledger 和默认行为披露形成了一个可审计的完整边界;正向/负向测试与远端 required checks 均覆盖了主要承诺。请继续把 missing/failed 当作 hold,不要把 rehearsal 结果用于 promotion。
English verdict: APPROVE for exact head 3400fab4231adf51c3b8137f7049940d29fd9b25. SQLite admission now fails closed before side effects, fast COUNT and authority invariants are preserved, and incomplete capacity evidence remains missing. The qualified Node 22.22.3 CI lane and required checks pass; this host’s Node 25.5.0/SQLite 3.51.2 rejection is expected.
Summary
SQLite head continuity wrapped COUNT in CAST, disabling the engine's fast-count optimization and aggregating every retained operation row. Move the conversion outside the scalar subquery: counts remain exact SQL TEXT, all continuity/digest checks remain, and the connection lifetime is unchanged. A regression captures the production query and verifies the fast Count path plus empty/gap/int64 boundary behavior.
Also reject embedded SQLite releases affected by the WAL-reset concurrency bug, using Node 22.22.3 / SQLite 3.51.3 as the SQLite reference while keeping the public Node 22.18 minimum for File. Extend the existing capacity entrypoint with explicit rehearsal and matched 64 KiB 10k/100k profiles, cold CLI, separate byte metrics and a passed/failed/missing ledger, plus real-process recovery tests.
The no-argument capacity command intentionally changes from the old 4 KiB/100k run to the small rehearsal. After the narrow SQL fix, all 10 measured checks pass; 11 unmeasured D2 requirements remain explicit holds. The original failed 2.213x head-growth measurement is retained as the baseline.
窄 SQL 修复恢复快速计数,保留完整性与文本计数语义。正式 5:1 对照的 head p95 增长降至 1.642x,未改变预算;完整 D2 仍待后续资格证据。
Issue Or Task
Validation
3400fab4231adf51c3b8137f7049940d29fd9b25. Current formal measurements, focused/full regressions and premerge cover the appended SQL fix. Original formal baseline:f6784e40845d36e7592c57c0b7620c5db744d6dc; the intermediate report metadata correction did not change workload or timers.staticpassednpm run typecheck:control-plane, runtime-workflow smoke, CI classifier tests, diff checks and public/private scans. The current SQL fix and full PR passed premerge.unitpassedreal_entrypointpassedtests/control_plane/test_sqlite_authority_cli.py: 5 passed. Capacity rehearsal and full profile exercise canonical status/quota reads, native mutation and isolated runtime shutdown.real_backendpassedregression_paritypassed34710d17aand candidate, using Node 22.18 / SQLite 3.50.2: baseline admits the vulnerable library; candidate rejects before authority directory creation. Default File head and receipts match. The fast-count regression fails on the prior production query and passes on the fix; existing corruption/recovery cases stay green. An injected vulnerable-version case traverses the actual provider entrypoint.integrationpassedmanualpassedintegrationnot_runstaticpassedmerge-gatepassed. CI run.Matched storage-axis summary (milliseconds; each group uses a new database). Host: macOS arm64, 18 logical CPUs and 64 GiB RAM; local temporary storage. The report does not claim hardware/OS qualification:
Full reports also contain p50/p99, Node/SQLite/source identifiers and sample counts. DB sizes at the target history, before the separate CLI mutations, are 696,594,432 / 6,966,067,200 bytes; sampled WAL is zero in both groups. These file sizes and application JSON bytes are not cumulative WAL or physical-write evidence. CLI mutation p95 difference is -5.033 ms against the <=200 ms increment budget.
Frontend / Visual Evidence
Type of Change
LoopX Area
Technical Direction
Core control-plane hardening
Long-horizon benchmark evidence
Operator surface and IM integration
Shared Goal Authority and cross-host coordination
Architecture and research incubator
Target base branch:
main; implementation baseline34710d17addbbdfd56e6ea1f03ec73519a4aa7fd.Direction tracker or promotion unit: RFC section 7.2 / D2. Default provider changes, active-Goal migration, persistent-format redesign and soak launch remain separately gated.
Shared-authority RFC fixture impact
loopx_coordination_production_scale_fixture_v0is unchanged. The capacity runner reusesauthorityProjectionFixturewith one fixed-size native synthetic Todo; it does not impersonate the complete production-scale workload.Boundary Checklist
none.Signed-off-bytrailer (git commit -s).