Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -129,3 +129,26 @@ the yielding and in-flight proof lifecycle belong to File. #4931's digest
window remains a separate optimization. The regression uses a private real
server and the existing mixed Todo/lease/decision fixture; production locators,
active Goals and raw evidence are never modified or published.

## Local provider cutover (`76ff7c73c`)

Recovery/audit #5140 and runtime fairness #5156 are merged. This delivery adds
reviewed File ↔ SQLite cutover for an already-promoted canonical Goal: verified
backup, exact source/fence binding, identity-bound target, history/receipt audit,
serialized selector publication, crash resume and reverse migration carrying
new writes. Persisted active leases hold migration even after expiry. It does
not stop Hosts or migrate independently-owned Turn/spend state.

The current inventory is three existing open PRs (#5054 retirement, #4931 SQLite
proof encoding, #5144 managed Host supervision), this cutover PR, and remaining
whole-Goal integration/default-entry scopes. Whole-Goal acceptance still needs
source drain and all retained consumers/external execution boundaries; defaults
still need new-Goal/settings/install adoption and bounded Python writer removal.
Do not subtract one from a broad work package merely because its cutover subitem
shipped. An exact remaining PR count cannot be promised until that integration
inventory and D2 results determine whether additional bounded fixes are needed.
Capacity, platform coverage and natural-time soak remain evidence gates rather
than PR quotas. PostgreSQL keeps the shared logical archive/audit contract;
service authentication, tenancy and failover are not qualified by this command.

See [the reviewed cutover journey](../../../../reference/file-authority-state-log.md#reviewed-filesqlite-cutover).
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# 本地默认切换:恢复审计与剩余交付范围

- 核对基线:2026-09-27 `157ab7b11`,加本次交付。
- 最新核对基线:`76ff7c73c`;下方历史段落保留其当时基线。
- 当前数量以末尾“本地 provider 切换”表为准,历史规划不是剩余 PR 倒计时。
- 归属:总目标 #4574 R5/G2;shared authority D2/D3;TS T3/T4。
- 取代[九月二十四日清单](2026-09-24-default-cutover-reconciliation.zh-CN.md)的
**当前数量口径**,不覆盖历史证据。
Expand Down Expand Up @@ -97,3 +98,28 @@ p95 或跨平台资格。单笔巨大事务、JSON 解析、其他同步 handler
让步及进行中证明的生命周期归 File;#4931 的 digest window 仍是独立优化。
回归使用私有真实 server 和既有混合 Todo/lease/decision fixture,不改生产 locator
及活跃 Goal,不发布原始证据。

## 本地 provider 切换(`76ff7c73c`)

#5140 恢复审计、#5156 共享运行时延迟修复已合入,不能重复列为未完成。
本次交付“已晋升 canonical Goal 的 File ↔ SQLite 审核切换”:备份并核对完整历史与
原回执,绑定 source revision/fence 和 target identity,串行发布 selector,支持进程
中断后的续传,以及携带最新历史的反向迁移。未结算租约(包括过期 active)拦截。
此处不自动停止 Host,不迁移 Turn/spend 的独立状态,也不等于全部旧 Goal 晋升。

| 当前交付范围 | PR / 状态 | 仍需证明的结果 |
| --- | --- | --- |
| 旧 Todo events 退役与 supervisor 日志隔离 | 已有 #5054,开放 | 消费者迁走后的旧分支删除 |
| SQLite retained proof 编码 | 已有 #4931,开放 | 在 #4224 冻结负载上的正式复测,不以小型迁移耗时替代 |
| 受管 Host 执行区间保护 | 已有 #5144,开放 | 续约/取消/旧 executor 接管;attached Host 边界另行明确 |
| 整 Goal 激活与回退集成 | 本次交付其中的本地 provider 切换子项 | 旧来源 drain、全部保留消费者和外部执行状态的组合验收仍未关闭 |
| 默认入口与有界 Python 退役 | 尚未实现的后续范围 | 新 Goal、设置、安装和各入口采用合格 profile;只删除 caller 已迁走的业务 writer |

因此当前可确定的是 **3 个已有开放 PR、当前 1 个切换 PR,以及上述剩余集成/默认
入口范围**。本次没有把宽泛的“整 Goal”行直接勾完,也没有据此将总数机械减一。
只有补齐消费者清单和 D2 实测后,才能判断剩余集成可合成一个 PR,还是需按具体
失败拆分;目前不能准确承诺“再 N 个就全量切换”。容量/平台/自然时间 soak 是独立
证据门,不是编码 PR 配额。PostgreSQL 仍复用共享逻辑历史与回执审计,但本地切换
入口明确不接受 PostgreSQL,服务认证/tenant/failover 不在此处偷换为已完成。

操作与恢复边界见[审核切换](../../../../reference/file-authority-state-log.md#reviewed-filesqlite-cutover)。
79 changes: 79 additions & 0 deletions docs/reference/file-authority-state-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,3 +234,82 @@ platform CI evidence; POSIX validation does not substitute for it.
升级失败时可能已有部分 store 完成,必须据实报告并重试,不能覆盖之后产生的写入。
备份仍是旧格式,恢复时应先在隔离目录升级。只回退二进制并不等于安全回退数据。
该方案减少重复存储和后续写入耗时,但冷校验仍验证全历史,可能更慢。

## Reviewed File/SQLite cutover

An already-promoted, quiescent canonical Goal can change between the built-in
File and SQLite providers. Stop its writers and settle its task leases first.
An expired but still `active` lease is a hold: expiry does not prove that the
old Host stopped. This command does not stop Hosts, settle Turns, retire leases,
change a registry, or remove the legacy writer fence. PostgreSQL service
activation and migration from a legacy Markdown Goal are separate operations.

```bash
# Use the same explicit runtime root for planning, preview and execution.
loopx --runtime-root /absolute/runtime --format json authority-archive plan-migration \
--goal-id example --provider sqlite --plan /absolute/migration-plan.json
# Review the saved plan and take PLAN_SHA256 from the planning response.
loopx --runtime-root /absolute/runtime --format json authority-archive migrate \
--goal-id example --plan /absolute/migration-plan.json --plan-sha256 PLAN_SHA256
loopx --runtime-root /absolute/runtime --format json authority-archive migrate \
--goal-id example --plan /absolute/migration-plan.json --plan-sha256 PLAN_SHA256 --execute
```

The plan binds the canonical runtime path, Goal, source store identity, revision,
cursor, projection digest, writer-fence digest and destination provider. A
changed source requires a new reviewed plan. Plans are created exclusively;
choose a new path instead of overwriting an already-reviewed artifact.

Migration copies complete committed projections, events and original receipts;
it does not rebuild Todos from display columns or reapply a metadata allowlist.
Historical metadata values, absent keys, explicit nulls, false, zero and empty
arrays remain distinct. Auditing excludes only the transaction's physical
provider revision, which legitimately changes with the backend. It cannot prove
that an earlier legacy-to-canonical capture included every external field, nor
does it copy attachment files or independently-owned Host/Turn stores.

Execution shares the maintenance guard used by canonical command writers. It
saves a verified logical backup below the runtime's
`authority-transition/local-provider/<plan digest>/`, binds the target identity,
restores missing history, independently audits every retained transaction and
receipt, then atomically publishes the selector. Selected local stores check
their identity on every operation. A missing/replaced selected store fails
closed; it is never recreated or silently replaced with another provider.

An implicit File source becomes an explicit identity-bound File selection before
SQLite preparation. This keeps the source readable while the target is being
built. It does not change the source's domain history. Backup and target creation
consume additional disk space; retain the backup and recovery record for retries
and investigation. No automatic cleanup deletes these recovery artifacts.

Retry **the same plan and digest** after interruption. A partial restore audits
its prefix before appending. If publication already happened, recovery audits
the retained prefix without discarding later target writes. A publication error
can return `authority_changed: null`: the outcome is uncertain, so retry rather
than assume the source is still selected. A completed plan superseded by another
migration cannot reactivate its former target.

To return to File, create a **new** plan with `--provider file`, then preview and
execute it. This carries the current SQLite history back to File and retains
new acknowledged writes. An old File history is reusable only if it is an exact
prefix. Divergent/extra target history rejects migration; do not delete it or
copy old bytes over live state to force acceptance.

Provider rollback is distinct from binary downgrade. Earlier binaries that do
not recognize explicit File selectors cannot operate this runtime. Keep a
migration-capable binary; do not remove the selector to make an old binary start.
The generic historical format check alone does not prove selector compatibility.
This operation qualifies local storage continuity, not D2 capacity/soak, all
Host lifecycles, all Goal consumers, or permission to enable a provider by default.

### 已晋升 Goal 的本地 provider 切换

先停止写入并结算租约,再通过 `plan-migration` 保存并审核计划,用输出的摘要调用
`migrate` 预览和执行。未结算的租约即使过期也会阻止迁移。TS 在同一个 canonical
写锁下完成备份、历史与回执核对、selector 发布及回读;Python 只传参数和结果。

中断后用同一计划重试;切换后产生的新写入会被保留。回退是另做一个 `--provider
file` 的新计划,把最新历史迁回 File,不能覆盖旧备份。此处不负责停 Host、结算
Turn、旧 Markdown Goal 晋升或 PostgreSQL 服务部署,也不代表默认值资格通过。
旧版本若不认识显式 File selector,会拒绝读取;保留支持迁移的运行时,不删除
selector 绕过检查。
30 changes: 26 additions & 4 deletions loopx/cli_commands/authority_archive.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,15 @@

from ..control_plane.effect_runtime import effect_runtime_result
from ..paths import DEFAULT_RUNTIME_ROOT, global_registry_path, resolve_runtime_root
from ..history import load_registry
from ..control_plane.projects.registry_codec import load_registry


def register_authority_archive_command(
subparsers: argparse._SubParsersAction[argparse.ArgumentParser],
add_subcommand_format: Callable[[argparse.ArgumentParser], None],
) -> None:
parser = subparsers.add_parser(
"authority-archive", help="Export, verify, audit or restore a canonical authority copy."
"authority-archive", help="Back up, audit, restore or migrate canonical local authority."
)
add_subcommand_format(parser)
actions = parser.add_subparsers(dest="authority_archive_action", required=True)
Expand All @@ -27,6 +27,15 @@ def register_authority_archive_command(
mode.add_argument("--execute", action="store_true")
upgrade.add_argument("--all-known", action="store_true", help="Include runtime roots of registered projects.")
mode.add_argument("--require-current", action="store_true", help="Fail if a format upgrade is needed; never write.")
for name in ("plan-migration", "migrate"):
action = actions.add_parser(name, help="Review or execute a quiescent File/SQLite provider migration.")
action.add_argument("--goal-id", required=True)
action.add_argument("--plan", type=Path, required=True)
if name == "plan-migration":
action.add_argument("--provider", choices=("file", "sqlite"), required=True)
else:
action.add_argument("--plan-sha256", required=True)
action.add_argument("--execute", action="store_true", help="Publish the verified provider; otherwise preview.")
for name in ("export", "verify", "restore", "audit"):
action = actions.add_parser(name)
action.add_argument("--archive", type=Path, required=True)
Expand Down Expand Up @@ -63,6 +72,14 @@ def handle_authority_archive_command(
elif args.authority_archive_action == "upgrade":
request.update(runtime_roots=authority_upgrade_roots(
registry_path, runtime_root_arg, all_known=args.all_known), execute=args.execute)
elif args.authority_archive_action in {"plan-migration", "migrate"}:
request.update(goal_id=args.goal_id, plan=str(args.plan.expanduser().resolve()),
runtime_root=str(resolve_runtime_root(load_registry(registry_path), runtime_root_arg,
registry_path=registry_path)))
if args.authority_archive_action == "plan-migration":
request["provider"] = args.provider
else:
request.update(plan_sha256=args.plan_sha256, execute=args.execute)
else:
request["archive"] = str(args.archive.expanduser().resolve())
if args.authority_archive_action == "export":
Expand All @@ -83,13 +100,18 @@ def handle_authority_archive_command(
"coordination.authority_archive.manage", request, timeout=300.0, retry_safe=False
)
except (OSError, RuntimeError, ValueError) as error:
result = {"status": "failed", "reason": str(error), "authority_changed": False}
uncertain = args.authority_archive_action == "migrate" and args.execute
result = {"status": "failed", "reason": str(error),
"authority_changed": None if uncertain else False}
if uncertain:
result.update(reason_code="migration_outcome_unknown", requires_same_plan_retry=True)
if (args.authority_archive_action == "upgrade" and args.require_current
and any(row.get("status") == "planned" for row in result.get("results", []))):
result.update(status="failed", reason="Authority format upgrade required before activating this runtime.")
print_payload(result, output_format(args), lambda value: (
f"Authority archive: {value.get('status')}\n"
f"{value.get('reason', 'Active authority selection is unchanged.')}\n"
f"{value.get('reason', 'Authority changed: ' + str(value.get('authority_changed')))}\n"
f"Plan digest: {value.get('plan_sha256', 'not applicable')}\n"
f"{value.get('audit', '')}"
))
return 1 if result.get("status") == "failed" else 0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import {FILE_AUTHORITY_JOURNAL_SCHEMA} from "./file_authority_journal.ts";
import {sqliteAuthorityRuntime} from "./sqlite_runtime.ts";
import {SQLITE_AUTHORITY_STORE_SCHEMA, sqliteAuthorityPath} from "./sqlite_authority_store.ts";
import {SQLITE_AUTHORITY_STORE_V1_SCHEMA} from "./sqlite_authority_migration.ts";
import {decodeLocalAuthoritySelection} from "./local_authority_provider.ts";
import {verifyAuthorityArchive} from "./authority_archive.ts";

type StoreInspection = {
Expand Down Expand Up @@ -87,9 +88,7 @@ export async function inspectAuthorityFormat(path: string): Promise<AuthorityFor
value.schema_version !== FILE_AUTHORITY_JOURNAL_SCHEMA);
}
if (value.schema_version === "loopx_local_authority_provider_v0") {
if (value.provider !== "sqlite" && value.provider !== "postgresql") throw new Error("Unknown selector provider");
requireAuthorityStoreId(value.store_identity, "selector store identity");
if (value.provider === "postgresql") requireAuthorityStoreId(value.tenant_id, "selector tenant id");
decodeLocalAuthoritySelection(value, requireAuthorityStoreId(value.goal_id, "goal id"));
return {artifact_kind: "provider_selector", status: "recognized", format: value.schema_version,
goal_id: requireAuthorityStoreId(value.goal_id, "goal id"), provider: value.provider ?? null,
verification: "metadata_only", migration_route: "resolve_selected_provider_before_migration"};
Expand Down
16 changes: 10 additions & 6 deletions loopx/control_plane/coordination/file_authority_store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -180,8 +180,9 @@ export class FileAuthorityStore implements AuthorityStore {
readonly path: string;
readonly identityPath: string;
private readonly existingOnly: boolean;
private readonly expectedIdentity: string | undefined;

constructor(directory: string, goalId: string, options: { existingOnly?: boolean } = {}) {
constructor(directory: string, goalId: string, options: { existingOnly?: boolean; expectedIdentity?: string } = {}) {
this.goalId = requireAuthorityStoreId(goalId, "goal id");
if (typeof directory !== "string" || directory.length === 0) {
throw new AuthorityStoreProtocolError("store directory is required");
Expand All @@ -191,6 +192,7 @@ export class FileAuthorityStore implements AuthorityStore {
this.path = join(this.directory, `authority-store-${digest}.json`);
this.identityPath = join(this.directory, "store-identity");
this.existingOnly = options.existingOnly === true;
this.expectedIdentity = options.expectedIdentity;
}

/** Narrow effect seam for crash-window qualification; not a semantic hook. */
Expand All @@ -214,20 +216,22 @@ export class FileAuthorityStore implements AuthorityStore {
private async readStoreIdentity(createIfMissing = !this.existingOnly): Promise<string> {
try {
const identity = await readFile(this.identityPath, "utf8");
if (!STORE_IDENTITY_PATTERN.test(identity)) {
throw new AuthorityStoreProtocolError("store identity does not match file:<32 lowercase hex>");
if (!STORE_IDENTITY_PATTERN.test(identity) ||
(this.expectedIdentity !== undefined && identity !== this.expectedIdentity)) {
throw new AuthorityStoreProtocolError("store identity is invalid or differs from the expected File lineage");
}
if (createIfMissing) await syncAuthorityDirectory(this.directory);
return identity;
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error;
}
if (!createIfMissing) throw new FileStoreUnavailableError("existing store identity is missing");
if (!createIfMissing || this.expectedIdentity !== undefined) throw new FileStoreUnavailableError("existing store identity is missing");
return await withFileMutationLock(this.identityPath, async () => {
try {
const identity = await readFile(this.identityPath, "utf8");
if (!STORE_IDENTITY_PATTERN.test(identity)) {
throw new AuthorityStoreProtocolError("store identity does not match file:<32 lowercase hex>");
if (!STORE_IDENTITY_PATTERN.test(identity) ||
(this.expectedIdentity !== undefined && identity !== this.expectedIdentity)) {
throw new AuthorityStoreProtocolError("store identity is invalid or differs from the expected File lineage");
}
await syncAuthorityDirectory(this.directory);
return identity;
Expand Down
2 changes: 2 additions & 0 deletions loopx/control_plane/coordination/local_authority_archive.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
/** Administrative archive transport. Large private state stays in local files;
* the managed effect runtime returns only compact integrity/readback facts. */
import {manageLocalAuthorityMigration} from "./local_authority_migration.ts";
import {inspectAuthorityFormat} from "./authority_format_inspection.ts";
import {upgradeAuthorityFormats} from "./authority_format_upgrade.ts";
import {mkdir, readFile} from "node:fs/promises";
Expand Down Expand Up @@ -35,6 +36,7 @@ export async function manageLocalAuthorityArchive(value: unknown,
}
return {...base, ...await upgradeAuthorityFormats(request.runtime_roots as string[], request.execute === true)};
}
if (request.action === "plan-migration" || request.action === "migrate") return await manageLocalAuthorityMigration(request);
const archive = path(request.archive, "archive path");
if (request.action === "verify") return {...base, status: "verified", archive: await verifyAuthorityArchive(archive)};
const goalId = requireAuthorityStoreId(request.goal_id, "goal id");
Expand Down
Loading
Loading