Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 23 additions & 17 deletions docs/architecture/rfcs/shared-goal-authority-state-provider-v0.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@
remains candidate evidence, not a merge gate or authority promotion
- PostgreSQL baseline: the TypeScript Stage 2B candidate implements the store
contract, transaction-local tenant context, forced row-level security, and
has passed a real PostgreSQL 16 transaction matrix. No shared authority
service, runtime caller, principal authentication/tenant authorization, or
bounded canonical commit admission, and has passed a real PostgreSQL 16
transaction matrix. No shared authority service, runtime caller, principal
authentication/tenant authorization, measured capacity/retention profile, or
authority promotion ships yet
- Language note: the
[Chinese version](./shared-goal-authority-state-provider-v0.zh-CN.md) and this
Expand Down Expand Up @@ -1258,6 +1259,13 @@ projection-plus-receipt commit, CAS contention, historical receipt replay,
operation fencing, ordered cursor scans, isolation of returned values, and
pre-write rejection of malformed JSON.

The PostgreSQL adapter also applies one provider-local resource guard before it
opens a connection: a commit whose canonical envelope exceeds the configured
`max_commit_bytes` is rejected as typed `store_capacity_exhausted`. The default
is 16 MiB and a deployment may lower it. This is an admission ceiling for one
atomic operation, not measured throughput evidence and not a retention or
partitioning design; those promotion holds remain open.

Real PostgreSQL qualification starts here, not at shadow or canary. A
PostgreSQL 16 instance passed the shared conformance matrix, same-head
concurrent CAS, tenant-scoped reuse of the same goal and operation ids,
Expand All @@ -1276,10 +1284,11 @@ The database runtime-role and RLS behavior within the service trust boundary is
now implemented and qualified. Service API authentication,
principal-to-tenant authorization, the production runtime caller,
restore-incarnation rotation, pool
exhaustion/cancellation/failover, one-way shadow parity, the TEST ONLY canary,
and authority-source promotion remain explicit holds. The next PostgreSQL
slice must qualify the authenticated service/deployment and failure boundary;
it must not treat database RLS as that missing API authorization layer.
exhaustion/cancellation/failover, retention/partitioning/measured capacity,
one-way shadow parity, the TEST ONLY canary, and authority-source promotion
remain explicit holds. The next PostgreSQL slice must qualify the authenticated
service/deployment and failure boundary; it must not treat database RLS or the
single-commit admission ceiling as those missing service and capacity layers.

The file-backed provider contract and executor are Stage 2; their first slice
is merged on `main` through #3529, and the evidence behind it is recorded in
Expand Down Expand Up @@ -2158,14 +2167,11 @@ that rewrite total. This rate is already enough to require provider-specific
capacity, latency, response-size, and recovery tests. Retaining everything in
one document is acceptable only for promotion bootstrap and bounded test goals.

### Sequence

A. transaction-bound capture feeding `coordination.runtime_shadow.commit`,
retiring the duplicate observation lineage; B. freeze and test the complete
Todo/lease manifests and omission/explicit-clear rules; C. implement the common
authority binding and profile conformance, including retention/capacity for the
selected provider; D. add provider-first CLI routing, the lock-owning promotion
orchestrator, and compatibility projection outbox on the shipped kernel; E. a
separately reviewed promotion PR deletes the reference-only duplicate aggregate,
flips holds and stage literals, and updates the execution ledger. A profile is
eligible only after A-D pass for its exact implementation and lineage.
### Parallel delivery plan

| Lane | May start | Scope and exit condition | Dependency |
| --- | --- | --- | --- |
| P. PostgreSQL provider plane | Now, from current `main` | Keep the existing `AuthorityStore` contract; finish schema migration/install ownership, authenticated service and tenant authorization, restore-incarnation rotation, pool/cancellation/failover behavior, and reviewed indexes, partitioning, retention, and measured capacity. Live PostgreSQL conformance remains mandatory. | Does not depend on #3870 and must not stack on its branch. This lane alone creates no runtime caller or promotion claim. |
| C. Canonical transaction capture | Now, by revising or replacing #3870 | Make the outbox transport complete versioned Todo/lease records into `coordination.runtime_shadow.commit`; retire the duplicate observation/local-shadow lineage and prove omission/explicit-clear behavior. | Can run in parallel with P, but both C and the selected provider profile must finish before parity or promotion integration. |
| I. Binding and qualification integration | After P and C | Bind one exact provider lineage, field manifest, source revision, digest, and cursor; run sustained transaction parity and provider-specific recovery/capacity qualification without consulting legacy state for missing fields. | This is the merge point between provider work and capture work. |
| F. Promotion and cleanup | After I and explicit maintainer approval | Add provider-first CLI routing, the lock-owning promotion orchestrator, compatibility projection outbox, post-promotion fenced export/rollback, then delete duplicate reference aggregates and flip the reviewed stage/hold declarations. | No profile is eligible until its exact implementation and lineage pass P, C, and I. |
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,10 @@
验证只接受这份 SDK 合同与本 checkout 的 helper;它仍是候选证据,不是合并门槛
或 authority promotion
- PostgreSQL 基线:TypeScript Stage 2B candidate 已实现 store contract、
transaction-local tenant context 与 forced row-level security,且已通过真实
PostgreSQL 16 transaction matrix;shared authority service、runtime caller、
principal authentication/tenant authorization 与 authority promotion 均尚未交付
transaction-local tenant context、forced row-level security 与有界 canonical
commit admission,且已通过真实 PostgreSQL 16 transaction matrix;shared
authority service、runtime caller、principal authentication/tenant authorization、
实测 capacity/retention profile 与 authority promotion 均尚未交付
- 语言说明:[英文版](./shared-goal-authority-state-provider-v0.md)与本中文版互为
语义镜像;两者不一致属于缺陷

Expand Down Expand Up @@ -1015,6 +1016,12 @@ conformance suite,覆盖 projection-plus-receipt 原子提交、CAS contention
replay、operation fencing、有序 cursor scan、返回值隔离,以及 malformed JSON 在写前
被拒绝。

PostgreSQL adapter 还会在打开连接之前执行一项 provider-local 资源门禁:canonical commit
envelope 超过配置的 `max_commit_bytes` 时,写入以 typed
`store_capacity_exhausted` 被拒绝。默认值为 16 MiB,部署可以调低。它只是单笔原子操作的
准入上限,不是实测 throughput 证据,也不是 retention/partitioning 设计;这些晋升 hold
仍然开放。

真实 PostgreSQL qualification 从这里开始,而不是等到 shadow 或 canary。一个真实
PostgreSQL 16 实例已通过共享 conformance matrix、同一 head 的并发 CAS、不同 tenant
复用相同 goal 与 operation id、transaction rollback 后不暴露 head 或 receipt、已提交
Expand All @@ -1028,10 +1035,11 @@ row,跨 context 写入失败,runtime role 也不能修改行政 metadata。F
保持不变,Agent 也不能获得注入的 pool。Service trust boundary 内的数据库
runtime-role/RLS 行为现已实现并完成资格化。Service API authentication、
principal-to-tenant authorization、production runtime caller、restore
incarnation rotation、pool exhaustion/cancellation/failover、单向 shadow parity、TEST
ONLY canary 与 authority-source promotion 仍是显式 hold。下一个 PostgreSQL 切片必须
资格化 authenticated service/deployment 与 failure boundary,不能把 database RLS 当成
仍缺失的 API authorization layer。
incarnation rotation、pool exhaustion/cancellation/failover、retention/partitioning/实测
capacity、单向 shadow parity、TEST ONLY canary 与 authority-source promotion 仍是显式
hold。下一个 PostgreSQL 切片必须资格化 authenticated service/deployment 与 failure
boundary,不能把 database RLS 或单笔 commit 准入上限当成仍缺失的 service 与 capacity
层。

File-backed provider 合同与 executor 属于 Stage 2;其第一个切片已通过 #3529
合入 `main`,证据记录在下方的 Stage 2 状态小节。该切片证明 aggregate 与 provider
Expand Down Expand Up @@ -1714,12 +1722,11 @@ record layout,不能由重写总量直接推断。这一频率已经要求逐
latency、response size 与 recovery。单文档全量保留只适用于 promotion bootstrap 与
有界 test goal。

### 顺序
### 并行交付计划

A. 用 transaction-bound capture 喂给 `coordination.runtime_shadow.commit`,退役重复
observation lineage;B. 冻结并测试完整 Todo/lease manifest,以及 omission/explicit-clear
规则;C. 实现共同 authority binding 与 profile conformance,包括所选 provider 的
retention/capacity;D. 在已交付 kernel 上加入 provider-first CLI routing、持锁 promotion
orchestrator 与兼容投影 outbox;E. 单独评审的 promotion PR 删除 reference-only 重复
aggregate,翻转 hold 与 stage literal,并更新执行台账。任何 profile 只有在其精确实现与
lineage 上通过 A-D 后才具备晋升资格。
| Lane | 何时开始 | 范围与退出条件 | 依赖 |
| --- | --- | --- | --- |
| P. PostgreSQL provider plane | 现在,从当前 `main` 开始 | 保持既有 `AuthorityStore` 合同;完成 schema migration/install ownership、authenticated service 与 tenant authorization、restore-incarnation rotation、pool/cancellation/failover 行为,以及经评审的 index、partition、retention 与实测 capacity。真实 PostgreSQL conformance 始终是强制门禁。 | 不依赖 #3870,也不得叠在其分支上。仅完成本 lane 不产生 runtime caller 或 promotion 声明。 |
| C. Canonical transaction capture | 现在,通过修订或替代 #3870 | 让 outbox 把完整、带版本的 Todo/lease record 传给 `coordination.runtime_shadow.commit`;退役重复 observation/local-shadow lineage,并证明 omission/explicit-clear 行为。 | 可与 P 并行;但 C 与选定 provider profile 都完成后,才能进入 parity 或 promotion 集成。 |
| I. Binding 与资格集成 | P 与 C 完成后 | 绑定一个精确 provider lineage、field manifest、source revision、digest 与 cursor;运行持续 transaction parity,以及 provider-specific recovery/capacity qualification;缺字段时不得查询 legacy state 补齐。 | 这是 provider 工作与 capture 工作的汇合点。 |
| F. Promotion 与清理 | I 完成且 maintainer 显式批准后 | 加入 provider-first CLI routing、持锁 promotion orchestrator、兼容投影 outbox、晋升后 fenced export/rollback;随后删除重复 reference aggregate,并翻转经评审的 stage/hold 声明。 | 一个 profile 的精确实现与 lineage 通过 P、C、I 前,不具备晋升资格。 |
1 change: 1 addition & 0 deletions loopx/control_plane/coordination/authority_store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ export const AUTHORITY_STORE_PROVIDER_PROFILES = {
"service_role_provisioning_and_audit_policy",
"restore_incarnation_rotation",
"failover_pool_exhaustion_and_cancellation",
"retention_partitioning_and_measured_capacity",
"shadow_parity_and_authority_source_promotion",
],
},
Expand Down
33 changes: 33 additions & 0 deletions loopx/control_plane/coordination/postgresql_authority_store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import type {
} from "./authority_store.ts";
import {
AuthorityStoreProtocolError,
canonicalAuthorityBytes,
canonicalAuthorityObject,
canonicalAuthorityObjectList,
normalizeAuthorityStoreCommit,
Expand All @@ -22,6 +23,7 @@ import {
const POSTGRESQL_STORE_IDENTITY_PATTERN = /^postgresql:[0-9a-f]{32}$/;
const POSTGRESQL_PROVIDER_REVISION_PATTERN = /^postgresql:([0-9a-f]{32}):([1-9]\d*)$/;
const POSTGRESQL_SCHEMA_VERSION = "loopx_postgresql_authority_store_v0";
export const DEFAULT_POSTGRESQL_MAX_COMMIT_BYTES = 16 * 1024 * 1024;

/**
* Structural subset of a server-owned PostgreSQL driver connection.
Expand All @@ -44,6 +46,12 @@ export interface PostgreSqlAuthorityDatabase {
export interface PostgreSqlAuthorityStoreOptions {
tenant_id: string;
goal_id: string;
/**
* Maximum canonical bytes admitted for one atomic commit. Deployments may
* lower this envelope; promotion still requires separately measured
* retention and sustained-capacity evidence.
*/
max_commit_bytes?: number;
}

interface HeadRow extends JsonObject {
Expand Down Expand Up @@ -269,6 +277,15 @@ function parseExpectedRevision(value: string | null): {
return { store_identity: `postgresql:${match[1]!}`, revision: match[2]! };
}

function canonicalCommitEnvelopeBytes(commit: AuthorityStoreCommit): number {
return canonicalAuthorityBytes({
operation_id: commit.operation_id,
events: commit.events,
next_projection: commit.next_projection,
receipts: commit.receipts,
}).byteLength;
}

function readFailure(error: unknown): AuthorityStoreReadFailure {
if (error instanceof AuthorityStoreProtocolError || error instanceof SyntaxError) {
return {
Expand Down Expand Up @@ -398,6 +415,7 @@ export class PostgreSqlAuthorityStore implements AuthorityStore {
readonly database: PostgreSqlAuthorityDatabase;
readonly tenantId: string;
readonly goalId: string;
readonly maxCommitBytes: number;

constructor(
database: PostgreSqlAuthorityDatabase,
Expand All @@ -406,6 +424,12 @@ export class PostgreSqlAuthorityStore implements AuthorityStore {
this.database = database;
this.tenantId = requireAuthorityStoreId(options.tenant_id, "tenant id");
this.goalId = requireAuthorityStoreId(options.goal_id, "goal id");
this.maxCommitBytes = options.max_commit_bytes ?? DEFAULT_POSTGRESQL_MAX_COMMIT_BYTES;
if (!Number.isSafeInteger(this.maxCommitBytes) || this.maxCommitBytes < 1) {
throw new AuthorityStoreProtocolError(
"PostgreSQL max commit bytes must be a positive safe integer",
);
}
}

private async connect(): Promise<PostgreSqlAuthorityConnection> {
Expand Down Expand Up @@ -496,6 +520,15 @@ export class PostgreSqlAuthorityStore implements AuthorityStore {
reason: error instanceof Error ? error.message : "invalid commit request",
};
}
const commitBytes = canonicalCommitEnvelopeBytes(normalized);
if (commitBytes > this.maxCommitBytes) {
return {
status: "failed",
reason_code: "store_capacity_exhausted",
reason:
`PostgreSQL authority commit is ${commitBytes} canonical bytes; configured maximum is ${this.maxCommitBytes}`,
};
}

let connection: PostgreSqlAuthorityConnection;
try {
Expand Down
5 changes: 5 additions & 0 deletions tests/control_plane_ts/authority_store.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,11 @@ test("provider profiles map one logical contract onto different backend primitiv
"service_role_provisioning_and_audit_policy",
),
);
assert.ok(
AUTHORITY_STORE_PROVIDER_PROFILES.postgresql.qualification_holds.includes(
"retention_partitioning_and_measured_capacity",
),
);
assert.notDeepEqual(
AUTHORITY_STORE_PROVIDER_PROFILES.file,
AUTHORITY_STORE_PROVIDER_PROFILES.nokv,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import test from "node:test";
import { Pool, type PoolClient } from "pg";

import {
DEFAULT_POSTGRESQL_MAX_COMMIT_BYTES,
installPostgreSqlAuthorityStoreSchema,
POSTGRESQL_AUTHORITY_STORE_SCHEMA_SQL,
PostgreSqlAuthorityStore,
Expand Down Expand Up @@ -362,6 +363,52 @@ test("PostgreSQL schema declares fail-closed tenant RLS on every scoped table",
);
});

test("PostgreSQL provider rejects an oversized commit before opening a connection", async () => {
let connectionAttempts = 0;
const disconnectedDatabase: PostgreSqlAuthorityDatabase = {
connect: async () => {
connectionAttempts += 1;
throw new Error("capacity admission must run before connect");
},
};
const store = new PostgreSqlAuthorityStore(disconnectedDatabase, {
tenant_id: "tenant-capacity",
goal_id: "goal-capacity",
max_commit_bytes: 256,
});
const oversized = commit(null, "operation-oversized", 1, 1);
oversized.next_projection.payload = "x".repeat(1024);

const result = await store.commitAuthority(oversized);
assert.equal(result.status, "failed");
if (result.status === "failed") {
assert.equal(result.reason_code, "store_capacity_exhausted");
assert.match(result.reason, /configured maximum is 256/);
}
assert.equal(connectionAttempts, 0);
});

test("PostgreSQL provider validates its commit capacity envelope", () => {
const disconnectedDatabase: PostgreSqlAuthorityDatabase = {
connect: async () => {
throw new Error("not reached");
},
};
const defaultStore = new PostgreSqlAuthorityStore(disconnectedDatabase, {
tenant_id: "tenant-capacity",
goal_id: "goal-capacity",
});
assert.equal(defaultStore.maxCommitBytes, DEFAULT_POSTGRESQL_MAX_COMMIT_BYTES);
assert.throws(
() => new PostgreSqlAuthorityStore(disconnectedDatabase, {
tenant_id: "tenant-capacity",
goal_id: "goal-capacity",
max_commit_bytes: 0,
}),
/max commit bytes must be a positive safe integer/,
);
});

test("PostgreSQL provider evicts and awaits cleanup-uncertain read connections", async () => {
const tenantId = "tenant-cleanup-fault";
const rollbackFailure = new Error("injected rollback failure");
Expand Down
Loading