Share PostgreSQL with Substrate

[iplay88keys]

Important

This PR requires a Substrate chart containing kagent-dev/substrate#32. The Kagent dependency must also be bumped after that chart is released.

Summary

• Default embedded Substrate to Kagent's selected PostgreSQL database while keeping Substrate objects in a dedicated substrate schema.
• Disable Substrate's bundled PostgreSQL so the default installation deploys only Kagent's bundled database.
• Create a release-scoped Substrate connection Secret from either database.postgres.url or Kagent's bundled PostgreSQL URL.
• Replace database.postgres.urlFile with the Kubernetes-native database.postgres.secretRef.{name,key} configuration.
• Keep ownership explicit: Kagent reads database.postgres.*, while embedded Substrate reads substrate.postgres.*.
• Allow Kagent and Substrate to share one DSN, use separate connections to the same server and database, or use entirely separate databases.
• Expose separate Secret-backed or inline Substrate runtime/DML and DDL connections.
• Document bundled, external, single-connection, and split-role configurations.

Configuration

The default development configuration shares Kagent's bundled database and uses separate schemas:

substrate:
  enabled: true

For an external database with separate Kagent, Substrate runtime, and Substrate DDL accounts:

database:
  postgres:
    bundled:
      enabled: false
    secretRef:
      name: kagent-postgres
      key: connectionString

substrate:
  enabled: true
  postgres:
    enabled: false
    schema: substrate
    connectionStringSecretRef:
      name: substrate-postgres
      key: runtimeConnectionString
    ddlConnectionStringSecretRef:
      name: substrate-postgres
      key: ddlConnectionString

Both Substrate connections may point to Kagent's PostgreSQL server and database. The dedicated schema prevents the generic Substrate runtime grants from covering Kagent tables. The DDL role owns and migrates the Substrate schema; the runtime role receives only the table and sequence access needed by ateapi.

Omitting the DDL connection preserves single-connection operation. Existing Secrets can also be shared by setting the same name and key under database.postgres.secretRef and substrate.postgres.connectionStringSecretRef.

[smeegoan]

Three gaps after this PR.

--postgres-schema defaults to public, and atepg.go assigns RuntimeParams["search_path"] unconditionally after parsing the DSN. In the shared-Secret setup this PR documents, a connection string carrying search_path=agents is therefore overridden to public on the ate-api-server side, silently, unless the flag is set to match.

Migrations cannot use a separate identity. RunUp runs in-process on the same connection string before the pool opens, so the role serving traffic must also hold DDL rights on its own schema, and every migrated object ends up owned by it. A schema owner distinct from its runtime consumer is not expressible.

Rotation has no input left. POSTGRES_DATABASE_URL is an env var, frozen for the life of the process. urlFile was the only input a rotation system could write to, and passfile is no substitute because ParseConfig resolves that once too. Is rotation meant to work another way?

[iplay88keys]

@smeegoan, thanks for taking a look! Just updating this PR and reviewing your comment. Just a heads up that Codex helped put together this response, though I reviewed it.

--postgres-schema defaults to public, and atepg.go assigns RuntimeParams["search_path"] unconditionally after parsing the DSN. In the shared-Secret setup this PR documents, a connection string carrying search_path=agents is therefore overridden to public on the ate-api-server side, silently, unless the flag is set to match.

This PR defaults substrate.postgres.schema to substrate, which becomes the explicit --postgres-schema=substrate argument. We intentionally do not rely on the DSN’s search_path; users can override the Substrate schema value.

Migrations cannot use a separate identity. RunUp runs in-process on the same connection string before the pool opens, so the role serving traffic must also hold DDL rights on its own schema, and every migrated object ends up owned by it. A schema owner distinct from its runtime consumer is not expressible.

Kagent already supports this operationally. An operator can create the schema/grants and run kagent db migrate up --db-url <migrator-url> with privileged credentials, then configure the controller with restricted runtime credentials and database.postgres.skipMigrations=true. The chart does not provision those roles or grants. RunUp is skipped in that mode.

On the Substrate side, even if we added a way to pre-apply and skip startup migrations, that would not provide complete DDL/DML separation. ateapi also performs runtime DDL to create and remove outbox partitions, so its runtime role must retain those privileges.

Operators can configure Substrate with a separate connection Secret and role, even when it points to the same PostgreSQL server and database. This keeps Substrate’s DDL privileges off Kagent’s runtime role, but Substrate currently has no supported mechanism for separating its own migration/DDL and runtime/DML identities. Additional Substrate work would be required for that stricter separation.

Rotation has no input left. POSTGRES_DATABASE_URL is an env var, frozen for the life of the process. urlFile was the only input a rotation system could write to, and passfile is no substitute because ParseConfig resolves that once too. Is rotation meant to work another way?

urlFile was read only once during startup, so changing it did not rotate an active pool. A Secret-backed environment variable has the same restart requirement. Live rotation would require explicit reload and pool reconnection support.

So the PR does not remove an existing Kagent credential-separation or live-rotation capability. The real follow-up is adding equivalent migration-role separation to Substrate if that is required for shared production databases.