Skip to content

chore(pg): remove native pg binding, compile real package from source - #10677

Draft
proggeramlug wants to merge 3 commits into
fix/10437-cjs-conditional-requirefrom
wip/pg-native-binding-removal
Draft

proggeramlug wants to merge 3 commits into
fix/10437-cjs-conditional-requirefrom
wip/pg-native-binding-removal

Conversation

@proggeramlug

@proggeramlug proggeramlug commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Deletes the native pg binding (crates/perry-ext-pg, sqlx::postgres + tokio bridge) and the duplicate pre-#466 in-tree pg implementation living inside crates/perry-stdlib/src/pg/ (the bundled-pg Cargo feature), plus every registry entry that pointed at either one. import ... from "pg" no longer resolves as a native module at all — it now compiles the real npm pg package from source, same as any other TypeScript/JavaScript dependency.

Must not merge before #10674 (fix/10437-cjs-conditional-require) — pg does not run without that fix: a literal require('pg-native') inside a conditional guard in pg/lib/native/client.js was hoisted to an eager static import and threw at program start. This branch is based on #10674's head (b1ba0caf5), not main.

Why two implementations

crates/perry-ext-pg (the registered [bindings.pg] binding, sqlx::postgres + perry-ffi) was the one actually wired up. But crates/perry-stdlib/src/pg/ turned out to be a second, older, full native reimplementation of pg (its own module doc: "pg compatible native implementation... drop-in replacement for the pg npm package using sqlx"), kept around behind a bundled-pg feature since before the #466 migration to a separate ext crate. It defined the exact same extern "C" symbol names (js_pg_client_new, js_pg_client_query, …) as perry-ext-pg, so in any build that linked both, the linker's link-order (perry-ext wins) silently discarded the stdlib copy — but it still compiled into every build that also used mysql2 (its #[cfg] was any(bundled-pg, bundled-mysql2)). Verified zero cross-references from mysql2's module before deleting it, so this is a clean removal, not a partial one.

What was deleted

  • crates/perry-ext-pg/ (Cargo.toml + 750-line lib.rs)
  • crates/perry-stdlib/src/pg/ (929 lines: connection.rs, pool.rs, result.rs, types.rs, mod.rs) and the bundled-pg / database-postgres Cargo features that gated it
  • the now-unused "postgres" sqlx feature on perry-stdlib's shared sqlx dependency (verified nothing outside the deleted pg/ module referenced sqlx::postgres/PgPool/etc. — mysql2's own sqlx dependency never requested it)
  • registry entries: crates/perry/well_known_bindings.toml ([bindings.pg] + upstream pin), NATIVE_MODULES in crates/perry-api-manifest/src/entries.rs, the manifest method/class rows in entries/part_1.rs and part_3.rs, the pg row in crates/perry-codegen/src/lower_call/native_table/databases.rs (7 NativeModSig rows — caught by every_dispatch_entry_has_manifest_counterpart, which fails on drift between this table and the manifest), crates/perry/src/commands/stdlib_features.rs, the bundled-pg/pg entries in optimized_libs/driver.rs and optimized_libs/freshness.rs, the workspace Cargo.toml member + path-dependency entries, and the perry-ext-pg entry in workspace-architecture.json

What was deliberately left alone

A grep sweep turned up roughly a dozen more "pg"-literal matches deep in perry-hir (local_natives.rs, native_new.rs, native_fetch.rs, module_decl.rs, stmt.rs, expr_call/static_and_instance.rs, expr_assign.rs) and perry-codegen (lower_call/builtin.rs's lower_builtin_new disambiguation, codegen/opts.rs's doc comment) — all ("pg", "connect") => Some("Client")-shaped type-narrowing heuristics for the removed native binding. Checked each one: every arm is reached only through ctx.lookup_native_module() / ctx.imported_class_sources lookups that require the module to have actually been classified native at import time. Since pg can no longer classify as native, these arms are unreachable dead code, not live landmines — confirmed both by reading the gating code and empirically, since the original compilability probe (which forced real-source pg via compilePackages while pg was still registered native) already exercised this exact "real pg source, non-native path" combination successfully. perry-codegen-js/src/emit/native.rs's browser-target "pg" => throw(...) arm is similarly dead (only reached for calls already lowered as native). Left as-is per the brief's "keep the diff tight" scope — noting them here for anyone doing a future dead-code sweep.

The real post-removal experience (no compilePackages entry for pg)

Tested with a from-scratch node_modules (npm install under Node 26.5.1) and a package.json containing only:

{
  "dependencies": { "pg": "^8" }
}

No perry.compilePackages key at all. perry compile pg_test.ts printed Compile package wildcard: expanded to 14 installed package(s) — when perry.compilePackages is entirely absent, Perry's default behavior sweeps every installed node_modules package as a compile-from-source candidate, which covered pg and all 13 of its transitive deps (pg-cloudflare, pg-connection-string, pg-pool, pg-protocol, pg-types, pgpass, pg-int8, postgres-array, postgres-date, postgres-interval, postgres-bytea, split2, xtend) automatically. It compiled all 43 modules, linked, and produced a 24.7 MB binary.

So: for a project where pg is the only (or main) native-ish dependency, no perry.compilePackages configuration is needed at all — just the ordinary npm "pg": "^8" entry. For a project with a larger, more heterogeneous node_modules (where blindly wildcard-compiling everything installed isn't desirable — some packages may not be TypeScript-subset-compatible), the recommended path is still an explicit perry.compilePackages list naming pg and the same 13 transitive packages above, scoping compilation precisely instead of relying on the wildcard.

Running the compiled binary against a Client({ host: "127.0.0.1", port: 5432, ... }) / client.connect() (no Postgres listening on this host): RESULT: ERROR Connection refused (os error 111) — a genuine OS-level net.connect() failure from the real npm pg source, reached with zero native-pg code anywhere in the tree. This is the same milestone the original compilability probe hit (which needed compilePackages forcing while the native binding still existed); this PR reaches it as the default, unforced behavior. No live Postgres was available to test a real query round-trip — that remains unverified, stated plainly rather than implied.

Validation (host: perrymaster, Node 26.5.1 at /opt/node-v26.5.1-linux-x64; box default is 26.8.1)

  • Build: cargo build --profile perry-dev -p perry -p perry-runtime-static -p perry-stdlib-static — clean, single invocation, confirmed the .a/binary mtimes moved after the final edit (an earlier E2E attempt caught a real coherence bug from building perry before the last source edit landed — a stale/current source-fingerprint mismatch between the compiler and a freshly auto-optimize-built runtime archive; rebuilding both together in one invocation fixed it, consistent with the Archives from separate cargo invocations can bundle different tokio builds off one Cargo.lock; the link guard catches it but two agents hit it today in unrelated work #10671 archive-coherence warning in the brief).
  • cargo test, all perry-dev:
    • perry-api-manifest: 39/39
    • perry-hir (lib + every integration test file, including unimplemented_api_check.rs): 459 lib tests + all integration suites, 0 failures
    • perry-codegen: all 36 integration test targets green (including manifest_consistency's every_dispatch_entry_has_manifest_counterpart, which initially caught the dispatch-table drift this PR now fixes). perry-codegen's own lib unit tests could not be compiled — pre-existing on the fix(cjs): defer conditional CommonJS require() init instead of hoisting (#10437) #10674 base commit itself (b1ba0caf5), unrelated to this change: instanceof_imported_rhs_tests.rs / new_builtin_shadow_tests.rs construct an ImportedClass literal missing the constructor_has_synthetic_arguments field, confirmed present at b1ba0caf5 before my first commit. Not mine to fix; named explicitly here as something I did not get a pass/fail count for.
    • perry-stdlib: 139/139
    • perry --bin perry (stdlib_features + optimized_libs filtered): 52/52 (matches chore: remove Tier A native bindings (fetch alias, tursodb, iroh) #10618's precedent number). Full unfiltered suite: 1129/1130 on the first run, with one commands::compile::geisterhand::…warm_archives_are_rebuilt_as_one_runtime_graph failure that re-ran green in isolation — a transient resource-contention artifact from a heavily shared host (multiple concurrent agents running their own cargo build --release, disk at 96-99% used throughout), not a pg regression; that test doesn't touch pg/mysql2/well_known_bindings at all.
    • perry-runtime (RUST_TEST_THREADS=1, untouched by this diff): 4039 passed, 2 failed, 4 ignored. Both failures are the two known pre-existing debug_assert!-gated cases named in the campaign brief (gc::tests::copy_slot_decode::…, gc::tests::heap_generation::…), which fail under perry-dev/release by construction (both profiles compile out debug_assert!). Confirmed unrelated to this change — perry-runtime is untouched by this diff.
  • Lint (SKIP_COMPILE_GATES=1 ./scripts/run_lint_gates.sh): 76 of 77 passed (compile tier skipped per host convention, 2 CI-only skipped). The one red, "Public benchmark evidence freshness", is the pre-existing, repo-wide red named in the brief — not chased. Along the way this also caught and fixed two gates this removal touches that PR chore: remove Tier A native bindings (fetch alias, tursodb, iroh) #10618's tier-A removal never exercised: workspace_architecture.py --check (the recorded baseline object in workspace-architecture.json needed workspace_members/decision_counts refreshed — no script flag does this automatically, recomputed via the module's own helpers) and string_payload_access_inventory.py (a per-file ratchet baseline with a stale perry-ext-pg entry).
  • Governance/registry gates (the ones this removal is actually about): binding_governance.py --check OK, binding_pins.mjs --check OK, unrooted_local_shape.py --check OK (baseline refreshed), check_file_size.sh OK.

package.json a user now needs

{
  "dependencies": { "pg": "^8" }
}

No perry.compilePackages entry required when pg is the project's only real dependency (Perry's no-config wildcard covers it and its transitive deps). For a project with other, unrelated npm packages installed, list pg plus its transitive deps explicitly in perry.compilePackages instead of relying on the wildcard sweeping everything: pg, pg-cloudflare, pg-connection-string, pg-pool, pg-protocol, pg-types, pgpass, pg-int8, postgres-array, postgres-date, postgres-interval, postgres-bytea, split2, xtend.

Scope

Removal only — no other binding touched. A sibling agent is concurrently removing the axios binding on its own branch against the same shared registry files (well_known_bindings.toml, entries.rs, stdlib_features.rs, workspace Cargo.toml, workspace-architecture.json); expect a merge conflict there, resolved by the merge train, not by either PR individually.

Removes crates/perry-ext-pg (sqlx::postgres + tokio bridge) and the
duplicate pre-#466 in-tree pg implementation in
crates/perry-stdlib/src/pg/ (bundled-pg feature), plus every registry
entry that pointed at them. import ... from "pg" now falls through to
real-source compilation instead of the native binding.

wip, base = PR #10674 (fix/10437-cjs-conditional-require) since pg
does not run without that fix.
…elines

- crates/perry-codegen/src/lower_call/native_table/databases.rs: drop
  the 7 pg NativeModSig rows (js_pg_* runtime symbols that no longer
  exist). Caught by perry-codegen's every_dispatch_entry_has_manifest_counterpart
  test, which fails on drift between this table and API_MANIFEST.
- docs/api/perry.d.ts, docs/src/api/reference.md: regenerated via
  --print-api-manifest (drops the pg module section).
- docs/src/native-libraries/governance.md: regenerated via
  binding_governance.py --table (drops the perry-ext-pg row).
- docs/src/native-libraries/overview.md: pg no longer routes to an
  in-tree native wrapper; updated the well-known-binding description.
- workspace-architecture.json: refreshed the recorded baseline
  (workspace_members 83->82, externalize 33->32) that
  workspace_architecture.py --check compares against.
- scripts/string_payload_access_baseline.txt,
  scripts/unrooted_local_shape_baseline.json: refreshed ratchet
  baselines now that perry-ext-pg/perry-stdlib/src/pg no longer
  contribute findings.
@coderabbitai

coderabbitai Bot commented Sep 18, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@proggeramlug

Copy link
Copy Markdown
Contributor Author

Heads-up before this is queued: this PR's recorded workspace baseline is stale and will fail workspace_architecture.py --check on rebase.

It records workspace_members 82 / externalize 32 / keep 45. Main (023dc0b653) is at 80 / 31 / 44, so a removal landing on it must produce 79 / 30 / 44 — not 82. Five sibling removal PRs carry the identical 82/32/45, which is also mutually impossible: six different crates cannot all produce the same transition.

Full table and reasoning in #10739. The short version, for whoever rebases this:

  • Recompute from the resolved tree and let workspace_architecture.py --check --print-summary reproduce the number independently. Do not derive it from 83 by arithmetic, and do not copy a sibling's figure or one quoted in a comment — they all go stale as the queue advances.
  • The same hazard applies to scripts/native_result_ledger.tsv, scripts/string_payload_access_baseline.txt and the governance/pins tables. Regenerate with their own scripts rather than resolving by hand.
  • MERGEABLE will not catch this. The counts sit on different JSON lines from the deleted crate entry, so git auto-merges both sides without a conflict — chore(bindings): remove axios native binding, compile real axios from source #10679 rebased onto current main today, came out MERGEABLE, and still carried 82/32/45.

Also relevant to the acceptance run whenever it happens: #10735 is live on main — require.main === module is true in every compiled CommonJS module, so any dependency with a CLI entry guard runs its CLI branch when merely imported. A fix is in flight. If acceptance fails in a way that looks like the package misbehaving at import time, test a dependency-free fixture that never mentions the package before attributing it to this removal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants