feat(server): add the CRM extension behind an optional second D1 database

[dcondrey]

Facet has never been able to hold a customer record, and the obvious way to add one would have quietly broken the thing being sold. Contacts are people who handed their details over directly. Analytics visitors are salted hashes with no cross-session identity. Put both in one database and the join between them is a foreign key someone adds later, with nothing structural in the way — the privacy claim would then depend on a code review noticing.

So the CRM is a separate D1 database, bound as the optional CRM_DB, mounted in the same Worker. With no binding there is no database, no migration, no table, and every /api/crm route returns 501 crm_unavailable — the same gate AE, AI, FACET_SIGNING_JWK, WEBHOOK_URL, SCITT_URL and SESSION_SECRET already use, and the only arrangement where "excluded" means the tables do not exist rather than that a flag hides them. It also buys the property this rests on: D1 cannot join across databases, so the contact→analytics link cannot be a foreign key. It has to be assembled in the Worker, which is what forces it through the consent check instead of past it.

The only bridge is external_user_id behind an active identified-tier consent record, and the column is only an index. findLinkedVisitorHashes returns the visitor hash out of the signed claims — verified against the deployment key, pinned by proof.kid, and required to assert this site and this tier — never out of the visitor_hash column beside it. A row hand-written into consent_records pointing at an arbitrary victim's hash therefore links nothing. events gains no person-keyed column, and a contact with no consent row has zero analytics linkage. The pinning is shared with verifyConsentRecord rather than copied, because the two callers bind different things afterwards: ingest knows the hash and window up front, the CRM derives the hash from the statement.

Nothing in the CRM caches a derived visitor hash, which is what makes retention work with no CRM-side purge at all. Contacts are deliberately not on the raw-event schedule — a contact is a business record with its own lifecycle, not telemetry — but the link is: once enforceRetention drops the consent record, the join simply stops resolving, and there is no stale copy anywhere to forget. A test ages a grant past the window, runs the ordinary purge job, and asserts the link severs while the contact survives untouched.

Erasure is real deletion and ships in v1, not as a follow-up. No tombstone, because a tombstone still holding an email is still that person's data. Deleting a contact also erases rather than revokes their consent rows: revoked_at would stop future elevation but leave the raw external_user_id at rest, which is precisely what the request was about. The pseudonymous event rows stay — with the consent record gone nothing can re-associate them with a person, and destroying the link is what erasing the identifiable data means here. A per-contact data-subject export lands alongside it, carrying the signed statements verbatim (they are PII-free by construction, so they add cryptographic evidence of what was consented to without widening what the export discloses) and stating events_truncated explicitly rather than silently returning a prefix that would read as a complete answer to a subject-access request.

Auth is a session cookie and deliberately never an API key. This is the one authenticated surface in the codebase that refuses clk_ keys, and the asymmetry is the point: a key is not secret the way a session is. /llms.txt advertises where to send one, and a public demo dashboard can ship with one compiled in (VITE_FACET_DEMO_API_KEY). A leaked key costs aggregate pageview counts; a key that could read /api/crm/contacts would cost customers' names, emails and phone numbers. analyst reads and writes, admin deletes and exports, and viewer — who can see aggregate analytics — gets nothing, because PII is a different kind of access rather than more of the same one. ADMIN_TOKEN is refused too, so the rule is uniform rather than mostly-uniform.

Binding this changes what the deployment is, so it changes what the deployment attests. privacyDpvClaims() took no env and hardcoded legitimate interest and pseudonymisation while being served at /.well-known/facet-privacy.json and embedded in the signed PrivacyAttestationCredential — so binding CRM_DB without touching it would have made the deployment cryptographically sign a false statement about itself. It now takes env: with the CRM bound it adds dpv:Store and dpv:Erase, adds dpv:Consent alongside legitimate interest, names dpv:CustomerRelationshipManagement, declares the pd: categories actually held, and stops letting dpv:Pseudonymisation stand as the sole measure while contact details sit in the clear. Every term was checked against the DPV 2.1 spec rather than recalled. Three existing tests asserted the old constant and now assert the shape the binding produces.

Migration tooling was single-database throughout. A second drizzle config, schema and migrations directory land with their own generate/apply scripts; vitest reads both sets, and the test-config generator injects CRM_DB with a distinct database_id — miniflare keys local databases by that id, so reusing one string would have silently made CRM_DB an alias of DB and destroyed the isolation the whole design exists for. A test asserts contacts is absent from the analytics database and events from the CRM one, so that claim is checked rather than assumed. wrangler.test.jsonc regenerates byte-identically apart from the new binding, verified against the tracked config rather than the local one. The shipped wrangler.jsonc carries the block commented out, with what enabling it implies spelled out next to it.

What the risk is not: no new identifier, no new column on events, no change to any analytics table, no change to ingest, and no migration for the existing database. Nothing in this PR alters a single existing route or response shape. A deployment that does not bind CRM_DB — which is every deployment until someone runs wrangler d1 create facet-crm — behaves byte-for-byte as before, right down to the DPV claims it serves and signs.

Two things found along the way that had to land with it. sites.team_id was writable by nothing in the shipped code: siteRole therefore always returned null, the dashboard-session branch of requireSiteAccess was unreachable in production, and the entire accounts/RBAC surface was exercisable only by a test setting the column with raw SQL. Gating contact PII on a team role would have meant gating it on something no operator could turn on, so PATCH /api/sites/:id/team closes it. And contactActivity counted pageviews as name = 'pageview' when this schema defines a pageview as name IS NULL — it would have reported zero pageviews for every real visitor, while the test fixture, written under the same wrong assumption, kept agreeing with it. It now imports pageviewCount/eventCount from db/stats.ts, so one contact's numbers cannot disagree with what /api/stats reports for the same rows. Separately, the duplicate-contact check walked only err.message, but drizzle wraps driver errors with the D1 error as cause, so a collision returned 500 instead of 409; it now walks the bounded cause chain.

Left out on purpose: no dashboard UI consumes any of this yet, and companies, deals and activities are v2–v4. Each is shippable alone on top of this.

The new safety assertions were mutation-checked — disabling the consent signature pinning fails exactly the forgery test, letting requireTeamRole fall through to API-key auth fails exactly the key-rejection test, and removing the requireCrm gate fails exactly the two 501 tests. The forged payload in that first test is deliberately complete (right site, right tier, uid-present, victim's hash) so that every non-cryptographic field check passes and only the crypto can reject it; a sparser payload would have passed with the pinning disabled.

pnpm lint, pnpm typecheck, pnpm test green: 1537 tests across 172 files, up from 1499/170.

---

Two defects found in self-review after the first commit, both in code this PR introduced, both fixed with a mutation-checked test on top.

SQLite ignores backslash escapes in LIKE unless an ESCAPE clause is present. The contact search prefixed metacharacters and stopped there, so \% was read as a literal backslash followed by the wildcard. Nothing matched — no name contains a backslash — which made the guard look correct while quietly making a literal % unsearchable. The covering test asserted only that q=% did not return everything, so it passed either way. It now seeds a contact whose name actually contains a % and requires that row back: matching neither everything nor nothing.

events_truncated compared against the wrong count. When contactActivity was moved onto the codebase's canonical pageview/custom-event split, this call site was not re-traced, so the export's cap flag compared returned rows against custom events only. A contact whose traffic is entirely pageviews — the common case — has zero of those, so the flag read false while a thousand rows were dropped. That is a subject-access request answered incorrectly, which is the one thing the flag exists to prevent. Now compared against the row total, covered by a test that seeds past the cap with pageviews specifically, since custom-event traffic would have passed either way.