Skip to content

Add native CockroachDB connector support - #3861

Open
viragtripathi wants to merge 1 commit into
PeerDB-io:mainfrom
viragtripathi:feat/cockroachdb-integration
Open

Add native CockroachDB connector support#3861
viragtripathi wants to merge 1 commit into
PeerDB-io:mainfrom
viragtripathi:feat/cockroachdb-integration

Conversation

@viragtripathi

@viragtripathi viragtripathi commented Jan 21, 2026

Copy link
Copy Markdown

Summary

Implements full CockroachDB integration as a source peer, following the same patterns as existing PostgreSQL and MySQL connectors.

What's included

Backend:

  • Native Go connector (490 lines) using pgx driver with CockroachDB-specific configuration
  • Schema introspection: GetSchemas, GetTables, GetColumns
  • Connection support for local and cloud deployments
  • TLS support (disabled by default for easier local development, matching Postgres behavior)
  • SSH tunnel support in backend (intentionally no UI - CockroachDB Cloud doesn't need SSH)
  • Version validation and variant detection (Cloud/Dedicated/Serverless)
  • Rust analyzer and catalog support for peer configuration

Frontend:

  • Complete UI integration for peer creation
  • Form validation and field helpers
  • CockroachDB logo and proper display names

Testing performed

  • ✅ UI peer creation with form validation
  • ✅ API peer creation and validation
  • ✅ Local CockroachDB connection (insecure mode)
  • ✅ Cloud CockroachDB connection (with TLS)
  • ✅ Schema introspection

Design Decisions

TLS Default: Set to false (matching Postgres) for easier local development. Cloud users will enable it as needed.

SSH Support: Backend implementation included but no UI exposed. CockroachDB Cloud deployments (primary use case) don't require SSH tunnels, keeping the UI simpler and more focused.

Note on SQL syntax

SQL syntax support (CREATE PEER FROM COCKROACHDB) requires updates to the sqlparser dependency and will be added in a follow-up PR.

Files changed

23 files changed, 936 insertions(+), 2 deletions(-)

@CLAassistant

CLAassistant commented Jan 21, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@viragtripathi
viragtripathi force-pushed the feat/cockroachdb-integration branch from eb56599 to 31e5672 Compare January 23, 2026 03:08
@viragtripathi
viragtripathi force-pushed the feat/cockroachdb-integration branch from 31e5672 to f16c84e Compare January 23, 2026 03:36
@serprex
serprex requested a review from jgao54 February 4, 2026 19:11
@serprex
serprex requested a review from Amogh-Bharadwaj April 6, 2026 04:06
@viragtripathi
viragtripathi force-pushed the feat/cockroachdb-integration branch from f16c84e to 2f33f5d Compare July 23, 2026 02:37
@viragtripathi
viragtripathi requested a review from a team as a code owner July 23, 2026 02:37
@viragtripathi
viragtripathi requested a deployment to external-contributor July 23, 2026 02:37 — with GitHub Actions Waiting
@viragtripathi
viragtripathi requested a deployment to external-contributor July 23, 2026 02:37 — with GitHub Actions Waiting
@viragtripathi
viragtripathi requested a deployment to external-contributor July 23, 2026 02:37 — with GitHub Actions Waiting
@viragtripathi
viragtripathi requested a deployment to external-contributor July 23, 2026 02:37 — with GitHub Actions Waiting
@viragtripathi

Copy link
Copy Markdown
Author

Rebased onto current main and expanded this from a peer-only connector to a full source integration:

  • CREATE PEER ... FROM COCKROACHDB now goes through the in-tree parser (following Drop sqlparser-rs (0.45) fork in favor of upstream (0.61) #4145), so the sqlparser fork dependency is gone and I am closing Add CockroachDB PeerType support sqlparser-rs#39
  • QRep replication using AS OF SYSTEM TIME partitioned reads with watermark based partitioning
  • CDC via sinkless changefeeds: resolved timestamp checkpoints, cursor resume with backoff, gapless snapshot handoff via cluster_logical_timestamp(), delete key fallback from the changefeed message key, replay dedup, and SQLSTATE based error classification. Changefeed webhook sink support is a natural follow-up for higher throughput deployments.
  • Unit tests, an e2e suite, and a CockroachDB service container wired into CI, plus Tilt dev environment wiring matching the other source databases

Tested against CockroachDB v25.4.13 (e2e suite, single node insecure) and a CockroachDB Cloud v26.2.1 cluster over verify-full TLS for connectivity, validation and schema introspection. A docs PR to peerdb-io/docs will follow once this merges.

Implements CockroachDB as a source peer with:
- Native Go connector using the pgx driver (peer creation, validation,
  version gating, variant detection, schema introspection)
- SQL syntax support (CREATE PEER ... FROM COCKROACHDB) via the in-tree
  parser, with nexus query routing through the Postgres executor since
  CockroachDB is wire compatible
- QRep replication with AS OF SYSTEM TIME partitioned reads and watermark
  based partitioning
- CDC via sinkless changefeeds: resolved timestamp checkpoints, cursor
  resume with backoff, gapless snapshot handoff using
  cluster_logical_timestamp, delete key fallback, replay dedup, SQLSTATE
  based error classification
- Complete UI integration including a changefeeds toggle
- Unit tests, an e2e suite, and CockroachDB wired into CI

Tested against CockroachDB v25.4.13 single node (insecure) with the e2e
suite and against a CockroachDB Cloud v26.2.1 cluster (TLS verify-full)
for connectivity, validation and schema introspection.
@viragtripathi
viragtripathi force-pushed the feat/cockroachdb-integration branch from 2f33f5d to 0fa9e5a Compare July 24, 2026 22:41
@viragtripathi
viragtripathi requested a deployment to external-contributor July 24, 2026 22:41 — with GitHub Actions Waiting
@viragtripathi
viragtripathi requested a deployment to external-contributor July 24, 2026 22:41 — with GitHub Actions Waiting
@viragtripathi
viragtripathi requested a deployment to external-contributor July 24, 2026 22:41 — with GitHub Actions Waiting
@viragtripathi
viragtripathi requested a deployment to external-contributor July 24, 2026 22:41 — with GitHub Actions Waiting
@viragtripathi

Copy link
Copy Markdown
Author

Updated: the rangefeed validation check queried crdb_internal.cluster_settings, which is gated behind allow_unsafe_internals from v26.1 onward, so on newer clusters the check silently returned nothing and mirror validation passed without verifying rangefeeds. Switched it (and the database variant lookup) to SHOW CLUSTER SETTING, which only needs VIEWCLUSTERSETTING, and added an e2e test covering the rangefeed validation path against a live cluster, including the disabled-setting error case.

Planned follow-ups: changefeed sink transports (webhook and Kafka) for deployments that need changefeed job semantics, protected timestamps through large initial scans, and DB Console metrics. The changefeed envelope parsing and checkpointing in this PR are transport independent, so those land as additional transports rather than a redesign.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants