Separate MySQL and MariaDB tests by ilidemi · Pull Request #4517 · PeerDB-io/peerdb

ilidemi · 2026-06-30T07:27:11Z

Changes:

Incorporate DBI-829: MariaDB as iteration dimension #4516 making Maria a separate DB in the matrix
Spawn out the maria-roots of e2e suites
Introduce TestIntegraion.../mysql and .../mariadb naming pattern for integration tests, filter from tilt with a matcher. MySQL-only are TestMySQLOnlyIntegration... to avoid running them from maria targets as parent tests are first ran to discover subtests. Unit tests are out of scope for tilt as they don't need any infra

Also GitHub seems to not handle indentation diffs very well, most of the line changes are just subtest indentation

codecov · 2026-06-30T07:29:04Z

❌ 8 Tests Failed:

Tests completed	Failed	Passed	Skipped
2653	8	2645	350

View the top 3 failed test(s) by shortest run time

github.com/PeerDB-io/peerdb/flow/connectors/mysql::TestMySQLOnlyIntegrationCloseSyncerWithTimeout

Stack Traces | 0.03s run time

=== RUN   TestMySQLOnlyIntegrationCloseSyncerWithTimeout
=== PAUSE TestMySQLOnlyIntegrationCloseSyncerWithTimeout
=== CONT  TestMySQLOnlyIntegrationCloseSyncerWithTimeout
2026/06/30 07:49:19 INFO Setting up SSH connection x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} Server=localhost:2222
2026/06/30 07:49:19 INFO Setting up SSH connection x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} Server=localhost:12001
2026/06/30 07:49:19 INFO Setting up SSH connection x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} Server=localhost:2222
2026/06/30 07:49:19 INFO Setting up SSH connection x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} Server=localhost:12003
2026/06/30 07:49:19 INFO Setting up SSH connection x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} Server=localhost:2222
2026/06/30 07:49:19 INFO Setting up SSH connection x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} Server=localhost:12005
2026/06/30 07:49:19 INFO Setting up SSH connection x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} Server=localhost:12002
2026/06/30 07:49:19 INFO Setting up SSH connection x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} Server=localhost:12004
    ssh_keepalive_test.go:293: 
        	Error Trace:	.../connectors/mysql/ssh_keepalive_test.go:105
        	            				.../connectors/mysql/ssh_keepalive_test.go:293
        	Error:      	Received unexpected error:
        	            	MySQL execute error: readInitialHandshake: io.ReadFull(header) failed. err EOF: connection was bad
        	Test:       	TestMySQLOnlyIntegrationCloseSyncerWithTimeout
        	Messages:   	Initial connection should work
--- FAIL: TestMySQLOnlyIntegrationCloseSyncerWithTimeout (0.03s)

github.com/PeerDB-io/peerdb/flow/connectors/mysql::TestMySQLOnlyIntegrationSSHKeepaliveTunnelDown

Stack Traces | 0.03s run time

=== RUN   TestMySQLOnlyIntegrationSSHKeepaliveTunnelDown
=== PAUSE TestMySQLOnlyIntegrationSSHKeepaliveTunnelDown
=== CONT  TestMySQLOnlyIntegrationSSHKeepaliveTunnelDown
    ssh_keepalive_test.go:129: 
        	Error Trace:	.../connectors/mysql/ssh_keepalive_test.go:68
        	            				.../connectors/mysql/ssh_keepalive_test.go:114
        	            				.../connectors/mysql/ssh_keepalive_test.go:129
        	Error:      	Received unexpected error:
        	            	MySQL execute error: ssh: rejected: connect failed (Connection refused)
        	Test:       	TestMySQLOnlyIntegrationSSHKeepaliveTunnelDown
        	Messages:   	Initial connection should work
--- FAIL: TestMySQLOnlyIntegrationSSHKeepaliveTunnelDown (0.03s)

github.com/PeerDB-io/peerdb/flow/connectors/mysql::TestMySQLOnlyIntegrationSyncerClose

Stack Traces | 0.03s run time

=== RUN   TestMySQLOnlyIntegrationSyncerClose
=== PAUSE TestMySQLOnlyIntegrationSyncerClose
=== CONT  TestMySQLOnlyIntegrationSyncerClose
    ssh_test.go:47: 
        	Error Trace:	.../connectors/mysql/ssh_test.go:38
        	            				.../connectors/mysql/ssh_test.go:47
        	Error:      	Received unexpected error:
        	            	MySQL execute error: ssh: rejected: connect failed (Connection refused)
        	Test:       	TestMySQLOnlyIntegrationSyncerClose
--- FAIL: TestMySQLOnlyIntegrationSyncerClose (0.03s)

github.com/PeerDB-io/peerdb/flow/connectors/mysql::TestMySQLOnlyIntegrationBinlogStalenessThreshold

Stack Traces | 0.04s run time

=== RUN   TestMySQLOnlyIntegrationBinlogStalenessThreshold
=== PAUSE TestMySQLOnlyIntegrationBinlogStalenessThreshold
=== CONT  TestMySQLOnlyIntegrationBinlogStalenessThreshold
    ssh_keepalive_test.go:334: 
        	Error Trace:	.../connectors/mysql/ssh_keepalive_test.go:105
        	            				.../connectors/mysql/ssh_keepalive_test.go:334
        	Error:      	Received unexpected error:
        	            	MySQL execute error: readInitialHandshake: io.ReadFull(header) failed. err EOF: connection was bad
        	Test:       	TestMySQLOnlyIntegrationBinlogStalenessThreshold
        	Messages:   	Initial connection should work
--- FAIL: TestMySQLOnlyIntegrationBinlogStalenessThreshold (0.04s)

github.com/PeerDB-io/peerdb/flow/connectors/mysql::TestMySQLOnlyIntegrationSSHKeepaliveCDCCloseHang

Stack Traces | 0.04s run time

=== RUN   TestMySQLOnlyIntegrationSSHKeepaliveCDCCloseHang
=== PAUSE TestMySQLOnlyIntegrationSSHKeepaliveCDCCloseHang
=== CONT  TestMySQLOnlyIntegrationSSHKeepaliveCDCCloseHang
    ssh_keepalive_test.go:238: 
        	Error Trace:	.../connectors/mysql/ssh_keepalive_test.go:68
        	            				.../connectors/mysql/ssh_keepalive_test.go:238
        	Error:      	Received unexpected error:
        	            	MySQL execute error: ssh: rejected: connect failed (Connection refused)
        	Test:       	TestMySQLOnlyIntegrationSSHKeepaliveCDCCloseHang
        	Messages:   	Initial connection should work
--- FAIL: TestMySQLOnlyIntegrationSSHKeepaliveCDCCloseHang (0.04s)

github.com/PeerDB-io/peerdb/flow/connectors/mysql::TestMySQLOnlyIntegrationSSHKeepaliveCDCHang

Stack Traces | 0.04s run time

=== RUN   TestMySQLOnlyIntegrationSSHKeepaliveCDCHang
=== PAUSE TestMySQLOnlyIntegrationSSHKeepaliveCDCHang
=== CONT  TestMySQLOnlyIntegrationSSHKeepaliveCDCHang
    ssh_keepalive_test.go:188: 
        	Error Trace:	.../connectors/mysql/ssh_keepalive_test.go:68
        	            				.../connectors/mysql/ssh_keepalive_test.go:188
        	Error:      	Received unexpected error:
        	            	MySQL execute error: ssh: rejected: connect failed (Connection refused)
        	Test:       	TestMySQLOnlyIntegrationSSHKeepaliveCDCHang
        	Messages:   	Initial connection should work
--- FAIL: TestMySQLOnlyIntegrationSSHKeepaliveCDCHang (0.04s)

github.com/PeerDB-io/peerdb/flow/connectors/mysql::TestMySQLOnlyIntegrationSSHKeepaliveLatency

Stack Traces | 0.04s run time

=== RUN   TestMySQLOnlyIntegrationSSHKeepaliveLatency
=== PAUSE TestMySQLOnlyIntegrationSSHKeepaliveLatency
=== CONT  TestMySQLOnlyIntegrationSSHKeepaliveLatency
    ssh_keepalive_test.go:136: 
        	Error Trace:	.../connectors/mysql/ssh_keepalive_test.go:68
        	            				.../connectors/mysql/ssh_keepalive_test.go:114
        	            				.../connectors/mysql/ssh_keepalive_test.go:136
        	Error:      	Received unexpected error:
        	            	MySQL execute error: ssh: rejected: connect failed (Connection refused)
        	Test:       	TestMySQLOnlyIntegrationSSHKeepaliveLatency
        	Messages:   	Initial connection should work
--- FAIL: TestMySQLOnlyIntegrationSSHKeepaliveLatency (0.04s)

github.com/PeerDB-io/peerdb/flow/connectors/mysql::TestMySQLOnlyIntegrationSSHResetPeer

Stack Traces | 0.04s run time

=== RUN   TestMySQLOnlyIntegrationSSHResetPeer
=== PAUSE TestMySQLOnlyIntegrationSSHResetPeer
=== CONT  TestMySQLOnlyIntegrationSSHResetPeer
    ssh_keepalive_test.go:143: 
        	Error Trace:	.../connectors/mysql/ssh_keepalive_test.go:68
        	            				.../connectors/mysql/ssh_keepalive_test.go:114
        	            				.../connectors/mysql/ssh_keepalive_test.go:143
        	Error:      	Received unexpected error:
        	            	MySQL execute error: ssh: rejected: connect failed (Connection refused)
        	Test:       	TestMySQLOnlyIntegrationSSHResetPeer
        	Messages:   	Initial connection should work
--- FAIL: TestMySQLOnlyIntegrationSSHResetPeer (0.04s)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

github-actions · 2026-06-30T07:31:33Z

❌ Test Failure

Analysis: A real workflow misconfiguration on the separate-mysql-mariadb-tests branch left DB_IMG/DB_ENV empty for some matrix combinations, causing docker run -e to fail deterministically (exit 125) and downstream test-result ingestion to fail.
Confidence: 0.95

⚠️ This appears to be a real bug - manual intervention needed

View workflow run

github-actions · 2026-06-30T07:52:33Z

🔄 Flaky Test Detected

Analysis: All failures are MariaDB/MySQL test suites failing instantly at setup with "connection refused"/EOF errors against localhost:3306, indicating the MariaDB service container failed to start or be ready — an infrastructure flake, not a code bug.
Confidence: 0.93

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-06-30T08:09:12Z

❌ Test Failure

Analysis: The PR's restructured TestMySQLOnlyIntegration* SSH-tunnel tests in flow/connectors/mysql fail deterministically and identically across all three MySQL matrix jobs with instant "ssh: rejected: connect failed (Connection refused)" errors, indicating a real PR-introduced bug in the SSH-tunnel-to-MySQL test setup rather than a flaky/intermittent failure.
Confidence: 0.9

⚠️ This appears to be a real bug - manual intervention needed

View workflow run

github-actions · 2026-06-30T08:17:49Z

❌ Test Failure

Analysis: Eight newly-added SSH-tunnel MySQL integration tests (TestMySQLOnlyIntegration*, nonexistent on main) fail 100% deterministically across both independent matrix jobs with "ssh: rejected: connect failed (Connection refused)" at initial connection, indicating a real setup/code defect in PR #4517's new tests rather than intermittent flakiness.
Confidence: 0.85

⚠️ This appears to be a real bug - manual intervention needed

View workflow run

dtunikov

looks good to me
i would maybe add smth to claude.md to make it more likely to catch cases when someone adds a test for mysql and forgets about mariadb (and vice versa), it mostly affects integration tests (not e2e suite):

func TestIntegrationGetTableSchemaCaseSensitiveIdentifiers(t *testing.T) {
	for _, tc := range []struct {
		name string
	}{
		{name: "mysql"},
		{name: "mariadb"},
	} {
		t.Run(tc.name, func(t *testing.T) {

pfcoperez · 2026-06-30T08:42:31Z

+func TestIntegrationMySQLGetMasterGTIDSet(t *testing.T) {
+	for _, tc := range []struct {
+		name string
+	}{


Can't this struct:

{ {name: "mysql"}, {name: "mariadb"}, }

Be factored out as a common constant or the table test structure factored out as a test helper?

This way we don't only stop repeating ourselves but also prevent accidentally missing one of the flavors.

It can but we then would lose the IDE codelenses as the VS Code extensions are not that smart to match through the layers of indirection

pfcoperez

Proceeding to force merge and update the branch protection

claude · 2026-06-30T09:11:02Z

Code review

One issue found. Checked for bugs and CLAUDE.md compliance.

Dependency version change in `.github/workflows/flow.yml` (lines 54-64)

Per .claude/REVIEW.md: When changes are bumping dependencies, check whether the version bump brings regressions. Read the upstream release notes and commit log for the bumped range.

This PR introduces new MariaDB Docker image versions and removes the previously pinned digest:

mariadb:lts-ubi9 -- was pinned to @sha256:55a81b2d..., now uses a floating tag
mariadb:12-ubi -- new major version
mariadb:13.0-ubi-rc -- new major version, release candidate (inherently unstable -- may cause CI flakiness)

See lines 54-64.

Please confirm that upstream MariaDB 12 and 13 release notes have been reviewed for breaking changes affecting the MySQL connector, and consider whether the RC image is appropriate for CI.

ilidemi · 2026-06-30T09:38:06Z

to make it more likely to catch cases when someone adds a test for mysql and forgets about mariadb (and vice versa)

Addressed in #4518. When this one is merged, I thiiink that one should switch to main as base and kick off the CI.

pfcoperez and others added 3 commits June 30, 2026 07:17

MariaDB as iteration dimension

86275f5

Separate MySQL and Maria tests

ec9a39a

lint

18a1696

ilidemi added 2 commits June 30, 2026 00:31

fix flow

d44e805

Separate peer name for maria

a9230ff

dtunikov approved these changes Jun 30, 2026

View reviewed changes

Fix SSH host

bc6e475

pfcoperez reviewed Jun 30, 2026

View reviewed changes

pfcoperez marked this pull request as ready for review June 30, 2026 08:54

pfcoperez requested a review from a team as a code owner June 30, 2026 08:54

pfcoperez approved these changes Jun 30, 2026

View reviewed changes

ilidemi mentioned this pull request Jun 30, 2026

Enforce the naming convention for MySQL integration tests #4518

Open

pfcoperez merged commit a7eee79 into main Jun 30, 2026
23 checks passed

pfcoperez deleted the separate-mysql-mariadb-tests branch June 30, 2026 09:43

pfcoperez mentioned this pull request Jun 30, 2026

DBI-829: MariaDB as iteration dimension #4516

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Separate MySQL and MariaDB tests - #4517

Separate MySQL and MariaDB tests#4517
pfcoperez merged 6 commits into
mainfrom
separate-mysql-mariadb-tests

ilidemi commented Jun 30, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 30, 2026

Uh oh!

github-actions Bot commented Jun 30, 2026

Uh oh!

github-actions Bot commented Jun 30, 2026

Uh oh!

github-actions Bot commented Jun 30, 2026

Uh oh!

dtunikov left a comment

Uh oh!

pfcoperez Jun 30, 2026

Uh oh!

ilidemi Jun 30, 2026 •

edited

Loading

Uh oh!

pfcoperez left a comment

Uh oh!

claude Bot commented Jun 30, 2026

Uh oh!

ilidemi commented Jun 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

ilidemi commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 8 Tests Failed:

Uh oh!

github-actions Bot commented Jun 30, 2026

❌ Test Failure

Uh oh!

github-actions Bot commented Jun 30, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented Jun 30, 2026

❌ Test Failure

Uh oh!

github-actions Bot commented Jun 30, 2026

❌ Test Failure

Uh oh!

dtunikov left a comment

Choose a reason for hiding this comment

Uh oh!

pfcoperez Jun 30, 2026

Choose a reason for hiding this comment

Uh oh!

ilidemi Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pfcoperez left a comment

Choose a reason for hiding this comment

Uh oh!

claude Bot commented Jun 30, 2026

Code review

Dependency version change in .github/workflows/flow.yml (lines 54-64)

Uh oh!

ilidemi commented Jun 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ilidemi commented Jun 30, 2026 •

edited

Loading

codecov Bot commented Jun 30, 2026 •

edited

Loading

ilidemi Jun 30, 2026 •

edited

Loading

Dependency version change in `.github/workflows/flow.yml` (lines 54-64)