feat(mysql): carry MySQL column DEFAULT through to ClickHouse on ADD COLUMN by dtunikov · Pull Request #4632 · PeerDB-io/peerdb

dtunikov · 2026-07-28T08:54:51Z

ALTER TABLE ... ADD COLUMN c INT DEFAULT 5 is DDL, not DML: MySQL makes existing rows read back as 5 without emitting a single row event for them. CDC therefore has nothing to replicate, and rows untouched since the ALTER show ClickHouse's type default (0/''/NULL) indefinitely — until a full table resync.

Emitting the DEFAULT on the destination ADD COLUMN fixes exactly that set: ClickHouse evaluates a column default at read time for parts written before the column existed, which is by construction the rows whose last change predates the ALTER. Rows changed after it carry real binlog values and win on version dedup, so the two mechanisms don't overlap.

Changes

FieldDescription.default_expr (optional string) carries a rendered SQL literal; unset means no default, or one we declined to translate.
MySQL DDL parsing renders constant defaults only, using standard SQL quote doubling so the literal stays dialect neutral. Signed literals arrive as a unary-minus wrapper and decimals as MyDecimal, both handled.
ClickHouse appends DEFAULT <literal> to ADD COLUMN, for shard and distributed tables.

Declined (unset, no DDL change): NULL (nullable columns already read back as NULL), CURRENT_TIMESTAMP/NOW()/UUID() and other expressions, bit/hex literals, and strings needing escapes beyond quote doubling. Expression defaults are deliberately not translated — read-time evaluation would give a different value per query, and MySQL gives each pre-existing row a distinct value for things like UUID().

If ClickHouse rejects a literal we translated, the ADD COLUMN is retried without it and a warning is logged. Losing a default only forfeits the historical backfill; stalling schema replay would stop the table.

Scope

MySQL source, ClickHouse destination, CDC DDL path only. Not covered: snapshot/resync CREATE TABLE (so a resync drops the default from the DDL — data stays correct since it re-copies real values), the gh-ost path (binlog row metadata carries no defaults), Postgres sources, other destinations.

Tests

TestAlterTableAddColumnDefault covers 27 rendering cases including the declined ones. Two e2e tests: Test_MySQL_AlterTableAddColumnDefault inserts a row before the ALTER so a plain source↔destination comparison only passes if the default carried across (29 type/default combinations), and Test_MySQL_AlterTableAddColumnDefaultUntranslated asserts declined and rejected defaults don't stall replication.

🤖 Generated with Claude Code

dtunikov · 2026-07-28T09:12:49Z

+
+			// retry without default if ch rejected the expression
+			addColumnWithDefaultFallback := func(tableName string) error {
+				err := addColumn(tableName, columnDef)


try to add with full col def (including default)
then retry with simple clickHouseColType if it's failed

if we want to, we can also add a metric + alert when this happens
or just fail with an error

Metric and a non-paging alert sounds like a good idea. Could also be a task to check these logs in a month, but alert is a more reliable way to have it happen and we want our defaults to be good

github-actions · 2026-07-28T09:19:58Z

❌ Test Failure

Analysis: Not flaky — Test_MySQL_AlterTableAddColumnDefault, the new test for this PR's own feature, fails deterministically on the MySQL 5.7 (mysql-pos) leg with "cannot convert bytes [98] to uint16enum" on every retry until timeout, because qvalue_convert.go has no ENUM-label handling for byte-string defaults.
Confidence: 0.93

⚠️ This appears to be a real bug - manual intervention needed

View workflow run

dtunikov · 2026-07-28T09:34:58Z

Which defaults are carried through

MySQL column	Example `DEFAULT`	Emitted on ClickHouse
`TINYINT`…`BIGINT`, signed and unsigned	`-128`, `18446744073709551615`	`-128`, `18446744073709551615`
`TINYINT(1)` / `BOOL`	`TRUE`	`1`
`YEAR`	`2021`	`2021`
`FLOAT`, `DOUBLE`	`-3.14`, `1e5`	`-3.14`, `100000`
`DECIMAL(p,s)`	`1.50`	`1.50` (scale as written)
`DATE`, `DATETIME(n)`, `TIME`	`'2020-01-02 03:04:05.678'`	`'2020-01-02 03:04:05.678'`
`CHAR`, `VARCHAR`, any charset	`'it''s'`	`'it''s'`
`ENUM`, `SET`	`'b'`, `'x,z'`	`'b'`, `'x,z'` — needs binlog row metadata, see below
`BINARY`, `VARBINARY`	`'abcd'`	`'abcd'`

Literals are rendered with SQL quote doubling rather than backslash escapes, so the value stays valid in any destination dialect.

Which are declined

The column is still added, just without a DEFAULT — identical to the behaviour before this PR.

Case	Example	Why
Expression defaults	`CURRENT_TIMESTAMP`, `NOW()`, `(UUID())`	These reach us as function calls. ClickHouse evaluates a column default when reading parts written before the column existed, so passing the expression through would make pre-`ALTER` rows return a different value on every query instead of the one constant MySQL froze at `ALTER` time.
`DEFAULT NULL`	`INT DEFAULT NULL`	Nothing to fill; the column already reads back `NULL`.
Bit literals	`BIT(8) DEFAULT b'101'`	Arrives as a `BinaryLiteral` with no dialect-neutral rendering.
Strings needing escapes	`'a\tb'`, `'a\\b'`	Declined rather than rendered ambiguously, since only quote doubling is applied.
`ENUM`/`SET` on MySQL < 8.0.1	`ENUM('a','b') DEFAULT 'b'`	Without binlog row metadata the member name can't be resolved from the binlog, so the destination column is `UInt16`/`UInt64` holding the ordinal or bitmask. The label MySQL gives as the default doesn't describe a value that column can hold.

TEXT, BLOB and JSON are absent because MySQL rejects a plain DEFAULT on them outright.

If a literal is rendered but ClickHouse rejects it anyway, ADD COLUMN is retried without the DEFAULT so schema replay for the table can't stall.

claude · 2026-07-28T09:41:19Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

dtunikov · 2026-07-28T10:51:39Z

 			nullable = false
-			break
+		case ast.ColumnOptionDefaultValue:
+			// Servers without binlog row metadata carry enums and sets as their ordinal or


that is unfortunate
but not much we can do here
on the qrep path we handled it using enum as unsigned conversion
but here can't avoid getting a plain string

ilidemi

Not a blocker for this PR, but is there a plan for ensuring we don't skip over DDL?

@fable Please look through the logic of mysql cdc.go and determine all the cases where a DDL with a column add might not get propagated as a schema delta. Ignore the ones where we fail to parse the query and emit a metric, but past that everything that belong to our tables should result in a delta applied on the destination

ilidemi · 2026-07-29T08:02:34Z

+
+			// retry without default if ch rejected the expression
+			addColumnWithDefaultFallback := func(tableName string) error {
+				err := addColumn(tableName, columnDef)


Metric and a non-paging alert sounds like a good idea. Could also be a task to check these logs in a month, but alert is a more reliable way to have it happen and we want our defaults to be good

ilidemi · 2026-07-29T08:30:20Z

+		}
+		return "'" + strings.ReplaceAll(v, "'", "''") + "'", true
+	default:
+		// DEFAULT NULL is a nil datum, bit literals are types.BinaryLiteral


Nit: could make exhaustive and warn/alert if something changes

ilidemi · 2026-07-29T08:41:14Z

Fable is ignoring us but there's another option https://claude.ai/code/session_01Sf6nJJgyfu4T8z1KxSmNar

github-actions · 2026-07-29T15:11:48Z

❌ Test Failure

Analysis: The PR's new Test_MySQL_AlterTableAddColumnDefault fails deterministically (4/4 MySQL+MariaDB→ClickHouse suites) on the ch-lts leg with a constant c_time type mismatch (QValueTime 13h14m15s vs QValueTimestamp 1970-01-01), because the test/backfill path doesn't handle the legacy non-Time64 ClickHouse mapping that Flag_ClickHouseTime64Enabled gates.
Confidence: 0.9

⚠️ This appears to be a real bug - manual intervention needed

View workflow run

handle mysql DEFAULT in DDL

f6e388c

dtunikov requested a review from a team as a code owner July 28, 2026 08:54

dtunikov commented Jul 28, 2026

View reviewed changes

fix enum/set on mysql without row metadata

b7a5e2e

dtunikov changed the title ~~Carry MySQL column DEFAULT through to ClickHouse on ADD COLUMN~~ feat: Carry MySQL column DEFAULT through to ClickHouse on ADD COLUMN Jul 28, 2026

dtunikov changed the title ~~feat: Carry MySQL column DEFAULT through to ClickHouse on ADD COLUMN~~ feat: carry MySQL column DEFAULT through to ClickHouse on ADD COLUMN Jul 28, 2026

dtunikov changed the title ~~feat: carry MySQL column DEFAULT through to ClickHouse on ADD COLUMN~~ feat(mysql): carry MySQL column DEFAULT through to ClickHouse on ADD COLUMN Jul 28, 2026

This was referenced Jul 28, 2026

WIP: feat(postgres): carry Postgres column DEFAULT through to ClickHouse on ADD COLUMN #4633

Closed

wip feat(postgres): carry ADD COLUMN ... DEFAULT to destination #4634

Closed

dtunikov commented Jul 28, 2026

View reviewed changes

dtunikov mentioned this pull request Jul 28, 2026

wip: feat(postgres): handle postgres DEFAULT in schema changes #4635

Draft

ilidemi approved these changes Jul 29, 2026

View reviewed changes

minor test comment fixes

d8584b3

dtunikov added 2 commits July 29, 2026 17:40

warn on unknown ddl default types

f14aad6

omit default when Flag_ClickHouseTime64Enabled is not enabled

f794378

Uh oh!

Conversation

dtunikov commented Jul 28, 2026

Changes

Scope

Tests

Uh oh!

dtunikov Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilidemi Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jul 28, 2026

❌ Test Failure

Uh oh!

dtunikov commented Jul 28, 2026

Which defaults are carried through

Which are declined

Uh oh!

claude Bot commented Jul 28, 2026

Code review

Uh oh!

dtunikov Jul 28, 2026

Choose a reason for hiding this comment

Uh oh!

ilidemi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ilidemi Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

ilidemi Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

ilidemi commented Jul 29, 2026

Uh oh!

github-actions Bot commented Jul 29, 2026

❌ Test Failure

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dtunikov Jul 28, 2026 •

edited

Loading