[backfill: ClickHouse/clickhouse-connect] Escaped single quotes in nested Enum names break parser inside Variant/Tuple

[alex-clickhouse]

Target client repo

ClickHouse/clickhouse-connect

Severity

sev:2 - Visible error (not silent corruption), real type pattern (Variant/Tuple of Enum with escaped-quote name), and no easy client-side workaround: the user would have to rename the column/enum to avoid embedded single quotes.

Description

ClickHouse Enum names may contain single quotes encoded as \'. The bare-Enum parsing path (parse_enum in clickhouse_connect/driver/parser.py) does handle this, and there are unit tests for it (tests/unit_tests/test_chtypes.py::test_enum_parse). However, when the same Enum appears as an element type inside a Variant(...) or Tuple(...), the wrapping parser (parse_columns in clickhouse_connect/driver/parser.py) mishandles \': it adds the unescaped ' to the label before the \ (because label += char runs at the bottom of the loop after label += expr[pos] runs in the escape branch). The Enum element string is then mangled and parse_enum later trips on a malformed numeric token.

This manifests on real wire traffic: ClickHouse's Native/RowBinary wire format does include the \' escape for Enum names embedded in a Variant(Enum(...), ...) column type. clickhouse-connect's transform layer reads that type string and feeds it to get_from_name, which fails.

Specifically, in parser.py::parse_columns around the if quote: branch (line 134-139): the escape handling assigns label += expr[pos] (the ') then advances pos, but the unconditional label += char at the bottom of the loop then appends the \. The result is that the order in label becomes '\ instead of \', which breaks subsequent boundary recognition for the closing quote of the Enum name.

ClickHouse server version

26.4.2.10 (HTTP, Native format), verified locally.

Reproduction

import clickhouse_connect

client = clickhouse_connect.get_client(
    host='localhost',
    settings={'allow_experimental_variant_type': 1, 'allow_suspicious_variant_types': 1},
)
client.command("DROP TABLE IF EXISTS t_enum_esc")
client.command(
    "CREATE TABLE t_enum_esc "
    "(v Variant(Enum8('foo\\'bar'=1,'baz'=2), String)) ENGINE=Memory"
)
client.command(
    "INSERT INTO t_enum_esc "
    "SELECT CAST('foo\\'bar' as Enum8('foo\\'bar'=1,'baz'=2))"
)
result = client.query('SELECT v FROM t_enum_esc')
print(result.result_rows)

Expected: [("foo'bar",)] (or similar successful read).

Actual:

File ".../clickhouse_connect/driver/parser.py", line 104, in parse_enum
    values.append(int("".join(value)))
ValueError: invalid literal for int() with base 10: "\\bar'1"

The query never returns; get_from_name("Variant(Enum8('foo\\'bar' = 1, 'baz' = 2), String)") raises before any rows can be parsed.

A standalone reproducer that does not need a running server:

from clickhouse_connect.datatypes.registry import get_from_name
get_from_name("Variant(Enum8('foo\\'bar' = 1, 'baz' = 2), String)")
# -> ValueError: invalid literal for int() with base 10: "\\bar'1"

Note: the bare-Enum case get_from_name("Enum8('foo\\'bar' = 1, 'baz' = 2)") works correctly. The bug only surfaces when the Enum with an escaped-quote name appears inside a Variant, Tuple, Nested, or other container type that routes through parse_columns.

Also note: the timezone-in-DateTime case from the source bug (Variant(DateTime('UTC'), String)) does not trigger this bug on the Native wire, because the ClickHouse server emits the type string with raw single quotes for DateTime('UTC') rather than DateTime(\'UTC\'). The \' form only appears for Enum element names, where the single quote is part of the name and must be escaped on the wire.

Suggested fix

In clickhouse_connect/driver/parser.py::parse_columns, the escape branch around lines 134-139 should not fall through to the unconditional label += char at the bottom of the loop. Either continue after handling the escape, or build the label inside an explicit if/else so the \ is appended before expr[pos] (matching the on-wire byte order). The same ordering question applies to parse_callable lines 41-46; both should be checked together.

A reasonable shape:

if quote:
    if char == quote:
        quote = None
        label += char
        continue
    if char == "\\" and expr[pos] == "'" and expr[pos:pos+4] != "' = " and expr[pos:pos+2] != "')":
        label += char           # the backslash
        label += expr[pos]      # the escaped quote
        pos += 1
        continue

Dedup search

gh issue list --repo ClickHouse/clickhouse-connect --search "enum escaped" --state all --limit 20
# -> 0 results
gh issue list --repo ClickHouse/clickhouse-connect --search "escape parser" --state all --limit 20
# -> 1 unrelated result (#300 about Date conversion on insert)
gh pr list --repo ClickHouse/clickhouse-connect --search "enum escaped" --state all --limit 20
# -> 0 results
gh issue list --repo ClickHouse/integrations-ai-playground --label backfill --label "target:clickhouse-connect" --search "enum escaped" --state all --limit 20
# -> 0 results
gh pr list --repo ClickHouse/integrations-ai-playground --search "enum escaped" --state all --limit 20
# -> 0 results

Source bug

ClickHouse/clickhouse-cpp#482