fix(promql): union (gauge, sum) scan so OTel-emitter cumulative sums under bare names resolve

[tsouza]

Summary

cerberus's PromQL matcher path used a Prom-naming heuristic
(internal/schema/Metrics.TableFor) to pick the metrics table:
_total/_count/_sum/_bucket suffix → Sum table, everything else
→ Gauge table. The heuristic mirrors the Prom-on-OTel remote-write
convention, but the OTel-Collector emitters PR #701 wires into the
quickstart (hostmetrics, sqlquery/clickhouse, prometheus/self)
ship cumulative sums under bare names that violate the convention
(system_cpu_time, clickhouse_event, otelcol_process_uptime,
…), so the matcher routed them to the Gauge table and returned zero
rows even though the row data lived in Sum.

The catalog endpoints (/api/v1/series, /api/v1/label/...) already
union all metric tables on the read side, so dashboards surfaced these
metrics in their pickers — but the matcher path silently diverged.
PR #701's otelcol-observability + clickhouse-observability
dashboards painted empty panels against fresh compose data; the
panel-kiosk + iterate-metrics-explorer sweeps caught the regression
as "Unable to fetch labels" + 10–11 console-error 400s per panel.

Root cause

internal/schema/otel.go::TableFor decides which CH table to scan for
a PromQL metric matcher based purely on the metric-name suffix. Cerberus's
self-telemetry happens to ship _total suffixes (so it works), but
upstream OTel emitters don't follow that convention — hostmetrics
emits system_cpu_time (Sum, no suffix), sqlquery emits
clickhouse_event (Sum, no suffix), prometheus/self emits
otelcol_process_uptime (mixed Gauge/Sum under bare names). The
matcher path committed to a single table; the read returned empty.

Prior PRs (#699, #706, #707, #708) each band-aided one symptom of
this single bug (labels endpoint, or-arm union, TimeUnix, allowlist
otelcol_* empties). All of them left the matcher-resolve heuristic
untouched.

Fix

• schema.Metrics.TablesFor returns the candidate table set: single
  table for suffix-routed names (preserves byte-stable SQL on the
  existing fixtures), [Gauge, Sum] for unsuffixed names.
• chplan.Scan.UnionTables is a new opt-in field — when set, the
  chsql emitter renders the scan as a CH merge(currentDatabase(), '<regex>') table function call. CH's merge() fans the scan across
  the matching tables in the named database; the PREWHERE on
  MetricName translates per-arm at planning time so granule pruning
  still fires. Verified via EXPLAIN.
• lowerVectorSelector picks the Scan shape via a scanFromTables
  helper: 1-element tables → legacy Scan{Table: …} shape (byte-stable
  for the suffix-routed fixtures); multi-element → Scan{UnionTables: …}. The histogram-companion + bucket-selector overrides keep
  single-table semantics.

Test plan

• All 4097 unit tests pass (go test ./...).
• All 268 chDB roundtrip fixtures pass (-tags chdb) including the
  new regression pin scan_unions_gauge_sum_for_unsuffixed_metric.txtar
  whose seed populates only the Sum table — a regression that
  dropped the Sum-table arm of merge() would return zero rows
  against this fixture.
• Live compose stack: every previously-failing query on PR feat(quickstart): rich ClickHouse / host / otelcol observability via OTel collector #701's
  compose-smoke iteration (run 26308908297) now returns data —
  otelcol_process_uptime, system_cpu_time, clickhouse_event,
  MergeTree-I/O rate(clickhouse_event{name=~...}[5m]) panel,
  the three-arm or shapes, the histogram_quantile over
  otelcol_processor_batch_batch_send_size_bucket.
• CI: compose-smoke should now pass against PR feat(quickstart): rich ClickHouse / host / otelcol observability via OTel collector #701's dashboards
  since the underlying data-routing bug is fixed (this PR doesn't
  include PR feat(quickstart): rich ClickHouse / host / otelcol observability via OTel collector #701's dashboards itself — that's still PR feat(quickstart): rich ClickHouse / host / otelcol observability via OTel collector #701).

Fixture churn

158 existing TXTAR fixtures absorb the FROM otel_metrics_gauge →
FROM merge(currentDatabase(), '^(otel_metrics_gauge|otel_metrics_sum)$')
change. The chplan IR pretty-printer (test/spec/chplan_print.go) now
renders Scan(merge(t1|t2)) for union scans.

Follow-up

Suffix-routed Sum metrics that happen to end in _count/_sum but
aren't histogram companions (system_cpu_logical_count,
system_processes_count etc.) still route to the histogram table via
HistogramCompanionColumn and return empty. The fix here is the same
shape (union the histogram-companion path with a fallback Sum scan
under the original name) — separate PR.

🤖 Generated with Claude Code