Skip to content

arrow_converter: three gaps that survive full typedColumns pinning #134

Description

@jghoman

Follow-ups from the 2026-09-02 session-replay pin work (charts#15090). Full pins close the cross-batch inference crashes for known fields, but three gaps remain in arrow_converter.py / main.py. Both adversarial review passes verified these empirically.

1. Uncaught OverflowError on integers ≥ 2^63 (poison-batch crash-loop)

A JSON integer in [2^63, ~1e21) raises OverflowError at pa.array() / from_pylist (arrow_converter.py:90, :354). Only ArrowInvalid/ArrowTypeError are caught; OverflowError escapes to the blanket handler in main.py:594 → fatal → restart → same offsets → crash-loop. JS emits integer notation below 1e21, so e.g. a corrupt timestamp inflating inter_action_gap_sum_of_squares_ms lands exactly in this window. typedColumns pins do not help — the raise happens before coercion.

Fix: catch OverflowError alongside the Arrow exceptions in the per-value fallback paths so the value NULLs and increments errors_total{type="column_coercion"}.

2. _floatify_integers_in_float_fields only fires on inferred-float batches

An all-int batch infers int64, and casting int64 > 2^53 to a double-pinned column raises ArrowInvalid → per-value NULL (silent data loss with metric) instead of nearest-double conversion. Mixed batches take the floatify path and convert correctly — same value, different outcome by batch composition.

Fix: floatify integers for any column pinned double via MILLPOND_TYPED_COLUMNS regardless of the inferred batch schema.

3. No structural defense against future unpinned numeric fields

The session tables are fixed-column ClickHouse schemas, so new fields arrive only via DDL migration in posthog/posthog — but when one does, it lands unpinned and can reintroduce the int64-vs-string concat crash (exactly how inter_action_gap_sum_ms bit us: absent from 32K sampled records, present in prod).

Fix: on pa.concat_tables type mismatch, unify the conflicting column to string (and count it) instead of dying; the pinned path stays authoritative for declared columns.

Also related

  • Whole-float-for-bigint is order-dependent: float-first batch casts 2.0→2 exactly; int-first mixed batch stringifies to "2.0" which NULLs on bigint coercion.
  • millpond pyducklake >=1.0.18 bump is still pending (Leak A parity).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions