Skip to content

loaders_extra.py raises a raw pandas internal error for table-oriented JSON #406

Description

@yakew7

Where: faircode/loaders_extra.py's JSON reading path (same area as the split/index-orient handling).

The gap: pandas.DataFrame.to_json(orient="table") - a real, valid, non-obscure pandas JSON export format (includes a "schema" key alongside "data") - isn't recognized at all. Verified directly:

>>> df.to_json('table.json', orient='table')
>>> from faircode.loaders_extra import read_table
>>> read_table('table.json')
ValueError: Mixing dicts with non-Series may lead to ambiguous ordering.

That message comes from deep inside pandas/core/internals/construction.py - exactly the kind of "internal, version-specific message" this module's own comment (describing why it special-cases split-orient) says it wants to avoid, but here for a completely valid JSON file pandas itself produces.

Why it matters: a user who exported their data with to_json(orient="table") (a reasonable choice - it's the only orientation that round-trips dtypes) gets a confusing, implementation-detail error instead of a clear "unsupported/unrecognized JSON shape" message.

Suggested fix: detect the {"schema": ..., "data": ...} shape the same way split-orient is already detected, and either dispatch to pd.read_json(path, orient="table") or raise the module's own clear ValueError for it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedExtra attention is needed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions