Where: faircode/loaders_extra.py's JSON orientation handling (the {"columns", "data"} <= keys split-orient check).
The gap: the existing code correctly detects and handles split-orient JSON, with a comment explaining why (a records-shaped file with columns literally named "columns"/"data" would otherwise silently misparse). A 4th real pandas export orientation - orient="index" - hits the same silent-wrong-shape problem and isn't detected at all. Verified directly:
>>> df = pd.DataFrame({'sex': ['M','F'], 'age': [30, 40]}, index=['a','b'])
>>> df.to_json('idx.json', orient='index')
>>> from faircode.loaders_extra import read_table
>>> read_table('idx.json')
a b
sex M F
age 30 40The result is transposed relative to the true data (sex/age end up as row labels, a/b as columns) - pd.read_json(path)'s default orientation guess ("columns") is structurally indistinguishable from a dict-of-dicts written with orient="index", so it silently produces the wrong shape instead of erroring.
Why it matters: unlike the already-handled split-orient ambiguity, this produces a plausible-looking but completely wrong DataFrame with no error anywhere - a user profiling this file gets confident-sounding, entirely incorrect results.
Suggested fix: detect the index-orient shape (a dict whose every top-level value is itself a dict, not matching the split-orient {"columns","data"} shape) and either parse it correctly via orient="index" or raise a clear "ambiguous JSON orientation" error rather than silently guessing wrong.
Where:
faircode/loaders_extra.py's JSON orientation handling (the{"columns", "data"} <= keyssplit-orient check).The gap: the existing code correctly detects and handles split-orient JSON, with a comment explaining why (a records-shaped file with columns literally named "columns"/"data" would otherwise silently misparse). A 4th real pandas export orientation -
orient="index"- hits the same silent-wrong-shape problem and isn't detected at all. Verified directly:The result is transposed relative to the true data (
sex/ageend up as row labels,a/bas columns) -pd.read_json(path)'s default orientation guess ("columns") is structurally indistinguishable from a dict-of-dicts written withorient="index", so it silently produces the wrong shape instead of erroring.Why it matters: unlike the already-handled split-orient ambiguity, this produces a plausible-looking but completely wrong DataFrame with no error anywhere - a user profiling this file gets confident-sounding, entirely incorrect results.
Suggested fix: detect the index-orient shape (a dict whose every top-level value is itself a dict, not matching the split-orient
{"columns","data"}shape) and either parse it correctly viaorient="index"or raise a clear "ambiguous JSON orientation" error rather than silently guessing wrong.