Skip to content

Web profiler's JS engine silently ignores an unmatched reference baseline, unlike the CLI #419

Description

@yakew7

Where: assets/profiler-engine.js's profile() (the reference-handling branch) vs faircode/profiler.py:412-424.

The gap: faircode/profiler.py explicitly raises when none of a --reference file's columns match a profiled dimension (if not any(d["name"] in o["reference"] for d in dimensions): raise ValueError(...)). The JS engine has no equivalent check - it applies the reference silently whenever o.reference is truthy, with no validation.

Repro (verified against both engines with the same mismatched reference):

# Python
>>> profile(df, {}, {"reference": {"totally_wrong_col": {"a": 0.5, "b": 0.5}}})
ValueError: reference file's column(s) don't match any profiled dimension: totally_wrong_col
// JS - same input, no error, reference silently ignored
const r = E.profile(table, {}, {reference: {totally_wrong_col: {a:0.5,b:0.5}}});
r.dimensions[0].reference  // undefined - silently did nothing

Why it matters - this is reachable through the live web UI: assets/profiler-ui.js's reference-upload handler lets a user upload any CSV as a reference baseline, and on a successful read it unconditionally sets referenceStatus.textContent = '⚖ scored vs ' + f.name regardless of whether the reference actually matched anything. A user who uploads a reference file with a typo'd or wrong column name sees a confident "scored vs myfile.csv" status while the reference silently did nothing - the same silent-no-op class of bug already fixed for the CLI (closing #364).

Suggested fix: port the same validation into the JS engine's profile(), and have profiler-ui.js call showError() when it throws, instead of showing the reference as successfully applied.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghelp wantedExtra attention is needed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions