Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ first shipped.
- **`get_explainer` had a path-traversal vulnerability via `slug`** (closes #387) - `slug="../README"` returned this repo's `README.md` content, a file entirely outside `explainers/`, since `slug` was used to build a filesystem path with zero validation. Now validated against the exact pattern every real slug matches before the path is ever built.
- **`profile_dataset`'s `cross` validation didn't reject the same column twice, unlike the CLI** (closes #389) - `cross=["sex","sex"]` silently produced a degenerate diagonal-only intersection table instead of erroring like `faircode profile --cross sex,sex` already does.
- **`get_benchmark_results` crashed uncaught or leaked raw pandas errors on a non-scalar filter value** (closes #390) - a dict value raised an uncaught `NotImplementedError` (not in the tool wrapper's except tuple); a list value raised `ValueError` with pandas' own internal wording. Each filter is now validated as a plain scalar first.
- **`get_benchmark_results` blamed the wrong file when its package mirror was missing** (closes #396) - the error now names the actual missing `faircode/_results_frozen/` path and points to `scripts.freeze_paper_results.mirror_for_mcp()`, avoiding advice that could trigger an unnecessary, citation-affecting re-freeze.
- **README.md's MCP section still listed only the original 3 tools** (closes #391) - `SPEC.md` section 11 was already correctly updated for all six.

## [2.1.1] - 02 Sep 2026
Expand Down
8 changes: 4 additions & 4 deletions faircode/mcp_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@
cli.py already wraps. See faircode/SPEC.md section 11 for the tool contract.

Phase 2 (list_explainers, get_explainer, get_benchmark_results) adds
read-only lookups against this repo's own explainers/ and
paper/results-frozen/ - the plan mentioned in CHANGELOG.md's Phase 1 note.
read-only lookups against package-internal mirrors of this repo's explainers/
and frozen benchmark results - the plan mentioned in CHANGELOG.md's Phase 1 note.

Needs the optional 'mcp' extra (`pip install faircode[mcp]`).

Expand Down Expand Up @@ -299,8 +299,8 @@ def _get_benchmark_results_impl(kind="fairness", audit=None, model=None, strateg
raise ValueError(f"kind must be one of {sorted(RESULTS_FROZEN_FILES)}, got {kind!r}")
if not path.is_file():
raise FileNotFoundError(
f"{path.name} not found - paper/results-frozen/ may not have been frozen yet "
"(see scripts/freeze_paper_results.py)")
f"{path} not found - restore the package mirror from paper/results-frozen/ "
"with `scripts.freeze_paper_results.mirror_for_mcp()`")
df = pd.read_csv(path)
for column, value in (("audit", audit), ("model", model), ("strategy", strategy),
("metric", metric), ("protected_attribute", protected_attribute)):
Expand Down
15 changes: 15 additions & 0 deletions tests/test_mcp_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

from faircode.mcp_server import ( # noqa: E402
_compare_datasets_impl,
RESULTS_FROZEN_FILES,
_get_benchmark_results_impl,
_get_explainer_impl,
_list_explainers_impl,
Expand Down Expand Up @@ -452,6 +453,20 @@ def test_get_benchmark_results_invalid_kind_raises():
_get_benchmark_results_impl(kind="not_a_real_kind")


def test_get_benchmark_results_missing_mirror_names_path_and_safe_recovery(
tmp_path, monkeypatch):
missing = tmp_path / "faircode" / "_results_frozen" / "results_fairness.csv"
monkeypatch.setitem(RESULTS_FROZEN_FILES, "fairness", missing)

with pytest.raises(FileNotFoundError) as exc_info:
_get_benchmark_results_impl()

message = str(exc_info.value)
assert str(missing) in message
assert "scripts.freeze_paper_results.mirror_for_mcp()" in message
assert "may not have been frozen" not in message


@pytest.mark.parametrize("bad_value", [{"x": 1}, ["a", "b"], (1, 2)])
def test_get_benchmark_results_non_scalar_filter_raises_clean_error(bad_value):
# A dict raised an uncaught NotImplementedError and a list raised a raw
Expand Down