MCP server for Fair Code - deliverable, scope, and getting involved

[yakew7]

@ahmdkaml @Shreyash0712 following up from #201 - splitting the MCP convo into its own thread since it deserves more than a reply buried at the bottom.

Shreyash asked the two questions that actually matter, so starting there:

"What will be our final deliverable?"

Phase 1 already shipped: faircode-mcp (pip install faircode[mcp]), a stdio MCP server exposing three tools -

• profile_dataset - representation profiling on any tabular file
• compare_datasets - drift between two datasets
• proxy_hints - chi-squared proxy detection (needs the proxy extra)

It's a thin wrapper, same functions cli.py already calls, no new analysis logic. It's local-only (stdio, no server, no network), so it's got the same trust boundary as running the CLI yourself - nothing about that changes just because an agent is the one calling it instead of a human typing the command.

Phase 2 (not built yet) is read-only lookups: list_audits/get_audit_result against paper/results-frozen/, and search_explainers/get_explainer against the explainer library. That one's smaller and doesn't depend on anything in Phase 1.

"Will agents be able to audit any dataset from our MCP?"

For the profiler side - yes. profile_dataset/compare_datasets/proxy_hints aren't scoped to the 7 audits in this repo, they'll run against whatever file path the agent points them at. That's the whole point of doing this over MCP instead of just documenting the CLI better - an agent helping someone build a model can profile their actual dataset mid-conversation instead of someone having to leave the chat, run a command, and paste output back in.

One thing worth being precise about though: this is the profiler (representation/demographic checks), not the full benchmark harness. The benchmark.py layer - the one that trains models, applies the five mitigation strategies, and produces the fairness metrics tied to the paper - is frozen and scoped to exactly the 7 domains in the paper. That's not going into the MCP surface, and I don't think it should while the freeze is on. So "audit" here means "check a dataset for representation gaps," not "run the full S0-S4 fairness benchmark on it."

---

Separate thing I wanted to put here since it's related: if either of you (or anyone reading this) wants to actually dig into Phase 2, or has ideas for what else would make sense as an MCP tool, that's exactly the kind of sustained area-ownership that turns into a CODEOWNERS entry down the line - same way it's worked for the rest of the project so far. Not asking for anything specific, just: if you're interested, the surface is small and self-contained right now, good time to get in on it before it grows.

Also open to hearing if anyone thinks this is a bad idea for some reason I haven't considered - genuinely asking, not just being polite about it.

[propcgamer20-png]

hmm i feel like this is a good idea and id be willing to contribute to this

[Shreyash0712]

Returning CSV rows for get_audit_result might result in massive text payloads that overwhelm smaller LLM context windows or hit token limits. I think either returning just the summary.csv data, or adding some filters to scope down the output from results_fairness.csv is a good idea.

Apart from that it's pretty straightforward.