SchemaShield Function Calling

SchemaShield separates a model's semantic tool choice from the syntax required by a downstream interface. It parses native Qwen tool calls into a canonical CallIR, validates arguments recursively, applies a short allowlist of safe type conversions, and serializes accepted calls into Python, JSON, or either of the two BFCL XML forms.

Unknown tools, unknown arguments, missing required values, ambiguous enum matches, fuzzy tool names, and non-finite numbers are rejected. Optional schema defaults are not inserted by default.

native tool call
  -> strict parser
  -> canonical CallIR
  -> recursive schema guard
  -> repair or abstain
  -> deterministic requested-format serializer

Quick start

The guard, parser, and serializers use the Python standard library.

python -m venv .venv
source .venv/bin/activate
python -m pip install -e .
python -m unittest discover -s tests -v
python examples/safe_delivery.py

BFCL subset run

The benchmark and model are external dependencies. Clone Gorilla separately and install the model extras:

git clone https://github.com/ShishirPatil/gorilla.git third_party/gorilla
git -C third_party/gorilla checkout 6ea57973c7a6097fd7c5915698c54c17c5b1b6c8
python -m pip install -e '.[model]'

Then run a small paired subset with explicit paths:

schema-shield-paired \
  --vendor-root third_party/gorilla/berkeley-function-call-leaderboard \
  --model-path /path/to/Qwen3-4B-Instruct-2507 \
  --split non_live --case-source official_format \
  --limit-per-category 1 --format-indices 0,9,12,21 \
  --omit-prompt --output-dir runs/subset

The command writes prompts or prompt hashes, raw generations, scored records, and a manifest to the output directory. runs/ is ignored by Git and should be reviewed before any artifact is shared.

Upstream attribution

The reported experiment used Gorilla/BFCL revision 6ea57973c7a6097fd7c5915698c54c17c5b1b6c8. The repository does not include BFCL questions, possible answers, evaluator source, Qwen weights, or run logs. The small formatter port is identified in NOTICE and in the source file.

Aggregate results and limitations are in RESULTS.md.

Scope

Schema validity is a static contract check, not proof that a tool call is semantically correct or safe to execute. The evaluation uses public BFCL cases and does not establish held-out generalization. The guard can also reject calls that BFCL accepts under looser symbolic-value rules.