Skip to content

Parametrized test: every demo recipe loads and runs clean #39

Description

@5uper0

examples/demo/ ships 14 recipe_*.json decision recipes (KYC, P2P dispute, DAO treasury, M&A term sheet, …), but only one — recipe_early_lease_exit.json — has an engine-level test (tests/test_real_decision_recipes.py). The other 13 are exercised only by whoever clicks them in the live demo. A malformed recipe (bad operator string, missing attr, broken utility bounds) would ship silently — and recipes are exactly the contribution we most want from outsiders, so the safety net should be automatic.

What is wanted

In tests/test_real_decision_recipes.py, add one pytest.mark.parametrize test over every examples/demo/recipe_*.json (glob the directory, so future recipes are covered with zero test edits) asserting for each:

  1. DecisionSpec.load(path) succeeds and the spec has ≥2 options and ≥2 parties;
  2. spec.run() returns status in {"agreed", "deadlock"} without raising;
  3. when status == "agreed", every party's transcript.verify_non_betrayal(party.to_sheet(), decision) is True;
  4. DecisionSpec.from_dict(spec.to_dict()) round-trips to the same decision.

Keep the existing lease-exit test as is — it pins a specific ratified outcome; the new test is the generic floor under all of them.

How to verify

.venv/bin/pytest -q tests/test_real_decision_recipes.py
.venv/bin/pytest -q

Expect 14 new passing parametrized cases (and the count grows by itself when a recipe lands).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions