Gate required evaluation cases

[brandonwise]

Summary

• Add required: true support for individual gate cases.
• Hard-fail the release gate when any required case fails, even if aggregate pass rate still meets the global threshold.
• Surface required-case failures in JSON and Markdown reports.
• Mark the demo refund workflow as required and document how to use required cases for launch-blocking journeys.
• Refresh tracked package metadata from the successful build.

Rationale

Current AI-agent telemetry points to validation and human review as the bottleneck. Aggregate pass rates can hide a failed critical journey, so PMs need a release control for must-pass workflows like checkout, compliance, safety escalation, or executive-demo paths.

Evidence:

• Reddit PM workflows: PMs are using AI for PRDs, ticket cleanup, code-aware docs, and repetitive work, but full PM agents still need sharp guardrails: https://www.reddit.com/r/ProductManagement/comments/1pxji99/how_are_pms_actually_using_ai_in_daytoday_work/
• Reddit evals discussion: PMs frame evals as product quality, but rubric agreement is the hard part: https://www.reddit.com/r/ProductManagement/comments/1umtcgx/experience_with_ai_evals/
• Product Hunt AI agents: current launches cluster around workplace assistants, autonomous ops orchestration, approvals, and audit trails: https://www.producthunt.com/categories/ai-agents
• Agentic coding research: adoption depends on the human and organizational processes that govern agent contributions: https://arxiv.org/abs/2607.14037

Validation

• python3 -m pytest tests/test_evaluator.py -q - PASS, 14 passed.
• python3 -m pytest -q - PASS, 30 passed.
• python3 -m ruff check . - PASS, all checks passed.
• python3 -m mypy src - PASS, no issues found in 5 source files.
• python3 -m build - PASS, built agent_release_gate-0.1.0.tar.gz and agent_release_gate-0.1.0-py3-none-any.whl.
• PYTHONPATH=src python3 -m agent_release_gate.cli evaluate --spec examples/spec.yaml --results examples/results.json --output /tmp/argate-required-pass.json --markdown /tmp/argate-required-pass.md - PASS, exit 0 with gate_passed: true and required: true on refund_status.
• PYTHONPATH=src python3 -c 'import json, subprocess, sys, tempfile; from pathlib import Path; d = Path(tempfile.mkdtemp()); spec = d / "spec.yaml"; results = d / "results.json"; spec.write_text("global:\n minimum_pass_rate: 0.5\ncases:\n - id: critical_checkout\n expected_all: [\"approved\"]\n required: true\n - id: optional_status\n expected_all: [\"status\"]\n", encoding="utf-8"); results.write_text(json.dumps({"cases": [{"id": "critical_checkout", "response": "manual review needed"}, {"id": "optional_status", "response": "status ready"}]}), encoding="utf-8"); completed = subprocess.run([sys.executable, "-m", "agent_release_gate.cli", "evaluate", "--spec", str(spec), "--results", str(results)], capture_output=True, text=True); print(completed.stdout); print(completed.stderr); ok = completed.returncode == 1 and "Required cases failed: critical_checkout" in completed.stdout; sys.exit(0 if ok else 1)' - PASS, outer smoke exits 0 after confirming the CLI hard-fails the required case.
• node /Users/bwise/.openclaw/workspace/projects/humanizer/src/cli.js score -f README.md --ignore-code - PASS, 1/100.
• git diff --check - PASS.

Note: an initial source-tree smoke without PYTHONPATH=src failed with ModuleNotFoundError because the package was not installed in that shell. The source-tree smoke commands above are the corrected validation path.