Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/agents/contract-validator/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Contract Validator Agent

An automated governed agent that validates Agent Contracts in pull requests using `scyvera.validate_contract()`.

## Purpose

The Contract Validator ensures that any Agent Contract YAML modified or added in pull requests conforms to the normative Agent Contract specifications (v1 and v1.1).

It operates as a governed agent inside the repository itself, subject to its own contract declared at `.github/agents/contract-validator/contract.yaml`.

## Contract Specification

- **Identity**: `contract-validator`
- **Domain**: `software`
- **Lifecycle**: `request-response` (triggered per PR event, stateless execution)
- **Permissions**: Read PR diffs, post PR comments (`github:pull-requests:read`, `github:pull-requests:write`)
- **Side Effects**: PR commentary only
- **Approval Points**: None (read-only validation feedback does not perform state modification or deployment)
- **Recovery**: Stops on unrecoverable validation runtime failure and logs diagnostics to GitHub Actions summary.
- **Replay Semantics**: Idempotent.

## How to Run Locally

You can run validation against any contract file or across the repository using `scyvera`:

```bash
# Install scyvera
pip install .

# Validate this agent's contract
scyvera validate .github/agents/contract-validator/contract.yaml

# Lint this agent's contract
scyvera lint .github/agents/contract-validator/contract.yaml
```

To run the full validation script used by CI locally:

```bash
python scripts/validate_pr_contracts.py --files .github/agents/contract-validator/contract.yaml
```
77 changes: 77 additions & 0 deletions .github/agents/contract-validator/contract.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
version: 1.1
system:
name: contract-validator
purpose: Automated PR contract validation and governance reporting
version: 1.0.0

domain: software

lifecycle:
mode: request-response
initiation: agent
resumability: stateless

inputs:
- name: pull_request_diff
type: string
description: Git diff and list of modified YAML files in the pull request
required: true
- name: github_token
type: string
description: GitHub Actions token for posting validation feedback
required: false

outputs:
- name: validation_report
type: string
description: Formatted Markdown summary of contract validation results
- name: pr_comment
type: string
description: Automated comment posted to the pull request

permissions:
- github:pull-requests:read
- github:pull-requests:write

side_effects:
- type: github_pr_comment
description: Posts validation feedback comment on the PR
irreversible: false

approval_points: []

recovery:
strategy: stop
details: Log validation error to Actions output and report failure on PR

replay:
mode: idempotent
details: Re-running workflow on the same PR commit produces identical output

dependencies:
- name: python
type: runtime
required: true
- name: scyvera
type: package
required: true
- name: PyYAML
type: package
required: true
- name: jsonschema
type: package
required: true

state:
persistence: ephemeral
scope: workflow-execution

observability:
level: audit
sinks:
- github_actions_logs
- github_pr_comments

risk:
level: low
category: automated-governance
89 changes: 89 additions & 0 deletions .github/workflows/validate-contracts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ on:
branches:
- main

permissions:
contents: read
pull-requests: write
issues: write

jobs:
build-and-test:
name: Build Wheel & Test Package
Expand Down Expand Up @@ -43,3 +48,87 @@ jobs:
scyvera init test_contract.yaml --name "CI System" --domain "software"
scyvera validate test_contract.yaml
scyvera lint test_contract.yaml

validate-pr-contracts:
name: Governed Contract Validator Agent
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'

steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install PyYAML jsonschema .

- name: Identify changed YAML files
id: changed-yaml
run: |
git fetch origin ${{ github.base_ref }} --depth=1
CHANGED_FILES=$(git diff --name-only origin/${{ github.base_ref }}...HEAD | grep -E '\.(yaml|yml)$' || true)
echo "Files to check:"
echo "$CHANGED_FILES"
# Convert to single-line spaced list
FILE_LIST=$(echo "$CHANGED_FILES" | tr '\n' ' ')
echo "files=$FILE_LIST" >> "$GITHUB_OUTPUT"

- name: Run Contract Validation
id: validation
run: |
if [ -n "${{ steps.changed-yaml.outputs.files }}" ]; then
python scripts/validate_pr_contracts.py --files ${{ steps.changed-yaml.outputs.files }} --output-md pr_report.md || echo "VALIDATION_FAILED=1" >> "$GITHUB_ENV"
else
echo "No YAML files modified in this pull request."
python scripts/validate_pr_contracts.py --output-md pr_report.md
fi
cat pr_report.md >> "$GITHUB_STEP_SUMMARY"

- name: Post PR Comment
if: always()
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
if (fs.existsSync('pr_report.md')) {
const body = fs.readFileSync('pr_report.md', 'utf8');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const botComment = comments.find(comment =>
comment.body.includes('Governed Agent**: [`contract-validator`]')
);
if (botComment) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: botComment.id,
body: body
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: body
});
}
}

- name: Advisory Status Check
run: |
if [ "$VALIDATION_FAILED" = "1" ]; then
echo "::warning title=Contract Validation Advisory::One or more contracts contain specification violations. Review the governance report."
# Note: per issue #33 constraints, do NOT hard block merge; warn and record advisory result.
fi
115 changes: 115 additions & 0 deletions scripts/validate_pr_contracts.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
#!/usr/bin/env python3
"""
PR Contract Validator Script

Validates changed YAML contract files in a pull request using scyvera.validate_contract()
and outputs structured markdown results for PR commenting and GitHub Actions step summaries.
"""

from __future__ import annotations

import argparse
from pathlib import Path
import sys
import yaml

ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT / "src"))

from scyvera import validate_contract

AGENT_CONTRACT_PATH = ".github/agents/contract-validator/contract.yaml"


def validate_files(files: list[str | Path]) -> tuple[int, int, str]:
ROOT = Path.cwd()
passed = 0
failed = 0
details: list[str] = []

for f in files:
p = Path(f)
if not p.is_file():
continue

try:
rel_path = p.relative_to(ROOT) if (p.is_absolute() and p.is_relative_to(ROOT)) else p
except AttributeError:
# Python < 3.9 compatibility fallback
try:
rel_path = p.relative_to(ROOT) if p.is_absolute() else p
except ValueError:
rel_path = p
except ValueError:
rel_path = p

try:
res = validate_contract(p)
if res.valid:
passed += 1
details.append(f"- `PASS` **`{rel_path}`**")
else:
failed += 1
error_list = "\n".join(
f" - `{e.path}`: {e.message}" if e.path else f" - {e.message}"
for e in res.errors
)
details.append(f"- `FAIL` **`{rel_path}`**\n{error_list}")
except (OSError, yaml.YAMLError, Exception) as exc:
failed += 1
details.append(f"- `FAIL` **`{rel_path}`**\n - Error: {exc}")

status_icon = "PASS" if failed == 0 else "WARNING"
summary_header = f"### Contract Governance Report ({status_icon})\n\n"
summary_body = (
f"**Governed Agent**: [`contract-validator`]({AGENT_CONTRACT_PATH})\n\n"
f"**Results Summary**:\n"
f"- Total contracts evaluated: `{passed + failed}`\n"
f"- Valid contracts: `{passed}`\n"
f"- Violations / errors: `{failed}`\n\n"
)

if details:
summary_body += "**File Details**:\n" + "\n".join(details) + "\n\n"
else:
summary_body += "No YAML contracts found to evaluate in this PR diff.\n\n"

summary_body += (
"> *Note: This check is advisory and enforces self-governance under the Agent Contract specification.*"
)

markdown_report = summary_header + summary_body
return passed, failed, markdown_report


def main() -> int:
parser = argparse.ArgumentParser(description="Validate PR Contract YAML files")
parser.add_argument(
"--files",
nargs="*",
default=[],
help="List of changed YAML file paths to validate",
)
parser.add_argument(
"--output-md",
default="",
help="Path to write the markdown summary output",
)

args = parser.parse_args()

files = [Path(f) for f in args.files if f.endswith((".yaml", ".yml"))]
passed, failed, report = validate_files(files)

print(report)

if args.output_md:
out_path = Path(args.output_md)
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_text(report, encoding="utf-8")

return 1 if failed > 0 else 0


if __name__ == "__main__":
sys.exit(main())
27 changes: 27 additions & 0 deletions tests/test_pr_validator.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
from pathlib import Path
import sys

from scripts.validate_pr_contracts import validate_files, AGENT_CONTRACT_PATH

def test_contract_validator_agent_contract_is_valid():
passed, failed, report = validate_files([AGENT_CONTRACT_PATH])
assert failed == 0
assert passed == 1
assert "Contract Governance Report (PASS)" in report
assert "contract-validator" in report

def test_validate_pr_contracts_handles_mixed_files(tmp_path):
valid_contract = tmp_path / "valid_contract.yaml"
valid_contract.write_text(
"""version: 1.1\nsystem:\n name: test-agent\nlifecycle:\n mode: request-response\n"""
)
invalid_contract = tmp_path / "invalid_contract.yaml"
invalid_contract.write_text(
"""version: 99\nsystem:\n name: test-agent\n"""
)

passed, failed, report = validate_files([valid_contract, invalid_contract])
assert passed == 1
assert failed == 1
assert "Contract Governance Report (WARNING)" in report
assert "FAIL" in report
Loading