Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions docs/a38-guard.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,8 +229,10 @@ The latest author report-like comment, ordered by `updated_at` and numeric comme

| Mode | Stable status context | Meaning |
| --- | --- | --- |
| `enforce` | `A38 / report (develop)` for target branch `develop` | Success only for valid evidence; otherwise failure. |
| `observe` | `A38 / report (observe: develop)` | Advisory status only; do not require this context for merging. |
| `enforce` | `A38 / report (develop)` for target branch `develop` | On a draft PR: omit the **blocking** commit status (not failure, not pending, not a fabricated pass); see the `not_applicable` success-clear carve-out below. The guard process exits 0 so `dfx pr guard` is not red merely for a missing draft report. An author report is still required before Ready. Once Ready (`draft=false`): success only for valid evidence; otherwise failure. |
| `observe` | `A38 / report (observe: develop)` | Advisory status only; do not require this context for merging. Unchanged on drafts. |

Configured `not_applicable` exclusions still publish success on the target enforce context to clear a wrong prior status, including on drafts; that success is not a test-pass claim.

Contexts use the **target branch name**, not the moving base SHA. Thus branch protection can require a stable name while a head targeting different branches gets distinct contexts. Supported branch names are bounded to 75 ASCII letters/digits, dots, underscores, hyphens and slashes; unsupported names fail closed. The exact base SHA remains in the comment and approval binding.

Expand All @@ -255,6 +257,8 @@ Issue-only events and the bot's own comments are ignored. The installed workflow

Closed PRs return `status: closed` and process exit zero without reading policy, pr-guard configuration or publishing comments/statuses, including when a PR closes during an all-open scan. Ignored events and empty all-open scans are also successful no-ops.

On an open **draft** in `enforce` mode the guard still publishes or updates its educational comment, but it does **not** create or update a blocking `A38 / report (<target>)` commit status (and does not post an invalidating `error` status on draft). Configured `not_applicable` exclusions may still write success on that context only to clear a wrong prior status; that is not a test-pass claim. Process exit is 0 so `dfx pr guard` is not red merely because a draft lacks an author report. Ready (`draft=false`) keeps success-only-for-valid-evidence and failure otherwise.

The bot marker is `<!-- PR-GUARD:A38:v1 -->`. Only comments owned by the numeric acting user may be updated. `/user` resolves normal tokens; fallback to the verified official Actions bot is allowed only when `GITHUB_ACTIONS=true`. Failed authentication outside Actions does not impersonate that bot. Existing identical comments/statuses are not reposted.

Before publication, the guard re-fetches head/base/branch/state, the trusted pr-guard configuration revision and bytes, the latest author report and any active migration approval. It checks again immediately before a success status and reassesses if evidence changed. GitHub offers no atomic transaction across comments, reviews and statuses: an edit after the final read is corrected by the next event or scheduled reconciliation.
Expand Down
2 changes: 1 addition & 1 deletion docs/a38.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ The report must contain the exact repository, current head, visibility, full req

Only the latest report-like comment by the authenticated PR author is considered. A newer failed or malformed report does not fall back to an older success. Editing or deleting reports, pushing commits and changing the target base require reassessment. Other contributors cannot satisfy the author's reporting requirement by posting a copied block.

The bot maintains one friendly comment with the rules, documentation and current problems. `observe` publishes an explicitly advisory status; `enforce` reports success only for valid evidence. Configure branch protection to require the appropriate enforced status after a successful rollout. A missing or invalid policy is a configuration failure, not permission to pass. A valid A38 report confirms the consistency of an author's declaration; it is not cryptographic proof that commands ran or proof that tests themselves are adequate.
The bot maintains one friendly comment with the rules, documentation and current problems. `observe` publishes an explicitly advisory status and is unchanged on drafts. `enforce` reports success only for valid evidence. On a **draft** pull request the guard publishes **no blocking** `A38 / report (<target>)` commit status (not failure, not pending, and not a fabricated pass) and exits 0 so the GitHub Actions check `dfx pr guard` is not red merely because a draft has no author report yet. Configured `not_applicable` exclusions may still write success on that context to clear a wrong prior status; that is not a test-pass claim. An author report is still required before Ready for review. Once the pull request is Ready (`draft=false`), keep current behaviour: valid evidence → success; missing or invalid evidence → failure. Configure branch protection to require the appropriate enforced status after a successful rollout. A missing or invalid policy is a configuration failure, not permission to pass. A valid A38 report confirms the consistency of an author's declaration; it is not cryptographic proof that commands ran or proof that tests themselves are adequate.

For the private opt-in equivalence, the final Ready join is a separate live read after the same SHA has been pushed and the unchanged generated report has been posted:

Expand Down
2 changes: 2 additions & 0 deletions docs/pull-request-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ Hosted CI and other applicable checks may fail. There is no promise that CI neve

Pending checks must be labeled **pending**. Do not fabricate a pass.

The blocking `A38 / report (<target>)` commit status is **omitted** on drafts (not pending, not failure, and not a fabricated pass). `observe` stays advisory and unchanged. Configured `not_applicable` exclusions may still write success on that context only to clear a wrong prior status; that is not a test-pass claim. Real red hosted CI remains a blocker. Once the pull request is Ready for review, A38 publishes success only for valid evidence and failure otherwise.

Repositories can enable the [guard's continuous readiness reconciliation](a38-guard.md#optional-continuous-readiness).
An open Ready PR returns to Draft with an explanatory comment when required CI
is missing, queued, running, blocked or failed, or GitHub confirms merge conflicts.
Expand Down
83 changes: 66 additions & 17 deletions src/agent_cli/a38_guard.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,8 @@ class PullSnapshot:
# Used to locate configuration; never a built-in scope rule by itself.
# Empty is allowed for closed no-ops; open assessments fail closed without it.
default_branch: str = ""
# True only when GitHub reports draft is JSON true; missing/None/False → not draft.
draft: bool = False


@dataclass(frozen=True)
Expand Down Expand Up @@ -156,6 +158,7 @@ class Assessment:
workflow_approvals: list[dict[str, Any]] = field(default_factory=list)
lifecycle_enabled: bool = False
lifecycle: dict[str, Any] = field(default_factory=dict)
draft: bool = False

def to_json(self) -> dict[str, Any]:
trusted = self.trusted_default_branch or self.default_branch
Expand Down Expand Up @@ -185,6 +188,7 @@ def to_json(self) -> dict[str, Any]:
"state": self.state_for_status,
"description": self.description,
"closed": self.closed,
"draft": self.draft,
"skip_publish": self.skip_publish,
"dry_run": self.dry_run,
"writes": list(self.writes),
Expand Down Expand Up @@ -640,6 +644,8 @@ def fetch_pull(api: GitHubApi, repo: str, number: int) -> PullSnapshot:
default_branch = _parse_default_branch(
base_repo.get("default_branch"), required=(state == "open")
)
# Fail closed for the draft exemption: only JSON true is draft.
draft = data.get("draft") is True
return PullSnapshot(
repo=repo,
number=number,
Expand All @@ -652,6 +658,7 @@ def fetch_pull(api: GitHubApi, repo: str, number: int) -> PullSnapshot:
author_login=author_login,
head_repo=head_repository,
default_branch=default_branch,
draft=draft,
)


Expand Down Expand Up @@ -985,22 +992,45 @@ def build_comment_body(assessment: Assessment) -> str:
problems = "; ".join(assessment.reasons) if assessment.reasons else "none"
if len(problems) > 800:
problems = problems[:799] + "…"
en = (
f"A38 {assessment.status}: "
+ (
"author local-CI report accepted for this head."
if assessment.ok and assessment.status == "pass"
else "author local-CI report missing or invalid for this head."
passing = assessment.ok and assessment.status == "pass"
if assessment.draft:
extra_en = (
" An author local-CI report is accepted for this head."
if passing
else " An author local-CI report is still required before Ready."
)
)
de = (
f"A38 {assessment.status}: "
+ (
"Autor-Local-CI-Report für diesen Head akzeptiert."
if assessment.ok and assessment.status == "pass"
else "Autor-Local-CI-Report für diesen Head fehlt oder ist ungültig."
extra_de = (
" Ein Autor-Local-CI-Report für diesen Head ist akzeptiert."
if passing
else " Ein Autor-Local-CI-Report ist vor Ready weiterhin erforderlich."
)
en = (
"A38: this pull request is a draft; "
"no blocking A38 report status is published until Ready for review."
+ extra_en
)
de = (
"A38: dieser Pull Request ist ein Draft; "
"bis Ready for review wird kein blockierender A38-Report-Status veröffentlicht."
+ extra_de
)
else:
en = (
f"A38 {assessment.status}: "
+ (
"author local-CI report accepted for this head."
if passing
else "author local-CI report missing or invalid for this head."
)
)
de = (
f"A38 {assessment.status}: "
+ (
"Autor-Local-CI-Report für diesen Head akzeptiert."
if passing
else "Autor-Local-CI-Report für diesen Head fehlt oder ist ungültig."
)
)
)
if assessment.mode == "observe":
en = "Observe mode (advisory, not branch-required). " + en
de = "Observe-Modus (Hinweis, nicht branch-pflichtig). " + de
Expand Down Expand Up @@ -1050,6 +1080,13 @@ def _status_bits(assessment: Assessment) -> None:
return
assessment.context = status_context_enforce(base)
assessment.observe_context = ""
if assessment.draft:
# Draft enforce: keep context for audit JSON; do not post success or failure.
assessment.state_for_status = ""
assessment.description = truncate_desc(
"draft: A38 status omitted until Ready"
)
return
if assessment.ok and assessment.status == "pass":
assessment.state_for_status = "success"
assessment.description = truncate_desc(f"pass for {assessment.head_sha[:7]}")
Expand Down Expand Up @@ -1094,6 +1131,7 @@ def assess_from_parts(
private=pull.private,
closed=pull.state != "open",
dry_run=dry_run,
draft=pull.draft,
standard_url=blob_url(CENTRAL_REPO, trusted_runtime_revision, POLICY_DOCS),
policy_url=blob_url(active_policy_repo, active_policy_sha, POLICY_PATH),
guard_docs_url=blob_url(CENTRAL_REPO, trusted_runtime_revision, GUARD_DOCS),
Expand Down Expand Up @@ -1344,6 +1382,7 @@ def _out_of_scope_assessment(
comment_body="",
skip_publish=False,
dry_run=dry_run,
draft=snap.draft,
)
_attach_trusted_config(assessment, trusted)
return assessment
Expand All @@ -1360,6 +1399,7 @@ def _snapshot_matches_assessment(fresh: PullSnapshot, assessment: Assessment) ->
and fresh.head_repo == assessment.head_repo
and fresh.private == assessment.private
and fresh.state == expected_state
and fresh.draft == assessment.draft
)


Expand Down Expand Up @@ -1400,7 +1440,7 @@ def assess_pull(
default_branch=snap.default_branch,
trusted_default_branch=snap.default_branch,
head_repo=snap.head_repo, private=snap.private,
state_for_status="", dry_run=dry_run,
state_for_status="", dry_run=dry_run, draft=snap.draft,
)
# Open PRs require trusted default_branch metadata to locate configuration.
if not snap.default_branch:
Expand Down Expand Up @@ -1636,6 +1676,8 @@ def _post_status(
assessment.description,
require_report=True,
)
elif assessment.draft:
assessment.writes.append("status:skipped:draft")
else:
_post_status(
assessment.context or status_context_enforce(assessment.base_ref),
Expand Down Expand Up @@ -1703,6 +1745,8 @@ def invalidate_status(api: GitHubApi, pull: PullSnapshot) -> None:
"""Best-effort error status on known head; never hide the original API failure."""
if pull.state != "open":
return
if pull.draft:
return
try:
status, _, _ = api.request(
"POST", f"/repos/{pull.repo}/statuses/{pull.head_sha}",
Expand Down Expand Up @@ -1915,8 +1959,13 @@ def _load_event(


def _assessment_exit_code(assessment: Assessment) -> int:
"""A closed PR is a successful no-op; observe remains advisory."""
return 0 if assessment.closed or assessment.ok or assessment.mode == "observe" else 1
"""Closed, observe, and draft enforce skips exit 0; Ready enforce failure exits 1."""
return 0 if (
assessment.closed
or assessment.ok
or assessment.mode == "observe"
or assessment.draft
) else 1


def main(argv: Sequence[str] | None = None, *, env: MutableMapping[str, str] | None = None,
Expand Down
Loading
Loading