From 1a1c6b7bf6ad676201ca656e7ab6b8e4d4c9db17 Mon Sep 17 00:00:00 2001 From: Anatoly Koyfman Date: Tue, 18 Aug 2026 11:55:30 +0000 Subject: [PATCH 1/8] Docs: Specify #154 pre-commit policy conflict diagnostic Document the read-only pre-commit conflict diagnostic (issue #154) across the AIAC requirement specs, ahead of code. Add a dedicated sub-PRD (policy-conflict-check.md) covering the interface, diagnostic-assembly pipeline, ConflictReport/status contract, testing tiers, and acceptance criteria; thread cross-references through the PRB sub-PRD, the AIAC Agent component PRD (endpoint, use case, file structure), and the master PRD (component summary, architectural decision, agent capability). The live /apply -> 422 contradiction path is unchanged. Assisted-By: Claude (Anthropic AI) Signed-off-by: Anatoly Koyfman --- aiac/docs/specs/PRD.md | 5 +- aiac/docs/specs/components/aiac-agent.md | 13 +- .../aiac-agent/policy-conflict-check.md | 283 ++++++++++++++++++ .../aiac-agent/policy-rules-builder.md | 8 + 4 files changed, 305 insertions(+), 4 deletions(-) create mode 100644 aiac/docs/specs/components/aiac-agent/policy-conflict-check.md diff --git a/aiac/docs/specs/PRD.md b/aiac/docs/specs/PRD.md index 7963edd48..6743e2f25 100644 --- a/aiac/docs/specs/PRD.md +++ b/aiac/docs/specs/PRD.md @@ -115,7 +115,7 @@ Nine components across five Kubernetes Pods plus a Python library layer, all imp | 5 | **Policy and Domain Knowledge RAG** | ChromaDB vector store holding the access control policy and domain knowledge in persistent, queryable form, populated via a co-located RAG Ingest Service. | | 6 | **Policy Guardrails Agent** | Verification gate co-located with ChromaDB and the RAG Ingest Service in the RAG Pod. Every document is checked before the RAG Ingest Service writes it to ChromaDB. Reachable only on the RAG Pod's loopback network — not exposed on the RAG Pod's ClusterIP Service. One service, two API families (`policy`, `domain-knowledge`); the `policy` family runs LLM-backed hygiene + corpus-contradiction checks (defined), `domain-knowledge` specced later. | | 7 | **Event Broker** | NATS JetStream pod that decouples event producers (Keycloak SPI listener, RAG Ingest Service) from the AIAC Agent. Provides durable, at-least-once delivery with automatic replay on Agent pod restart. Competing consumer model ensures each event is processed exactly once. | -| 8 | **AIAC Agent** | LangGraph-based AI agent triggered by Event Broker subscriptions (`aiac.apply.>` subjects) and directly by the operator (`rebuild` only). Retrieves the current policy from the RAG store, interprets it against live PDP state, and applies the required policy changes immediately. | +| 8 | **AIAC Agent** | LangGraph-based AI agent triggered by Event Broker subscriptions (`aiac.apply.>` subjects) and directly by the operator (`rebuild` only). Retrieves the current policy from the RAG store, interprets it against live PDP state, and applies the required policy changes immediately. Also exposes a **read-only pre-commit policy conflict diagnostic** (`POST /policy/check`) that surveys a candidate policy for grant/prohibit contradictions without mutating policy state. | | 9 | **Python library** | Python API library provides typed access to IdP and policy services via `aiac.idp.configuration`, `aiac.policy.model`, `aiac.policy.model_store.library`, `aiac.pdp.policy.library`, and `aiac.policy.computation` modules backed by generic Pydantic models. | ### High-level architecture @@ -312,6 +312,7 @@ All inter-pod traffic is Kubernetes ClusterIP. External access is exclusively vi - **Guardrails verification is a synchronous, per-document, pre-flight, fail-closed gate.** The RAG Ingest Service calls the Policy Guardrails Agent once per document before making any ChromaDB mutation; any rejection fails the whole request with nothing written, and an unreachable or erroring agent is treated the same as a rejection unless verification is explicitly disabled via `AIAC_GUARDRAILS_ENABLED`. - **The Policy Guardrails Agent has no Event Broker involvement.** It neither publishes nor consumes NATS subjects; the RAG Ingest Service's existing `aiac.apply.policy.build` publish is unchanged. - **AIAC Agent is stateless.** Changes are applied immediately on trigger — no pending session or human confirmation step. +- **Pre-commit conflict diagnostic is a separate read-only path.** `POST /policy/check` surveys a candidate policy for grant/prohibit contradictions but **never mutates policy state**, returns a `ConflictReport` rather than applying anything, and returns **`200` (not `422`) on a found conflict** — a found conflict is a successful diagnosis, not a failure. It is deliberately **distinct** from the live `/apply` contradiction contract (which raises `PolicyContradictionError` → `422` and aborts on the first genuine conflict), which stays byte-for-byte unchanged. Full spec: [components/aiac-agent/policy-conflict-check.md](components/aiac-agent/policy-conflict-check.md). - **Event Broker decouples all automated triggers from the Agent.** The Keycloak SPI listener and RAG Ingest Service publish to NATS subjects; the Agent subscribes as a durable competing consumer. This removes all direct dependencies between trigger sources and the Agent. - **`rebuild` bypasses the Event Broker.** It is an operator-only command issued directly via HTTP (`kubectl port-forward`). It is never published to NATS and has no NATS listener. - **NATS consumer is a thin adapter.** It receives events from the Event Broker and calls the same internal handler functions used by the debug HTTP endpoints. No business logic lives in the consumer. @@ -421,6 +422,8 @@ FastAPI + LangGraph service (`0.0.0.0:7070`). Receives automated triggers via th All sub-agent `StateGraph` instances are logically separated modules running within a single pod and process. Sub-UC agents produce `list[PolicyRule]` and call `compute_and_apply(rules)` — they do not call `aiac.policy.model_store.library` or `aiac.pdp.policy.library` directly. The **Policy Update** sub-agents compute a minimal rule delta between the current ChromaDB policy and live OPA state. The **Rebuild** variant additionally clears the Policy Model Store and all OPA policy rules before recomputing. The **Role Update** orchestrator computes rules for all services affected by the role change. The **Service Onboarding** orchestrator classifies the new service via the pod's `rossoctl.io/type` label (for agents reads the `AgentCard` CR; for tools calls `tools/list` on the MCP endpoint discovered via K8s Service label lookup), then computes rules and calls `compute_and_apply`. Stateless; changes are applied immediately. Integrated retry with differentiated error codes per upstream. +The Agent additionally exposes a **read-only pre-commit policy conflict diagnostic** at `POST /policy/check`: given candidate policy text plus a target service id, it surveys that service's focal entities and returns a `ConflictReport` (all grant/prohibit contradictions at once, with verbatim quotes) — it never mutates policy state, never calls the PCE, and does **not** `422` on a found conflict (a found conflict is a successful `200` diagnosis), distinct from the live `/apply` contradiction path. + **Full spec:** [components/aiac-agent.md](components/aiac-agent.md) --- diff --git a/aiac/docs/specs/components/aiac-agent.md b/aiac/docs/specs/components/aiac-agent.md index 7f44b86bc..115d9da1a 100644 --- a/aiac/docs/specs/components/aiac-agent.md +++ b/aiac/docs/specs/components/aiac-agent.md @@ -132,8 +132,9 @@ Each use case (and the UC1 Orchestrator) is specified in a dedicated sub-PRD: | Policy Update | [aiac-agent/uc2-policy-update.md](aiac-agent/uc2-policy-update.md) | `aiac.apply.policy.build`, `POST /apply/policy/build`, `POST /apply/policy/rebuild` | | | Role Update | [aiac-agent/uc3-role-update.md](aiac-agent/uc3-role-update.md) | `aiac.apply.role.{id}`, `POST /apply/role/{id}` | | | Service Offboarding | (see PCE `decommission`) | `POST /apply/offboard/{service_id}` (`aiac.apply.offboard.{id}` — NATS wiring is a follow-up) | Thin sub-agent; calls the PCE's `decommission(service_id)` **directly** (whole-service teardown, not a rule fold — bypasses the PRB and `compute_and_apply`). Keyed by **clientId, not UUID** (an offboarded client is gone from `get_services()`). | +| Policy Conflict Check (pre-commit diagnostic) | [aiac-agent/policy-conflict-check.md](aiac-agent/policy-conflict-check.md) | `POST /policy/check` (HTTP only — not routed through the Event Broker) | **Read-only** diagnostic: candidate `policy_text` + service id → `ConflictReport`. Reuses the PRB propose/precheck/audit machinery in a **separate assembly** (record-not-raise + `explain`). Does **not** call the PCE / `compute_and_apply`; never mutates policy; returns `200` on a found conflict. | -> **Note:** Each producing sub-agent (UC1–UC3) calls the **shared Policy Rules Builder** directly, merges the results, and returns `list[PolicyRule]` to the Controller. The Controller calls `compute_and_apply(merged_rules)` from `aiac.policy.computation` (PCE) once. Policy rule application is fully specified in [policy-computation-engine.md](policy-computation-engine.md). The Policy Rules Builder is specified in [aiac-agent/policy-rules-builder.md](aiac-agent/policy-rules-builder.md). **UC4 (Service Offboarding) is the exception:** it produces no rules — its handler resolves the clientId and calls the PCE's authoritative `decommission(service_id)` (specified in [policy-computation-engine.md → Decommission](policy-computation-engine.md#decommission-service-offboard)) to tear down the service's entire policy footprint. +> **Note:** Each producing sub-agent (UC1–UC3) calls the **shared Policy Rules Builder** directly, merges the results, and returns `list[PolicyRule]` to the Controller. The Controller calls `compute_and_apply(merged_rules)` from `aiac.policy.computation` (PCE) once. Policy rule application is fully specified in [policy-computation-engine.md](policy-computation-engine.md). The Policy Rules Builder is specified in [aiac-agent/policy-rules-builder.md](aiac-agent/policy-rules-builder.md). **UC4 (Service Offboarding) is the exception:** it produces no rules — its handler resolves the clientId and calls the PCE's authoritative `decommission(service_id)` (specified in [policy-computation-engine.md → Decommission](policy-computation-engine.md#decommission-service-offboard)) to tear down the service's entire policy footprint. **Policy Conflict Check is a further exception:** it is a **read-only** agent capability (not a UC number — the informal "UC4" is offboarding's), producing neither rules nor any PCE call. It surveys a candidate policy and returns a `ConflictReport`; the caller decides what to do with it. Full spec: [aiac-agent/policy-conflict-check.md](aiac-agent/policy-conflict-check.md). ### IdP access — library, not service @@ -151,6 +152,7 @@ Every sub-agent (UC1 Provision + Service Policy Builder, UC2 Build + Rebuild, UC | POST | `/apply/role/{role_id}` | Role Update | Role | | POST | `/apply/service/{service_id}` | Service Onboarding | Provision | | POST | `/apply/offboard/{service_id}` | Service Offboarding | Offboard (calls PCE `decommission` directly) | +| POST | `/policy/check` | Policy Conflict Check (diagnostic) | `check_policy_conflicts` — read-only; returns a `ConflictReport` JSON body | `GET /health` is a bare liveness/readiness probe: the Controller is stateless (no local state, no connection held at rest), so it answers `200 {"status": "ok"}` whenever the process is serving, dispatching to no handler and touching no upstream. Upstream reachability (IdP, PCE, NATS) is validated per-request by the handlers. The k8s Deployment wires both the readiness and liveness probes to it. @@ -158,6 +160,8 @@ The `/apply/offboard/{service_id}` path uses the `{service_id:path}` converter ( The `/apply/*` endpoints return bare HTTP status codes: `200 OK` on success (no response body), and the status codes from the Error Handling table on upstream failure. Success responses carry no body; upstream failures are raised as FastAPI `HTTPException`s, so error responses carry FastAPI's default JSON error body (`{"detail": ...}`) alongside the status code. Summary, applied-rule details, and debug information are written to the service log. Validation failures surface as an error status and log entry; detailed reporting is specified in [policy-rules-builder.md](aiac-agent/policy-rules-builder.md). +`POST /policy/check` is the **first endpoint that returns a JSON success body** (a `ConflictReport`), unlike the bare-status-code `/apply/*` routes. It is the read-only pre-commit conflict diagnostic: a **found conflict is a successful `200` diagnosis, NOT `422`** — only a pre-survey failure (IdP unreachable, unknown service, missing `policy_text`) is non-2xx. This is the opposite of the live `/apply` contradiction path, which raises `PolicyContradictionError` → `422`. Full spec: [aiac-agent/policy-conflict-check.md](aiac-agent/policy-conflict-check.md). + --- ## Configuration @@ -214,7 +218,7 @@ aiac/src/aiac/ ├── shared/ ← project-level shared: run_upstream (upstream.py) — transport retry primitive └── agent/ ├── controller/ - ├── shared/ ← flatten_role (roles.py) + ├── shared/ ← flatten_role (roles.py); focal_entities.py (resolve_focal_entities — D13, shared by live build() + diagnostic) ├── uc/ │ ├── onboarding/ │ │ ├── orchestrator.py ← sequences provision → policy_builder, returns list[PolicyRule] @@ -223,8 +227,11 @@ aiac/src/aiac/ │ ├── policy_update/ │ │ ├── build/ ← calls PRB, returns list[PolicyRule]; TBD internals │ │ └── rebuild/ ← delegates to Build; TBD internals - │ └── role_update/ ← calls PRB with (role, all_scopes), returns list[PolicyRule] + │ ├── role_update/ ← calls PRB with (role, all_scopes), returns list[PolicyRule] + │ └── policy_check/ ← read-only diagnostic: check_policy_conflicts(policy_text, service_id) → ConflictReport └── policy_rules_builder/ ← shared; called by Service Policy Builder, Build, and Role sub-agent + ├── diagnostic.py ← parallel diagnostic assembly (START-seeds-text, _audit_diagnostic record-not-raise, terminal _explain) + └── diagnostic_models.py ← ConflictReport + conflict/unevaluated row models ``` Docker build command (run from repo root): diff --git a/aiac/docs/specs/components/aiac-agent/policy-conflict-check.md b/aiac/docs/specs/components/aiac-agent/policy-conflict-check.md new file mode 100644 index 000000000..6a81dcaa0 --- /dev/null +++ b/aiac/docs/specs/components/aiac-agent/policy-conflict-check.md @@ -0,0 +1,283 @@ +# Sub-PRD: AIAC Agent — Policy Conflict Check (pre-commit diagnostic) + +> **Depends on:** [`../aiac-agent.md`](../aiac-agent.md) — Controller, Shared Module, Configuration, Error Handling, Runtime. + +> **Sits next to the live PRB contradiction path.** This diagnostic reuses the Policy Rules Builder machinery specified in [`policy-rules-builder.md`](policy-rules-builder.md), but is a **separate, read-only** path. The live `/apply` → `PolicyContradictionError` → HTTP 422 contract documented there is **UNCHANGED** by this feature. + +## Description + +The **Policy Conflict Check** is a **read-only pre-commit diagnostic** a caller invokes **before** +onboarding/committing a policy, to discover grant/prohibit contradictions ahead of time. Given a +candidate `policy_text` and a target service, it surveys **that service's** focal entities against +the live IdP catalog, runs the **same** proposer → precheck → audit machinery the live path uses, +and — instead of aborting on the first genuine contradiction the way `/apply` does — **records every +genuine contradiction and continues**. It returns a single `ConflictReport` listing **all** conflicts +at once (each with verbatim, substring-validated quotes, a plain-language explanation, and a conflict +`kind`), or the rendered form of `status: no_conflict` ("No conflict."). + +This is a **diagnostic gate**, not the commit. It **never mutates policy state**, never calls the PCE +(`compute_and_apply`), and never blocks anything by itself; the **caller decides** what to do with the +report. It does **not** replace or modify the live `/apply` → 422 path. + +--- + +## Interface + +- **New Controller route** — `POST /policy/check` (**recommended**; the exact path is a #154 open item, + `/policy/check` vs `/policy/conflicts`). The route is a **thin serialization shell** over a testable + plain function: + + ```python + def check_policy_conflicts(policy_text: str, service_id: str) -> ConflictReport: ... + ``` + + This mirrors how the repo separates graph logic from the `/apply` route, keeping the LLM-heavy logic + drivable by the `-m llm` suite without HTTP. + +- **Inputs:** + - **`policy_text`** — the **candidate prose supplied directly as an argument**, *not* read from + `AIAC_POLICY_FILE`. The diagnostic assembly's first node injects the supplied text instead of the + live graph's file-reading fetch. + - **`service_id`** — the target service's **Keycloak internal client UUID** (`s.id == service_id`), + matching `POST /apply/service/{service_id}`. + +- **Catalog resolution — live from the IdP.** Roles/scopes are resolved **live from the IdP** for the + target service — the **same** `own_scopes` / `candidate_roles` / `other_scopes` resolution + `builder.py` already performs (see [`uc1-service-onboarding.md` → Service Policy Builder](uc1-service-onboarding.md#sub-agent-service-policy-builder)). + This is deliberately **not** a text-only endpoint: conflicts are structural (same role + same scope), + which only means something against the real, typed entity set. It reuses `_precheck`'s hallucination + filtering so every reported entity is real. + +- **Scope of one check — per-service.** "All conflicts" = all conflicts involving **that service's** + focal entities, matching the commit unit (you onboard one service at a time). The whole-policy loop + (survey every already-onboarded service) is **deferred** — see **Out of scope**. + +--- + +## Reused machinery + code-contact deltas (D11–D14) + +The diagnostic is a **separate assembly** that reuses `_propose` / `_precheck` / the proposer + +auditor prompts **unchanged**, swaps in a **record-not-raise** audit node, and adds a **terminal +`explain` node**. A separate **sequential** survey orchestrator drives the per-entity units alongside +`builder.py`'s loop. Four code-contact points make the "reuse unchanged" framing plannable against the +current PRB code: + +- **D11 — the structured `kind` is produced by the new `explain` node, not read from the auditor.** + In the current code the conflict *kind* is not a typed value — it exists only as prose inside + `Contradiction.description`; `_precheck` collapses both kinds into one overlap signal. So the + `explain` call **classifies** `kind ∈ {direct, coarse_scope}` itself, from the policy text + the + `(role, scope)` pair, with the auditor's own `description` passed in as a **hint**. Chosen over adding + a `kind` field to the shared `Contradiction` / `AuditVerdict` models, which would touch the live + `/apply` path that D1/D8 require to stay byte-for-byte unchanged. (The auditor's `description` — the + adjudicator's own conclusion, not the proposer's `reasoning` that D9 excludes — is a safe hint.) +- **D12 — name→id join + run-direction tagging in the engine.** Audit output carries only **name + strings** (no ids, no typed objects). The survey orchestrator re-joins each name to the resolved + entity set (D13) to recover the id, and tags each recorded conflict with its **run direction** at + record time — scope-focal run ⇒ focal is the scope, candidate is the role; role-focal run ⇒ focal is + the role, candidate is the scope — which is how the report's `role(name+id)` / `scope(name+id)` sides + are assigned. +- **D13 — prerequisite refactor: extract a standalone resolver.** Focal-entity resolution is currently + inlined in `ServicePolicyBuilder.build()` and entangled with the fan-out loop. Extract it into a pure + `resolve_focal_entities(service_id, service_type) -> FocalEntitySet` (typed: `own_scopes`, + `own_roles`, `candidate_roles`, `other_scopes`), and have **both** the live `build()` and the + diagnostic call it. This is a **pure extraction — no behavior change to the live path** (covered by + existing builder tests). The existing `HTTPException(502/404)` on IdP-unreachable / unknown-service + moves into the resolver and directly satisfies this feature's pre-survey HTTP boundary. +- **D14 — "separate assembly" = fork the audit node + build a parallel graph; there is no swap seam.** + The raise is hard-coded in `_audit` with no strategy seam, and the live START node (`_fetch`) calls + the hard-coded `get_policy_source()` and overwrites any input `policy_text`. So there is **NO + node-swap / source-injection seam**. The diagnostic instead uses: (i) a START node that **seeds + `policy_text` from input state** (no file read); (ii) `_propose` / `_precheck` reused unchanged; + (iii) a forked `_audit_diagnostic` that **records** contradictions into state instead of raising; + (iv) a terminal `_explain` node (D11). The live `graph.py` `_audit` and the `build_*` entry points + stay **untouched**. + +--- + +## Settled design decisions + +(Condensed from handoff 04 §4, decisions D1–D10.) + +- **D1 — Separate diagnostic, live path untouched.** A read-only pre-commit tool, distinct from + `/apply`. The live commit path keeps raising `PolicyContradictionError` → 422. +- **D2 — Text + live IdP catalog** (not text-only). A structural conflict definition requires a real + typed entity set; text-only extraction cannot dedupe/validate entity names or drop hallucinations, + which is exactly wrong for a gate. +- **D3 — Provenance is conflict-only.** Clean rules carry **no** citation. Only auditor-confirmed + genuine conflicts get quotes, produced by a dedicated explanation prompt. Many rules have no quotable + statement (description-derived grants, exclusivity-complement denies, deny-by-default), so forcing + per-rule citations would maximize hallucination where there's nothing to cite. +- **D4 — Explain only auditor-confirmed genuine conflicts** — not every `_precheck` overlap. The + auditor's genuine-vs-slip adjudication keeps proposer noise out of the report; slips still trigger the + normal retry (which usually erases the bogus overlap). +- **D5 — Verbatim + validated quotes.** The explanation prompt returns exact substrings of the policy + text; the tool checks each is a substring (whitespace-normalized). Each side is a **list of spans** + ("one or more" statements). On validation failure: **keep** the conflict, set `quotes_verified=false`, + fall back to the auditor `description`. +- **D6 — Report both conflict kinds:** `direct` **and** `coarse_scope` granularity. A coarse-scope + contradiction ("management granted, writing forbidden") is genuine and more insidious; the machinery + already distinguishes the two, so it is nearly free. +- **D7 — Collect-all survey, never abort.** Run **every** focal entity to completion; accumulate + conflicts + un-evaluatable entities. The audit node in diagnostic mode **records** genuine + contradictions instead of raising. +- **D8 — Separate diagnostic assembly**, reusing `_propose` / `_precheck` / prompts unchanged; swap in a + record-not-raise audit node + a new terminal `explain` node; a separate survey orchestrator alongside + `builder.py`'s loop. A `diagnostic: bool` flag *inside* the live audit node was **rejected** — it + risks a conflict that should 422 a commit silently becoming a recorded-and-ignored report. +- **D9 — Explanation prompt: one call per conflict pair.** Inputs = policy text + the one + `(role, scope)` pair + the auditor's kind label. **Not** the proposer's free-form `reasoning` (LLM + narration that could anchor extraction onto a hallucinated justification). Conflicts are rare + (usually 0), so per-pair isolation gives the cleanest verbatim extraction with no cross-pair + contamination. +- **D10 — Identity by entity id; no cross-run reconciliation.** The fan-out is disjoint (scope-focal = + candidate role × own scope; role-focal = own role × other-service scope; own vs. other split by + `serviceId`), so the same pair is never decided twice. A conflict is always a within-single-run + overlap. + +--- + +## Pipeline (diagnostic assembly) + +Per focal entity: + +``` +inject candidate policy_text (START seeds text from input; replaces the file-reading _fetch) + → _propose (reused unchanged) + → _precheck (reused unchanged; flags candidates in both the grant and prohibit lists) + → _audit_diagnostic: + genuine contradiction → RECORD (do NOT raise), continue + proposer slip → ordinary rejection → re-propose (≤ MAX_AUDIT_RETRIES) + retry budget exhausted → mark entity UNEVALUATED (do NOT raise) + → _explain (terminal; runs only if this entity has recorded genuine conflicts): + one LLM call per (role, scope) pair + → granting_quotes[], prohibiting_quotes[] (verbatim, validated substrings) + → explanation, kind (kind classified here — D11) + → quotes_verified (false ⇒ fall back to the auditor description) +``` + +A **sequential** survey orchestrator (alongside the onboarding builder's loop) runs **every** focal +entity to completion — the first conflict never aborts — then assembles the report. Concurrency is +deferred (see **Out of scope**). A CHECK-node touch on the component Mermaid diagram is **optional** and +noted here rather than forced into that diagram. + +--- + +## Report + status contract + +``` +ConflictReport: + conflicts: [{ focal: { name, id, type }, # type ∈ {"role", "scope"} — which side drove the run + role: { name, id }, + scope: { name, id }, + kind, # kind ∈ {"direct", "coarse_scope"} + granting_quotes: list[str], # verbatim substrings of policy_text + prohibiting_quotes: list[str], # verbatim substrings of policy_text + explanation, + quotes_verified }] + unevaluated: [{ focal: { name, id, type }, + reason, # enum { "nonconvergence" } + detail }] # optional free-text + status: "no_conflict" | "conflicts_found" | "incomplete" +``` + +### Status enum + precedence (RESOLVED — authoritative) + +The status is derived by this precedence, exactly: + +``` +if conflicts: -> conflicts_found +elif evaluated_count == 0 or unevaluated: -> incomplete +else: -> no_conflict +``` + +Both `incomplete` disjuncts are **load-bearing**: `evaluated_count == 0` catches the empty-input / +zero-focal case; `unevaluated != []` catches retry-exhaustion on some entities while the rest are clean. +`no_conflict` is reached **only** when ≥1 entity was evaluated **and** `conflicts == []` **and** +`unevaluated == []`. The literal string "No conflict." is the *rendered* form of `no_conflict` — never a +bare string that could mask an incomplete run (guards against an outage looking identical to a clean +policy). + +### Quote-validator rule + +**Whitespace-normalize ONLY** before the substring check: collapse all whitespace runs (including +newlines) to a single space and trim the ends. Deliberately **no** case-folding and **no** smart-quote / +punctuation normalization — the quote must be findable *as written* in the author's prose; normalizing +punctuation would let a non-findable near-quote pass. On mismatch (including smart-quote mismatches): +**keep** the conflict, set `quotes_verified=false`, and fall back to the auditor `description`. Never a +silent "fix." `granting_quotes` / `prohibiting_quotes` are `list[str]` of verbatim substrings — not +offset spans (nothing consumes offsets). + +### HTTP status boundary + +- **Pre-survey failure** — can't build the entity set (IdP unreachable, unknown service, missing + `policy_text`) ⇒ **non-2xx**, **no report** (502 for upstream IdP; 400/422 for bad input). The D13 + resolver's existing `HTTPException(502/404)` satisfies this directly. +- **Per-entity failure during the survey** — an entity won't converge / exhausts retries ⇒ **200**, + entity listed under `unevaluated`, status ≠ `no_conflict`. +- **Any completed survey** — conflicts or clean, possibly partial ⇒ **200, never 422**. A found + conflict is a *successful diagnosis*, not an error — unlike the live `/apply`, this tool does **not** + 422 on conflict. This is the controller's **first JSON response body** (existing `/apply/*` routes + return bare status codes). + +### Report boundary note + +`status == no_conflict` means "no conflict introduced by **this service's** rules," **not** "the global +policy is conflict-free." This is documented in the report so callers don't over-read a per-service +result. + +--- + +## Testing + +Three tiers, mirroring the repo's existing split (deterministic unit tests patch the `_structured_call` +seam; the opt-in `-m llm` suite runs the real model — see [`policy-rules-builder.md` → Testing](policy-rules-builder.md#testing)): + +1. **Deterministic unit (patch `_structured_call`)** — the bulk. The shared `_structured_call` LLM seam + already covers proposer, auditor, **and** the new per-pair explanation call (a `side_effect` list + drives all three), so no new patch seam is introduced. Cover: survey **orchestration** (all entities + run; first conflict does **not** abort), conflict → **report assembly**, the **`unevaluated` path** on + retry exhaustion, the **zero-evaluated guard**, **`status` derivation** (the precedence above), and + the **verbatim-quote validator** as a pure function (substring match, whitespace normalization, + `quotes_verified=false` fallback to `description`). +2. **Live-LLM (`-m llm`)** — small, **blatant** planted fixtures: one **direct** conflict, one + **coarse-scope** conflict, and one **clean** policy. Assert **structural** properties only: clean ⇒ + `no_conflict`; planted ⇒ the confirmed set **contains** the planted pair(s), and **every** returned + quote is a verbatim substring of the policy text. **Do not** assert exact quote strings or explanation + wording (model nondeterminism; convergence on subtle prose is known-fragile). +3. **Route** — one thin test that the endpoint calls the survey function and serializes report + status + codes (function patched, no LLM). + +--- + +## Out of scope + +- **Whole-policy audit** (survey *every* already-onboarded service): desired later as a **loop over the + per-service check** — a **separate effort**, not this work. +- **Survey concurrency:** assume **sequential** (matches `builder.py`); parallelize only if latency + demands. +- **Route path name:** decide at implementation (`/policy/check` vs `/policy/conflicts`). +- **Subtle-prose robustness / PRB precedence tuning** (explicit prohibition vs description-derived + grant): a separate PRB-quality concern, not a correctness gate for this feature. +- **ALLOW-vs-DENY precedence / tie-break at enforcement time:** a distinct PCE/Rego concern (tracked in + #124). This tool *reports* contradictions; it does not *resolve* them. +- **No change to the live `/apply` path** or its 422 contradiction contract. +- **Provenance on non-conflicting rules:** descoped — clean rules carry no citation. + +--- + +## Acceptance criteria + +(Carried over from #154 / handoff 04 §9.) + +1. `POST` with a clean candidate policy + a real service ⇒ 200, `status: no_conflict`. +2. `POST` with a candidate policy containing a **direct** and a **coarse-scope** conflict ⇒ 200, + `status: conflicts_found`, **both** reported in a single response, each with validated verbatim + `granting_quotes` / `prohibiting_quotes` (or `quotes_verified=false` + description fallback). +3. An entity that cannot converge appears under `unevaluated` with a reason; the run still returns 200 + and does **not** report `no_conflict`. +4. IdP unreachable / unknown service / missing `policy_text` ⇒ non-2xx, no report. +5. Zero entities evaluated ⇒ `status: incomplete`, never `no_conflict`. +6. The live `/apply` path and its 422 behavior are unchanged (regression check). +7. Deterministic unit tests cover orchestration, report assembly, `unevaluated`, the zero-evaluated + guard, and the quote validator/fallback; the `-m llm` suite asserts containment + substring-validity + on planted/clean fixtures. diff --git a/aiac/docs/specs/components/aiac-agent/policy-rules-builder.md b/aiac/docs/specs/components/aiac-agent/policy-rules-builder.md index 2ca2d4c8d..8a0f1dc3e 100644 --- a/aiac/docs/specs/components/aiac-agent/policy-rules-builder.md +++ b/aiac/docs/specs/components/aiac-agent/policy-rules-builder.md @@ -250,6 +250,13 @@ The PRB is the producer that must **guarantee** this — it must never pass a co downstream. Detection and reporting live here; the **treatment** of a reported contradiction (surface to a human, partial-apply, re-author the policy, split the scope) is a **separate, deferred** task. +> **Read-only pre-commit diagnostic.** The collect-all, quote-bearing form of that treatment — a +> read-only pre-commit tool that surveys a candidate policy, records **all** genuine conflicts at once +> (never aborting on the first), and returns a `ConflictReport` with verbatim quotes — is now specified +> in [`policy-conflict-check.md`](policy-conflict-check.md). It reuses this module's proposer / precheck +> / audit machinery but is a **separate assembly**; the live `/apply` → `PolicyContradictionError` → 422 +> path documented in this section is **UNCHANGED** by it. + - **Detection is deterministic** (in `precheck`): `conflict_names = granted_names ∩ denied_names`, after candidate-set filtering. Precheck resolves nothing; it only stores the overlap. Because the derived exclusivity complement is disjoint from grants by construction, overlap can arise **only** @@ -319,6 +326,7 @@ needs the full onboarding stack), `llm` needs only an LLM endpoint. Both are des | UC1 — Service Onboarding | Service Policy Builder sub-agent | `build_scope_rules(other_roles, scope)` per agent/tool scope + `build_role_rules(role, other_scopes)` per agent role (agent path only) | | UC2 — Policy Update (Build) | Build sub-agent | TBD | | UC3 — Role Update | Role sub-agent | `build_role_rules(role, all_scopes)` — one call | +| Policy Conflict Check (diagnostic) | Controller (`POST /policy/check`) → `check_policy_conflicts` | A parallel diagnostic assembly reusing propose / precheck / audit (record-not-raise + a terminal `explain` node) — see [`policy-conflict-check.md`](policy-conflict-check.md) | --- From 4bd4a93438e25e6d2c2f72671b15162a48c6ce68 Mon Sep 17 00:00:00 2001 From: Anatoly Koyfman Date: Wed, 19 Aug 2026 08:32:26 +0000 Subject: [PATCH 2/8] Refactor: Extract focal-entity resolver (prefactor for #154) Extract the focal-entity resolution inlined in ServicePolicyBuilder.build() into a standalone resolve_focal_entities(service_id, service_type) -> FocalEntitySet in agent/shared/focal_entities.py, callable by both the live builder and the upcoming policy-conflict diagnostic. Pure extraction, no behavior change to the live /apply path: existing builder tests pass unchanged. The HTTPException(502) (IdP unreachable) / HTTPException(404) (unknown service) boundary moves into the resolver, where the diagnostic's pre-survey HTTP boundary needs it. builder.py keeps routing service_type through its parameter (not focus.type), preserving the live /apply contract. Refs #155. Part of #154. Assisted-By: Claude (Anthropic AI) Signed-off-by: Anatoly Koyfman --- aiac/src/aiac/agent/shared/focal_entities.py | 146 +++++++++++++ .../uc/onboarding/policy_builder/builder.py | 92 ++------- aiac/test/agent/shared/__init__.py | 0 aiac/test/agent/shared/test_focal_entities.py | 194 ++++++++++++++++++ 4 files changed, 356 insertions(+), 76 deletions(-) create mode 100644 aiac/src/aiac/agent/shared/focal_entities.py create mode 100644 aiac/test/agent/shared/__init__.py create mode 100644 aiac/test/agent/shared/test_focal_entities.py diff --git a/aiac/src/aiac/agent/shared/focal_entities.py b/aiac/src/aiac/agent/shared/focal_entities.py new file mode 100644 index 000000000..f541a480d --- /dev/null +++ b/aiac/src/aiac/agent/shared/focal_entities.py @@ -0,0 +1,146 @@ +"""Focal-entity resolution for a target service (shared). + +Extracted from ``ServicePolicyBuilder.build()`` (D13) so both the live Service Policy +Builder and the read-only Policy Conflict Check diagnostic resolve the **same** typed entity +set from the live IdP catalog. This is a **pure extraction** — the resolution logic is +byte-for-byte the same split the builder performed inline; no live behavior changes. + +The focus service is resolved from ``get_services()`` by ``id`` (the Keycloak internal client +UUID the ``/apply/service/{id}`` route and ``Trigger.entity_id`` carry — **not** +``serviceId``/clientId, which may be a slash-bearing SPIFFE URI). Candidates are +excluded/included by **ownership** (role id / ``scope.serviceId``), never by name: + +- ``own_roles`` / ``own_scopes`` — the focus service's own ``aiac.managed`` roles/scopes. +- ``candidate_roles`` — the flattened, de-duplicated union of (a) other services' + ``aiac.managed`` roles (``kind=Agent``) and (b) realm roles held by at least one user + (composite-expanded, and not owned by any service; ``kind=User``). +- ``other_scopes`` — other services' ``aiac.managed`` scopes, sourced from ``get_services()`` + so each scope carries its owning ``serviceId`` (the SPM routing key the PCE needs). + +IdP access is via the **idp-library** ``Configuration`` seam. Callers that already hold a +``Configuration`` (e.g. the live builder, whose ``_config`` seam existing tests patch) pass it +in via ``config``; callers that don't (e.g. the diagnostic) let the resolver create the +default-realm one. ``HTTPException(502)`` is raised on IdP-unreachable, ``HTTPException(404)`` +on unknown service — this is the feature's pre-survey HTTP boundary. +""" + +from fastapi import HTTPException +from pydantic import BaseModel, ConfigDict + +from aiac.agent.shared.roles import flatten_role +from aiac.idp.configuration.api import Configuration +from aiac.idp.configuration.models import Role, Scope, ServiceType + + +class FocalEntitySet(BaseModel): + """Typed result of :func:`resolve_focal_entities` — the focus service's own entities plus + the candidate universes it maps against. ``service_type`` is echoed from the caller's + parameter (the requested classification for the fan-out), **not** ``focus.type``.""" + + model_config = ConfigDict(arbitrary_types_allowed=True) + + own_scopes: list[Scope] + own_roles: list[Role] + candidate_roles: list[Role] + other_scopes: list[Scope] + service_type: ServiceType + + +def _config() -> Configuration: + return Configuration.for_default_realm() + + +def _flatten_dedup(roles: list[Role]) -> list[Role]: + """Union of every role's closure, de-duplicated by ``role.id``.""" + out: list[Role] = [] + seen: set[str] = set() + for role in roles: + for member in flatten_role(role): + if member.id not in seen: + seen.add(member.id) + out.append(member) + return out + + +def resolve_focal_entities( + service_id: str, + service_type: ServiceType, + *, + config: Configuration | None = None, +) -> FocalEntitySet: + """Resolve the focus service's own entities + candidate universes from the live IdP catalog. + + ``service_id`` is the Keycloak internal client UUID (``Service.id``), not the + human-readable ``serviceId``/clientId. ``service_type`` is the requested classification and + is echoed onto the result unchanged (the caller routes on it — it is **not** derived from + ``focus.type``). Pass ``config`` to reuse an existing ``Configuration`` seam; otherwise the + default-realm one is created. + + Raises ``HTTPException(502)`` when the IdP Configuration Service is unreachable and + ``HTTPException(404)`` when ``service_id`` is absent from the catalog. + """ + config = config or _config() + + try: + services = config.get_services() + subjects = config.get_subjects() + except Exception as e: + raise HTTPException( + 502, f"IdP Configuration Service unavailable for service {service_id!r}: {e}" + ) + + # The trigger id is the Keycloak internal client UUID (Service.id), not the human-readable + # clientId (Service.serviceId): the /apply/service/{id} route is keyed on the UUID because a + # clientId can be a slash-bearing SPIFFE URI the single-segment route cannot carry. + focus = next((s for s in services if s.id == service_id), None) + if focus is None: + raise HTTPException(404, f"service {service_id!r} not found in IdP catalog") + + own_roles = [r for r in focus.roles if r.aiac_managed] + own_scopes = [s for s in focus.scopes if s.aiac_managed] + + # kind=Agent rides through unchanged from get_services() → routes to source_roles in the PCE. + other_agent_roles = [ + r + for other in services + if other.serviceId != focus.serviceId + for r in other.roles + if r.aiac_managed + ] + + # User roles are membership-derived, not aiac.managed: a realm role qualifies iff a user + # holds it directly or via a composite parent they hold, and no service owns it. + service_owned_ids = {r.id for s in services for r in s.roles} + user_roles_by_id: dict[str, Role] = {} + for subject in subjects: + for role in subject.roles: + # NB: flatten_role on an agent composite role would yield children whose kind + # defaults to User (composites endpoint doesn't carry per-service kind) — a latent + # edge case if a user is ever assigned a composite agent role. Not hit here. + for member in flatten_role(role): + if member.id not in service_owned_ids: + user_roles_by_id[member.id] = member + user_roles = list(user_roles_by_id.values()) + + # Other services' aiac.managed scopes, sourced from get_services() (mirroring + # other_agent_roles) so each scope carries its owning serviceId — the SPM routing key the + # PCE needs. The global get_scopes() endpoint returns scopes with an empty serviceId, which + # would both (a) fail to exclude the focus's own scopes (``"" != focus.serviceId`` is always + # true) and (b) route any resulting rule to ``SPM("")``, a 422 dead-end. + other_scopes = [ + s + for other in services + if other.serviceId != focus.serviceId + for s in other.scopes + if s.aiac_managed + ] + + candidate_roles = _flatten_dedup(user_roles + other_agent_roles) + + return FocalEntitySet( + own_scopes=own_scopes, + own_roles=own_roles, + candidate_roles=candidate_roles, + other_scopes=other_scopes, + service_type=service_type, + ) diff --git a/aiac/src/aiac/agent/uc/onboarding/policy_builder/builder.py b/aiac/src/aiac/agent/uc/onboarding/policy_builder/builder.py index f0a536c28..daed94df1 100644 --- a/aiac/src/aiac/agent/uc/onboarding/policy_builder/builder.py +++ b/aiac/src/aiac/agent/uc/onboarding/policy_builder/builder.py @@ -20,14 +20,20 @@ services' ``aiac.managed`` roles carry ``kind=Agent``; realm roles held by at least one user (composite-expanded, and not owned by any service) carry ``kind=User``. This keeps ``subject_roles``/``source_roles`` routing correct downstream in the PCE. -""" -from fastapi import HTTPException +The focal-entity resolution itself (the own-scope / candidate-role / other-scope split, and +the IdP-unreachable / unknown-service ``HTTPException(502/404)`` boundary) lives in the shared +``resolve_focal_entities`` (D13) so the read-only Policy Conflict Check diagnostic can reuse +the exact same entity set. This module keeps only the fan-out loop over that set. The local +``_config`` seam is preserved and threaded into the resolver so existing tests patch it as +before. +""" from aiac.agent.policy_rules_builder.graph import build_role_rules, build_scope_rules +from aiac.agent.shared.focal_entities import resolve_focal_entities from aiac.agent.shared.roles import flatten_role from aiac.idp.configuration.api import Configuration -from aiac.idp.configuration.models import Role, ServiceType +from aiac.idp.configuration.models import ServiceType from aiac.policy.model.models import PolicyRule @@ -35,84 +41,18 @@ def _config() -> Configuration: return Configuration.for_default_realm() -def _flatten_dedup(roles): - """Union of every role's closure, de-duplicated by ``role.id``.""" - out = [] - seen: set[str] = set() - for role in roles: - for member in flatten_role(role): - if member.id not in seen: - seen.add(member.id) - out.append(member) - return out - - class ServicePolicyBuilder: @staticmethod def build(service_id: str, service_type: ServiceType) -> list[PolicyRule]: - config = _config() - - try: - services = config.get_services() - subjects = config.get_subjects() - except Exception as e: - raise HTTPException( - 502, f"IdP Configuration Service unavailable for service {service_id!r}: {e}" - ) - - # The trigger id is the Keycloak internal client UUID (Service.id), not the human-readable - # clientId (Service.serviceId): the /apply/service/{id} route is keyed on the UUID because a - # clientId can be a slash-bearing SPIFFE URI the single-segment route cannot carry. - focus = next((s for s in services if s.id == service_id), None) - if focus is None: - raise HTTPException(404, f"service {service_id!r} not found in IdP catalog") - - own_roles = [r for r in focus.roles if r.aiac_managed] - own_scopes = [s for s in focus.scopes if s.aiac_managed] - - # kind=Agent rides through unchanged from get_services() → routes to source_roles in the PCE. - other_agent_roles = [ - r - for other in services - if other.serviceId != focus.serviceId - for r in other.roles - if r.aiac_managed - ] - - # User roles are membership-derived, not aiac.managed: a realm role qualifies iff a user - # holds it directly or via a composite parent they hold, and no service owns it. - service_owned_ids = {r.id for s in services for r in s.roles} - user_roles_by_id: dict[str, Role] = {} - for subject in subjects: - for role in subject.roles: - # NB: flatten_role on an agent composite role would yield children whose kind - # defaults to User (composites endpoint doesn't carry per-service kind) — a latent - # edge case if a user is ever assigned a composite agent role. Not hit here. - for member in flatten_role(role): - if member.id not in service_owned_ids: - user_roles_by_id[member.id] = member - user_roles = list(user_roles_by_id.values()) - - # Other services' aiac.managed scopes, sourced from get_services() (mirroring - # other_agent_roles) so each scope carries its owning serviceId — the SPM routing key the - # PCE needs. The global get_scopes() endpoint returns scopes with an empty serviceId, which - # would both (a) fail to exclude the focus's own scopes (``"" != focus.serviceId`` is always - # true) and (b) route any resulting rule to ``SPM("")``, a 422 dead-end. - other_scopes = [ - s - for other in services - if other.serviceId != focus.serviceId - for s in other.scopes - if s.aiac_managed - ] - - candidate_roles = _flatten_dedup(user_roles + other_agent_roles) + # service_type is routed through the parameter (the requested classification for the + # fan-out), never conflated with focus.type — see #154 AC#6. + focal = resolve_focal_entities(service_id, service_type, config=_config()) rules: list[PolicyRule] = [] - for scope in own_scopes: - rules.extend(build_scope_rules(candidate_roles, scope)) + for scope in focal.own_scopes: + rules.extend(build_scope_rules(focal.candidate_roles, scope)) if service_type is ServiceType.AGENT: - for own_role in own_roles: + for own_role in focal.own_roles: for role in flatten_role(own_role): - rules.extend(build_role_rules(role, other_scopes)) + rules.extend(build_role_rules(role, focal.other_scopes)) return rules diff --git a/aiac/test/agent/shared/__init__.py b/aiac/test/agent/shared/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/aiac/test/agent/shared/test_focal_entities.py b/aiac/test/agent/shared/test_focal_entities.py new file mode 100644 index 000000000..b50e23543 --- /dev/null +++ b/aiac/test/agent/shared/test_focal_entities.py @@ -0,0 +1,194 @@ +"""Unit tests for the extracted focal-entity resolver (D13). + +``resolve_focal_entities`` is the pure resolution logic lifted out of +``ServicePolicyBuilder.build()`` so both the live builder and the read-only Policy Conflict +Check diagnostic share one typed entity set. These tests exercise the resolver directly with a +mocked ``Configuration`` (passed via the ``config`` seam — no live IdP, no LLM): + +- the own-scope / candidate-role / other-scope ownership split (by role id / ``scope.serviceId``, + never by name), +- composite-role flatten + de-dup of the candidate universe, +- membership-derived user roles vs ``aiac.managed`` agent roles, with self-owned exclusion, +- ``service_type`` echoed from the parameter (not ``focus.type``), +- the ``HTTPException(502/404)`` pre-survey boundary. + +The live-builder-facing behavior is covered separately by ``test_builder.py``; here we assert on +the ``FocalEntitySet`` fields the diagnostic will consume. +""" + +from unittest.mock import MagicMock + +import pytest +from fastapi import HTTPException + +from aiac.agent.shared.focal_entities import FocalEntitySet, resolve_focal_entities +from aiac.idp.configuration.models import RoleKind, Scope, Service, ServiceType, Subject +from aiac.idp.configuration.models import Role as RoleModel + +FOCUS_ID = "svc-focus" +OTHER_ID = "svc-other" + + +def _role(name, *, role_id=None, composite=False, children=None, kind=RoleKind.USER, aiac_managed=True): + return RoleModel( + id=role_id or f"{name}-id", + name=name, + description=name, + composite=composite, + childRoles=children or [], + attributes={"aiac.managed": ["true"]} if aiac_managed else {}, + kind=kind, + ) + + +def _scope(name, *, scope_id=None, service_id="", aiac_managed=True): + return Scope( + id=scope_id or f"{name}-id", + name=name, + description=name, + attributes={"aiac.managed": "true"} if aiac_managed else {}, + serviceId=service_id, + ) + + +def _service(service_id, *, ref=None, roles=None, scopes=None, service_type=ServiceType.TOOL): + return Service( + id=service_id, + serviceId=ref or service_id, + enabled=True, + type=service_type, + roles=roles or [], + scopes=scopes or [], + ) + + +def _subject(username, *, roles=None, subject_id=None): + return Subject(id=subject_id or f"{username}-id", username=username, enabled=True, roles=roles or []) + + +def _resolve(service_type, *, services, subjects, service_id=FOCUS_ID): + conf = MagicMock() + conf.get_services.return_value = services + conf.get_subjects.return_value = subjects + return resolve_focal_entities(service_id, service_type, config=conf) + + +class TestSplit: + def test_own_and_candidate_and_other_universes_partitioned_by_ownership(self): + own_role = _role("weather.agent") + own_scope = _scope("weather.forecast", service_id=FOCUS_ID) + other_role = _role("github.agent", kind=RoleKind.AGENT) + other_scope = _scope("github.issue", service_id=OTHER_ID) + focus = _service(FOCUS_ID, roles=[own_role], scopes=[own_scope], service_type=ServiceType.AGENT) + other = _service(OTHER_ID, roles=[other_role], scopes=[other_scope]) + + result = _resolve(ServiceType.AGENT, services=[focus, other], subjects=[]) + + assert isinstance(result, FocalEntitySet) + assert [s.name for s in result.own_scopes] == ["weather.forecast"] + assert [r.name for r in result.own_roles] == ["weather.agent"] + assert [r.name for r in result.candidate_roles] == ["github.agent"] + assert [s.name for s in result.other_scopes] == ["github.issue"] + + def test_non_aiac_managed_own_entities_dropped(self): + managed_scope = _scope("weather.forecast", service_id=FOCUS_ID) + builtin_scope = _scope("profile", service_id=FOCUS_ID, aiac_managed=False) + managed_role = _role("weather.agent") + builtin_role = _role("uma_protection", aiac_managed=False) + focus = _service( + FOCUS_ID, + roles=[managed_role, builtin_role], + scopes=[managed_scope, builtin_scope], + service_type=ServiceType.AGENT, + ) + + result = _resolve(ServiceType.AGENT, services=[focus], subjects=[]) + + assert [s.name for s in result.own_scopes] == ["weather.forecast"] + assert [r.name for r in result.own_roles] == ["weather.agent"] + + +class TestCandidateRoles: + def test_composite_other_role_flattened_and_deduped_by_id(self): + reader = _role("github.reader", role_id="reader-id", kind=RoleKind.AGENT) + admin = _role( + "github.admin", role_id="admin-id", composite=True, children=[reader], kind=RoleKind.AGENT + ) + focus = _service(FOCUS_ID, scopes=[_scope("weather.forecast", service_id=FOCUS_ID)]) + other = _service(OTHER_ID, roles=[admin, reader]) + + result = _resolve(ServiceType.TOOL, services=[focus, other], subjects=[]) + + assert [r.id for r in result.candidate_roles] == ["admin-id", "reader-id"] + + def test_user_role_included_as_candidate_and_self_owned_role_excluded(self): + own_role = _role("weather.admin", role_id="own-role-id") + user_role = _role("realm.viewer", kind=RoleKind.USER, aiac_managed=False) + # a user also holds the focus's own role — ownership must still exclude it + subject = _subject("alice", roles=[user_role, own_role]) + focus = _service(FOCUS_ID, roles=[own_role], scopes=[_scope("weather.forecast", service_id=FOCUS_ID)]) + other = _service(OTHER_ID) + + result = _resolve(ServiceType.TOOL, services=[focus, other], subjects=[subject]) + + assert [r.name for r in result.candidate_roles] == ["realm.viewer"] + assert result.candidate_roles[0].kind == RoleKind.USER + + def test_other_scopes_excludes_focus_owned_by_serviceid(self): + own_scope = _scope("shared.scope", scope_id="own-scope-id", service_id=FOCUS_ID) + other_scope = _scope("shared.scope", scope_id="other-scope-id", service_id=OTHER_ID) + # same-named scope, different owner: exclusion is by serviceId, not name + focus = _service(FOCUS_ID, scopes=[own_scope]) + other = _service(OTHER_ID, scopes=[other_scope]) + + result = _resolve(ServiceType.TOOL, services=[focus, other], subjects=[]) + + assert [s.id for s in result.other_scopes] == ["other-scope-id"] + + +class TestServiceType: + def test_service_type_echoed_from_parameter_not_focus_type(self): + # focus.type is TOOL, but the requested classification is AGENT — the result must carry + # the parameter, never focus.type. + focus = _service(FOCUS_ID, service_type=ServiceType.TOOL) + + result = _resolve(ServiceType.AGENT, services=[focus], subjects=[]) + + assert result.service_type is ServiceType.AGENT + + +class TestErrors: + def test_idp_unreachable_raises_502(self): + conf = MagicMock() + conf.get_services.side_effect = RuntimeError("HTTP 503") + with pytest.raises(HTTPException) as ei: + resolve_focal_entities(FOCUS_ID, ServiceType.TOOL, config=conf) + assert ei.value.status_code == 502 + + def test_get_subjects_unreachable_raises_502(self): + conf = MagicMock() + conf.get_services.return_value = [_service(FOCUS_ID)] + conf.get_subjects.side_effect = RuntimeError("HTTP 500") + with pytest.raises(HTTPException) as ei: + resolve_focal_entities(FOCUS_ID, ServiceType.TOOL, config=conf) + assert ei.value.status_code == 502 + + def test_unknown_service_raises_404(self): + with pytest.raises(HTTPException) as ei: + _resolve(ServiceType.TOOL, services=[_service(OTHER_ID)], subjects=[], service_id="nope") + assert ei.value.status_code == 404 + + def test_focus_resolved_by_uuid_not_clientid(self): + uuid = "f5592be1-uuid" + client_id = "spiffe://localtest.me/ns/team1/sa/github-agent" + focus = _service(uuid, ref=client_id, scopes=[_scope("weather.forecast", service_id=uuid)]) + other = _service(OTHER_ID, ref="svc-other-client") + + # the UUID resolves + result = _resolve(ServiceType.TOOL, services=[focus, other], subjects=[], service_id=uuid) + assert [s.name for s in result.own_scopes] == ["weather.forecast"] + + # the clientId does not + with pytest.raises(HTTPException) as ei: + _resolve(ServiceType.TOOL, services=[focus, other], subjects=[], service_id=client_id) + assert ei.value.status_code == 404 From 114a505fe1d98333e248d6c0fb9466bc63297b81 Mon Sep 17 00:00:00 2001 From: Anatoly Koyfman Date: Wed, 19 Aug 2026 08:32:40 +0000 Subject: [PATCH 3/8] Feat: Add conflict report + status models (#154) Add agent/policy_rules_builder/diagnostic_models.py with the ConflictReport / Conflict / Unevaluated models, the EntityRef / FocalRef refs, and the FocalType / ConflictKind / ConflictStatus / UnevaluatedReason enums -- the stable structured shape the diagnostic engine, survey use-case, and route all serialize. Encodes the pinned focal{name,id,type} / role{name,id} / scope{name,id} / granting_quotes / prohibiting_quotes / quotes_verified contract. Includes ConflictReport.derive_status / from_survey encoding the status precedence (conflicts_found > incomplete > no_conflict) for the survey use-case to apply. The live /apply models (Contradiction, AuditVerdict) are untouched -- no kind field added to shared models, per D11. Refs #156. Part of #154. Assisted-By: Claude (Anthropic AI) Signed-off-by: Anatoly Koyfman --- .../policy_rules_builder/diagnostic_models.py | 150 +++++++++++++++ .../test_diagnostic_models.py | 182 ++++++++++++++++++ 2 files changed, 332 insertions(+) create mode 100644 aiac/src/aiac/agent/policy_rules_builder/diagnostic_models.py create mode 100644 aiac/test/agent/policy_rules_builder/test_diagnostic_models.py diff --git a/aiac/src/aiac/agent/policy_rules_builder/diagnostic_models.py b/aiac/src/aiac/agent/policy_rules_builder/diagnostic_models.py new file mode 100644 index 000000000..0d4e523ff --- /dev/null +++ b/aiac/src/aiac/agent/policy_rules_builder/diagnostic_models.py @@ -0,0 +1,150 @@ +"""Structured models for the read-only policy **conflict-check** diagnostic (feature #154). + +This module defines ONLY the stable serialization shape shared by the diagnostic engine, the +survey use-case, and the ``POST /policy/check`` route — no pipeline logic. The live ``/apply`` +path and its models (``PolicyRule``, ``Contradiction``, ``AuditVerdict`` in ``graph.py`` / +``policy.model.models``) are deliberately untouched (decision D11): the typed conflict ``kind`` +lives here, produced by the diagnostic's ``explain`` node, rather than being bolted onto the +shared ``Contradiction`` / ``AuditVerdict`` that the byte-for-byte-stable live path depends on. + +Style mirrors the rest of the agent/policy layer: ``str``-subclass ``Enum``s (so +``FocalType.ROLE == "role"`` holds and they serialize as their string value) and pydantic +``BaseModel`` with ``ConfigDict(extra="ignore")``. +""" + +from enum import Enum + +from pydantic import BaseModel, ConfigDict + + +class FocalType(str, Enum): + """Which axis the focal entity sits on — i.e. which side drove the survey run. A + ``str`` enum, so ``FocalType.ROLE == "role"`` holds and it serializes as ``"role"`` / + ``"scope"``.""" + + ROLE = "role" + SCOPE = "scope" + + +class ConflictKind(str, Enum): + """The two recognized contradiction kinds (D6). ``direct`` = the same (role, scope) pair is + both granted and prohibited; ``coarse_scope`` = a coarse capability is granted while a finer + one it subsumes is prohibited (a granularity mismatch). Classified by the ``explain`` node + (D11), not read from the auditor. A ``str`` enum, so ``ConflictKind.DIRECT == "direct"``.""" + + DIRECT = "direct" + COARSE_SCOPE = "coarse_scope" + + +class ConflictStatus(str, Enum): + """The report's top-level outcome. EXACTLY three values (the precedence that selects among + them is documented on :meth:`ConflictReport.derive_status`). ``no_conflict`` is a *positive* + clean result and is only ever reached when ≥1 entity was evaluated with nothing outstanding — + never a stand-in for an incomplete/failed run.""" + + NO_CONFLICT = "no_conflict" + CONFLICTS_FOUND = "conflicts_found" + INCOMPLETE = "incomplete" + + +class UnevaluatedReason(str, Enum): + """Why a focal entity could not be evaluated. Currently a single value: ``nonconvergence`` + (the entity exhausted the audit retry budget without a verdict). Kept as an enum so callers + switch on a stable token, with the free-text ``detail`` carrying specifics.""" + + NONCONVERGENCE = "nonconvergence" + + +class EntityRef(BaseModel): + """Minimal ``{name, id}`` reference to a resolved IdP entity (role or scope). The ``id`` is + the Keycloak entity id re-joined from the audit output's name string (D12).""" + + model_config = ConfigDict(extra="ignore") + + name: str + id: str + + +class FocalRef(EntityRef): + """A ``{name, id, type}`` reference to the *focal* entity of a run — an :class:`EntityRef` + plus the axis (``type``) it sits on, which tells the reader which side drove the run.""" + + type: FocalType + + +class Conflict(BaseModel): + """One genuine, auditor-confirmed grant/prohibit contradiction for a single (role, scope) + pair. ``focal`` records which side drove the run; ``role`` / ``scope`` are the two colliding + entities regardless of direction. Quotes are verbatim substrings of the candidate policy text + (D5); when substring validation fails, the conflict is kept, ``quotes_verified`` is ``False``, + and the explanation falls back to the auditor description.""" + + model_config = ConfigDict(extra="ignore") + + focal: FocalRef + role: EntityRef + scope: EntityRef + kind: ConflictKind + granting_quotes: list[str] = [] + prohibiting_quotes: list[str] = [] + explanation: str + quotes_verified: bool + + +class Unevaluated(BaseModel): + """A focal entity the survey could not evaluate (e.g. retry-budget exhaustion). Listed so a + partial run is never mistaken for a clean one — its presence forces status away from + ``no_conflict``.""" + + model_config = ConfigDict(extra="ignore") + + focal: FocalRef + reason: UnevaluatedReason = UnevaluatedReason.NONCONVERGENCE + detail: str | None = None + + +class ConflictReport(BaseModel): + """The single object the diagnostic returns: every conflict found across the surveyed + focal entities, every entity that could not be evaluated, and the derived ``status``.""" + + model_config = ConfigDict(extra="ignore") + + conflicts: list[Conflict] = [] + unevaluated: list[Unevaluated] = [] + status: ConflictStatus + + @staticmethod + def derive_status( + conflicts: list[Conflict], + unevaluated: list[Unevaluated], + evaluated_count: int, + ) -> ConflictStatus: + """Apply the authoritative status precedence (spec §"Status enum + precedence"): + + ``conflicts_found`` ⇔ ``conflicts`` non-empty; else ``incomplete`` ⇔ ``evaluated_count + == 0`` OR ``unevaluated`` non-empty; else ``no_conflict``. Both ``incomplete`` disjuncts + are load-bearing: ``evaluated_count == 0`` catches the empty-input / zero-focal case, + ``unevaluated != []`` catches retry-exhaustion on some entities while the rest are clean. + ``no_conflict`` is reached only when ≥1 entity was evaluated AND ``conflicts == []`` AND + ``unevaluated == []`` — so an outage never looks identical to a clean policy.""" + if conflicts: + return ConflictStatus.CONFLICTS_FOUND + if evaluated_count == 0 or unevaluated: + return ConflictStatus.INCOMPLETE + return ConflictStatus.NO_CONFLICT + + @classmethod + def from_survey( + cls, + conflicts: list[Conflict], + unevaluated: list[Unevaluated], + evaluated_count: int, + ) -> "ConflictReport": + """Assemble a report, deriving ``status`` from the survey outcome via + :meth:`derive_status`. The survey use-case (out of scope for the model layer) owns *what* + counts as an evaluated entity; this only encodes the precedence.""" + return cls( + conflicts=conflicts, + unevaluated=unevaluated, + status=cls.derive_status(conflicts, unevaluated, evaluated_count), + ) diff --git a/aiac/test/agent/policy_rules_builder/test_diagnostic_models.py b/aiac/test/agent/policy_rules_builder/test_diagnostic_models.py new file mode 100644 index 000000000..91d4620af --- /dev/null +++ b/aiac/test/agent/policy_rules_builder/test_diagnostic_models.py @@ -0,0 +1,182 @@ +"""Unit tests for the policy conflict-check diagnostic models (#156).""" + +import pytest +from pydantic import ValidationError + +from aiac.agent.policy_rules_builder.diagnostic_models import ( + Conflict, + ConflictKind, + ConflictReport, + ConflictStatus, + EntityRef, + FocalRef, + FocalType, + Unevaluated, + UnevaluatedReason, +) + + +# --- enum values (pinned exactly) ------------------------------------------------------------- + + +def test_focal_type_values(): + assert [m.value for m in FocalType] == ["role", "scope"] + assert FocalType.ROLE == "role" # str-enum identity + + +def test_conflict_kind_values(): + assert [m.value for m in ConflictKind] == ["direct", "coarse_scope"] + assert ConflictKind.COARSE_SCOPE == "coarse_scope" + + +def test_status_values_exactly(): + assert {m.value for m in ConflictStatus} == {"no_conflict", "conflicts_found", "incomplete"} + + +def test_unevaluated_reason_values(): + assert [m.value for m in UnevaluatedReason] == ["nonconvergence"] + + +# --- ref models ------------------------------------------------------------------------------- + + +def test_entity_ref_shape(): + ref = EntityRef(name="reader", id="r-1") + assert ref.model_dump() == {"name": "reader", "id": "r-1"} + + +def test_focal_ref_is_entity_ref_plus_type(): + focal = FocalRef(name="reader", id="r-1", type=FocalType.ROLE) + assert isinstance(focal, EntityRef) + assert focal.model_dump() == {"name": "reader", "id": "r-1", "type": "role"} + + +def test_focal_ref_type_is_constrained(): + with pytest.raises(ValidationError): + FocalRef(name="x", id="y", type="subject") + + +# --- Conflict --------------------------------------------------------------------------------- + + +def _conflict(**over): + base = dict( + focal=FocalRef(name="reader", id="r-1", type=FocalType.ROLE), + role=EntityRef(name="reader", id="r-1"), + scope=EntityRef(name="repo:write", id="s-1"), + kind=ConflictKind.DIRECT, + granting_quotes=["reader may write"], + prohibiting_quotes=["reader must not write"], + explanation="granted and prohibited the same pair", + quotes_verified=True, + ) + base.update(over) + return Conflict(**base) + + +def test_conflict_full_shape_serializes(): + dumped = _conflict().model_dump() + assert dumped == { + "focal": {"name": "reader", "id": "r-1", "type": "role"}, + "role": {"name": "reader", "id": "r-1"}, + "scope": {"name": "repo:write", "id": "s-1"}, + "kind": "direct", + "granting_quotes": ["reader may write"], + "prohibiting_quotes": ["reader must not write"], + "explanation": "granted and prohibited the same pair", + "quotes_verified": True, + } + + +def test_conflict_quotes_default_empty(): + c = Conflict( + focal=FocalRef(name="s", id="s-1", type=FocalType.SCOPE), + role=EntityRef(name="r", id="r-1"), + scope=EntityRef(name="s", id="s-1"), + kind=ConflictKind.COARSE_SCOPE, + explanation="e", + quotes_verified=False, + ) + assert c.granting_quotes == [] + assert c.prohibiting_quotes == [] + + +def test_quotes_are_list_of_str(): + assert _conflict().granting_quotes == ["reader may write"] + assert isinstance(_conflict().prohibiting_quotes, list) + + +# --- Unevaluated ------------------------------------------------------------------------------ + + +def test_unevaluated_defaults_reason_and_optional_detail(): + u = Unevaluated(focal=FocalRef(name="r", id="r-1", type=FocalType.ROLE)) + assert u.reason == UnevaluatedReason.NONCONVERGENCE + assert u.detail is None + assert u.model_dump() == { + "focal": {"name": "r", "id": "r-1", "type": "role"}, + "reason": "nonconvergence", + "detail": None, + } + + +def test_unevaluated_with_detail(): + u = Unevaluated( + focal=FocalRef(name="r", id="r-1", type=FocalType.ROLE), + detail="exhausted 3 retries", + ) + assert u.detail == "exhausted 3 retries" + + +# --- ConflictReport --------------------------------------------------------------------------- + + +def test_report_empty_defaults(): + report = ConflictReport(status=ConflictStatus.NO_CONFLICT) + assert report.conflicts == [] + assert report.unevaluated == [] + dumped = report.model_dump() + assert dumped["status"] == "no_conflict" + assert dumped["conflicts"] == [] + assert dumped["unevaluated"] == [] + + +def test_report_round_trips_via_json(): + report = ConflictReport.from_survey([_conflict()], [], evaluated_count=1) + reloaded = ConflictReport.model_validate_json(report.model_dump_json()) + assert reloaded == report + + +# --- derive_status precedence ----------------------------------------------------------------- + + +def test_derive_status_no_conflict(): + assert ( + ConflictReport.derive_status([], [], evaluated_count=3) == ConflictStatus.NO_CONFLICT + ) + + +def test_derive_status_conflicts_found_takes_precedence_over_unevaluated(): + u = Unevaluated(focal=FocalRef(name="r", id="r-1", type=FocalType.ROLE)) + # conflicts win even when there are also unevaluated entities + assert ( + ConflictReport.derive_status([_conflict()], [u], evaluated_count=1) + == ConflictStatus.CONFLICTS_FOUND + ) + + +def test_derive_status_incomplete_when_zero_evaluated(): + assert ConflictReport.derive_status([], [], evaluated_count=0) == ConflictStatus.INCOMPLETE + + +def test_derive_status_incomplete_when_unevaluated_present(): + u = Unevaluated(focal=FocalRef(name="r", id="r-1", type=FocalType.ROLE)) + assert ( + ConflictReport.derive_status([], [u], evaluated_count=5) == ConflictStatus.INCOMPLETE + ) + + +def test_from_survey_encodes_precedence(): + assert ConflictReport.from_survey([], [], 0).status == ConflictStatus.INCOMPLETE + assert ConflictReport.from_survey([], [], 2).status == ConflictStatus.NO_CONFLICT + assert ConflictReport.from_survey([_conflict()], [], 1).status == ConflictStatus.CONFLICTS_FOUND From 0e7777409fbcf2f78577949edd632b3a65eef67b Mon Sep 17 00:00:00 2001 From: Anatoly Koyfman Date: Wed, 19 Aug 2026 08:42:29 +0000 Subject: [PATCH 4/8] Feat: Add policy-conflict diagnostic engine + explain prompt (#154) Add agent/policy_rules_builder/diagnostic.py: a parallel diagnostic graph (START->seed->propose->precheck->audit_diagnostic->{explain|retry|END}) that reuses the live proposer/precheck/_structured_call unchanged but seeds policy_text from input (no file read), RECORDS genuine contradictions instead of raising, marks non-converging entities unevaluated, and adds a terminal explain node. The explain node classifies kind in {direct, coarse_scope} (D11), extracts verbatim granting/prohibiting quotes validated by a whitespace-normalized substring check, and does the name->id join + run-direction tagging (D12). On quote-validation failure the conflict is kept with quotes_verified=false and the auditor description as fallback. Adds build_explain_messages to prompts.py (append-only). Exposes run_role_diagnostic / run_scope_diagnostic per-entity entry points returning DiagnosticResult(conflicts, unevaluated) for the survey use-case to fan out over. The live graph.py is byte-for-byte unedited; the live /apply -> 422 path is untouched (D14). Refs #157. Part of #154. Assisted-By: Claude (Anthropic AI) Signed-off-by: Anatoly Koyfman --- .../agent/policy_rules_builder/diagnostic.py | 450 ++++++++++++++++++ .../agent/policy_rules_builder/prompts.py | 61 +++ .../policy_rules_builder/test_diagnostic.py | 274 +++++++++++ 3 files changed, 785 insertions(+) create mode 100644 aiac/src/aiac/agent/policy_rules_builder/diagnostic.py create mode 100644 aiac/test/agent/policy_rules_builder/test_diagnostic.py diff --git a/aiac/src/aiac/agent/policy_rules_builder/diagnostic.py b/aiac/src/aiac/agent/policy_rules_builder/diagnostic.py new file mode 100644 index 000000000..f8434a3e4 --- /dev/null +++ b/aiac/src/aiac/agent/policy_rules_builder/diagnostic.py @@ -0,0 +1,450 @@ +"""Read-only policy Conflict-Check diagnostic engine (feature #154, task #157). + +A PARALLEL diagnostic graph that sits NEXT TO the live Policy Rules Builder graph in +``graph.py`` and reuses its proposer / precheck / auditor machinery UNCHANGED, but differs +in three ways (design decisions D8/D11/D14 in +``docs/specs/components/aiac-agent/policy-conflict-check.md``): + + (a) START **seeds ``policy_text`` from the input state** -- it does NOT read + ``AIAC_POLICY_FILE`` via ``get_policy_source()`` the way the live ``_fetch`` does. The + candidate policy prose is supplied directly by the caller. + (b) The audit node **RECORDS** genuine contradictions into state instead of RAISING + ``PolicyContradictionError`` (the live ``_audit`` raises). On retry-budget exhaustion it + marks the focal entity ``unevaluated`` -- again, never raising. + (c) A terminal ``explain`` node (D11) CLASSIFIES the conflict ``kind`` and extracts verbatim, + substring-validated quotes for each recorded (role, scope) contradiction. There is NO + ``build`` node -- the diagnostic emits a ``ConflictReport`` fragment, not ``PolicyRule``s. + +The live ``graph.py`` (``_audit`` raise, ``_fetch`` START, ``build_*`` entry points) is left +BYTE-FOR-BYTE UNCHANGED; this module imports the reusable pieces from it. Every LLM turn -- +propose, audit, explain -- flows through the SAME ``graph._structured_call`` seam, so a single +``side_effect`` patch on ``aiac.agent.policy_rules_builder.graph._structured_call`` drives all +three in order (mirrors ``test_graph.py``). + +This engine runs ONE focal entity end-to-end (one role-focal run or one scope-focal run). The +survey orchestrator (#158) loops it over every focal entity in a resolved ``FocalEntitySet`` and +assembles the top-level ``ConflictReport`` from the accumulated ``conflicts`` + ``unevaluated``. + +Public per-entity entry points (what #158 calls): + + run_role_diagnostic(policy_text, role, scopes, *, focal_entities=None) -> DiagnosticResult + run_scope_diagnostic(policy_text, roles, scope, *, focal_entities=None) -> DiagnosticResult + +each returning ``DiagnosticResult(conflicts=[...], unevaluated=[...])``. Per the resolver's +fan-out (#155): a scope-focal run is ``candidate_roles`` against one ``own_scope``; a role-focal +run (AGENT services only) is one ``own_role`` against ``other_scopes``. +""" + +from typing import Any, NamedTuple, TypedDict + +from langgraph.graph import END, START, StateGraph +from pydantic import BaseModel + +from aiac.agent.shared.focal_entities import FocalEntitySet +from aiac.idp.configuration.models import Role, Scope + +from . import graph as _graph +from .diagnostic_models import ( + Conflict, + ConflictKind, + EntityRef, + FocalRef, + FocalType, + Unevaluated, + UnevaluatedReason, +) +from .graph import ( + MAX_AUDIT_RETRIES, + AuditVerdict, + RoleSelection, + ScopeSelection, + _precheck, + _propose, + _PRBWorking, + _role_cands, + _role_focal, + _scope_cands, + _scope_focal, + _ROLE_CONTRACT, + _ROLE_DIRECTION, + _SCOPE_CONTRACT, + _SCOPE_DIRECTION, +) +from .prompts import build_auditor_messages, build_explain_messages + + +# --------------------------------------------------------------------------- # +# Explain-node output schema (driven through graph._structured_call). # +# --------------------------------------------------------------------------- # +class ExplainResult(BaseModel): + """Structured output of the terminal ``explain`` LLM call for ONE (role, scope) contradiction. + + ``kind`` is CLASSIFIED here (D11), not read from the auditor. ``granting_quotes`` / + ``prohibiting_quotes`` are meant to be verbatim substrings of the candidate policy text; the + engine validates each with :func:`_verify_quote` and, on any failure, keeps the conflict but + sets ``quotes_verified=False`` and falls back to the auditor description for the explanation.""" + + kind: ConflictKind + granting_quotes: list[str] = [] + prohibiting_quotes: list[str] = [] + explanation: str = "" + + +# --------------------------------------------------------------------------- # +# Diagnostic state (extends the shared working state from graph.py). # +# --------------------------------------------------------------------------- # +class DiagnosticState(_PRBWorking): + """The live ``_PRBWorking`` fields (``policy_text``, ``selected_names``, ``denied_names``, + ``conflict_names``, ``exclusive``, ``reasoning``, ``approved``, ``audit_feedback``, + ``retry_count``, ``rules``) plus the diagnostic-only accumulators. + + ``focal_entities`` carries the resolved :class:`FocalEntitySet` for the run so the name->id + join (D12) has the full typed context available; the join itself is performed against this + run's own typed candidate lists (``scopes`` / ``roles``), which are the exact slice of that + set that drove the run. ``recorded_contradictions`` is what the forked audit node writes + instead of raising; ``conflicts`` / ``unevaluated`` are the report fragments this run emits.""" + + focal_entities: FocalEntitySet | None + recorded_contradictions: list[Any] # list[Contradiction] recorded (not raised) by _audit_diagnostic + conflicts: list[Conflict] + unevaluated: list[Unevaluated] + + +class RoleDiagnosticState(DiagnosticState): + role: Role + scopes: list[Scope] + + +class ScopeDiagnosticState(DiagnosticState): + roles: list[Role] + scope: Scope + + +class DiagnosticResult(NamedTuple): + """Per-entity output the survey (#158) collects and concatenates across all focal entities.""" + + conflicts: list[Conflict] + unevaluated: list[Unevaluated] + + +# --------------------------------------------------------------------------- # +# Pure quote validator (D5 + spec "Quote-validator rule"). # +# --------------------------------------------------------------------------- # +def _normalize_ws(text: str) -> str: + """Collapse every whitespace run (spaces, tabs, newlines) to a single space and trim the ends. + ``str.split()`` with no separator splits on ANY run of whitespace, so ``" ".join(s.split())`` + performs exactly the specified normalization.""" + return " ".join(text.split()) + + +def _verify_quote(quote: str, policy_text: str) -> bool: + """True iff ``quote`` is a whitespace-normalized substring of ``policy_text``. + + Whitespace-normalize ONLY (both sides): collapse whitespace runs to a single space and trim. + Deliberately NO case-folding and NO punctuation / smart-quote normalization -- the quote must + be findable *as written* in the author's prose, so a near-quote (wrong case, curly vs straight + quotes, altered punctuation) correctly FAILS and forces ``quotes_verified=False``.""" + return _normalize_ws(quote) in _normalize_ws(policy_text) + + +# --------------------------------------------------------------------------- # +# Forked nodes (record-not-raise audit; terminal explain; input-seeding START).# +# --------------------------------------------------------------------------- # +def _seed_policy_text(state: DiagnosticState) -> dict[str, Any]: + """START node: seed ``policy_text`` from the INPUT state (candidate prose supplied by the + caller). Deliberately does NOT call ``get_policy_source()`` -- this is the one place the live + ``_fetch`` reads ``AIAC_POLICY_FILE`` and overwrites the input, which a pre-commit check on a + *candidate* policy must not do (D14).""" + return {"policy_text": state["policy_text"]} + + +def _audit_diagnostic( + state: DiagnosticState, + *, + focal: str, + candidates: str, + direction: str, + focal_ref: FocalRef, +) -> dict[str, Any]: + """Forked audit node. Runs the SAME auditor call as the live ``_audit`` (same + ``build_auditor_messages`` + ``AuditVerdict`` via ``_structured_call``) but ROUTES DIFFERENTLY: + + - genuine contradictions -> RECORD into state (never raise); route to ``explain``. + - approved -> no conflict for this entity; route to END. + - ordinary rejection -> feed the reason back and re-propose (<= MAX_AUDIT_RETRIES). + - retry budget exhausted -> mark the entity ``unevaluated`` (nonconvergence); route to END. + """ + verdict = _graph._structured_call( + AuditVerdict, + build_auditor_messages( + state["policy_text"], + focal, + candidates, + state["selected_names"], + state["denied_names"], + state["conflict_names"], + direction=direction, + ), + ) + if verdict.contradictions: + # RECORD, do not raise (the live path raises here). Routed to explain by _route_diagnostic. + return {"recorded_contradictions": list(verdict.contradictions), "approved": False} + if verdict.approved: + return {"approved": True} + if state["retry_count"] >= MAX_AUDIT_RETRIES: + # Non-convergence: mark UNEVALUATED (do not raise) so a partial survey is never mistaken + # for a clean one (its presence forces status away from no_conflict in #158). + return { + "approved": False, + "unevaluated": [ + Unevaluated( + focal=focal_ref, + reason=UnevaluatedReason.NONCONVERGENCE, + detail=verdict.reason, + ) + ], + } + # Ordinary rejection: thread the reason back and re-propose on the shared retry budget. + return {"approved": False, "audit_feedback": verdict.reason, "retry_count": state["retry_count"] + 1} + + +def _explain( + state: DiagnosticState, + *, + focal_type: FocalType, + focal_obj: Role | Scope, + candidate_by_name: dict[str, Role | Scope], +) -> dict[str, Any]: + """Terminal node (D11). ONE ``_structured_call`` per recorded (role, scope) contradiction. + + Inputs to each call = candidate ``policy_text`` + the one (role, scope) pair + the auditor's + ``description`` as a HINT ONLY (never the proposer's reasoning). Classifies ``kind``, extracts + verbatim ``granting_quotes`` / ``prohibiting_quotes``, and validates each with + :func:`_verify_quote`. On ANY quote failure (or no quotes at all) the conflict is KEPT with + ``quotes_verified=False`` and the explanation falls back to the auditor ``description``. + + Name->id join + run-direction tagging (D12): the focal side is ``focal_obj`` (tagged with + ``focal_type``); the candidate side is re-joined by name against this run's typed candidate + set. A role-focal run => focal is the role; a scope-focal run => focal is the scope.""" + policy_text = state["policy_text"] + focal_ref = FocalRef(name=focal_obj.name, id=focal_obj.id, type=focal_type) + out: list[Conflict] = [] + for contradiction in state["recorded_contradictions"]: + candidate = candidate_by_name.get(contradiction.candidate_name) + if candidate is None: + # Defensive: precheck already filtered names to the candidate set, so an unjoinable + # name should not occur. Skip rather than emit a conflict with a fabricated id. + continue + if focal_type is FocalType.ROLE: + role_obj, scope_obj = focal_obj, candidate + else: + role_obj, scope_obj = candidate, focal_obj + + result = _graph._structured_call( + ExplainResult, + build_explain_messages( + policy_text, + _role_focal(role_obj), + _scope_focal(scope_obj), + contradiction.description, + ), + ) + granting = list(result.granting_quotes) + prohibiting = list(result.prohibiting_quotes) + # Verified only when there is at least one quote AND every quote is a verbatim substring. + verified = bool(granting or prohibiting) and all( + _verify_quote(q, policy_text) for q in granting + prohibiting + ) + explanation = result.explanation if verified else contradiction.description + out.append( + Conflict( + focal=focal_ref, + role=EntityRef(name=role_obj.name, id=role_obj.id), + scope=EntityRef(name=scope_obj.name, id=scope_obj.id), + kind=result.kind, + granting_quotes=granting, + prohibiting_quotes=prohibiting, + explanation=explanation, + quotes_verified=verified, + ) + ) + return {"conflicts": out} + + +def _route_diagnostic(state: DiagnosticState) -> str: + """Route out of the audit node. Recorded contradictions -> explain; a clean approval or a + non-convergence mark -> END; anything else (ordinary rejection with budget left) -> retry.""" + if state.get("recorded_contradictions"): + return "explain" + if state.get("approved"): + return "end" + if state.get("unevaluated"): + return "end" + return "retry" + + +# --------------------------------------------------------------------------- # +# Assembly (parallel to graph._assemble; no build node; record-not-raise). # +# --------------------------------------------------------------------------- # +def _assemble_diagnostic(state_type: type, seed, propose, precheck, audit_diagnostic, explain): + """Wire START->seed->propose->precheck->audit_diagnostic->{explain | retry->propose | END}. + Mirrors ``graph._assemble`` but swaps ``_fetch`` for the input-seeding START node, drops the + ``build`` node, and replaces the two-way approved/rejected route with the three-way + explain/retry/end route of the record-not-raise audit node.""" + g = StateGraph(state_type) + g.add_node("seed", seed) + g.add_node("propose", propose) + g.add_node("precheck", precheck) + g.add_node("audit", audit_diagnostic) + g.add_node("explain", explain) + g.add_edge(START, "seed") + g.add_edge("seed", "propose") + g.add_edge("propose", "precheck") + g.add_edge("precheck", "audit") + g.add_conditional_edges("audit", _route_diagnostic, {"explain": "explain", "retry": "propose", "end": END}) + g.add_edge("explain", END) + return g.compile() + + +def build_role_diagnostic_graph(): + """Role-focal diagnostic run: the focal entity is a ROLE; the candidates are SCOPES.""" + + def seed(s: RoleDiagnosticState) -> dict[str, Any]: + return _seed_policy_text(s) + + def propose(s: RoleDiagnosticState) -> dict[str, Any]: + return _propose( + s, + focal=_role_focal(s["role"]), + candidates=_scope_cands(s["scopes"]), + contract=_ROLE_CONTRACT, + direction=_ROLE_DIRECTION, + schema=RoleSelection, + names_field="granted_scope_names", + denied_names_field="denied_scope_names", + exclusive_field="grant_is_exclusive", + ) + + def precheck(s: RoleDiagnosticState) -> dict[str, Any]: + return _precheck(s, candidate_names={sc.name for sc in s["scopes"]}) + + def audit(s: RoleDiagnosticState) -> dict[str, Any]: + return _audit_diagnostic( + s, + focal=_role_focal(s["role"]), + candidates=_scope_cands(s["scopes"]), + direction=_ROLE_DIRECTION, + focal_ref=FocalRef(name=s["role"].name, id=s["role"].id, type=FocalType.ROLE), + ) + + def explain(s: RoleDiagnosticState) -> dict[str, Any]: + return _explain( + s, + focal_type=FocalType.ROLE, + focal_obj=s["role"], + candidate_by_name={sc.name: sc for sc in s["scopes"]}, + ) + + return _assemble_diagnostic(RoleDiagnosticState, seed, propose, precheck, audit, explain) + + +def build_scope_diagnostic_graph(): + """Scope-focal diagnostic run: the focal entity is a SCOPE; the candidates are ROLES.""" + + def seed(s: ScopeDiagnosticState) -> dict[str, Any]: + return _seed_policy_text(s) + + def propose(s: ScopeDiagnosticState) -> dict[str, Any]: + return _propose( + s, + focal=_scope_focal(s["scope"]), + candidates=_role_cands(s["roles"]), + contract=_SCOPE_CONTRACT, + direction=_SCOPE_DIRECTION, + schema=ScopeSelection, + names_field="roles_with_access_names", + denied_names_field="roles_denied_access_names", + exclusive_field="access_is_exclusive", + ) + + def precheck(s: ScopeDiagnosticState) -> dict[str, Any]: + return _precheck(s, candidate_names={r.name for r in s["roles"]}) + + def audit(s: ScopeDiagnosticState) -> dict[str, Any]: + return _audit_diagnostic( + s, + focal=_scope_focal(s["scope"]), + candidates=_role_cands(s["roles"]), + direction=_SCOPE_DIRECTION, + focal_ref=FocalRef(name=s["scope"].name, id=s["scope"].id, type=FocalType.SCOPE), + ) + + def explain(s: ScopeDiagnosticState) -> dict[str, Any]: + return _explain( + s, + focal_type=FocalType.SCOPE, + focal_obj=s["scope"], + candidate_by_name={r.name: r for r in s["roles"]}, + ) + + return _assemble_diagnostic(ScopeDiagnosticState, seed, propose, precheck, audit, explain) + + +# Module-level compile is safe (never builds the LLM), mirroring ROLE_GRAPH / SCOPE_GRAPH. +ROLE_DIAGNOSTIC_GRAPH = build_role_diagnostic_graph() +SCOPE_DIAGNOSTIC_GRAPH = build_scope_diagnostic_graph() + + +def _base_state(policy_text: str, focal_entities: FocalEntitySet | None) -> dict[str, Any]: + return { + "policy_text": policy_text, + "selected_names": [], + "denied_names": [], + "conflict_names": [], + "exclusive": False, + "reasoning": "", + "approved": False, + "audit_feedback": None, + "retry_count": 0, + "rules": [], + "focal_entities": focal_entities, + "recorded_contradictions": [], + "conflicts": [], + "unevaluated": [], + } + + +def run_role_diagnostic( + policy_text: str, + role: Role, + scopes: list[Scope], + *, + focal_entities: FocalEntitySet | None = None, +) -> DiagnosticResult: + """Run ONE role-focal diagnostic entity end-to-end and return its conflicts + unevaluated. + + ``role`` is the focal entity (one ``own_role``); ``scopes`` are the candidate scopes it is + checked against (``other_scopes`` in the resolver fan-out). ``policy_text`` is the candidate + prose (seeded directly -- no file read). Never raises on a genuine conflict or non-convergence; + both are returned in the :class:`DiagnosticResult` for #158 to accumulate.""" + state: RoleDiagnosticState = {**_base_state(policy_text, focal_entities), "role": role, "scopes": scopes} # type: ignore[assignment] + out = ROLE_DIAGNOSTIC_GRAPH.invoke(state) + return DiagnosticResult(conflicts=out["conflicts"], unevaluated=out["unevaluated"]) + + +def run_scope_diagnostic( + policy_text: str, + roles: list[Role], + scope: Scope, + *, + focal_entities: FocalEntitySet | None = None, +) -> DiagnosticResult: + """Run ONE scope-focal diagnostic entity end-to-end and return its conflicts + unevaluated. + + ``scope`` is the focal entity (one ``own_scope``); ``roles`` are the candidate roles it is + checked against (``candidate_roles`` in the resolver fan-out). ``policy_text`` is the candidate + prose (seeded directly -- no file read). Never raises on a genuine conflict or non-convergence; + both are returned in the :class:`DiagnosticResult` for #158 to accumulate.""" + state: ScopeDiagnosticState = {**_base_state(policy_text, focal_entities), "roles": roles, "scope": scope} # type: ignore[assignment] + out = SCOPE_DIAGNOSTIC_GRAPH.invoke(state) + return DiagnosticResult(conflicts=out["conflicts"], unevaluated=out["unevaluated"]) diff --git a/aiac/src/aiac/agent/policy_rules_builder/prompts.py b/aiac/src/aiac/agent/policy_rules_builder/prompts.py index 5de55f218..2a3d7bab4 100644 --- a/aiac/src/aiac/agent/policy_rules_builder/prompts.py +++ b/aiac/src/aiac/agent/policy_rules_builder/prompts.py @@ -189,3 +189,64 @@ def build_auditor_messages( "mistake, leave `contradictions` empty and reject with a reason so it can re-propose." ) return [SystemMessage(content=_AUDITOR_SYSTEM), HumanMessage(content=body)] + + +# --------------------------------------------------------------------------- # +# Explain prompt — read-only policy Conflict-Check diagnostic (feature #154). # +# # +# Used ONLY by the diagnostic assembly's terminal `explain` node, once per # +# auditor-confirmed (role, scope) contradiction. It does NOT re-adjudicate the # +# conflict; the auditor already ruled it genuine. It (1) CLASSIFIES the kind # +# (direct vs coarse_scope -- D11: the typed kind is produced here, not read # +# from the auditor), (2) extracts VERBATIM granting/prohibiting quotes from the # +# candidate policy text, and (3) explains the collision in plain language. # +# # +# The candidate policy_text is shown RAW (not wrapped in _policy_block): the # +# quotes are validated as substrings of exactly this text, and the baseline # +# layer is grants-only so it can never be a source of a prohibiting quote. # +# The auditor's own `description` is passed as a HINT ONLY (D9/D11) -- never # +# the proposer's free-form reasoning, which could anchor extraction onto a # +# hallucinated justification. # +# --------------------------------------------------------------------------- # +_EXPLAIN_SYSTEM = ( + "You explain a single, ALREADY-CONFIRMED access-policy contradiction for one (role, scope) pair. " + "An auditor has already ruled that the policy genuinely BOTH grants and prohibits this pair -- do " + "NOT re-litigate whether the conflict exists. Your job has three parts.\n" + "1) CLASSIFY the kind, choosing EXACTLY one:\n" + " - \"direct\": the policy grants and prohibits the SAME capability for this pair -- a head-on " + "grant-vs-prohibit collision on the same scope.\n" + " - \"coarse_scope\": a coarse/broad capability is granted while a finer operation it INCLUDES is " + "prohibited (or vice versa) -- a granularity mismatch (e.g. management granted, writing forbidden).\n" + "2) QUOTE the colliding statements VERBATIM. granting_quotes and prohibiting_quotes are each a list " + "of one or more EXACT substrings copied character-for-character from the POLICY text below: the " + "statement(s) that GRANT access go in granting_quotes, the statement(s) that PROHIBIT or RESTRICT " + "it go in prohibiting_quotes. Copy the author's words exactly -- do NOT paraphrase, fix spelling or " + "punctuation, change quotation marks, or add ellipses. Quote ONLY from the POLICY text, never from " + "the auditor hint.\n" + "3) EXPLAIN the collision in one or two plain sentences.\n" + "The AUDITOR HINT describes the collision to help you locate and classify it; treat it strictly as " + "a hint -- it is NOT policy text and must never be quoted." +) + + +def build_explain_messages( + policy_text: str, + role: str, + scope: str, + description_hint: str, +) -> list[BaseMessage]: + """Messages for the diagnostic `explain` node: classify a confirmed (role, scope) contradiction + and extract verbatim granting/prohibiting quotes from ``policy_text``. + + ``policy_text`` is the RAW candidate policy (the same text the quote validator checks substrings + against -- deliberately NOT wrapped in the baseline ``_policy_block``). ``role`` / ``scope`` are + human-readable descriptions of the colliding pair. ``description_hint`` is the auditor's own + ``Contradiction.description`` (a hint for classification/location only -- never a quote source).""" + body = ( + f"POLICY:\n{policy_text}\n\n" + f"ROLE:\n{role}\n\nSCOPE:\n{scope}\n\n" + f"AUDITOR HINT (not policy text -- do not quote):\n{description_hint}\n\n" + "Classify the conflict kind, extract verbatim granting_quotes and prohibiting_quotes from the " + "POLICY text above, and explain the collision." + ) + return [SystemMessage(content=_EXPLAIN_SYSTEM), HumanMessage(content=body)] diff --git a/aiac/test/agent/policy_rules_builder/test_diagnostic.py b/aiac/test/agent/policy_rules_builder/test_diagnostic.py new file mode 100644 index 000000000..f2df1739b --- /dev/null +++ b/aiac/test/agent/policy_rules_builder/test_diagnostic.py @@ -0,0 +1,274 @@ +"""Deterministic unit tests for the read-only Conflict-Check diagnostic engine (#157). + +Every LLM turn -- proposer, auditor, and the terminal explain call -- flows through the SAME +``graph._structured_call`` seam the live path uses, so ONE ``side_effect`` list drives all three +in order (mirrors ``test_graph.py``). No live endpoint is touched and, unlike the live graph, the +diagnostic seeds ``policy_text`` from input, so ``get_policy_source`` is never patched here. + +Coverage: single-entity end-to-end RECORDING (not raising) a genuine contradiction -> classified +Conflict with verbatim quotes (role + scope directions); retry-budget exhaustion -> unevaluated +(no raise); explain kind classification; substring-validation failure -> quotes_verified=False + +description fallback; and ``_verify_quote`` as a pure function. +""" + +from contextlib import ExitStack +from unittest.mock import patch + +from aiac.agent.policy_rules_builder.diagnostic import ( + ExplainResult, + _verify_quote, + run_role_diagnostic, + run_scope_diagnostic, +) +from aiac.agent.policy_rules_builder.diagnostic_models import ConflictKind, FocalType +from aiac.agent.policy_rules_builder.graph import AuditVerdict, Contradiction, RoleSelection, ScopeSelection +from aiac.idp.configuration.models import Role, Scope + +_SEAM = "aiac.agent.policy_rules_builder.graph._structured_call" + + +def _role(id="r-dev", name="developer") -> Role: + return Role(id=id, name=name, composite=False, childRoles=[]) + + +def _scope(id="s-iss", name="issues") -> Scope: + return Scope(id=id, name=name) + + +def _patch_calls(stack: ExitStack, side_effect): + return stack.enter_context(patch(_SEAM, side_effect=side_effect)) + + +# --------------------------------------------------------------------------- # +# 1 — role-focal run: a genuine contradiction is RECORDED (not raised) and # +# turned into a classified Conflict with verbatim, validated quotes. # +# --------------------------------------------------------------------------- # +def test_role_run_records_genuine_conflict_with_verbatim_quotes(): + role = _role("r-dev", "developer") + issues = _scope("s-iss", "issues") + policy = "Developers may read issues.\nDevelopers must not modify issues." + + with ExitStack() as stack: + _patch_calls( + stack, + [ + RoleSelection( + granted_scope_names=["issues"], + denied_scope_names=["issues"], + reasoning="may read but must not modify", + ), + AuditVerdict( + approved=False, + contradictions=[ + Contradiction(candidate_name="issues", description="coarse-scope granularity mismatch") + ], + ), + ExplainResult( + kind=ConflictKind.COARSE_SCOPE, + granting_quotes=["Developers may read issues."], + prohibiting_quotes=["Developers must not modify issues."], + explanation="issues covers both read and write; reading is granted but writing is forbidden", + ), + ], + ) + result = run_role_diagnostic(policy, role, [issues]) + + assert result.unevaluated == [] + assert len(result.conflicts) == 1 + c = result.conflicts[0] + assert c.kind is ConflictKind.COARSE_SCOPE + assert c.quotes_verified is True + assert c.focal.type is FocalType.ROLE + assert (c.focal.name, c.focal.id) == ("developer", "r-dev") + assert (c.role.name, c.role.id) == ("developer", "r-dev") + assert (c.scope.name, c.scope.id) == ("issues", "s-iss") + assert c.granting_quotes == ["Developers may read issues."] + assert c.prohibiting_quotes == ["Developers must not modify issues."] + # Every reported quote is a verbatim substring of the candidate policy text. + for q in c.granting_quotes + c.prohibiting_quotes: + assert _verify_quote(q, policy) + + +# --------------------------------------------------------------------------- # +# 2 — scope-focal run: the direction flips (focal is the SCOPE, candidate is # +# the ROLE), and a DIRECT conflict is classified as such. # +# --------------------------------------------------------------------------- # +def test_scope_run_records_conflict_direction_flipped_and_direct_kind(): + scope = _scope("s-audit", "audit-log") + intern = _role("r-int", "intern") + policy = "Interns may access the audit-log. Interns may not access the audit-log." + + with ExitStack() as stack: + _patch_calls( + stack, + [ + ScopeSelection( + roles_with_access_names=["intern"], + roles_denied_access_names=["intern"], + reasoning="granted and forbidden for interns", + ), + AuditVerdict( + approved=False, + contradictions=[Contradiction(candidate_name="intern", description="direct conflict")], + ), + ExplainResult( + kind=ConflictKind.DIRECT, + granting_quotes=["Interns may access the audit-log."], + prohibiting_quotes=["Interns may not access the audit-log."], + explanation="the same access is both granted and prohibited", + ), + ], + ) + result = run_scope_diagnostic(policy, [intern], scope) + + assert result.unevaluated == [] + assert len(result.conflicts) == 1 + c = result.conflicts[0] + assert c.kind is ConflictKind.DIRECT + assert c.quotes_verified is True + assert c.focal.type is FocalType.SCOPE + assert (c.focal.name, c.focal.id) == ("audit-log", "s-audit") + assert (c.scope.name, c.scope.id) == ("audit-log", "s-audit") + assert (c.role.name, c.role.id) == ("intern", "r-int") + + +# --------------------------------------------------------------------------- # +# 3 — a clean approval records NO conflict and NO unevaluated (routes to END # +# without visiting explain). # +# --------------------------------------------------------------------------- # +def test_clean_approval_records_nothing(): + role = _role("r-dev", "developer") + issues = _scope("s-iss", "issues") + + with ExitStack() as stack: + _patch_calls( + stack, + [ + RoleSelection(granted_scope_names=["issues"], reasoning="granted"), + AuditVerdict(approved=True), + ], + ) + result = run_role_diagnostic("Developers may use issues.", role, [issues]) + + assert result.conflicts == [] + assert result.unevaluated == [] + + +# --------------------------------------------------------------------------- # +# 4 — retry-budget exhaustion marks the entity UNEVALUATED (nonconvergence) # +# and does NOT raise; no conflicts are produced. # +# --------------------------------------------------------------------------- # +def test_retry_budget_exhaustion_marks_unevaluated_no_raise(): + role = _role("r-dev", "developer") + issues = _scope("s-iss", "issues") + + def se(schema, messages): + if schema is AuditVerdict: + return AuditVerdict(approved=False, reason="still not right") + return RoleSelection(granted_scope_names=["issues"], reasoning="r") + + with ExitStack() as stack: + _patch_calls(stack, se) + result = run_role_diagnostic("Some policy.", role, [issues]) + + assert result.conflicts == [] + assert len(result.unevaluated) == 1 + u = result.unevaluated[0] + assert u.reason.value == "nonconvergence" + assert u.focal.type is FocalType.ROLE + assert (u.focal.name, u.focal.id) == ("developer", "r-dev") + assert u.detail == "still not right" + + +# --------------------------------------------------------------------------- # +# 5 — substring-validation FAILURE: the explain call returns a non-substring # +# granting quote, so the conflict is KEPT with quotes_verified=False and # +# the explanation falls back to the auditor description. # +# --------------------------------------------------------------------------- # +def test_quote_validation_failure_sets_unverified_and_falls_back_to_description(): + role = _role("r-dev", "developer") + issues = _scope("s-iss", "issues") + policy = "Developers may read issues.\nDevelopers must not modify issues." + audit_desc = "coarse-scope granularity mismatch on issues" + + with ExitStack() as stack: + _patch_calls( + stack, + [ + RoleSelection( + granted_scope_names=["issues"], + denied_scope_names=["issues"], + reasoning="r", + ), + AuditVerdict( + approved=False, + contradictions=[Contradiction(candidate_name="issues", description=audit_desc)], + ), + ExplainResult( + kind=ConflictKind.COARSE_SCOPE, + # Not a verbatim substring of the policy (paraphrased) -> validation fails. + granting_quotes=["Developers are allowed to read every issue"], + prohibiting_quotes=["Developers must not modify issues."], + explanation="a model paraphrase that must NOT survive fallback", + ), + ], + ) + result = run_role_diagnostic(policy, role, [issues]) + + assert len(result.conflicts) == 1 + c = result.conflicts[0] + assert c.quotes_verified is False + assert c.explanation == audit_desc # fell back to the auditor description, not the model prose + assert c.kind is ConflictKind.COARSE_SCOPE # classification is still kept + + +# --------------------------------------------------------------------------- # +# 6 — empty quotes are treated as an unverified citation (kept, fallback). # +# --------------------------------------------------------------------------- # +def test_empty_quotes_are_unverified_and_fall_back(): + role = _role("r-dev", "developer") + issues = _scope("s-iss", "issues") + audit_desc = "direct conflict on issues" + + with ExitStack() as stack: + _patch_calls( + stack, + [ + RoleSelection(granted_scope_names=["issues"], denied_scope_names=["issues"], reasoning="r"), + AuditVerdict( + approved=False, + contradictions=[Contradiction(candidate_name="issues", description=audit_desc)], + ), + ExplainResult(kind=ConflictKind.DIRECT, granting_quotes=[], prohibiting_quotes=[], explanation="x"), + ], + ) + result = run_role_diagnostic("Developers policy about issues.", role, [issues]) + + c = result.conflicts[0] + assert c.quotes_verified is False + assert c.explanation == audit_desc + + +# --------------------------------------------------------------------------- # +# 7 — _verify_quote as a pure function: substring match, whitespace # +# normalization (incl. newlines), non-match, and NO case-folding. # +# --------------------------------------------------------------------------- # +def test_verify_quote_exact_substring(): + assert _verify_quote("must not modify", "Developers must not modify issues.") + + +def test_verify_quote_normalizes_whitespace_runs_and_newlines(): + policy = "Developers may\tread\nissues." + # Multiple spaces, a tab, and a newline in the source all collapse to single spaces. + assert _verify_quote("Developers may read issues.", policy) + # A multiline quote also normalizes to match single-spaced source prose. + assert _verify_quote("read\n issues.", "Developers may read issues.") + + +def test_verify_quote_rejects_non_substring(): + assert not _verify_quote("Developers may deploy to prod", "Developers may read issues.") + + +def test_verify_quote_is_case_sensitive(): + # No case-folding: a wrong-case near-quote must FAIL (it is not findable as written). + assert not _verify_quote("developers may read issues.", "Developers may read issues.") From ab6bb85de22483220f02ff92285af335bbcff59c Mon Sep 17 00:00:00 2001 From: Anatoly Koyfman Date: Wed, 19 Aug 2026 08:51:23 +0000 Subject: [PATCH 5/8] Feat: Add policy-conflict survey use-case (#154) Add agent/uc/policy_check/check.py::check_policy_conflicts(policy_text, service_id) -> ConflictReport: a sequential, read-only survey that runs every focal entity of the target service through the diagnostic graph to completion (the first conflict never aborts), accumulates all conflicts + unevaluated entries, and derives status via ConflictReport.from_survey with the zero-evaluated guard. service_type is derived read-only from the IdP catalog (focus.type of the service resolved by id) -- Provision is never run, so the survey never mutates IdP state. The resolver's HTTPException(502/404) pre-survey boundary is preserved on both the type lookup and resolve_focal_entities. Fan-out mirrors builder.py exactly (scope-focal over own_scopes; role-focal over flatten_role(own_role) x other_scopes for AGENT). A found conflict returns a report, never raises. Refs #158. Part of #154. Assisted-By: Claude (Anthropic AI) Signed-off-by: Anatoly Koyfman --- .../aiac/agent/uc/policy_check/__init__.py | 1 + aiac/src/aiac/agent/uc/policy_check/check.py | 105 +++++++ aiac/test/agent/uc/policy_check/__init__.py | 0 aiac/test/agent/uc/policy_check/test_check.py | 288 ++++++++++++++++++ 4 files changed, 394 insertions(+) create mode 100644 aiac/src/aiac/agent/uc/policy_check/__init__.py create mode 100644 aiac/src/aiac/agent/uc/policy_check/check.py create mode 100644 aiac/test/agent/uc/policy_check/__init__.py create mode 100644 aiac/test/agent/uc/policy_check/test_check.py diff --git a/aiac/src/aiac/agent/uc/policy_check/__init__.py b/aiac/src/aiac/agent/uc/policy_check/__init__.py new file mode 100644 index 000000000..c3cad07c1 --- /dev/null +++ b/aiac/src/aiac/agent/uc/policy_check/__init__.py @@ -0,0 +1 @@ +"""Read-only Policy Conflict Check survey use-case (feature #154, task #158).""" diff --git a/aiac/src/aiac/agent/uc/policy_check/check.py b/aiac/src/aiac/agent/uc/policy_check/check.py new file mode 100644 index 000000000..694cf0ea5 --- /dev/null +++ b/aiac/src/aiac/agent/uc/policy_check/check.py @@ -0,0 +1,105 @@ +"""Policy Conflict Check survey use-case (feature #154, task #158). + +A **sequential, read-only** survey that runs EVERY focal entity of a target service through the +Conflict-Check diagnostic graph (#157) to completion, accumulates every run's ``conflicts`` + +``unevaluated``, and returns a single :class:`ConflictReport`. Unlike the live ``/apply`` path, +the first genuine conflict does NOT abort the survey — all entities are always run, so the report +lists ALL of that service's conflicts at once. + +This is the diagnostic counterpart of ``ServicePolicyBuilder.build()`` (the live fan-out loop in +``uc/onboarding/policy_builder/builder.py``): it resolves the SAME typed entity set via the shared +``resolve_focal_entities`` (#155) and mirrors that loop's fan-out EXACTLY (scope-focal over every +own scope; role-focal over every flattened own role, AGENT services only). It differs in three +deliberate ways: + + * it is READ-ONLY — it never runs Provision (which MUTATES the IdP) and never calls the PCE / + orchestrator. The target service is a PRE-EXISTING catalog entry, so its ``service_type`` is + read straight from the catalog (``focus.type``) rather than being (re)discovered by Provision. + * it drives ``run_scope_diagnostic`` / ``run_role_diagnostic`` (the record-not-raise diagnostic + graph), not ``build_scope_rules`` / ``build_role_rules`` (the live raise-on-conflict builder). + * a found conflict is a SUCCESSFUL diagnosis, not an error — it is recorded, never raised. Only + the pre-survey resolver boundary (``HTTPException(502)`` IdP-unreachable / + ``HTTPException(404)`` unknown-service) propagates. + +``service_type`` note (the one real design decision — done READ-ONLY): the resolver needs a +``service_type``, but we must NOT run Provision to (re)discover it. For a pre-existing catalog +entry ``focus.type`` IS the authoritative classification, so we resolve the focus service by +``id`` from the same ``get_services()`` catalog the resolver reads and take ``focus.type``. The +builder's "never conflate service_type with focus.type" caution applies ONLY to the live +onboarding path (where a NEW service's type is being discovered), not to this read-only diagnostic +over an existing service. The same ``Configuration`` seam (``focal_entities._config`` / +``Configuration.for_default_realm``) is reused for BOTH the type lookup and the resolver, so the +``HTTPException(502/404)`` pre-survey boundary is preserved and tests patch a single seam. +""" + +from fastapi import HTTPException + +from aiac.agent.policy_rules_builder.diagnostic import run_role_diagnostic, run_scope_diagnostic +from aiac.agent.policy_rules_builder.diagnostic_models import ConflictReport +from aiac.agent.shared import focal_entities as _focal_entities +from aiac.agent.shared.focal_entities import resolve_focal_entities +from aiac.agent.shared.roles import flatten_role +from aiac.idp.configuration.models import ServiceType + + +def check_policy_conflicts(policy_text: str, service_id: str) -> ConflictReport: + """Survey a target service's focal entities for grant/prohibit contradictions in ``policy_text``. + + ``service_id`` is the Keycloak internal client UUID (``Service.id``), matching the + ``/apply/service/{service_id}`` route. Returns one :class:`ConflictReport` with every conflict + found across ALL of the service's focal entities, every entity that could not be evaluated, and + the derived ``status``. Never raises on a found conflict or a non-converging entity; only the + resolver's pre-survey ``HTTPException(502/404)`` propagates. + """ + # Reuse the resolver's own Configuration seam so a single patch point drives both the + # read-only service_type lookup and resolve_focal_entities (and the 502/404 boundary). + config = _focal_entities._config() + + # Read-only service_type derivation: the focus service pre-exists in the catalog, so its + # own catalog type is authoritative. Wrap the lookup in the SAME 502/404 boundary the + # resolver uses (it re-reads the catalog itself for the entity split). + try: + services = config.get_services() + except Exception as e: + raise HTTPException( + 502, f"IdP Configuration Service unavailable for service {service_id!r}: {e}" + ) + focus = next((s for s in services if s.id == service_id), None) + if focus is None: + raise HTTPException(404, f"service {service_id!r} not found in IdP catalog") + service_type = focus.type + + focal = resolve_focal_entities(service_id, service_type, config=config) + + all_conflicts = [] + all_unevaluated = [] + evaluated_count = 0 + + def _accumulate(result) -> None: + # An entity is "evaluated" iff its run did NOT land in unevaluated (a run yields either a + # verdict — clean or recorded conflicts — or a nonconvergence mark, never both). This is + # exactly the count ConflictReport.from_survey's precedence expects for the zero-evaluated + # guard: >=1 evaluated + no conflicts + nothing unevaluated => no_conflict. + nonlocal evaluated_count + all_conflicts.extend(result.conflicts) + all_unevaluated.extend(result.unevaluated) + if not result.unevaluated: + evaluated_count += 1 + + # Fan-out mirrors builder.py EXACTLY. First conflict never aborts — every entity runs. + for scope in focal.own_scopes: + _accumulate( + run_scope_diagnostic( + policy_text, focal.candidate_roles, scope, focal_entities=focal + ) + ) + if service_type is ServiceType.AGENT: + for own_role in focal.own_roles: + for role in flatten_role(own_role): + _accumulate( + run_role_diagnostic( + policy_text, role, focal.other_scopes, focal_entities=focal + ) + ) + + return ConflictReport.from_survey(all_conflicts, all_unevaluated, evaluated_count) diff --git a/aiac/test/agent/uc/policy_check/__init__.py b/aiac/test/agent/uc/policy_check/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/aiac/test/agent/uc/policy_check/test_check.py b/aiac/test/agent/uc/policy_check/test_check.py new file mode 100644 index 000000000..34c393540 --- /dev/null +++ b/aiac/test/agent/uc/policy_check/test_check.py @@ -0,0 +1,288 @@ +"""Deterministic unit tests for the Policy Conflict Check survey use-case (#158). + +The survey drives the #157 diagnostic graph over EVERY focal entity of a target service. Two +seams are patched, exactly as the existing suites do: + + * the IdP catalog seam ``focal_entities._config`` (a ``MagicMock`` backs ``get_services`` / + ``get_subjects``) — the SAME seam ``test_focal_entities.py`` / ``test_builder.py`` use, reused + for both the read-only ``service_type`` lookup and ``resolve_focal_entities``; + * the LLM seam ``graph._structured_call`` — every proposer / auditor / explain turn of every + entity flows through it, so ONE ``side_effect`` (dispatched on the requested schema) drives all + of them (mirrors ``test_diagnostic.py`` / ``test_graph.py``). No live LLM, no cluster. + +Coverage: clean policy over >=1 evaluated entity -> ``no_conflict``; ALL conflicts across ALL +focal entities in one report with the first conflict NOT aborting; a non-converging entity -> +``unevaluated`` with status != ``no_conflict``; the ``unevaluated`` disjunct load-bearing even with +an evaluated entity; zero focal entities -> ``incomplete`` (never ``no_conflict``, no LLM call); the +AGENT role-focal fan-out; and the resolver's ``HTTPException(502/404)`` pre-survey boundary. +""" + +from unittest.mock import MagicMock, patch + +import pytest +from fastapi import HTTPException + +from aiac.agent.policy_rules_builder.diagnostic import ExplainResult +from aiac.agent.policy_rules_builder.diagnostic_models import ConflictKind, ConflictStatus +from aiac.agent.policy_rules_builder.graph import ( + AuditVerdict, + Contradiction, + RoleSelection, + ScopeSelection, +) +from aiac.agent.shared import focal_entities +from aiac.agent.uc.policy_check.check import check_policy_conflicts +from aiac.idp.configuration.models import RoleKind, Scope, Service, ServiceType, Subject +from aiac.idp.configuration.models import Role as RoleModel + +_SEAM = "aiac.agent.policy_rules_builder.graph._structured_call" + +FOCUS_ID = "svc-focus" +OTHER_ID = "svc-other" + + +# --------------------------------------------------------------------------- # +# Fixture builders (mirror test_builder.py / test_focal_entities.py). # +# --------------------------------------------------------------------------- # +def _role(name, *, role_id=None, composite=False, children=None, kind=RoleKind.USER, aiac_managed=True): + return RoleModel( + id=role_id or f"{name}-id", + name=name, + description=name, + composite=composite, + childRoles=children or [], + attributes={"aiac.managed": ["true"]} if aiac_managed else {}, + kind=kind, + ) + + +def _scope(name, *, scope_id=None, service_id="", aiac_managed=True): + return Scope( + id=scope_id or f"{name}-id", + name=name, + description=name, + attributes={"aiac.managed": "true"} if aiac_managed else {}, + serviceId=service_id, + ) + + +def _service(service_id, *, ref=None, roles=None, scopes=None, service_type=ServiceType.TOOL): + return Service( + id=service_id, + serviceId=ref or service_id, + enabled=True, + type=service_type, + roles=roles or [], + scopes=scopes or [], + ) + + +def _subject(username, *, roles=None): + return Subject(id=f"{username}-id", username=username, enabled=True, roles=roles or []) + + +def _run(policy, *, services, subjects=None, side_effect, service_id=FOCUS_ID): + """Run ``check_policy_conflicts`` with both the catalog seam and the LLM seam patched.""" + conf = MagicMock() + conf.get_services.return_value = services + conf.get_subjects.return_value = subjects or [] + with ( + patch.object(focal_entities, "_config", return_value=conf), + patch(_SEAM, side_effect=side_effect), + ): + return check_policy_conflicts(policy, service_id) + + +# --------------------------------------------------------------------------- # +# 1 — clean policy over >=1 evaluated entity => no_conflict. # +# --------------------------------------------------------------------------- # +def test_clean_policy_over_one_entity_is_no_conflict(): + focus = _service(FOCUS_ID, scopes=[_scope("reports", service_id=FOCUS_ID)]) + other = _service(OTHER_ID, roles=[_role("intern", kind=RoleKind.AGENT)]) + + def se(schema, messages): + if schema is ScopeSelection: + return ScopeSelection(roles_with_access_names=["intern"], reasoning="ok") + if schema is AuditVerdict: + return AuditVerdict(approved=True) + raise AssertionError(f"unexpected schema {schema}") + + report = _run("Interns may access reports.", services=[focus, other], side_effect=se) + + assert report.status is ConflictStatus.NO_CONFLICT + assert report.conflicts == [] + assert report.unevaluated == [] + + +# --------------------------------------------------------------------------- # +# 2 — ALL conflicts across ALL focal entities in ONE report; the first # +# conflict does NOT abort (two own scopes, each surfacing a conflict). # +# --------------------------------------------------------------------------- # +def test_all_conflicts_reported_first_conflict_does_not_abort(): + policy = "Interns may access reports. Interns may not access reports." + focus = _service( + FOCUS_ID, + scopes=[_scope("scope-a", service_id=FOCUS_ID), _scope("scope-b", service_id=FOCUS_ID)], + ) + other = _service(OTHER_ID, roles=[_role("intern", kind=RoleKind.AGENT)]) + + def se(schema, messages): + if schema is ScopeSelection: + return ScopeSelection( + roles_with_access_names=["intern"], + roles_denied_access_names=["intern"], + reasoning="both granted and denied", + ) + if schema is AuditVerdict: + return AuditVerdict( + approved=False, + contradictions=[Contradiction(candidate_name="intern", description="direct conflict")], + ) + if schema is ExplainResult: + return ExplainResult( + kind=ConflictKind.DIRECT, + granting_quotes=["Interns may access reports."], + prohibiting_quotes=["Interns may not access reports."], + explanation="the same access is both granted and prohibited", + ) + raise AssertionError(f"unexpected schema {schema}") + + report = _run(policy, services=[focus, other], side_effect=se) + + assert report.status is ConflictStatus.CONFLICTS_FOUND + # BOTH own-scope entities ran and each surfaced a conflict — the first did not abort the survey. + assert len(report.conflicts) == 2 + assert {c.focal.name for c in report.conflicts} == {"scope-a", "scope-b"} + for c in report.conflicts: + assert c.kind is ConflictKind.DIRECT + assert c.quotes_verified is True + assert (c.role.name, c.scope.name) == ("intern", c.focal.name) + + +# --------------------------------------------------------------------------- # +# 3 — a non-converging entity appears under unevaluated; status != no_conflict. # +# --------------------------------------------------------------------------- # +def test_nonconverging_entity_is_unevaluated_and_not_no_conflict(): + focus = _service(FOCUS_ID, scopes=[_scope("reports", service_id=FOCUS_ID)]) + other = _service(OTHER_ID, roles=[_role("intern", kind=RoleKind.AGENT)]) + + def se(schema, messages): + if schema is ScopeSelection: + return ScopeSelection(roles_with_access_names=["intern"], reasoning="r") + if schema is AuditVerdict: + return AuditVerdict(approved=False, reason="still not right") + raise AssertionError(f"unexpected schema {schema}") + + report = _run("Some policy about reports.", services=[focus, other], side_effect=se) + + assert report.status is not ConflictStatus.NO_CONFLICT + assert report.status is ConflictStatus.INCOMPLETE + assert report.conflicts == [] + assert len(report.unevaluated) == 1 + assert report.unevaluated[0].focal.name == "reports" + assert report.unevaluated[0].reason.value == "nonconvergence" + + +# --------------------------------------------------------------------------- # +# 3b — the unevaluated disjunct is load-bearing: one entity evaluated clean + # +# one non-converging => incomplete (evaluated_count>=1 but unevaluated!=[]).# +# --------------------------------------------------------------------------- # +def test_mixed_evaluated_and_unevaluated_is_incomplete(): + focus = _service( + FOCUS_ID, + scopes=[_scope("clean-scope", service_id=FOCUS_ID), _scope("bad-scope", service_id=FOCUS_ID)], + ) + other = _service(OTHER_ID, roles=[_role("intern", kind=RoleKind.AGENT)]) + + def se(schema, messages): + if schema is ScopeSelection: + return ScopeSelection(roles_with_access_names=["intern"], reasoning="r") + if schema is AuditVerdict: + # The focal scope name is embedded in the auditor messages; bad-scope never converges. + if "bad-scope" in str(messages): + return AuditVerdict(approved=False, reason="nope") + return AuditVerdict(approved=True) + raise AssertionError(f"unexpected schema {schema}") + + report = _run("Policy about scopes.", services=[focus, other], side_effect=se) + + assert report.status is ConflictStatus.INCOMPLETE # not no_conflict, despite one clean entity + assert report.conflicts == [] + assert [u.focal.name for u in report.unevaluated] == ["bad-scope"] + + +# --------------------------------------------------------------------------- # +# 4 — zero focal entities => incomplete (never no_conflict); no LLM call. # +# --------------------------------------------------------------------------- # +def test_zero_focal_entities_is_incomplete_never_no_conflict(): + focus = _service(FOCUS_ID) # no own scopes, TOOL => no role-focal runs, no candidates + + def se(schema, messages): # pragma: no cover - must never be reached + raise AssertionError("no focal entities => the LLM seam must never be called") + + report = _run("Any candidate policy.", services=[focus], side_effect=se) + + assert report.status is ConflictStatus.INCOMPLETE + assert report.conflicts == [] + assert report.unevaluated == [] + + +# --------------------------------------------------------------------------- # +# 5 — AGENT service: the role-focal fan-out runs in addition to scope-focal. # +# --------------------------------------------------------------------------- # +def test_agent_service_runs_scope_and_role_focal_entities(): + focus = _service( + FOCUS_ID, + roles=[_role("weather.agent")], + scopes=[_scope("weather.forecast", service_id=FOCUS_ID)], + service_type=ServiceType.AGENT, + ) + other = _service( + OTHER_ID, + roles=[_role("github.agent", kind=RoleKind.AGENT)], + scopes=[_scope("github.issue", service_id=OTHER_ID)], + ) + seen_schemas = [] + + def se(schema, messages): + seen_schemas.append(schema) + if schema is ScopeSelection: + return ScopeSelection(roles_with_access_names=["github.agent"], reasoning="ok") + if schema is RoleSelection: + return RoleSelection(granted_scope_names=["github.issue"], reasoning="ok") + if schema is AuditVerdict: + return AuditVerdict(approved=True) + raise AssertionError(f"unexpected schema {schema}") + + report = _run("Weather agent policy.", services=[focus, other], side_effect=se) + + assert report.status is ConflictStatus.NO_CONFLICT + assert report.conflicts == [] + assert report.unevaluated == [] + # Both the scope-focal (ScopeSelection) and role-focal (RoleSelection) fan-outs were exercised. + assert ScopeSelection in seen_schemas + assert RoleSelection in seen_schemas + + +# --------------------------------------------------------------------------- # +# 6 — resolver pre-survey HTTP boundary propagates (unknown service / IdP down).# +# --------------------------------------------------------------------------- # +def test_unknown_service_raises_404(): + def se(schema, messages): # pragma: no cover - resolution fails before any LLM turn + raise AssertionError("unreachable") + + with pytest.raises(HTTPException) as ei: + _run("policy", services=[_service(OTHER_ID)], side_effect=se, service_id="nope") + assert ei.value.status_code == 404 + + +def test_idp_unreachable_raises_502(): + conf = MagicMock() + conf.get_services.side_effect = RuntimeError("HTTP 503") + with ( + patch.object(focal_entities, "_config", return_value=conf), + patch(_SEAM, side_effect=AssertionError("unreachable")), + ): + with pytest.raises(HTTPException) as ei: + check_policy_conflicts("policy", FOCUS_ID) + assert ei.value.status_code == 502 From 3b812b234e8ee5df2ac0eb87d5e685a860408fb1 Mon Sep 17 00:00:00 2001 From: Anatoly Koyfman Date: Wed, 19 Aug 2026 09:01:04 +0000 Subject: [PATCH 6/8] Feat: Add POST /policy/check conflict-check route (#154) Thin serialization shell over check_policy_conflicts: any completed survey returns 200 with the ConflictReport JSON body (the controller's first JSON response body). Unlike the live /apply path, a found conflict is a recorded diagnosis and never 422; the resolver's pre-survey HTTPException(502/404) propagates unchanged as a bare non-2xx with no report body, and a missing policy_text is a FastAPI validation 422. The /apply routes and the PolicyContradictionError/PolicyRulesBuilderError -> 422 handlers are untouched. Refs #159. Part of #154. Assisted-By: Claude (Anthropic AI) Signed-off-by: Anatoly Koyfman --- aiac/src/aiac/agent/controller/routes.py | 24 +++ .../controller/test_policy_check_route.py | 188 ++++++++++++++++++ 2 files changed, 212 insertions(+) create mode 100644 aiac/test/agent/controller/test_policy_check_route.py diff --git a/aiac/src/aiac/agent/controller/routes.py b/aiac/src/aiac/agent/controller/routes.py index 85dc4e870..519f8b90f 100644 --- a/aiac/src/aiac/agent/controller/routes.py +++ b/aiac/src/aiac/agent/controller/routes.py @@ -16,12 +16,15 @@ import uvicorn from fastapi import FastAPI, Request from fastapi.responses import JSONResponse, Response +from pydantic import BaseModel from aiac.agent.eventbus.consumer import lifespan +from aiac.agent.policy_rules_builder.diagnostic_models import ConflictReport from aiac.agent.policy_rules_builder.graph import ( PolicyContradictionError, PolicyRulesBuilderError, ) +from aiac.agent.uc.policy_check.check import check_policy_conflicts from aiac.agent.uc.offboarding.offboard import offboard_service from aiac.agent.uc.onboarding.orchestrator import onboard_service from aiac.agent.uc.policy_update.build import build_policy @@ -107,6 +110,27 @@ def apply_offboard(service_id: str) -> Response: return Response(status_code=200) +class PolicyCheckRequest(BaseModel): + """Body for ``POST /policy/check``: candidate ``policy_text`` to survey against the focal + entities of ``service_id`` (the Keycloak internal client UUID, matching + ``/apply/service/{service_id}``). ``policy_text`` is required — its absence is a FastAPI + validation 422 with no report body.""" + + policy_text: str + service_id: str + + +# The read-only conflict-check survey (feature #154). UNLIKE the live /apply routes, this is a +# diagnostic: ANY completed survey is a success and returns 200 with the ConflictReport body — +# a found conflict is a recorded finding, NOT a 422. Only the resolver's pre-survey boundary +# (HTTPException 502 IdP-unreachable / 404 unknown-service) propagates, unchanged, as the bare +# non-2xx error with no report body (we deliberately do NOT catch it). FastAPI serializes the +# returned ConflictReport pydantic model to the JSON response body. +@app.post("/policy/check") +def policy_check(body: PolicyCheckRequest) -> ConflictReport: + return check_policy_conflicts(body.policy_text, body.service_id) + + def main() -> None: uvicorn.run(app, host="0.0.0.0", port=7070) diff --git a/aiac/test/agent/controller/test_policy_check_route.py b/aiac/test/agent/controller/test_policy_check_route.py new file mode 100644 index 000000000..dbad013c6 --- /dev/null +++ b/aiac/test/agent/controller/test_policy_check_route.py @@ -0,0 +1,188 @@ +"""Unit tests for the ``POST /policy/check`` conflict-check route (feature #154, task #159). + +This is the diagnostic serialization shell: it calls the read-only survey use-case +(``check_policy_conflicts``) and serializes the returned :class:`ConflictReport` as a JSON +response body. The use-case is patched at the routes-module boundary — no live IdP, no LLM, no +real diagnostic graph. UNLIKE the live ``/apply`` path, a found conflict is a successful diagnosis +and returns 200 (never 422); only the survey's pre-survey ``HTTPException(502/404)`` propagates. +""" + +from unittest.mock import patch + +from fastapi import HTTPException +from fastapi.testclient import TestClient + +from aiac.agent.controller.routes import app +from aiac.agent.policy_rules_builder.diagnostic_models import ( + Conflict, + ConflictKind, + ConflictReport, + ConflictStatus, + EntityRef, + FocalRef, + FocalType, + Unevaluated, + UnevaluatedReason, +) + +client = TestClient(app) + + +def _focal(name: str = "editor", id: str = "r-1") -> FocalRef: + return FocalRef(name=name, id=id, type=FocalType.ROLE) + + +def _conflict() -> Conflict: + return Conflict( + focal=_focal(), + role=EntityRef(name="editor", id="r-1"), + scope=EntityRef(name="write", id="s-1"), + kind=ConflictKind.DIRECT, + granting_quotes=["editors may write"], + prohibiting_quotes=["editors must not write"], + explanation="write is both granted and prohibited for editor", + quotes_verified=True, + ) + + +def _unevaluated() -> Unevaluated: + return Unevaluated( + focal=_focal("viewer", "r-2"), + reason=UnevaluatedReason.NONCONVERGENCE, + detail="retry budget exhausted", + ) + + +def test_clean_report_returns_200_no_conflict(): + # A survey that evaluated ≥1 entity with nothing outstanding is a positive clean result. + report = ConflictReport.from_survey([], [], evaluated_count=2) + assert report.status is ConflictStatus.NO_CONFLICT + with patch( + "aiac.agent.controller.routes.check_policy_conflicts", return_value=report + ): + resp = client.post( + "/policy/check", json={"policy_text": "editors may read", "service_id": "svc-1"} + ) + + assert resp.status_code == 200 + body = resp.json() + assert body["status"] == "no_conflict" + assert body["conflicts"] == [] + assert body["unevaluated"] == [] + + +def test_conflicts_found_returns_200_not_422(): + # A found conflict is a recorded diagnosis, NOT a policy-input error — it must be 200, unlike + # the live /apply path which maps a contradiction to 422. + report = ConflictReport.from_survey([_conflict()], [], evaluated_count=1) + assert report.status is ConflictStatus.CONFLICTS_FOUND + with patch( + "aiac.agent.controller.routes.check_policy_conflicts", return_value=report + ): + resp = client.post( + "/policy/check", json={"policy_text": "contradictory", "service_id": "svc-1"} + ) + + assert resp.status_code == 200 + body = resp.json() + assert body["status"] == "conflicts_found" + assert len(body["conflicts"]) == 1 + c = body["conflicts"][0] + assert c["kind"] == "direct" + assert c["role"]["name"] == "editor" + assert c["scope"]["name"] == "write" + assert c["granting_quotes"] == ["editors may write"] + assert c["prohibiting_quotes"] == ["editors must not write"] + + +def test_unevaluated_present_returns_200_and_not_no_conflict(): + # A partial run (some entity did not converge) must never look clean — status is forced away + # from no_conflict, and it is still a completed survey ⇒ 200. + report = ConflictReport.from_survey([], [_unevaluated()], evaluated_count=1) + assert report.status is not ConflictStatus.NO_CONFLICT + with patch( + "aiac.agent.controller.routes.check_policy_conflicts", return_value=report + ): + resp = client.post( + "/policy/check", json={"policy_text": "some policy", "service_id": "svc-1"} + ) + + assert resp.status_code == 200 + body = resp.json() + assert body["status"] != "no_conflict" + assert len(body["unevaluated"]) == 1 + assert body["unevaluated"][0]["reason"] == "nonconvergence" + + +def test_incomplete_zero_evaluated_returns_200(): + # Zero focal entities evaluated (empty-input / no-focal case) ⇒ incomplete, still 200. + report = ConflictReport.from_survey([], [], evaluated_count=0) + assert report.status is ConflictStatus.INCOMPLETE + with patch( + "aiac.agent.controller.routes.check_policy_conflicts", return_value=report + ): + resp = client.post( + "/policy/check", json={"policy_text": "some policy", "service_id": "svc-1"} + ) + + assert resp.status_code == 200 + assert resp.json()["status"] == "incomplete" + + +def test_pre_survey_http_502_propagates_with_no_report_body(): + # The resolver's IdP-unreachable boundary must escape the diagnostic unchanged: bare 502, + # no report (FastAPI renders the HTTPException as its default error body). + with patch( + "aiac.agent.controller.routes.check_policy_conflicts", + side_effect=HTTPException(502, "IdP Configuration Service unavailable"), + ): + resp = client.post( + "/policy/check", json={"policy_text": "p", "service_id": "svc-down"} + ) + + assert resp.status_code == 502 + body = resp.json() + assert "status" not in body + assert "conflicts" not in body + + +def test_pre_survey_http_404_propagates_with_no_report_body(): + # Unknown-service boundary likewise propagates as a bare 404 with no report body. + with patch( + "aiac.agent.controller.routes.check_policy_conflicts", + side_effect=HTTPException(404, "service not found in IdP catalog"), + ): + resp = client.post( + "/policy/check", json={"policy_text": "p", "service_id": "svc-missing"} + ) + + assert resp.status_code == 404 + body = resp.json() + assert "status" not in body + assert "conflicts" not in body + + +def test_missing_policy_text_is_422_validation_and_never_calls_survey(): + # policy_text is a required field on the request model — its absence is a FastAPI validation + # 422 (no report body), and the survey is never invoked. + with patch("aiac.agent.controller.routes.check_policy_conflicts") as survey: + resp = client.post("/policy/check", json={"service_id": "svc-1"}) + + assert resp.status_code == 422 + assert "status" not in resp.json() + survey.assert_not_called() + + +def test_route_calls_survey_with_posted_policy_text_and_service_id(): + # The thin shell forwards exactly what was posted to the use-case. + report = ConflictReport.from_survey([], [], evaluated_count=1) + with patch( + "aiac.agent.controller.routes.check_policy_conflicts", return_value=report + ) as survey: + resp = client.post( + "/policy/check", + json={"policy_text": "editors may read", "service_id": "svc-abc"}, + ) + + assert resp.status_code == 200 + survey.assert_called_once_with("editors may read", "svc-abc") From 2df72c4770a08b32e48fdb447816132318e792fd Mon Sep 17 00:00:00 2001 From: Anatoly Koyfman Date: Wed, 19 Aug 2026 09:01:17 +0000 Subject: [PATCH 7/8] Test: Add live-LLM conflict fixtures + /apply 422 regression guard (#154) Two tiers in the new test/agent/policy_check/: - test_conflict_check_live_llm.py (-m llm, deselected by default): drives check_policy_conflicts end-to-end through the real LLM with only the catalog seam (focal_entities._config) stubbed and _structured_call left live. Planted direct + coarse-scope + clean fixtures on the scope-focal branch (Tool focus). Structural assertions only: clean => no_conflict; planted => confirmed set contains the planted (role, scope) pair, and every returned quote is a verbatim whitespace-normalized substring of policy_text (reusing the engine's _verify_quote). Skips cleanly when LLM env is unset. - test_apply_conflict_regression.py (deterministic, default suite): pins the unchanged live contract -- build_role_rules raises PolicyContradictionError on a genuine contradiction, and POST /apply/service/{id} maps it to 422 with the PCE never called. Tests only; no production code changed. Refs #160. Part of #154. Assisted-By: Claude (Anthropic AI) Signed-off-by: Anatoly Koyfman --- aiac/test/agent/policy_check/__init__.py | 0 .../test_apply_conflict_regression.py | 102 ++++++++++ .../test_conflict_check_live_llm.py | 188 ++++++++++++++++++ 3 files changed, 290 insertions(+) create mode 100644 aiac/test/agent/policy_check/__init__.py create mode 100644 aiac/test/agent/policy_check/test_apply_conflict_regression.py create mode 100644 aiac/test/agent/policy_check/test_conflict_check_live_llm.py diff --git a/aiac/test/agent/policy_check/__init__.py b/aiac/test/agent/policy_check/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/aiac/test/agent/policy_check/test_apply_conflict_regression.py b/aiac/test/agent/policy_check/test_apply_conflict_regression.py new file mode 100644 index 000000000..82adb8f95 --- /dev/null +++ b/aiac/test/agent/policy_check/test_apply_conflict_regression.py @@ -0,0 +1,102 @@ +"""Regression guard: the live ``/apply`` conflict path is UNCHANGED by the conflict-check diagnostic. + +This test is **deterministic** (NOT marked ``integration`` / ``llm``) so it runs in the default +suite and under ``-m "not integration"``. Feature #154's design rests on D1/D8 -- the read-only +diagnostic is a *separate* assembly and the safety-critical live ``/apply`` graph stays +byte-for-byte unchanged, still **raising** ``PolicyContradictionError`` -> HTTP 422 on a genuine +grant/deny contradiction (the diagnostic *records* instead). This guard pins both ends of that live +contract, independent of the #159 ``/policy/check`` route: + + 1. **Builder level** -- with ``graph._structured_call`` patched so the auditor returns a genuine + ``Contradiction``, ``build_role_rules`` RAISES ``PolicyContradictionError`` and returns no rule + set (fail-closed). Mirrors ``test_graph.py``'s genuine-overlap slice. + 2. **Route level** -- ``POST /apply/service/{id}`` maps that ``PolicyContradictionError`` to HTTP + 422 and never reaches the PCE. ``onboard_service`` is patched to raise (so no cluster / LLM is + needed), mirroring how ``test/agent/controller/test_routes.py`` drives ``/apply``. +""" + +from contextlib import ExitStack +from unittest.mock import patch + +import pytest +from fastapi.testclient import TestClient + +from aiac.agent.controller.routes import app +from aiac.agent.policy_rules_builder.graph import ( + AuditVerdict, + Contradiction, + PolicyContradictionError, + RoleSelection, + build_role_rules, +) +from aiac.idp.configuration.models import Role, Scope + +client = TestClient(app) + + +class _Source: + """Stub PolicySource whose ``fetch()`` returns a fixed policy string (mirrors ``test_graph.py``).""" + + def __init__(self, text: str = "POLICY"): + self.text = text + + def fetch(self) -> str: + return self.text + + +def test_live_build_role_rules_still_raises_policy_contradiction(): + # A genuine grant/deny overlap on the same coarse candidate: the proposer lists `issues` in BOTH + # its grant and prohibit lists; the auditor adjudicates it GENUINE. The live builder must RAISE + # (fail-closed) -- the diagnostic's record-not-raise fork must not have leaked into this path. + role = Role(id="r-dev", name="developer", composite=False) + issues = Scope(id="s-iss", name="issues") + + with ExitStack() as stack: + stack.enter_context( + patch("aiac.agent.policy_rules_builder.graph.get_policy_source", return_value=_Source()) + ) + stack.enter_context( + patch( + "aiac.agent.policy_rules_builder.graph._structured_call", + side_effect=[ + RoleSelection( + granted_scope_names=["issues"], + denied_scope_names=["issues"], + grant_is_exclusive=False, + reasoning="may read issues but must not modify them", + ), + AuditVerdict( + approved=False, + contradictions=[ + Contradiction( + candidate_name="issues", + description="coarse-scope granularity mismatch: issues covers read and write", + ) + ], + ), + ], + ) + ) + with pytest.raises(PolicyContradictionError) as exc: + build_role_rules(role, [issues]) + + # The raise carries the focal identity and the genuine contradiction(s) -- no rule set comes back. + assert role.name in exc.value.focal + assert [c.candidate_name for c in exc.value.contradictions] == ["issues"] + + +def test_apply_service_maps_policy_contradiction_to_422_and_skips_pce(): + # The Controller maps a PolicyContradictionError raised inside the onboarding handler to HTTP 422 + # (a policy finding, not a 500), and the PCE is never reached. onboard_service is patched to raise + # directly so the route mapping is exercised without a cluster or the LLM. + with ( + patch( + "aiac.agent.controller.routes.onboard_service", + side_effect=PolicyContradictionError("focal-svc", []), + ), + patch("aiac.agent.controller.routes.compute_and_apply") as pce, + ): + resp = client.post("/apply/service/svc-conflict") + + assert resp.status_code == 422 + pce.assert_not_called() diff --git a/aiac/test/agent/policy_check/test_conflict_check_live_llm.py b/aiac/test/agent/policy_check/test_conflict_check_live_llm.py new file mode 100644 index 000000000..4dfff10f8 --- /dev/null +++ b/aiac/test/agent/policy_check/test_conflict_check_live_llm.py @@ -0,0 +1,188 @@ +"""Live-LLM planted-fixture suite for the Policy Conflict Check diagnostic (feature #154, task #160). + +Mirrors ``test/agent/policy_rules_builder/test_graph_live_llm.py``: it runs the **real** LLM +end-to-end through the read-only conflict-check survey (``check_policy_conflicts``) and asserts +**structural** properties of the emitted ``ConflictReport`` -- never exact quote strings or +explanation wording (model nondeterminism; convergence on subtle prose is known-fragile). + +Only the IdP catalog is stubbed -- the ``focal_entities._config`` seam ``check_policy_conflicts`` +reuses for BOTH the ``service_type`` lookup and ``resolve_focal_entities`` (a single patch point, +preserving the 502/404 boundary). ``graph._structured_call`` is deliberately left **live** so the +real proposer / auditor / explain prompts run: a prompt-engineering regression (a missed conflict, +a hallucinated deny, a non-verbatim quote) fails a fixture here where a mocked suite could not see +it. The candidate ``policy_text`` is supplied **directly** to ``check_policy_conflicts`` (the +diagnostic seeds it from input -- no policy-source stub needed). + +Gating: the module is marked **both** ``integration`` and ``llm``. ``integration`` -> the routine +``-m "not integration"`` run deselects it (its collected count is unchanged). ``llm`` -> it can be +selected on its own (``-m llm``) without a cluster or Keycloak. The autouse +``require_env_or_skip("LLM_BASE_URL", "LLM_MODEL", "LLM_API_KEY")`` fixture makes every test **skip +cleanly** (never crash, never false-pass) when the endpoint is unset. + +Planting choice (documented): all three fixtures are planted on the **scope-focal** branch of the +survey. The focus service is a **Tool** that owns exactly **one** scope (the focal scope); the +offending pair's role is a **user-held realm role** (a ``candidate_role``, resolved from a stubbed +subject). Focus ``type == Tool`` means the AGENT-only role-focal branch never runs, so the survey +evaluates exactly the one planted scope-focal entity -- minimal surface, least nondeterminism, and +the direct/coarse collision is adjudicated against a real, typed ``(candidate_role, own_scope)`` +pair exactly as a scope-focal run does. +""" + +from unittest.mock import patch + +import pytest + +from aiac.agent.policy_rules_builder.diagnostic import _verify_quote +from aiac.agent.policy_rules_builder.diagnostic_models import ConflictStatus +from aiac.idp.configuration.models import Role, Scope, Service, ServiceType, Subject +from aiac.agent.uc.policy_check.check import check_policy_conflicts +from test.integration.launcher import require_env_or_skip + +pytestmark = [pytest.mark.integration, pytest.mark.llm] + +# The Keycloak internal client UUID the /apply/service/{id} route and check_policy_conflicts key on. +FOCUS_ID = "svc-focus-uuid" + + +# --------------------------------------------------------------------------- # +# Harness # +# --------------------------------------------------------------------------- # +@pytest.fixture(autouse=True) +def _require_llm_env(): + """Skip the whole suite cleanly unless a real LLM endpoint is configured. Autouse so every + fixture is gated without repeating the call; runs before the patched survey.""" + require_env_or_skip("LLM_BASE_URL", "LLM_MODEL", "LLM_API_KEY") + + +class _FakeConfig: + """Duck-typed stand-in for ``Configuration`` -- ``check_policy_conflicts`` and + ``resolve_focal_entities`` only ever call ``get_services()`` / ``get_subjects()`` on it. Patched + in at ``focal_entities._config`` so the single seam drives both the ``service_type`` lookup and + the resolver.""" + + def __init__(self, services: list[Service], subjects: list[Subject]): + self._services = services + self._subjects = subjects + + def get_services(self) -> list[Service]: + return self._services + + def get_subjects(self) -> list[Subject]: + return self._subjects + + +def _scope(id: str, name: str, description: str) -> Scope: + """An aiac.managed scope owned by the focus Tool -- becomes an ``own_scope`` (a scope-focal run).""" + return Scope( + id=id, + name=name, + description=description, + attributes={"aiac.managed": "true"}, + serviceId="focus-tool", + ) + + +def _role(id: str, name: str, description: str) -> Role: + """A realm role held by the stubbed user -- becomes a ``candidate_role`` (membership-derived, not + aiac.managed, and not owned by any service).""" + return Role(id=id, name=name, description=description, composite=False, childRoles=[]) + + +def _run(policy_text: str, *, focal_scope: Scope, candidate_role: Role): + """Drive ``check_policy_conflicts`` end-to-end with a stubbed one-scope Tool focus and a single + user-held candidate role. Only the catalog seam is patched; the LLM is left live.""" + focus = Service( + id=FOCUS_ID, + serviceId="focus-tool", + enabled=True, + type=ServiceType.TOOL, # Tool => role-focal (AGENT-only) branch never runs. + roles=[], + scopes=[focal_scope], + ) + subject = Subject(id="u-planted", username="planted-user", enabled=True, roles=[candidate_role]) + with patch( + "aiac.agent.shared.focal_entities._config", + return_value=_FakeConfig([focus], [subject]), + ): + return check_policy_conflicts(policy_text, FOCUS_ID) + + +def _pairs(report) -> set[tuple[str, str]]: + """Confirmed conflict set as ``(role.name, scope.name)`` pairs (matched by name/id-agnostic name).""" + return {(c.role.name, c.scope.name) for c in report.conflicts} + + +def _assert_quotes_verbatim(report, policy_text: str) -> None: + """Every string in every conflict's ``granting_quotes`` / ``prohibiting_quotes`` must be a + verbatim (whitespace-normalized) substring of ``policy_text`` -- reusing the engine's own + ``_verify_quote`` so the normalization matches exactly.""" + for conflict in report.conflicts: + for quote in conflict.granting_quotes + conflict.prohibiting_quotes: + assert _verify_quote(quote, policy_text), ( + f"quote is not a verbatim substring of policy_text: {quote!r}" + ) + + +# --------------------------------------------------------------------------- # +# Fixture 1 -- DIRECT conflict (scope-focal). The policy both grants and # +# prohibits the SAME (developer, source-write) pair on the same action ("write"),# +# a direct contradiction. The confirmed set must contain that pair. # +# --------------------------------------------------------------------------- # +def test_direct_conflict_contains_planted_pair(): + write = _scope("sc-write", "source-write", "Write and modify source code in the repository.") + developer = _role("role-dev", "developer", "A software developer.") + + policy = ( + "Developers may write to the source code repository. " + "Developers must not write to the source code repository." + ) + + report = _run(policy, focal_scope=write, candidate_role=developer) + + assert report.status == ConflictStatus.CONFLICTS_FOUND + assert ("developer", "source-write") in _pairs(report) + _assert_quotes_verbatim(report, policy) + + +# --------------------------------------------------------------------------- # +# Fixture 2 -- COARSE-SCOPE conflict (scope-focal). The focal scope is COARSE # +# ("manage ... reading and modifying"); the policy grants the read facet and # +# prohibits the write facet of that single coarse scope -- a granularity mismatch # +# on (developer, issues-manage). The confirmed set must contain that pair. # +# --------------------------------------------------------------------------- # +def test_coarse_scope_conflict_contains_planted_pair(): + issues = _scope( + "sc-iss", + "issues-manage", + "Manage the issue tracker, including reading and modifying issues.", + ) + developer = _role("role-dev", "developer", "A software developer.") + + policy = ( + "Developers may read the issue tracker. " + "Developers must not modify the issue tracker." + ) + + report = _run(policy, focal_scope=issues, candidate_role=developer) + + assert report.status == ConflictStatus.CONFLICTS_FOUND + assert ("developer", "issues-manage") in _pairs(report) + _assert_quotes_verbatim(report, policy) + + +# --------------------------------------------------------------------------- # +# Fixture 3 -- CLEAN policy (scope-focal). A single grant, no prohibition: # +# exactly one entity evaluated, no conflict, nothing unevaluated => no_conflict. # +# --------------------------------------------------------------------------- # +def test_clean_policy_is_no_conflict(): + deploy = _scope("sc-dep", "deploy", "Deploy the application to production.") + operator = _role( + "role-ops", "operator", "An operations engineer who deploys and runs the application." + ) + + policy = "Operators may deploy the application to production." + + report = _run(policy, focal_scope=deploy, candidate_role=operator) + + assert report.status == ConflictStatus.NO_CONFLICT + assert report.conflicts == [] From 760eebb85313a7e70fe3099acf763b5ea681fd6f Mon Sep 17 00:00:00 2001 From: Anatoly Koyfman Date: Wed, 26 Aug 2026 14:28:07 +0000 Subject: [PATCH 8/8] Style: Sort controller import block (isort/ruff I001) Reorder the uc.* imports in controller/routes.py so offboarding < onboarding < policy_check, resolving ruff I001. The aiac/ tree is outside the repo pre-commit ruff scope (authbridge/-only), so this was not caught in CI. Import-only change; no behavior change. Addresses a review nit on #154. Assisted-By: Claude (Anthropic AI) Signed-off-by: Anatoly Koyfman --- aiac/src/aiac/agent/controller/routes.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aiac/src/aiac/agent/controller/routes.py b/aiac/src/aiac/agent/controller/routes.py index 519f8b90f..a06fffbab 100644 --- a/aiac/src/aiac/agent/controller/routes.py +++ b/aiac/src/aiac/agent/controller/routes.py @@ -24,9 +24,9 @@ PolicyContradictionError, PolicyRulesBuilderError, ) -from aiac.agent.uc.policy_check.check import check_policy_conflicts from aiac.agent.uc.offboarding.offboard import offboard_service from aiac.agent.uc.onboarding.orchestrator import onboard_service +from aiac.agent.uc.policy_check.check import check_policy_conflicts from aiac.agent.uc.policy_update.build import build_policy from aiac.agent.uc.policy_update.rebuild import rebuild_policy from aiac.agent.uc.role_update.role import update_role