Skip to content

feat(test): write down the safety invariants and test them as invariants - #71

Open
mkoushni wants to merge 3 commits into
praxis-proxy:mainfrom
mkoushni:feat/test-safety-invariants
Open

feat(test): write down the safety invariants and test them as invariants#71
mkoushni wants to merge 3 commits into
praxis-proxy:mainfrom
mkoushni:feat/test-safety-invariants

Conversation

@mkoushni

@mkoushni mkoushni commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Write the fail-closed promise down as testable claims in docs/safety-invariants.md (linked from CONTRIBUTING) and drive them with a reusable fault-injection plugin and PDP resolver.
  • Contain panics in serial, transform, and audit the same way concurrent already did: route through on_error instead of unwinding execute(). Halt vs continue still follows can_block; that difference is documented with a reason.
  • Table-driven catalog: every plugin phase × {panic, error, timeout} under on_error: fail, plus ignore/disable for serial, transform, and audit. A sequential panic under ignore still runs later audit; under fail it does not.
  • Malformed config covers unknown key, misspelled top-level block, and unparseable YAML. Omitted on_error in YAML is Fail. Missing subject.id pins the differential allowlist row (Cedar dispatch error, CEL eval error, OPA default deny). Malformed policy pins Cedar dispatch / CEL compile / OPA compile-or-dispatch.
  • PDP {panic, error, timeout} cells drive the evaluator wrapper (FaultPdp), not the Cedar/CEL/OPA resolvers themselves. Real dialects are covered for missing attribute and malformed policy. A new phase or shipped dialect fails the build until a cell is added.

Closes #24

Test plan

  • cargo test -p praxis-policy-core --test safety_invariants
  • cargo test -p praxis-policy-apl-core --test safety_invariants
  • cargo test -p praxis-policy-pdp-diff safety

The fail-closed promise lived in comments and per-seam judgment. Codify
it as a catalog and contain panics in every awaited phase the way
concurrent already did.

Fixes praxis-proxy#24

Signed-off-by: mkoushni <mkoushni@redhat.com>
A one-plugin catalog could not fail if a contained serial panic still
skipped later audit, and several cells only asserted "not allow". Name
the decision per cell and cover ignore/disable outside concurrent.

Signed-off-by: mkoushni <mkoushni@redhat.com>

@praxis-bot praxis-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: feat(test): write down the safety invariants and test them as invariants

Strong work. The containment model is sound: serial, transform, and audit panics now route through on_error the same way concurrent already did. The catalog approach -- exhaustive matches that fail the build when a phase or dialect is added -- is exactly the right mechanism for these invariants. The fault-injection infrastructure is well-factored and the test matrix is thorough.

Two findings, both medium.

Severity File Finding
Medium executor.rs ContainedOutcome::Lost non-blocking paths skip OnError::Disable
Medium plugin.rs is_dispatch_phase takes self by value; all other PluginMode methods take &self

Comment thread crates/ppe-core/src/executor.rs
Comment thread crates/ppe-core/src/plugin.rs Outdated
A non-blocking Lost outcome recorded the error but skipped the Disable
circuit breaker that Error, Timeout, and Panic already trip. Match the
other PluginMode predicates on the receiver too.

Signed-off-by: mkoushni <mkoushni@redhat.com>
@araujof araujof added the tests label Sep 3, 2026
@araujof araujof moved this from Backlog to In progress in Praxis Policy Engine (PPE) Sep 3, 2026
@araujof araujof added this to the 0.1.2 milestone Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

feat(test): write down the safety invariants and test them as invariants

3 participants