Skip to content

testing-fixtures.md

Latest commit

 

History

History
75 lines (54 loc) · 2.95 KB

testing-fixtures.md

File metadata and controls

75 lines (54 loc) · 2.95 KB
Raw

Testing Fixtures

Fixture apps prove that Rulepath works across supported frameworks and data layers.

Layout

fixtures/
  express_prisma/
    safe/
    unsafe/
  fastapi_sqlalchemy/
    safe/
    unsafe/
  django_drf/
    safe/
    unsafe/
  nextjs_prisma_authjs/
    safe/
    unsafe/
  v1_rule_matrix/
    safe/
    unsafe/

Each fixture should be small, readable, and focused on one analysis behavior.

Language adapters also carry parser-level unit coverage for imports, symbols, calls, decorators, and suppressions. Fixture scans should assert the end-to-end path: parser facts, framework route extraction, data-layer operation extraction, trace propagation, and final findings.

Required Scenarios

Each fixture family should cover:

  • Route discovery.
  • Request source extraction.
  • Authentication evidence.
  • Authorization evidence.
  • Object-scope evidence.
  • Tenant-scope evidence.
  • Service-layer tracing.
  • ORM sink detection.
  • Finding emission.
  • Review-hint emission.
  • Baseline behavior.
  • CI failure behavior.
  • Suppression behavior.

Acceptance Milestones

The first milestone is:

rulepath scan fixtures/express_prisma/unsafe

It should discover an Express route, trace to a Prisma sink, detect request-controlled IDs and body data, and emit INV001 plus INV002.

The second milestone mirrors that behavior for FastAPI and SQLAlchemy.

The Django/DRF fixture pair verifies DRF route extraction and Django ORM extraction together. The unsafe fixture should emit unscoped resource access for a request-controlled object lookup on the inferred InvoiceViewSet.get_object route, while the safe fixture should remain clean because tenant filtering or object permission evidence is recognized.

The Next.js/Prisma/Auth.js fixture pair verifies App Router route extraction, dynamic [invoiceId] source IDs, Auth.js evidence labels, and Prisma mutation findings. The unsafe fixture should emit findings for unscoped access, client-controlled mutation data, and missing operation authorization; the safe fixture should remain clean because Auth.js, permission, tenant, and allowlisted field evidence are present.

v1 Rule Matrix

fixtures/v1_rule_matrix is a compact Express/Prisma fixture pair used to lock rule and hint coverage that is not naturally exercised by the framework parity fixtures.

The unsafe matrix intentionally includes:

  • State transition, idempotency, bulk mutation, and export cases for INV004 through INV007.
  • Unconfigured resource, authorization-looking helper, workflow transition, money-like operation, unknown export, and auth-without-scope cases for HINT001 through HINT006.

The safe matrix mirrors those cases with configured transition, idempotency, authorization, and tenant-scope evidence. It should emit no findings and no review hints.

Together with the unsafe Express/Prisma fixture, the matrix-backed CLI tests assert coverage for every INV001 through INV008 finding and every HINT001 through HINT006 review hint.