Skip to content

Latest commit

Β 

History

31 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

🌐 PersonaDiff

Evidence-First Multi-Persona Web Journey Comparison & Differential Audit Platform

Node.js TypeScript Playwright Fastify React Security License

Execute identical bounded web journeys across isolated browser personas, capture cryptographic evidence, and compute explainable comparison metrics without claiming causation.

Quick Start β€’ Key Features β€’ Architecture β€’ Metrics Engine β€’ Security β€’ Documentation


PersonaDiff β€” Comparison Runs Dashboard
Comparison Runs Dashboard β€” Real-time execution cards, status filters, and historical runs overview across isolated Playwright contexts.



Launch Live Multi-Persona Audit
Audit Launch Wizard β€” Target presets, custom journey script builder, and regional persona matrix.
Cross-Persona Divergence Analysis
Cross-Persona Divergence Analysis β€” Cryptographic side-by-side comparison with automated content divergence alerts.
Evidence Forensic Replay
Evidence Forensic Replay β€” Targetless replay inspector to review captured DOM timeline snapshots.

πŸ“‘ Table of Contents


πŸ” Overview

Modern web platforms dynamically alter content, geo-pricing, feature availability, and localization based on user signals: geography, language preference, viewport dimensions, authentication status, or request headers. Auditing these discrepancies responsibly demands absolute browser isolation, cryptographic provenance, and statistically honest diffing.

PersonaDiff is a production-grade differential web auditing system that allows operators and researchers to:

  1. Orchestrate Parallel Journeys: Execute identical navigation scripts across 2 to $N$ isolated personas simultaneously with zero cross-context leakage.
  2. Capture Cryptographic Evidence: Collect full-resolution screenshots, raw DOM snapshots, and network traces with pre-storage PII redaction and SHA-256 hash chaining.
  3. Compute Deterministic Diffs: Calculate structural DOM similarity, text cosine distance, ranking permutations, numeric deltas, and redirect routes.
  4. Replay Without Live Targets: Reconstruct and inspect past runs step-by-step from immutable storage without generating outbound network requests.
  5. Enforce Ethical & Security Boundaries: Enforce strict surface allowlisting, pre-navigation SSRF IP checks, and non-causal reporting standards.

πŸ’‘ 100% Self-Contained Local Mode: PersonaDiff includes a built-in deterministic fixture service (http://localhost:4300) pre-approved for local testing, demo recordings, and offline evaluations with zero third-party consent required.


✨ Key Features

Feature Description
πŸ›‘οΈ Zero-Leakage Isolation Launches dedicated Playwright contexts per persona with strict lifecycle teardown, ensuring cookies, localStorage, and caches never cross boundaries.
πŸ”’ Defense-in-Depth SSRF Guard Validates target URLs against strict surface allowlists and blocks loopback, private RFC-1918 subnets, and cloud metadata IPs (169.254.169.254).
🧹 Pre-Storage PII Redaction Automatically strips auth tokens, passwords, session cookies, and sensitive parameters before persisting evidence artifacts.
πŸ“Š Deterministic Comparison Engine Evaluates element presence, tokenized text cosine similarity, rank shift, numeric deltas, and redirect routes with confidence scoring.
πŸ€– AI-Assisted Divergence Insights Provides explainable, non-causal visual and regional pricing analysis powered by Google Gemini.
⏰ Continuous Monitoring Schedules recurring background audits with configurable intervals and real-time execution countdowns.
πŸŽ₯ Targetless Replay Mode Reconstructs captured journeys offline directly from stored DOM snapshots and screenshots without contacting external hosts.
πŸ“¦ Tamper-Evident Export Bundles Generates verifiable export packages with cryptographic SHA-256 checksums and immutable audit manifests.

πŸ—οΈ System Architecture

                               β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                               β”‚   Web Operator UI      β”‚
                               β”‚  (React 18 + Vite)     β”‚
                               β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                           β”‚ HTTP / REST
                               β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                               β”‚     Fastify REST API   β”‚
                               β”‚  (RBAC, Rate Limits)   β”‚
                               β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜
                                     β”‚            β”‚
                      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”        β”Œβ”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                      β”‚ PostgreSQL DB  β”‚        β”‚  Redis Queue   β”‚
                      β”‚ (Runs, Audit)  β”‚        β”‚  (Broker)      β”‚
                      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜        β””β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”˜
                                                  β”‚            β”‚
                          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”       β”Œβ”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                          β”‚ Playwright Browser Workerβ”‚       β”‚ Comparison Worker        β”‚
                          β”‚  β€’ Isolated Contexts     β”‚       β”‚  β€’ Normalization Engine  β”‚
                          β”‚  β€’ SSRF & Route Guards   β”‚       β”‚  β€’ Jaccard / Cosine Diff β”‚
                          β”‚  β€’ Pre-Storage Redaction β”‚       β”‚  β€’ Rank / Delta Metrics  β”‚
                          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜       β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                      β”‚                                  β”‚
                                      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                                    β”‚
                                      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                                      β”‚ S3 / MinIO Object Store  β”‚
                                      β”‚  (Immutable Artifacts)   β”‚
                                      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Execution Lifecycle

  1. Request Ingestion: REST API validates tenant permissions, idempotency keys, and registers the comparison run.
  2. Worker Dispatch: Parallel job requests are queued via Redis for both browser execution and differential analysis.
  3. Isolated Capture: Worker spawns ephemeral, isolated Playwright contexts, enforces SSRF rules, redacts PII, and streams assets to MinIO/S3.
  4. Metric Computation: Worker Compare calculates deterministic text, DOM, and visual similarity scores.
  5. Cryptographic Manifest: SHA-256 hashes are computed for all artifacts and saved into PostgreSQL for tamper-evident provenance.

πŸš€ Quick Start

1. Prerequisites

  • Node.js 24+ & npm 10+
  • Docker Compose v2

2. Installation & Infrastructure

# 1. Clone the repository
git clone https://github.com/omerfarooq223/ParallelWeb.git
cd ParallelWeb

# 2. Install dependencies & Playwright Chromium
npm install
npx playwright install chromium

# 3. Start local backing infrastructure (PostgreSQL, Redis, MinIO, OTel)
npm run stack:up

3. Launch the Web Operator UI

# Start the web client development server
npm run dev --workspace=@ai-parallel-web/web

Open http://localhost:5173 in your browser to access the operator dashboard.

4. Verify System Health

  • API Health Readiness: http://localhost:3000/health/ready
  • Local Deterministic Fixtures: http://localhost:4300/fixture

πŸ”Œ API Usage & Role-Based Tokens

PersonaDiff includes pre-seeded development tokens for role-based access control (RBAC):

Role Development Bearer Token Permissions & Scope
admin pw-admin-token-dev-only-0001 Full tenant management, runs, policy updates, and system metrics
operator pw-operator-token-dev-only-001 Audit creation, live execution triggers, cancel, and diff inspection
viewer pw-viewer-token-dev-only-0001 Read-only access to completed audit manifests and offline replays

Trigger a Comparison Run via cURL

curl --request POST http://localhost:3000/v1/runs \
  --header "Authorization: Bearer pw-operator-token-dev-only-001" \
  --header "Idempotency-Key: audit-run-$(date +%s)" \
  --header "Content-Type: application/json" \
  --data '{
    "surfaceId": "00000000-0000-4000-8000-000000000010",
    "journeyVersionId": "00000000-0000-4000-8000-000000000020",
    "personaVersionIds": [
      "00000000-0000-4000-8000-000000000030",
      "00000000-0000-4000-8000-000000000031"
    ]
  }'

πŸ“ Comparison Engine & Metrics

PersonaDiff uses mathematically rigorous, deterministic algorithms to compare captures without bias:

Metric Algorithm / Formula Flag Threshold Purpose
DOM Element Presence Jaccard Similarity on Element Sets $< 0.90$ Detects missing or extra rendered UI containers
Text Content Similarity Tokenized Cosine & Jaccard Overlap $< 0.95$ Identifies copy, title, and descriptive text changes
Rank / Order Shift $\frac{\text{Position-Changed Items}}{\text{Total Items}}$ $> 0.0$ Detects personalized sorting or item substitution
Numeric Delta $\frac{|V_{\text{variant}} - V_{\text{control}}|}{|V_{\text{control}}|}$ $> 1.0%$ Detects price, fee, or quantity adjustments
Redirect Path Diff Normalized URL Path Matching Non-Identical Flags routing or localized redirect discrepancies
Timing Delta $|T_{\text{variant}} - T_{\text{control}}|$ (ms) $> 1000\text{ms}$ Measures load duration and latency variance

βš–οΈ Non-Causal Reporting Standard: All observations are reported as "Observed differences under recorded conditions". PersonaDiff strictly avoids inferring algorithmic intent, discriminatory motive, or causal mechanism.


πŸ›‘οΈ Security & Ethical Boundaries

Security and ethical guardrails are deeply embedded into the platform architecture:

  • Strict Egress Containment: Restrictive route interception blocks network calls outside registered surface domains.
  • SSRF Defense-in-Depth: Pre-navigation DNS resolution and CIDR filtering block intranet IPs (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 127.0.0.1, ::1, 169.254.169.254).
  • Automated Data Minimization: 30-day retention policies with cascading artifact deletion workflows.
  • Sandboxed Replay: Targetless replay executes untrusted historical HTML within secure, sandboxed iframes.

πŸ“‚ Repository Layout

ParallelWeb/
β”œβ”€β”€ apps/
β”‚   β”œβ”€β”€ api/                 # Fastify REST API service (auth, routes, orchestration)
β”‚   β”œβ”€β”€ browser-spike/       # Spike runner & surface policy testing harness
β”‚   β”œβ”€β”€ fixture/             # Deterministic local target fixture service
β”‚   β”œβ”€β”€ web/                 # React 18 + Vite operator interface
β”‚   β”œβ”€β”€ worker-browser/      # Playwright browser automation worker pool
β”‚   └── worker-compare/      # Asynchronous comparison & metric worker
β”œβ”€β”€ packages/
β”‚   β”œβ”€β”€ auth/                # RBAC roles and permission evaluators
β”‚   β”œβ”€β”€ capture/             # PII redaction engine, manifests & retention workflows
β”‚   β”œβ”€β”€ comparison/          # Deterministic metric algorithms & normalization
β”‚   β”œβ”€β”€ contracts/           # OpenAPI 3.1 specs, JSON schemas & TypeScript types
β”‚   β”œβ”€β”€ db/                  # PostgreSQL client pool, migrations & repositories
β”‚   β”œβ”€β”€ domain/              # State machine, reconciliation & export builders
β”‚   β”œβ”€β”€ observability/       # OpenTelemetry, Prometheus metrics & structured logger
β”‚   β”œβ”€β”€ storage/             # S3 / MinIO immutable artifact adapter
β”‚   └── test-fixtures/       # Shared golden test payloads
β”œβ”€β”€ config/                  # Surface policies & tooling configurations
β”œβ”€β”€ docs/                    # Architecture, security, specifications & runbooks
β”‚   └── screenshots/         # UI showcase screenshots & evidence captures
β”œβ”€β”€ infra/                   # Docker compose stack & database migration scripts
└── tests/                   # Contract, integration, failure-injection & security tests

πŸ§ͺ Testing & Verification

PersonaDiff maintains high test coverage across unit, integration, contract, and security layers:

# Run the full automated test suite
npm test

# Run comprehensive workspace verification (Formatting + Lint + TypeCheck + Tests)
npm run check

# Run execution isolation tests (Playwright context separation)
npm test -- tests/integration/execution-isolation.test.ts

# Run security & SSRF defense test suites
npm test -- tests/security

# Validate OpenAPI contracts against schemas
npm run contracts:validate

πŸ“š Documentation Index

Topic Reference Document Description
πŸ›οΈ Architecture Decisions docs/adr/ ADR-0001 through ADR-0005 documenting key technical decisions
πŸ”’ Security & Threat Model docs/security/threat-model.md Comprehensive STRIDE threat model & attack surface mitigations
πŸ“‹ Privacy & Data Map docs/security/privacy-data-map.md Field-by-field lifecycle, PII masking, and retention rules
πŸ“œ Acceptable Use Policy docs/security/acceptable-use-policy.md Responsible research guidelines and operational guardrails
πŸ“Š Metric Specifications docs/spec/comparison-metrics.md Metric algorithms, normalization logic, and thresholds
πŸ› οΈ Operations & Runbooks docs/operations/operations-handoff.md Production deployment, monitoring, and operational handoff

PersonaDiff β€’ Developed by Muhammad Umar Farooq β€’ Built with ❀️ for deterministic, evidence-first web auditing.

About

Production-grade differential web auditing system using Playwright for parallel browser persona execution, cryptographic evidence capture, and deterministic DOM/text/visual comparison metrics with security guardrails

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors

Languages