GitHub - obrienma/sentinel-l7: AI-Driven Observability Engine: A high-performance monitoring system for Finance/Medical/SaaS. Sentinel-L7 does not monitor infrastructure; it monitors Business Intent.

Sentinel-L7 is a multi-process Laravel application built to explore production patterns for async message processing, semantic caching, and fault-tolerant distributed systems. It processes any scored event stream — financial events, medical access logs, SaaS API activity, raw system telemetry — and classifies each event against an indexed corpus of domain-specific policy documents to determine whether it exceeds a risk threshold. A compliance engine (AML, GDPR, HIPAA) is the domain used here; the architecture is domain-agnostic.

%%{init: {'themeVariables': {'fontSize': '10px'}, 'flowchart': {'nodeSpacing': 15, 'rankSpacing': 30}}}%%
flowchart LR
    subgraph Ingestion
        A[Events<br/>XADD]
        SL[synapse-l4]
    end

    subgraph SentinelL7["Sentinel-L7"]
        subgraph Processing
            B[Worker Pool<br/>PHP]
        end
        subgraph Intelligence
            CASCADE[Cache → RAG → LLM → Fallback]
        end
        subgraph Interface
            MCP[MCP Server]
            E[React<br/>Dashboard]
        end
        subgraph Persistence
            D[(Neon<br/>Postgres)]
        end
    end

    AR[Arbiter-L8<br/>External Eval Harness]

    subgraph DC["Downstream Consumers"]
        direction TB
        LE[Ledger-L5]
        RL[Rhizome-Lens]
    end

    Interface ~~~ AR
    Interface ~~~ DC

    A -->|Redis Stream| B
    SL -->|Validated Events| B
    B -->|Evaluation Request| CASCADE
    CASCADE -->|Verdict| D
    D -->|Query| E
    D -->|Usage Events| LE
    SentinelL7 -->|OTel Traces/Logs| RL
    AR -->|MCP: analyze-transaction<br/>driver override| MCP
    AR -->|OTel Metrics/Traces| RL

    click CASCADE "docs/diagrams/TRANSACTION_PIPELINE.md" "See the full transaction pipeline diagram"
    click SL "https://github.com/obrienma/synapse-l4#readme" "Go to Synapse-L4 repo"
    click AR "https://github.com/obrienma/Arbiter-L8#readme" "Go to Arbiter-L8 repo"
    click LE "https://github.com/obrienma/Ledger-L5#readme" "Go to Ledger-L5 repo"
    click RL "https://github.com/obrienma/Rhizome-Lens#readme" "Go to Rhizome-Lens repo"

    class A,D teal
    class B purple
    class MCP,E amber
    class SL,AR,LE,RL,CASCADE clickable

    classDef clickable fill:#1d4ed8,stroke:#1e40af,stroke-width:2px,color:#ffffff
    classDef teal fill:#E1F5EE,stroke:#0F6E56,color:#04342C
    classDef purple fill:#EEEDFE,stroke:#534AB7,color:#26215C
    classDef amber fill:#FDF0D5,stroke:#9A6B0A,color:#4A3200

    style Ingestion fill:#D9D9D9,stroke:#5F5E5A,color:#000000
    style SentinelL7 fill:#D9D9D9,stroke:#5F5E5A,color:#000000
    style DC fill:#D9D9D9,stroke:#5F5E5A,color:#000000

    style Processing fill:#FFFFFF,stroke:#5F5E5A,color:#000000
    style Intelligence fill:#FFFFFF,stroke:#5F5E5A,color:#000000
    style Interface fill:#FFFFFF,stroke:#5F5E5A,color:#000000
    style Persistence fill:#FFFFFF,stroke:#5F5E5A,color:#000000

The "Cache → RAG → LLM → Fallback" box collapses the full cascade — see docs/diagrams/TRANSACTION_PIPELINE.md for the step-by-step breakdown, or the Pipeline Diagram under Architecture for the internal sequencing.

🧰 Stack

🚀 Backend & Ingestion

Laravel 12 / PHP 8.4: Service container, queue, and Artisan command bus drive three long-running processes — web, transaction worker, and axiom worker — each consuming its own Redis Stream via XREADGROUP.
Upstash Redis Streams: XADD / XREADGROUP / XAUTOCLAIM provide at-least-once delivery with a Pending Entry List; the web process never blocks waiting for analysis.

⚡ AI & Vector

Ollama (default) + Gemini Flash + OpenRouter: LLM analysis runs through a swappable ComplianceDriver interface backed by a Laravel Service Manager; switching providers is a single env-var change, not a code change. Default is local/self-hosted qwen3.5:9b-q4_K_M via Ollama (ADR-0027) — no external API quota on the compliance-analysis path.
Upstash Vector: Named-namespace strategy (ADR-0026) — transactions (semantic cache, ≥ 0.90 threshold — ADR-0015) cuts repeat LLM calls by 80%+; policies (RAG corpus, ≥ 0.70, domain-filtered) grounds compliance rulings in indexed regulatory documents (AML, HIPAA, GDPR). No data lives in Upstash's implicit default namespace.

👁️ Frontend & Observability

React 19 + Inertia.js + shadcn/ui: Server-driven SPA — no API layer needed; dashboard, compliance events, and CSV export all use Inertia page components with a dark-default shadcn/Tailwind v4 theme.
OpenTelemetry: Wide spans on both worker processes export via OTLP to a companion Grafana stack (Tempo + Prometheus); traceparent propagated from Synapse-L4 stream entries continues the upstream trace as a child span (ADR-0024).

🧪 Testing & Infrastructure

Pest: Feature, unit, and architecture tests; arch tests in tests/ArchTest.php enforce domain layer isolation — Http and Redis facade imports are banned inside App\Services\Sentinel\Logic.
Neon PostgreSQL + Railway: Serverless Postgres (non-pooled host — SELECT … FOR UPDATE SKIP LOCKED requirement) for compliance_events and transactions; Railway hosts all three processes.

🚀 Running the Project

✅ Prerequisites

PHP 8.4+ with Composer
Node.js 20+
Upstash account — Redis Streams + Vector namespaces (transactions, policies)
Neon PostgreSQL database
Gemini API key (or OpenRouter key if using that driver)

Note

Developed on WSL2 (Ubuntu) and deployed to Railway. Other environments may work but are untested.

⚡ Quick Start

# 1. Install dependencies
composer install && npm install

# 2. Copy env and fill in Upstash, Neon, and Gemini credentials
cp .env.example .env

# 3. Run migrations
php artisan migrate

# 4. Index policy documents into the vector knowledge base
php artisan sentinel:ingest

# 5. Start all processes (web + queue + logs + Vite + axiom watcher)
composer dev

# 6. In a separate terminal, generate transactions
php artisan sentinel:stream --limit=100

# 7. Open the dashboard
open http://localhost:8000/dashboard

📦 Artisan Commands

Command	Description
`composer dev`	Start all five processes: web, queue, logs, Vite, axiom watcher
`php artisan sentinel:stream --limit=100`	Simulate a transaction stream
`php artisan sentinel:watch`	Transaction worker (run alongside `sentinel:stream` for manual testing)
`php artisan sentinel:watch-axioms`	Axiom stream worker
`php artisan sentinel:ingest`	Index policy docs into vector KB (bumps policy epoch)
`php artisan sentinel:reset-metrics`	Reset dashboard counters
`php artisan sentinel:export-ground-truth --count=200 --output=ground-truth.json`	Export pre-AI labeled transactions (Arbiter-L8 offline eval ground truth)
`./vendor/bin/pest --filter=TestName`	Run a single test
`./vendor/bin/pint`	Run the Pint linter

🏗️ Architecture

🔀 Pipeline Diagram

%%{init: {'themeVariables': {'fontSize': '10px'}, 'flowchart': {'nodeSpacing': 15, 'rankSpacing': 25}}}%%
flowchart LR
    EH[EventHorizon]
    XY[Xylem-L6]
    AR[Arbiter-L8<br/>external eval harness]

    EH ~~~ AR

    subgraph Ingestion
        A[Events<br/>XADD]
        SL[synapse-l4]
    end

    subgraph SentinelL7["sentinel-l7"]
        subgraph Processing
            B[Worker Pool<br/>PHP]
        end
        subgraph Intelligence
            CACHE[Semantic Cache]
            LLM[Ollama LLM + RAG<br/>pluggable drivers]
            FALLBACK[Rule-based Fallback]
        end
        subgraph Interface
            E[React<br/>Dashboard]
        end
        subgraph Persistence
            D[(Neon<br/>Postgres)]
        end
    end

    subgraph "Downstream Consumers"
        LE[Ledger-L5]
        RL[Rhizome-Lens]
    end

    EH -->|Telemetry Events| SL
    XY -->|SaaS Activity| SL
    A -->|Redis Stream| B
    SL -->|Validated Events| B
    B -->|Evaluation Request| CACHE
    CACHE -->|Cache Miss| LLM
    LLM -->|Low Confidence| FALLBACK
    B -->|Persist Results| D
    D -->|Query| E
    D -->|Usage Events| LE
    SentinelL7 -->|OTel Traces/Logs| RL
    AR -->|HTTP POST /ingest| SL
    AR -->|MCP: analyze-transaction<br/>driver override| B
    AR -->|OTel Metrics/Traces| RL

    click EH "https://github.com/obrienma/EventHorizon#readme" "Go to EventHorizon repo"
    click XY "https://github.com/obrienma/Xylem-L6#readme" "Go to Xylem-L6 repo"
    click SL "https://github.com/obrienma/synapse-l4#readme" "Go to Synapse-L4 repo"
    click AR "https://github.com/obrienma/Arbiter-L8#readme" "Go to Arbiter-L8 repo"
    click LE "https://github.com/obrienma/Ledger-L5#readme" "Go to Ledger-L5 repo"
    click RL "https://github.com/obrienma/Rhizome-Lens#readme" "Go to Rhizome-Lens repo"

    classDef clickable fill:#1d4ed8,stroke:#1e40af,stroke-width:2px,color:#ffffff
    class EH,XY,SL,AR,LE,RL clickable

🗂️ Processing Layers

The system is composed of three long-running processes plus a shared intelligence and persistence layer.

Layer	Key Files	Purpose & Responsibilities
🌐 Web	`app/Http/Controllers/` · `resources/js/Pages/`	Dashboard & API: Inertia/React dashboard, compliance event pages, CSV export endpoint, HTTP rate-limited routes.
⚡ Transaction Worker	`app/Console/Commands/WatchTransactions.php` · `app/Services/TransactionProcessorService.php`	Stream Consumer: `XREADGROUP` on `transactions`; semantic cache check → optional AI analysis → `XACK`. `XAUTOCLAIM` recovery pass at top of every loop iteration.
🔷 Axiom Worker	`app/Console/Commands/WatchAxioms.php` · `app/Services/AxiomProcessorService.php`	Axiom Consumer: `XREADGROUP` on `synapse:axioms`; threshold routing (`anomaly_score > 0.8`) → AI audit narrative → Postgres. Every Axiom persisted — no silent drops.
🧠 AI Layer	`app/Contracts/ComplianceDriver.php` · `app/Services/ComplianceManager.php`	Driver Abstraction: Resolves `ollama` (default), `gemini`, or `openrouter` from env via Laravel Service Manager; domain logic only depends on the `ComplianceDriver` interface.
💾 Vector Layer	`app/Services/VectorCacheService.php` · `app/Services/EmbeddingService.php`	Semantic Cache + RAG: Upstash Vector `transactions` namespace (cache, ≥ 0.90) + `policies` namespace (RAG, ≥ 0.70, domain-filtered); fingerprint embedding via Ollama `nomic-embed-text` (768-dim) or Gemini `embedding-001` (1536-dim), swappable via `SENTINEL_EMBEDDING_DRIVER`.
🔌 MCP	`app/Mcp/Servers/SentinelServer.php` · `routes/ai.php`	Agent Protocol: Model Context Protocol endpoint at `POST /mcp`; exposes `analyze_transaction`, `search_policies`, and `get_recent_transactions` tools. Synchronous JSON-RPC over HTTP — no queue, no durability. Consumed by interactive AI agents (Claude Desktop, Cursor, VS Code Copilot) and by Arbiter-L8, an external eval harness. See 🔌 MCP Server below for details.

📐 Scale & Fault Tolerance

Note

Because both workers implement XAUTOCLAIM-based self-healing and the persistence layer uses idempotent writes (firstOrCreate + partial unique index on source_id), the worker pool can safely scale horizontally with zero risk of data duplication. Losing a worker doesn't stop recovery — any running sibling claims orphaned messages on its next loop iteration (ADR-0022). A delivery-count guard hard-ACKs poison messages at delivery_count >= 3 so a reliably crashing message cannot circulate indefinitely.

🧩 Laravel Patterns

Service Manager driver abstraction — ComplianceManager extends Laravel's Manager; swap AI providers via SENTINEL_AI_DRIVER env var, no code change required
Arch-test-enforced domain isolation — Pest architecture tests assert App\Services\Sentinel\Logic cannot import Http or Redis facades; enforced in tests/ArchTest.php
Policy epoch invalidation — cached compliance verdicts carry an MD5 of the policy corpus; mismatched epochs on cache hits trigger re-analysis so no verdict survives a policy update unexamined
Prompt versioning — all LLM templates live in prompts/ as versioned Markdown with changelogs and Used by: lists; the active ComplianceDriver loads the compiled .txt form at runtime; prompt drift is visible in git like code drift
Named rate limiters — RateLimiter::for() limiters on login, signup, and /dashboard/stream; all thresholds backed by RATE_LIMIT_* env vars via config/sentinel.php

🔌 MCP Server

Mcp::web('/mcp', SentinelServer::class) (routes/ai.php) registers a plain synchronous HTTP route — the same request/response model as any Laravel controller, not a queued job. A caller's request is handled inline, in-process, and the compliance verdict is returned in that same HTTP response. Full tool request/response shapes are documented in API.md.

Tools:

analyze_transaction — runs a transaction through the full pipeline (semantic cache → policy RAG → AI analysis, or Tier 3 fallback on infra failure)
search_policies — semantic search over the indexed policy knowledge base (ns:policies, threshold ≥ 0.70)
get_recent_transactions — reads the live Redis feed of recently processed transactions

analyze_transaction also accepts an optional driver (gemini/openrouter/ollama/vertexai) that forces a specific ComplianceManager provider instead of the configured default. Setting it bypasses the semantic vector cache entirely (no read, no write) and never falls back to Tier 3 on failure — a driver-override call always reflects that one provider's live verdict.

Note

Because the endpoint is a synchronous, non-durable RPC, a verdict only exists for the lifetime of the HTTP request that produced it — there's no queue or database row to recover it from if the caller drops the connection mid-call. This is a deliberate scope boundary, not an oversight: the durable, at-least-once pipeline (sentinel:watch on the transactions stream) is what production transactions go through. The MCP endpoint exists for callers that want an on-demand, synchronous verdict and can tolerate re-running a lost call — it was never meant to carry the same delivery guarantees as the stream pipeline.

Known consumers:

Arbiter-L8 — an external Python evaluation harness. Its adapters/sentinel_l7.py speaks MCP-over-HTTP directly to /mcp to score compliance verdicts against labeled ground truth (offline) and to run cross-provider disagreement checks via the driver override (online) — see the "Per-request ComplianceManager driver override" entry under Shipped Features.
Interactive AI agents — Claude Desktop, Cursor, VS Code Copilot — call the tools ad hoc during a chat/coding session.

Processing Loop

sequenceDiagram
    autonumber
    participant S as Redis Stream
    participant W as Sentinel Worker
    participant V as Upstash Vector
    participant G as Gemini AI

    S->>W: Fetch Transaction (XREADGROUP)

    note over W,V: 2a. Semantic Cache Check (Namespace: transactions)
    W->>V: Search Similar Results

    alt Pattern Similarity ≥ 0.90
        V-->>W: Return Cached Risk Report
        Note over W: Bypasses LLM (Fast Path)
    else Pattern New or Low Score
        note over W,V: 2b. Policy Retrieval (Namespace: policies)
        W->>V: Fetch Relevant Regulatory Rules
        V-->>W: Return AML/HIPAA Context
        W->>G: Analyze Intent + Policy Context
        G-->>W: Policy-Grounded Risk Analysis
        W->>V: Upsert New Vector + Metadata
    end

    W->>S: Acknowledge (XACK)

Axiom Ingestion (Synapse-L4 → Compliance Events)

sequenceDiagram
    autonumber
    participant SL4 as Synapse-L4
    participant AS as synapse:axioms
    participant AW as Axiom Worker
    participant V as Upstash Vector
    participant G as Gemini AI
    participant DB as Postgres

    SL4->>AS: XADD (source_id, anomaly_score, status)
    AS->>AW: XREADGROUP (axiom-workers)

    alt anomaly_score > 0.8
        note over AW,V: Policy Retrieval (Namespace: policies, ≥ 0.70)
        AW->>V: Fetch Regulatory Context
        V-->>AW: Policy Chunks
        AW->>G: Analyze Axiom + Policy Context
        G-->>AW: Audit Narrative + Risk Level
        AW->>DB: firstOrCreate compliance_events (routed_to_ai=true)
    else score ≤ 0.8
        AW->>DB: firstOrCreate compliance_events (routed_to_ai=false)
    end

    alt source_id not yet seen
        DB-->>AW: row created
    else re-delivery (same source_id)
        DB-->>AW: existing row returned — no duplicate
    end

    AW->>AS: XACK (remove from PEL)

    note over AW: Next loop iteration: XAUTOCLAIM
    note over AW: any message idle > 30s → reassign + reprocess

Message Lifecycle (Fault Tolerance)

stateDiagram-v2
    [*] --> New: XADD to Stream
    New --> Pending: Worker Reads (XREADGROUP)

    state Pending {
        [*] --> Processing
        Processing --> Success: XACK (Done)
        Processing --> Zombie: Worker Crashed
    }

    Zombie --> Processing: Sibling worker XAUTOCLAIM (min-idle > 30s)
    Success --> [*]

Service Layer

classDiagram
    direction TB

    class ComplianceDriver {
        <<interface>>
        +analyze(array data) array
        +analyzeTransaction(array data) array
    }

    class AbstractComplianceDriver {
        <<abstract>>
        #callModel(string prompt) string
        +analyze(array data) array
        +analyzeTransaction(array data) array
    }

    class OllamaDriver {
        #callModel(string prompt) string
    }

    class GeminiDriver {
        #callModel(string prompt) string
    }

    class OpenRouterDriver {
        #callModel(string prompt) string
    }

    class ComplianceManager {
        -Application app
        +driver(string name) ComplianceDriver
        #createOllamaDriver() ComplianceDriver
        #createGeminiDriver() ComplianceDriver
        #createOpenrouterDriver() ComplianceDriver
        +getDefaultDriver() string
    }

    class ComplianceEngine {
        -ComplianceDriver ai
        +__construct(ComplianceDriver ai)
        +process(array transaction) array
    }

    ComplianceDriver <|.. AbstractComplianceDriver : Realizes
    AbstractComplianceDriver <|-- OllamaDriver : Extends
    AbstractComplianceDriver <|-- GeminiDriver : Extends
    AbstractComplianceDriver <|-- OpenRouterDriver : Extends
    ComplianceManager ..> ComplianceDriver : Resolves
    ComplianceEngine o-- ComplianceDriver : Injected

Domain Logic Hierarchy

graph LR
    subgraph "Protected Core"
        Domain[App\Services\Sentinel\Logic]
    end

    subgraph "Infrastructure"
        Http[Laravel Http Facade]
        Redis[Redis Facade]
    end

    subgraph "Entry Points"
        Web[App\Http\Controllers]
        Console[App\Console\Commands]
    end

    Console --> Domain
    Web --> Domain
    Domain -.->|Forbidden| Http
    Domain -.->|Forbidden| Redis
    Domain -->|Allowed| Contract[ComplianceDriver Interface]

🔭 Observability

🖥️ Application UI

The application UI (/dashboard) is a server-driven React/Inertia SPA with four main surfaces:

Live transaction feed — real-time stream of processed transactions with risk level, cache-hit/miss indicator, and AI routing signal
Compliance Events — paginated audit trail of every persisted compliance_events row; toggle between flagged-only and all events; CSV export with optional date-range filter
Backpressure widget — consumer lag stat card reading sentinel:consumer_lag (10s TTL), colour-coded emerald/amber/red against the lag_warn/lag_pause thresholds; shows a dash when the worker is offline
Metrics counters — processed count, cache hit rate, flagged event count; reset with php artisan sentinel:reset-metrics

Note

Additional screenshots coming — the GIFs at the top of this README show the dashboard and terminal worker in action.

🔍 Overview

The Axiom Worker emits one axiom.process span per message — decorated with source_id, anomaly_score, domain, and routed_to_ai attributes — and continues the traceparent propagated from Synapse-L4, stitching the full cross-service trace. For deeper visibility, Sentinel exports to a companion Grafana monitoring stack (Tempo + Prometheus) that surfaces service health, processing latency p50/p95/p99, and AI routing signals that the HTTP response can't show.

The dashboard's AI Analysis by Driver and AI Confidence panels read ai.driver / ai.confidence attributes, which AxiomProcessorService::routeToAi() only sets when ComplianceDriver::analyze() succeeds. With a placeholder API key the call throws — the failure surfaces in the AI Errors panel and the two AI panels stay empty. To populate them:

Set a working credential for the active SENTINEL_AI_DRIVER: ollama (default) → reachable OLLAMA_URL + pulled OLLAMA_CHAT_MODEL, no API key; openrouter → OPENROUTER_API_KEY (+ OPENROUTER_MODEL); gemini → GEMINI_API_KEY (+ GEMINI_FLASH_URL).
Send Axioms with anomaly_score > AXIOM_AUDIT_THRESHOLD (default 0.8) so they route to AI — sub-threshold Axioms never emit axiom.ai_analysis attributes.
Run php artisan sentinel:watch-axioms with the OTel exporter pointed at the collector.

No dashboard change is needed once a driver call succeeds — the queries are already correct.

📊 Grafana Dashboard

Tip

The dashboard lives in rhizome-observability. All 9 panels are TraceQL-metrics queries over axiom.process / axiom.ai_analysis span attributes — no Prometheus counters required. Requires Tempo ≥ 2.7 with filter_server_spans: false (Sentinel spans are INTERNAL-kind).

📚 Docs

File	Contents	Last updated
README.md	Project overview	2026-07-17
ARCHITECTURE.md	System design, data flows, worker loop structure	—
SERVICES.md	Per-service reference	—
API.md	HTTP + MCP routes	—
AI_PIPELINE.md	Gemini driver, RAG pipeline, prompt versioning	—
TESTING.md	Test strategy, known gaps	—
USER_STORIES.md	Compliance officer, platform engineer, AI agent	—
DEV_GETTING_STARTED.md	Full local setup walkthrough	—
journal.md	Engineering journal — one entry per phase	—
adr/	Architecture Decision Records (ADR-0001 – ADR-0032)	—

🗺️ Roadmap

📋 Planned

🐛 Known issues

Semantic cache can permanently amplify a single wrong verdict for narrow-profile merchants. The Upstash Vector cache (similarity threshold 0.90) matches on embedding similarity, not transaction identity. A merchant profile whose transactions are narrow enough in amount range and message wording (e.g. the suspicious-category simulation profile) can embed near-identically across every transaction it generates — so if the first one is ever misanalyzed, every subsequent similar transaction inherits that one stale, wrong cached verdict indefinitely, rather than getting an independent re-analysis. Discovered during Arbiter-L8's Phase 3 step 8 live judge validation (worked around there via the per-request driver override, which bypasses the cache entirely — see Arbiter-L8's docs/journal/arbiter-l8-2026-07-04T1720-ground-truth-export-and-judge-validation.md). Not yet fixed here; no cache-invalidation or per-merchant TTL exists today.

📦 Production-Ready Baseline

Tip

22+ features shipped | Pest architecture + unit suite green

🔍 View shipped features...

🔁 Core Ingestion & Stream Reliability

Core pipeline — Redis Streams, semantic cache, fault tolerance (XCLAIM)
Backpressure step 2 — XREADGROUP + XAUTOCLAIM self-healing worker pool (ADR-0022): transaction stream migrated to consumer group sentinel-consumers; XAUTOCLAIM embedded at top of each worker loop; dead-letter guard ACKs poison messages at delivery_count >= 3; dedicated reclaimer daemon removed
Backpressure step 3 — graduated consumer lag signal (ADR-0023): worker writes XPENDING count to sentinel:consumer_lag (TTL 10s); producer applies soft-limit sleep (500ms, configurable) at lag > 50, spin-wait at lag > 200
Weighted transaction simulation — simulation.merchants config holds weighted profiles (category, weight, amount range, currencies, is_threat) instead of a flat uniform-probability list; TransactionStreamService::generate() samples via an index-repetition pool so traffic mix reflects realistic merchant volume
Benchmark seeder — database/seeders/TransactionSeeder.php runs N simulated transactions through the live pipeline and reports cache hit rate, fallbacks, embedding API call count, and threat rate

🧠 AI Compliance Engine

ComplianceDriver stack — OllamaDriver (qwen3.5:9b-q4_K_M, default — ADR-0027), GeminiDriver (Gemini Flash), OpenRouterDriver (OpenAI-compatible), VertexAIDriver (Claude Sonnet 4.6 via Vertex AI/Agent Platform, service account + IAM auth — ADR-0030), all sharing policy RAG/quality-scoring/response-parsing via AbstractComplianceDriver; swap via env, ComplianceManager (Laravel Service Manager pattern)
VertexAIDriver (ADR-0030, amended) — fourth compliance driver, SENTINEL_AI_DRIVER=vertexai, calls Claude Sonnet 4.6 via publishers/anthropic/models/claude-sonnet-4-6:rawPredict (Anthropic Messages API shape, not Gemini's generateContent); auth via google/auth's ServiceAccountCredentials (roles/aiplatform.user) minting a per-call OAuth2 bearer token, wrapped in a small VertexAiTokenService so VertexAIDriverTest can mock the token boundary instead of hitting Google's real OAuth2 endpoint; 'vertexai' added to AnalyzeTransaction's MCP tool DRIVERS allowlist per the ADR. No free tier — billed per-token at direct-API rates; every request explicitly sets thinking: disabled + effort: low since Sonnet 4.6's default (high-effort adaptive thinking) is expensive overkill for this driver's short JSON-classification workload
Ollama as default compliance-analysis driver (ADR-0027) — SENTINEL_AI_DRIVER defaults to ollama; verified live against the real host (raw JSON-mode call, and a full TransactionProcessorService cache-miss call producing a correctly-flagged critical-risk verdict with real policy citations). think: false avoids a ~20x latency penalty from qwen3.5's reasoning trace; Gemini/OpenRouter remain available via env override, no removal
Policy RAG — sentinel:ingest chunking pipeline, policies/ corpus, score-aware query formulation
Domain-scoped RAG retrieval — domain metadata tag at ingest; server-side filter at query time; retrieval quality logging
Output quality scoring — 4-signal rubric on every compliance driver response; low quality score warning when score ≤ 1
policy_refs fabrication guard (ADR-0034) — AbstractComplianceDriver forces policy_refs = [] whenever the retrieval that produced the response had chunk_count === 0, regardless of what the model returned; a policy_refs fabricated on empty retrieval warning logs the correction so the fabrication rate stays observable. Live sample data (49 cache_miss calls, openrouter) showed the model asserting non-empty policy_refs on 18/49 zero-chunk retrievals at avg. confidence 0.88. ADR-0019's has_policy_refs rubric signal now inherits the corrected value automatically. Scope: only the structured policy_refs field is guarded — a citation embedded in free-text narrative (the field actually persisted to compliance_events.audit_narrative) is untouched, and correspondence between a cited ref and an actually-retrieved chunk isn't checked when chunk_count > 0; both are open TODOs.
Retrieval coverage logging — mean_score and under_indexed per RAG query; Log::warning fires when a domain filter returns < 2 chunks
EmbeddingDriver stack (ADR-0025) — GeminiEmbeddingDriver, OllamaEmbeddingDriver (nomic-embed-text v1.5, 768-dim, task-prefixed search_document/search_query inputs), EmbeddingManager (Service Manager pattern), swap via SENTINEL_EMBEDDING_DRIVER; EmbeddingService now delegates to the resolved driver instead of calling Gemini directly. Live in this environment: Upstash Vector index recreated at 768-dim, policy KB re-ingested against Ollama.
Named Vector namespaces (ADR-0026) — VectorCacheService no longer has any bare/default-namespace methods; transaction semantic cache moved from Upstash's implicit default namespace to an explicit transactions namespace, matching policies. Sets the pattern for future namespaces (e.g. telemetry) and tenant-prefixed namespacing.
ADR-0007 Tier 2 drift closed — TransactionProcessorService now calls ComplianceDriver::analyzeTransaction() (Gemini/OpenRouter + policy RAG) on a cache miss instead of the rule-based ThreatAnalysisService; ThreatAnalysisService is reserved for Tier 3 (infra failure) as ADR-0007 originally specified. New transaction-compliance-analysis prompt added for the transaction-shaped query.
Per-request ComplianceManager driver override — TransactionProcessorService::process() and the analyze_transaction MCP tool accept an optional driver (gemini/openrouter/ollama/vertexai) that bypasses the semantic vector cache entirely (no read, no write) and never falls back to Tier 3 on failure, so the same transaction can be scored through two different providers for cross-provider disagreement measurement. Built for Arbiter-L8's online disagreement layer.

🔷 Axiom / Synapse-L4 Integration

Synapse-L4 Axiom ingestion — synapse:axioms Redis stream + sentinel:watch-axioms worker
Synapse-L4 Python sidecar — FastAPI LLM judge pass + Redis emitter
Idempotent Axiom persistence — firstOrCreate + partial unique index on source_id + UniqueConstraintViolationException catch; re-delivered stream messages never produce duplicate compliance_events rows
Early-exit idempotency in AxiomProcessorService — EXISTS check on source_id before AI routing; duplicate re-deliveries short-circuit before Gemini is called; DB-layer firstOrCreate remains as concurrent-race fallback
XCLAIM recovery for synapse:axioms consumer group — XAUTOCLAIM embedded in worker loop (ADR-0022)
Rule-based Tier 3 fallback for Axioms — AxiomThreatAnalysisService gives AxiomProcessorService an ADR-0007-style deterministic verdict (risk_level: high, threshold-referencing narrative) when Gemini/OpenRouter throws, instead of persisting risk_level: unknown / narrative: null; driver_used is stamped fallback so the degraded path is observable

💾 Persistence & Audit

compliance_events audit trail — Postgres persistence with source_id correlation
Transaction history — processed transactions persisted to Postgres transactions table
Compliance dashboard — Flags / Events nav pages surfacing compliance_events
Compliance report CSV export — GET /compliance/export streams flagged/all events chunked at 500 rows; optional from/to date filters; UI date-range picker on the Compliance page
Transaction-pipeline idempotency guard — partial unique index on transactions.txn_id (excludes driver_override, which intentionally writes multiple rows per transaction) plus an early-exit dedup check in TransactionProcessorService::process(), closing an XAUTOCLAIM-redelivery double-billing risk found while drafting ADR-0028; mirrors the Axiom pipeline's existing source_id dedup pattern
Billing classification (ADR-0028, Accepted) — defines which transactions.source/compliance_events.driver_used rows Ledger-L5's usage-pull query should treat as billable vs. cache-savings vs. excluded; no sentinel-l7 instrumentation beyond the idempotency guard above, which was a prerequisite for the classification to be safe to rely on
GET /usage endpoint (ADR-0029) — dual per-pipeline cursor pull (since_transactions/since_compliance_events, both auto-increment id) with a config-backed page size and safety-lag window; X-Ledger-Api-Key-gated (VerifyLedgerApiKey middleware), HTTPS-enforced outside local/testing, 401 on missing/invalid key

👁️ Frontend & Operations

React 19 + shadcn/ui dashboard with live transaction feed
Backpressure dashboard widget — consumer lag stat card reads sentinel:consumer_lag (10s TTL); colour-coded emerald/amber/red against lag_warn/lag_pause config thresholds; dash when worker is offline
HTTP rate limiting — named RateLimiter::for() limiters on login (5/min per IP), signup (10/hr per IP), /dashboard/stream (20/min per authenticated user); all thresholds config-backed via RATE_LIMIT_* env vars

🔭 Observability

MCP server — exposes analyze_transaction, search_policies, get_recent_transactions tools to AI agents via Model Context Protocol at POST /mcp
OTel instrumentation (Phase 2) — OtelServiceProvider bootstraps SDK (BatchSpanProcessor → OTLP HTTP); AxiomProcessorService wraps processing in wide spans with source_id, anomaly_score, domain, routed_to_ai attributes; traceparent extracted from stream entries to continue Synapse-L4 trace as child span (ADR-0024)
Grafana dashboard — "Sentinel-L7 Service" — 9 panels, TraceQL-metrics queries over axiom.process / axiom.ai_analysis spans (no Prometheus counters required); throughput by risk level/domain/AI routing, latency p50/p95/p99, anomaly-score and AI-confidence aggregates

Name		Name	Last commit message	Last commit date
.claude		.claude
.github		.github
.observability		.observability
app		app
bootstrap		bootstrap
config		config
database		database
docs		docs
policies		policies
prompts		prompts
public		public
resources		resources
routes		routes
storage		storage
tests		tests
.dockerignore		.dockerignore
.editorconfig		.editorconfig
.env.example		.env.example
.gitattributes		.gitattributes
.gitignore		.gitignore
.nvmrc		.nvmrc
CLAUDE.md		CLAUDE.md
Dockerfile		Dockerfile
Procfile		Procfile
README.md		README.md
artisan		artisan
components.json		components.json
composer.json		composer.json
composer.lock		composer.lock
jsconfig.json		jsconfig.json
package-lock.json		package-lock.json
package.json		package.json
phpunit.xml		phpunit.xml
render.yaml		render.yaml
system.md		system.md
vite.config.js		vite.config.js

Navigation Menu

Repository files navigation

📋 Contents

🧰 Stack

🚀 Running the Project

✅ Prerequisites

⚡ Quick Start

📦 Artisan Commands

🏗️ Architecture

🔀 Pipeline Diagram

🗂️ Processing Layers

📐 Scale & Fault Tolerance

🧩 Laravel Patterns

🔌 MCP Server

Processing Loop

Axiom Ingestion (Synapse-L4 → Compliance Events)

Message Lifecycle (Fault Tolerance)

Service Layer

Domain Logic Hierarchy

🔭 Observability

🖥️ Application UI

🔍 Overview

📊 Grafana Dashboard

📚 Docs

🗺️ Roadmap

📋 Planned

🐛 Known issues

📦 Production-Ready Baseline

🔁 Core Ingestion & Stream Reliability

🧠 AI Compliance Engine

🔷 Axiom / Synapse-L4 Integration

💾 Persistence & Audit

👁️ Frontend & Operations

🔭 Observability

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

📋 Contents

🧰 Stack

🚀 Running the Project

✅ Prerequisites

⚡ Quick Start

📦 Artisan Commands

🏗️ Architecture

🔀 Pipeline Diagram

🗂️ Processing Layers

📐 Scale & Fault Tolerance

🧩 Laravel Patterns

🔌 MCP Server

Processing Loop

Axiom Ingestion (Synapse-L4 → Compliance Events)

Message Lifecycle (Fault Tolerance)

Service Layer

Domain Logic Hierarchy

🔭 Observability

🖥️ Application UI

🔍 Overview

📊 Grafana Dashboard

📚 Docs

🗺️ Roadmap

📋 Planned

🐛 Known issues

📦 Production-Ready Baseline

🔁 Core Ingestion & Stream Reliability

🧠 AI Compliance Engine

🔷 Axiom / Synapse-L4 Integration

💾 Persistence & Audit

👁️ Frontend & Operations

🔭 Observability

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages