Architecture

Architecture

System overview

                          +------------------+
                          |   Dashboard      |
                          |   (React + TS)   |
                          |   29,600 LOC     |
                          +--------+---------+
                                   |
                              HTTP/SSE
                                   |
+-------------+           +--------+---------+          +-----------------+
|  CLI        |---------->|   FastAPI API    |--------->|  Queue          |
|  (27 cmds)  |   REST    |   84 endpoints  |  arq/    |  (arq / asyncio)|
+-------------+           |   5,700 LOC     |  async   +-----------------+
                          +--------+---------+                  |
                                   |                            v
                          +--------+---------+          +-----------------+
                          |  DAG Engine      |          |  Scheduler      |
                          |  (dag.py 1,200L) |          |  (APScheduler)  |
                          +--------+---------+          +-----------------+
                                   |
                          +--------+---------+
                          |  Executor        |
                          |  (4,750 LOC)     |
                          |  15 step types   |
                          +--------+---------+
                                   |
              +--------------------+--------------------+
              |                    |                    |
     +--------v-------+  +--------v-------+  +--------v-------+
     |  Sandshore      |  |  Tools         |  |  Memory        |
     |  Runtime        |  |  Registry      |  |  System        |
     |  (sandbox pool) |  |  56 connectors |  |  (Mem0 + Neo4j)|
     +--------+--------+  +----------------+  +----------------+
              |
    +---------+---------+---------+---------+
    |         |         |         |         |
  +--+--+  +--+--+  +--+--+  +--+--+
  | E2B |  |Docker|  |Local|  | CF  |
  |Cloud|  |Seccomp| |  Dev |  |Edge |
  +-----+  +------+  +-----+  +-----+

Module structure

src/sandcastle/
  __init__.py              # Package init, version
  __main__.py              # CLI entry point (3,046 lines, 27 commands)
  main.py                  # FastAPI app factory, lifespan
  config.py                # Settings (pydantic-settings, env vars)
  sdk.py                   # Python SDK for programmatic use (1,226 lines)

  api/
    routes.py              # 84 API endpoints (5,698 lines)
    schemas.py             # Pydantic response models (789 lines)
    auth.py                # HMAC-SHA256 auth, admin guard
    a2a.py                 # Google Agent-to-Agent protocol
    agui.py                # CopilotKit AG-UI SSE streaming
    security_headers.py    # CSP, X-Frame-Options, etc.
    rate_limit.py          # Sliding window, Redis or in-memory

  engine/
    dag.py                 # DAG builder and validator (1,211 lines)
    executor.py            # Step execution engine (4,754 lines)
    sandshore.py           # Sandbox runtime pool (701 lines)
    backends.py            # E2B, Docker, Local, CF backends (813 lines)
    providers.py           # Model registry + failover (237 lines)
    memory.py              # Agent memory v2 (727 lines)
    optimizer.py           # Cost optimization (522 lines)
    autopilot.py           # A/B testing for workflows (508 lines)
    policy.py              # Guardrails and policy engine (469 lines)
    generator.py           # NL-to-workflow generation (718 lines)
    eval.py                # Evaluation framework (528 lines)
    pdf.py                 # PDF report generation (1,257 lines)
    storage.py             # S3/MinIO artifact storage (147 lines)
    events.py              # Event bus for SSE (90 lines)
    crypto.py              # Fernet credential encryption
    license.py             # Ed25519 license verification
    tools/
      registry.py          # Tool registry + credential management (3,168 lines)
      connectors/          # 56 .mjs connector scripts (6,800+ lines)

  models/
    db.py                  # SQLAlchemy async models (runs, keys, versions, etc.)

  queue/
    worker.py              # arq worker / in-process fallback
    scheduler.py           # APScheduler for cron workflows

  templates/
    __init__.py            # Template parser
    *.yaml                 # 118 built-in workflow templates

  webhooks/
    dispatcher.py          # Webhook delivery with retry

dashboard/                 # React 18 + TypeScript + Vite + Tailwind v4
  src/
    pages/                 # 20 full pages
    components/            # Shared UI components
    api/                   # API client + mock mode
    hooks/                 # Custom React hooks
    lib/                   # Utilities, template packs

Data flow: workflow execution

1. Input - User provides workflow (YAML or natural language) + input data
2. Parse - DAG builder validates YAML, resolves dependencies, builds execution graph
3. Schedule - Steps with met dependencies are queued for execution
4. Execute - Each step runs in a sandbox (E2B/Docker/Local/CF):
   • Sandbox boots with the appropriate runner (Claude or OpenAI-compatible)
   • Tools are mounted based on step configuration
   • Agent executes with the step prompt + context from previous steps
   • Output is captured and stored
5. Fan-out - Steps with depends_on resolved trigger downstream steps
6. Stream - Real-time SSE events pushed to dashboard and CLI
7. Store - Run data, outputs, artifacts saved to DB + S3
8. Complete - Final output aggregated, webhooks dispatched, PDF available

Database schema

SQLAlchemy async with SQLite (local) or PostgreSQL (production):

• Run - Workflow execution instances with status, timing, cost
• StepResult - Per-step outputs, token counts, duration
• WorkflowVersion - Git-like versioning (draft/production/archived)
• ApiKey - HMAC-hashed keys with expiry, CIDR allowlists, rotation
• Schedule - Cron triggers linked to workflows
• Approval - Human-in-the-loop approval queue
• Memory - Agent memory entries with embeddings
• ToolConnection - Encrypted credentials for integrations
• AutoPilotExperiment - A/B test configurations and results
• Setting - Persistent key-value config

Technology stack

Layer	Technology
Language (backend)	Python 3.12+
Framework	FastAPI + Uvicorn
Database	SQLAlchemy 2.0 async (SQLite / PostgreSQL)
Queue	arq (Redis) / in-process asyncio
Scheduler	APScheduler
Dashboard	React 18 + TypeScript + Vite
Styling	Tailwind CSS v4
Charts	Recharts
Workflow viz	@xyflow/react
Build	hatchling (Python), Vite (JS)
Package	PyPI: sandcastle-ai
AI	Anthropic Claude, OpenAI, Google Gemini, MiniMax
Sandbox	E2B SDK, aiodocker, Cloudflare Workers
Memory	Mem0 + optional Neo4j
Encryption	cryptography (Fernet, Ed25519)