TradingSimulator

A modular platform to practice day trading in a realistic, low-latency environment using live or paper market data. It provides reproducible simulation, risk controls, and rich observability, enabling you to design, test, and refine trading strategies end-to-end.

---

🎯 Objectives & Scope

• Real-time market data ingestion (Alpaca WebSocket: trades/quotes/bars).
• Deterministic paper execution simulator (fills, slippage, partials).
• Risk engine with portfolio/risk limits.
• Strategy interfaces: manual UI + programmatic SDK.
• Replay of historical topics for backtests & drills.
• Dashboards & alerts in Grafana (latency, PnL, risk, data quality).

---

🏗️ High-Level Architecture

          Alpaca Market Data (WS/REST)
                    │
             [Ingestor Service]
                    │  (normalized events)
                 Kafka Topics
        ┌───────────┼────────────┐
 [Strategy Apps]   [Simulator]   [ETL/Storage]
        │                │              │
  Orders/Signals  Executions/PnL   TimescaleDB/Parquet
        │                │              │
      Redis <——— State/positions ——→ Grafana (dashboards/alerts)
                    │
              Risk Engine (sync path)

---

📂 Project Structure

trading-simulator/
├─ services/
│  ├─ ingestor/            # Alpaca → Kafka
│  ├─ simulator/           # Paper execution
│  ├─ risk-engine/         # Order validation
│  ├─ strategy-sdk/        # Programmatic strategy SDK
│  └─ ui-manual/           # Optional Streamlit/Next.js manual UI
│
├─ libs/
│  ├─ python/trading_common/
│  │  ├─ schemas/          # Avro/Proto schemas
│  │  ├─ kafka.py          # Kafka helpers
│  │  ├─ metrics.py        # Prometheus helpers
│  │  └─ time.py
│  └─ go/                  # Future Go libs
│
├─ infra/
│  ├─ docker-compose.yaml  # Full stack (Redpanda, Redis, Timescale, Prometheus, Grafana)
│  ├─ kafka/               # Topic config + create_topics.sh
│  ├─ prometheus/          # prometheus.yml
│  └─ grafana/             # provisioning for datasources + dashboards
│
├─ config/
│  ├─ .env.example
│  ├─ ingestor.env.example
│  ├─ simulator.env.example
│  ├─ risk-engine.env.example
│  └─ symbols.yaml
│
├─ scripts/
│  ├─ dev_up.sh
│  ├─ dev_down.sh
│  ├─ seed_timescaledb.sql
│  └─ wait_for_port.sh
│
├─ tests/
│  └─ e2e/                 # End-to-end tests across services
│
├─ .github/workflows/      # CI pipelines
├─ .pre-commit-config.yaml
├─ Makefile
├─ README.md               # ← this file
└─ technical_doc.md        # Extended architecture & plan

---

⚡ Tech Stack

• Streaming: Redpanda (Kafka-compatible), Schema Registry.
• State/cache: Redis.
• DB: TimescaleDB (hypertables for trades, positions, PnL).
• Metrics/Tracing: Prometheus + OpenTelemetry → Grafana.
• Runtime: Python (Ingestor, Simulator), optional Go for high-throughput services.
• Packaging/Deploy: Docker Compose (dev), Helm/Kubernetes (future).

---

🚦 Current Phase

✅ Phase 0 — Environment setup complete:

• Redpanda + Console up, topics provisioned.
• TimescaleDB seeded with hypertables.
• Grafana + Prometheus online with pre-provisioned datasources/dashboards.

➡️ Next: Phase 1 — implement Ingestor (Alpaca → Kafka).

---

📊 Grafana Dashboards

Provisioned automatically at startup:

• Infra Health — verifies Redpanda, Redis, Prometheus health.
• Trading Pipeline (placeholders) — panels for ingest lag, simulator latency, order throughput, PnL (will light up as services emit metrics).

---

🛠️ Development Workflow

# bring up infra stack
make up

# watch logs
make logs

# seed TimescaleDB tables
make seed

# run tests
make test

# stop stack
make down

---

📈 Execution Plan (Phased)

• Phase 1: Ingestor service → Kafka topics.
• Phase 2: Simulator service → fills, PnL, positions.
• Phase 3: Risk engine (limits, sync validation).
• Phase 4: Strategy SDK + manual trading UI.
• Phase 5: Observability dashboards + alerts.
• Phase 6: Replay/backtesting service.
• Phase 7: Kubernetes deployment + CI/CD.
• Phase 8: Extensions (options, futures, RL ghost trader).

---

📚 References

• Technical Design Doc — detailed architecture, schemas, and phased roadmap.