Complete event-driven backtest engine v1

.github/workflows/tests.yml

@@ -0,0 +1,14 @@
+name: tests
+on: [push, pull_request]
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+      - run: pip install -e ".[dev]"
+      - run: python -m unittest discover -s tests -v
+      - run: ruff check engine tests run.py

.gitignore

@@ -1,3 +1,11 @@
+.venv/
 __pycache__/
-*.pyc
+*.py[cod]
+.pytest_cache/
+.ruff_cache/
+*.egg-info/
+build/
+dist/
+data/*.csv
 data/*.png
+!data/.gitkeep

README.md

@@ -1,37 +1,63 @@
-# backtest-engine
+# Backtest Engine
 
-An **event-driven backtesting engine, built from scratch** in Python — no
-backtesting libraries doing the work. It mirrors the decoupled architecture real
-trading systems use: components communicate only through events on a queue.
+A compact event-driven backtesting engine built from scratch in Python. Its main research rule is explicit: **signals are generated at `close[t]`, filled at `open[t+1]`, and marked at `close[t+1]`**. That prevents the same-close look-ahead error common in first backtests.
 
-## Architecture
+## What version 1 includes
 
-    DataHandler --MarketEvent--> Strategy --SignalEvent--> Portfolio
-        ^                                                      |
-        |                                                  OrderEvent
-        |                                                      v
-        +------------- FillEvent <--- ExecutionHandler <-------+
+- Injected pandas DataFrames for deterministic, network-free tests
+- Target-weight sizing, cash reservation, and order rejection
+- Directional slippage and per-share/minimum commissions
+- Order, fill, trade, position, cash, rejection, and equity ledgers
+- A cost-matched buy-and-hold benchmark
+- Transaction-cost sensitivity at 0, 1, 5, and 10 bps
+- Tests for timing, cash, multiple symbols, costs, exits, missing bars, and final marking
 
-Each component is independent and swappable:
+## Event flow
 
-| Module | Responsibility |
-|--------|----------------|
-| `engine/data.py` | Streams historical bars, emits a `MarketEvent` per day |
-| `engine/strategy.py` | Turns market data into `SignalEvent`s (example: MA crossover) |
-| `engine/portfolio.py` | Tracks cash/positions/equity; sizes orders; books fills |
-| `engine/execution.py` | Simulated broker with slippage + commission |
-| `engine/backtest.py` | The event loop that drives everything |
-| `engine/metrics.py` | Sharpe, drawdown, returns from the equity curve |
+```text
+bar opens -> pending orders fill -> bar closes -> strategy signals
+     ^                                                |
+     |                                                v
+next bar <--- order waits in broker <--- portfolio target weights
+```
 
-## Run
+## Install and test
 
-    pip install -r requirements.txt
-    python run.py            # runs a 20/50 MA crossover on AAPL, saves the equity curve
-    python tests/test_portfolio.py   # unit tests for portfolio accounting
+```bash
+python -m venv .venv
+source .venv/bin/activate        # Windows PowerShell: .venv\Scripts\Activate.ps1
+pip install -e ".[dev]"
+python -m unittest discover -s tests -v
+```
 
-## Extending it
+Run the SPY example (this step downloads market data):
 
-Write a new strategy by subclassing `Strategy` and implementing
-`calculate_signals(event)` — the rest of the engine is unchanged. That decoupling
-is the whole point: the same engine can drive any strategy, and each component can
-be tested in isolation.
+```bash
+python run.py
+```
+
+Outputs are saved under `data/`, including every ledger, the cost-sensitivity table, and an equity chart.
+
+## Minimal network-free use
+
+```python
+import pandas as pd
+from engine.backtest import Backtest
+from engine.strategy import MovingAverageCross
+
+bars = pd.DataFrame(
+    {"Open": [100, 101, 102], "Close": [101, 102, 103]},
+    index=pd.date_range("2024-01-01", periods=3),
+)
+results = Backtest(
+    ["TEST"],
+    price_data={"TEST": bars},
+    strategy_cls=MovingAverageCross,
+    strategy_kwargs={"short": 1, "long": 2},
+).run()
+print(results["fills"])
+```
+
+## Scope and limitations
+
+This is an educational daily-bar simulator, not a production trading system. It does not model partial market liquidity, limit orders, corporate-action edge cases, borrow, taxes, or intraday queue position. See [methodology](docs/METHODOLOGY.md), [design](docs/DESIGN.md), and [results](docs/RESULTS.md).

data/.gitkeep

@@ -0,0 +1 @@
+

docs/DESIGN.md

@@ -0,0 +1,21 @@
+# Design
+
+The engine separates market data, strategy logic, portfolio accounting, and execution through typed queue events. This makes timing observable and lets each component be replaced independently.
+
+## Bar lifecycle
+
+For each timestamp, the data handler publishes a market event. The broker first fills orders that were pending from an earlier timestamp at the current open. The strategy then observes the current close and may publish target weights. The portfolio converts changed targets to orders, which remain pending until another market event. Finally, the portfolio marks cash and positions at the current close.
+
+An order submitted on the final bar is cancelled with `end_of_data`; it is never silently filled at the same close.
+
+## Accounting invariants
+
+- Long-only positions cannot fall below zero.
+- New buy orders reserve estimated cash.
+- Gaps beyond the reserve buffer cause a smaller fill or rejection, never negative cash.
+- Equity equals cash plus each position valued at the current close.
+- Every order ends as filled, partially filled, rejected, or cancelled.
+
+## Intentionally excluded
+
+Version 1 does not add more strategies. Its goal is a reliable engine boundary, auditable ledgers, and honest evaluation—not a strategy catalogue.

docs/METHODOLOGY.md

@@ -0,0 +1,17 @@
+# Methodology
+
+## Timing convention
+
+A strategy may use information through `close[t]`. Its order becomes eligible at `open[t+1]`. Portfolio equity is marked at `close[t+1]`. The test suite asserts the signal timestamp is earlier than the fill timestamp.
+
+## Costs
+
+Market orders receive adverse slippage: buys pay above the open and sells receive below it. Commissions are the greater of a minimum charge and a per-share charge. `run.py` repeats the experiment at 0, 1, 5, and 10 basis points.
+
+## Comparison
+
+The example compares the one included moving-average rule with buy-and-hold. Both start with identical capital and use the same entry commission/slippage assumptions. This is a baseline, not evidence of a profitable strategy.
+
+## Research discipline
+
+For strategy research beyond the example, split data chronologically into training, validation, and untouched test periods. Choose rules only with training/validation, report the untouched result once, and examine sensitivity across parameters, costs, and market regimes. Walk-forward evaluation should retrain only on information available at each historical date.

docs/RESULTS.md

@@ -0,0 +1,14 @@
+# Results
+
+`python run.py` downloads SPY daily bars from 2015 through 2024, runs a 20/50-day moving-average rule, compares it with cost-matched buy-and-hold, and writes reproducible outputs to `data/`.
+
+Expected files:
+
+- `equity_curve.png`
+- `cost_sensitivity.csv`
+- `orders.csv`, `fills.csv`, `trades.csv`
+- `positions.csv`, `cash.csv`, `equity.csv`, `rejections.csv`
+
+Numerical results are deliberately generated rather than hard-coded because the upstream adjusted dataset can change. Interpret the study as an engine demonstration. A simple moving-average result can depend heavily on the chosen period, parameters, market regime, and assumed costs; it is not investment advice or a claim of deployable alpha.
+
+The most important version-one result is structural: automated tests verify next-bar execution, event order, cost direction, cash constraints, multi-symbol reservations, exits, common-calendar handling, final marking, and end-of-data cancellation.

engine/backtest.py

@@ -1,35 +1,82 @@
-"""Orchestrates the event loop that drives the whole simulation."""
+"""Event-loop orchestration with explicit close-signal/next-open execution."""
+
+from __future__ import annotations
+
 import queue
+
+import pandas as pd
+
 from .data import DataHandler
-from .portfolio import Portfolio
 from .execution import SimulatedExecutionHandler
+from .portfolio import Portfolio
 
 
 class Backtest:
-    def __init__(self, symbols, start, initial_capital, strategy_cls, end=None, **strat_kwargs):
+    def __init__(
+        self,
+        symbols,
+        start=None,
+        initial_capital=100_000.0,
+        strategy_cls=None,
+        end=None,
+        price_data=None,
+        commission_per_share=0.005,
+        minimum_commission=1.0,
+        slippage_bps=1.0,
+        strategy_kwargs=None,
+        **legacy_strategy_kwargs,
+    ):
+        if strategy_cls is None:
+            raise ValueError("strategy_cls is required")
         self.events = queue.Queue()
-        self.data = DataHandler(self.events, symbols, start, end)
-        self.strategy = strategy_cls(self.data, self.events, **strat_kwargs)
+        self.data = DataHandler(self.events, symbols, start, end, price_data=price_data)
+        kwargs = dict(strategy_kwargs or {})
+        kwargs.update(legacy_strategy_kwargs)
+        self.strategy = strategy_cls(self.data, self.events, **kwargs)
         self.portfolio = Portfolio(self.data, self.events, initial_capital)
-        self.execution = SimulatedExecutionHandler(self.data, self.events)
+        self.execution = SimulatedExecutionHandler(
+            self.data,
+            self.events,
+            commission_per_share=commission_per_share,
+            minimum_commission=minimum_commission,
+            slippage_bps=slippage_bps,
+        )
 
     def run(self):
         while self.data.continue_backtest:
             self.data.update_bars()
             if not self.data.continue_backtest:
                 break
+            market_event = None
             while True:
                 try:
-                    ev = self.events.get(False)
+                    event = self.events.get_nowait()
                 except queue.Empty:
                     break
-                if ev.type == "MARKET":
-                    self.strategy.calculate_signals(ev)
-                    self.portfolio.update_timeindex(ev)
-                elif ev.type == "SIGNAL":
-                    self.portfolio.update_signal(ev)
-                elif ev.type == "ORDER":
-                    self.execution.execute_order(ev)
-                elif ev.type == "FILL":
-                    self.portfolio.update_fill(ev)
-        return self.portfolio.equity_curve
+                if event.type == "MARKET":
+                    market_event = event
+                    self.execution.on_market(event)
+                    self.strategy.calculate_signals(event)
+                elif event.type == "SIGNAL":
+                    self.portfolio.update_signal(event)
+                elif event.type == "ORDER":
+                    self.execution.execute_order(event)
+                elif event.type == "FILL":
+                    self.portfolio.update_fill(event)
+            if market_event is not None:
+                self.portfolio.mark_to_market(market_event)
+
+        for order in self.execution.cancel_all():
+            self.portfolio.cancel_order(order)
+        return self.results()
+
+    def results(self):
+        return {
+            "equity": pd.DataFrame(self.portfolio.equity_curve),
+            "orders": pd.DataFrame(self.portfolio.orders),
+            "fills": pd.DataFrame(self.portfolio.fills),
+            "trades": pd.DataFrame(self.portfolio.trades),
+            "positions": pd.DataFrame(self.portfolio.positions_ledger),
+            "cash": pd.DataFrame(self.portfolio.cash_ledger),
+            "rejections": pd.DataFrame(self.portfolio.rejections),
+        }