Classical ML and LSTM models for forecasting the NASDAQ-100 close price, benchmarked on both a one-day-ahead and a 7-day recursive multi-step horizon, with an evaluation of hand-engineered technical indicators (RSI, ATR, Bollinger Band width, Momentum, MACD).
This project builds a full time-series forecasting pipeline for the NASDAQ-100 index — leak-free feature engineering, train-only scaling, sliding-window sequence construction — and benchmarks four models against each other:
- Linear Regression, Random Forest, and XGBoost on flattened 30-day windows
- A PyTorch LSTM (64 hidden units) trained on the same windows
Every model is evaluated both as a single-step next-day predictor and as a 7-day recursive forecaster, and the classical models are re-run on an enhanced feature set that adds five technical indicators to see whether they actually help.
| Model | RMSE | MAE | MAPE | R² |
|---|---|---|---|---|
| Linear Regression | 0.0756 | 0.0561 | 0.0228 | 0.9877 |
| Random Forest | 1.7875 | 1.6523 | 0.6062 | -5.8726 |
| XGBoost | 1.8274 | 1.6954 | 0.6239 | -6.1824 |
| LSTM | 0.5117 | 0.3776 | 0.1216 | 0.4313 |
Random Forest and XGBoost score negative R² — worse than predicting the historical mean. This isn't a bug in the models; it's a scaling artifact explained in the report. Tree ensembles can't extrapolate beyond the value range seen in training, and the NASDAQ-100 index trended well above that range by the test period.
| Model | RMSE (Base) | R² (Base) | RMSE (Enhanced) | R² (Enhanced) |
|---|---|---|---|---|
| Linear Regression | 0.0756 | 0.9877 | 0.1828 | 0.9282 |
| Random Forest | 1.7875 | -5.8726 | 1.7726 | -5.7583 |
| XGBoost | 1.8444 | -6.3168 | 1.8499 | -6.3602 |
Adding RSI / ATR / Bollinger width / Momentum / MACD only helped Random Forest, and only marginally — the indicators are highly collinear with features already in the base engineered set, and the extra dimensions hurt Linear Regression's generalization.
| Model | Mean RMSE (7-day) | Mean MAE | Runtime |
|---|---|---|---|
| Linear Regression | 0.0772 | 0.0587 | 0.53s |
| Random Forest | 1.7905 | 1.6555 | 101.89s |
| XGBoost | 1.8750 | 1.7467 | 0.95s |
| LSTM | 0.5331 | 0.3989 | 14.70s |
.
├── notebook/
│ └── nasdaq100_forecasting.ipynb # full pipeline: data prep → features → models → evaluation
├── images/ # output plots referenced in this README and the report
├── NasdaqCast_Technical_Report.pdf # full write-up with methodology, results, and analysis
├── requirements.txt
└── README.md
- Data preparation — load daily NASDAQ-100 OHLCV, set a sorted datetime index, fill any gaps.
- Chronological split — 70% train / 15% validation / 15% test, no shuffling.
- Feature engineering —
Return,Range, moving averages, rolling volatility, lagged closes, log-returns, plus (for the enhanced set) RSI, ATR, Bollinger Band width, Momentum, and MACD — all computed causally per split to avoid leakage. - Normalization —
MinMaxScalerfit exclusively on the training partition. - Windowing — 30-day input windows, 7-day output horizon, built independently per split.
- Modeling — Linear Regression / Random Forest / XGBoost on flattened windows, and an LSTM (64 units → dropout 0.2 → dense) trained with early stopping.
- Multi-step forecasting — recursive one-step rollout to 7 days ahead for every model.
- Evaluation — RMSE, MAE, MAPE, R² at every horizon step, with loss curves and actual-vs-predicted plots.
git clone https://github.com/<your-username>/nasdaqcast.git
cd nasdaqcast
pip install -r requirements.txt
jupyter notebook notebook/nasdaq100_forecasting.ipynbThe notebook expects a NASDAQ100_data.csv file (daily OHLCV, with Price/Date, Close, High,
Low, Open, Volume columns) in the working directory.
The full technical report — methodology, all four result tables, plots, and a discussion of why the
tree ensembles underperform and what would fix it — is in
NasdaqCast_Technical_Report.pdf.
- Linear Regression is the strongest baseline on this scaled, level-based target — it extrapolates linearly, while Random Forest and XGBoost saturate and fail badly once test-period prices move outside the training range.
- Technical indicators are not a free win. They only helped the model (Random Forest) that's least sensitive to collinearity, and hurt the model (Linear Regression) most sensitive to it.
- Recursive multi-step error compounds but stays bounded over a 7-day horizon for the models that actually track the series — LSTM RMSE grows ~7% from day 1 to day 7, Linear Regression roughly doubles in absolute terms but from a very small base.
- Re-run the LSTM on the enhanced (20-feature) input set.
- Refit or roll the scaler over time (or forecast returns instead of price levels) to fix the tree-model extrapolation failure.
- Add a direct multi-horizon model and a sequence-to-sequence LSTM decoder as alternatives to the recursive strategy.
- Hyperparameter tuning for Random Forest / XGBoost beyond near-default settings.
MIT — see LICENSE.