🧠 TinyTimeTransformer

A lightweight transformer model for univariate time series forecasting, built using PyTorch.
This project demonstrates how compact attention-based models can predict future values in sequential data—here, applied to cryptocurrency closing prices.

---

📈 Example Output

---

🔧 Features

• Tiny transformer adapted from GPT-style attention
• Predicts future values from historical price sequences
• Fully Dockerized for easy training & evaluation
• Compares favorably to LSTM and linear models
• Code structured for reuse and extension

---

📁 Project Structure

TinyTimeTransformer/
├── data/
│   ├── raw/                ← Original CSV (e.g. Bitcoin)
│   ├── processed/          ← Numpy-ready training data
│   └── process_data.py     ← Preprocessing script
├── models/
│   └── tiny_transformer.py ← Model definition
├── train.py                ← Training script
├── evaluate.py             ← Evaluation + plotting
├── Dockerfile              ← Docker support
├── requirements.txt        ← Python dependencies
└── README.md

---

🚀 Getting Started

1. Preprocess Data

python data/process_data.py

This will generate:

• X_train.npy, Y_train.npy
• X_val.npy, Y_val.npy
• stats.txt (for denormalization if needed)

2. Train Model

python train.py

3. Evaluate Predictions

python evaluate.py

---

🐳 Run with Docker

Build

docker build -t tiny-time-transformer .

Evaluate

docker run --rm -v $(pwd):/app tiny-time-transformer

Train (optional)

docker run --rm -v $(pwd):/app tiny-time-transformer python train.py

---

📊 Dataset

Uses a public cryptocurrency price dataset with the following structure:

Cryptocurrency Price History (Kaggle)

Date, Open, High, Low, Close, Volume, Marketcap

Preprocessing focuses on the Close column using a sliding window approach.

---

🔍 Future Ideas

• Multi-feature transformer (OHLCV)
• Add baseline LSTM comparison
• Extend to real-world financial/eeg/IoT data
• Use Mamba for long-sequence efficiency

---

📜 License

MIT License

---

🤝 Contact

Hossein Dahaei
GitHub • LinkedIn • dahaeehossein@gmail.com