Xiangqi-R1: Enhancing Spatial Strategic Reasoning in LLMs for Chinese Chess via Reinforcement Learning

This repository contains the implementation of Xiangqi-R1: Enhancing Spatial Strategic Reasoning in LLMs for Chinese Chess via Reinforcement Learning

---

📦 Install Environment

pip install -r requirements.txt

---

🚀 Quick Start

1. 🧠 Train the Model (SFT + RL)

Step 1: Supervised Fine-Tuning Stage 1 (Basic Move Generation)

cd code/sft
python sft_stage1.py

Step 2: Supervised Fine-Tuning Stage 2 (Strategic Analysis)

cd code/sft  
python sft_stage2.py

Step 3: Reinforcement Learning Fine-Tuning

cd code/rl
python r1.py

---

2. 📁 Project Structure

xiangqi/
├── cchess/                  # Core Chinese chess library
│   ├── board.py            # Board representation and game logic
│   ├── game.py             # Game state management
│   ├── piece.py            # Chess piece implementations
│   ├── move.py             # Move validation and notation
│   └── engine.py           # Chess engine integration
├── code/
│   ├── sft/
│   │   ├── sft_stage1.py   # Basic move generation training
│   │   ├── sft_stage2.py   # Strategic analysis training
│   │   └── ds_config.json  # DeepSpeed configuration
│   ├── rl/
│   │   ├── r1.py           # Reinforcement learning training
│   │   ├── rewards.py      # Reward function definitions
│   │   └── ds_config.json  # DeepSpeed configuration
│   └── evaluate/
│       ├── eval_model.py   # Model evaluation scripts
│       └── online_model_eval.py
├── data_process/
│   ├── xiangqi_crawler.py  # Web scraping for game data
│   ├── data_init.py        # Data preprocessing
│   └── filter/             # Data filtering utilities  
├── pikafish/               # Chess engine binaries
│   ├── pikafish-*          # Engine variants for different CPUs
│   └── pikafish.nnue       # Neural network evaluation file
└── data/
    └── test_data.json      # Test datasets

---

🙏 Acknowledgments

This project uses the Chinese chess library from cchess with minor modifications. We thank the original authors for their excellent work.