golm is a language model that lives entirely in the universe of Go moves. There is no natural language, no board representation, no hand-crafted features — only a sequence of move tokens (B[pd], W[dp], …). The model learns Go purely by predicting the next move in a stream of professional and AI games, the same way a language model learns English by predicting the next word.
Built on nanoGPT. Includes a GTP-compatible engine for play in Sabaki, GoGui, or any GTP-aware interface.
This project explores two related questions:
-
Can a decoder-only LLM develop an implicit world model of Go purely from next-token prediction on move sequences? The hypothesis is that a model trained only on move histories must internally reconstruct board state — including captures, liberty counts, and game phase — to predict the next move well. No explicit board representation is provided; any spatial or tactical understanding must emerge from the sequence alone.
-
How far can small architectural and data changes take a mid-sized model in terms of playing strength? The aim is to find the highest practical playing strength achievable through targeted improvements to encoding, attention structure, and training data — without adding search, MCTS, or hand-crafted Go knowledge. Reproducing AlphaGo or AlphaZero is explicitly not the goal; the interest is in what the pure language modeling paradigm can achieve on its own.
Current experiments include structured move embeddings (decomposing each token into color + row + col components) and learned per-head spatial attention biases based on board-coordinate distances.
- ~194M parameter transformer (20 layers, 14 heads, 896 embedding dim)
- Vocabulary of 731 tokens:
B[xx]/W[xx]for all 19×19 intersections, pass, game-result, and special tokens — nothing else - Structured embedding: each move token decomposed into color + row + col sub-embeddings
- Spatial attention bias: per-head learned bias based on (Δrow, Δcol, color-pair) between query and key positions
- Trained on ~1.6M professional and KataGo AI games (≈3.8B tokens)
uv sync
uv sync --group dev # for testsSources:
- KataGo rating games: https://katagoarchive.org
- Fox 9-dan / professional: https://github.com/featurecat/go-dataset
Download KataGo games:
python scripts/download_katago.pyPrepare training data (streaming, no large memory allocation):
python data/sgf/prepare.pypython train.py config/train_sgf.pyResume from checkpoint by setting init_from = "resume" in config/train_sgf.py.
Generate opening sequences from a trained checkpoint:
python scripts/demo_opening.py --games 3 --moves 30
python scripts/demo_opening.py --greedy --moves 50 --games 1Run as a GTP engine (connects to Sabaki / GoGui):
python gtp_engine.pyOptions:
| Flag | Default | Description |
|---|---|---|
--greedy |
off | Greedy decoding instead of sampling |
--temp |
0.8 | Sampling temperature |
--top-k |
3 | Top-k candidates |
--top-p |
0.9 | Nucleus sampling threshold |
--resign-ratio |
0.54 | Resign when opponent holds this fraction of total points |
--device |
cuda | cuda or cpu |
The engine supports 19×19 only. It resigns when clearly losing, passes to end the game when appropriate, and avoids filling opponent territory in the endgame.
uv run pytest