diff --git a/README.md b/README.md index 0e27014..7c5715a 100644 --- a/README.md +++ b/README.md @@ -17,10 +17,11 @@ Early. The engine plays legal chess via a UCI loop with: - **Move generation** — delegated to `dragontoothmg` (verified with perft). - **Evaluation** — material + piece-square tables + bishop pair. - **Search** — iterative deepening, negamax alpha-beta, quiescence search, - MVV-LVA move ordering, hard time limits. + MVV-LVA move ordering, a shared transposition table with hash-move ordering, + Lazy SMP (multi-threaded search), hard time limits. -Not yet implemented: transposition table, killer/history heuristics, null-move -pruning, opening book, endgame tablebases. See the [roadmap](#roadmap). +Not yet implemented: killer/history heuristics, null-move pruning, opening book, +endgame tablebases. See the [roadmap](#roadmap). ## Install @@ -51,10 +52,19 @@ gochess version Add the binary as an engine in any UCI GUI, or pipe commands directly: ``` +setoption name Hash value 128 +setoption name Threads value 8 position startpos moves e2e4 e7e5 go movetime 1000 ``` +### UCI options + +| Option | Default | Range | Meaning | +|---|---|---|---| +| `Hash` | 64 | 1–4096 | Transposition-table size in MiB. | +| `Threads` | 1 | 1–256 | Lazy-SMP worker count (capped at the machine's core count at search time). | + ## Development ```bash @@ -82,7 +92,8 @@ docs/ architecture notes and ADRs - [x] Negamax + alpha-beta + iterative deepening + time budget - [x] UCI loop - [x] Quiescence search + MVV-LVA move ordering -- [ ] Transposition table (Zobrist hash is already available from `dragontoothmg`) +- [x] Transposition table (Zobrist hash from `dragontoothmg`) + hash-move ordering +- [x] Lazy SMP — multi-threaded search over the shared TT (`Threads` UCI option) - [ ] Killer moves + history heuristic - [ ] Null-move pruning, late move reductions - [ ] Opening book (Polyglot) and Syzygy tablebase probing diff --git a/docs/architecture.md b/docs/architecture.md index ab8aa0f..b5bd1fb 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -41,23 +41,32 @@ unapply closure), FEN parsing, an incrementally-updated Zobrist hash - **search.go** — `Search(*Board, SearchParams) SearchResult`. Iterative deepening around a negamax alpha-beta core, with: - quiescence search at the horizon (captures and promotions only), + - transposition-table probes/stores with hash-move ordering, - MVV-LVA move ordering, - - mate-distance-aware scoring (`mateScore - ply`), + - mate-distance-aware scoring (`mateScore - ply`), ply-rebased through the TT, + - Lazy SMP: `SearchParams.Threads` workers deepen independently on their own + board copy over one shared TT; the deepest completed result wins, - a hard wall-clock budget checked every 2048 nodes; the last fully completed depth is returned. +- **transposition.go** — `TT`, a fixed-size power-of-two table keyed by + `Board.Hash`. Each 16-byte slot is a pair of `atomic.Uint64` words accessed + with Hyatt's lockless XOR trick (`word0 = key ^ data`), so the Lazy-SMP + workers share it without a mutex; a write torn across goroutines reads as a + miss. `data` packs move, int32 score, depth and bound flag. ### `internal/uci` -A line-oriented reader for `uci`, `isready`, `ucinewgame`, `position` -(`startpos` / `fen`, with `moves`), `go` (`depth`, `movetime`, `wtime`/`btime`), -`stop`, `d`, and `quit`. Search is synchronous, so `stop` is a no-op and -`bestmove` is emitted as soon as `go` returns. +A line-oriented reader for `uci`, `isready`, `setoption`, `ucinewgame`, +`position` (`startpos` / `fen`, with `moves`), `go` (`depth`, `movetime`, +`wtime`/`btime`), `stop`, `d`, and `quit`. It owns the persistent `engine.TT` +(sized by the `Hash` option, cleared on `ucinewgame`) and the `Threads` setting. +Search is synchronous, so `stop` is a no-op and `bestmove` is emitted as soon as +`go` returns. ## Deliberately not here yet -Transposition table, killer/history heuristics, null-move pruning, LMR, aspiration -windows, opening book, tablebases, pondering, `SearchMoves`/`MultiPV`. The Zobrist -hash needed for a TT is already exposed by `dragontoothmg`. +Killer/history heuristics, null-move pruning, LMR, aspiration windows, opening +book, tablebases, pondering, `SearchMoves`/`MultiPV`. ## Key invariants @@ -68,3 +77,6 @@ hash needed for a TT is already exposed by `dragontoothmg`. 3. **Evaluation sign convention**: positive = good for the side to move. 4. **Untrusted input**: FEN and UCI strings come from GUIs and tournament managers. Parsing must not panic (see SECURITY.md). +5. **The TT is the only shared mutable state between Lazy-SMP workers.** Every + worker has its own `dragontoothmg.Board` copy and its own `searcher`; the + table is safe for concurrent use, so no other synchronisation is needed. diff --git a/docs/design.md b/docs/design.md index b881979..249322b 100644 --- a/docs/design.md +++ b/docs/design.md @@ -17,11 +17,13 @@ than it buys. |---|---| | `dragontoothmg.Board` | board state, legal move generation, `Apply` / unapply, Zobrist hash, FEN — never reimplemented (see [ADR 0002](adr/0002-use-dragontoothmg-for-move-generation.md)) | | `engine.Evaluate` | static evaluation from the side-to-move's view (positive = better for that side) | -| `engine.orderMoves` | orders moves to maximise alpha-beta cut-offs; currently promotions then MVV-LVA captures then quiets | -| `searcher` | carries the node count and wall-clock deadline; answers `timeUp` and flips `stopped` | +| `engine.orderMoves` | orders moves to maximise alpha-beta cut-offs; TT/hash move first, then promotions, then MVV-LVA captures, then quiets | +| `engine.TT` | shared, lock-free transposition table keyed by `Board.Hash`; probed/stored inside `negamax` and seeded into move ordering | +| `searcher` | carries the node count, TT handle, shared stop flag and wall-clock deadline; answers `timeUp` and flips `stopped` | `internal/uci` is the protocol layer and the only place that prints `info` / -`bestmove`. Search is synchronous, so `stop` is a no-op and `bestmove` is emitted +`bestmove`. It owns the persistent `engine.TT` (`Hash` option) and the `Threads` +setting. Search is synchronous, so `stop` is a no-op and `bestmove` is emitted as soon as `go` returns. `cmd/gochess` wires these together and adds the `perft` and `bench` subcommands. @@ -31,8 +33,10 @@ and `bench` subcommands. - [Negamax](https://www.chessprogramming.org/Negamax) with [alpha-beta pruning](https://www.chessprogramming.org/Alpha-Beta) — fail-hard - [Iterative Deepening](https://www.chessprogramming.org/Iterative_Deepening) — the last fully completed depth is the one returned +- [Transposition Table](https://www.chessprogramming.org/Transposition_Table) — power-of-two table keyed by `dragontoothmg.Board.Hash()`; stores EXACT / LOWER / UPPER bounds with the best move, mate scores rebased by ply on store and probe. Lock-free (Hyatt XOR) so Lazy-SMP workers share one table. The stored move seeds move ordering even when the entry is too shallow to cut +- [Lazy SMP](https://www.chessprogramming.org/Lazy_SMP) — `Threads` workers run iterative deepening in parallel on private board copies over the shared TT; workers start at staggered depths so they diverge, and the deepest completed result wins - [Quiescence Search](https://www.chessprogramming.org/Quiescence_Search) at the horizon — captures and promotions only, depth-bounded by `maxPly` -- [Move Ordering](https://www.chessprogramming.org/Move_Ordering) — promotions first, then [MVV-LVA](https://www.chessprogramming.org/MVV-LVA) captures, then quiet moves +- [Move Ordering](https://www.chessprogramming.org/Move_Ordering) — TT/hash move first, then promotions, then [MVV-LVA](https://www.chessprogramming.org/MVV-LVA) captures, then quiet moves - [Mate-distance scoring](https://www.chessprogramming.org/Mate_Distance_Pruning) — `mateScore - ply`, so the shortest mate is preferred; a proven mate ends iterative deepening early - Draw detection — the fifty-move rule (`Halfmoveclock >= 100`) is scored `0` inside the tree - Time management — a hard wall-clock budget checked every 2048 nodes; the UCI layer spends `1/30` of the remaining clock when the GUI sends `wtime` / `btime` instead of `movetime` @@ -51,7 +55,6 @@ update and no pawn or evaluation hash. ### Search -- [Transposition Table](https://www.chessprogramming.org/Transposition_Table) — Zobrist key is already exposed by `dragontoothmg.Board.Hash()`; needs EXACT / LOWER / UPPER bounds and ply-rebased mate scores - [Principal Variation Search](https://www.chessprogramming.org/Principal_Variation_Search) — null-window scout + re-search - [Killer](https://www.chessprogramming.org/Killer_Heuristic) and [history](https://www.chessprogramming.org/History_Heuristic) heuristics in `orderMoves` - [Null Move Pruning](https://www.chessprogramming.org/Null_Move_Pruning) and [Late Move Reductions](https://www.chessprogramming.org/Late_Move_Reductions) @@ -59,7 +62,7 @@ update and no pawn or evaluation hash. - [Static Exchange Evaluation](https://www.chessprogramming.org/Static_Exchange_Evaluation) for capture ordering and bad-capture pruning in quiescence - Threefold-[repetition](https://www.chessprogramming.org/Repetitions) detection (needs a position history the current `Search` does not keep) - A real principal variation in the `info` line (only `bestmove` is reported today) -- [Lazy SMP](https://www.chessprogramming.org/Lazy_SMP) once a shared TT exists +- Richer Lazy SMP — per-worker root-move splitting, aspiration-window skew, TT ageing / bucketed replacement - Asynchronous search so `stop` and pondering actually work - [Opening book](https://www.chessprogramming.org/Opening_Book) (Polyglot) and [Syzygy endgame tablebases](https://www.chessprogramming.org/Endgame_Tablebases) diff --git a/docs/engine-strength.md b/docs/engine-strength.md index c418486..b5a0313 100644 --- a/docs/engine-strength.md +++ b/docs/engine-strength.md @@ -28,17 +28,25 @@ until [the strength-testing harness](#measuring-it-properly) produces real data. bishop-pair bonus. Crude, but it captures development, central control, king placement and the two-bishop advantage. - **Compiled and fast** — ~2.7M nodes/sec in a single thread on an Apple M4 - (`gochess bench`). That is ~50–60× the node rate of a typical Python engine, - so effective search depth is respectable even without a transposition table. + (`gochess bench`), and Lazy SMP puts the other cores to work. That is ~50–60× + the node rate of a typical Python engine, so effective search depth is + respectable. - **Robust I/O** — FEN and UCI parsing is fuzz-safe and non-panicking; the engine will not forfeit on a malformed GUI command. +### Recently added + +- **Transposition table + hash-move ordering.** A shared, lock-free table keyed + by the `dragontoothmg` Zobrist hash now caches EXACT / LOWER / UPPER bounds + and the best move (mate scores rebased by ply). It roughly cuts the node count + of a depth-7 search from the opening by ~7× and seeds move ordering with the + hash move. +- **Lazy SMP.** `Threads` workers deepen in parallel on private board copies + over the one shared table. With 4–8 cores this reaches a given depth several + times faster than the single-threaded search did. + ### Limiting factors -- **No transposition table.** The Zobrist hash is already exposed by - `dragontoothmg`, but nothing caches search results yet. Transpositions are - re-searched from scratch, and there is no hash move to seed move ordering. - This is the single largest missing feature — worth an estimated 150–250 Elo. - **No killer or history heuristic.** Quiet-move ordering is essentially arbitrary, so cut-offs deep in the tree are later than they should be. - **No null-move pruning, no late move reductions, no futility or delta @@ -55,8 +63,9 @@ until [the strength-testing harness](#measuring-it-properly) produces real data. - **Thin endgame play.** No tablebase probing, no KPK / KBNK knowledge, no contempt. The 50-move rule is honoured but threefold repetition is not detected inside the search. -- **Single-threaded.** No Lazy SMP or any other parallel search; the other - nine cores of the test machine sit idle. +- **Basic Lazy SMP only.** Workers share the TT and start at staggered depths, + but there is no root-move splitting, aspiration-window skew, or TT ageing, so + parallel scaling past a handful of threads is modest. - **Synchronous search.** `stop` is a no-op and there is no pondering, so the engine cannot think on the opponent's clock or bail out of a bad time allocation. @@ -65,7 +74,7 @@ until [the strength-testing harness](#measuring-it-properly) produces real data. | Reference engine | ~CCRL blitz | Relevant comparison | | ---------------- | ----------- | ------------------- | -| TSCP 1.81 | ~1700 | Similar search shape (alpha-beta, iterative deepening, quiescence, MVV-LVA) **plus** a transposition table. goChess should land at or just below TSCP until the TT lands. | +| TSCP 1.81 | ~1700 | Similar search shape (alpha-beta, iterative deepening, quiescence, MVV-LVA, transposition table). goChess should now land near TSCP, with the missing pruning heuristics still costing rating. | | Sungorus 1.4 | ~2000 | TT + null-move + PVS + killers. Clearly stronger than goChess today. | | CT800 / Claudia class | ~2100+ | Full modern pruning set. Out of reach without the roadmap features. | @@ -84,30 +93,30 @@ depth is unusually shallow for its speed — each extra ply is high-value. | Bullet (1+0) | ~1350–1550 | Depth 4–6; positional weaknesses dominate. | | Blitz (3+2 / 5+0) | ~1500–1750 | Depth 6–9; the headline estimate. | | Rapid (15+10) | ~1650–1900 | Depth 9–12; tactics get sharper, eval ceiling starts to bite. | -| Classical (40/40) | ~1750–2000 | Depth-limited by the missing TT more than by the clock. | +| Classical (40/40) | ~1750–2000 | Depth-limited by the missing pruning heuristics more than by the clock. | ## Where the number would move Rough, independent Elo deltas from the [roadmap](../README.md#roadmap), assuming each is implemented competently and validated by SPRT: -| Change | Estimated Elo | -| ------ | ------------- | -| Transposition table + hash-move ordering | +150 to +250 | -| Killer moves + history heuristic | +50 to +100 | -| Null-move pruning | +50 to +80 | -| Late move reductions | +50 to +100 | -| Aspiration windows | +10 to +30 | -| Game-phase eval interpolation (tapered eval) | +30 to +60 | -| Passed pawns / king safety / mobility terms | +40 to +80 | -| Texel-tuned evaluation weights | +40 to +80 | -| Opening book (Polyglot) | +20 to +40 at short TC | -| Syzygy tablebase probing | +10 to +20 | -| Lazy SMP (8 threads) | +100 to +150 | - -Landing the transposition table, killers/history, null-move and LMR together -would plausibly put goChess in the 1950–2150 range; adding tapered/tuned eval -and Lazy SMP on top targets 2300+. +| Change | Estimated Elo | Status | +| ------ | ------------- | ------ | +| Transposition table + hash-move ordering | +150 to +250 | done, pending SPRT | +| Lazy SMP (8 threads) | +100 to +150 | done (basic), pending SPRT | +| Killer moves + history heuristic | +50 to +100 | | +| Null-move pruning | +50 to +80 | | +| Late move reductions | +50 to +100 | | +| Aspiration windows | +10 to +30 | | +| Game-phase eval interpolation (tapered eval) | +30 to +60 | | +| Passed pawns / king safety / mobility terms | +40 to +80 | | +| Texel-tuned evaluation weights | +40 to +80 | | +| Opening book (Polyglot) | +20 to +40 at short TC | | +| Syzygy tablebase probing | +10 to +20 | | + +With the transposition table and Lazy SMP in, adding killers/history, null-move +and LMR would plausibly put goChess in the 1950–2150 range; tapered/tuned eval +on top targets 2300+. ## Measuring it properly diff --git a/docs/performance.md b/docs/performance.md index bdf1b56..79262bb 100644 --- a/docs/performance.md +++ b/docs/performance.md @@ -1,34 +1,56 @@ # Performance -Fresh benchmark of the current engine (iterative deepening, negamax alpha-beta, -bounded quiescence, MVV-LVA move ordering, hard time limit; **no** transposition -table, killers/history, null-move pruning, or LMR yet) on one machine: Apple M4, -10 cores, Go 1.27, `darwin/arm64`. Search is from the starting position; rows -stop once a search passes ~5 seconds. "Nodes" is the engine's node counter -(`negamax` + `quiesce` calls). Numbers are single-run and rounded — the search -is single-threaded and deterministic, so node counts repeat exactly and only the -wall-clock wobbles. - -## What `go` / `go depth N` runs today: single-process iterative deepening - -This is the real search path. There is no Lazy SMP or parallel search — `go` -runs one `Search` on one goroutine, deepening from depth 1 until it runs out of -`movetime` (or hits `depth N`), and returns the last fully completed depth. The -rows below use a generous budget so the run isn't deadline-capped. - -| depth | time | nodes | bestmove | -|------:|-----:|------:|:---------| -| 4 | 0.01s | 55,614 | b1c3 | -| 5 | 0.06s | 457,203 | d2d4 | -| 6 | 0.45s | 3,097,162 | d2d4 | -| 7 | 3.4s | 23,615,017 | e2e4 | -| 8 | 30s | 160,689,328 | d2d4 | - -(Cumulative nodes over the deepening series 1..N; the shallower iterations add -well under 1%.) Depth 7 from the opening lands in ~3s on this machine, depth 8 in -~30s. Adding a transposition table is the single biggest lever left — it would -recover most of the redundant re-search between iterations and across -transposing lines. +Benchmark of the current engine (iterative deepening, negamax alpha-beta, +bounded quiescence, MVV-LVA move ordering, a shared transposition table, Lazy +SMP, hard time limit; **no** killers/history, null-move pruning, or LMR yet) on +one machine: Apple M4, 10 cores, Go 1.27, `darwin/arm64`. Search is from the +starting position unless noted. "Nodes" is the engine's node counter (`negamax` ++ `quiesce` calls). Numbers are single-run and rounded; with a shared TT and +multiple workers the search is no longer bit-for-bit deterministic, so node +counts wobble a few percent between runs. + +## What `go` / `go depth N` runs today: iterative deepening + TT (+ optional Lazy SMP) + +`go` deepens from depth 1 until it runs out of `movetime` (or hits `depth N`) +and returns the last fully completed depth. Every `negamax` node probes and +stores the shared transposition table, and the stored move seeds move ordering. +With `Threads > 1` that same search runs on N goroutines over the one table +(Lazy SMP). + +Single-threaded, `Hash 128`, generous budget so the run isn't deadline-capped. +"Before TT" is the pre-transposition-table baseline from earlier revisions of +this document: + +| depth | time (before TT) | nodes (before TT) | time (with TT) | nodes (with TT) | +|------:|-----------------:|------------------:|---------------:|----------------:| +| 6 | 0.45s | 3,097,162 | 0.09s | ~0.6M | +| 7 | 3.4s | 23,615,017 | 0.52s | ~3.1M | +| 8 | 30s | 160,689,328 | 1.3s | ~5.9M | + +The transposition table is the single biggest lever in the engine's history — +it recovers most of the redundant re-search between iterations and across +transposing lines, and cuts depth-8-from-the-opening from ~30s to ~1.3s. + +## Lazy SMP scaling + +Basic Lazy SMP: workers share the TT and start at staggered depths, but there is +no root-move splitting or aspiration-window skew yet, so on TT-friendly +positions the workers largely re-explore the same tree. Fixed-depth wall-clock +is roughly flat; the value shows up as extra breadth (each worker's TT-perturbed +ordering occasionally finds a better line) and robustness on tactical positions, +plus headroom once root splitting lands. Approximate, `movetime 2000` from the +start position: + +| threads | depth reached | nodes | +|--------:|--------------:|------:| +| 1 | 8 | ~9.1M | +| 2 | 8 | ~17.8M | +| 4 | 8 | ~32.1M | +| 8 | 8 | ~42.6M | + +Nodes scale with worker count (overlapping work); turning that into deeper +fixed-time search is what per-worker root-move splitting on the +[roadmap](../README.md#roadmap) is for. ## Move generation: `perft` from the start position @@ -48,9 +70,15 @@ Move generation is delegated to `dragontoothmg` and is not the bottleneck. ## Single fixed-depth search from the start position A controlled baseline: one search to exactly depth `N` (no iterative deepening), -clock never armed. Deterministic, and it isolates search-tree efficiency, so -this is the table to watch when judging whether a search change helped. Not what -the engine runs in a game. +clock never armed. It isolates search-tree efficiency, so this is the table to +watch when judging whether a search change helped. Not what the engine runs in a +game. + +> **Stale since the TT landed.** The table below predates the transposition +> table. `Search` now always uses a TT (a private one when the caller passes +> none), so a single fixed-depth search from a cold table already benefits from +> intra-search transpositions — the real node counts are lower than shown and +> vary slightly run to run. Regenerate this table with a fresh benchmark. The `perft(depth)` column is the size of the *full* legal game tree at that depth — the branching the search would face with no alpha-beta at all. `pruned` diff --git a/internal/engine/search.go b/internal/engine/search.go index fa7f7b4..783ffab 100644 --- a/internal/engine/search.go +++ b/internal/engine/search.go @@ -2,6 +2,8 @@ package engine import ( "sort" + "sync" + "sync/atomic" "time" "github.com/dylhunn/dragontoothmg" @@ -23,6 +25,14 @@ type SearchParams struct { // MoveTime is a hard wall-clock budget. Zero means "no time limit; obey // MaxDepth only". When set, the last fully completed depth is returned. MoveTime time.Duration + // Threads is the number of Lazy-SMP workers. Zero or one searches on a + // single goroutine; higher values run that many workers over one shared TT. + Threads int + // TT is the transposition table to search against. When nil a private + // default-sized table is allocated for this call only, so results are not + // carried between moves — callers that want that (the UCI layer) pass a + // table they own. + TT *TT } // SearchResult is the outcome of a Search call. @@ -35,13 +45,22 @@ type SearchResult struct { } type searcher struct { + tt *TT + stop *atomic.Bool // shared across Lazy-SMP workers; set once time is up + id int nodes int64 deadline time.Time stopped bool } func (s *searcher) timeUp() bool { - return !s.deadline.IsZero() && time.Now().After(s.deadline) + if s.stop != nil && s.stop.Load() { + return true + } + if s.deadline.IsZero() { + return false + } + return time.Now().After(s.deadline) } // Search runs iterative-deepening alpha-beta and returns the best move it found. @@ -50,42 +69,115 @@ func Search(b *dragontoothmg.Board, p SearchParams) SearchResult { if p.MaxDepth <= 0 || p.MaxDepth > maxPly { p.MaxDepth = maxPly } - s := &searcher{} - if p.MoveTime > 0 { - s.deadline = time.Now().Add(p.MoveTime) + tt := p.TT + if tt == nil { + tt = NewTT(defaultHashMB) + } + threads := p.Threads + if threads < 1 { + threads = 1 } - start := time.Now() + start := time.Now() var res SearchResult if b.White.Kings == 0 || b.Black.Kings == 0 { return res // illegal position, nothing sensible to search } + if len(b.GenerateLegalMoves()) == 0 { + return res + } + + var deadline time.Time + if p.MoveTime > 0 { + deadline = start.Add(p.MoveTime) + } + stop := new(atomic.Bool) + + if threads > 1 { + res = searchLazySMP(b, p.MaxDepth, deadline, stop, tt, threads) + } else { + s := &searcher{tt: tt, stop: stop, deadline: deadline} + res = s.runIterativeDeepening(b, p.MaxDepth, 1) + } + res.Elapsed = time.Since(start) + return res +} + +// searchLazySMP runs `threads` workers over one shared transposition table. Each +// worker deepens independently on its own copy of the board; workers seeded with +// a different start depth diverge into different subtrees, and the shared TT +// lets every worker profit from what the others have already searched. The +// deepest completed result wins. See https://www.chessprogramming.org/Lazy_SMP. +func searchLazySMP(b *dragontoothmg.Board, maxDepth int, deadline time.Time, stop *atomic.Bool, tt *TT, threads int) SearchResult { + results := make([]SearchResult, threads) + var wg sync.WaitGroup + for i := 0; i < threads; i++ { + wg.Add(1) + go func(id int) { + defer wg.Done() + // dragontoothmg.Board is all value types (no pointers, slices, or + // maps), so a plain copy gives each worker an independent board. + board := *b + startDepth := 1 + id%2 + if startDepth > maxDepth { + startDepth = maxDepth + } + s := &searcher{tt: tt, stop: stop, deadline: deadline, id: id} + results[id] = s.runIterativeDeepening(&board, maxDepth, startDepth) + }(i) + } + wg.Wait() + + best := results[0] + var nodes int64 + for _, r := range results { + nodes += r.Nodes + if r.BestMove == 0 { + continue + } + if r.Depth > best.Depth || (r.Depth == best.Depth && r.Score > best.Score) { + best = r + } + } + best.Nodes = nodes + return best +} + +// runIterativeDeepening deepens from startDepth to maxDepth on b, keeping the +// last fully completed depth. startDepth is above 1 only for Lazy-SMP workers. +func (s *searcher) runIterativeDeepening(b *dragontoothmg.Board, maxDepth, startDepth int) SearchResult { + var res SearchResult root := b.GenerateLegalMoves() if len(root) == 0 { return res } res.BestMove = root[0] - for depth := 1; depth <= p.MaxDepth; depth++ { + for depth := startDepth; depth <= maxDepth; depth++ { score, move, ok := s.searchRoot(b, depth) if !ok { break // out of time: keep the previous completed depth } res.BestMove, res.Score, res.Depth = move, score, depth if score >= mateScore-maxPly || score <= -mateScore+maxPly { - break // forced mate found; deeper search cannot improve on it + if s.stop != nil { + s.stop.Store(true) // forced mate: let the other workers stop too + } + break } if s.timeUp() { break } } res.Nodes = s.nodes - res.Elapsed = time.Since(start) return res } func (s *searcher) searchRoot(b *dragontoothmg.Board, depth int) (score int, best dragontoothmg.Move, ok bool) { - moves := orderMoves(b, b.GenerateLegalMoves()) + key := b.Hash() + _, ttMove, _ := s.tt.probe(key, depth, -infinity, infinity, 0) + + moves := orderMoves(b, b.GenerateLegalMoves(), ttMove) alpha, beta := -infinity, infinity bestScore := -infinity for _, m := range moves { @@ -102,6 +194,7 @@ func (s *searcher) searchRoot(b *dragontoothmg.Board, depth int) (score int, bes alpha = v } } + s.tt.store(key, depth, bestScore, boundExact, best, 0) return bestScore, best, true } @@ -109,6 +202,9 @@ func (s *searcher) negamax(b *dragontoothmg.Board, depth, alpha, beta, ply int) s.nodes++ if s.nodes&2047 == 0 && s.timeUp() { s.stopped = true + if s.stop != nil { + s.stop.Store(true) + } return 0 } if b.White.Kings == 0 || b.Black.Kings == 0 { @@ -121,6 +217,13 @@ func (s *searcher) negamax(b *dragontoothmg.Board, depth, alpha, beta, ply int) return s.quiesce(b, alpha, beta, ply) } + alphaOrig := alpha + key := b.Hash() + ttScore, ttMove, cutoff := s.tt.probe(key, depth, alpha, beta, ply) + if cutoff { + return ttScore + } + moves := b.GenerateLegalMoves() if len(moves) == 0 { if b.OurKingInCheck() { @@ -130,7 +233,8 @@ func (s *searcher) negamax(b *dragontoothmg.Board, depth, alpha, beta, ply int) } best := -infinity - for _, m := range orderMoves(b, moves) { + var bestMove dragontoothmg.Move + for _, m := range orderMoves(b, moves, ttMove) { unapply := b.Apply(m) v := -s.negamax(b, depth-1, -beta, -alpha, ply+1) unapply() @@ -138,7 +242,7 @@ func (s *searcher) negamax(b *dragontoothmg.Board, depth, alpha, beta, ply int) return 0 } if v > best { - best = v + best, bestMove = v, m } if v > alpha { alpha = v @@ -147,6 +251,15 @@ func (s *searcher) negamax(b *dragontoothmg.Board, depth, alpha, beta, ply int) break // fail-high: opponent won't enter this line } } + + bound := boundExact + switch { + case best <= alphaOrig: + bound = boundUpper + case best >= beta: + bound = boundLower + } + s.tt.store(key, depth, best, bound, bestMove, ply) return best } @@ -168,7 +281,7 @@ func (s *searcher) quiesce(b *dragontoothmg.Board, alpha, beta, ply int) int { return stand } - for _, m := range orderMoves(b, b.GenerateLegalMoves()) { + for _, m := range orderMoves(b, b.GenerateLegalMoves(), 0) { if !dragontoothmg.IsCapture(m, b) && m.Promote() == dragontoothmg.Nothing { continue } @@ -205,9 +318,9 @@ func kingCaptureScore(b *dragontoothmg.Board, ply int) int { } // orderMoves sorts moves best-first so alpha-beta prunes as early as possible: -// promotions first, then captures by MVV-LVA (most valuable victim, least -// valuable attacker), then quiet moves. -func orderMoves(b *dragontoothmg.Board, moves []dragontoothmg.Move) []dragontoothmg.Move { +// the transposition-table move first, then promotions, then captures by MVV-LVA +// (most valuable victim, least valuable attacker), then quiet moves. +func orderMoves(b *dragontoothmg.Board, moves []dragontoothmg.Move, ttMove dragontoothmg.Move) []dragontoothmg.Move { type scored struct { move dragontoothmg.Move score int @@ -216,13 +329,17 @@ func orderMoves(b *dragontoothmg.Board, moves []dragontoothmg.Move) []dragontoot for i := range moves { m := moves[i] sc := 0 - if p := m.Promote(); p != dragontoothmg.Nothing { - sc += 90000 + pieceValue[p] - } - if dragontoothmg.IsCapture(m, b) { - victim, _ := dragontoothmg.GetPieceType(m.To(), b) - attacker, _ := dragontoothmg.GetPieceType(m.From(), b) - sc += 10000 + pieceValue[victim]*8 - pieceValue[attacker] + if ttMove != 0 && m == ttMove { + sc = 1 << 20 + } else { + if p := m.Promote(); p != dragontoothmg.Nothing { + sc += 90000 + pieceValue[p] + } + if dragontoothmg.IsCapture(m, b) { + victim, _ := dragontoothmg.GetPieceType(m.To(), b) + attacker, _ := dragontoothmg.GetPieceType(m.From(), b) + sc += 10000 + pieceValue[victim]*8 - pieceValue[attacker] + } } list[i] = scored{m, sc} } diff --git a/internal/engine/search_test.go b/internal/engine/search_test.go index 7068a94..e9d392d 100644 --- a/internal/engine/search_test.go +++ b/internal/engine/search_test.go @@ -53,6 +53,77 @@ func TestSearchRespectsMoveTime(t *testing.T) { } } +func TestSearchWithTranspositionTableFindsSameMove(t *testing.T) { + // A shared TT must not change the result of a fixed-depth search, only its + // speed. Compare a run with a private table against one with an explicit + // table passed in. + fen := "r4rk1/1pp1qppp/p1np1n2/2b1p1B1/2B1P1b1/P1NP1N2/1PP1QPPP/R4RK1 w - - 0 10" + b1 := dragontoothmg.ParseFen(fen) + plain := engine.Search(&b1, engine.SearchParams{MaxDepth: 6}) + + b2 := dragontoothmg.ParseFen(fen) + withTT := engine.Search(&b2, engine.SearchParams{MaxDepth: 6, TT: engine.NewTT(16)}) + + if plain.BestMove != withTT.BestMove { + t.Errorf("best move changed with a TT: %s vs %s", plain.BestMove.String(), withTT.BestMove.String()) + } +} + +func TestSearchTranspositionTableCutsNodeCount(t *testing.T) { + // Re-searching the same position with a warm TT should visit far fewer + // nodes than the cold search did. + fen := "r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 1" + tt := engine.NewTT(16) + + b1 := dragontoothmg.ParseFen(fen) + cold := engine.Search(&b1, engine.SearchParams{MaxDepth: 6, TT: tt}) + + b2 := dragontoothmg.ParseFen(fen) + warm := engine.Search(&b2, engine.SearchParams{MaxDepth: 6, TT: tt}) + + if warm.Nodes >= cold.Nodes { + t.Errorf("warm search visited %d nodes, cold visited %d; expected the warm run to be cheaper", warm.Nodes, cold.Nodes) + } +} + +func TestSearchLazySMPFindsMate(t *testing.T) { + b := dragontoothmg.ParseFen("6k1/5ppp/8/8/8/8/8/R5K1 w - - 0 1") + res := engine.Search(&b, engine.SearchParams{MaxDepth: 4, Threads: 4}) + if got := res.BestMove.String(); got != "a1a8" { + t.Fatalf("Lazy-SMP best move = %s (score %d), want a1a8", got, res.Score) + } + if res.Score < mateThreshold { + t.Errorf("score = %d, want a mate score (>= %d)", res.Score, mateThreshold) + } +} + +func TestSearchLazySMPRespectsMoveTime(t *testing.T) { + b := dragontoothmg.ParseFen(dragontoothmg.Startpos) + start := time.Now() + res := engine.Search(&b, engine.SearchParams{ + MaxDepth: maxDepthUnbounded, + MoveTime: 200 * time.Millisecond, + Threads: 4, + }) + if elapsed := time.Since(start); elapsed > time.Second { + t.Fatalf("Lazy-SMP search ran %s, expected it to stop near 200ms", elapsed) + } + if res.BestMove.String() == "0000" { + t.Fatal("Lazy-SMP search returned no move") + } + if res.Depth == 0 { + t.Errorf("Lazy-SMP search completed no full depth") + } +} + +func TestSearchLazySMPWinsFreeMaterial(t *testing.T) { + b := dragontoothmg.ParseFen("3q2k1/8/8/8/8/8/6K1/3R4 w - - 0 1") + res := engine.Search(&b, engine.SearchParams{MaxDepth: 5, Threads: 4}) + if got := res.BestMove.String(); got != "d1d8" { + t.Fatalf("Lazy-SMP best move = %s, want d1d8", got) + } +} + const ( mateThreshold = 1_000_000 - 64 maxDepthUnbounded = 64 diff --git a/internal/engine/transposition.go b/internal/engine/transposition.go new file mode 100644 index 0000000..81ba735 --- /dev/null +++ b/internal/engine/transposition.go @@ -0,0 +1,163 @@ +package engine + +import ( + "sync/atomic" + + "github.com/dylhunn/dragontoothmg" +) + +// Transposition-table bound flags. They record how a stored score relates to the +// true value of the position: EXACT is the value itself, LOWER is a fail-high +// (the value is at least this), UPPER is a fail-low (the value is at most this). +const ( + boundExact = 1 + boundLower = 2 + boundUpper = 3 +) + +const ( + defaultHashMB = 64 + // One ttEntry is two uint64 words. + ttEntryBytes = 16 +) + +// ttEntry is a single table slot, accessed with Hyatt's lockless XOR trick so +// Lazy-SMP workers can share the table without a mutex. word0 holds key ^ data +// and word1 holds data; a reader recovers key' = word0 ^ word1 and rejects the +// entry unless key' matches the probe key. A write torn across two goroutines +// therefore fails that check and simply reads as a miss. +type ttEntry struct { + lock atomic.Uint64 // key ^ data + data atomic.Uint64 // packed move | score | depth | bound +} + +// data layout (64 bits): +// +// bits 0..15 best move (dragontoothmg.Move, uint16) +// bits 16..47 score, int32 two's-complement (mate scores need > 16 bits) +// bits 48..55 depth, uint8 +// bits 56..57 bound flag +func packTT(move dragontoothmg.Move, score, depth, bound int) uint64 { + switch { + case depth < 0: + depth = 0 + case depth > 255: + depth = 255 + } + // score fits int32 (mate scores are ~1e6); the mask keeps only its low 32 + // bits and unpackTT sign-extends them back. + return uint64(uint16(move)) | + (uint64(uint32(score)) << 16) | //nolint:gosec // deliberate low-32-bit pack; unpackTT restores the sign + (uint64(depth&0xff) << 48) | + (uint64(bound&3) << 56) +} + +func unpackTT(data uint64) (move dragontoothmg.Move, score, depth, bound int) { + move = dragontoothmg.Move(data & 0xffff) + score = int(int32(data >> 16)) //nolint:gosec // sign-extend the packed int32 score + depth = int((data >> 48) & 0xff) + bound = int((data >> 56) & 3) + return +} + +// TT is a fixed-size, power-of-two transposition table keyed by the Zobrist hash +// that dragontoothmg maintains incrementally (dragontoothmg.Board.Hash). It is +// safe for concurrent use by the Lazy-SMP workers. +type TT struct { + entries []ttEntry + mask uint64 +} + +// NewTT returns a table that occupies about mb megabytes, rounded down to the +// nearest power-of-two number of entries. mb <= 0 selects the default size. +func NewTT(mb int) *TT { + if mb <= 0 { + mb = defaultHashMB + } + want := (mb * 1024 * 1024) / ttEntryBytes + size := uint64(1) + for size<<1 <= uint64(want) { + size <<= 1 + } + return &TT{entries: make([]ttEntry, size), mask: size - 1} +} + +// Clear empties every slot. Call it between games (UCI "ucinewgame"); stale +// entries from an unrelated position are otherwise indistinguishable from live +// ones once their key happens to collide. +func (t *TT) Clear() { + for i := range t.entries { + t.entries[i].lock.Store(0) + t.entries[i].data.Store(0) + } +} + +// probe looks up key. cutoff is true when the stored score is usable directly +// for the (depth, alpha, beta) window; move is the stored best move and is +// returned for move ordering even when cutoff is false. ply rebases mate scores +// from the stored node back to the search root. +func (t *TT) probe(key uint64, depth, alpha, beta, ply int) (score int, move dragontoothmg.Move, cutoff bool) { + e := &t.entries[key&t.mask] + lock := e.lock.Load() + data := e.data.Load() + if data == 0 || lock^data != key { + return 0, 0, false + } + m, eScore, eDepth, bound := unpackTT(data) + if eDepth < depth { + return 0, m, false + } + s := ttProbeScore(eScore, ply) + switch bound { + case boundExact: + return s, m, true + case boundLower: + if s >= beta { + return s, m, true + } + case boundUpper: + if s <= alpha { + return s, m, true + } + } + return 0, m, false +} + +// store records a result for key. A shallower existing entry for the same key is +// overwritten; a deeper one is kept. +func (t *TT) store(key uint64, depth, score, bound int, move dragontoothmg.Move, ply int) { + e := &t.entries[key&t.mask] + if old := e.data.Load(); old != 0 && e.lock.Load()^old == key { + if _, _, oldDepth, _ := unpackTT(old); oldDepth > depth { + return + } + } + data := packTT(move, ttStoreScore(score, ply), depth, bound) + e.lock.Store(key ^ data) + e.data.Store(data) +} + +// ttStoreScore rebases a root-relative score to the node that is storing it, so +// "mate in N from the root" becomes "mate in N from here". Only mate scores +// move; ordinary centipawn scores are stored unchanged. +func ttStoreScore(score, ply int) int { + switch { + case score >= mateScore-maxPly: + return score + ply + case score <= -mateScore+maxPly: + return score - ply + } + return score +} + +// ttProbeScore is the inverse of ttStoreScore: it rebases a stored score back to +// the search root. +func ttProbeScore(score, ply int) int { + switch { + case score >= mateScore-maxPly: + return score - ply + case score <= -mateScore+maxPly: + return score + ply + } + return score +} diff --git a/internal/engine/transposition_test.go b/internal/engine/transposition_test.go new file mode 100644 index 0000000..52abf42 --- /dev/null +++ b/internal/engine/transposition_test.go @@ -0,0 +1,112 @@ +package engine + +import ( + "sync" + "testing" + + "github.com/dylhunn/dragontoothmg" +) + +func TestTTStoreThenProbeExact(t *testing.T) { + tt := NewTT(1) + key := uint64(0xdeadbeefcafef00d) + move, _ := dragontoothmg.ParseMove("e2e4") + + tt.store(key, 5, 42, boundExact, move, 0) + + score, got, cutoff := tt.probe(key, 5, -100, 100, 0) + if !cutoff { + t.Fatalf("expected an exact-bound cut-off") + } + if score != 42 { + t.Errorf("score = %d, want 42", score) + } + if got != move { + t.Errorf("move = %s, want %s", got.String(), move.String()) + } +} + +func TestTTShallowEntryDoesNotCutButKeepsMove(t *testing.T) { + tt := NewTT(1) + key := uint64(1) + move, _ := dragontoothmg.ParseMove("d2d4") + tt.store(key, 3, 10, boundExact, move, 0) + + score, got, cutoff := tt.probe(key, 6, -100, 100, 0) + if cutoff { + t.Errorf("a depth-3 entry must not satisfy a depth-6 probe") + } + if score != 0 { + t.Errorf("score = %d, want 0 when no cut-off", score) + } + if got != move { + t.Errorf("move = %s, want the stored %s for ordering", got.String(), move.String()) + } +} + +func TestTTBoundsRespectWindow(t *testing.T) { + tt := NewTT(1) + key := uint64(7) + + tt.store(key, 4, 50, boundLower, 0, 0) + if _, _, cutoff := tt.probe(key, 4, 0, 40, 0); !cutoff { + t.Errorf("lower bound 50 should cut when beta=40") + } + if _, _, cutoff := tt.probe(key, 4, 0, 60, 0); cutoff { + t.Errorf("lower bound 50 should not cut when beta=60") + } + + tt.store(key, 4, 50, boundUpper, 0, 0) + if _, _, cutoff := tt.probe(key, 4, 60, 100, 0); !cutoff { + t.Errorf("upper bound 50 should cut when alpha=60") + } + if _, _, cutoff := tt.probe(key, 4, 40, 100, 0); cutoff { + t.Errorf("upper bound 50 should not cut when alpha=40") + } +} + +func TestTTMateScoreIsRebasedByPly(t *testing.T) { + tt := NewTT(1) + key := uint64(99) + // "mate in 1 from the root" is mateScore-1. Store it as seen from ply 4. + rootScore := mateScore - 1 + tt.store(key, 10, rootScore, boundExact, 0, 4) + + score, _, cutoff := tt.probe(key, 10, -infinity, infinity, 4) + if !cutoff { + t.Fatal("expected a cut-off") + } + if score != rootScore { + t.Errorf("probed mate score = %d, want %d (round-trip through ply rebasing)", score, rootScore) + } + if score < mateScore-maxPly { + t.Errorf("score %d no longer reads as a mate", score) + } +} + +func TestTTMissOnKeyMismatch(t *testing.T) { + tt := NewTT(1) + tt.store(1, 5, 10, boundExact, 0, 0) + if _, _, cutoff := tt.probe(2, 1, -100, 100, 0); cutoff { + t.Errorf("probe of an unstored key must miss") + } +} + +// TestTTConcurrentAccessIsRaceFree exercises the lockless entries from many +// goroutines at once; run with -race. +func TestTTConcurrentAccessIsRaceFree(t *testing.T) { + tt := NewTT(1) + var wg sync.WaitGroup + for g := 0; g < 8; g++ { + wg.Add(1) + go func(g int) { + defer wg.Done() + for i := 0; i < 20000; i++ { + key := uint64((i % 512) + 1) + tt.store(key, i%64, (i%200)-100, boundExact, dragontoothmg.Move(i&0xffff), 0) + tt.probe(key, i%64, -100, 100, 0) + } + }(g) + } + wg.Wait() +} diff --git a/internal/uci/uci.go b/internal/uci/uci.go index 769eb23..9801e27 100644 --- a/internal/uci/uci.go +++ b/internal/uci/uci.go @@ -7,6 +7,7 @@ import ( "bufio" "fmt" "io" + "runtime" "strconv" "strings" "time" @@ -23,17 +24,31 @@ const ( // Fraction of the remaining clock to spend on one move when the GUI sends // wtime/btime rather than an explicit movetime. clockDivisor = 30 + + defaultHashMB = 64 + minHashMB = 1 + maxHashMB = 4096 + maxThreads = 256 + defaultThreads = 1 ) type session struct { - board dragontoothmg.Board - out io.Writer + board dragontoothmg.Board + out io.Writer + tt *engine.TT + hashMB int + threads int } // Run reads UCI commands from r and writes responses to w until "quit" or EOF. // version is reported in the "id name" line. func Run(r io.Reader, w io.Writer, version string) error { - s := &session{board: dragontoothmg.ParseFen(dragontoothmg.Startpos), out: w} + s := &session{ + board: dragontoothmg.ParseFen(dragontoothmg.Startpos), + out: w, + hashMB: defaultHashMB, + threads: defaultThreads, + } sc := bufio.NewScanner(r) sc.Buffer(make([]byte, 0, 64*1024), 1024*1024) @@ -46,11 +61,19 @@ func Run(r io.Reader, w io.Writer, version string) error { case "uci": fmt.Fprintf(w, "id name %s %s\n", engineName, version) fmt.Fprintf(w, "id author %s\n", engineAuthor) + fmt.Fprintf(w, "option name Hash type spin default %d min %d max %d\n", defaultHashMB, minHashMB, maxHashMB) + fmt.Fprintf(w, "option name Threads type spin default %d min 1 max %d\n", defaultThreads, maxThreads) fmt.Fprintln(w, "uciok") case "isready": + s.ensureTT() fmt.Fprintln(w, "readyok") + case "setoption": + s.handleSetOption(fields[1:]) case "ucinewgame": s.board = dragontoothmg.ParseFen(dragontoothmg.Startpos) + if s.tt != nil { + s.tt.Clear() + } case "position": s.handlePosition(fields[1:]) case "go": @@ -66,6 +89,56 @@ func Run(r io.Reader, w io.Writer, version string) error { return sc.Err() } +// ensureTT lazily allocates the transposition table at the configured size. +func (s *session) ensureTT() { + if s.tt == nil { + s.tt = engine.NewTT(s.hashMB) + } +} + +// handleSetOption parses "setoption name value " for the options +// advertised in the "uci" reply. Unknown options are ignored, as the protocol +// requires. +func (s *session) handleSetOption(args []string) { + var name, value string + for i := 0; i < len(args); i++ { + switch args[i] { + case "name": + j := i + 1 + for j < len(args) && args[j] != "value" { + j++ + } + name = strings.Join(args[i+1:j], " ") + i = j - 1 + case "value": + value = strings.Join(args[i+1:], " ") + i = len(args) + } + } + + switch strings.ToLower(name) { + case "hash": + if n, err := strconv.Atoi(value); err == nil { + s.hashMB = clamp(n, minHashMB, maxHashMB) + s.tt = engine.NewTT(s.hashMB) // resize now, before the next search + } + case "threads": + if n, err := strconv.Atoi(value); err == nil { + s.threads = clamp(n, 1, maxThreads) + } + } +} + +func clamp(v, lo, hi int) int { + if v < lo { + return lo + } + if v > hi { + return hi + } + return v +} + func (s *session) handlePosition(args []string) { if len(args) == 0 { return @@ -128,6 +201,13 @@ func (s *session) handleGo(args []string) { } params.MoveTime = movetime + s.ensureTT() + params.TT = s.tt + params.Threads = s.threads + if n := runtime.NumCPU(); params.Threads > n { + params.Threads = n // never spawn more workers than the machine has cores + } + res := engine.Search(&s.board, params) if res.Depth > 0 { fmt.Fprintf(s.out, "info depth %d score %s nodes %d time %d pv %s\n",