diff --git a/docs/design.md b/docs/design.md new file mode 100644 index 0000000..b881979 --- /dev/null +++ b/docs/design.md @@ -0,0 +1,91 @@ +# Design + +How the engine is put together, what each technique buys, and the backlog of +ideas not yet implemented. For the component diagram and invariants see +[architecture.md](architecture.md); for build, perft, and strength-testing +workflow see [development.md](development.md); for the reasons behind the big +choices see the [ADRs](adr/). + +## Architecture + +`engine.Search` is the search core. It is a plain function around a `searcher` +value (node counter + deadline) with a few free collaborators rather than an +injected object graph — small enough that dependency injection would cost more +than it buys. + +| collaborator | responsibility | +|---|---| +| `dragontoothmg.Board` | board state, legal move generation, `Apply` / unapply, Zobrist hash, FEN — never reimplemented (see [ADR 0002](adr/0002-use-dragontoothmg-for-move-generation.md)) | +| `engine.Evaluate` | static evaluation from the side-to-move's view (positive = better for that side) | +| `engine.orderMoves` | orders moves to maximise alpha-beta cut-offs; currently promotions then MVV-LVA captures then quiets | +| `searcher` | carries the node count and wall-clock deadline; answers `timeUp` and flips `stopped` | + +`internal/uci` is the protocol layer and the only place that prints `info` / +`bestmove`. Search is synchronous, so `stop` is a no-op and `bestmove` is emitted +as soon as `go` returns. `cmd/gochess` wires these together and adds the `perft` +and `bench` subcommands. + +## Implemented + +### [Search](https://www.chessprogramming.org/Search) + +- [Negamax](https://www.chessprogramming.org/Negamax) with [alpha-beta pruning](https://www.chessprogramming.org/Alpha-Beta) — fail-hard +- [Iterative Deepening](https://www.chessprogramming.org/Iterative_Deepening) — the last fully completed depth is the one returned +- [Quiescence Search](https://www.chessprogramming.org/Quiescence_Search) at the horizon — captures and promotions only, depth-bounded by `maxPly` +- [Move Ordering](https://www.chessprogramming.org/Move_Ordering) — promotions first, then [MVV-LVA](https://www.chessprogramming.org/MVV-LVA) captures, then quiet moves +- [Mate-distance scoring](https://www.chessprogramming.org/Mate_Distance_Pruning) — `mateScore - ply`, so the shortest mate is preferred; a proven mate ends iterative deepening early +- Draw detection — the fifty-move rule (`Halfmoveclock >= 100`) is scored `0` inside the tree +- Time management — a hard wall-clock budget checked every 2048 nodes; the UCI layer spends `1/30` of the remaining clock when the GUI sends `wtime` / `btime` instead of `movetime` +- King-capture guard — scores the illegal position `dragontoothmg` can hand back when the side not to move was already in check, instead of panicking on an empty king bitboard + +### [Evaluation](https://www.chessprogramming.org/Evaluation) + +- Material — standard centipawn values (`P 100 · N 320 · B 330 · R 500 · Q 900`) +- [Piece-square tables](https://www.chessprogramming.org/Piece-Square_Tables) — Michniewski's "simplified evaluation function", single (non-tapered) set; black reads the vertically mirrored square (`sq^56`) +- [Bishop pair](https://www.chessprogramming.org/Bishop_Pair) — `+30` for holding both bishops + +Evaluation is recomputed from scratch on every call; there is no incremental +update and no pawn or evaluation hash. + +## Backlog + +### Search + +- [Transposition Table](https://www.chessprogramming.org/Transposition_Table) — Zobrist key is already exposed by `dragontoothmg.Board.Hash()`; needs EXACT / LOWER / UPPER bounds and ply-rebased mate scores +- [Principal Variation Search](https://www.chessprogramming.org/Principal_Variation_Search) — null-window scout + re-search +- [Killer](https://www.chessprogramming.org/Killer_Heuristic) and [history](https://www.chessprogramming.org/History_Heuristic) heuristics in `orderMoves` +- [Null Move Pruning](https://www.chessprogramming.org/Null_Move_Pruning) and [Late Move Reductions](https://www.chessprogramming.org/Late_Move_Reductions) +- [Aspiration Windows](https://www.chessprogramming.org/Aspiration_Windows) around the previous iteration's score +- [Static Exchange Evaluation](https://www.chessprogramming.org/Static_Exchange_Evaluation) for capture ordering and bad-capture pruning in quiescence +- Threefold-[repetition](https://www.chessprogramming.org/Repetitions) detection (needs a position history the current `Search` does not keep) +- A real principal variation in the `info` line (only `bestmove` is reported today) +- [Lazy SMP](https://www.chessprogramming.org/Lazy_SMP) once a shared TT exists +- Asynchronous search so `stop` and pondering actually work +- [Opening book](https://www.chessprogramming.org/Opening_Book) (Polyglot) and [Syzygy endgame tablebases](https://www.chessprogramming.org/Endgame_Tablebases) + +### Evaluation + +- [Tapered eval](https://www.chessprogramming.org/Tapered_Eval) — mid/endgame PST pairs interpolated by game phase ([PeSTO](https://www.chessprogramming.org/PeSTO%27s_Evaluation_Function)) +- Pawn structure — [passed](https://www.chessprogramming.org/Passed_Pawn) / [isolated](https://www.chessprogramming.org/Isolated_Pawn) / [doubled](https://www.chessprogramming.org/Doubled_Pawn) / [backward](https://www.chessprogramming.org/Backward_Pawn) pawns +- [Mobility](https://www.chessprogramming.org/Mobility), [rook on open file](https://www.chessprogramming.org/Rook_on_Open_File), [knight outposts](https://www.chessprogramming.org/Outpost), [king safety](https://www.chessprogramming.org/King_Safety), [tempo](https://www.chessprogramming.org/Tempo) +- [Incremental updates](https://www.chessprogramming.org/Incremental_Updates) of material + PST on make/unmake +- [Pawn](https://www.chessprogramming.org/Pawn_Hash_Table) and [evaluation](https://www.chessprogramming.org/Evaluation_Hash_Table) hash tables + +### Alternative search algorithms to evaluate + +- [MTD(f)](https://www.chessprogramming.org/MTD\(f\)) +- [NegaC*](https://www.chessprogramming.org/NegaC*) + +## Infrastructure the design assumes + +- **Move generation is never reimplemented.** Any discrepancy is a `dragontoothmg` + bug or an API misuse, found and fixed with a perft test (`engine.PerftDivide` + bisects against a reference). +- **`Search` leaves the board unmodified** — every `Apply` is paired with its + unapply, including on early returns. +- **Evaluation sign convention**: positive = good for the side to move. +- **Untrusted input**: FEN and UCI strings come from GUIs and tournament + managers; parsing must not panic (see [SECURITY.md](../SECURITY.md)). +- **Strength changes are proven by a match**, not intuition — SPRT via + `cutechess-cli` against the previous build (see + [development.md](development.md#strength-testing)). diff --git a/docs/engine-strength.md b/docs/engine-strength.md new file mode 100644 index 0000000..c418486 --- /dev/null +++ b/docs/engine-strength.md @@ -0,0 +1,126 @@ +# Engine Strength + +**Estimated playing strength: ~1500–1750 Elo (likely ~1600) on the CCRL blitz +scale.** + +This is a reasoned estimate from the feature set and search speed, **not a +measured result**. goChess has not yet played a rated match or an SPRT gauntlet +against calibrated opponents. Treat the number as an order-of-magnitude guide +until [the strength-testing harness](#measuring-it-properly) produces real data. + +## How the estimate is derived + +### Positive contributors + +- **Move generation** — delegated to `dragontoothmg`, a magic-bitboard legal + move generator, and verified exact with perft (start position to depth 6, + Kiwipete to depth 4). No legality bugs, no pseudo-legal filtering cost. +- **Search** — iterative-deepening negamax with alpha-beta pruning, a fail-hard + window, and mate-distance-aware scoring (`mateScore - ply`, so the engine + prefers the shortest mate and the longest defence). +- **Quiescence search** at the horizon (captures and promotions only), so the + static evaluation is never taken in the middle of a capture sequence — this + alone removes most one-move blunders. +- **Move ordering** — promotions first, then captures by MVV-LVA (most valuable + victim, least valuable attacker), then quiet moves. Good ordering is what + makes alpha-beta actually prune. +- **Evaluation** — material values + Michniewski piece-square tables + a + bishop-pair bonus. Crude, but it captures development, central control, king + placement and the two-bishop advantage. +- **Compiled and fast** — ~2.7M nodes/sec in a single thread on an Apple M4 + (`gochess bench`). That is ~50–60× the node rate of a typical Python engine, + so effective search depth is respectable even without a transposition table. +- **Robust I/O** — FEN and UCI parsing is fuzz-safe and non-panicking; the + engine will not forfeit on a malformed GUI command. + +### Limiting factors + +- **No transposition table.** The Zobrist hash is already exposed by + `dragontoothmg`, but nothing caches search results yet. Transpositions are + re-searched from scratch, and there is no hash move to seed move ordering. + This is the single largest missing feature — worth an estimated 150–250 Elo. +- **No killer or history heuristic.** Quiet-move ordering is essentially + arbitrary, so cut-offs deep in the tree are later than they should be. +- **No null-move pruning, no late move reductions, no futility or delta + pruning, no aspiration windows.** The tree is close to full-width + alpha-beta + quiescence. Effective middlegame depth is roughly 6–9 ply at + blitz time controls, where a pruning-heavy engine of the same speed would + reach 12–16. +- **No search extensions** (check extensions, singular extensions). Tactical + lines that need one extra ply past the horizon are missed. +- **Hand-set evaluation weights, never tuned.** No Texel tuning, no + game-phase interpolation (a single PST set is used from opening to endgame), + no explicit terms for passed pawns, pawn structure, mobility, rook-on-open- + file, or king safety beyond the king PST. +- **Thin endgame play.** No tablebase probing, no KPK / KBNK knowledge, no + contempt. The 50-move rule is honoured but threefold repetition is not + detected inside the search. +- **Single-threaded.** No Lazy SMP or any other parallel search; the other + nine cores of the test machine sit idle. +- **Synchronous search.** `stop` is a no-op and there is no pondering, so the + engine cannot think on the opponent's clock or bail out of a bad time + allocation. + +## Calibration against known engines + +| Reference engine | ~CCRL blitz | Relevant comparison | +| ---------------- | ----------- | ------------------- | +| TSCP 1.81 | ~1700 | Similar search shape (alpha-beta, iterative deepening, quiescence, MVV-LVA) **plus** a transposition table. goChess should land at or just below TSCP until the TT lands. | +| Sungorus 1.4 | ~2000 | TT + null-move + PVS + killers. Clearly stronger than goChess today. | +| CT800 / Claudia class | ~2100+ | Full modern pruning set. Out of reach without the roadmap features. | + +The feature set most closely resembles a "first working alpha-beta engine" — +tactically sound at shallow depth, positionally simplistic, and losing rating to +every missing pruning technique. The fast node rate keeps it from dropping into +true beginner territory. + +## Time-control sensitivity + +Deeper search helps goChess more than most engines, because without pruning its +depth is unusually shallow for its speed — each extra ply is high-value. + +| Time control | Estimated Elo | Notes | +| ------------ | ------------- | ----- | +| Bullet (1+0) | ~1350–1550 | Depth 4–6; positional weaknesses dominate. | +| Blitz (3+2 / 5+0) | ~1500–1750 | Depth 6–9; the headline estimate. | +| Rapid (15+10) | ~1650–1900 | Depth 9–12; tactics get sharper, eval ceiling starts to bite. | +| Classical (40/40) | ~1750–2000 | Depth-limited by the missing TT more than by the clock. | + +## Where the number would move + +Rough, independent Elo deltas from the [roadmap](../README.md#roadmap), +assuming each is implemented competently and validated by SPRT: + +| Change | Estimated Elo | +| ------ | ------------- | +| Transposition table + hash-move ordering | +150 to +250 | +| Killer moves + history heuristic | +50 to +100 | +| Null-move pruning | +50 to +80 | +| Late move reductions | +50 to +100 | +| Aspiration windows | +10 to +30 | +| Game-phase eval interpolation (tapered eval) | +30 to +60 | +| Passed pawns / king safety / mobility terms | +40 to +80 | +| Texel-tuned evaluation weights | +40 to +80 | +| Opening book (Polyglot) | +20 to +40 at short TC | +| Syzygy tablebase probing | +10 to +20 | +| Lazy SMP (8 threads) | +100 to +150 | + +Landing the transposition table, killers/history, null-move and LMR together +would plausibly put goChess in the 1950–2150 range; adding tapered/tuned eval +and Lazy SMP on top targets 2300+. + +## Measuring it properly + +The estimate above is replaced by data as soon as there is a gauntlet result. +The intended process (see [docs/development.md](development.md#strength-testing)): + +1. Build the candidate and the previous release as two binaries. +2. Run `cutechess-cli` self-play with an SPRT stopping rule + (`elo0=0 elo1=5`, `alpha=beta=0.05`) at `tc=10+0.1`, from a varied EPD + opening book, `-repeat`, `-concurrency 8`. +3. Only merge a search/eval change if it passes SPRT (or is Elo-neutral and + justified on other grounds). +4. Periodically run a gauntlet against a ladder of rated reference engines + (TSCP, Sungorus, Fruit 2.1, …) to anchor an absolute CCRL-comparable number. + +Until step 4 has run, cite this document as an estimate, not a rating. diff --git a/docs/performance.md b/docs/performance.md new file mode 100644 index 0000000..bdf1b56 --- /dev/null +++ b/docs/performance.md @@ -0,0 +1,81 @@ +# Performance + +Fresh benchmark of the current engine (iterative deepening, negamax alpha-beta, +bounded quiescence, MVV-LVA move ordering, hard time limit; **no** transposition +table, killers/history, null-move pruning, or LMR yet) on one machine: Apple M4, +10 cores, Go 1.27, `darwin/arm64`. Search is from the starting position; rows +stop once a search passes ~5 seconds. "Nodes" is the engine's node counter +(`negamax` + `quiesce` calls). Numbers are single-run and rounded — the search +is single-threaded and deterministic, so node counts repeat exactly and only the +wall-clock wobbles. + +## What `go` / `go depth N` runs today: single-process iterative deepening + +This is the real search path. There is no Lazy SMP or parallel search — `go` +runs one `Search` on one goroutine, deepening from depth 1 until it runs out of +`movetime` (or hits `depth N`), and returns the last fully completed depth. The +rows below use a generous budget so the run isn't deadline-capped. + +| depth | time | nodes | bestmove | +|------:|-----:|------:|:---------| +| 4 | 0.01s | 55,614 | b1c3 | +| 5 | 0.06s | 457,203 | d2d4 | +| 6 | 0.45s | 3,097,162 | d2d4 | +| 7 | 3.4s | 23,615,017 | e2e4 | +| 8 | 30s | 160,689,328 | d2d4 | + +(Cumulative nodes over the deepening series 1..N; the shallower iterations add +well under 1%.) Depth 7 from the opening lands in ~3s on this machine, depth 8 in +~30s. Adding a transposition table is the single biggest lever left — it would +recover most of the redundant re-search between iterations and across +transposing lines. + +## Move generation: `perft` from the start position + +Move generation is delegated to `dragontoothmg` and is not the bottleneck. +`gochess perft N` walks the full legal game tree with no evaluation or pruning: + +| depth | nodes | time | nps | +|------:|------:|-----:|----:| +| 4 | 197,281 | 5ms | 42M | +| 5 | 4,865,609 | 47ms | 105M | +| 6 | 119,060,324 | 699ms | 170M | +| 7 | 3,195,901,860 | 19.9s | 160M | + +~160–170M nodes/sec at the deeper counts, all of which match the published +[perft results](https://www.chessprogramming.org/Perft_Results). + +## Single fixed-depth search from the start position + +A controlled baseline: one search to exactly depth `N` (no iterative deepening), +clock never armed. Deterministic, and it isolates search-tree efficiency, so +this is the table to watch when judging whether a search change helped. Not what +the engine runs in a game. + +The `perft(depth)` column is the size of the *full* legal game tree at that +depth — the branching the search would face with no alpha-beta at all. `pruned` +is `1 - nodes/perft`. + +| depth | time | nodes | perft(depth) | pruned | +|------:|-----:|------:|-------------:|-------:| +| 2 | 0.000s | 452 | 400 | −13% | +| 3 | 0.002s | 5,318 | 8,902 | 40% | +| 4 | 0.011s | 49,804 | 197,281 | 75% | +| 5 | 0.056s | 401,589 | 4,865,609 | 91.8% | +| 6 | 0.40s | 2,639,959 | 119,060,324 | 97.8% | +| 7 | 2.9s | 20,517,855 | 3,195,901,860 | 99.4% | +| 8 | 27s | 137,074,311 | 84,998,978,956 | 99.84% | + +The `pruned` column is an *estimate*: the two counts aren't the same unit. +"nodes" counts every node the search visits (internal nodes and quiescence +included, and quiescence looks past `depth` in forcing lines); `perft` is only +the leaves at exactly `depth`. At depth 2 the search actually expands *more* +nodes than the full tree — quiescence chases every capture sequence to its end +and there is no TT to catch repeats — so `pruned` goes negative. From depth 4 on +the trend is real: MVV-LVA ordering plus alpha-beta take the tree from "search +all of it" to "search one node in ~620" by depth 8. + +The effective branching factor across the deeper rows is roughly 7 (≈√35, the +alpha-beta ideal for a ~35-move position would be ~6). Closing that last gap — +and cutting the constant factor — is what the transposition table, killer moves, +and null-move pruning on the [roadmap](../README.md#roadmap) are for.