uber
diff --git a/‎doc/rfc/submitqueue/speculation-generator.md‎
Lines changed: 171 additions & 0 deletions b/‎doc/rfc/submitqueue/speculation-generator.md‎
Lines changed: 171 additions & 0 deletions
@@ -0,0 +1,171 @@
+# Speculation Path Generation
+
+## Summary
+
+The merge queue can build a batch early by assuming whether each of its dependencies will succeed or fail. These assumptions create many possible build paths, but CI can run only a few. The default generator returns the most likely paths first and stops when the build budget is full.
+
+It does that by treating each batch's possible paths as a binary decision tree and always extending the most probable partial path, across all batches at once. Paths come out in probability order, and only the paths actually requested are ever created. In the architecture of [speculation.md](speculation.md), this is the Generator inside the default Speculator.
+
+## The problem
+
+A batch waiting on n undecided batches ahead of it has one possible path per combination of outcomes — 2ⁿ in all. A batch ten deep in a conflict chain has 1,024 possible futures, and CI can afford perhaps two or three of them. The rest of the system asks the generator for candidates one at a time, best first, and stops asking once the build budget is full.
+
+So the generator must return paths in order of how likely they are to be useful, never return the same path twice, never return a path that contradicts a known outcome, and do work proportional to the number of paths requested — not to 2ⁿ.
+
+## The queue in this doc
+
+One worked example runs through the whole document. Four batches are in flight. A, B, and C form a conflict chain; D conflicts with nothing:
+
+```
+main ◀── A ◀── B ◀── C        B conflicts with A, and C conflicts with B —
+                              each batch records only its direct conflicts
+         D                    no conflicts — D depends on nothing
+```
+
+This is the **DAG**: the queue's batches and the dependency **edges** between them ("edge" is a DAG term only — a direct conflict making one batch depend on another). Each batch stores only its **direct** conflicts — C's stored list is just `[B]` — so longer chains exist only as chains of edges. The generator builds the DAG from those edges each run and walks it from the roots down to find each batch's **ancestors**: every batch that can reach it along dependency edges. For C that walk collects B directly and, through B, A. The DAG says *what is undecided*; it is the input, not the thing searched.
+
+A pluggable scorer estimates how likely each batch's build is to succeed. Only batches that appear as someone's dependency are ever asked about — A and B here; nothing depends on C or D, so their probabilities never matter:
+
+| dependency | P(succeeds) |
+| --- | --- |
+| A | 0.9 |
+| B | 0.8 |
+
+## The decision: one tree per batch
+
+Each batch being evaluated gets its own **speculation tree** — a binary tree with one level per *ancestor* in the DAG, in queue order. Walking the DAG from its roots downward is what produces the levels: a batch's ancestors are always earlier in the queue, so the walk visits them in the order the tree decides them. (speculation.md calls the batch a path builds the path's *head*.)
+
+C makes the point: it conflicts directly only with B, yet its tree has a level for A. Which B there is to stack on depends on A — B built with A and B built without A are different changes — so A's outcome reaches C through the chain even though they never touch the same code. One assumption per ancestor is what makes a path self-contained: it pins the exact stack of changes the build sits on, with no reference to any other path.
+
+- The **root** is the batch with no decisions taken yet. Its **path probability** is **1**: nothing has been assumed, so nothing can be wrong.
+- A **decision** is the binary choice at a level: the ancestor at that level *succeeds* or *fails*. Taking a decision multiplies the path probability by that side's chance — p for succeeds, 1−p for fails.
+- A **node** is a partial path: the batch plus the decisions taken so far.
+- A **leaf** is a node with every level decided.
+- A **path** is the root-to-leaf decision sequence — the tree-side term for what the code records as a `SpeculationPath`. Its path probability is the probability that every assumption on it holds.
+
+The queue above has four trees, and their shapes fall straight out of each batch's ancestors:
+
+```
+ batch A — no ancestors        batch B — ancestor A            batch C — ancestors A and B
+
+        {} 1.0                        {} 1.0                              {} 1.0
+                                ┌───────┴───────┐                 ┌──────────┴──────────┐
+   the root is already       A✓ ×0.9         A✗ ×0.1          A✓ ×0.9               A✗ ×0.1
+   a leaf: one path,            │               │                 │                     │
+   no assumptions              0.9             0.1               0.9                   0.1
+                              leaf            leaf         ┌──────┴──────┐       ┌──────┴──────┐
+ batch D — same as A:                                   B✓ ×0.8      B✗ ×0.2  B✓ ×0.8      B✗ ×0.2
+ its tree is one node,                                      │            │        │            │
+ one path, probability 1                                  0.72          0.18     0.08         0.02
+```
+
+Read A's tree first, because it answers the question the notation invites. A has no ancestors, so there is nothing to decide: the root *is* a leaf, and A has exactly one path — build A directly, assume nothing, probability 1. The `{}` at every root means "no decisions taken *yet*", not "assumed empty dependencies"; for A the two coincide because nothing in the queue is ahead of it. D is the same.
+
+B's tree has one level: build on top of A (`[A✓]`, 0.9) or build without A (`[A✗]`, 0.1). C's has two levels and four leaves — its four possible futures, from `[A✓ B✓]` at 0.72 down to `[A✗ B✗]` at 0.02. Nobody builds these trees up front — they exist so the next section can walk them.
+
+## How generation works
+
+The generator keeps one list of partial paths, ordered by path probability, holding every tree's frontier at once. It starts with just the four roots on the list, then repeats four plain steps each time a candidate is requested:
+
+1. Take the most probable partial path off the list.
+2. If every one of its ancestors is decided, it is complete — return it as the next candidate.
+3. Otherwise, take its next undecided ancestor both ways: one copy assuming *succeeds*, one assuming *fails*, each with its probability multiplied in.
+4. Put both copies back on the list and go to step 1.
+
+The ordered list is a priority queue in the implementation; nothing else is needed. Traced on the example queue (a node is written as its batch plus the decisions taken; the four roots tie at 1.0 and order by batch ID, per the ties section):
+
+| step | taken (probability) | put back | returned |
+| --- | --- | --- | --- |
+| 1 | `A {}` (1.0) | — | **A: `[]`** |
+| 2 | `B {}` (1.0) | `B [A✓]` (0.9), `B [A✗]` (0.1) | — |
+| 3 | `C {}` (1.0) | `C [A✓]` (0.9), `C [A✗]` (0.1) | — |
+| 4 | `D {}` (1.0) | — | **D: `[]`** |
+| 5 | `B [A✓]` (0.9) | — | **B: `[A✓]`** |
+| 6 | `C [A✓]` (0.9) | `C [A✓B✓]` (0.72), `C [A✓B✗]` (0.18) | — |
+| 7 | `C [A✓B✓]` (0.72) | — | **C: `[A✓ B✓]`** |
+
+Four candidates out — the unconditional builds of A and D, B stacked on A, C stacked on both — in exactly the order a human would fund them. If the caller kept asking, the rest would follow in descending order: 0.18, 0.1, 0.08, 0.02.
+
+The point is what happens when the caller *stops*. After seven steps the list holds `C [A✓B✗]` (0.18), `B [A✗]` (0.1), and `C [A✗]` (0.1), and nothing else exists. `C [A✗]` is a single uncreated-below node standing in for C's entire without-A subtree — two paths, neither created, both reachable later if the caller keeps asking and skipped forever if it does not. The succeed-or-fail choice at each level is made only when a partial path is actually taken off the list — as generation progresses, never speculatively ahead of it.
+
+The guarantees, each with its one-line reason:
+
+- **Candidates are returned in non-increasing probability.** Every decision multiplies by a probability ≤ 1, so extending a partial path never raises it; taking the most probable entry first means no later path can beat an earlier one.
+- **Every combination exactly once.** Each partial path is created exactly once (by extending its unique parent), so continuing until no candidates remain returns every tree's every leaf, none twice.
+- **Work is proportional to what is requested.** Returning one path costs at most one step per tree level; nothing below an untaken node is ever created.
+- **Impossible decisions are not silently dropped.** A side with probability 0 makes its whole subtree probability 0 — it sorts after every possible path, but is still returned if the caller keeps asking. Nothing is discarded.
+
+## Path probability vs. the stored ranking value
+
+Everything above ranks by path probability, and that ranking is exactly what the implementation preserves — but it cannot compute the products literally. A batch with 7,100 ancestors at 0.9 each has a best path probability of 0.9⁷¹⁰⁰ ≈ 10⁻³²⁵, which is below the smallest number float64 can represent: the product rounds to exactly 0, every path of that batch ties at zero, and the order is lost.
+
+So the implementation never stores the probability itself. It stores the probability's **logarithm** as the **ranking value**, which turns products into sums:
+
+```
+log(p₁ × p₂ × ⋯ × pₙ) = log p₁ + log p₂ + ⋯ + log pₙ
+```
+
+Because log is strictly increasing, comparing ranking values orders paths *identically* to comparing the path probabilities themselves — and the sums stay comfortably finite: that wide batch's best ranking value is 7,100 × log 0.9 ≈ −748, an ordinary float64. Mapping C's tree once:
+
+| node | path probability | ranking value (log) |
+| --- | --- | --- |
+| `C {}` (root) | 1.0 | 0 |
+| `C [A✓]` | 0.9 | −0.105 |
+| `C [A✓B✓]` | 0.72 | −0.328 |
+
+Multiplying in a decision's probability becomes adding its log — a number ≤ 0, which restates "extending a partial path never raises it". A probability of 0 becomes −∞, which sorts below every finite value, matching the impossible-decision rule above.
+
+The ranking value is what a candidate carries out of the generator. It orders candidates within one run and means nothing across runs — the next run rescores from scratch.
+
+## Example across two runs
+
+The generator keeps no state between runs: each run builds trees from the queue's current state, and what changed since last time shows up as smaller trees. (What the rest of the system does with the candidates — funding, cancelling, merging — is covered in [speculation.md](speculation.md).)
+
+**Run 1.** The queue is as above; the four most probable paths are the ones the trace returned:
+
+```
+A: []  1.0   ·   D: []  1.0   ·   B: [A✓]  0.9   ·   C: [A✓ B✓]  0.72
+```
+
+**Between runs.** A's build fails. A had one unconditional path, so no other future exists in which it passes: A's outcome is now known.
+
+**Run 2.** A is no longer a batch being evaluated, so it gets no tree. As an *ancestor* it is now a fact, not a guess: every path fixes A to the known outcome (*fails*), and A's level disappears from B's and C's trees:
+
+```
+ batch B — A fixed to fails         batch C — A fixed to fails, one level left
+
+     {} 1.0                                {} 1.0
+                                     ┌───────┴───────┐
+  root is now a leaf:             B✓ ×0.8         B✗ ×0.2
+  the one path, [A✗],                │               │
+  probability 1.0                   0.8             0.2
+                                   leaf            leaf
+```
+
+The first candidates returned are `B: [A✗]` and `D: []`, both at probability 1.0, then `C: [A✗ B✓]` at 0.8. Look at B's closely: it is the *same path* that had probability 0.1 in run 1 — the assumptions are identical, so its identity is identical — but what was a long shot is now a certainty, because A's failure stopped being a probability and became a fact. Probabilities mean nothing across runs; a path's identity is what survives, which is how the rest of the system recognizes paths it already acted on. And no path returned in run 2 contradicts the known outcome: the `[A✓]` halves of both trees are simply gone.
+
+Rebuilding from scratch each run is cheap by construction: every ancestor whose outcome becomes known shrinks the affected trees by a level, and within a run only the partial paths actually taken off the list ever exist.
+
+## Edge cases and ties
+
+Trees are built over what is genuinely undecided; everything else is settled before generation starts.
+
+- **Ancestors are derived every run, never stored.** A batch's stored dependency list holds its direct conflicts only; the generator rebuilds the DAG from the run's snapshot and walks it to collect each batch's ancestors. Nothing persists a flattened chain — the DAG's edges are the single source of truth, and a chain that changes shape between runs (a batch resolving, a batch cancelled away) is simply walked differently next time.
+- **An ancestor whose outcome is known** is fixed to it — Succeeded fixes *succeeds*; Failed or Cancelled fixes *fails* — and its level drops out of the affected trees, exactly as A's did in run 2. An ancestor that is still being cancelled is not yet known and keeps its level.
+- **An ancestor absent from the run's snapshot** cannot be scored. Each run hands the generator the batches the controller read, and a referenced batch ID with no batch among them has nothing to score — and no edges of its own, so the walk also stops there. It contributes one level with a fixed default probability of succeeding, 0.95, a constant in the generator chosen on the observation that nearly every batch a queue accepts does build successfully; it only affects ranking, never which paths exist.
+- **Each unique ancestor is scored once per run**, however many trees it appears in. In the example, A is a level in both B's and C's trees but costs one scorer call; the whole queue costs two (A and B — nothing depends on C or D, so they are never scored). Returning candidates triggers no further scoring: by the first request, every probability the run needs is known.
+- **Exact ties need a fixed order**, or two runs over the same queue could propose different paths. Ties break, in order: higher probability, then fewer decisions taken, then the path that stays on likelier sides longer (at exactly 0.5, succeeds counts as likelier), then the batch's ID. The trace already used the last rule twice: the four roots tie at 1.0 and order as A, B, C, D; `B [A✓]` and `C [A✓]` tie at 0.9 and B goes first. The likelier-sides rule matters for batches whose decisions tie: two batches each with one ancestor at exactly 0.5 return as *first batch's likelier path, second batch's likelier path, first's other, second's other* — the batches interleave, so one batch's tied subtree cannot absorb the whole budget before the other is offered at all. Because each partial path is created exactly once, no two distinct entries ever compare fully equal: every step has a unique winner, so repeated runs agree.
+
+## Alternative considered: flip-based generation
+
+The current implementation reaches the same output from the opposite direction: it starts from each batch's single most likely path and produces every other path as a set of deviations ("flips") from it, ranked by what each flip costs in probability. It does slightly less priority-queue work per returned path. The tree was chosen because every piece of its state is directly meaningful — a partial path and its probability — while the flip formulation is legible only through derived bookkeeping (a preferred side per dependency, per-flip costs, rules to keep floating-point totals stable), which makes it harder to understand and maintain. Both return the same paths in the same order.
+
+## Implementation impact
+
+The generator package is rewritten around the tree vocabulary — root, decision, node, leaf, path — with no change to its interfaces, its place in the default Speculator, the entities, or the scorer contract. Ranking values are unchanged up to floating-point rounding (the same logs added in a different order), and the set of paths returned is identical; only the order of *exact* ties is governed by the rule above rather than the old formulation's tie rule.
+
+Deriving ancestors from direct edges reaches slightly beyond the generator, because a path carries one assumption per ancestor and everything that reads paths must agree on what the ancestors are:
+
+- One shared definition of "a batch's ancestors in this snapshot" is used both by the generator to shape trees and by the speculate controller to validate returned paths, so the two can never disagree.
+- The controller's read step follows chains of direct edges when collecting finalized batches that are still referenced, since a chain's older links no longer appear in any one batch's stored list.
+- The entity comment on the stored dependency list currently claims it holds the transitive closure; it holds direct conflicts, and the comment is corrected with the implementation.