Span of control for hierarchical LLM agent trees: the data, a queueing model, and the reference bound behind shipping ToolsEnabled agent trees four wide.
ToolsEnabled runs agents as trees. A coordinator hands work to its direct reports, and a report can coordinate reports of its own. The deployed product allowed eight live direct reports per coordinator, four levels deep: 1 + 8 + 64 + 512 = 585 agents.
That is a fan-out bound, the same shape as one gate output driving eight inputs. I asked which ideas from electrical engineering could help, and what the bound actually controls.
Fan-out caps how many reports a coordinator drives. Reports also send results, questions and blockers back up. Seen from the coordinator, that traffic is fan-in: N sources, one server, one message per turn.
Hardware handles fan-in with interrupt controllers and handshakes:
- buffered request lines;
- status registers;
- arbitration;
- direct data movement;
- coalescing;
- ready/valid backpressure.
The lesson carries over: a cap on the number of sources does not limit their traffic.
- The bound. At most eight live children per parent, four levels deep. A child holds a seat only while it is starting or running; drafts and finished agents hold none. The cap is checked where a seat is claimed, not only where a child is added.
- The controller/manager role split. Five design choices went into scope. This is the only one that has shipped. The other four did not ship, and eight research questions stayed queued.
The data come from one deployed tree: 36 agents over 11.2 hours on 2026-09-10/11. This is operational data, not a controlled experiment.
| Measure | Value |
|---|---|
| Wait before a manager's turn picked up a message | median 708 s, p90 1,976 s (n = 166) |
| Same wait, messages to workers | median 3 s (n = 308) |
| Accepted messages to one manager that never became a turn | 108 |
| Busy share of alive time, Claude agents | managers 0.909, workers 0.601 |
That manager heard from 14 agents in 6.0 alive hours. The bound held it to eight live reports at a time, but it did not limit the messages. Provider usage limits and a 30-minute turn timeout stopped agents long before the tree came near 585.
This is a model, not a measurement. It is a seeded M/G/1 queue of one coordinator, fed by the measured producer rate (5.67 messages per hour) and the measured coordinator turn times.
| Producers per coordinator | 2 | 4 | 8 | 13 |
|---|---|---|---|---|
| Utilization, producers only | 0.27 | 0.55 | 1.09 | 1.77 |
| Median wait in an 11.2 h window (s) | 0 | 14 | 1,730 | 8,706 |
Past a utilization of 1, the queue grows for as long as the load lasts. Adding the coordinator's measured other work puts four producers at 0.99.
On 2026-09-11 I decided to ship with a width of four and unchanged depth, which is 85 agents per tree. I deferred the fuller 8:1 implementation and the research behind it.
The evidence is the measurements above and a draft, unreviewed span rule:
- At most four busy producers per coordinator held at four.
- The same rule failed at thirteen.
- Nothing in between was measured.
The width change was in development when this was written.
- EE-Q01 Priority-based attention arbitration.
- EE-Q02 Coalescing routine updates.
- EE-Q03 Capacity-aware dispatch.
- EE-Q04 Semantic context pruning.
- EE-Q05 Is eight the right bound?
- EE-Q06 Fixed per-child service slots.
- EE-Q07 Whole-group completion barriers.
- EE-Q08 Mandatory delegation quota.
| Path | Contents |
|---|---|
data/ |
Text-free timing and state records, the codebook, and summary.py, which recomputes every figure above |
model/ |
The queueing model and its tests |
code/ |
The seat-counting bound (live children only) with tests; MIT excerpt from the ToolsEnabled app |
paper/ |
The preprint: Fan-Out Is Not Fan-In: Span of Control for Hierarchical LLM Agent Trees |
To reproduce everything:
python3 data/summary.py
python3 -m unittest discover -s data -p 'test_*.py'
python3 model/coordinator_queue.py
python3 -m unittest discover -s model -p 'test_*.py'
cd code && node --test
The scripts need Python 3 and Node 20 or later, with no dependencies. The raw session logs are private and are not included; data/CODEBOOK.md explains how the records were derived.
MIT. See LICENSE, NOTICE and CITATION.cff.
Josh Pinckard · josh@toolsenabled.ai · https://toolsenabled.ai