Problem
The agent benchmark driver only knows two arms, stim and control. Expo published @expo/agent-cli (1.0.16, 2026-09-16), an agent-facing CLI over Expo CLI / EAS CLI / expo-doctor with status, dev --ios, smoke, and agents:setup. There is no way to measure it against Stim with the existing readiness protocol without hand-editing the driver, so the comparison cannot be reproduced or audited.
Evidence
scripts/agent-benchmark/driver.mjs gates dispatch, smoke, prompts, PATH, skill profile, watcher device selection, collection, cleanup, and report pairing on arm === 'stim' / arm === 'control'; run-guards.mjs hardcodes arms = new Set(['stim', 'control']) for targets.json.npm view @expo/agent-cli version → 1.0.16; expo-agent-cli dev --help shows it opens the app on the booted simulator, so the control device rules (create and boot a named iPhone 17 / iOS 26.5 simulator) still apply.expo/skills ships no agent-cli skill; the agent guidance agents:setup installs is the managed AGENTS.md block plus help workflow.
Cause
Arm identity is a two-value enum spread across the driver rather than a description of what differs between arms (tool on PATH, guidance, prompt line, device rules).
Fix idea
Add an expo-agent-cli arm that inherits every control rule (no Stim on PATH, same sandbox policy, benchmark-named simulator, agent-device proof, control cleanup) and differs only by a pinned expo-agent-cli shim on PATH, a guidance file rendered from the agents:setup managed block and help workflow, and the prompt line telling the agent to use it in place of Expo CLI. Report pairs stim against any non-stim arm. Validate with one iOS JavaScript cell per arm on a root that reuses the 1.0.0 golden.
Problem
The agent benchmark driver only knows two arms,
stimandcontrol. Expo published@expo/agent-cli(1.0.16, 2026-09-16), an agent-facing CLI over Expo CLI / EAS CLI / expo-doctor withstatus,dev --ios,smoke, andagents:setup. There is no way to measure it against Stim with the existing readiness protocol without hand-editing the driver, so the comparison cannot be reproduced or audited.Evidence
scripts/agent-benchmark/driver.mjsgatesdispatch,smoke, prompts, PATH, skill profile, watcher device selection, collection, cleanup, and report pairing onarm === 'stim'/arm === 'control';run-guards.mjshardcodesarms = new Set(['stim', 'control'])fortargets.json.npm view @expo/agent-cli version→1.0.16;expo-agent-cli dev --helpshows it opens the app on the booted simulator, so the control device rules (create and boot a named iPhone 17 / iOS 26.5 simulator) still apply.expo/skillsships no agent-cli skill; the agent guidanceagents:setupinstalls is the managedAGENTS.mdblock plushelp workflow.Cause
Arm identity is a two-value enum spread across the driver rather than a description of what differs between arms (tool on PATH, guidance, prompt line, device rules).
Fix idea
Add an
expo-agent-cliarm that inherits every control rule (no Stim on PATH, same sandbox policy, benchmark-named simulator, agent-device proof, control cleanup) and differs only by a pinnedexpo-agent-clishim on PATH, a guidance file rendered from theagents:setupmanaged block andhelp workflow, and the prompt line telling the agent to use it in place of Expo CLI. Report pairsstimagainst any non-stim arm. Validate with one iOS JavaScript cell per arm on a root that reuses the 1.0.0 golden.