I build AI agent systems end to end: the tools they call, the guardrails that decide whether they hold, and the measurement that says whether it works.

Most of what's here was built while running an agent against a live broker order API, on its own channel, with no human approving each action. That setting is useful mainly because it prices its own mistakes. A guard that can be routed around gets routed around, and you find out the same week. The repositories below are the parts of that work that stand on their own.

Every one runs offline, with no API key, and says what it doesn't prove.

blind-agent-eval An evaluation harness that cannot leak the answer to the model it's evaluating. It replays a historical day to an LLM agent one bar at a time and scores every decision against what happened next. The agent never sees a future bar — not because it's asked not to look, but because the future bars are never handed to the function that builds what it reads. The side channel is the harder half: a decider is an agent with a shell, so the auditor greps the decider's own transcript for forbidden reads inside the decision window and fails closed. run_demo.sh --leak plants a cheater and shows it caught, named and timestamped. 42 + 24 + 10 + 6 assertions · stdlib only · synthetic corpus, no market data

holdout-your-metrics Anything that optimizes against your evaluation suite will eventually learn the suite instead of the thing the suite stands for, and when it does, every number you're looking at goes up. So split the checks into trained and holdout, show the optimizer only the trained half, and measure both. The draw is hash-chained before the first cycle runs. The demo makes the point in six lines: two runs both reach a perfect trained score, and only one is killed. 53 tests · 0.19s · no dependencies

agent-loop-fenced An autonomous research loop, and the fence that keeps its objective from being gamed by the agent optimising against it. Hand an optimiser a number and ask it to raise the number, and every split that number is computed on becomes training data. Ships both arrangements so the fence is demonstrable rather than asserted: the unfenced run reports 102.39 and is earned on six trades, surviving audit on two. 82 tests · a corrected statistic, with its broken predecessor pinned by a regression test

prereg-harness Specifications frozen by SHA-256 in an append-only, hash-chained registry before anything runs. A null verdict is refused at the write unless it carries the smallest effect it could have detected, because "no edge found" has to mean "no edge above X found". 68-check mutation suite · 34 further tests · chain verifies from genesis

when-the-metric-lies A four-day futures backtest reported a profit on trades that had in fact lost money. Dropped UDP packets left cancelled orders resting in the reconstructed book, so the engine kept filling against liquidity that was no longer there, and every standard defence in front of it passed. The writeup derives why an error of that shape can only ever flatter the result, and ships the three guards built afterwards. 51 tests · 0.09s · stdlib only

agent-guardrails An agent that can take an irreversible action needs a gate that isn't the agent's own judgement. A send guard that denies across four tool surfaces at once while leaving drafting alone. A test scoper that picks suites from the git diff rather than from whatever felt relevant. A write guard that makes a named set of files unwritable to the process — and covers subprocess, because an agent that shells out walks straight past a guard that only knows Python's open().

llm-pipeline-checked A nine-pass pipeline turning a lecture video into a knowledge graph, arranged so no model output reaches the next model until code has checked it. A claim the extractor invented gets re-derived from the artifacts on disk, and becomes a fail row instead of a graph node. 13 tests · 2.7s · mock backend, no API key

said-so A check that refuses to believe an agent's sentence. It shipped the afternoon an agent wrote "next wake 14:45" and never called the scheduler.

nris A retrieval system over a research-paper corpus, built by reading the retrieval literature and implementing it: RAPTOR, STORM, PaperQA2's reranking pass, HyDE. Cost governance sits in the call path rather than in a policy document, so every paid call prices itself first, caches, and stops to ask above a threshold. 171 tests · sockets disabled in the harness, so no test can reach the network

oas-execute-mcp A safety layer between a language model and a broker order. Seven tools over a swappable backend, with the failure modes tested rather than assumed.

harness-scheduler-probe Mines an agent runtime's own session transcripts to compare every scheduling request against what actually arrived. It found the runtime silently rewrites any request under 60 seconds and quantizes delivery to the next whole minute — which makes the largest legal request the worst one to make. Then it corrects for it. 20 tests · runs on a clean machine with no prior state

These are components of a larger system that runs against a broker API. The execution loop, the broker adapter, the account state and the risk fuses aren't published and won't be, because they're inseparable from the credentials they run under. What's here is the part that stands on its own. Read access to more can be arranged for a serious conversation.

Also on this profile: nouns-rot-verbs-hold, a field report on what held and what didn't when an agent decided and submitted orders without per-trade approval. And mole-deep, a mobile-first game in one self-contained HTML file, no build step, written in a month in a domain I hadn't worked in.

📍 Eibar, Basque Country · CET · remote · zorionarrillaga@gmail.com