Problem
In the Inspect smoke test on beets repo, the agent registered all GT tools (groundtruth_brief, groundtruth_search, groundtruth_context) but only called groundtruth_brief once — at step 15 of 15.
The agent preferred its built-in file reading over GT tools for the first 14 steps, exploring the repo manually instead of using the pre-indexed knowledge graph.
Root Cause
System prompt doesn't emphasize GT tool usage strongly enough. The agent treats GT tools as optional, not primary.
Fix
Add explicit instruction to system prompt: "Before exploring files manually, always call groundtruth_brief first to get repo overview and key files."
Priority: Critical — this determines whether GT adds value at all
Category: Prompt Engineering
Problem
In the Inspect smoke test on beets repo, the agent registered all GT tools (
groundtruth_brief,groundtruth_search,groundtruth_context) but only calledgroundtruth_briefonce — at step 15 of 15.The agent preferred its built-in file reading over GT tools for the first 14 steps, exploring the repo manually instead of using the pre-indexed knowledge graph.
Root Cause
System prompt doesn't emphasize GT tool usage strongly enough. The agent treats GT tools as optional, not primary.
Fix
Add explicit instruction to system prompt: "Before exploring files manually, always call groundtruth_brief first to get repo overview and key files."
Priority: Critical — this determines whether GT adds value at all
Category: Prompt Engineering