Skip to content

Updated agent prompt: avoid unnecessary implementation details in response, don't always create gadgets - #489

Open
jonesphillip wants to merge 5 commits into
mainfrom
pjones/agent-context
Open

Updated agent prompt: avoid unnecessary implementation details in response, don't always create gadgets#489
jonesphillip wants to merge 5 commits into
mainfrom
pjones/agent-context

Conversation

@jonesphillip

Copy link
Copy Markdown
Contributor

Noticing a couple of things in agent behavior that can be improved:

  • Responses often include phrases like like "binding" or "X (bindings)'s API". Those are implementation details that describe why agent is calling tools. We don't need the agent to describe implementation details to users unless prompted to do so.
  • We are a little too quick to create gadgets. If a user wants help writing an email, we should respond in text instead of building gadget.

The goal of this PR is to make the agent communicate clearly for nontechnical users, avoid unnecessary implementation details, and use better judgment about when to create a gadget.

I ran 14 scenarios across Sonnet 4.6, GPT-5.6 Sol, and GLM 5.3, comparing the original and updated prompts (summary below). Overall saw improvement. Specifically saw fewer unnecessary gadgets and more direct answers in chat where warranted. Also saw less unprompted narration about bindings and APIs describing how agent was getting answers for users.

Scenario Result What changed
Simple greeting Improved All three models gave a neutral greeting without steering users toward building gadgets.
Summarize supplied information Baseline
One-off calculation Baseline
Read a connected spreadsheet Improved Sonnet removed setup narration and delivered the analysis directly.
Request document access Baseline
Explicit gadget creation Improved GLM described useful features without explaining storage or callbacks.
Update an existing gadget Baseline
Technical details when requested Baseline
Vague request for expense help Improved All three avoided creating an app immediately.
Spawned agent reporting results Improved Sonnet led with the result and removed narration about bindings and APIs.
Draft a email follow-up Improved Sonnet drafted the email instead of creating an email generator; GLM dropped its unsolicited gadget offer.
Query top CRM opportunities Improved GLM returned the correct results without suggesting a dashboard gadget.
Create a slide deck Improved Sonnet and GLM gave shorter confirmations focused on the slides they created.
Interpret a bare company name Improved Sol and GLM asked about the user’s broader intent instead of assuming they wanted an app.

…ontechnical users, avoid unnecessary implementation details, and use better judgment about when to create a gadget
@github-actions github-actions Bot added the kernel Changes to the Workshop kernel label Sep 13, 2026
@ask-bonk

ask-bonk Bot commented Sep 13, 2026

Copy link
Copy Markdown
  • [P2] Preserve prompt precedencepackages/workshop-backend/src/agent.ts:2506: shared guidance is appended after deployment instructions, creating unresolved conflicts with admin guardrails. Place it before instanceInstructions, which explicitly defer to preceding instructions.

github run

@github-actions

Copy link
Copy Markdown

Preview: pr489-pjones-agent-context

https://pr489-pjones-agent-context-router.cloudflare-os-previews.workers.dev

Dashboard · deleted when this PR closes

@ask-bonk

ask-bonk Bot commented Sep 13, 2026

Copy link
Copy Markdown

LGTM!

github run

Comment thread packages/workshop-backend/src/agent.ts Outdated
@ask-bonk

ask-bonk Bot commented Sep 14, 2026

Copy link
Copy Markdown

LGTM!

github run

@ndisidore ndisidore left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to lean the prompt less coding specific

Comment thread packages/workshop-backend/src/agent.ts Outdated
Co-authored-by: Nathan Disidore <nathan@cloudflare.com>
@jonesphillip
jonesphillip marked this pull request as ready for review September 14, 2026 16:13

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@ask-bonk

ask-bonk Bot commented Sep 14, 2026

Copy link
Copy Markdown
  • [P1] Escape template-literal backtickspackages/workshop-backend/src/agent.ts:684: raw backticks around createGadget terminate SYSTEM_PROMPT, causing a TypeScript syntax error. Use \createGadget``.

github run

@AshishKumar4

Copy link
Copy Markdown
Contributor

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kernel Changes to the Workshop kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants