Agent-Failure-Modes

A runnable repository for handling three most common AI agent failure modes.

If you're building (or reviewing) an agent that can call tools on its own, this repo is for you. It's built around a simple helpdesk triage agent using Microsoft Semantic Kernel in C#/.NET. The agent logic is trivial on purpose, so all your attention goes to the safeguards wrapped around it.

🎓 The solutions in this repo are simplified so the idea is easy to see. For converting this to production code, see Key takeaways / production notes

---

What can go wrong with your AI agent?

• The agent won't stop. It keeps retrying the same failing action, burning tokens and API calls, because nothing tells it "this isn't working, stop and hand off."
• The agent forgets what matters. As a conversation grows, something has to give way to fit the context window and if you truncate blindly, the very first instruction (often the most important one) is usually the first thing dropped.
• The agent reaches for more than it should. A confused model, a manipulated prompt, or a buggy tool-selection step can lead an agent to attempt an action it's not supposed to do like delete a record, grant a permission.

These are the most common failure modes that must be handled before shipping any AI agent in prod. This repo shows you how to add those guardrails in plain C# code.

What you'll learn

• How to wrap an agent's reasoning loop with an iteration/token/time budget and a stall detector, so it always resolves to continue, exit, or escalate-to-human
• How to keep critical instructions alive in a long conversation using pinned facts, rolling summarization, and token-budget-based trimming.
• How to enforce least privilege for agent tool calls with a deny-by-default allow-list, a permission ceiling, and short-lived, scoped credential leases.
• How to structure this kind of safety code so it's a genuine choke point the agent can't talk its way around.

The base example

The scenario is an IT helpdesk triage agent: it looks up tickets, resets passwords, and escalates to a human when it's stuck. All of its tools are mock stubs — LookupTicket, ResetPassword, and EscalateToHuman return fake strings, and nothing here talks to a real ticketing system. There's also one deliberately sensitive pair of tools, DeleteUser / GrantAdmin, included only so the permission-ceiling safeguard has something dangerous to deny.

   ┌──────────┐        ┌───────────────────┐        ┌────────────────────┐
   │  Agent   │──1──▶  │   LoopGuard        │──2──▶  │  PermissionGate     │
   │ (Semantic│        │ (budget + stall    │        │ (allow-list +       │
   │  Kernel) │        │  detection)        │        │  ceiling + lease)   │
   └────┬─────┘        └───────────────────┘        └─────────┬───────────┘
        │                                                       │
        │        ┌────────────────────────┐                    │
        └───3───▶│   ContextManager        │◀───────────────────┘
                  │ (pinned facts + rolling │
                  │  summarization)         │
                  └────────────┬────────────┘
                                │
                                ▼
                     ┌────────────────────┐
                     │   Mock Helpdesk     │
                     │   Tools (stubs)     │
                     └────────────────────┘

1. Every loop iteration is reported to the LoopGuard before another iteration is allowed to run.
2. Every tool call is checked by the PermissionGate before it's actually invoked.
3. Every turn added to the conversation passes through the ContextManager, which keeps pinned facts alive and summarizes the rest once it's over budget.

The three failure modes

1. Task loop failure

The problem: an agent stuck trying (and re-trying) something it can't actually accomplish, with no mechanism to notice and stop.

The technique: a LoopGuard tracks iteration count, a token/time budget, and repeated ("stalled") tool calls, and returns a structured decision — Continue, Exit, or EscalateToHuman — every single iteration.

// src/Safeguards/LoopControl/LoopGuard.cs
var guard = new LoopGuard(new LoopBudget { MaxIterations = 4, StallThreshold = 2 });
var evaluation = guard.Evaluate("lookup_ticket(TCK-1042)", tokensUsedThisStep: 80);
// evaluation.Decision => Continue | Exit | EscalateToHuman

📖 Deep dive: docs/01-task-loop-failure.md

2. Context drop

The problem: long conversations exceed the context/token budget, and naive truncation drops the oldest turn — often exactly where the critical instruction lives.

The technique: a ContextManager pins "always-keep" facts, keeps the most recent turns verbatim, and rolls the middle of the conversation into a summary instead of deleting it.

// src/Safeguards/ContextManagement/ContextManager.cs
context.PinFact(TurnRole.System, "SYSTEM/CRITICAL: never auto-close this VIP ticket.");
context.AddTurn(TurnRole.User, "...");
Console.WriteLine(context.EnforceBudget()); // summarizes older turns once over budget

📖 Deep dive: docs/02-context-drop.md

3. Permission escalation

The problem: an agent (confused, manipulated, or buggy) attempts an action like deleting a user, granting admin. Something it should not be able to perform autonomously.

The technique: a PermissionGate enforces a deny-by-default allow-list plus a hard permission ceiling for sensitive tools, and requires a live, correctly-scoped, non-expired CredentialLease for every call.

// src/Safeguards/Permissions/PermissionGate.cs
var gate = new PermissionGate(allowList: new[] { "reset_password", "delete_user" },
                               sensitiveCeiling: new[] { "delete_user" });
gate.TryAuthorize("delete_user", lease); // => DeniedByCeiling, even though it's allow-listed

📖 Deep dive: docs/03-permission-escalation.md

Repository structure

Agent-Failure-Modes/
├── README.md                     # you are here
├── Agent-Failure-Modes.sln        # solution wiring all projects together
├── src/
│   ├── Agent/                    # simple Semantic Kernel triage agent (SK wiring + config)
│   ├── Tools/                    # mock helpdesk tools (stubs, incl. sensitive ones)
│   └── Safeguards/
│       ├── LoopControl/          # iteration/budget + stall detection + exit/escalate decision
│       ├── ContextManagement/    # pinned facts + summarization + token-budget trimming
│       └── Permissions/          # allow-list + permission ceiling + short-lived credential lease
├── samples/                      # runnable demos, one per failure mode
│   ├── LoopFailureDemo/
│   ├── ContextDropDemo/
│   └── PermissionEscalationDemo/
└── docs/                         # one explainer markdown per failure mode
    ├── 01-task-loop-failure.md
    ├── 02-context-drop.md
    └── 03-permission-escalation.md

Prerequisites

• .NET SDK 8.0 or later.

• No API key required to run the three safeguard demos. They're designed to run fully offline using scripted/deterministic data, so you can see every safeguard fire without spending a cent or configuring anything.

• If you want to wire the Agent project up to a real Semantic Kernel model, set these environment variables (there are no secrets checked into this repo):

  Variable	Purpose	Default
  AGENT_MODEL_ID	Chat completion model/deployment name	gpt-4o-mini (small & cheap — swap to gpt-4o for a more capable model)
  AGENT_API_KEY	OpenAI-compatible API key	(none — required only for live calls)
  AGENT_ENDPOINT	Optional custom endpoint, e.g. Azure OpenAI	(none — defaults to public OpenAI)

How to run

# Build everything
dotnet build

# Safeguard #1 — watch the loop budget count down and escalate to a human
dotnet run --project samples/LoopFailureDemo

# Safeguard #2 — watch a long conversation get summarized while a pinned
# critical instruction survives to the end
dotnet run --project samples/ContextDropDemo

# Safeguard #3 — watch safe tool calls succeed, sensitive ones get denied by
# the permission ceiling, and an expired credential lease get rejected
dotnet run --project samples/PermissionEscalationDemo

Each demo prints a narrated, step-by-step trace to the console. You'll see the budget or token count ticking down, the exact decision made at each step, and a final summary explaining what just happened and why it matters.

Key takeaways / production notes

These are teaching illustrations, not a production-ready SDK. Before adapting this pattern for real systems, consider hardening:

• LoopGuard: feed it real token usage from your model provider's response metadata, and make EscalateToHuman actually notify a human (Slack/Teams/PagerDuty) instead of just logging.
• ContextManager: replace the toy string-concatenation summarizer with a real (ideally cheap) summarization model call, and use your provider's actual tokenizer instead of the length / 4 heuristic.
• PermissionGate / CredentialBroker: back the credential lease with a real short-lived-credential mechanism (cloud IAM temp tokens, Vault/Key Vault dynamic secrets, short-exp signed JWTs), and log every permission decision for auditability.
• All three safeguards should be tested independently of the LLM.

Further reading

• Microsoft Semantic Kernel — the agent orchestration framework used here.
• docs/01-task-loop-failure.md, docs/02-context-drop.md, docs/03-permission-escalation.md — the deep-dive explainer for each safeguard.

Contributing

Issues and PRs that improve clarity, fix bugs, or add another well-scoped failure mode/safeguard are welcome.

License

No license file is currently included. Treat this repository as "all rights reserved" for reuse beyond personal learning until a license is added.