[Epic]: Support Agent Skills via On-Demand SKILL.md Discovery

[sami-marreed]

Feature Request

Add support for Agent Skills — a dynamic, on-demand skill discovery and injection system powered by SKILL.md files. Instead of stuffing every possible instruction into the agent's initial system prompt (which wastes tokens and can confuse the model), cuga would use a lightweight skill registry pattern where full instructions are loaded only when needed.

The first reference skill to validate the system against is the Anthropic PPTX skill, which exercises the full breadth of what skill execution requires: Python scripts, Node.js tooling (pptxgenjs), and shell commands (LibreOffice, pdftoppm) — making it an ideal integration test for the entire skill pipeline.

---

Motivation / Problem

As cuga handles increasingly complex and varied user tasks, the system prompt grows unwieldy. Including every possible workflow guide upfront:

• Wastes context window tokens on irrelevant instructions
• Increases latency and cost per request
• Can confuse the agent with conflicting or noisy guidance

A skill system solves this by keeping the system prompt lean and loading task-specific instructions on-demand.

---

Use Case

As a developer using cuga for diverse tasks (git releases, deployments, presentations, code reviews, etc.), I want cuga to automatically discover relevant skill files in my project (.cuga/skills/) or globally (~/.config/cuga/skills/), surface them to the agent only when relevant, and inject full instructions only when the agent actually needs them — keeping every interaction fast, focused, and token-efficient.

---

Proposed Solution

A complete end-to-end skills flow:

Step 1: Skill Discovery

Before each session, cuga scans project-local (.cuga/skills/) and global (~/.config/cuga/skills/) directories for SKILL.md files. It reads the YAML frontmatter (name, description) of each file and injects a lightweight <available_skills> list into the agent's system prompt.

Step 2: Agent Decision & Tool Call

The agent analyzes the user's prompt and, if it matches a skill description, emits a tool call to load the full skill:

skill({ name: "pptx" })

Step 3: Permission Gate

Each skill can define a permission policy (e.g., ask, auto, deny). If set to ask, cuga pauses and requests human approval before loading or executing. This is the primary safety layer for sensitive skills.

Step 4: Context Injection

If permitted, cuga reads the full Markdown body of the matched SKILL.md and injects it into the agent's context. This file contains the heavy-lifting instructions: specific rules, workflow constraints, architecture guidelines, or exact shell commands.

Step 5: Execution

The agent executes the skill using its standard tools (bash, write, edit). These tools remain governed by their own permission settings (e.g., bash with ask mode still requires approval per command).

The PPTX skill is a great first test because it requires all three execution modalities:

• Python: python -m markitdown presentation.pptx, python scripts/thumbnail.py
• Node.js / npm: npm install -g pptxgenjs for creating slides from scratch
• Shell tools: soffice --headless --convert-to pdf, pdftoppm for rendering

Step 6: Sandbox Integration

Skill execution should respect the configured sandbox environment:

• Containerized sandboxes (e.g., rootless Podman/Docker via an entersh-style script): skill bash commands run inside the container, isolating them from the host OS
• Plugin sandboxes (e.g., Daytona, DevContainers): skills execute within the managed remote environment

---

Alternatives Considered

• Static system prompt expansion: Add all skill instructions upfront. Works but doesn't scale — wastes tokens and degrades quality as the number of skills grows.
• Manual user invocation: Require users to explicitly load a skill file. Removes the "automatic discovery" value and adds friction.
• LLM tool use without SKILL.md: Use generic tool calls with inline descriptions. Loses the ability for users to customize and extend skills via plain Markdown files in their own repo.

---

Priority

High - Important for my workflow

Implementation Complexity (if known)

Complex - Significant development effort

---

Additional Context

This pattern is directly inspired by the Cursor Agent Skills system, which uses a similar SKILL.md + on-demand injection model. The key differentiator for cuga is the permission gate (Step 3) and first-class sandbox integration (Step 6), making it safer for agentic use in production and CI environments.

The Anthropic PPTX skill is the reference implementation to validate against. It is representative of real-world skill complexity: multi-step QA loops, subagent delegation, cross-tool dependencies (Python + Node + system binaries), and a rich set of design guidelines injected as context.

Skill file example (.cuga/skills/pptx/SKILL.md frontmatter):

---
name: pptx
description: "Use this skill any time a .pptx file is involved — creating, reading, editing, or converting presentations."
permissions:
  bash: ask
---

---

Checklist

• I have searched existing issues and feature requests to ensure this is not a duplicate
• I have provided a clear use case and motivation for this feature
• I am willing to help test this feature once implemented
• I am interested in contributing to the implementation of this feature

[Harmedox]

Great proposal!

One small convention suggestion for the skill markdown format: consider encouraging a ## Policy section separate from instructions. Using the PPTX skill as an example:

---                                                                                         
name: pptx      
description: Generate PowerPoint presentations                                              
---                                                                                         
                                                                                            
## Policy                                                                                   
                
- Never execute downloaded scripts; only run bundled tooling                                
- Limit shell commands to python3 and node — no arbitrary binaries
- Do not embed external URLs as images without user confirmation                            
                                                                                            
## Instructions                                                                             
                                                                                            
1. Use python-pptx to generate slides...                                                    

This doesn't change the mechanics — both sections get injected in step 4 the same way. But the separation is useful for:

• Readability — skill authors can see at a glance what the agent must not do vs what it should do
• Portability — teams can review and audit constraints independently from implementation details
• Future enforceability — a structured policy section is parseable by tooling that might want to enforce constraints at runtime, rather than relying solely on the LLM's compliance

Right now the permission gate (step 3) governs whether a skill loads. But there's no equivalent for constraining behavior during execution. A ## Policy convention at least makes that gap visible and lays groundwork for addressing it.

[sami-marreed]

@coderabbitai Can u address how skills will be also handles in both local and prod mode, e.g. hosted? per thread etc with sandbox like https://github.com/alibaba/OpenSandbox

[coderabbitai]

Coding Plan

Summary

• Build a standalone SkillRegistry that parses SKILL.md files from configured directories, extracting YAML frontmatter for name/description summaries, a separate ## Policy section for behavioral constraints, and preserving remaining markdown as content for on-demand loading
• Inject skill summaries into the system prompt via a new template section, and provide a skill tool that loads full content (including policy) into conversation context using the existing per-turn injection pattern
• Parse permission requirements from the ## Policy section content rather than structured YAML frontmatter, keeping all policy information in the markdown body
• Add environment-aware execution support differentiating local (filesystem scan, shared registry) from hosted (tenant-scoped, potentially DB-backed) modes, with explicit OpenSandbox adapter integration

Design Choices

Design Choice 1: How should skill permissions interact with the existing tool approval system?

Options Considered:

1. Parse the ## Policy section for specific permission indicators (e.g., a Permission: ask line or pattern)
2. Default to auto-approve for skill loading, relying on existing ToolApproval policies for bash/write/edit operations
3. Create a simple convention where policy section starting with [ask] or similar marker indicates approval needed

Chosen Option: 1

Rationale: Parse the ## Policy section for permission indicators using a simple pattern match, falling back to auto if none found. This preserves the skill-level permission gate while keeping permissions in the markdown body as requested.

Design Choice 2: Where should available skills appear in the system prompt?

Options Considered:

1. In the ## What You Have Access To bullet list alongside tools and variables
2. As a new ## Available Skills section after ## Connected Applications
3. Via the special_instructions slot

Chosen Option: 2

Rationale: Create a dedicated ## Available Skills section after ## Connected Applications for clear separation and discoverability, following the existing structural pattern.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Skill Discovery Infrastructure

Build the foundational components for discovering, parsing, and caching SKILL.md files from configured directories. This phase establishes the skill registry that other components will consume.

Task 1: Add Skill Configuration Options

Add configuration settings to enable the skills feature and specify directories to scan for SKILL.md files.

• Add features.skills_enabled boolean flag with Validator default False in src/cuga/config.py
• Add advanced_features.skill_directories list with Validator default [] in src/cuga/config.py
• Add corresponding entries to settings.toml with documentation comments
• Include default paths logic: when skill_directories is empty and skills_enabled is true, default to [".cuga/skills", "~/.config/cuga/skills"]

Task 2: Create Skill Data Models

Define Pydantic models representing skill metadata and the parsed structure of a SKILL.md file with separate fields for content and policy.

• Create src/cuga/backend/skills/models.py with:
  • SkillMetadata model with name, description, file_path, directory fields (lightweight, for prompt summaries)
  • Skill model extending metadata with:
    • content: str — the main markdown body (instructions, examples, guidelines)
    • policy: Optional[str] — the extracted ## Policy section content, stored separately for future runtime enforcement
• No YAML frontmatter permission fields — permissions are parsed from the policy section content at runtime

Task 3: Implement SKILL.md Parser

Create a parser that extracts YAML frontmatter (name, description only), the ## Policy section, and remaining markdown content from SKILL.md files.

• Create src/cuga/backend/skills/parser.py with a parse_skill_file(path: Path) -> Skill function
• Use pyyaml or standard library to parse YAML frontmatter (content between --- delimiters) for name and description only
• Extract the ## Policy section (or ## Instructions as alternate heading) as a distinct field:
  • Search for heading matching ## Policy or ## Instructions (case-insensitive)
  • Extract content until the next ## heading or end of file
  • Store in Skill.policy field
• Store remaining markdown (excluding policy section) in Skill.content field
• Handle malformed files gracefully with warnings rather than failures
• Validate required fields (name, description) exist in frontmatter

Task 4: Implement Skill Registry

Create the core registry class that scans directories, maintains a cache of skill metadata, and provides lookup methods.

• Create src/cuga/backend/skills/registry.py with SkillRegistry class
• Implement scan_directories() method that walks configured paths looking for SKILL.md files (support both flat and nested structures like .cuga/skills/pptx/SKILL.md)
• Implement get_skill_summaries() -> List[SkillMetadata] returning lightweight metadata (no content) for prompt injection
• Implement get_skill(name: str) -> Optional[Skill] returning full skill with content
• Implement reload() method for hot-reloading during development
• Make registry a singleton or provide factory function accessible from graph nodes

🤖 Prompt for AI agents

Implement the foundational skill discovery infrastructure — configuration, data
models, parser, and registry — that all subsequent phases will depend on.

**Task 1: Configuration (`src/cuga/config.py` and `settings.toml`)**
- Add `features.skills_enabled` boolean with `Validator` default `False`
- Add `advanced_features.skill_directories` list with `Validator` default `[]`
- Add corresponding documented entries to `settings.toml`
- Add default paths logic: when `skill_directories` is empty and
`skills_enabled` is true, default to `[".cuga/skills", "~/.config/cuga/skills"]`

**Task 2: Data Models (`src/cuga/backend/skills/models.py`)**
- Create Pydantic model `SkillMetadata` with fields: `name`, `description`,
`file_path`, `directory` (lightweight, for prompt summaries)
- Create Pydantic model `Skill` extending `SkillMetadata` with:
  - `content: str` — the main markdown body (instructions, examples, guidelines)
- `policy: Optional[str]` — the extracted `## Policy` section content, stored
separately for future runtime enforcement
- Do NOT include YAML frontmatter permission fields — permissions are parsed
from the policy section content at runtime

**Task 3: SKILL.md Parser (`src/cuga/backend/skills/parser.py`)**
- Implement `parse_skill_file(path: Path) -> Skill`
- Parse YAML frontmatter delimited by `---` using `pyyaml` or the standard
library, extracting only `name` and `description`
- Extract the `## Policy` section (or `## Instructions` as an alternate heading,
case-insensitive):
  - Search for a heading matching `## Policy` or `## Instructions`
  - Extract content until the next `##` heading or end of file
  - Store in `Skill.policy`
- Store remaining markdown (excluding the policy section) in `Skill.content`
- Handle malformed files gracefully with warnings rather than raising exceptions
- Validate that required fields (`name`, `description`) exist in frontmatter

**Task 4: Skill Registry (`src/cuga/backend/skills/registry.py`)**
- Create `SkillRegistry` class
- Implement `scan_directories()` that walks configured paths for `SKILL.md`
files, supporting both flat and nested directory structures (e.g.,
`.cuga/skills/pptx/SKILL.md`)
- Implement `get_skill_summaries() -> List[SkillMetadata]` returning lightweight
metadata without content
- Implement `get_skill(name: str) -> Optional[Skill]` returning the full skill
including content and policy
- Implement `reload()` for hot-reloading during development
- Expose the registry as a singleton or via a factory function accessible from
graph nodes

Phase 2: Agent Integration

Integrate the skill system into the agent's prompt and execution flow, enabling the agent to see available skills and load them on demand.

Task 1: Create the Skill Tool

Implement the skill tool that the agent calls to load a skill's full content into context.

• Create src/cuga/backend/skills/skill_tool.py with the skill loading function
• Function signature: async def load_skill(name: str, state: CugaLiteState) -> str
• Lookup skill in registry; return error message if not found
• On success, store loaded skill in state metadata for context injection
• Return confirmation message indicating skill was loaded and is now active

Task 2: Register Skill Tool in Tool Context

Wire the skill tool into the agent's available tools so it can be called via the CodeAct pattern.

• Modify prepare_tools_and_apps() in src/cuga/backend/cuga_graph/nodes/cuga_lite/cuga_lite_graph.py
• When skills_enabled is true and skills are available, add the skill function to tools_context_dict
• Add a tool definition dict for skill to tools_for_prompt with name, description ("Load a skill's full instructions by name"), and parameters schema
• Wrap the skill function with make_tool_awaitable() like other tools

Task 3: Add Available Skills to System Prompt

Modify the system prompt template to display available skill summaries.

• Add available_skills parameter to create_mcp_prompt() in src/cuga/backend/cuga_graph/nodes/cuga_lite/prompt_utils.py
• Modify src/cuga/backend/cuga_graph/nodes/cuga_lite/prompts/mcp_prompt.jinja2:
  • Add {% if available_skills %} block after the ## Connected Applications section
  • Render as ## Available Skills with a list of skill name + description pairs
  • Include brief instruction: "Call skill(name="...") to load full instructions for a skill"
• Apply same changes to mcp_prompt_todos.jinja2 for consistency
• Pass skill summaries from registry in prepare_tools_and_apps()

Task 4: Inject Loaded Skill Content into Context

When a skill is loaded, inject its full markdown content (including policy section) into the conversation context following existing patterns.

• In call_model() within cuga_lite_graph.py, check state metadata for loaded skill
• If present, append ## Active Skill: {skill_name}\n{skill_content}\n\n## Skill Policy\n{skill_policy} to the last user message (inject both content and policy together)
• Use a flag (e.g., skill_content_injected) to avoid re-injection on subsequent turns
• Clear loaded skill from metadata after injection to allow loading different skills

Task 5: Update State Models for Skill Tracking

Extend CugaLiteState to track loaded skills and injection status with proper thread/session scoping.

• Add loaded_skill: Optional[str] field to cuga_lite_metadata schema or directly to CugaLiteState
• Add skill_content_injected: bool flag for tracking injection
• These fields are automatically thread/session-scoped because:
  • CugaLiteState is a Pydantic model instantiated per-subgraph invocation
  • State is checkpointed by LangGraph using the thread_id key passed via configurable
  • Each HTTP request receives a unique thread_id (from X-Thread-ID header or auto-generated UUID)
  • MemorySaver isolates checkpoints by thread_id, preventing cross-session state leakage
• In hosted multi-tenant deployments, thread_id uniqueness per request ensures skill state does not bleed across users or sessions

🤖 Prompt for AI agents

Integrate the skill system into the agent's prompt and execution flow so the
agent can discover available skills and load them on demand.

**Task 1: Skill Tool (`src/cuga/backend/skills/skill_tool.py`)**
- Implement `async def load_skill(name: str, state: CugaLiteState) -> str`
- Look up the skill in the registry; return an error message if not found
- On success, store the loaded skill in state metadata for later context
injection
- Return a confirmation message indicating the skill was loaded and is now
active

**Task 2: Register Skill Tool
(`src/cuga/backend/cuga_graph/nodes/cuga_lite/cuga_lite_graph.py`)**
- In `prepare_tools_and_apps()`, when `skills_enabled` is true and skills are
available:
  - Add the `skill` function to `tools_context_dict`
- Add a tool definition dict for `skill` to `tools_for_prompt` with name,
description (`"Load a skill's full instructions by name"`), and parameters
schema
- Wrap the skill function with `make_tool_awaitable()` following the pattern
used for other tools

**Task 3: System Prompt Template Updates**
- In `src/cuga/backend/cuga_graph/nodes/cuga_lite/prompt_utils.py`, add
`available_skills` parameter to `create_mcp_prompt()`
- In `src/cuga/backend/cuga_graph/nodes/cuga_lite/prompts/mcp_prompt.jinja2`:
- After the `## Connected Applications` section, add a `{% if available_skills
%}` block
- Render a `## Available Skills` section listing each skill's name and
description
- Include the instruction: `Call \`skill(name="...")\` to load full instructions
for a skill`
- Apply the same changes to `mcp_prompt_todos.jinja2`
- Pass skill summaries from the registry inside `prepare_tools_and_apps()`

**Task 4: Context Injection (`cuga_lite_graph.py` — `call_model()`)**
- Check state metadata for a loaded skill each turn
- If present and not yet injected, append `## Active Skill:
{skill_name}\n{skill_content}\n\n## Skill Policy\n{skill_policy}` to the last
user message (inject both content and policy together)
- Set a `skill_content_injected` flag to prevent re-injection on subsequent
turns
- Clear the loaded skill from metadata after injection to allow loading
different skills

**Task 5: State Model Updates (`CugaLiteState`)**
- Add `loaded_skill: Optional[str]` to `cuga_lite_metadata` or directly to
`CugaLiteState`
- Add `skill_content_injected: bool` tracking flag
- Note that these fields are automatically thread/session-scoped via LangGraph's
`thread_id` checkpoint mechanism (`MemorySaver` isolates checkpoints by
`thread_id`)
- In hosted multi-tenant deployments, per-request `thread_id` uniqueness ensures
no cross-session state leakage

Phase 3: Permission Gate

Integrate skill loading with the existing permission system to enable human approval for sensitive skills, parsing permission requirements from the policy section content.

Task 1: Implement Permission Parser for Policy Section

Create a utility to extract permission requirements from the ## Policy section content.

• Create src/cuga/backend/skills/permissions.py with permission parsing logic
• Define SkillPermission enum: AUTO, ASK, DENY
• Implement parse_permission(policy_content: Optional[str]) -> SkillPermission:
  • Search for patterns like Permission: ask, Requires approval, [ask] at start of policy
  • Search for Permission: deny or [deny] patterns
  • Default to AUTO if no permission indicator found
• Keep parsing simple and well-documented so skill authors understand the convention

Task 2: Generate Tool Approval Policies from Skill Permissions

Auto-generate ToolApproval policies for skills that require permission (ASK mode).

• In SkillRegistry, after scanning and parsing, determine permission for each skill using the permission parser
• For skills with ASK permission, generate ToolApproval policy objects
• Set required_tools: ["skill"] and configure the policy to match when the skill name is in the code
• Store generated policies in a way that PolicyAgent can access them (register with PolicyStorage or create a skill-specific policy provider)
• Set appropriate approval_message using skill name and description

Task 3: Implement Skill Permission Check in Tool Handler

Add permission checking to the skill tool that triggers HITL interrupt when required.

• Before loading skill content in load_skill(), retrieve the skill and parse its permission
• If permission is ASK, integrate with ToolApprovalHandler to create an approval interrupt
• If permission is DENY, return an error message without loading
• If permission is AUTO (default), proceed without interruption
• Reuse existing approval action creation from followup_model.py for consistent UI

Task 4: Handle Skill Tool Approval Resumption

Ensure skill loading completes correctly after user approval.

• Extend ToolApprovalHandler.handle_approval_resumption() or create parallel handler for skill approvals
• On approval, complete the skill loading and context injection
• On denial, return appropriate message to agent indicating skill access was denied
• Clean up approval metadata after resolution

🤖 Prompt for AI agents

Integrate skill loading with the existing HITL permission system to gate
sensitive skill activation through user approval, using parsed permissions from
the `## Policy` section content.

**Task 1: Permission Parser (`src/cuga/backend/skills/permissions.py`)**
- Define `SkillPermission` enum with values: `AUTO`, `ASK`, `DENY`
- Implement `parse_permission(policy_content: Optional[str]) ->
SkillPermission`:
- Search for patterns like `Permission: ask`, `Requires approval`, `[ask]` at
start of policy → return `ASK`
  - Search for `Permission: deny` or `[deny]` patterns → return `DENY`
  - Default to `AUTO` if no permission indicator is found
- Keep parsing logic simple and document conventions clearly for skill authors

**Task 2: Auto-generate Tool Approval Policies
(`src/cuga/backend/skills/registry.py`)**
- After scanning and parsing in `SkillRegistry`, determine each skill's
permission using `parse_permission()` on its policy field
- For skills with `ASK` permission, generate `ToolApproval` policy objects
- Set `required_tools: ["skill"]` and configure the policy to match when the
skill name is present in the code
- Register policies with `PolicyStorage` or provide a skill-specific policy
provider accessible by `PolicyAgent`
- Set `approval_message` using the skill's name and description

**Task 3: Permission Check in Skill Tool
(`src/cuga/backend/skills/skill_tool.py`)**
- At the start of `load_skill()`, retrieve the skill and call
`parse_permission()` on its policy field
- `DENY`: return an error message immediately without loading
- `ASK`: integrate with `ToolApprovalHandler` to create an approval interrupt,
reusing approval action creation from `followup_model.py` for consistent UI
- `AUTO` (default): proceed without interruption

**Task 4: Approval Resumption Handling**
- Extend `ToolApprovalHandler.handle_approval_resumption()` or create a parallel
handler for skill approvals
- On approval: complete the skill loading and trigger context injection
- On denial: return an appropriate message to the agent indicating skill access
was denied
- Clean up all approval metadata after resolution

Phase 4: Environment-Aware Execution

Ensure skill execution works correctly across deployment modes (local vs hosted) and integrates with sandbox environments including OpenSandbox.

Task 1: Local vs Hosted Skill Registry Scoping

Adapt the skill registry to work correctly in both local and multi-tenant hosted deployments.

• In local mode (storage.mode = "local"):
  • SkillRegistry scans filesystem paths directly (.cuga/skills/, ~/.config/cuga/skills/)
  • Skills are loaded once at startup and cached in-memory
  • Single shared registry instance for the process
• In hosted mode (storage.mode = "prod"):
  • Consider tenant isolation requirements — skills may be tenant-specific
  • Add tenant_id scoping to registry lookups using get_tenant_id() from config
  • Option A: Store skill metadata in the relational DB (PolicyStorage pattern) with tenant scoping
  • Option B: Keep filesystem-based but scope directories per tenant (e.g., /skills/{tenant_id}/)
  • Start with Option B for simplicity, document path to Option A for full DB-backed skills

Task 2: Thread-Scoped Skill Registry Access

Ensure skill registry access in graph nodes respects per-session context.

• Create factory function get_skill_registry(config: RunnableConfig) -> SkillRegistry
• In prepare_tools_and_apps(), access registry via factory with current config
• For local mode, return singleton instance (all sessions share same skills)
• For hosted mode, return tenant-scoped instance using tenant_id from configurable or state.service_scope
• This mirrors the PolicyConfigurable.from_config() pattern already in use

Task 3: OpenSandbox Adapter Integration

Add explicit support for OpenSandbox as a sandbox backend with mapping of skill operations to its API.

• Create src/cuga/backend/cuga_graph/nodes/cuga_lite/executors/opensandbox/opensandbox_executor.py
• Subclass RemoteExecutor with implementations for execute_for_cuga_lite and execute_for_code_agent
• Map skill operations to OpenSandbox API:
  • bash commands → sandbox.commands.run() — execute shell commands
  • File writes → sandbox.files.write_files() — write generated files
  • Python code execution → CodeInterpreter.codes.run() — execute Python in isolated environment
  • Cleanup → sandbox.kill() — terminate sandbox after execution
• Add routing in CodeExecutor:
  • Add _opensandbox_executor singleton and _get_opensandbox_executor() factory
  • Extend mode literal type to include 'opensandbox'
  • Add config flag advanced_features.opensandbox_sandbox for auto-selection

Task 4: Skill Execution Environment Documentation

Document how skills execute across different sandbox backends.

• Add src/cuga/backend/skills/README.md documenting:
  • Skill file format (YAML frontmatter, ## Policy section, content body)
  • Permission conventions (how to indicate ask/deny in policy section)
  • Execution flow across sandbox backends (local exec, E2B, Docker, OpenSandbox)
  • Multi-tenant considerations for hosted deployments
• Skills execute via existing bash/write/edit tools — no special skill-specific execution path
• Sandbox backend selection is orthogonal to skill system; skills work with any configured backend

🤖 Prompt for AI agents

Ensure the skill system works correctly across all deployment modes and sandbox
backends, and add an OpenSandbox executor adapter.

**Task 1: Local vs Hosted Registry Scoping**
- In local mode (`storage.mode = "local"`):
- `SkillRegistry` scans filesystem paths directly (`.cuga/skills/`,
`~/.config/cuga/skills/`)
- Load once at startup, cache in-memory, use a single shared instance per
process
- In hosted mode (`storage.mode = "prod"`):
- Add `tenant_id` scoping to registry lookups via `get_tenant_id()` from config
- Implement Option B: scope filesystem directories per tenant (e.g.,
`/skills/{tenant_id}/`)
- Document the path to Option A (full DB-backed skills via `PolicyStorage`
pattern) for future work

**Task 2: Thread-Scoped Registry Access**
- Create factory function `get_skill_registry(config: RunnableConfig) ->
SkillRegistry`
- In `prepare_tools_and_apps()`, access the registry via this factory using the
current config
- For local mode, return the singleton instance
- For hosted mode, return a tenant-scoped instance using `tenant_id` from
`configurable` or `state.service_scope`
- Mirror the `PolicyConfigurable.from_config()` pattern already in use

**Task 3: OpenSandbox Executor
(`src/cuga/backend/cuga_graph/nodes/cuga_lite/executors/opensandbox/opensandbox_executor.py`)**
- Subclass `RemoteExecutor` and implement `execute_for_cuga_lite` and
`execute_for_code_agent`
- Map operations to OpenSandbox API:
  - `bash` commands → `sandbox.commands.run()`
  - File writes → `sandbox.files.write_files()`
  - Python code execution → `CodeInterpreter.codes.run()`
  - Cleanup → `sandbox.kill()`
- In `CodeExecutor`:
- Add `_opensandbox_executor` singleton and `_get_opensandbox_executor()`
factory
  - Extend the mode literal type to include `'opensandbox'`
  - Add config flag `advanced_features.opensandbox_sandbox` for auto-selection

**Task 4: Documentation (`src/cuga/backend/skills/README.md`)**
- Document the SKILL.md file format: YAML frontmatter, `## Policy` section, and
content body
- Document permission conventions for indicating `ask`/`deny` in the policy
section
- Document execution flow across sandbox backends: local exec, E2B, Docker,
OpenSandbox
- Document multi-tenant considerations for hosted deployments
- Note that skills execute via existing bash/write/edit tools — no special
skill-specific execution path — and sandbox backend selection is orthogonal to
the skill system

Research

The cuga agent is built on LangGraph with a CodeAct execution pattern where the LLM generates Python code that is executed in a sandbox. The main execution loop lives in cuga_lite_graph.py with three core nodes: prepare_tools_and_apps (builds the system prompt and tool context), call_model (LLM interaction with per-turn context injection), and sandbox (code execution). System prompts are Jinja2 templates (mcp_prompt.jinja2) with slots for tools, apps, and special instructions. Context is injected mid-conversation by appending sections like ## Task Guidance or ## Available Variables to user messages. The policy system handles tool approval via ToolApproval policies that trigger HITL interrupts. State is scoped per-session via LangGraph's thread_id checkpoint mechanism using MemorySaver, and configuration uses Dynaconf with validators in config.py. The sandbox system supports multiple backends (local exec, E2B, Docker) routed through CodeExecutor.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Implement the foundational skill discovery infrastructure — configuration, data
models, parser, and registry — that all subsequent phases will depend on.

**Task 1: Configuration (`src/cuga/config.py` and `settings.toml`)**
- Add `features.skills_enabled` boolean with `Validator` default `False`
- Add `advanced_features.skill_directories` list with `Validator` default `[]`
- Add corresponding documented entries to `settings.toml`
- Add default paths logic: when `skill_directories` is empty and
`skills_enabled` is true, default to `[".cuga/skills", "~/.config/cuga/skills"]`

**Task 2: Data Models (`src/cuga/backend/skills/models.py`)**
- Create Pydantic model `SkillMetadata` with fields: `name`, `description`,
`file_path`, `directory` (lightweight, for prompt summaries)
- Create Pydantic model `Skill` extending `SkillMetadata` with:
  - `content: str` — the main markdown body (instructions, examples, guidelines)
- `policy: Optional[str]` — the extracted `## Policy` section content, stored
separately for future runtime enforcement
- Do NOT include YAML frontmatter permission fields — permissions are parsed
from the policy section content at runtime

**Task 3: SKILL.md Parser (`src/cuga/backend/skills/parser.py`)**
- Implement `parse_skill_file(path: Path) -> Skill`
- Parse YAML frontmatter delimited by `---` using `pyyaml` or the standard
library, extracting only `name` and `description`
- Extract the `## Policy` section (or `## Instructions` as an alternate heading,
case-insensitive):
  - Search for a heading matching `## Policy` or `## Instructions`
  - Extract content until the next `##` heading or end of file
  - Store in `Skill.policy`
- Store remaining markdown (excluding the policy section) in `Skill.content`
- Handle malformed files gracefully with warnings rather than raising exceptions
- Validate that required fields (`name`, `description`) exist in frontmatter

**Task 4: Skill Registry (`src/cuga/backend/skills/registry.py`)**
- Create `SkillRegistry` class
- Implement `scan_directories()` that walks configured paths for `SKILL.md`
files, supporting both flat and nested directory structures (e.g.,
`.cuga/skills/pptx/SKILL.md`)
- Implement `get_skill_summaries() -> List[SkillMetadata]` returning lightweight
metadata without content
- Implement `get_skill(name: str) -> Optional[Skill]` returning the full skill
including content and policy
- Implement `reload()` for hot-reloading during development
- Expose the registry as a singleton or via a factory function accessible from
graph nodes
===============================================================================

Task: 2

Integrate the skill system into the agent's prompt and execution flow so the
agent can discover available skills and load them on demand.

**Task 1: Skill Tool (`src/cuga/backend/skills/skill_tool.py`)**
- Implement `async def load_skill(name: str, state: CugaLiteState) -> str`
- Look up the skill in the registry; return an error message if not found
- On success, store the loaded skill in state metadata for later context
injection
- Return a confirmation message indicating the skill was loaded and is now
active

**Task 2: Register Skill Tool
(`src/cuga/backend/cuga_graph/nodes/cuga_lite/cuga_lite_graph.py`)**
- In `prepare_tools_and_apps()`, when `skills_enabled` is true and skills are
available:
  - Add the `skill` function to `tools_context_dict`
- Add a tool definition dict for `skill` to `tools_for_prompt` with name,
description (`"Load a skill's full instructions by name"`), and parameters
schema
- Wrap the skill function with `make_tool_awaitable()` following the pattern
used for other tools

**Task 3: System Prompt Template Updates**
- In `src/cuga/backend/cuga_graph/nodes/cuga_lite/prompt_utils.py`, add
`available_skills` parameter to `create_mcp_prompt()`
- In `src/cuga/backend/cuga_graph/nodes/cuga_lite/prompts/mcp_prompt.jinja2`:
- After the `## Connected Applications` section, add a `{% if available_skills
%}` block
- Render a `## Available Skills` section listing each skill's name and
description
- Include the instruction: `Call \`skill(name="...")\` to load full instructions
for a skill`
- Apply the same changes to `mcp_prompt_todos.jinja2`
- Pass skill summaries from the registry inside `prepare_tools_and_apps()`

**Task 4: Context Injection (`cuga_lite_graph.py` — `call_model()`)**
- Check state metadata for a loaded skill each turn
- If present and not yet injected, append `## Active Skill:
{skill_name}\n{skill_content}\n\n## Skill Policy\n{skill_policy}` to the last
user message (inject both content and policy together)
- Set a `skill_content_injected` flag to prevent re-injection on subsequent
turns
- Clear the loaded skill from metadata after injection to allow loading
different skills

**Task 5: State Model Updates (`CugaLiteState`)**
- Add `loaded_skill: Optional[str]` to `cuga_lite_metadata` or directly to
`CugaLiteState`
- Add `skill_content_injected: bool` tracking flag
- Note that these fields are automatically thread/session-scoped via LangGraph's
`thread_id` checkpoint mechanism (`MemorySaver` isolates checkpoints by
`thread_id`)
- In hosted multi-tenant deployments, per-request `thread_id` uniqueness ensures
no cross-session state leakage
===============================================================================

Task: 3

Integrate skill loading with the existing HITL permission system to gate
sensitive skill activation through user approval, using parsed permissions from
the `## Policy` section content.

**Task 1: Permission Parser (`src/cuga/backend/skills/permissions.py`)**
- Define `SkillPermission` enum with values: `AUTO`, `ASK`, `DENY`
- Implement `parse_permission(policy_content: Optional[str]) ->
SkillPermission`:
- Search for patterns like `Permission: ask`, `Requires approval`, `[ask]` at
start of policy → return `ASK`
  - Search for `Permission: deny` or `[deny]` patterns → return `DENY`
  - Default to `AUTO` if no permission indicator is found
- Keep parsing logic simple and document conventions clearly for skill authors

**Task 2: Auto-generate Tool Approval Policies
(`src/cuga/backend/skills/registry.py`)**
- After scanning and parsing in `SkillRegistry`, determine each skill's
permission using `parse_permission()` on its policy field
- For skills with `ASK` permission, generate `ToolApproval` policy objects
- Set `required_tools: ["skill"]` and configure the policy to match when the
skill name is present in the code
- Register policies with `PolicyStorage` or provide a skill-specific policy
provider accessible by `PolicyAgent`
- Set `approval_message` using the skill's name and description

**Task 3: Permission Check in Skill Tool
(`src/cuga/backend/skills/skill_tool.py`)**
- At the start of `load_skill()`, retrieve the skill and call
`parse_permission()` on its policy field
- `DENY`: return an error message immediately without loading
- `ASK`: integrate with `ToolApprovalHandler` to create an approval interrupt,
reusing approval action creation from `followup_model.py` for consistent UI
- `AUTO` (default): proceed without interruption

**Task 4: Approval Resumption Handling**
- Extend `ToolApprovalHandler.handle_approval_resumption()` or create a parallel
handler for skill approvals
- On approval: complete the skill loading and trigger context injection
- On denial: return an appropriate message to the agent indicating skill access
was denied
- Clean up all approval metadata after resolution
===============================================================================

Task: 4

Ensure the skill system works correctly across all deployment modes and sandbox
backends, and add an OpenSandbox executor adapter.

**Task 1: Local vs Hosted Registry Scoping**
- In local mode (`storage.mode = "local"`):
- `SkillRegistry` scans filesystem paths directly (`.cuga/skills/`,
`~/.config/cuga/skills/`)
- Load once at startup, cache in-memory, use a single shared instance per
process
- In hosted mode (`storage.mode = "prod"`):
- Add `tenant_id` scoping to registry lookups via `get_tenant_id()` from config
- Implement Option B: scope filesystem directories per tenant (e.g.,
`/skills/{tenant_id}/`)
- Document the path to Option A (full DB-backed skills via `PolicyStorage`
pattern) for future work

**Task 2: Thread-Scoped Registry Access**
- Create factory function `get_skill_registry(config: RunnableConfig) ->
SkillRegistry`
- In `prepare_tools_and_apps()`, access the registry via this factory using the
current config
- For local mode, return the singleton instance
- For hosted mode, return a tenant-scoped instance using `tenant_id` from
`configurable` or `state.service_scope`
- Mirror the `PolicyConfigurable.from_config()` pattern already in use

**Task 3: OpenSandbox Executor
(`src/cuga/backend/cuga_graph/nodes/cuga_lite/executors/opensandbox/opensandbox_executor.py`)**
- Subclass `RemoteExecutor` and implement `execute_for_cuga_lite` and
`execute_for_code_agent`
- Map operations to OpenSandbox API:
  - `bash` commands → `sandbox.commands.run()`
  - File writes → `sandbox.files.write_files()`
  - Python code execution → `CodeInterpreter.codes.run()`
  - Cleanup → `sandbox.kill()`
- In `CodeExecutor`:
- Add `_opensandbox_executor` singleton and `_get_opensandbox_executor()`
factory
  - Extend the mode literal type to include `'opensandbox'`
  - Add config flag `advanced_features.opensandbox_sandbox` for auto-selection

**Task 4: Documentation (`src/cuga/backend/skills/README.md`)**
- Document the SKILL.md file format: YAML frontmatter, `## Policy` section, and
content body
- Document permission conventions for indicating `ask`/`deny` in the policy
section
- Document execution flow across sandbox backends: local exec, E2B, Docker,
OpenSandbox
- Document multi-tenant considerations for hosted deployments
- Note that skills execute via existing bash/write/edit tools — no special
skill-specific execution path — and sandbox backend selection is orthogonal to
the skill system

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- @coderabbitai You can skip phase 3. Add a simple unit test case for phase 2.
- @coderabbitai For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!

[sami-marreed]

@coderabbitai here's a sample how to use OpenSandbox

import asyncio
from datetime import timedelta

from code_interpreter import CodeInterpreter, SupportedLanguage
from opensandbox import Sandbox
from opensandbox.models import WriteEntry

async def main() -> None:
# 1. Create a sandbox
sandbox = await Sandbox.create(
"opensandbox/code-interpreter:v1.0.2",
entrypoint=["/opt/opensandbox/code-interpreter.sh"],
env={"PYTHON_VERSION": "3.11"},
timeout=timedelta(minutes=10),
)

async with sandbox:

    # 2. Execute a shell command
    execution = await sandbox.commands.run("echo 'Hello OpenSandbox!'")
    print(execution.logs.stdout[0].text)

    # 3. Write a file
    await sandbox.files.write_files([
        WriteEntry(path="/tmp/hello.txt", data="Hello World", mode=644)
    ])

    # 4. Read a file
    content = await sandbox.files.read_file("/tmp/hello.txt")
    print(f"Content: {content}") # Content: Hello World

    # 5. Create a code interpreter
    interpreter = await CodeInterpreter.create(sandbox)

    # 6. Execute Python code (single-run, pass language directly)
    result = await interpreter.codes.run(
          """
              import sys
              print(sys.version)
              result = 2 + 2
              result
          """,
          language=SupportedLanguage.PYTHON,
    )

    print(result.result[0].text) # 4
    print(result.logs.stdout[0].text) # 3.11.14

# 7. Cleanup the sandbox
await sandbox.kill()

if name == "main":
asyncio.run(main())

[coderabbitai]

I've updated the plan above based on your feedback. Ready to proceed when you are!