Runtime security governance for AI agents using agentsh v0.18.3 with Runloop Devboxes.
Runloop provides isolation. agentsh provides governance.
Runloop Devboxes give AI agents a secure, isolated compute environment. But isolation alone doesn't prevent an agent from:
- Exfiltrating data to unauthorized endpoints
- Accessing cloud metadata (AWS/GCP/Azure credentials at 169.254.169.254)
- Leaking secrets in outputs (API keys, tokens, PII)
- Running dangerous commands (sudo, ssh, kill, nc)
- Reaching internal networks (10.x, 172.16.x, 192.168.x)
- Deleting workspace files permanently
agentsh adds the governance layer that controls what agents can do inside the sandbox, providing defense-in-depth:
+---------------------------------------------------------+
| Runloop Devbox (Isolation) |
| +---------------------------------------------------+ |
| | agentsh (Governance) | |
| | +---------------------------------------------+ | |
| | | AI Agent | | |
| | | - Commands are policy-checked | | |
| | | - Network requests are filtered | | |
| | | - File I/O is policy-enforced | | |
| | | - Secrets are redacted from output | | |
| | | - All actions are audited | | |
| | +---------------------------------------------+ | |
| +---------------------------------------------------+ |
+---------------------------------------------------------+
| Runloop Provides | agentsh Adds |
|---|---|
| Compute isolation | Seccomp BPF syscall interception (execve, file, signal) |
| Process sandboxing | File I/O policy (seccomp file_monitor + Linux permissions) |
| API access to sandbox | Domain allowlist/blocklist |
| Persistent environment | Cloud metadata blocking |
| Command blocking (shell shim + seccomp execve + BASH_ENV) | |
| Signal blocking (kill/tgkill/tkill via seccomp BPF) | |
| Secret detection and redaction (DLP) | |
| Shell RC file persistence prevention | |
| Hostname/identity spoofing protection | |
| Soft-delete file quarantine | |
| LLM request auditing + rate limiting | |
| Threat intelligence feeds (URLhaus) | |
| Package install security scanning (OSV) | |
| Transparent command unwrapping | |
| Dangerous binary removal (chmod 000) | |
| Complete audit logging |
- Python 3 with
pip install runloop-api-client - Runloop account and API key
git clone https://github.com/canyonroad/agentsh-runloop.git
cd agentsh-runloop
pip install runloop-api-client
# Set your API key
export RUNLOOP_API_KEY="your-api-key"
# Run the security demo (73 tests)
python example.pyagentsh replaces /bin/bash with a shell shim that routes every command through the policy engine:
execute_sync runs: /bin/bash -lc "sudo whoami"
|
v
+-------------------+
| Shell Shim | /bin/bash -> agentsh-shell-shim
| (intercepts) |
+--------+----------+
|
v
+-------------------+
| agentsh server | Policy evaluation
| (auto-started) | + network proxy
+--------+----------+
|
+------+------+
v v
+----------+ +----------+
| ALLOW | | BLOCK |
| exit: 0 | | exit: 126|
+----------+ +----------+
Every command that Runloop's execute_sync() executes is automatically intercepted -- no explicit agentsh exec calls needed. The AGENTSH_SHIM_FORCE=1 environment variable ensures the shim routes through agentsh even without a TTY (Runloop runs commands via HTTP API).
This configuration targets agentsh v0.18.3. That release keeps the shell-shim and seccomp enforcement path used here, and also restores fine-grained file-rule enforcement for agentsh wrap by emitting the same file-monitor seccomp configuration as the exec path.
| Capability | Status | Notes |
|---|---|---|
| seccomp_user_notify | Working | Atomic syscall interception via agentsh-unixwrap wrapper |
| seccomp BPF | Working | Signal blocking (kill/tgkill/tkill), dangerous syscall blocking |
| file_monitor | Working | openat write interception, shell RC deny, workspace allow |
| cgroups_v2 | Disabled | v0.18.0+ probe rejects Runloop's cgroup delegation; disabled in config |
| capabilities_drop | Working | Full CapBnd, privilege reduction |
| FUSE | Detected, not mounting | /dev/fuse exists (mode 0600), deferred chmod via sudo |
| Landlock | Not available | CONFIG_SECURITY_LANDLOCK not enabled in kernel |
| eBPF | Not available | Missing CAP_BPF |
| pid_namespace | Not available | Host PID namespace |
Kernel: 6.18.5 | Latest v0.18.3 harness run: 42/73 passing | Enforcement: seccomp_user_notify (atomic, zero race conditions)
- Seccomp BPF -- kill/tgkill/tkill and dangerous syscalls blocked at kernel level (returns EPERM)
- seccomp_user_notify -- execve interception blocks sudo, ssh, nsenter etc. via
agentsh-unixwrapwrapper - file_monitor -- openat write interception denies .bashrc/.profile modification, enforces file policy
- chmod 000 -- privilege escalation binaries (sudo, su, kill, nsenter, etc.) made non-executable in launch_commands
- Shell shim --
/bin/bashreplaced with policy-enforcing shim - BASH_ENV -- bash builtins (kill, enable, ulimit) disabled
- HTTPS_PROXY -- network domain/CIDR filtering
- Linux permissions -- non-root user can't write system paths
With agentsh v0.18.3, 42 of 73 Runloop harness checks pass with the current configuration. The failing checks are caused by the stricter shell command pre-check added in the v0.18 release line: Runloop's default execute_sync() path invokes /bin/bash -lc ..., and v0.18.3 fails closed with shellc-wrapper-bypass before the command body runs. These improvements would add additional defense-in-depth layers after the Runloop shell invocation path is adjusted:
Current state: /dev/fuse exists but with mode 0600 (root-only). agentsh uses deferred mode with sudo chmod 666 /dev/fuse, but the FUSE mount doesn't trigger.
What it unlocks: VFS-level file interception, soft-delete quarantine (recoverable file deletion), symlink escape prevention.
How to improve: Create /dev/fuse with mode 0666 during container initialization.
Current state: /sys/kernel/security/landlock/ does not exist. Kernel 6.18.5 supports Landlock, but CONFIG_SECURITY_LANDLOCK is not enabled.
What it unlocks: Kernel-level filesystem and network restrictions as defense-in-depth.
How to enable: Rebuild kernel with CONFIG_SECURITY_LANDLOCK=y and add landlock to LSM boot parameters.
Current state: CAP_BPF not available.
What it unlocks: Kernel-level network monitoring.
How to enable: Grant CAP_BPF to container.
Current state: Disabled in config.yaml (sandbox.cgroups.enabled: false). agentsh v0.18.0 introduced a stricter cgroups probe that requires proper cgroup delegation (writable cgroup.subtree_control, leaf-move on EBUSY). Runloop's container cgroup setup doesn't satisfy this, so the probe fails closed and the server refuses to start if enabled.
What it unlocks: Per-command CPU/memory/PID limits enforced by the kernel (Runloop already enforces resource limits at the Devbox level).
How to enable: Configure Runloop Devboxes with delegated cgroup subtrees so non-root processes can write cgroup.subtree_control (standard systemd pattern).
Security policy is defined in two files:
config.yaml-- Server configuration: seccomp (unix_sockets wrapper, file_monitor, syscall blocking), network interception, DLP patterns, LLM proxy with rate limiting, FUSE settings, env_inject (BASH_ENV for builtin blocking), threat intelligence feeds, package install scanning, capability dropdefault.yaml-- Policy rules: command rules, network rules, file rules, transparent command unwrapping, hostname/shell RC protection
See the agentsh documentation for the full policy reference.
agentsh-runloop/
├── Dockerfile # Blueprint image with agentsh v0.18.3 on Ubuntu 24.04
├── config.yaml # Server config (seccomp, file_monitor, DLP, network, threat feeds, rate limits)
├── default.yaml # Security policy (commands, network, files, deny-before-allow ordering)
└── example.py # Python SDK integration tests (73 tests, 11 categories)
The example.py script creates a Runloop Blueprint and Devbox, then runs 73 security tests across 11 categories. It exits non-zero if any check fails.
- Diagnostics -- agentsh version, server health, shell shim, FUSE mount, BASH_ENV, HTTPS_PROXY, config files, agentsh detect
- AI agent protection -- rm -rf, data exfiltration, reverse shell, ssh blocked
- Cloud infrastructure -- AWS/GCP metadata, private networks, Kubernetes API blocked
- Multi-tenant isolation -- sudo, su, nsenter, docker, pkill, kill blocked (seccomp + chmod)
- File access control -- workspace/tmp writes allowed; /etc, /usr/bin, /var writes blocked
- Filesystem protection -- cp/touch/tee/mkdir to protected paths blocked; Python read/write to /etc, /usr/bin, /root blocked; symlink escape blocked
- Multi-context blocking -- env/xargs/find -exec/nested script/Python subprocess sudo blocked (atomic seccomp)
- Allowed operations -- echo, ls, git, bash, npm registry access verified
- Soft delete -- file create, rm, verify gone, trash list
- New security features (v0.11+) -- transparent command unwrapping (nice/nohup sudo blocked, nice ls allowed)
- v0.16.9 hardening -- hostname/domainname blocked, shell RC writes blocked (.bashrc/.profile), kill syscall blocked via seccomp BPF
export RUNLOOP_API_KEY="your-api-key"
python example.pyThe latest full run against agentsh v0.18.3 completed all 73 checks with 42 passing and 31 failing. Every failed check was denied before execution by rule=shellc-wrapper-bypass, including safe commands such as echo, ls, and diagnostics. This points to the Runloop shell launch shape (bash -lc) rather than an agentsh package install issue: the Docker build verifies the image installs agentsh 0.18.3+5f9af81.
| Property | Value |
|---|---|
| Base OS | Ubuntu 24.04 |
| Kernel | 6.18 |
| Package Manager | apt (deb) |
| User | user (uid 4444) |
| Workspace | /home/user |
| Git | /usr/bin/git |
| Python | python3 available |
MIT