Evolve Agent v0.3: evaluation-driven evolution

.github/workflows/evolve-agent-ci.yml

@@ -0,0 +1,31 @@
+name: evolve-agent-ci
+
+on:
+  push:
+    paths:
+      - "evolve-agent/**"
+      - ".github/workflows/evolve-agent-ci.yml"
+  pull_request:
+    paths:
+      - "evolve-agent/**"
+      - ".github/workflows/evolve-agent-ci.yml"
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: evolve-agent
+    strategy:
+      matrix:
+        node-version: [22, 24]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: ${{ matrix.node-version }}
+      - run: npm install --no-audit --no-fund
+      - run: npm run check

evolve-agent/.env.example

@@ -0,0 +1,69 @@
+# Model
+OPENAI_API_KEY=
+OPENAI_MODEL=gpt-5.6-sol
+OPENAI_VERIFIER_MODEL=gpt-5.6-sol
+EVOLVE_REASONING_EFFORT=high
+
+# State and workspace
+# EVOLVE_HOME must remain outside EVOLVE_WORKSPACE. When omitted, Evolve Agent
+# derives a per-workspace directory under XDG_STATE_HOME or ~/.local/state.
+# EVOLVE_HOME=/absolute/path/outside/the/workspace
+EVOLVE_WORKSPACE=.
+
+# Hardened execution — Docker is the default and fails closed until a pinned
+# image is both allowlisted and already present locally.
+EVOLVE_EXECUTOR=docker
+EVOLVE_DOCKER_BINARY=docker
+# Example shape only: repository/name@sha256:<64 lowercase hex characters>
+# EVOLVE_DOCKER_DEFAULT_IMAGE=node:22-bookworm-slim@sha256:...
+# EVOLVE_DOCKER_ALLOWED_IMAGES=node:22-bookworm-slim@sha256:...
+EVOLVE_DOCKER_ALLOWED_NETWORKS=
+EVOLVE_DOCKER_USER=65532:65532
+EVOLVE_DOCKER_MAX_MEMORY_MB=2048
+EVOLVE_DOCKER_MAX_CPUS=2
+EVOLVE_DOCKER_MAX_PIDS=256
+EVOLVE_DOCKER_MAX_TMPFS_MB=256
+EVOLVE_DOCKER_REQUIRE_ROOTLESS=false
+
+# Host execution is an explicit unsafe escape hatch.
+EVOLVE_ALLOW_LOCAL_EXECUTOR=false
+
+# Process boundary
+EVOLVE_ALLOWED_COMMANDS=git,node,npm,npx,pnpm,python,python3,pytest,vitest,tsc
+EVOLVE_NON_INTERACTIVE=false
+
+# Only names in this list can be materialized as short-lived read-only files.
+EVOLVE_SECRET_ALLOWLIST=
+EVOLVE_SECRET_TTL_MS=300000
+
+# Crash and duplicate-run protection
+EVOLVE_LEASE_TTL_MS=30000
+EVOLVE_LEASE_HEARTBEAT_MS=10000
+
+# Evaluation automation is conservative by default. Fixture capture and shadow
+# traffic are opt-in. Promotion is always explicit; rollback may be automatic.
+EVOLVE_EVAL_CAPTURE_COMMITTED=false
+EVOLVE_EVAL_SHADOW_PERCENT=0
+EVOLVE_EVAL_MONITOR_PROMOTED=true
+
+# Offline evaluation policy
+EVOLVE_EVAL_MIN_FIXTURES=3
+EVOLVE_EVAL_REPEATS=1
+EVOLVE_EVAL_MAX_NEW_FAILURES=0
+EVOLVE_EVAL_MAX_SUCCESS_REGRESSION=0
+EVOLVE_EVAL_MAX_SCORE_REGRESSION=0.02
+EVOLVE_EVAL_MIN_SUCCESS_IMPROVEMENT=0.05
+EVOLVE_EVAL_MIN_SCORE_IMPROVEMENT=0.02
+EVOLVE_EVAL_MAX_TOKEN_RATIO=1.2
+EVOLVE_EVAL_MAX_TOOL_RATIO=1.2
+EVOLVE_EVAL_EFFICIENCY_RATIO=0.9
+EVOLVE_EVAL_CONFIDENCE=0.9
+EVOLVE_EVAL_BOOTSTRAP_SAMPLES=1000
+
+# Shadow canary and production rollback envelope
+EVOLVE_EVAL_CANARY_MIN_SAMPLES=5
+EVOLVE_EVAL_MONITOR_MIN_SAMPLES=10
+EVOLVE_EVAL_MONITOR_WINDOW=50
+EVOLVE_EVAL_MONITOR_MAX_SUCCESS_DROP=0.1
+EVOLVE_EVAL_MONITOR_MAX_SCORE_DROP=0.1
+EVOLVE_EVAL_MONITOR_MAX_TOKEN_RATIO=1.5

evolve-agent/.gitignore

@@ -0,0 +1,8 @@
+node_modules/
+dist/
+.test-dist/
+coverage/
+.env
+.evolve/
+*.log
+.DS_Store

evolve-agent/CHANGELOG.md

@@ -0,0 +1,65 @@
+# Changelog
+
+## 0.3.0 — Evaluation-Driven Evolution
+
+### Added
+
+- content-addressed replay fixtures captured from clean committed Episodes
+- supporting-Episode leakage exclusion and frozen training provenance
+- paired baseline-versus-candidate replay harness
+- exact proposal-trace matching and executed-argument evidence validation
+- success, verifier, token, tool-call, duration, trace, and safety metrics
+- deterministic paired bootstrap confidence intervals
+- configurable non-regression and improvement gates
+- Ed25519-signed offline, canary, and monitor reports
+- shadow canaries that cannot alter production answers
+- rolling production monitoring against a signed canary envelope
+- automatic rollback on production regression
+- `quarantined` Skill state
+- evaluation fixture, report, shadow, monitor, verify, and promotion CLI commands
+- evaluation authority enforcement inside `SkillStore`
+- 12 new evaluation and integration tests, bringing the suite to 38 tests
+
+### Changed
+
+- package version raised to 0.3.0
+- manual canary-score recording replaced by report-backed evaluation flow
+- promotion now requires verified signed offline and canary reports
+- runtime checkpoints record active Skill IDs and final verifier score
+- committed Episode capture and shadow sampling are opt-in; promoted-Skill monitoring remains enabled by default
+
+### Security
+
+- direct low-level promotion fails closed without a signed-report verifier
+- report tampering, Skill/report identity mismatch, authority-key mismatch, and policy-hash mismatch block promotion
+- fixture imports and captures validate evidence provenance and integrity
+- candidate training Episodes cannot approve the candidate they produced
+
+## 0.2.0 — Hardened Execution
+
+### Added
+
+- executor interface with Docker and explicit local backends
+- immutable image and network policy objects
+- Docker resource and privilege hardening
+- short-lived file secret broker with output redaction
+- execution receipts with command and policy hashes
+- Episode leases, heartbeats, duplicate-run protection, and stale recovery
+- `doctor`, `executors list`, and `secrets sweep` diagnostics
+- dedicated hardened-execution design documentation
+- 14 security tests, bringing the suite to 26 tests
+
+### Changed
+
+- Docker became the default process executor
+- local execution became disabled by default
+- state home defaults outside the workspace and inside-workspace state is rejected
+- truncated process output terminates execution and marks it unsuccessful
+
+## 0.1.0 — Evidence-Gated Kernel
+
+- bounded autonomous loop
+- evidence artifacts and hash-chained ledger
+- exact approval capabilities
+- independent final verification
+- governed memory and candidate Skill lifecycle

evolve-agent/CONTRIBUTING.md

@@ -0,0 +1,14 @@
+# Contributing
+
+1. Create a focused branch and keep unrelated application changes out of the PR.
+2. Add or update tests for every invariant touched.
+3. Run `npm run check` and `npm pack --dry-run` before publishing.
+4. Keep tool, executor, fixture, evaluator, and promotion permissions narrow and fail closed.
+5. Do not add unrestricted shell execution, silent approval bypasses, implicit image pulls, broad network defaults, automatic Skill promotion, or unsigned promotion shortcuts.
+6. A mutating tool must define risk class, validation, evidence output, and rollback behavior.
+7. An executor must state which isolation properties it actually enforces and encode them in its receipt.
+8. An evaluation metric must define its direction, pairing unit, failure semantics, sample requirements, and regression threshold.
+9. Evaluation fixtures must preserve provenance and must not reuse a candidate's supporting Episodes as approval data.
+10. Never put secret values, private signing keys, or production credentials in arguments, logs, artifacts, fixtures, tests, or committed environment files.
+11. Keep agent authority and evaluation state outside task-mounted workspaces.
+12. Document the trusted computing base and residual risk; do not describe containers, model verification, or signed reports as stronger guarantees than they provide.

evolve-agent/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 DCLXAI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.