FlowEvo

Self-Evolving Agents through the Co-Evolution of Workflows and Executable Skills

Published as a conference paper at the Third Conference on Language Modeling (COLM) 2026.

Paper: COLM 2026 (OpenReview)  |  arXiv:2607.21596

FlowEvo is a training-free framework for agents that improve over time by compiling successful execution traces into reusable, directly-executable skills, then routing future tasks to the cheapest-yet-reliable path: direct skill replay, skill-conditioned workflow generation, or pure dynamic planning. A governance layer continuously evaluates whether injected skills help or hurt and suppresses those that cause negative transfer.

This repository contains the public release of the FlowEvo codebase accompanying the paper.

Repository layout

• src/agent/ — planner, generator, executor, retriever
• src/compiler/ — trace-to-skill compilation and admission
• src/memory/ — skill registry, policy, template, primitive, and trace stores
• src/governance/ — contrastive evaluation and utility scoring
• src/maintenance/ — governance kernel that coordinates lifecycle updates
• src/runtime/ — LLM client, generation settings, and config loader
• src/core/ — shared schemas and utilities
• src/env/ — sandbox helpers for code execution
• src/eval/ — benchmark runner (runner.py) and verifier (verifier.py)
• src/alfworld_/ — ALFWorld environment adapter, executor, compiler, and validation entry (run_20task_validation.py)
• src/code_math/ — HumanEval / MBPP / GSM8K / MATH benchmark runner
• configs/ — runtime configuration templates

Setup

python -m venv .venv
source .venv/bin/activate
pip install -e .

Configuring the LLM backend

The runtime talks to any OpenAI-compatible chat-completions endpoint through the openrouter provider. configs/default.yaml targets openai/gpt-4o-mini via OpenRouter, the shared backbone used in the paper; point base_url / model at another endpoint to switch backbones.

Create a local override that is never committed:

cp configs/local.example.yaml configs/local.yaml
# edit configs/local.yaml to set your api_key

The API key can also be supplied via the OPENROUTER_API_KEY environment variable.

Local / self-hosted models

The runtime speaks the OpenAI chat-completions protocol, so any compatible server (vLLM, llama.cpp, LM Studio, Ollama, …) works by pointing base_url at it in configs/local.yaml:

llm:
  provider: openai_compatible   # `openrouter` is accepted as an alias
  base_url: http://localhost:8000/v1
  model: my-served-model-name   # api_key may be omitted for local servers

Reasoning models

Models that emit hidden reasoning (Nemotron, DeepSeek-R1, Qwen-thinking, …) spend output tokens before writing the answer. With the default budgets (draft.max_output_tokens: 900; ALFWorld 256/500/200) the reply can come back with empty content and the runtime raises LLMClientError: … returned empty content (finish_reason=length …). Raise max_output_tokens (and the llm.alfworld budgets), or set grow_on_truncation: true to let the client retry with a doubled budget. Both are off by default so published numbers are unaffected; see configs/local.example.yaml.

Quick check

All runners accept --limit N; a five-task run verifies the setup before committing to a full benchmark:

python -m src.code_math.runner --benchmark humaneval --limit 5 \
    --config-path configs/default.yaml --output-dir runs/smoke --conditions cot_baseline

Note: the root .gitignore excludes /docs/, /results/, /analysis/ and similar runtime directories; put committed documentation elsewhere.

Running code / math benchmarks

python -m src.code_math.runner \
    --benchmark humaneval \
    --config-path configs/default.yaml \
    --output-dir runs/humaneval_demo \
    --conditions cot_baseline ours

Supported benchmarks: humaneval, mbpp, gsm8k, math.

Supported conditions: io_baseline, cot_baseline, full_library, expel, and ours (FlowEvo's compile + reuse + adaptive-escalation pipeline).

Running ALFWorld

ALFWorld is an optional extra; install it and download the game files once:

pip install -e ".[alfworld]"
alfworld-download

python -m src.alfworld_.run_20task_validation \
    --config-path configs/default.yaml \
    --output-dir runs/alfworld_demo \
    --conditions full_library

Supported ALFWorld conditions: pure_dynamic, compile_only, layer1_only, layer1_2, layer1_3, full_library, expel, and no_governance.

Running democase_sql

democase_sql/ is a small, self-contained domain adapter that applies the same compile → replay → govern loop to natural-language questions over a SQLite database (e.g. "get the address of customer X"). It follows the ALFWorld adapter layout, needs no external dataset, and can run fully offline with an oracle generator.

# 1. build the deterministic demo database (customers, addresses, orders, ...)
python -m democase_sql.db.build_db

# 2. offline run with the oracle generator (no API key needed)
python -m democase_sql.runner --generator oracle     --output-dir democase_sql/runs/oracle_demo     --conditions pure_dynamic layer1_only full_library ours

# 3. real run through the configured LLM backend
python -m democase_sql.runner --generator llm     --config-path configs/default.yaml     --output-dir democase_sql/runs/llm_demo     --conditions pure_dynamic ours

# tests (offline)
python -m pytest democase_sql/tests -q

Supported conditions: pure_dynamic, compile_only, layer1_only, full_library, ours, and no_governance. Each run writes per-episode traces, the compiled skill library (skill_library.json) and a report.md under --output-dir. Reasoning models need a larger output budget; use --max-output-tokens 4096 (the default) or higher.

See democase_sql/README.md for the design of the three skill layers, governance, error handling and how to point it at your own database.

Citation

If you use FlowEvo in your research, please cite our paper:

@inproceedings{ren2026flowevo,
  title         = {FlowEvo: Self-Evolving Agents through the Co-Evolution of Workflows and Executable Skills},
  author        = {Ren, Zeyu and Yue, Ling and Li, Ran and Wang, Yishu and Xu, Shengxiang and Liu, Hanmo and Pan, Shaowu and Di, Shimin},
  booktitle     = {Third Conference on Language Modeling (COLM)},
  year          = {2026},
  address       = {San Francisco, CA, USA},
  url           = {https://openreview.net/forum?id=hU2N7IIkcE},
  eprint        = {2607.21596},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI}
}

COLM proceedings are published on OpenReview and are not assigned DOIs, so the entry carries the OpenReview url in place of a doi field.