diff --git a/.agent/CONTINUITY.md b/.agent/CONTINUITY.md new file mode 100644 index 0000000..c194e89 --- /dev/null +++ b/.agent/CONTINUITY.md @@ -0,0 +1,23 @@ +# Project continuity + +## 2026-07-18: guided Codex CLI dry-run backend + +- Added a first-party `codex-cli` backend so an operator's saved ChatGPT Codex login can power + planning, implementation, and review without copying credentials into a repository container. +- The controller owns every Codex argument. Configuration may name only the Codex executable and + model; extra CLI arguments and environment pass-through are rejected. +- Codex 0.145.0 or newer is required because the adapter uses permission profiles and a fixed set of + current feature-disable controls. Model-run commands receive only minimal runtime reads and the + temporary workspace, with network disabled and credential-like workspace files denied. User + config, project instruction injection, rules, hooks, apps, web search, subagents, and shell + snapshots are disabled for each run. Repository-local Codex skills are refused and execution uses + an empty isolated `HOME`; only `CODEX_HOME` remains visible to the CLI process for saved auth. +- Planning and review receive a read-only workspace. Implementation receives workspace write. + Leftovers still performs its own Git metadata checks, canonical diff gates, offline container + verification, independent review, telemetry validation, and cleanup proof. +- `leftovers setup codex` creates a new mode-0600 dry-run config only after the operator confirms the + repository AI policy, SPDX license, offline test argv, and quota envelope. It detects but does not + install Python, Git, Codex, `gh`, or Docker/Podman, and it does not install a scheduler. +- The Codex CLI backend remains lower assurance than a fresh VM/microVM and is intentionally blocked + from `draft-pr` mode. A later publication PR should add a sealed approval/resume boundary before + reconsidering that restriction. diff --git a/.gitignore b/.gitignore index 704851d..02f0f83 100644 --- a/.gitignore +++ b/.gitignore @@ -8,5 +8,6 @@ __pycache__/ dist/ build/ .leftovers/ -.agent/ +.agent/* +!.agent/CONTINUITY.md *.log diff --git a/README.md b/README.md index c8529cf..a0578dd 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,8 @@ review—not more unsolicited pull requests. runs too close to reset are rejected. - Planning and implementation prompt contracts, fresh independent review, and deterministic controller-rendered draft-PR text from verified evidence. +- Guided, owner-only Codex CLI setup plus a first-party execute-only adapter that uses a saved Codex + login, strict structured output, least-privilege permission profiles, and provider usage receipts. - Docker/Podman command construction with no GitHub credential in the worker. - Offline operator-curated verification commands plus structural rename/file-mode, dependency, license, secret, size, and forbidden-path gates. @@ -78,6 +80,20 @@ or broker cutoff when the provider supports one. ## Quick start +For the shortest supported execute-only path with a ChatGPT Codex plan, run the guided wizard: + +```sh +leftovers --config config/leftovers.toml setup codex +``` + +It checks Python, Git, a read-only GitHub discovery token, Codex login/version/model, `gh`, the +container runtime, and the local sandbox image; asks the operator to confirm the allowlisted +repository, AI policy, license, offline test argv, and quota allocation; and creates a new owner-only +config in `dry-run` mode. It does not install packages, copy tokens, overwrite a config, schedule +runs, or enable publication. See [`docs/CODEX_CLI.md`](docs/CODEX_CLI.md). + +For a generic container provider adapter: + 1. Copy and curate the example configuration: ```sh @@ -119,11 +135,12 @@ or broker cutoff when the provider supports one. PYTHONPATH=src python3 -m leftovers --config config/leftovers.toml run --execute ``` -Execution requires the configured agent command and container runtime. The stock sandbox image does -not embed a model provider or credentials; derive a provider-specific image or use a trusted host -CLI with its own sandbox. No runnable provider adapter ships in v0.1, and the host option is -explicitly lower assurance. See [`docs/AGENT_ADAPTERS.md`](docs/AGENT_ADAPTERS.md) for the exact -stdin/result-file contract and credential tradeoffs. +Execution requires the configured agent and container runtime. The stock sandbox image does not +embed a model provider or credentials. The first-party Codex CLI backend is execute-only and uses +the host CLI's saved login plus least-privilege permission profiles; generic providers still require +a reviewed adapter image or trusted host CLI. Host and Codex CLI backends remain lower assurance and +cannot enable draft publication. See [`docs/AGENT_ADAPTERS.md`](docs/AGENT_ADAPTERS.md) for the exact +contract and credential tradeoffs. ## Prove the control plane before using it @@ -218,6 +235,7 @@ remaining v0.1 gaps in [`SECURITY.md`](SECURITY.md) before enabling writes. - [`PROTOCOL.md`](PROTOCOL.md): prompt/result contracts and state invariants. - [`SECURITY.md`](SECURITY.md): threat model, hard gates, and assurance limits. - [`docs/AGENT_ADAPTERS.md`](docs/AGENT_ADAPTERS.md): provider adapter contract and v0.1 limits. +- [`docs/CODEX_CLI.md`](docs/CODEX_CLI.md): guided Codex setup, execution boundary, and limits. - [`docs/OPERATIONS.md`](docs/OPERATIONS.md): activation, scheduler installation, and recovery. - [`docs/TELEMETRY.md`](docs/TELEMETRY.md): exact quota/check-in semantics, dashboard boundary, and rehearsal evidence. diff --git a/SECURITY.md b/SECURITY.md index fc8b1a1..e251b82 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -30,6 +30,11 @@ and partial publication or cleanup failures. - GitHub issue/base state is rechecked before publication. - Publisher uses an isolated Git HOME, disabled hooks/credential helpers, an ephemeral askpass script, and a token held only in its subprocess environment. +- The first-party Codex CLI dry-run backend accepts no user CLI flags or environment pass-through, + disables automatic instruction/config/rule/hook/app/network surfaces, uses stage-specific + least-privilege permission profiles, and converts only closed-schema output and bounded usage + telemetry. Repository-local Codex skills are refused, execution uses an empty isolated home, and + ambient GitHub, provider-token, SSH-agent, and cloud credentials are not forwarded. - Publisher identity must match configured expected login and immutable GitHub user ID before writes. - Container removal requires exact managed/job/stage labels and a post-removal absence check; workspace deletion runs only afterward and requires a managed marker, expected prefix, and @@ -60,7 +65,7 @@ Do not describe these as solved: human accepts the supply-chain/exfiltration risk. - Agent provider authentication is deployment-specific. Baking credentials into an image is unsafe. Prefer a model/tool broker or a provider CLI whose own sandbox keeps credentials outside tool - reach. The host backend is lower assurance. + reach. Host and Codex CLI backends are lower assurance and cannot publish. - Secret regexes are not proof of absence. Production should add a dedicated scanner and entropy/ historical-secret checks. - Sensitive-issue label and text matching is a conservative gate, not semantic proof that an issue diff --git a/config/leftovers.example.toml b/config/leftovers.example.toml index 48299c1..17d198f 100644 --- a/config/leftovers.example.toml +++ b/config/leftovers.example.toml @@ -69,6 +69,8 @@ tmpfs_size = "512m" [agent] backend = "container" +# For the guided first-party Codex CLI backend, generate a separate owner-only config with +# `leftovers --config config/leftovers.toml setup codex`; do not hand-copy saved auth into this file. # Build a provider-specific derivative of sandbox/Dockerfile that exposes this command and writes # strict JSON to LEFTOVERS_RESULT_PATH. Never add GitHub credentials to that image. command = ["your-agent-cli", "--prompt-stdin"] diff --git a/docs/AGENT_ADAPTERS.md b/docs/AGENT_ADAPTERS.md index 44a98a8..4153026 100644 --- a/docs/AGENT_ADAPTERS.md +++ b/docs/AGENT_ADAPTERS.md @@ -1,9 +1,28 @@ # Agent adapters -Leftovers v0.1 defines a provider-neutral process contract; it does **not** ship a runnable OpenAI, -Anthropic, local-model, or other provider adapter. The stock sandbox image supplies the execution -environment only. A deployment must build and review its own adapter before `run --execute` can -complete. +Leftovers defines a provider-neutral process contract and ships one first-party execute-only +integration for Codex CLI. Anthropic, local-model, and other providers still require a deployment to +build and review its own adapter. The stock sandbox image supplies the execution environment only. + +## First-party Codex CLI backend + +`agent.backend = "codex-cli"` invokes only the configured Codex executable; extra user-supplied +arguments and `pass_environment` entries are rejected. The controller selects the model and builds +all noninteractive, structured-output, permission, feature-disable, and result-path arguments. +Planning and review are read-only; implementation can write only the temporary workspace. Model-run +commands have no network and do not inherit the Codex process environment. + +The Codex process receives only the small host environment needed to find its saved login. GitHub, +OpenAI API, Codex access-token, SSH-agent, cloud, and arbitrary variables are not forwarded. User +config and automatic project instruction injection are disabled, as are project rules, hooks, apps, +web search, subagents, and remote plugins. Exact final usage is converted from Codex JSONL into the +normal adapter telemetry protocol. The execution uses an empty isolated `HOME`, denies `.agents` and +`.codex` reads, and refuses a repository-local `.agents/skills` tree so repository skills cannot +become a higher-priority instruction channel. + +This backend requires Codex CLI 0.145.0 or newer and a model present in its bundled catalog. It is a +lower-assurance host process and is rejected in `draft-pr` mode. Use `leftovers setup codex` and see +[`CODEX_CLI.md`](CODEX_CLI.md) for activation and limitations. ## Process contract @@ -54,10 +73,10 @@ container, and a networked stage could expose the secret. For higher assurance, use an external model/tool broker that keeps provider credentials outside the worker and exposes only the minimum inference operation. A provider CLI on the host may keep its -credential outside the repository container, but `agent.backend = "host"` is the lower-assurance -profile and cannot be used with v0.1 draft publication. Direct provider credentials plus bridge -networking should be limited to curated, explicitly risk-accepted dry runs; `network = "none"` -cannot reach a hosted model API. +credential outside the repository container, but `host` and `codex-cli` are lower-assurance profiles +and cannot be used with draft publication. Direct provider credentials plus bridge networking +should be limited to curated, explicitly risk-accepted dry runs; `network = "none"` cannot reach a +hosted model API from a generic container adapter. Do not claim autonomous operation until the chosen adapter, credential topology, image digest, network policy, and all stage outputs have been exercised in execute-only runs with no remote write. diff --git a/docs/CODEX_CLI.md b/docs/CODEX_CLI.md new file mode 100644 index 0000000..6020ecf --- /dev/null +++ b/docs/CODEX_CLI.md @@ -0,0 +1,103 @@ +# Codex CLI backend + +Leftovers includes a controller-owned `codex-cli` backend for execute-only dry runs using an +operator's saved Codex CLI login. It is the shortest supported path from a ChatGPT Codex plan to a +locally verified candidate patch. It is not a plan-balance scraper, shared quota pool, remote worker, +or automatic PR publisher. + +## Prerequisites + +- Python 3.11 or newer, Git, and Leftovers; +- Codex CLI 0.145.0 or newer; +- a saved Codex login (`codex login`, then `codex login status`); +- a read-only public-repository token in `GITHUB_TOKEN` for live scouting; +- Docker or Podman plus a locally built `leftovers-sandbox:latest` image for offline test execution + and cleanup proof; and +- `gh` only for a future, separately reviewed publication path. + +ChatGPT sign-in uses included subscription access when the account and workspace support it. API-key +login uses separately billed API usage. Leftovers does not pass `OPENAI_API_KEY`, `CODEX_API_KEY`, +`CODEX_ACCESS_TOKEN`, GitHub credentials, SSH agent sockets, or arbitrary environment values into +the agent process. Prefer OS keyring credential storage and treat file-based `auth.json` as a +password. See the official [Codex authentication](https://learn.chatgpt.com/docs/auth) documentation. + +## Guided setup + +For an interactive owner-only configuration wizard: + +```sh +leftovers --config config/leftovers.toml setup codex +``` + +The wizard asks for one allowlisted `owner/name` repository, a reviewed SPDX license, an HTTPS +source showing that AI-assisted contributions are permitted, one or more offline test argv arrays, +and an explicit daily or weekly token envelope. It writes a new mode-`0600` config in `dry-run` mode +and refuses to overwrite any existing file or symlink. + +A non-interactive example is: + +```sh +leftovers --config config/leftovers.toml setup codex \ + --repository owner/project \ + --ai-policy-url https://github.com/owner/project/blob/main/CONTRIBUTING.md \ + --ai-policy-reviewed \ + --allowed-license MIT \ + --test-command-json '["python","-m","pytest","-q"]' \ + --allocated-tokens 150000 +``` + +Setup only diagnoses prerequisites. It does not install host packages, copy a token, log in on the +operator's behalf, build the sandbox image, enable publication, or install a scheduler. Review every +generated repository field before running a live scout. + +## Execution boundary + +For each model stage, Leftovers constructs `codex exec` arguments itself and uses ephemeral, +noninteractive, strict structured output. The adapter: + +- selects the configured model explicitly; +- disables approval prompts while keeping a least-privilege permission profile; +- grants model-run commands only minimal runtime reads and the temporary repository; +- keeps planning/review read-only and grants workspace write only during implementation; +- disables model-command network, web search, user config, automatic `AGENTS.md` injection, + execution rules, hooks, apps, memories, goals, subagents, remote plugins, and shell snapshots; +- uses an empty isolated `HOME` for repository execution while retaining only `CODEX_HOME` for the + CLI's own saved authentication; +- denies `.agents`, `.codex`, common `.env`, PEM, and key-file reads inside the workspace, and + refuses repositories that contain a discoverable `.agents/skills` tree; +- captures exact final usage from Codex JSONL and validates it through the existing telemetry + protocol; and +- writes the final JSON through a closed, stage-specific schema outside the model's workspace. + +The controller then checks Git metadata and the canonical diff, runs only operator-curated offline +commands inside the hardened container, performs a fresh review, and proves label-scoped container +cleanup before deleting the workspace. The coding process never receives GitHub publication +credentials. The official [`codex exec`](https://learn.chatgpt.com/docs/developer-commands?surface=cli#cli-codex-exec) +and [permission profiles](https://learn.chatgpt.com/docs/permissions) documentation describe the +underlying Codex controls. + +## Activation + +Run these in order: + +```sh +leftovers --config config/leftovers.toml validate +leftovers --config config/leftovers.toml doctor +leftovers --config config/leftovers.toml scout +leftovers --config config/leftovers.toml run --execute +``` + +Inspect the hash-chained journal and cleanup receipt after every execute-only run. Complete at least +three successful dry runs before considering any publication work. + +## Limits + +- Codex still runs as a local process under the operator account. Permission profiles materially + reduce model-command access, but this is not equivalent to a disposable VM/microVM. +- A container runtime remains mandatory because setup and verification commands do not run on the + host. +- The configured token envelope is local admission control. It is not an exact view of remaining + ChatGPT plan allowance and cannot force a provider-side cutoff. +- `codex-cli` is rejected in `draft-pr` mode. Publication remains a separate future hardening step. +- The selected model must remain available in the installed Codex CLI's bundled catalog; `doctor` + fails closed when the CLI, login, version, or model check fails. diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md index 700865c..2f54da5 100644 --- a/docs/OPERATIONS.md +++ b/docs/OPERATIONS.md @@ -2,11 +2,13 @@ ## First activation -1. Create `config/leftovers.toml` from the example. +1. For Codex CLI, run `leftovers --config config/leftovers.toml setup codex`; for a generic adapter, + create `config/leftovers.toml` from the example. 2. Curate a small repository allowlist and record current licenses, contribution rules, AI policy, default branch, forbidden paths, and exact offline checks. If AI contributions are allowed, record the policy's HTTPS source and the date it was actually checked. -3. Build a provider-specific agent image from `sandbox/Dockerfile` without GitHub credentials. +3. Build the sandbox image used for offline verification. Generic container agents also need a + provider-specific derivative of `sandbox/Dockerfile` without GitHub credentials. 4. Run `validate`, `doctor`, fixture scout, the OCI training cycle, live scout, and at least three execute-only dry runs. 5. Inspect audit journals and confirm every temporary workspace is gone. diff --git a/src/leftovers/cli.py b/src/leftovers/cli.py index 037ec2f..5439928 100644 --- a/src/leftovers/cli.py +++ b/src/leftovers/cli.py @@ -13,10 +13,18 @@ from .audit import redact from .budget import BudgetLedger +from .codex_adapter import inspect_codex_cli from .config import AppConfig, ConfigError, load_config from .dashboard import DashboardUnavailable, serve_dashboard from .github import FixtureIssueSource, GitHubClient, GitHubError from .models import RunStage +from .onboarding import ( + DEFAULT_CODEX_MODEL, + CodexSetupInputs, + container_image_available, + parse_argv_json, + setup_codex, +) from .orchestrator import ContributionOrchestrator, ranked_to_dict from .publisher import GhPublisher, PublicationError from .rehearsal import ( @@ -60,6 +68,50 @@ def _parser() -> argparse.ArgumentParser: help="TOML configuration path (default: config/leftovers.toml)", ) subparsers = parser.add_subparsers(dest="command", required=True) + setup = subparsers.add_parser( + "setup", help="create a new dry-run configuration with guided prerequisite checks" + ) + setup.add_argument("provider", choices=("codex",)) + setup.add_argument("--repository", help="allowlisted GitHub repository as owner/name") + setup.add_argument("--ai-policy-url", help="reviewed HTTPS AI-contribution policy URL") + setup.add_argument( + "--ai-policy-reviewed", + action="store_true", + help="confirm that the policy URL was reviewed and currently permits AI assistance", + ) + setup.add_argument( + "--test-command-json", + action="append", + default=[], + help='reviewed offline test argv as JSON, for example ["python","-m","pytest","-q"]', + ) + setup.add_argument( + "--allowed-license", + action="append", + default=[], + help="reviewed SPDX license identifier; repeat when needed", + ) + setup.add_argument( + "--allow-label", + action="append", + default=[], + help="maintainer-signal issue label; defaults to help wanted and good first issue", + ) + setup.add_argument("--default-branch", default="main") + setup.add_argument("--model", default=DEFAULT_CODEX_MODEL) + setup.add_argument( + "--allocated-tokens", + type=_bounded_integer(100_000, 10_000_000, "allocated tokens"), + help="explicit daily or weekly token envelope allocated to Leftovers", + ) + setup.add_argument( + "--reserve-tokens", + type=_bounded_integer(0, 9_900_000, "reserve tokens"), + default=20_000, + ) + setup.add_argument("--window", choices=("daily", "weekly"), default="daily") + setup.add_argument("--timezone", default="America/Phoenix") + setup.add_argument("--runtime", choices=("docker", "podman"), default="docker") subparsers.add_parser("validate", help="validate configuration and exit") subparsers.add_parser("doctor", help="check local runtime prerequisites without remote writes") @@ -149,6 +201,67 @@ def _source(config: AppConfig, fixture: Path | None) -> FixtureIssueSource | Git return FixtureIssueSource(fixture.resolve()) if fixture else GitHubClient(config.github) +def _prompt_required(value: str | None, label: str) -> str: + if value and value.strip(): + return value.strip() + if not sys.stdin.isatty(): + raise ConfigError(f"setup requires --{label.replace('_', '-')}") + response = input(f"{label.replace('_', ' ').capitalize()}: ").strip() + if not response: + raise ConfigError(f"setup requires {label.replace('_', ' ')}") + return response + + +def _setup_inputs(args: argparse.Namespace) -> CodexSetupInputs: + repository = _prompt_required(args.repository, "repository") + ai_policy_url = _prompt_required(args.ai_policy_url, "ai_policy_url") + reviewed = args.ai_policy_reviewed + if not reviewed and sys.stdin.isatty(): + answer = input( + "Have you reviewed that policy and confirmed AI-assisted contributions are allowed? " + "[y/N]: " + ).strip() + reviewed = answer.casefold() in {"y", "yes"} + raw_commands = list(args.test_command_json) + if not raw_commands and sys.stdin.isatty(): + raw_commands.append( + input('Offline test argv JSON (for example ["python","-m","pytest","-q"]): ') + ) + commands = tuple(parse_argv_json(value) for value in raw_commands) + licenses = list(args.allowed_license) + if not licenses and sys.stdin.isatty(): + licenses = [ + item.strip() + for item in input("Reviewed SPDX license identifier(s), comma-separated: ").split(",") + if item.strip() + ] + allocated_tokens = args.allocated_tokens + if allocated_tokens is None and sys.stdin.isatty(): + raw_tokens = input("Token envelope allocated to Leftovers [150000]: ").strip() or "150000" + try: + allocated_tokens = int(raw_tokens) + except ValueError as exc: + raise ConfigError("allocated tokens must be an integer") from exc + if allocated_tokens is None: + raise ConfigError("setup requires --allocated-tokens") + labels = tuple(args.allow_label or ("help wanted", "good first issue")) + return CodexSetupInputs( + repository=repository, + ai_policy_url=ai_policy_url, + ai_policy_reviewed=reviewed, + test_commands=commands, + allowed_licenses=tuple(licenses), + allow_labels=labels, + default_branch=args.default_branch, + model=args.model, + allocated_tokens=allocated_tokens, + reserve_tokens=args.reserve_tokens, + window=args.window, + timezone=args.timezone, + runtime=args.runtime, + ) + + def _doctor(config: AppConfig) -> tuple[bool, list[dict[str, Any]]]: checks: list[dict[str, Any]] = [] @@ -173,6 +286,12 @@ def add(name: str, ok: bool, detail: str, severity: str = "error") -> None: runtime_present, f"{config.sandbox.runtime} is required for container-agent and verification stages", ) + add( + "sandbox_image", + runtime_present + and container_image_available(config.sandbox.runtime, config.sandbox.image), + f"configured image {config.sandbox.image} must already exist locally", + ) add( "pinned_image", "@sha256:" in config.sandbox.image, @@ -182,9 +301,31 @@ def add(name: str, ok: bool, detail: str, severity: str = "error") -> None: add( "agent_backend", config.agent.backend == "container", - "host agents rely on the provider CLI's own sandbox and are a lower-assurance profile", + "host and Codex CLI agents remain dry-run, lower-assurance profiles", severity="warning", ) + if config.agent.backend == "codex-cli": + inspection = inspect_codex_cli(config.agent.command[0], config.agent.model) + add( + "codex_cli", + inspection.executable is not None, + "Codex CLI must be installed at the configured path", + ) + add( + "codex_version", + inspection.version_supported, + "Codex CLI 0.145.0 or newer is required by the hardened adapter", + ) + add( + "codex_login", + inspection.authenticated, + "Codex CLI must report an active saved login", + ) + add( + "codex_model", + inspection.model_available, + f"configured model {config.agent.model} must exist in the bundled model catalog", + ) add( "rootless_runtime", False, @@ -375,6 +516,12 @@ def _training_payload(args: argparse.Namespace, config: AppConfig) -> tuple[int, def main(argv: list[str] | None = None) -> int: args = _parser().parse_args(argv) try: + if args.command == "setup": + if args.provider != "codex": + raise ConfigError("unsupported setup provider") + status, payload = setup_codex(args.config, _setup_inputs(args)) + print(json.dumps(payload, indent=2, sort_keys=True)) + return status config = load_config(args.config) if args.command == "validate": print(json.dumps({"valid": True, "config": str(args.config.resolve())}, indent=2)) diff --git a/src/leftovers/codex_adapter.py b/src/leftovers/codex_adapter.py new file mode 100644 index 0000000..41f3fb2 --- /dev/null +++ b/src/leftovers/codex_adapter.py @@ -0,0 +1,427 @@ +from __future__ import annotations + +import json +import os +import re +import shutil +import stat +import subprocess +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from .models import TokenUsage, utc_now + + +CODEX_PROVIDER = "openai-codex-cli" +CODEX_ADAPTER_VERSION = "leftovers-codex-cli-v1" +MIN_CODEX_VERSION = (0, 145, 0) +_MAX_MODEL_CATALOG_BYTES = 2_000_000 +_CODEX_VERSION = re.compile(r"\bcodex-cli\s+(\d+)\.(\d+)\.(\d+)") +_DISABLED_FEATURES = ( + "apps", + "goals", + "hooks", + "memories", + "multi_agent", + "remote_plugin", + "shell_snapshot", +) + + +class CodexAdapterError(RuntimeError): + pass + + +@dataclass(frozen=True) +class CodexCliInspection: + executable: str | None + version: str | None + version_supported: bool + authenticated: bool + model_available: bool + + @property + def ready(self) -> bool: + return bool( + self.executable + and self.version_supported + and self.authenticated + and self.model_available + ) + + +def resolve_codex_executable(value: str = "codex") -> str | None: + candidate = shutil.which(value) + if candidate is None: + return None + return str(Path(candidate).resolve()) + + +def codex_process_environment(isolated_home: Path | None = None) -> dict[str, str]: + """Keep login discovery available without forwarding ambient worker credentials.""" + allowed = { + "CODEX_HOME", + "HOME", + "LANG", + "LC_ALL", + "LOGNAME", + "PATH", + "SHELL", + "TMPDIR", + "USER", + } + environment = {name: value for name, value in os.environ.items() if name in allowed} + if isolated_home is not None: + actual_home = os.environ.get("HOME") + codex_home = os.environ.get("CODEX_HOME") + if codex_home is None and actual_home: + codex_home = str(Path(actual_home) / ".codex") + if not codex_home: + raise CodexAdapterError("Codex auth home could not be determined") + environment["CODEX_HOME"] = codex_home + environment["HOME"] = str(isolated_home.resolve()) + return environment + + +def validate_codex_workspace(workspace: Path) -> None: + """Reject repository skill discovery that would outrank the untrusted-source contract.""" + agents = workspace / ".agents" + try: + agents_info = agents.lstat() + except FileNotFoundError: + return + if stat.S_ISLNK(agents_info.st_mode) or not stat.S_ISDIR(agents_info.st_mode): + raise CodexAdapterError("repository .agents path is not a real directory") + skills = agents / "skills" + try: + skills.lstat() + except FileNotFoundError: + return + raise CodexAdapterError("repository-local Codex skills are refused in unattended runs") + + +def _run_codex_probe(argv: list[str], *, timeout: int = 20) -> subprocess.CompletedProcess[str]: + try: + return subprocess.run( + argv, + env=codex_process_environment(), + stdin=subprocess.DEVNULL, + capture_output=True, + text=True, + timeout=timeout, + check=False, + ) + except (OSError, subprocess.TimeoutExpired) as exc: + raise CodexAdapterError(f"Codex CLI probe failed: {type(exc).__name__}") from exc + + +def inspect_codex_cli(executable: str, model: str) -> CodexCliInspection: + resolved = resolve_codex_executable(executable) + if resolved is None: + return CodexCliInspection(None, None, False, False, False) + + try: + version_result = _run_codex_probe([resolved, "--version"]) + except CodexAdapterError: + return CodexCliInspection(resolved, None, False, False, False) + version_text = version_result.stdout.strip() + match = _CODEX_VERSION.search(version_text) + parsed = tuple(int(part) for part in match.groups()) if match else None + version_supported = bool( + version_result.returncode == 0 and parsed is not None and parsed >= MIN_CODEX_VERSION + ) + + try: + login_result = _run_codex_probe([resolved, "login", "status"]) + except CodexAdapterError: + authenticated = False + else: + authenticated = login_result.returncode == 0 + + model_available = False + try: + models_result = _run_codex_probe([resolved, "debug", "models", "--bundled"]) + except CodexAdapterError: + models_result = None + if models_result is not None and models_result.returncode == 0 and ( + len(models_result.stdout.encode("utf-8")) <= _MAX_MODEL_CATALOG_BYTES + ): + try: + payload = json.loads(models_result.stdout) + except (json.JSONDecodeError, RecursionError): + payload = None + if isinstance(payload, dict) and isinstance(payload.get("models"), list): + model_available = any( + isinstance(item, dict) and item.get("slug") == model + for item in payload["models"] + ) + + return CodexCliInspection( + executable=resolved, + version=version_text or None, + version_supported=version_supported, + authenticated=authenticated, + model_available=model_available, + ) + + +def _string_schema(*, minimum: int = 0, maximum: int = 8_192) -> dict[str, Any]: + return {"type": "string", "minLength": minimum, "maxLength": maximum} + + +def _string_array(*, minimum: int = 0) -> dict[str, Any]: + return { + "type": "array", + "minItems": minimum, + "maxItems": 128, + "items": _string_schema(minimum=1, maximum=4_096), + } + + +def stage_result_schema(stage: str) -> dict[str, Any]: + string = _string_schema(maximum=8_192) + nonempty = _string_schema(minimum=1, maximum=8_192) + argv = { + "type": "array", + "maxItems": 64, + "items": _string_schema(maximum=4_096), + } + if stage == "planning": + properties: dict[str, Any] = { + "status": {"type": "string", "enum": ["planned", "blocked", "failed"]}, + "acceptance_criteria": _string_array(), + "reproduction": { + "type": "object", + "properties": {"argv": argv, "observed": string}, + "required": ["argv", "observed"], + "additionalProperties": False, + }, + "root_cause": { + "type": "array", + "maxItems": 128, + "items": { + "type": "object", + "properties": {"path": string, "evidence": string}, + "required": ["path", "evidence"], + "additionalProperties": False, + }, + }, + "steps": _string_array(), + "tests": {"type": "array", "maxItems": 64, "items": argv}, + "risks": _string_array(), + "estimated_remaining_tokens": { + "type": "integer", + "minimum": 0, + "maximum": 1_000_000_000, + }, + "stop_conditions": _string_array(), + "reason": string, + } + elif stage == "implementation": + properties = { + "status": {"type": "string", "enum": ["implemented", "blocked", "failed"]}, + "summary": string, + "changed_files": _string_array(), + "commands": { + "type": "array", + "maxItems": 128, + "items": { + "type": "object", + "properties": { + "argv": argv, + "exit_code": {"type": "integer"}, + "summary": string, + }, + "required": ["argv", "exit_code", "summary"], + "additionalProperties": False, + }, + }, + "acceptance_criteria": { + "type": "array", + "maxItems": 128, + "items": { + "type": "object", + "properties": {"criterion": string, "evidence": string}, + "required": ["criterion", "evidence"], + "additionalProperties": False, + }, + }, + "remaining_risks": _string_array(), + "reason": string, + } + elif stage == "review": + properties = { + "verdict": {"type": "string", "enum": ["approve", "revise", "abandon"]}, + "findings": { + "type": "array", + "maxItems": 128, + "items": { + "type": "object", + "properties": { + "severity": { + "type": "string", + "enum": ["blocker", "major", "minor"], + }, + "summary": nonempty, + "evidence": nonempty, + "path": {"type": ["string", "null"], "maxLength": 4_096}, + }, + "required": ["severity", "summary", "evidence", "path"], + "additionalProperties": False, + }, + }, + "missing_verification": _string_array(), + "pr_claims_supported": {"type": "boolean"}, + } + else: + raise CodexAdapterError(f"unsupported Codex stage: {stage}") + return { + "$schema": "https://json-schema.org/draft/2020-12/schema", + "type": "object", + "properties": properties, + "required": list(properties), + "additionalProperties": False, + } + + +def write_stage_schema(path: Path, stage: str) -> None: + encoded = (json.dumps(stage_result_schema(stage), sort_keys=True) + "\n").encode("utf-8") + flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0) + descriptor = os.open(path, flags, 0o600) + try: + info = os.fstat(descriptor) + if not stat.S_ISREG(info.st_mode) or info.st_uid != os.getuid() or info.st_nlink != 1: + raise CodexAdapterError("Codex output schema is not an owner-controlled regular file") + os.fchmod(descriptor, 0o600) + pending = memoryview(encoded) + while pending: + written = os.write(descriptor, pending) + if written < 1: + raise CodexAdapterError("Codex output schema write made no progress") + pending = pending[written:] + os.fsync(descriptor) + finally: + os.close(descriptor) + + +def _permission_config(profile: str, access: str) -> str: + workspace_rules = ( + '{"."="' + + access + + '",".git"="read",".agents"="deny",".codex"="deny",' + + '".env"="deny",".env.*"="deny","*.env"="deny",' + + '"*/*.env"="deny","*/*/*.env"="deny","**/*.pem"="deny",' + + '"**/*.key"="deny"}' + ) + return ( + f'permissions.{profile}.filesystem={{glob_scan_max_depth=3,":minimal"="read",' + f'":workspace_roots"={workspace_rules}}}' + ) + + +def build_codex_argv( + executable: str, + *, + stage: str, + workspace: Path, + schema_path: Path, + result_path: Path, + model: str, + read_only_workspace: bool, +) -> list[str]: + if stage not in {"planning", "implementation", "review"}: + raise CodexAdapterError(f"unsupported Codex stage: {stage}") + if read_only_workspace != (stage != "implementation"): + raise CodexAdapterError("Codex stage and workspace permission do not match") + profile = "leftovers-read" if read_only_workspace else "leftovers-write" + access = "read" if read_only_workspace else "write" + argv = [ + executable, + "--ask-for-approval", + "never", + "--cd", + str(workspace.resolve()), + "--model", + model, + "--config", + f'default_permissions="{profile}"', + "--config", + _permission_config(profile, access), + "--config", + f"permissions.{profile}.network.enabled=false", + "--config", + "project_doc_max_bytes=0", + "--config", + "project_doc_fallback_filenames=[]", + "--config", + "mcp_servers={}", + "--config", + 'web_search="disabled"', + "--config", + "check_for_update_on_startup=false", + "--config", + 'shell_environment_policy.inherit="none"', + "--config", + 'shell_environment_policy.set={CI="1"}', + ] + for feature in _DISABLED_FEATURES: + argv.extend(["--disable", feature]) + argv.extend( + [ + "exec", + "--ephemeral", + "--ignore-user-config", + "--ignore-rules", + "--strict-config", + "--json", + "--color", + "never", + "--output-schema", + str(schema_path.resolve()), + "--output-last-message", + str(result_path.resolve()), + "-", + ] + ) + return argv + + +def parse_codex_usage(output: str) -> TokenUsage: + latest: dict[str, Any] | None = None + for line in output.splitlines(): + if not line.startswith("{") or len(line) > 65_536: + continue + try: + event = json.loads(line) + except (json.JSONDecodeError, RecursionError): + continue + if ( + isinstance(event, dict) + and event.get("type") == "turn.completed" + and isinstance(event.get("usage"), dict) + ): + latest = event["usage"] + if latest is None: + raise CodexAdapterError("Codex JSONL did not contain a final usage receipt") + values = { + "input_tokens": latest.get("input_tokens"), + "cached_input_tokens": latest.get("cached_input_tokens", 0), + "output_tokens": latest.get("output_tokens"), + "reasoning_tokens": latest.get( + "reasoning_output_tokens", latest.get("reasoning_tokens", 0) + ), + } + if any(type(value) is not int or not 0 <= value <= 1_000_000_000 for value in values.values()): + raise CodexAdapterError("Codex JSONL usage values are invalid") + if values["cached_input_tokens"] > values["input_tokens"]: + raise CodexAdapterError("Codex cached input usage exceeds input usage") + if values["reasoning_tokens"] > values["output_tokens"]: + raise CodexAdapterError("Codex reasoning usage exceeds output usage") + return TokenUsage( + **values, + total_tokens=values["input_tokens"] + values["output_tokens"], + source="provider_response", + exact=True, + reported_at=utc_now(), + ) diff --git a/src/leftovers/config.py b/src/leftovers/config.py index 452dbb9..e6b0ebb 100644 --- a/src/leftovers/config.py +++ b/src/leftovers/config.py @@ -568,8 +568,8 @@ def _validate(config: AppConfig) -> None: and 1 <= config.sandbox.timeout_seconds <= 7_200 ): raise ConfigError("sandbox resource limits are outside conservative bounds") - if config.agent.backend not in {"container", "host"}: - raise ConfigError("agent.backend must be container or host") + if config.agent.backend not in {"container", "host", "codex-cli"}: + raise ConfigError("agent.backend must be container, host, or codex-cli") if not config.agent.command: raise ConfigError("agent.command must be a non-empty argv array") if not config.agent.command[0].strip(): @@ -588,6 +588,20 @@ def _validate(config: AppConfig) -> None: len(argument) > 4_096 or "\0" in argument for argument in config.agent.command ): raise ConfigError("agent.command exceeds the argv safety limits") + if config.agent.backend == "codex-cli": + executable_name = Path(config.agent.command[0]).name.casefold() + if len(config.agent.command) != 1 or executable_name not in {"codex", "codex.exe"}: + raise ConfigError( + "codex-cli backend command must contain only the Codex executable path" + ) + if config.agent.provider != "openai-codex-cli": + raise ConfigError("codex-cli backend provider must be openai-codex-cli") + if config.agent.model == "unconfigured": + raise ConfigError("codex-cli backend requires an explicit model") + if not config.agent.checkin_required or not config.agent.usage_reporting_required: + raise ConfigError("codex-cli backend requires check-in and usage reporting") + if config.agent.pass_environment: + raise ConfigError("codex-cli backend does not accept pass_environment entries") if any(_ENVIRONMENT_NAME.fullmatch(name) is None for name in config.agent.pass_environment): raise ConfigError("agent.pass_environment contains an invalid variable name") if ( @@ -634,7 +648,9 @@ def _validate(config: AppConfig) -> None: if config.publication.mode == "draft-pr" and not config.publication.draft: raise ConfigError("v1 only publishes draft PRs") if config.publication.mode == "draft-pr" and config.agent.backend != "container": - raise ConfigError("draft publication requires the container agent backend") + raise ConfigError( + "draft publication requires the container agent backend; codex-cli is dry-run only" + ) if ( config.publication.mode == "draft-pr" and _PINNED_IMAGE.fullmatch(config.sandbox.image) is None diff --git a/src/leftovers/onboarding.py b/src/leftovers/onboarding.py new file mode 100644 index 0000000..ebd0060 --- /dev/null +++ b/src/leftovers/onboarding.py @@ -0,0 +1,401 @@ +from __future__ import annotations + +import json +import os +import shutil +import stat +import subprocess +import sys +import tempfile +from dataclasses import dataclass +from datetime import UTC, datetime +from pathlib import Path + +from .codex_adapter import CODEX_PROVIDER, inspect_codex_cli +from .config import ConfigError, load_config + + +DEFAULT_CODEX_MODEL = "gpt-5.6-luna" + + +@dataclass(frozen=True) +class CodexSetupInputs: + repository: str + ai_policy_url: str + ai_policy_reviewed: bool + test_commands: tuple[tuple[str, ...], ...] + allowed_licenses: tuple[str, ...] + allow_labels: tuple[str, ...] + default_branch: str + model: str + allocated_tokens: int + reserve_tokens: int + window: str + timezone: str + runtime: str + + +def parse_argv_json(value: str) -> tuple[str, ...]: + try: + payload = json.loads(value) + except json.JSONDecodeError as exc: + raise ConfigError("test commands must be JSON argv arrays") from exc + if ( + not isinstance(payload, list) + or not payload + or len(payload) > 64 + or any( + not isinstance(argument, str) + or not argument + or len(argument) > 4_096 + or "\0" in argument + for argument in payload + ) + ): + raise ConfigError("each test command must be a non-empty JSON string array") + return tuple(payload) + + +def _toml_string(value: str) -> str: + return json.dumps(value, ensure_ascii=True) + + +def _toml_strings(values: tuple[str, ...]) -> str: + return "[" + ", ".join(_toml_string(value) for value in values) + "]" + + +def _toml_commands(commands: tuple[tuple[str, ...], ...]) -> str: + return "[" + ", ".join(_toml_strings(command) for command in commands) + "]" + + +def _validate_inputs(inputs: CodexSetupInputs) -> None: + if not inputs.ai_policy_reviewed: + raise ConfigError( + "setup requires an explicit confirmation that the repository AI policy was reviewed" + ) + if not inputs.test_commands: + raise ConfigError("setup requires at least one operator-reviewed test command") + if not inputs.allowed_licenses: + raise ConfigError("setup requires at least one reviewed SPDX license identifier") + if not inputs.allow_labels: + raise ConfigError("setup requires at least one maintainer-signal issue label") + if inputs.window not in {"daily", "weekly"}: + raise ConfigError("setup window must be daily or weekly") + if inputs.runtime not in {"docker", "podman"}: + raise ConfigError("setup runtime must be docker or podman") + if inputs.allocated_tokens < 1 or inputs.reserve_tokens < 0: + raise ConfigError("allocated and reserve token values must be positive") + if inputs.allocated_tokens - inputs.reserve_tokens < 100_000: + raise ConfigError( + "allocated tokens must leave at least 100000 after reserve for the default P95 gate" + ) + + +def render_codex_config(inputs: CodexSetupInputs, codex_executable: str) -> str: + _validate_inputs(inputs) + checked_at = datetime.now(UTC).date().isoformat() + policy_forbid_paths = _toml_strings( + ( + ".github/workflows/**", + "SECURITY.md", + "CODEOWNERS", + ".gitmodules", + ".gitattributes", + "**/*.pem", + "**/*.key", + ) + ) + return f'''version = 1 +state_dir = ".leftovers/state" +temp_root = ".leftovers/workspaces" + +[github] +api_url = "https://api.github.com" +token_env = "GITHUB_TOKEN" +api_version = "2026-03-10" +request_timeout_seconds = 20 +max_read_requests_per_run = 500 + +[budget] +source = "fixed" +fixed_remaining_tokens = {inputs.allocated_tokens} +maximum_tokens = {inputs.allocated_tokens} +reserve_tokens = {inputs.reserve_tokens} +minimum_spendable_tokens = 30000 +safety_multiplier = 1.25 +window = {_toml_string(inputs.window)} +timezone = {_toml_string(inputs.timezone)} +reset_hour = 0 +reset_weekday = 0 +max_run_seconds = 3600 +reset_safety_seconds = 300 + +[discovery] +query = {_toml_string(f'is:issue is:open no:assignee -linked:pr label:"{inputs.allow_labels[0]}"')} +per_repo_limit = 20 +max_candidates = 100 + +[scoring] +minimum_score = 55 +repository_impact_weight = 0.28 +urgency_weight = 0.22 +user_demand_weight = 0.15 +maintainer_signal_weight = 0.15 +tractability_weight = 0.12 +neglect_weight = 0.08 +technical_risk_penalty = 0.20 +collision_risk_penalty = 0.12 +scope_uncertainty_penalty = 0.08 + +[policy] +require_unassigned = true +require_no_open_linked_pr = true +require_license = true +max_changed_files = 20 +max_changed_lines = 1200 +max_patch_bytes = 1000000 +ai_policy_max_age_days = 90 +deny_labels = ["security", "vulnerability", "legal", "needs-design", "breaking-change", "wontfix"] +forbid_paths = {policy_forbid_paths} +forbid_dependency_changes = true + +[sandbox] +runtime = {_toml_string(inputs.runtime)} +image = "leftovers-sandbox:latest" +network = "none" +memory = "4g" +cpus = 2.0 +pids_limit = 256 +timeout_seconds = 1800 +tmpfs_size = "512m" + +[agent] +backend = "codex-cli" +command = [{_toml_string(codex_executable)}] +provider = "{CODEX_PROVIDER}" +model = {_toml_string(inputs.model)} +checkin_required = true +usage_reporting_required = true +checkin_timeout_seconds = 30 +heartbeat_timeout_seconds = 120 +timeout_seconds = 3600 +max_output_bytes = 65536 +estimated_tokens_p50 = 40000 +estimated_tokens_p95 = 80000 +max_repair_cycles = 1 +pass_environment = [] + +[publication] +mode = "dry-run" +external_writes_acknowledged = false +require_cli_flag = true +draft = true +fork = true +branch_prefix = "leftovers" +disclose_ai_assistance = true +max_prs_per_window = 1 +max_open_prs_per_repository = 1 +repository_cooldown_days = 7 + +[[repositories]] +slug = {_toml_string(inputs.repository)} +enabled = true +importance = 0.5 +default_branch = {_toml_string(inputs.default_branch)} +allowed_licenses = {_toml_strings(inputs.allowed_licenses)} +allow_labels = {_toml_strings(inputs.allow_labels)} +deny_labels = ["needs maintainer decision"] +setup_commands = [] +test_commands = {_toml_commands(inputs.test_commands)} +forbid_paths = ["infra/**", "releases/**"] +max_changed_files = 12 +max_changed_lines = 600 +network = "none" +require_human_approval = true +ai_contributions_allowed = true +ai_policy_url = {_toml_string(inputs.ai_policy_url)} +ai_policy_checked_at = {_toml_string(checked_at)} +''' + + +def _write_new_config(path: Path, content: str) -> Path: + expanded = path.expanduser() + if expanded.is_symlink() or expanded.exists(): + raise ConfigError(f"refusing to overwrite existing setup config: {expanded}") + parent = expanded.parent + if parent.is_symlink(): + raise ConfigError(f"setup config parent may not be a symlink: {parent}") + parent.mkdir(parents=True, exist_ok=True, mode=0o700) + parent_info = parent.lstat() + if not stat.S_ISDIR(parent_info.st_mode) or parent_info.st_uid != os.getuid(): + raise ConfigError("setup config parent is not an owner-controlled directory") + parent = parent.resolve() + target = parent / expanded.name + descriptor, temporary_name = tempfile.mkstemp(prefix=".leftovers-setup-", dir=parent) + temporary = Path(temporary_name) + try: + os.fchmod(descriptor, 0o600) + pending = memoryview(content.encode("utf-8")) + while pending: + written = os.write(descriptor, pending) + if written < 1: + raise ConfigError("setup config write made no progress") + pending = pending[written:] + os.fsync(descriptor) + os.close(descriptor) + descriptor = -1 + load_config(temporary) + try: + os.link(temporary, target, follow_symlinks=False) + except FileExistsError as exc: + raise ConfigError(f"refusing to overwrite existing setup config: {target}") from exc + finally: + if descriptor >= 0: + os.close(descriptor) + temporary.unlink(missing_ok=True) + info = target.lstat() + if not stat.S_ISREG(info.st_mode) or info.st_uid != os.getuid() or info.st_nlink != 1: + raise ConfigError("created setup config failed ownership and file-type verification") + os.chmod(target, 0o600) + return target + + +def _gh_authenticated(executable: str | None) -> bool: + if executable is None: + return False + try: + result = subprocess.run( + [executable, "auth", "status", "--hostname", "github.com"], + env={name: os.environ[name] for name in ("HOME", "PATH") if name in os.environ}, + stdin=subprocess.DEVNULL, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + timeout=15, + check=False, + ) + except (OSError, subprocess.TimeoutExpired): + return False + return result.returncode == 0 + + +def container_image_available(runtime: str, image: str = "leftovers-sandbox:latest") -> bool: + executable = shutil.which(runtime) + if executable is None: + return False + try: + result = subprocess.run( + [executable, "image", "inspect", image], + env=os.environ.copy(), + stdin=subprocess.DEVNULL, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + timeout=20, + check=False, + ) + except (OSError, subprocess.TimeoutExpired): + return False + return result.returncode == 0 + + +def setup_codex(path: Path, inputs: CodexSetupInputs) -> tuple[int, dict[str, object]]: + _validate_inputs(inputs) + codex = inspect_codex_cli("codex", inputs.model) + configured_executable = codex.executable or "codex" + config_path = _write_new_config(path, render_codex_config(inputs, configured_executable)) + git_present = shutil.which("git") is not None + runtime_present = shutil.which(inputs.runtime) is not None + image_present = runtime_present and container_image_available(inputs.runtime) + gh = shutil.which("gh") + gh_authenticated = _gh_authenticated(gh) + github_read_token = bool(os.environ.get("GITHUB_TOKEN")) + checks = [ + { + "name": "python", + "ok": sys.version_info >= (3, 11), + "severity": "error", + "detail": "Python 3.11 or newer is required", + }, + { + "name": "git", + "ok": git_present, + "severity": "error", + "detail": "Git is required for temporary repository acquisition", + }, + { + "name": "container_runtime", + "ok": runtime_present, + "severity": "error", + "detail": f"{inputs.runtime} is required for offline verification and cleanup proof", + }, + { + "name": "sandbox_image", + "ok": image_present, + "severity": "error", + "detail": "leftovers-sandbox:latest must be built locally before execute runs", + }, + { + "name": "codex_cli", + "ok": codex.executable is not None, + "severity": "error", + "detail": "Codex CLI must be installed and available on PATH", + }, + { + "name": "codex_version", + "ok": codex.version_supported, + "severity": "error", + "detail": "Codex CLI 0.145.0 or newer is required by the hardened adapter", + }, + { + "name": "codex_login", + "ok": codex.authenticated, + "severity": "error", + "detail": "Codex CLI must have an active saved login; run codex login", + }, + { + "name": "codex_model", + "ok": codex.model_available, + "severity": "error", + "detail": f"configured model {inputs.model} must exist in the bundled catalog", + }, + { + "name": "github_read_token", + "ok": github_read_token, + "severity": "error", + "detail": "GITHUB_TOKEN must contain a read-only public-repository token for scouting", + }, + { + "name": "github_cli", + "ok": gh is not None, + "severity": "warning", + "detail": "gh is needed only after a later publication review", + }, + { + "name": "github_login", + "ok": gh_authenticated, + "severity": "warning", + "detail": "gh authentication is needed only after a later publication review", + }, + ] + errors = [check for check in checks if not check["ok"] and check["severity"] == "error"] + next_steps = [ + f"leftovers --config {config_path} validate", + f"leftovers --config {config_path} doctor", + f"leftovers --config {config_path} scout", + ] + if not codex.authenticated: + next_steps.insert(0, "codex login") + if not image_present: + next_steps.insert(0, "make sandbox-image") + if not github_read_token: + next_steps.insert(0, "set GITHUB_TOKEN to a read-only public-repository token") + return (0 if not errors else 3), { + "configured": True, + "ready": not errors, + "config": str(config_path), + "mode": "dry-run", + "checks": checks, + "next_steps": next_steps, + "publication_enabled": False, + "packages_installed": False, + "scheduler_installed": False, + } diff --git a/src/leftovers/prompt_templates/system.md b/src/leftovers/prompt_templates/system.md index 4151098..1f6e4f0 100644 --- a/src/leftovers/prompt_templates/system.md +++ b/src/leftovers/prompt_templates/system.md @@ -20,9 +20,10 @@ data into a patch or result. Make the smallest complete change. Never claim a co its captured result proves it. Stop when the task involves security disclosure, ambiguous product decisions, forbidden paths, unexpected credentials, scope-limit breach, or insufficient evidence. -Your final response must be strict JSON written to: +Your final response must be strict JSON delivered through the adapter at: `{{LEFTOVERS_RESULT_PATH}}` -Do not use Markdown fences around that file's JSON. Normal progress text on stdout is permitted, -but the result file is authoritative. +The adapter, not a model-generated command, owns that output path. Return only the stage JSON as +your final message; do not try to create or modify the result file with a tool. Do not use Markdown +fences around the JSON. The adapter-produced result file is authoritative. diff --git a/src/leftovers/runner.py b/src/leftovers/runner.py index 3c94a3b..1a01b70 100644 --- a/src/leftovers/runner.py +++ b/src/leftovers/runner.py @@ -18,6 +18,16 @@ from typing import Any from .audit import redact +from .codex_adapter import ( + CODEX_ADAPTER_VERSION, + CodexAdapterError, + build_codex_argv, + codex_process_environment, + parse_codex_usage, + resolve_codex_executable, + validate_codex_workspace, + write_stage_schema, +) from .config import AgentConfig, SandboxConfig from .models import AgentResult, CommandResult, TokenUsage, isoformat, utc_now from .prompts import RenderedPrompt @@ -336,6 +346,39 @@ def _bounded(value: str, maximum: int) -> str: return "[TRUNCATED]\n" + value[-maximum:] +def _append_adapter_telemetry(path: Path, event: dict[str, Any]) -> None: + encoded = (json.dumps(event, separators=(",", ":"), sort_keys=True) + "\n").encode() + if len(encoded) > _TELEMETRY_MAX_LINE_BYTES: + raise AgentOutputError("adapter telemetry line is oversized") + flags = ( + os.O_WRONLY + | os.O_APPEND + | os.O_CREAT + | getattr(os, "O_NOFOLLOW", 0) + ) + descriptor = os.open(path, flags, 0o600) + try: + info = os.fstat(descriptor) + if ( + not stat.S_ISREG(info.st_mode) + or info.st_uid != os.getuid() + or info.st_nlink != 1 + ): + raise AgentOutputError("adapter telemetry must be an owner-controlled regular file") + if info.st_size + len(encoded) > _TELEMETRY_MAX_BYTES: + raise AgentOutputError("adapter telemetry exceeds the byte limit") + os.fchmod(descriptor, 0o600) + pending = memoryview(encoded) + while pending: + written = os.write(descriptor, pending) + if written < 1: + raise AgentOutputError("adapter telemetry write made no progress") + pending = pending[written:] + os.fsync(descriptor) + finally: + os.close(descriptor) + + def execute( argv: list[str], *, @@ -615,7 +658,8 @@ def run_agent( telemetry_callback: Callable[[dict[str, Any]], None] | None = None, ) -> AgentResult: output_dir, result_path, telemetry_path = self._output(workspace, stage) - for stale_path in (result_path, telemetry_path): + schema_path = output_dir / "result.schema.json" + for stale_path in (result_path, telemetry_path, schema_path): if stale_path.exists(): stale_path.unlink() monitor = _AdapterTelemetryMonitor( @@ -624,8 +668,44 @@ def run_agent( telemetry_callback, allow_synthetic_usage=self.allow_synthetic_usage, ) + codex_sequence = 0 + codex_heartbeat_at = time.monotonic() + + def append_codex_event(event_type: str, **fields: Any) -> None: + nonlocal codex_sequence + codex_sequence += 1 + _append_adapter_telemetry( + telemetry_path, + { + "version": 1, + "sequence": codex_sequence, + "type": event_type, + **fields, + "observed_at": isoformat(utc_now()), + }, + ) + + if self.agent.backend == "codex-cli": + try: + validate_codex_workspace(workspace) + except CodexAdapterError as exc: + raise RunnerError(str(exc)) from exc + append_codex_event( + "checkin", + provider=self.agent.provider, + model=self.agent.model, + adapter_version=CODEX_ADAPTER_VERSION, + capabilities=["structured-output", "provider-usage", "permission-profile"], + ) def telemetry_tick() -> None: + nonlocal codex_heartbeat_at + if ( + self.agent.backend == "codex-cli" + and time.monotonic() - codex_heartbeat_at >= 30 + ): + append_codex_event("heartbeat") + codex_heartbeat_at = time.monotonic() if telemetry_callback is not None: telemetry_callback( { @@ -657,6 +737,73 @@ def telemetry_tick() -> None: timeout=_effective_timeout(self.agent.timeout_seconds, deadline), on_tick=telemetry_tick, ) + elif self.agent.backend == "codex-cli": + executable = resolve_codex_executable(self.agent.command[0]) + if executable is None: + raise RunnerError( + "Codex CLI is not installed or its configured path is unavailable" + ) + config_path = workspace / ".git/config" + local_config_before = config_path.read_bytes() + before = self._host_readonly_fingerprint(workspace) if read_only_workspace else None + prompt_text = prompt.text.replace("{{LEFTOVERS_RESULT_PATH}}", str(result_path)) + try: + isolated_home = workspace.parent / "codex-agent-home" + isolated_home.mkdir(mode=0o700, exist_ok=True) + home_info = isolated_home.lstat() + if ( + not stat.S_ISDIR(home_info.st_mode) + or stat.S_ISLNK(home_info.st_mode) + or home_info.st_uid != os.getuid() + ): + raise CodexAdapterError("Codex isolated home is not owner-controlled") + os.chmod(isolated_home, 0o700) + write_stage_schema(schema_path, stage) + codex_argv = build_codex_argv( + executable, + stage=stage, + workspace=workspace, + schema_path=schema_path, + result_path=result_path, + model=self.agent.model, + read_only_workspace=read_only_workspace, + ) + command_result = execute( + codex_argv, + cwd=workspace, + env=codex_process_environment(isolated_home), + stdin=prompt_text, + timeout=_effective_timeout(self.agent.timeout_seconds, deadline), + max_output_bytes=self.agent.max_output_bytes, + on_tick=telemetry_tick, + ) + if command_result.passed: + usage = parse_codex_usage(command_result.stdout_tail) + append_codex_event( + "usage", + input_tokens=usage.input_tokens, + output_tokens=usage.output_tokens, + cached_input_tokens=usage.cached_input_tokens, + reasoning_tokens=usage.reasoning_tokens, + total_tokens=usage.total_tokens, + source=usage.source, + exact=usage.exact, + final=True, + ) + except CodexAdapterError as exc: + raise RunnerError(str(exc)) from exc + if not config_path.is_file() or config_path.read_bytes() != local_config_before: + raise RunnerError(f"Codex agent modified Git control metadata during {stage}") + from .policy import unsafe_git_configuration + + dangerous_config = unsafe_git_configuration(workspace) + if dangerous_config: + raise RunnerError( + "repository contains unsafe local Git configuration: " + + ", ".join(dangerous_config) + ) + if before is not None and self._host_readonly_fingerprint(workspace) != before: + raise RunnerError(f"Codex agent modified the workspace during read-only {stage}") else: config_path = workspace / ".git/config" local_config_before = config_path.read_bytes() diff --git a/tests/test_cli.py b/tests/test_cli.py index ecc9e36..1361f37 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -26,6 +26,40 @@ def to_dict(self) -> dict[str, object]: class CliTests(unittest.TestCase): + def test_setup_codex_does_not_require_an_existing_config(self) -> None: + stdout = io.StringIO() + report = {"configured": True, "ready": True} + with ( + patch("leftovers.cli.load_config", side_effect=AssertionError("must not load")), + patch("leftovers.cli.setup_codex", return_value=(0, report)) as setup, + redirect_stdout(stdout), + ): + status = main( + [ + "--config", + "new.toml", + "setup", + "codex", + "--repository", + "owner/repo", + "--ai-policy-url", + "https://github.com/owner/repo/blob/main/CONTRIBUTING.md", + "--ai-policy-reviewed", + "--allowed-license", + "MIT", + "--test-command-json", + '["python","-m","unittest"]', + "--allocated-tokens", + "150000", + ] + ) + self.assertEqual(status, 0) + self.assertEqual(json.loads(stdout.getvalue()), report) + inputs = setup.call_args.args[1] + self.assertEqual(inputs.repository, "owner/repo") + self.assertEqual(inputs.test_commands, (("python", "-m", "unittest"),)) + self.assertTrue(inputs.ai_policy_reviewed) + def test_cleanup_protects_container_and_reserved_controller_runs(self) -> None: with tempfile.TemporaryDirectory() as directory: root = Path(directory) diff --git a/tests/test_codex_adapter.py b/tests/test_codex_adapter.py new file mode 100644 index 0000000..c9217a2 --- /dev/null +++ b/tests/test_codex_adapter.py @@ -0,0 +1,173 @@ +import json +import os +import stat +import tempfile +import unittest +from pathlib import Path +from subprocess import CompletedProcess +from unittest import mock + +from leftovers.codex_adapter import ( + CodexAdapterError, + build_codex_argv, + codex_process_environment, + inspect_codex_cli, + parse_codex_usage, + stage_result_schema, + validate_codex_workspace, + write_stage_schema, +) + + +class CodexAdapterTests(unittest.TestCase): + def test_cli_inspection_requires_current_version_login_and_model(self) -> None: + probes = [ + CompletedProcess(["codex", "--version"], 0, "codex-cli 0.145.0-alpha.18\n", ""), + CompletedProcess(["codex", "login", "status"], 0, "logged in\n", ""), + CompletedProcess( + ["codex", "debug", "models", "--bundled"], + 0, + json.dumps({"models": [{"slug": "gpt-5.6-luna"}]}), + "", + ), + ] + with ( + mock.patch( + "leftovers.codex_adapter.resolve_codex_executable", + return_value="/bin/codex", + ), + mock.patch("leftovers.codex_adapter._run_codex_probe", side_effect=probes), + ): + inspection = inspect_codex_cli("codex", "gpt-5.6-luna") + self.assertTrue(inspection.ready) + self.assertEqual(inspection.version, "codex-cli 0.145.0-alpha.18") + + def test_controller_builds_least_privilege_argv(self) -> None: + root = Path(tempfile.mkdtemp()) + self.addCleanup(lambda: __import__("shutil").rmtree(root)) + workspace = root / "repo" + workspace.mkdir() + argv = build_codex_argv( + "/opt/codex/bin/codex", + stage="implementation", + workspace=workspace, + schema_path=root / "schema.json", + result_path=root / "result.json", + model="gpt-5.6-luna", + read_only_workspace=False, + ) + joined = "\n".join(argv) + self.assertNotIn("--sandbox", argv) + self.assertIn("--ask-for-approval", argv) + self.assertIn("--ignore-user-config", argv) + self.assertIn("--ignore-rules", argv) + self.assertIn("project_doc_max_bytes=0", argv) + self.assertIn("mcp_servers={}", argv) + self.assertIn('web_search="disabled"', argv) + self.assertIn('default_permissions="leftovers-write"', argv) + self.assertIn('":workspace_roots"={"."="write"', joined) + self.assertIn('".git"="read"', joined) + self.assertIn('".agents"="deny"', joined) + self.assertIn('".codex"="deny"', joined) + self.assertIn('permissions.leftovers-write.network.enabled=false', argv) + for feature in ("apps", "hooks", "multi_agent", "remote_plugin"): + self.assertIn(feature, argv) + self.assertNotIn("--dangerously-bypass-approvals-and-sandbox", argv) + self.assertEqual(argv[-1], "-") + + def test_stage_permissions_must_match_stage(self) -> None: + with self.assertRaisesRegex(CodexAdapterError, "permission do not match"): + build_codex_argv( + "codex", + stage="review", + workspace=Path("/tmp/repo"), + schema_path=Path("/tmp/schema"), + result_path=Path("/tmp/result"), + model="model", + read_only_workspace=False, + ) + + def test_codex_process_environment_drops_all_ambient_credentials(self) -> None: + source = { + "HOME": "/home/operator", + "PATH": "/usr/bin", + "GITHUB_TOKEN": "github-secret", + "GH_TOKEN": "gh-secret", + "OPENAI_API_KEY": "api-secret", + "CODEX_ACCESS_TOKEN": "codex-secret", + "SSH_AUTH_SOCK": "/tmp/agent.sock", + "AWS_SECRET_ACCESS_KEY": "cloud-secret", + } + with mock.patch.dict(os.environ, source, clear=True): + environment = codex_process_environment() + self.assertEqual(environment, {"HOME": "/home/operator", "PATH": "/usr/bin"}) + with tempfile.TemporaryDirectory() as directory: + with mock.patch.dict(os.environ, source, clear=True): + isolated = codex_process_environment(Path(directory)) + self.assertEqual(isolated["HOME"], str(Path(directory).resolve())) + self.assertEqual(isolated["CODEX_HOME"], "/home/operator/.codex") + self.assertNotIn("GITHUB_TOKEN", isolated) + + def test_repository_local_codex_skills_are_refused(self) -> None: + with tempfile.TemporaryDirectory() as directory: + workspace = Path(directory) + validate_codex_workspace(workspace) + (workspace / ".agents" / "skills").mkdir(parents=True) + with self.assertRaisesRegex(CodexAdapterError, "skills are refused"): + validate_codex_workspace(workspace) + + def test_provider_usage_is_parsed_from_final_jsonl_event(self) -> None: + output = "\n".join( + [ + json.dumps({"type": "turn.started"}), + json.dumps( + { + "type": "turn.completed", + "usage": { + "input_tokens": 100, + "cached_input_tokens": 40, + "output_tokens": 20, + "reasoning_output_tokens": 5, + }, + } + ), + ] + ) + usage = parse_codex_usage(output) + self.assertEqual(usage.total_tokens, 120) + self.assertEqual(usage.reasoning_tokens, 5) + self.assertTrue(usage.exact) + self.assertEqual(usage.source, "provider_response") + + def test_missing_or_invalid_provider_usage_fails_closed(self) -> None: + with self.assertRaisesRegex(CodexAdapterError, "usage receipt"): + parse_codex_usage('{"type":"turn.failed"}') + with self.assertRaisesRegex(CodexAdapterError, "cached input"): + parse_codex_usage( + json.dumps( + { + "type": "turn.completed", + "usage": { + "input_tokens": 1, + "cached_input_tokens": 2, + "output_tokens": 1, + }, + } + ) + ) + + def test_stage_schemas_are_closed_and_written_owner_only(self) -> None: + for stage in ("planning", "implementation", "review"): + with self.subTest(stage=stage): + schema = stage_result_schema(stage) + self.assertIs(schema["additionalProperties"], False) + self.assertEqual(set(schema["required"]), set(schema["properties"])) + with tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "schema.json" + write_stage_schema(path, "review") + self.assertEqual(stat.S_IMODE(path.stat().st_mode), 0o600) + self.assertEqual(json.loads(path.read_text())["additionalProperties"], False) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_config.py b/tests/test_config.py index df0c024..bb6d14c 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -37,6 +37,62 @@ def test_minimal_config_loads(self) -> None: self.assertEqual(config.repositories[0].slug, "owner/repo") self.assertEqual(config.github.api_version, "2026-03-10") + def test_codex_cli_backend_has_a_fixed_adapter_contract(self) -> None: + configured = BASE.replace( + 'backend = "container"\ncommand = ["agent"]', + 'backend = "codex-cli"\n' + 'command = ["/opt/codex/bin/codex"]\n' + 'provider = "openai-codex-cli"\n' + 'model = "gpt-5.6-luna"\n' + 'checkin_required = true\n' + 'usage_reporting_required = true', + ) + config = load_config(self.write(configured)) + self.assertEqual(config.agent.backend, "codex-cli") + + def test_codex_cli_backend_rejects_user_arguments_and_environment(self) -> None: + base = BASE.replace( + 'backend = "container"\ncommand = ["agent"]', + 'backend = "codex-cli"\n' + 'command = ["codex", "--yolo"]\n' + 'provider = "openai-codex-cli"\n' + 'model = "gpt-5.6-luna"\n' + 'checkin_required = true\n' + 'usage_reporting_required = true', + ) + with self.assertRaisesRegex(ConfigError, "only the Codex executable"): + load_config(self.write(base)) + exposed = base.replace( + 'command = ["codex", "--yolo"]', + 'command = ["codex"]\npass_environment = ["SAFE_LOOKING_VALUE"]', + ) + with self.assertRaisesRegex(ConfigError, "does not accept pass_environment"): + load_config(self.write(exposed)) + + def test_codex_cli_backend_cannot_enable_draft_publication(self) -> None: + digest = "a" * 64 + configured = ( + BASE.replace( + "[agent]", + f'[sandbox]\nimage = "leftovers@sha256:{digest}"\n\n[agent]', + ) + .replace( + 'backend = "container"\ncommand = ["agent"]', + 'backend = "codex-cli"\n' + 'command = ["codex"]\n' + 'provider = "openai-codex-cli"\n' + 'model = "gpt-5.6-luna"\n' + 'checkin_required = true\n' + 'usage_reporting_required = true', + ) + .replace( + 'mode = "dry-run"', + 'mode = "draft-pr"\nexpected_login = "leftovers-bot"\nexpected_user_id = 1', + ) + ) + with self.assertRaisesRegex(ConfigError, "codex-cli is dry-run only"): + load_config(self.write(configured)) + def test_unknown_keys_are_rejected(self) -> None: with self.assertRaisesRegex(ConfigError, "unknown key"): load_config(self.write(BASE + "\n[github]\ntyop = true\n")) diff --git a/tests/test_onboarding.py b/tests/test_onboarding.py new file mode 100644 index 0000000..236e648 --- /dev/null +++ b/tests/test_onboarding.py @@ -0,0 +1,105 @@ +import stat +import tempfile +import unittest +from pathlib import Path +from unittest import mock + +from leftovers.codex_adapter import CodexCliInspection +from leftovers.config import ConfigError, load_config +from leftovers.onboarding import CodexSetupInputs, parse_argv_json, setup_codex + + +class OnboardingTests(unittest.TestCase): + def inputs(self) -> CodexSetupInputs: + return CodexSetupInputs( + repository="owner/repo", + ai_policy_url="https://github.com/owner/repo/blob/main/CONTRIBUTING.md", + ai_policy_reviewed=True, + test_commands=(("python", "-m", "unittest"),), + allowed_licenses=("MIT",), + allow_labels=("help wanted",), + default_branch="main", + model="gpt-5.6-luna", + allocated_tokens=150_000, + reserve_tokens=20_000, + window="daily", + timezone="America/Phoenix", + runtime="docker", + ) + + def test_setup_writes_valid_owner_only_dry_run_config(self) -> None: + with tempfile.TemporaryDirectory() as directory: + target = Path(directory) / "config" / "leftovers.toml" + inspection = CodexCliInspection( + "/Applications/ChatGPT.app/Contents/Resources/codex", + "codex-cli 0.145.0", + True, + True, + True, + ) + with ( + mock.patch("leftovers.onboarding.inspect_codex_cli", return_value=inspection), + mock.patch("leftovers.onboarding.shutil.which", return_value="/usr/bin/tool"), + mock.patch("leftovers.onboarding._gh_authenticated", return_value=True), + mock.patch("leftovers.onboarding.container_image_available", return_value=True), + mock.patch.dict("os.environ", {"GITHUB_TOKEN": "read-only-test-token"}), + ): + status, report = setup_codex(target, self.inputs()) + self.assertEqual(status, 0) + self.assertTrue(report["ready"]) + self.assertFalse(report["publication_enabled"]) + self.assertFalse(report["packages_installed"]) + self.assertEqual(stat.S_IMODE(target.stat().st_mode), 0o600) + config = load_config(target) + self.assertEqual(config.agent.backend, "codex-cli") + self.assertEqual(config.agent.command, (inspection.executable,)) + self.assertEqual(config.publication.mode, "dry-run") + self.assertTrue(config.repositories[0].require_human_approval) + + def test_setup_refuses_overwrite_and_symlink_targets(self) -> None: + with tempfile.TemporaryDirectory() as directory: + target = Path(directory) / "leftovers.toml" + target.write_text("keep") + inspection = CodexCliInspection("/usr/bin/codex", "codex-cli 1.0.0", True, True, True) + with ( + mock.patch("leftovers.onboarding.inspect_codex_cli", return_value=inspection), + self.assertRaisesRegex(ConfigError, "refusing to overwrite"), + ): + setup_codex(target, self.inputs()) + self.assertEqual(target.read_text(), "keep") + target.unlink() + outside = Path(directory) / "outside" + outside.write_text("keep") + target.symlink_to(outside) + with ( + mock.patch("leftovers.onboarding.inspect_codex_cli", return_value=inspection), + self.assertRaisesRegex(ConfigError, "refusing to overwrite"), + ): + setup_codex(target, self.inputs()) + self.assertEqual(outside.read_text(), "keep") + + def test_setup_requires_policy_test_license_and_real_budget_confirmation(self) -> None: + unsafe = self.inputs() + for replacement, message in ( + ({"ai_policy_reviewed": False}, "policy was reviewed"), + ({"test_commands": ()}, "test command"), + ({"allowed_licenses": ()}, "SPDX"), + ({"allocated_tokens": 100_000}, "at least 100000"), + ): + with self.subTest(message=message): + values = {**unsafe.__dict__, **replacement} + with self.assertRaisesRegex(ConfigError, message): + setup_codex(Path("unused"), CodexSetupInputs(**values)) + + def test_test_commands_are_strict_json_argv_arrays(self) -> None: + self.assertEqual( + parse_argv_json('["python","-m","pytest","-q"]'), + ("python", "-m", "pytest", "-q"), + ) + for unsafe in ('"python -m pytest"', "[]", '["sh", 2]', '["sh", ""]'): + with self.subTest(unsafe=unsafe), self.assertRaises(ConfigError): + parse_argv_json(unsafe) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_runner.py b/tests/test_runner.py index 00962dc..a1a78cb 100644 --- a/tests/test_runner.py +++ b/tests/test_runner.py @@ -1,3 +1,4 @@ +import json import os import tempfile import unittest @@ -20,6 +21,78 @@ class RunnerTests(unittest.TestCase): + def test_codex_cli_backend_converts_structured_output_and_usage(self) -> None: + root = Path(tempfile.mkdtemp()) + self.addCleanup(lambda: __import__("shutil").rmtree(root)) + workspace = root / "repo" + workspace.mkdir() + subprocess = __import__("subprocess") + subprocess.run(["git", "init", "-q"], cwd=workspace, check=True) + runner = AgentRunner( + SandboxConfig(runtime="docker", image="image@sha256:abc"), + AgentConfig( + backend="codex-cli", + command=("codex",), + provider="openai-codex-cli", + model="gpt-5.6-luna", + checkin_required=True, + usage_reporting_required=True, + ), + ) + result_payload = { + "status": "implemented", + "summary": "implemented fixture", + "changed_files": ["example.py"], + "commands": [], + "acceptance_criteria": [{"criterion": "works", "evidence": "fixture"}], + "remaining_risks": [], + "reason": "", + } + captured: dict[str, object] = {} + + def fake_execute(argv: list[str], **kwargs: object) -> CommandResult: + captured["argv"] = argv + captured["env"] = kwargs["env"] + result_path = Path(argv[argv.index("--output-last-message") + 1]) + result_path.write_text(json.dumps(result_payload)) + usage_event = { + "type": "turn.completed", + "usage": { + "input_tokens": 100, + "cached_input_tokens": 25, + "output_tokens": 20, + "reasoning_output_tokens": 5, + }, + } + return CommandResult(tuple(argv), 0, 0.01, json.dumps(usage_event), "") + + events: list[dict[str, object]] = [] + with ( + mock.patch("leftovers.runner.resolve_codex_executable", return_value="/usr/bin/codex"), + mock.patch("leftovers.runner.execute", side_effect=fake_execute), + ): + result = runner.run_agent( + "implementation", + workspace, + RenderedPrompt("implementation", "task", "0" * 64), + "run-1", + read_only_workspace=False, + telemetry_callback=events.append, + ) + self.assertEqual(result.status, "implemented") + self.assertIsNotNone(result.usage) + assert result.usage is not None + self.assertEqual(result.usage.total_tokens, 120) + self.assertEqual([event["type"] for event in events], ["checkin", "usage"]) + argv = captured["argv"] + self.assertIsInstance(argv, list) + self.assertIn('default_permissions="leftovers-write"', argv) + environment = captured["env"] + self.assertIsInstance(environment, dict) + self.assertNotIn("GITHUB_TOKEN", environment) + self.assertNotIn("CODEX_ACCESS_TOKEN", environment) + self.assertEqual(Path(environment["HOME"]).name, "codex-agent-home") + def test_host_agent_git_config_mutation_is_rejected_before_controller_git(self) -> None: root = Path(tempfile.mkdtemp()) self.addCleanup(lambda: __import__("shutil").rmtree(root))