Skip to content

feat: audit Codex context savings - #3

Merged
Evostructs merged 1 commit into
mainfrom
agent/codex-context-savings
Aug 17, 2026
Merged

feat: audit Codex context savings#3
Evostructs merged 1 commit into
mainfrom
agent/codex-context-savings

Conversation

@Evostructs

Copy link
Copy Markdown
Owner

What

  • add denser audit, a fail-closed behavior-fidelity gate with a required known-bad control
  • add a versioned Codex CLI text-only/v1 profile for pre-bundled text decisions
  • publish a reproducible, seeded standard-vs-text-only benchmark and complete per-call evidence
  • package the work as 0.2.0a1 with an updated citation and case study

Why

A shorter instruction file is not the same as lower end-to-end model input. This change measures provider-reported full input and refuses to call unchanged scores convincing unless the workload also catches a known-bad change.

Recorded result

Codex CLI 0.147.0, gpt-5.6-sol, medium reasoning, three trials per case, randomized submission order:

  • release-operation decisions: 27/27 in each profile; 20,294.11 -> 18,154.00 input tokens/call (-10.55%)
  • automation permission routing: 15/15 in each profile; 20,619.00 -> 18,434.00 input tokens/call (-10.60%)
  • 84/84 calls passed; 0 operational errors; 0 transport fallbacks

The first strict run exposed one missing output-contract behavior. text-only/v1 fixes that boundary, and the complete 84-call audit was rerun.

Checks

  • 181 tests passed locally (2 integration tests deselected)
  • Ruff formatting and lint passed
  • mypy passed
  • wheel and source archive built successfully
  • wheel installed in a fresh Python 3.12 environment; CLI reported 0.2.0a1
  • staged-content sensitive-pattern scan found no credentials or local user paths

Scope

text-only is only for tasks whose complete input is already in the request. Coding, repository, shell, network, plugin, app, skill, or memory work must keep the standard profile.

@Evostructs
Evostructs merged commit 2e0ca1b into main Aug 17, 2026
5 checks passed
@Evostructs
Evostructs deleted the agent/codex-context-savings branch August 18, 2026 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant