Skip to content

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Turntable

Evidence-disciplined adversarial review for Codex.

Turntable is an experimental Codex skill designed to reduce the "human pleaser" failure mode during reviews. It rotates an agent from collaborative creation into a falsification workflow: define the review surface, search for realistic failure, capture reproducible receipts, and remove criticism that cannot survive an evidence gate.

Turntable is research-informed, not independently proven. Its cited research supports the problem and parts of the method; it does not establish that this skill outperforms an ordinary review. The repository includes a blinded evaluation protocol so that claim can be tested rather than assumed.

The joke is from The Office. The method is not a joke.

Why

AI assistants can favor user-affirming answers over critical engagement. That is useful friction reduction during creation, but a liability during review. Asking an agent to become angry or hostile can simply replace agreeable theater with adversarial theater.

Turntable instead asks:

  • What claim can be falsified?
  • What fails under realistic production conditions?
  • Which evidence establishes the failure?
  • What was not checked?
  • Which criticism should be deleted because it is unsupported?

Method

  1. Neutralize ownership: Convert user assertions into questions and treat ownership as irrelevant.

  2. Define scope and safety: Record success criteria, trust boundaries, assumptions, exclusions, and authorized verification methods.

  3. Search for failure: Check relevant correctness, production, security, privacy, UX, accessibility, testing, AI, and product risks.

  4. Build atomic findings: Keep one claim per finding with a location, trigger, expected behavior, impact, and fix.

  5. Gate hallucinations: Separate severity, evidence status, and confidence. Delete unsupported findings.

  6. Verify and report coverage: Prefer reproducible checks, disclose what was not tested, and accept a clean review as valid.

Install

From the repository root, copy the skill contents into the Codex skill directory:

skill_dir="${CODEX_HOME:-$HOME/.codex}/skills/turntable"
mkdir -p "$skill_dir"
cp -R skills/turntable/. "$skill_dir/"

The command is safe to rerun and does not create a nested turntable/turntable directory.

Invoke it in Codex:

Use $turntable to review this PR for production risks.

Other useful prompts:

Use $turntable to critique this product idea before launch.
Use $turntable to review this agent workflow for hallucination and prompt-injection risk.
Use $turntable to find the bugs a polite review might miss without inventing new ones.

Safety

Turntable uses local, read-only, and reversible verification by default. Reviewing a live product does not authorize form submissions, state changes, credential use, scanning, exploitation, or load testing. Those actions require explicit authorization for the exact target and action.

Evaluation

The benchmark in evals/ contains flawed artifacts and clean controls with hidden oracles. It is designed for paired baseline-versus-Turntable runs in fresh contexts.

python3 evals/benchmark.py check
python3 -m unittest discover -s evals -p 'test_*.py'
python3 evals/benchmark.py prepare --out /tmp/turntable-eval --seed 0

See evals/README.md for the blinded protocol and scoring format. Do not market Turntable as empirically superior until repeated results across models and domains support that claim.

Research Basis

The included reference distinguishes evidence motivating the method from evidence that would validate Turntable itself. See skills/turntable/references/research.md.

License

MIT

About

Adversarial Codex skill for evidence-backed AI, product, and code reviews

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages