Ditto has real usage signals, but it does not yet have controlled evidence showing when the personal layer helps.
Build a standalone, disabled-by-default benchmark harness for Ditto Proof v1. This issue covers the harness only, not provider runs or public claims.
Acceptance criteria
- Freeze Ditto at
v0.3.7 / 5f4008b0c0df40dcadb92c8fd1ba4dcf3aee40d0 for every scored cell.
- Build exactly 24 paired comparisons and 48 isolated cell records: two systems, three task families, two variants, and two trials.
- Keep host-native persistent personalization absent in both conditions.
- Require fresh workspaces and dedicated homes for every cell.
- Record exact system identity, host version, tools, permissions, budgets, instruction hashes, fixture commit/hash, attempts, and artifact hashes.
- Make evidence append-only. A correction creates a superseding record instead of overwriting one.
- Allow one retry only for a provider or host failure before meaningful output.
- Require both
--execute and an exact manifest-hash approval before any provider command can run.
- Keep all private profiles, transcripts, reviewer records, and run artifacts outside Git.
- Leave
ditto.py, mining behavior, profiles, skills, and normal commands unchanged.
- Keep the existing Ditto test suite green and add focused Windows, UTF-8, privacy, and isolation coverage.
Not part of this issue
- Selecting or paying for systems
- Running the scored benchmark
- Recruiting reviewers
- Publishing proof clips or performance claims
- Antigravity changes
Implementation stops at a verified local harness and a separate execution approval gate.
Ditto has real usage signals, but it does not yet have controlled evidence showing when the personal layer helps.
Build a standalone, disabled-by-default benchmark harness for Ditto Proof v1. This issue covers the harness only, not provider runs or public claims.
Acceptance criteria
v0.3.7/5f4008b0c0df40dcadb92c8fd1ba4dcf3aee40d0for every scored cell.--executeand an exact manifest-hash approval before any provider command can run.ditto.py, mining behavior, profiles, skills, and normal commands unchanged.Not part of this issue
Implementation stops at a verified local harness and a separate execution approval gate.