Skip to content

Build an information-sensitive live SC2 belief benchmark - #1

Draft
jwm-dev wants to merge 4 commits into
mainfrom
codex/temporal-predictive-state
Draft

Build an information-sensitive live SC2 belief benchmark#1
jwm-dev wants to merge 4 commits into
mainfrom
codex/temporal-predictive-state

Conversation

@jwm-dev

@jwm-dev jwm-dev commented Aug 12, 2026

Copy link
Copy Markdown
Owner

What changed

  • Replaced the weak generic macro-intent test with a full-game TvZ hidden-commitment benchmark.
  • Added one shared-opening Zerg opponent with roach/ravager, mutalisk, and ling-flood branches that share all branch tech.
  • Added irreversible anti-armor, anti-air, and anti-swarm doctrines plus an auditable legal scanner-sweep action.
  • Added deterministic current-observation and privileged-state experts and a recurrent legal-history controller trained from privileged action advantages.
  • Extended full-game evaluation to 1,500 game seconds, added paired tournament/summarization tooling, committed the frozen checkpoint and aggregate artifacts, and added no-leak regression tests.
  • Rewrote REPORT.md around the controlled result and its limitations.

Result

Across six paired held-out contexts:

  • observation: 3-0-3, score 0.500
  • recurrent belief: 6-0-0, score 1.000
  • privileged oracle: 6-0-0, score 1.000
  • belief without scouting: 3-1-2, score 0.583

The recurrent controller recovers 100% of the observed oracle gap on this small mechanism test. The clean separation is concentrated in the mutalisk branch, so the report recommends a larger uniformly action-sensitive replication before architecture tuning.

Validation

  • .venv\Scripts\python -m pytest -q: 14 passed, 1 skipped
  • .venv\Scripts\ruff check .: passed
  • Frozen model loads in the bot environment (64 hidden units, 100 legal features)
  • Aggregate summary and figure regenerated from the four raw phase directories
  • git diff --check: passed

@jwm-dev jwm-dev changed the title Add temporal predictive-state analysis Test whether compact SC2 history is a decision state Aug 12, 2026
@jwm-dev jwm-dev changed the title Test whether compact SC2 history is a decision state Build and test live SC2 belief controllers Aug 12, 2026
@jwm-dev jwm-dev changed the title Build and test live SC2 belief controllers Build an information-sensitive live SC2 belief benchmark Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant