Non-record: BigramHash(4096) + Cosine EMA + LZMA-9

[Alfaxad]

Summary

• Non-record submission tested on 1xH100 (940 steps, 10 min wallclock)
• Built on PR Record: LeakyReLU² + Legal Score-First TTT + Parallel Muon — val_bpb 1.1194 (3-seed mean) #549 stack with 4 targeted changes:
  • BigramHash expanded 2048→4096 for richer local context
  • Cosine EMA schedule (0.99→0.999) replacing fixed 0.997
  • Earlier late QAT activation (threshold 0.15→0.10)
  • LZMA preset 6→9 for maximum compression

Results (1xH100)

Metric	Value
Pre-quant val BPB	1.3628
Legal TTT BPB	1.4775
Artifact size	7.9MB (49% of 16MB)
Steps (1xH100, 10 min)	940

Notes

• 8xH100 evaluation pending (budget constraint) — on 8xH100 this runs ~7500 steps
• Artifact is very compact (7.9MB), leaving room for model expansion
• The cosine EMA creates a quantization gap (1.37→2.21 roundtrip) that TTT partially recovers — fixed EMA may be preferable
• Developed using ShinkaEvolve (Sakana AI) evolutionary code optimization with GPT-5.4 + Gemini 3 Pro as mutation operators

Test plan

• Verified script runs on 1xH100 with torchrun
• Training completes within 10 min wallclock
• Artifact under 16MB (7.9MB)
• Legal TTT produces valid BPB score
• 8xH100 evaluation (pending compute budget)

🤖 Generated with Claude Code