Skip to content

Add int8 shared-block bigram+cache prototype#704

Open
sauravtom wants to merge 1 commit intoopenai:mainfrom
sauravtom:codex/int8-bigram-cache
Open

Add int8 shared-block bigram+cache prototype#704
sauravtom wants to merge 1 commit intoopenai:mainfrom
sauravtom:codex/int8-bigram-cache

Conversation

@sauravtom
Copy link

Summary

  • add shared-block transformer support (12 logical layers from a 2-block bank) plus LSQ-lite QAT hooks in train_gpt.py
  • add bigram + short cache logit fusion, label smoothing, EMA/SWA options, and int8 export extras for priors/config
  • add submission folder records/track_10min_16mb/2026-03-25_Int8_Bigram_Cache_Proto with README, submission.json, train.log placeholder, and runnable train script copy

Testing

  • python3 -m py_compile train_gpt.py

Notes

  • This PR is a prototype submission package while compute-backed full 8xH100 run logs are pending.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant