Apple Silicon dual-backend port of autoresearch (PyTorch MPS + MLX) with full Muon optimizer
-
Updated
Mar 23, 2026 - Python
#
muon-optimizer
Here are 17 public repositories matching this topic...
Unofficial Optimized Implementation for NorMuon
machine-learning optimizer artificial-intelligence muon llm muon-optimizer normuon normuon-optimizer
-
Updated
Apr 8, 2026 - Python
Megatron-LM fork for experiments on Alps
-
Updated
May 18, 2026 - Python
LLM pretraining from scratch on FineWeb dataset (architecture and all components explained), plus optimal use of GPU on SLURM cluster
rope model-evaluation gpu-optimization gqa layernorm rmsnorm llm-training llm-inference llm-evaluation swiglu bpe-tokenizer flashattention muon-optimizer
-
Updated
May 12, 2026 - Python
Don't tune the knee. Measure it.
-
Updated
Aug 25, 2026 - Python
A performance-optimized Muon optimizer implementation for PyTorch
-
Updated
Jun 3, 2026 - Python
PyTorch DDP and multi-GPU LLM training — from a minimal distributed example to NanoGPT speedruns, Muon, H100 profiling, and modded-nanogpt. FBA LAB https://bubblnet.com
training gpu modal pytorch ddp pytorch-tutorial gpt-2 distributed-data-parallel marin nanogpt llm-training h100 qwen deepseek olmo llm-systems muon-optimizer modded-nanogpt
-
Updated
Aug 20, 2026 - Python
🤖 Ultron (113M) — A SOTA LLM pre-trained from scratch on 10B FineWeb-Edu tokens featuring the Muon Optimizer, RoPE, GQA, SwiGLU, and zero-copy streaming. "There are no strings on me."
deep-neural-networks deep-learning pytorch muon rope pre-training pre-trained-model huggingface gqa huggingface-transformers llm muon-optimizer fineweb-edu
-
Updated
Aug 19, 2026 - Python
≈125M decoder-only transformer from first principles with RoPE, RMSNorm, QK-Norm, and Muon + AdamW.
-
Updated
Aug 14, 2026 - Python
NerVE: Nonlinear Eigenspectrum Dynamics in LLM Feed-Forward Networks (ICLR 2026)
transformers feedforward-neural-network representation-learning spectral-analysis adam-optimizer deep-learning-framework interpretable-machine-learning llms muon-optimizer eigenspectrum
-
Updated
Mar 16, 2026 - Jupyter Notebook
High-performance CUDA implementation of Muon optimizer for LLM training. Features Newton-Schulz polar decomposition, cuBLAS acceleration, and transpose optimization for 8x FLOP savings on transformer FFN layers. Benchmarked on NVIDIA A100 with Llama 3.1 8B architectures (4096×11008 weights).
-
Updated
Dec 21, 2025 - Python
OpenAI parameter-golf challenge: train the smallest LM that fits in 16MB. My SP8192 frontier submission (openai/parameter-golf#1887): 11L x 512d w/ targeted middle recurrence, MuonEq-R optimizer, int6 GPTQ+SDClip, Brotli-11 compression, legal score-first TTT. 8xH100 DDP.
openai quantization ddp model-compression distributed-training llm gptq h100 language-model-training muon-optimizer
-
Updated
Apr 28, 2026 - Python
From-scratch 125M LLM: 300K steps, 57.8B-token corpus, 282K tok/s on 2x H100
-
Updated
Aug 13, 2026 - Python
Official code for optimizer-induced spectral scaling laws
-
Updated
Jul 20, 2026 - Jupyter Notebook
Few-Shot Adaptation for Vision-Language Models. Implements Base-to-Novel generalization on CLIP using LoRA, LP++, and Muon Optimizer to enhance performance on the Oxford Flowers-102 dataset.
-
Updated
Dec 24, 2025 - Jupyter Notebook
Exploring new trendencies in numerical optimization applied on PINNs. An example with the nonlinear TDSE.
physics quantum-mechanics m3 lbfgs-algorithm tdse physics-informed-neural-networks nested-learning muon-optimizer time-dependent-schrodinger multiscale-momentum-muon
-
Updated
Jun 9, 2026 - Jupyter Notebook
ARS: Slide directly with the geodesic of the loss landscape to the global optimum.
adam-optimizer ars second-order-optimization free-energy-principle flatness formalized-realism muon-optimizer
-
Updated
Aug 21, 2026 - Python
Improve this page
Add a description, image, and links to the muon-optimizer topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the muon-optimizer topic, visit your repo's landing page and select "manage topics."