expert-streaming

Here are 4 public repositories matching this topic...

xero00000 / expert-streaming-engine

Run oversized sparse MoE GGUF models with bounded NVMe-to-RAM-to-VRAM caching, native multi-GPU planning, and a Linux desktop Studio.

desktop-app linux cuda inference multi-gpu tauri mixture-of-experts llama-cpp local-llm gguf expert-streaming

Updated Sep 7, 2026
C++

haiyan-ai-lab / adaptive-moe-inference

Star

Adaptive MoE inference for Kimi K3 — beyond-memory expert streaming, reversible runtime profiles, measured optimization results, and an open NVIDIA/GPU/NPU adaptation roadmap.

cuda nvidia-gpu mixture-of-experts model-optimization llm-inference expert-streaming kimi-k3 moe-inference gpu-npu

Updated Aug 12, 2026
Python

AstrolexisAI / MnemoCUDA

Star

Expert streaming inference engine for MoE models larger than VRAM — run 235B+ models on consumer GPUs

gpu cuda inference moe quantization nvme vram mixture-of-experts llm expert-streaming

Updated Mar 30, 2026
C

CostanzoPadovano / llama.cpp-deepseek-v4-streaming

Star

Experimental llama.cpp runtime for bounded DeepSeek V4 MoE streaming on dual 16 GB GPUs

cuda moe llama-cpp gguf deepseek-v4 expert-streaming

Updated Aug 8, 2026
C++

Add this topic to your repo

To associate your repository with the expert-streaming topic, visit your repo's landing page and select "manage topics."

Learn more