benmaster82/colizig

A from-scratch Zig inference engine for large MoE models - stream experts from SSD → RAM → VRAM and keep the resident working set small. ColiZig explores a different way to run Mixture-of-Experts models: instead of loading the entire model into memory, it treats expert weights as a tiered working set and streams them between SSD, RAM and opt. VRAM

语言：Zig · GitHub 星标：4 · Fork：1

话题：cuda · engine · llm · llm-inference · mixture-of-experts · moe · nvidia-gpu · qwen3

在 GitHub 上查看原仓库