JustVugg/colibri

A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.

语言：C · GitHub 星标：48,457 · Fork：5,637

在 GitHub 上查看原仓库