Helldez/BigMoeOnEdge

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

语言：C++ · GitHub 星标：697 · Fork：81

话题：android · cpp · edge-ai · gemma · gguf · inference · llama-cpp · llm

在 GitHub 上查看原仓库