NeelM0906/Mference

Swift + Metal MoE inference for Apple Silicon: Qwen 3.6 35B at 23.5–29.3 tok/s decode with 2.20× faster long-prompt prefill on a 24 GB M5; Gemma 4 26B in ~2 GB, DeepSeek-V4-Flash 284B, Inkling-Small 276B, native Mac app, CLI, and OpenAI-compatible server.

语言：Swift · GitHub 星标：122 · Fork：17

话题：apple-silicon · deepseek · gemma · llm-inference · macos · metal · mixture-of-experts · qwen

在 GitHub 上查看原仓库