rulith-dev/rulith-inference

Rulith Inference, formerly Strix Llama. Serving a 125B MoE model fast on one AMD Strix Halo machine, on Windows: a patched llama.cpp, a manager, and a desktop app.

语言：Python · GitHub 星标：48 · Fork：6

话题：amd · gfx1151 · llama-cpp · local-llm · moe · qwen · rocm · strix-halo

在 GitHub 上查看原仓库