xigh/herbert-rs

Local LLM inference engine written from scratch in Rust — hand-written AVX-512 assembly kernels, Metal & Vulkan compute shaders. Supports Qwen3, Mistral3, ... Q4/INT8/BF16 quantization.

语言：Rust · GitHub 星标：44 · Fork：5

话题：gemma · ia · inference · inference-engine · llm · moe · opensource · openweight

在 GitHub 上查看原仓库