sauravsingla/MemVanta

Low-memory C++20 LLM inference runtime for quantized GGUF models on CPU — mmap-backed weights, paged KV cache, Q4/Q8 kernels, and reproducible llama.cpp benchmarks.

语言：C++ · GitHub 星标：40 · Fork：40

话题：benchmarking · cpp · cpp20 · cpu-inference · edge-ai · gguf · inference · kv-cache

在 GitHub 上查看原仓库