1115714829/llama.cpp-v100

llama.cpp optimized for NVIDIA Tesla V100 (SM70): Qwen3.8-Flash-Next with MTP (prefill 3402 tok/s at 200K context on 6x V100-16GB; also runs on 2x V100-16GB) and Qwen3.8-27B with DFlash2 (decode up to 220 tok/s); 2-6 GPUs, 262K-512K context, multimodal, concurrent serving, built-in web UI.

语言：C++ · GitHub 星标：68 · Fork：13

话题：cuda · gguf · llama-cpp · llm-inference · local-llm · long-context · mixture-of-experts · moe

在 GitHub 上查看原仓库