swellweb/reame

CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accurate than tuning the engine. MIT, benchmarks and failures included.

语言：C++ · GitHub 星标：114 · Fork：10

话题：arm · cpu · edge-ai · gguf · inference · kv-cache · llama-cpp · llm

在 GitHub 上查看原仓库