JohnScheuer/mini-llm-inference-engine

High-performance Transformer inference engine in C++/CUDA — 32k tok/s peak, 87% INT8 Tensor Core ceiling, roofline-validated on RTX 2070

语言：C++ · GitHub 星标：9 · Fork：1

在 GitHub 上查看原仓库