mddunlap924/LLM-Inference-Serving

This repository demonstrates LLM execution on CPUs using packages like llamafile, emphasizing low-latency, high-throughput, and cost-effective benefits for inference and serving.

语言：Jupyter Notebook · GitHub 星标：9 · Fork：1

话题：deepspeed · large-language-models · llamacpp · llamafile · llm-inference · llm-serving · llms · vllm

在 GitHub 上查看原仓库