OpenCSGs/llm-inference

llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource management, monitoring, and more.

语言：Python · GitHub 星标：99 · Fork：17

话题：deepspeed · llama-cpp · llm-inference · ray · transformer · vllm

在 GitHub 上查看原仓库