gitkaz/mlx_gguf_server

This is a FastAPI based LLM server. Load multiple LLM models (MLX or llama.cpp) simultaneously using multiprocessing.

语言：Python · GitHub 星标：19 · Fork：4

话题：llamacpp · llm · mlx

在 GitHub 上查看原仓库