perk11/large-model-proxy

Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on different ports and loading/unloading them on demand

语言：Go · GitHub 星标：88 · Fork：9

话题：large-models · llms · localllm · proxy

在 GitHub 上查看原仓库