NetEase-Media/grps

Deep Learning Deployment Framework: Supports tf/torch/trt/trtllm/vllm and other NN frameworks. Support dynamic batching, and streaming modes. It is dual-language compatible with Python and C++, offering scalability, extensibility, and high performance. It helps users quickly deploy models and provide services through HTTP/RPC interfaces.

语言：C++ · GitHub 星标：172 · Fork：11

话题：dynamic-batching · serving · tensorflow · tensorrt · tensorrt-llm · torch · triton-inference-server · vllm

在 GitHub 上查看原仓库