alesha-pro/vllm-cross-model-kv

Cross-model KV cache transfer in vLLM: a smaller model prefills, a bigger one answers from its cache (independent implementation of arXiv:2608.03893, measured on 2x RTX 3090)

语言：Python · GitHub 星标：19 · Fork：1

在 GitHub 上查看原仓库