Bruce-Lee-LY/decoding_attention

Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.

语言：C++ · GitHub 星标：48 · Fork：4

话题：cuda · cuda-core · decoding-attention · flash-attention · flashinfer · flashmla · gpu · gqa

在 GitHub 上查看原仓库