Bruce-Lee-LY/flash_attention_inference

Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

语言：C++ · GitHub 星标：46 · Fork：7

话题：cuda · cutlass · flash-attention · flash-attention-2 · gpu · inference · large-language-model · llm

在 GitHub 上查看原仓库