feifeibear/long-context-attention

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

语言：Python · GitHub 星标：696 · Fork：87

话题：attention-is-all-you-need · deepspeed-ulysses · llm-inference · llm-training · pytorch · ring-attention

在 GitHub 上查看原仓库