rishiraj/jaxotron

Minimal, hackable 4D-parallel LLM pre-training and inference in pure JAX. Data, Tensor (+ Sequence), Pipeline, and Context parallelism (plus FSDP/ZeRO-3) for Llama 3.x, Qwen3, Qwen2.5, and SmolLM2 models in ~2,500 lines of readable code.

语言：Python · GitHub 星标：19 · Fork：0

在 GitHub 上查看原仓库