dogmaticdev/SIMD-Tokenizer

A Hyper Optimzed Tokenizer written in handwritten assembly. Made for SSE2 cpu architectures.

语言：Assembly · GitHub 星标：5 · Fork：0

在 GitHub 上查看原仓库