sefineh-ai/Amharic-Tokenizer

Syllable-aware BPE tokenizer for the Amharic language (አማርኛ) – fast, accurate, trainable.

语言：Python · GitHub 星标：99 · Fork：16

话题：african-languages · amharic · amharic-tokenizer · amharictokenizer · amhtokenizer · bpe · bpe-tokenizer · deep-learning

在 GitHub 上查看原仓库