clipperhouse/uax29

A tokenizer based on Unicode text segmentation (UAX #29), for Go. Split graphemes, words, sentences.

语言：Go · GitHub 星标：125 · Fork：7

话题：go · golang · nlp · tokenization · tokenizer · uax29 · unicode

在 GitHub 上查看原仓库