huggingface/datatrove

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

语言：Python · GitHub 星标：3,417 · Fork：327

在 GitHub 上查看原仓库