facebookresearch/Exact-Byte-Level-Probabilities-from-Tokenized-LMs
Example implementation of "Exact Byte-Level Probabilities from Tokenized Language Models for FIM-Tasks and Model Ensembles" by Buu Phan, Brandon Amos, Itai Gat, Marton Havasi, Matthew Muckley, and Karen UllrichWork conducted as part of a MetaAI internship.
正在读取项目信息…