twistedcubic/attention-rank-collapse

[ICML 2021 Oral] We show pure attention suffers rank collapse, and how different mechanisms combat it.

语言：Jupyter Notebook · GitHub 星标：173 · Fork：11

在 GitHub 上查看原仓库