guillaume-chevalier/Linear-Attention-Recurrent-Neural-Network

A recurrent attention module consisting of an LSTM cell which can query its own past cell states by the means of windowed multi-head attention. The formulas are derived from the BN-LSTM and the Transformer Network. The LARNN cell with attention can be easily used inside a loop on the cell state, just like any other RNN. (LARNN)

语言：Jupyter Notebook · GitHub 星标：148 · Fork：32

话题：attention-is-all-you-need · attention-mechanism · attention-model · lstm · pytorch · recurrent-neural-networks · rnn

在 GitHub 上查看原仓库