b61e45bcb205020

http://nhiro.org.s3.amazonaws.com/5/8/5881af9f79578d6134e4d49f89a43e28.jpg https://gyazo.com/5881af9f79578d6134e4d49f89a43e28

(OCR text)

21

数式読解

A Note on BPTT for LSTM LM

Tomonari MASADA @ Nagasaki University

January 15, 2015

1 Forward pass

K is the vocabulary size. N is the number of hidden layers. Dn is the number of hidden units at the nth

layer. The input sequence is X = (xi,... , XT), where each at (xt1, ., xtK) is a one-hot vector.

input gates

forget gates

. η: cells

w: output gates

h cell outputs

LSTMによる隠れ層がN層重なっている構造