#### 从 RNN 说起

$h = f(x, U)\\ y = f(h, V)$

$h_t = f(x, h_{t-1}, U, W)\\ y_t = g(h_t, V)$

#### 正式引入注意力机制

Recurrent neural networks has been firmly established as state of the art approaches in sequence modeling and transduction problems such as language modeling and machine translation.