Deep Recurrent Neural Networks (RNN) are a type of Artificial Neural Network that takes the networks previous hidden state as part of its input, effectively allowing the network to have a memory. This makes RNNs useful for modeling sequential or time-series data such as stock prices. However, training RNNs on sequences greater than a few hundred time steps can be difficult. In this post, we will explore three tools that can allow for more efficient training of RNN models with long sequences: Optimizers, Gradient Clipping, and Batch Sequence Length.
Read the rest of the article at Mindboard’s Medium channel.