Understanding Seq2Seq Neural Networks - Decoder Outputs and the Fully Connected Layer

This article explores the decoder part of a Seq2Seq neural network, focusing on the embedding values, LSTM layers, and the fully connected layer that transforms the LSTM outputs.

Details

The article explains the process of decoding in a Seq2Seq neural network. After the encoder has finished processing the input sentence, the decoder starts with the embedding values for the <start> token. The decoder then uses two layers of LSTM cells, each with two LSTM cells, to perform computations on the input embeddings. The output values (also known as the short-term memories or hidden states) from the top LSTM layer are then transformed using additional weights and biases in a fully connected layer. This fully connected layer is an important part of the decoder, as it helps to generate the output sequence based on the LSTM outputs.