Generating the Output with Softmax in Seq2Seq Neural Networks
This article explains how the output of a Seq2Seq neural network is generated using a fully connected layer and a Softmax function to select the final output word.
Why it matters
Understanding how the output is generated in a Seq2Seq model is crucial for building effective neural machine translation systems.
Key Points
- 1The fully connected layer transforms the LSTM cell outputs into a vector of scores for each output word
- 2The Softmax function is applied to the fully connected layer outputs to select the most likely output word
- 3The selected output word is the Spanish translation of the input English word
Details
The article describes the final stage of a Seq2Seq neural network, where the outputs from the LSTM decoder cells are passed through a fully connected layer. This fully connected layer has two inputs (the two values from the top LSTM layer) and four outputs (one for each word in the Spanish vocabulary). The connections between the inputs and outputs have associated weights and biases. The output of the fully connected layer is then passed through a Softmax function, which selects the most likely output word by normalizing the scores into a probability distribution. The selected output word is the Spanish translation of the input English word.
No comments yet
Be the first to comment