repeat vector and return_sequence in Keras

 

 

Essentially, return_sequences=True returns all the outputs the encoder observed in the past,

where RepeatVector repeats the very last output of the encoder

 

some tips for the use of the two statements:

one example is machine translation. For example, if you have a seq2seq model and you don't want to use teacher forcing and want a quick and dirty solution, you can pass in the last state of the encoder RNN (last blue box) using RepeatVector and return_sequences=False. But if you are to compute attention weights for the encoder states, you will need to use return_sequences=True as you need all encoder states to compute attention weights.