What are the main differences between the encoder and decoder in a Transformer?
Answer
The encoder builds rich bidirectional representations of the source sequence: every token attends to every other token. The decoder is built for generation: its self-attention is causally masked so each position only sees the past, and it inserts an extra cross-attention sub-layer that reads the encoder’s output. Same building blocks, different wiring for two different jobs: understanding vs generating.
(1) Self-Attention Masking: Encoder self-attention is unmasked (full bidirectional context); decoder self-attention is masked so position 
attends only to 
.
(2) Cross-Attention: Absent in the encoder; present in every decoder layer: queries from the decoder state, keys/values from the encoder output.
(3) Inputs & Role: Encoder consumes the source sequence once; decoder consumes the shifted-right target (teacher forcing) and produces next-token distributions.
Figure 1: Two stacks, three differences: the decoder adds a causal mask on self-attention and a cross-attention bridge to the encoder output.
Mathematical Formulation (decoder self-attention mask):
Where:
is the causal mask: 
above the diagonal zeroes out future positions after softmax; the encoder simply omits .
index query and key positions; 
means “past or present only”.
is the causal mask: 
above the diagonal zeroes out future positions after softmax; the encoder simply omits 
index query and key positions; 
means “past or present only”.| Aspect | Encoder | Decoder |
|---|---|---|
| Self-attention | Unmasked: all positions | Masked: past positions only (causal) |
| Cross-attention | Not present | Present: attends to encoder outputs |
| Positional encoding | Added to source embeddings | Added to target embeddings (shifted right) |
| Input | Source sequence | Shifted target + encoder outputs |
| Function | Encode source into contextual representations | Generate target autoregressively with source context |
Table 1: Encoder vs decoder at a glance. The decoder is a superset: same sub-layers plus masking and the cross-attention bridge.
Figure 2: Q/K/V sources for the three attention types: only cross-attention mixes sequences: Q from the decoder, K/V from the encoder.
Why “Shifted Right”: During training the decoder receives the target sequence shifted one position right (prefixed with a start token), so the prediction at position is supervised against token while the input only ever reveals tokens before it. This is teacher forcing, and it keeps training fully parallel despite autoregressive inference.
Leave a Reply