Positional Encoding
Positional encoding is how a Transformer knows word order. This topic compares sinusoidal, learned and rotary schemes such as RoPE, and explains why context length claims often hinge on this one mechanism.
Self-attention treats a sentence as a bag of tokens, so positional encoding supplies the missing sense of order. Early Transformers added sinusoidal patterns, many current large models rotate token vectors with RoPE instead, and the choice shapes how far a model can read before distant text turns fuzzy. Following this topic makes long-context marketing numbers much easier to judge.
