Look at everything, weigh what matters
Transformers were originally built for language, and their core mechanism is self-attention. Instead of only looking at nearby words the way a sliding filter looks at nearby pixels, self-attention lets every element of the input directly look at every other element, and learn how much attention to pay to each one, no matter how far apart they are in the sequence. In a sentence like 'the trophy didn't fit in the suitcase because it was too big,' figuring out what 'it' refers to requires connecting a word to another word far earlier in the sentence, and self-attention does this directly, in a single step, rather than needing to pass information through many intermediate layers.
Concretely, for every token the model computes a query, a key, and a value. The query represents what this token is looking for, the keys represent what every other token has to offer, and comparing queries against keys produces attention weights that say how relevant each other token is. Those weights are then used to blend the values into a new representation for each token, one that has effectively gathered relevant context from across the entire input at once.