Attention

Every word decides which other words are worth reading.

Part of the What's inside track on lAItest.

"The trophy did not fit in the suitcase because it was too big." What is "it"?

Change one adjective and the answer moves to a different noun.

Attention is a word choosing what to read.

For each word, the model produces a score against every other word in the passage. Those scores are turned into weights that add up to one, and the word rebuilds itself as a blend of the others in those proportions. A high weight means read this closely. A low weight means mostly ignore it.

Try it

Focus "it". Then switch to the "too small" sentence and focus "it" again. One adjective changed; nothing else did. This step is an interactive widget; open the lesson to use it.

A common misconception

Commonly believed: The model resolves "it" by applying a grammar rule someone wrote down.

Actually: No rule exists anywhere in the file. The scores fall out of parameters trained on text where that pattern held. That is why attention handles cases a hand-written rule would miss — and why it can also point confidently at the wrong noun with no warning that it has.

In one sentence

Attention is the model deciding, for every word, which other words are worth reading.