AI Knowledge HubAI & Models
attention
Instead of weighing every word equally, the model links words that belong together. In 'the animal ate the branch', attention captures how 'animal' relates to 'branch'. How strongly those links are computed decides most of natural-language performance.
It is the core mechanism behind translation and summarization.
Why it mattersVisualizing which words a model attends to produces an attention map. It is the first place to look when asking why a passage or an answer came out that way.
ExampleInstead of weighing every word equally, the model links words that belong together. In 'the animal ate the branch', attention captures how 'animal' relates to 'branch'. How strongly those links are computed decides most of natural-language performance.