Inner Products Measure Alignment in Attention
Compute a dot product, read its sign and magnitude geometrically, and see how query-key scores become attention weights.
Tag
#alignment
1articlewith this tag.
Tag
1articlewith this tag.
Compute a dot product, read its sign and magnitude geometrically, and see how query-key scores become attention weights.