Focus Weight Hunt

Focus Weight Hunt is an attention mechanism game that teaches the algorithm inside every transformer model ever built — not through equations, but through an interactive heat map you control with your hands. The attention mechanism allows a language model to resolve ambiguity, track pronouns, match verbs to their subjects across long distances, and understand the same word differently in different contexts. It does this by computing, for every token in a sequence, a weight distribution over every other token — how much should this word attend to every other word when building its representation? The weights must sum to one. Competition for attention is the mechanism. The game gives you the heat map and asks you to fill it in. A sentence appears. A target word is highlighted. You set the attention weights by tapping a grid — dim cells for low attention, vivid amber for high. The softmax constraint is enforced in real time: increasing one weight decreases all others. When the weights are set correctly, the output visualisation shows what the mechanism has computed: the target word's representation rendered as a colour blend of the tokens it attended to, proportional to the weights you assigned. A word attending eighty percent to its antecedent absorbs eighty percent of that antecedent's colour. Abstract computation made physical. Ten levels build from adjacent word relationships to pronoun resolution, lexical ambiguity, multi-head attention, and cross-lingual alignment. The Winograd Schema level shows both readings of an ambiguous sentence as competing heat maps. The polysemy level shows the same word producing completely different attention patterns — and completely different output representations — in two different sentences. The translation level shows cross-attention breaking down for structurally mismatched languages. After ten levels, the player understands why context matters, why transformers outperformed recurrent networks, and what a language model is actually computing when it processes a sentence.