Watch a tiny model write one token at a time

This toy has read 17 short lines and only looks at the last 3 tokens of the text so far. A real model has read far more text, works in smaller pieces of words, and can use everything written so far. The loop is the same: one pass, one token, add it, repeat. The toy takes a token’s chance from how often it followed those 3 tokens in its lines.