cache, all chats
Starts from the Qwen3-8B settings saved with the lesson code. Assumes full attention in every layer, the cache stored at the value type shown, and nothing in GPU memory but weights and cache (no activations, framework overhead or fragmentation). GiB and MiB are powers of 1,024. 8-bit and 4-bit only divide bytes per value; real quantised caches also store scale factors. Qwen3-8B has 32 query heads sharing 8 KV heads: set KV heads to 32 to see the cache without sharing.
Every message has the same token count, as in the lesson's tokens_read: before reply k the app sends 2k − 1 messages, and reply tokens are not counted as read. The prefix-cache row is the lower bound where each of the 2 × replies × tokens written tokens is processed once; it needs the earlier tokens' cache kept in GPU memory, which is the other tab.