Memory a language model needs

Pick a model and how precisely its numbers are stored, then drag the two sliders. The bar shows what the card's memory must hold.

Model

Bits stored per weight

Context length of each chat

Chats running at once

of memory needed

Does it fit on one card?

Sizes use GiB (2³⁰ bytes). A card's label says GB; this page reads it as GiB, except the L40S, whose 48 GB is 45 GiB with error-correcting memory on. A server claims about of a card by default, so the fit test uses that share. Working memory and the fixed allowance are example estimates.