Pick a model, a number of GPUs and a link. See what each GPU holds and how long one word takes.
Time per word, by GPU count
A toy. One made-up GPU: 80 GiB of memory (73.6 usable) that reads 2,039 GB a second. The fast link moves 600 GB a second and the slow link 64. Memory holds one chat of 8,192 words. Reading and waiting do not overlap here. The toy keeps one link for every count, though a model spread over several servers meets a slower link between them.