Two GPUs, shared or split

Requests a second: Prompt tokens: Link speed:

Toy values and 600 invented requests, not a measurement of any real GPU or link.

More settings

Reply words:

How many words a typical reply has. Each reply is half, the same or one and a half times this.

First word by:

A request is right only if its first word comes this soon.

Gap at most:

And only if no pause between two of its words is longer than this.