A tiny model server under load

	p50	p99
First word
Between words

Chats a second: Cache pages: Batch limit:

A toy server and 400 invented chats, not a measurement. A load test on your own traffic decides.

More settings

Shuffle the chats