streaming
3 posts tagged streaming, newest first.
Posts
Checking the new chunk first: a repeated end tag in streamed replies in Xinference
Once a streamed reply's reasoning had ended, a later </think> in the answer was read as the end of the reasoning again.
A start tag inserted ahead of a newline: streamed reasoning in Xinference
A streamed reply that starts with a lone newline chunk, with reasoning_content on, put the model's own <think> inside reasoning_content.
Four early exits dropped the chunk: empty reasoning streams in Xinference
A streamed chat reply with no text at all, from a reasoning model that needs the <think> tag added, reached the client as data: [DONE] and nothing else.