JouleCloudinferencedocsmodelspricingcomparestatus

Streaming preview

Streaming follows the OpenAI Server-Sent Events protocol: set stream: true and read incremental delta chunks.

Preview. Streaming is part of the OpenAI-compatible protocol JouleCloud implements, but end-to-end passthrough through the branded gateway is not yet verified for every model. Verify against your own key before you depend on it in production; the non-streaming path is fully supported.

Request

{
  "model": "qwen2.5-7b-instruct",
  "messages": [{"role": "user", "content": "Count to three."}],
  "stream": true
}

Response stream

The response is a text/event-stream. Each event is a data: line carrying a JSON chunk; new tokens arrive in choices[0].delta.content. The stream ends with a [DONE] sentinel.

data: {"choices":[{"delta":{"role":"assistant"}}]}

data: {"choices":[{"delta":{"content":"One"}}]}

data: {"choices":[{"delta":{"content":", two"}}]}

data: {"choices":[{"delta":{"content":", three."}}]}

data: {"choices":[{"delta":{},"finish_reason":"stop"}]}

data: [DONE]

With the SDKs

The OpenAI SDKs handle the SSE parsing for you — pass stream=True (Python) or stream: true (JS) and iterate the returned stream. Point the client at https://api.jouledns.com/v1 as usual.