Streaming

Set stream: true and receive the answer as it is written, over server-sent events.

Stream whenever a person is waiting, and whenever the answer might be long.

It feels faster. First words appear in a second or two instead of after the whole answer.
It avoids the 95-second cap. Blocking requests are cut off there; streamed ones are not.
It lets you stop early. Close the connection and xKiro aborts the upstream call, so you stop paying.

stream = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    messages=[{"role": "user", "content": "Write a haiku about caching."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

The wire format#

Responses are text/event-stream. Each event is a data: line holding one JSON object.

Chat Completions

data: {"id":"chatcmpl-9f1","object":"chat.completion.chunk","model":"openai/gpt-5.6-sol","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-9f1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Cache"},"finish_reason":null}]}

data: {"id":"chatcmpl-9f1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" hits"},"finish_reason":null}]}

data: {"id":"chatcmpl-9f1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":12,"completion_tokens":17,"total_tokens":29}}

data: [DONE]

Usage arrives at the end

Token counts are on the final chunk, not the first. If you meter usage yourself, read it there — and keep reading until [DONE], or you will record zero.

Messages (Anthropic)

The Anthropic dialect uses named events rather than one repeated shape. A turn opens with message_start, then pairs of content_block_start / content_block_delta / content_block_stop, and closes with message_delta and message_stop.

Messages

event: message_start
data: {"type":"message_start","message":{"id":"msg_01ABC","role":"assistant","content":[],"usage":{"input_tokens":12,"output_tokens":0}}}

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Cache"}}

event: content_block_stop
data: {"type":"content_block_stop","index":0}

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":17}}

event: message_stop
data: {"type":"message_stop"}

A stream that ends without a terminator is an error

A well-formed turn always ends with message_delta and message_stop (or [DONE]). If the connection closes before that, treat it as a failure and retry — not as an empty but successful answer.

Errors mid-stream#

Once the first byte is sent the HTTP status is already 200, so a later failure arrives as an event rather than a status code. Handle it explicitly — a reader that ignores it will show the user a truncated answer with no explanation.

Error frame

data: {"error":{"message":"Upstream provider is unavailable. Please retry.","type":"api_error","code":"upstream_error"}}

data: [DONE]

Raw SSE reader that handles errors

const res = await fetch("https://api.xkiro.com/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.XKIRO_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "openai/gpt-5.6-sol",
    stream: true,
    messages: [{ role: "user", content: "Hello" }],
  }),
});

if (!res.ok || !res.body) throw new Error(`HTTP ${res.status}`);

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "";

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });
  // Events are separated by a blank line. Keep the tail: a chunk boundary can
  // land in the middle of one, and parsing half an event throws.
  const events = buffer.split("\n\n");
  buffer = events.pop() ?? "";

  for (const event of events) {
    const line = event.split("\n").find((l) => l.startsWith("data:"));
    if (!line) continue;

    const payload = line.slice(5).trim();
    if (payload === "[DONE]") return;

    const json = JSON.parse(payload);
    if (json.error) throw new Error(json.error.message);

    const delta = json.choices?.[0]?.delta?.content;
    if (delta) process.stdout.write(delta);
  }
}

Cancelling#

Stopping the stream stops the work. xKiro detects the closed connection and aborts the upstream request, so you are billed only for what was generated up to that point.

Abort after 10 seconds

const controller = new AbortController();
setTimeout(() => controller.abort(), 10_000);

const stream = await client.chat.completions.create(
  {
    model: "openai/gpt-5.6-sol",
    messages: [{ role: "user", content: "Write an essay." }],
    stream: true,
  },
  { signal: controller.signal },
);

If you run a proxy in front#

Disable response buffering, or the whole answer arrives at once and streaming buys you nothing. In nginx: proxy_buffering off;.
Do not retry POST requests on timeout. A retried request runs a second time and is billed twice. In nginx: proxy_next_upstream off;.
Raise read timeouts above your longest expected generation, otherwise the proxy cuts a healthy stream.

See also Idempotency & retries for what happens when a request is sent twice.