Streaming

Stream tokens in real time using Server-Sent Events — works with any OpenAI-compatible client.

Streaming

Streaming returns tokens as they're generated instead of waiting for the full response. This dramatically reduces time-to-first-token and makes chat interfaces feel responsive.

How It Works

Set stream=True (Python) or stream: true (Node.js). The API returns a stream of Server-Sent Events, each containing a delta with the new content. The stream ends with data: [DONE].

Python

``python


from openai import OpenAI

client = OpenAI(
    api_key="YOUR_QWENAPI_KEY",
    base_url="https://dashscope-us.aliyuncs.com/compatible-mode/v1",
)

stream = client.chat.completions.create(
    model="qwen3.5-plus",
    messages=[{"role": "user", "content": "Write a short poem about the ocean."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.content:
        print(delta.content, end="", flush=True)



Node.js

`javascript


import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.QWENAPI_KEY,
  baseURL: "https://dashscope-us.aliyuncs.com/compatible-mode/v1",
});

const stream = await client.chat.completions.create({
  model: "qwen3.5-plus",
  messages: [{ role: "user", content: "Write a short poem about the ocean." }],
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content ?? "";
  process.stdout.write(content);
}



Raw SSE Format

Each event looks like:


data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"content":"Hello"},"index":0}]}

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"delta":{"content":" world"},"index":0}]}

data: [DONE]



Streaming with Tool Calls

Tool call arguments stream incrementally. Accumulate the arguments string across chunks before parsing:

`python


tool_call_args = ""
tool_call_name = ""

stream = client.chat.completions.create(
    model="qwen3.5-plus",
    messages=[{"role": "user", "content": "What's the weather in Austin?"}],
    tools=tools,
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.tool_calls:
        tc = delta.tool_calls[0]
        if tc.function.name:
            tool_call_name = tc.function.name
        if tc.function.arguments:
            tool_call_args += tc.function.arguments

Parse after stream ends
import json
args = json.loads(tool_call_args)



Streaming with Thinking Mode

When enable_thinking=True, the model streams its reasoning before the final answer. Thinking content arrives in delta.reasoning_content (or a block in delta.content, depending on the model version):

`python


stream = client.chat.completions.create(
    model="qwen3-max",
    messages=[{"role": "user", "content": "What is 144 * 37?"}],
    stream=True,
    extra_body={"enable_thinking": True},
)

for chunk in stream:
    delta = chunk.choices[0].delta
    # Reasoning content (thinking)
    if hasattr(delta, "reasoning_content") and delta.reasoning_content:
        print(delta.reasoning_content, end="", flush=True)
    # Final answer
    elif delta.content:
        print(delta.content, end="", flush=True)



Usage Statistics

Token counts are included in the final chunk's usage field. To receive them, set stream_options:

`python


stream = client.chat.completions.create(
    model="qwen3.5-flash",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.usage:
        print(f"Tokens: {chunk.usage.prompt_tokens} in, {chunk.usage.completion_tokens} out")



Performance Tips

Use the US endpoint (dashscope-us.aliyuncs.com) from North America for lower latency.

Prefer qwen3.5-flash or qwen-turbo for streaming chat interfaces where speed matters more than depth.

Set max_tokens to avoid unexpectedly long responses that keep the stream open.


Handle disconnects — implement retry logic with exponential backoff if the stream drops mid-response.


Error Handling

Errors during streaming are returned as a final SSE event with an error field, or as an HTTP error before the stream starts. Always wrap stream iteration in a try/except:

`python


try:
    for chunk in stream:
        ...
except Exception as e:
    print(f"Stream error: {e}")