Batch API
Process large volumes of requests asynchronously at 50% off standard pricing.
Batch API
The Batch API lets you submit large numbers of requests as a single job and retrieve results when processing is complete. Batch jobs are processed within 24 hours and cost 50% less than synchronous requests.
When to Use Batch
- Evaluating a dataset against a model
- Generating embeddings for a large document corpus
- Running nightly classification or extraction pipelines
- Any workload where you don't need results in real time
Pricing
How It Works
1. Prepare a JSONL file — one request per line
2. Upload the file via the Files API
3. Create a batch job referencing the file
4. Poll until the job completes
5. Download the output file and parse results
Step 1: Prepare the JSONL File
Each line is a JSON object with a custom_id, method, url, and body:
``jsonl
{"custom_id": "req-001", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "qwen3.5-flash", "messages": [{"role": "user", "content": "Classify the sentiment: 'I love this product!'"}], "max_tokens": 10}}
{"custom_id": "req-002", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "qwen3.5-flash", "messages": [{"role": "user", "content": "Classify the sentiment: 'Terrible experience.'"}], "max_tokens": 10}}
{"custom_id": "req-003", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "qwen3.5-flash", "messages": [{"role": "user", "content": "Classify the sentiment: 'It was okay, nothing special.'"}], "max_tokens": 10}}
`
custom_id is your identifier — it's echoed back in the output so you can match results to inputs.
Step 2: Upload the File
`python
from openai import OpenAI
client = OpenAI(
api_key="YOUR_QWENAPI_KEY",
base_url="https://dashscope-us.aliyuncs.com/compatible-mode/v1",
)
with open("requests.jsonl", "rb") as f:
file = client.files.create(file=f, purpose="batch")
print(file.id) # file-abc123
`
Step 3: Create the Batch Job
`python
batch = client.batches.create(
input_file_id=file.id,
endpoint="/v1/chat/completions",
completion_window="24h",
)
print(batch.id) # batch-xyz789
print(batch.status) # validating
`
Step 4: Poll for Completion
`python
import time
while True:
batch = client.batches.retrieve(batch.id)
print(f"Status: {batch.status} — {batch.request_counts.completed}/{batch.request_counts.total}")
if batch.status in ("completed", "failed", "cancelled", "expired"):
break
time.sleep(30)
`
Batch statuses: validating → in_progress → completed (or failed / cancelled / expired).
Step 5: Download Results
`python
if batch.status == "completed":
output = client.files.content(batch.output_file_id)
for line in output.text.strip().split("\n"):
result = json.loads(line)
custom_id = result["custom_id"]
content = result["response"]["body"]["choices"][0]["message"]["content"]
print(f"{custom_id}: {content}")
`
Output Format
Each line in the output JSONL corresponds to one input request:
`json
{
"id": "batch_req_abc",
"custom_id": "req-001",
"response": {
"status_code": 200,
"body": {
"choices": [{"message": {"content": "Positive"}, "finish_reason": "stop"}],
"usage": {"prompt_tokens": 24, "completion_tokens": 1}
}
},
"error": null
}
`
Failed requests have a non-null error field and a non-200 status_code. The rest of the batch still completes.
Complete Example
`python
import json
import time
from openai import OpenAI
client = OpenAI(
api_key="YOUR_QWENAPI_KEY",
base_url="https://dashscope-us.aliyuncs.com/compatible-mode/v1",
)
Upload
with open("requests.jsonl", "rb") as f:
file = client.files.create(file=f, purpose="batch")
Create
batch = client.batches.create(
input_file_id=file.id,
endpoint="/v1/chat/completions",
completion_window="24h",
)
Poll
while batch.status not in ("completed", "failed", "cancelled", "expired"):
time.sleep(30)
batch = client.batches.retrieve(batch.id)
Download
if batch.status == "completed":
output = client.files.content(batch.output_file_id)
results = [json.loads(line) for line in output.text.strip().split("\n")]
print(f"Completed {len(results)} requests")
`
Limits
Batches that don't complete within 24 hours are marked expired. Partial results are not available for expired batches — resubmit the failed requests.
Cancellation
`python
client.batches.cancel(batch.id)
``
Cancellation is best-effort. Requests already in flight will complete; queued requests are cancelled.