Batch requests
A batch is a list of chat requests submitted in one call and collected later. Use it for work that does not need an answer immediately: classifying a file line by line, rewriting a set of descriptions, running the same prompt against many inputs.
Submit a batch
POST /v1/batches takes a model and an array of chat requests. Up to 1,000 items at a time.
curl -s https://api.autarcho.com/v1/batches \
-H "Authorization: Bearer ak_..." \
-H "content-type: application/json" \
-d '{
"model": "<model id>",
"items": [
{ "messages": [{ "role": "user", "content": "Name three uses for a shed." }] },
{ "messages": [{ "role": "user", "content": "Rewrite this in one sentence: ..." }] }
]
}' The answer carries a batch_id, a status URL, a results URL, and the window your
results are retained for.
Check a batch
GET /v1/batches/{id} reports how many items are queued, running, finished or
failed, and how much credit the submission holds. GET /v1/batches lists your
submissions newest first, which is how you find a batch again after losing the response.
curl -s https://api.autarcho.com/v1/batches/<batch_id> \ -H "Authorization: Bearer ak_..."
Fetch results
GET /v1/batches/{id}/results returns JSON Lines, one line per finished
request, with the question it answered and the answer.
{"seq":1,"state":"done","prompt":"Name three uses for a shed.","content":"Storage, workshop, office.","finish_reason":"stop"}
{"seq":2,"state":"done","prompt":"Rewrite this in one sentence: ...","content":"The shed is used for storage.","finish_reason":"stop"} Items finish out of order. Read the whole set and match lines by seq rather than
assuming the highest number you have seen means the batch is complete. A line with no text says
why: the item failed, or its result had already expired.
Cancel a batch
POST /v1/batches/{id}/cancel refunds every item that has not started. Items
already finished, or running at that moment, are not affected.
What a batch costs
The price is the same per generated token as a chat request. The amount held when you submit is an upper bound, calculated from the item count, the token budget and the price; the charge is the sum of what each item actually generated. An item that fails without producing any text is refunded.
Billing covers the per-token rate, and the API reference covers the request fields inside an item.