Batch requests

A batch is a list of chat requests submitted in one call and collected later. Use it for work that does not need an answer immediately: classifying a file line by line, rewriting a set of descriptions, running the same prompt against many inputs.

Submit a batch

POST /v1/batches takes a model and an array of chat requests. Up to 1,000 items at a time.

curl -s https://api.autarcho.com/v1/batches \
  -H "Authorization: Bearer ak_..." \
  -H "content-type: application/json" \
  -d '{
    "model": "<model id>",
    "items": [
      { "messages": [{ "role": "user", "content": "Name three uses for a shed." }] },
      { "messages": [{ "role": "user", "content": "Rewrite this in one sentence: ..." }] }
    ]
  }'

The answer carries a batch_id, a status URL, a results URL, and the window your results are retained for.

Check a batch

GET /v1/batches/{id} reports how many items are queued, running, finished or failed, and how much credit the submission holds. GET /v1/batches lists your submissions newest first, which is how you find a batch again after losing the response.

curl -s https://api.autarcho.com/v1/batches/<batch_id> \
  -H "Authorization: Bearer ak_..."

Fetch results

GET /v1/batches/{id}/results returns JSON Lines, one line per finished request, with the question it answered and the answer.

{"seq":1,"state":"done","prompt":"Name three uses for a shed.","content":"Storage, workshop, office.","finish_reason":"stop"}
{"seq":2,"state":"done","prompt":"Rewrite this in one sentence: ...","content":"The shed is used for storage.","finish_reason":"stop"}

Items finish out of order. Read the whole set and match lines by seq rather than assuming the highest number you have seen means the batch is complete. A line with no text says why: the item failed, or its result had already expired.

Cancel a batch

POST /v1/batches/{id}/cancel refunds every item that has not started. Items already finished, or running at that moment, are not affected.

What a batch costs

The price is the same per generated token as a chat request. The amount held when you submit is an upper bound, calculated from the item count, the token budget and the price; the charge is the sum of what each item actually generated. An item that fails without producing any text is refunded.

Billing covers the per-token rate, and the API reference covers the request fields inside an item.