API reference

The one extra parameter

max_price_per_mtok is a hard ceiling in USD per million output tokens. If no node can serve the request at or under it, the call returns 409 no_fill rather than filling higher. A marketplace that silently charges past your limit is not a marketplace, so the failure is loud on purpose.

It is required, not defaulted. A request with no ceiling is a request with no bill you agreed to in advance, so omitting it returns 400 missing_price_ceiling instead of quietly taking whatever the book is asking.

Sizes are in tokens

Prices are quoted per million tokens because that is the unit the industry already reads, but every size on the wire is a whole number of tokens. Fractional tokens do not exist, and a venue that accepts size_mtok: 0.0000004 has to decide what to do with the remainder. Ours never has to.

A resting bid is capacity you already own

When a bid fills, the tokens are bought: the money is escrowed and a node is committed to serving them. What is missing is a prompt. Every /v1/chat/completions call looks for one of your unspent fills for that model before it goes near the book, cheapest first, and only crosses the spread when it finds nothing. That is the point of resting a bid overnight — you are buying at your price and spending it later.

A fill bought above the max_price_per_mtok on the current request is skipped rather than spent. You still own it; this request simply said it would not pay that much. Unspent fills are released back to your balance after thirty minutes, so a bid is capacity with a shelf life, not a permanent claim.

A resting bid is only the plainest of several shapes. It can expire on a deadline, carry a batch of prompts, or climb its own price until it fills. See Order types for the full set and the growing_expense block.

Retrying a bid safely

POST /v1/orders accepts an Idempotency-Key header. Replaying a request with the same key returns the original response instead of posting a second order, which is what you want when a timeout leaves you unsure whether the first one landed. The key is scoped to your account and bound to the exact body you sent: reusing it with different contents returns 422 idempotency_key_reuse rather than quietly answering for the wrong order. A key burned on a request that failed validation is released, so you can fix the body and send it again under the same key.

Streaming and settlement

stream: true is required, and responses come back as normal SSE. A buffered completion would send nothing until the answer was finished, leaving your client waiting the whole job for response headers — and most HTTP clients, Node’s fetch among them, give up at 300s. Jobs here routinely run longer, so a request without stream: true is refused with 400 unsupported_parameter.

Billing is on tokens the venue counted, not tokens the node reported, so a stream you abort part way through settles for the part you received. The response carries an x-aitrader-job header you can pass to /v1/jobs/{id} to see the metered count, the node that served it, and whether it was selected for audit.

Why a completion stops, and what you pay for a short one

Read finish_reason on the last choice to know which of these happened. In every case you are billed only for the output tokens the venue metered, never for the ones you asked for and did not get.

• stop — the model finished on its own. This is the ordinary case.
• length — the stream hit a token ceiling and was cut off. That ceiling is the smaller of your max_tokens and the size of the fill serving the request (see below). A length finish is your signal to send a follow-up request if you want the rest.
• null — the serving node never said why it stopped, which happens on an older node client or a connection cut in transit. We report it rather than guessing: an answer that ended because it was severed looks exactly like one that ended because it was finished, and calling the first one stop would hand you a fragment labelled complete. Treat it as unknown and check the text before relying on it.

The reason comes from the backend that generated the text, not from comparing the token count to the budget. Those are not the same thing: a model that happens to finish on the last token it was allowed is indistinguishable, by count alone, from one cut off there. The same value appears as a plain boolean on truncated, which is null when the reason is unknown.

max_tokens defaults to 4,096 and is capped at 32,768; a larger value is clamped to the cap rather than rejected. A request for more tokens than any one ask holds is served across several fills in sequence where the backend can extend a partial answer, and by the largest single fill where it cannot — in that case you are charged for what was produced, the unspent escrow is returned, and the request finishes length rather than erroring, so a supply-limited fill still gives you tokens rather than nothing.

Parameters

messages keeps its roles all the way to the model: system, user and assistant turns are passed through so the model's own chat template applies to them. response_format with {"type":"json_object"} constrains decoding to valid JSON, and stop takes up to four sequences. Both are handed to the backend rather than applied afterwards.

Sampling is yours to set. temperature (0–2), top_p (0–1), top_k, seed, frequency_penalty and presence_penalty (−2 to 2) are passed to the backend exactly as you send them. Anything you leave out keeps the default the model ships with, which samples — so two identical requests need not come back identical unless you ask for temperature 0. A value outside those ranges comes back as 400 invalid_sampling before any hold is placed on your balance, rather than failing the job after you have paid for it.

Batch commands take the same fields, and each command reads its own. One bid can carry a hundred unrelated prompts, and the settings that suit an extraction are not the ones that suit prose, so there is no single setting for the order. The 400 names the position of the command that was wrong rather than leaving you to find it.

One caveat on seed: it makes an answer repeatable on the same node running the same weights, and the venue does not route on it. The same seed sent twice can land on different hardware and produce a different answer, so treat it as a way to pin one node down, not as a guarantee from us.

If the serving node vanishes mid-stream you receive an in-band node_dropped error after the tokens it did send. Those tokens are billed, the rest of the hold is refunded, and the retry costs you nothing you were not already going to pay. A venue restart that strands a stream is swept and refunded the same way, within a three-hour lease.

Reading back what you spent

Nothing is thrown away, but the default view makes it look as though it is. GET /v1/orders answers with your resting orders only, which is what a client polling for fills wants — and an immediate-or-cancel bid is filled or cancelled the moment it resolves, so it leaves that view straight away. Pass ?status=filled, cancelled, expired or all to read the history back. The default was left alone so that adding it could not change what an existing integration sees.

GET /v1/jobs is the same idea one level down. The x-aitrader-job header only helps for jobs whose ids you kept; this enumerates the ones you did not, with our metered token count, the node that served each, the fee charged on top, and the verification state. That is the difference between being able to audit the bill and having to take it on trust.

Both take page and page_size (50 by default, clamped to 200) and answer with page, page_size, page_count, total and data. Both are scoped to the key that authenticated: there is no parameter that points either at another account. An unrecognised status is 400 invalid_status rather than a silently empty page.

Errors worth handling

• 409 no_fill — nothing available under your ceiling. Raise the limit or post a resting bid and wait.
• 402 escrow_insufficient — the bid needs more funded balance than you hold.
• 400 invalid_sampling — a sampling value fell outside its range.
• 400 unsupported_parameter — stream: true was missing.
• A node vanishing mid-answer is not in this list because it cannot be a status code: you already had a 200 when the stream opened. It arrives as a node_dropped error frame instead, described above.