Ways to buy inference under the rate you pay now

Start with a ceiling, not a migration

Every request here carries max_price_per_mtok, a hard limit in USD per million output tokens. It is required rather than defaulted, and nothing fills above it: if no node can serve the request at or under your number, the call returns 409 no_fill instead of quietly filling higher.

That error is usually described as a failure. For a buyer who already has a provider it is the useful half of the design. Set the ceiling to the rate you are paying today and there are exactly two outcomes: the market beats your rate and you take the fill, or it does not and you fall through to the provider you were going to use anyway. There is no third outcome where you overpay, because the most you can be charged is a number you wrote in your own code.

So the cost of evaluating this is bounded by integration time rather than by spend, which is the opposite of the usual trial where you have to move real traffic to find out. Leave the fallback in permanently and you have a router that takes the cheaper of two prices per request, forever.

import os
from openai import OpenAI, APIStatusError

market = OpenAI(
    base_url="https://api.ai-trader.dev/v1",
    api_key=os.environ["AITRADER_KEY"],
)
incumbent = OpenAI()  # whatever you already pay, unchanged

# You pay $0.90/Mtok output today. Set the ceiling under that, so a
# fill still beats your rate once the buyer fee is added on top.
CEILING = 0.86


def complete(**kwargs):
    try:
        return market.chat.completions.create(
            **kwargs, extra_body={"max_price_per_mtok": CEILING}
        )
    except APIStatusError as err:
        if err.status_code != 409:  # 409 no_fill: nothing under the ceiling
            raise
        return incumbent.chat.completions.create(**kwargs)

Trim the ceiling a little under your contracted rate, as above. The limit governs the token price, and the venue's buyer fee sits on top of it, so a ceiling set exactly at your current rate can clear marginally above it once the fee is added. Pricing & fees has the number and the arithmetic.

You are not billed for input tokens

Only output is metered. Prompt tokens are not billed — prefill is compute-bound and cheap next to decode, and metering it separately doubles the surface for disputes without moving the total much.

For conversational traffic that is a modest difference. For data processing it is often the whole bill. The shape of extraction and classification work is a long input and a tiny output: four thousand tokens of contract, invoice or support thread go in, and a label, a score or a short JSON object comes out. Priced per input token, the document dominates the invoice and the answer is a rounding error. Priced only on output, the document is free and you are billed for the twelve tokens you actually wanted.

Which means the comparison worth running is not our headline rate against your headline rate. It is your monthly bill against the output tokens inside it. The more lopsided your input-to-output ratio, the wider the gap — and document processing is the most lopsided workload there is.

Patience is the discount

Supply here is idle consumer GPUs, so it arrives on a human schedule: cards free up overnight and at weekends, when their owners are not using them and the alternative is earning nothing. Work with no turnaround requirement can simply wait for that.

A resting bid is how you wait. Post below the current mid and it sits on the book, filling as cheap asks appear, and each fill is capacity you own and spend later — a completion drains your unspent fills before it ever crosses the spread. A bid can also carry the work with it: send a commands array and the venue drains the prompts as fills land, so you post the batch before you leave and collect from results_url in the morning.

POST /v1/orders
Authorization: Bearer sk_live_...
{
  "side": "bid",
  "model": "llama-3.1-8b-instruct",
  "limit_per_mtok": 0.18,
  "expires_in": 28800,
  "commands": [
    { "prompt": "Label the sentiment of this review: ...", "max_tokens": 8 },
    { "prompt": "Extract every line item as JSON: ...",    "max_tokens": 512 }
  ]
}

201 Created
{
  "id": "bd_71aa",
  "status": "open",
  "batch": { "commands": 2, "results_url": "/v1/orders/bd_71aa" }
}

expires_in is what stops patience becoming neglect. The order withdraws itself at the deadline and refunds whatever never filled, so an overnight batch that the market never came down to meet is a bid you got back, not one you are still holding at noon.

When the deadline is real but not immediate

Between "now" and "whenever" there is work that has to be done by a particular hour but does not care which hour before it. A growing_expense block covers that case: the bid starts at a price you would love, steps up on a timer, and stops climbing the moment it fills or reaches a cap you set. It rises only as far as it has to, so a bid that fills on the second step never reaches the fifth, and you get something close to a guaranteed fill without paying the ceiling for it. See Order types for the full set.

Point your own model at the docs

The endpoint is OpenAI-compatible, so the integration is a base URL and one extra field. That is small enough that writing it by hand is not the interesting part — hand the docs to whatever coding agent you already run and let it do the edit.

/llms.txt exists for exactly this. It is the entire API as one plain-text file — auth, every endpoint, the order types, the billing rules, the error codes, the parameters we refuse, and the current model table pulled from the live book. One fetch, no crawling, and enough to write a correct integration without reading anything else.

Fetch https://ai-trader.dev/llms.txt — it is the whole API in one file.

Our service calls OpenAI in src/inference.py. Add AI-Trader as a
price-capped first choice: the same call, with base_url
https://api.ai-trader.dev/v1 and max_price_per_mtok set to CEILING.
On 409 no_fill, fall through to the existing client untouched.
Change no prompt, no model choice, and no retry logic.

We mention this because the honest pitch for a marketplace is that it is cheap to check and cheap to abandon. If an afternoon of an agent's time tells you our prices do not beat yours, you have lost an afternoon and kept a price-capped fallback that costs nothing while it sits unused. If they do beat yours, the same code that answered the question is the code that goes to production.

Where the book is the wrong tool

Worth knowing before you spend the afternoon:

• Latency-critical interactive traffic. Depth varies with who is online. A 409 is fine inside a batch runner and bad in front of a user waiting on a cursor.
• Frontier closed models. The contracts here are open models at a named quantisation. If your pipeline needs a specific proprietary model, nothing on this book substitutes for it.
• Reproducible output across nodes. You can set seed and the rest of the sampling parameters, and they are passed to the backend untouched — but say nothing and the model keeps the default it ships with, which samples, and the venue does not route on a seed. The same request can land on different hardware holding a different quantisation, and two honest nodes diverge at the first near-tie. If you need bit-identical answers, pin the work yourself.
• Contracted capacity. There is no reserved throughput to buy. You are bidding against whatever is connected, which is why the fallback path is not optional.