Hosts

Node API reference

The protocol the node client speaks. You never call these by hand — ai-trader does — but this is the contract behind register, bench, serve, and the auto-undercut ask.

Endpoints

POST	/v1/nodes	Register a machine. Authenticated with your account key; returns a node key.
POST	/v1/node/bench	Record measured tok/s per model. Only benchmarked models may be quoted.
POST	/v1/node/asks	Rest an ask. A new ask for a model replaces your resting one.
WS	/v1/node/ws	Hold the link. Jobs arrive down it; generated text goes back up it.

1 · Register the machine

Provider key in, node key out

http

POST /v1/nodes
Authorization: Bearer pk_live_...        # your account key (sk_live_ works too)
{
  "handle": "quartzite",
  "gpu": "NVIDIA GeForce RTX 4090",
  "accelerator": "cuda",                 # cuda | metal | rocm | cpu
  "vram_gb": 24,
  "mem_bandwidth_gbs": 1008,
  "region": "us-west"
}

201 Created
{ "id": "nd_7f31", "node_key": "nk_live_..." }

2 · Report the benchmark

Measured, not declared

http

POST /v1/node/bench
Authorization: Bearer nk_live_...        # the node key from register
{
  "models": [
    { "model": "llama-3.1-8b-instruct", "tok_per_sec": 142.9 },
    { "model": "qwen2.5-coder-32b",     "tok_per_sec": 38.2 }
  ]
}

200 OK
{ "node_id": "nd_7f31", "registered": ["llama-3.1-8b-instruct", "qwen2.5-coder-32b"] }

3 · Rest an ask

A new price replaces the old

http

POST /v1/node/asks
Authorization: Bearer nk_live_...
{
  "model": "llama-3.1-8b-instruct",
  "floor_per_mtok": 0.18,                # your price, USD per Mtok
  "size_tokens": 8000000,
  "expires_in": 86400
}

201 Created
{
  "id": "sk_4a90",
  "status": "open",
  "filled_tokens": 0,
  "remaining_tokens": 8000000,
  "replaced_asks": 1                     # your prior ask for this model was retired
}

4 · Hold the link and serve

http

GET /v1/node/ws                          (WebSocket)

→ {"t":"auth","key":"nk_live_..."}
← {"t":"ready","node_id":"nd_7f31"}

← {"t":"job","job":{"jobId":"jb_4c1f","modelId":"llama-3.1-8b-instruct",
     "prompt":"Explain a limit order book.","maxTokens":512}}

The node dials out and keeps one socket open for the whole session. It authenticates with a first frame rather than a header, then waits. When one of your asks is the cheapest eligible one a job frame arrives; you forward its prompt to your local backend and send the tokens back up the same socket as they decode.

http

→ {"t":"chunk","job":"jb_4c1f","text":"A limit order book "}
→ {"t":"chunk","job":"jb_4c1f","text":"is a list of resting orders…"}
→ {"t":"done","job":"jb_4c1f","tokens":128,"reason":"stop"}

# generation failed instead
→ {"t":"error","job":"jb_4c1f","message":"model is not pulled"}

# stop decoding and send no trailer.
# reason "budget": the text you sent is delivered and paid for.
# reason "gone":   the buyer left; this job earns nothing.
← {"t":"stop","job":"jb_4c1f","reason":"budget"}

The done trailer is not optional bookkeeping: it is the only way the venue learns whether the model finished or was cut off at maxTokens. A job that ends without one settles marked un-attested.

5 · Prove the link is still there

Because a dead socket looks open

http

→ {"t":"ping"}
← {"t":"pong"}

A link can die without either end seeing a close — a yanked cable, a sleeping laptop, a NAT table that forgot the flow. The venue reaps its side with WebSocket ping frames, but a client cannot always observe those, so ask in-band: send ping on an interval and expect pong straight back. Treat a few missed replies as a dropped link, reconnect, and re-rest your asks. ai-trader does this every 30s and gives up after 90s of silence. A node that skips it stays online in its own eyes while earning nothing.

Two keys, two scopes

You authenticate to register with your account key — the same key you buy with. Accounts created on the Host side carry a pk_live_ prefix and ones created on the Prompt side an sk_live_ prefix; the prefix is a label, not a scope, and either works here. Everything after that uses the node key it hands back: the account and node behind an ask are fixed by the credential, so a node can only ever post supply for itself.

A new ask replaces the old one

Posting an ask for a model you already quote does not stack a second order beside the first — it retires the resting one and rests the new price in its place. The response says how many it replaced in replaced_asks. This is why lowering your floor is simply a matter of posting again, or restarting serve after editing config.toml: the config is the source of truth, and each reconnect re-rests it over whatever was there.

Asks escrow nothing, so replacing one moves no money. It is a status change and nothing more; any fills already struck against the old ask keep their jobs and settle on their own.

Auto-undercut: let your ask chase the fill

An ask can lower its own price over time until it wins flow. Add an auto_undercut block and the venue steps your floor down toward a hard minimum on a timer, re-checking the book at each step:

http

POST /v1/node/asks
{
  "model": "llama-3.1-8b-instruct",
  "floor_per_mtok": 0.22,          # start here
  "size_tokens": 8000000,
  "auto_undercut": {
    "step_per_mtok": 0.005,        # shave this each interval
    "every_seconds": 300,          # every five minutes
    "floor_per_mtok": 0.12         # never go below this
  }
}

It descends only as far as it must: the first interval at which your ask becomes the cheapest eligible one wins a fill and the descent stops. Reach the floor without filling and it simply rests there. Use it to hold the top of the queue on a busy model without babysitting a price, and set the floor to the number below which the electricity is not worth it.

Staying competitive

Jobs route to the lowest ask at or below the buyer's limit, price first and arrival time only as a tie-break. Being online longer earns you nothing if someone rests a cheaper ask — uptime is deliberately not a routing input, because paying for it starves new supply and rewards never restarting. To win more flow, lower your floor.

Small models are where a modest box competes best: the weights fit with room for KV cache, the bandwidth ceiling is high, and the edge is volume rather than margin per token. A single consumer card serving an 8B earns real but modest money, and it is worth doing because the hardware and the power are largely sunk. See the install guide for the bench that decides what you may list, and pricing for the fee schedule.

Errors worth handling

409 not_benchmarked — you quoted a model this node has no benchmark on record for. Run bench first.
4401 on the socket — the key was refused, or an operator suspended the node. Reconnecting will not help until the key is replaced.
4400 on the socket — a frame arrived before auth.
Frames naming a job that did not route to you, or whose buyer has already gone, are dropped silently. Nothing is billed and nothing is delivered, so treat a job with no stop and no acknowledgement as abandoned rather than retrying it.

Preview

Every endpoint here answers on the development venue, and the node client drives them end to end. The hosted installer is the packaging still to come; see Install the node.