AI Server docs / API & clients

API & clients

AI Suite apps need zero configuration — they discover the server themselves. This page is for everything else: your own code, OpenAI-compatible tools, and scripts.

Authentication

Authorization: Bearer <api-key>

API keys are created on the server's API keys page (or its CLI) and sent as a standard bearer token. Loopback calls on the server's own machine need no key. A gateway accepts the same keys as a single server — clients never know the difference.

Endpoints (OpenAI-compatible)

Endpoint	What it does
`POST /v1/chat/completions`	Chat, including streaming (SSE), tool calling, and vision (`image_url` content parts).
`POST /v1/embeddings`	Text embeddings.
`POST /v1/images/generations`	Image generation — text-to-image plus image-to-image / inpainting via base64 init and mask images in the body.
`POST /v1/audio/speech`	Text-to-speech, streamed audio.
`POST /v1/audio/transcriptions`	Speech-to-text (multipart file upload).
`/v1/audio/transcriptions/stream`	Live speech-to-text over WebSocket (binary PCM frames up, transcript deltas down).
`GET /v1/models`	Models available to serve; a gateway answers with the union across its worker farm.

Additional native endpoints (voice cloning, vision analysis, model install/pull, usage, metrics) exist alongside these; the in-app Help lists the full surface, and a machine-readable spec is planned.

Point standard tooling at it

# Python (openai SDK)
from openai import OpenAI
client = OpenAI(base_url="http://your-server:8080/v1", api_key="<api-key>")
r = client.chat.completions.create(model="<model-id>",
        messages=[{"role": "user", "content": "Hello"}], stream=True)
for chunk in r:
    print(chunk.choices[0].delta.content or "", end="")

# curl
curl -N http://your-server:8080/v1/chat/completions \
  -H "Authorization: Bearer <api-key>" -H "Content-Type: application/json" \
  -d '{"model":"<model-id>","messages":[{"role":"user","content":"Hello"}],"stream":true}'

Tools with an "OpenAI-compatible / custom base URL" setting (chat UIs, coding assistants, editor plugins) work the same way: base URL http://your-server:8080/v1, your API key, pick a model from /v1/models.

Error & streaming semantics worth coding for

429 — you hit a rate limit, quota, or the free-tier meter (below). When upgrading would lift the limit, the response carries X-AISuite-Upgrade: 1.
503 + Retry-After — honest backpressure: every worker is saturated. Wait the indicated seconds and retry; do not hammer.
Streaming is standard SSE (stream: true). Keep proxy buffering off anywhere you put your own proxy in front.
Behind a gateway, transient worker failures are retried server-side before you see them; responses carry X-AISuite-Backend naming the machine that answered — useful in bug reports.

Free-tier metering for generic clients

On the Free tier, AI Suite apps are unlimited, but generic clients (curl, SDKs, third-party tools) are metered to a small allowance — currently 1 request/minute and 10/day — after which the server answers 429 with X-AISuite-Upgrade: 1. Any paid tier removes the meter entirely. The line exists so the free tier can power the app suite without giving away an unlimited general-purpose inference server.

Legacy compatibility endpoint

Servers upgraded from AI Server 2.0.1 keep serving that generation's local-engine wire format so old setups don't break. This compatibility surface is sunset on 2026-12-31 — move generic integrations to the /v1 endpoints above before then.

Next: Gateway & operations · back to docs home