AI Server documentation.
AI Server runs chat, embeddings, image, speech and vision models on hardware you control, behind one OpenAI-compatible API. Pick the path that matches your job.
Windows app 2.2.6 · Container images 2.5.2 · Helm chart 0.2.2
Decide
What AI Server is, what it costs, the business case and the risks.
- What AI Server is — an overview for decision-makersAI Server in one page — what it does, who it is for, how it fits with AI Suite and AI Client, the editions, and what it takes to run.
- AI Server editions, pricing and how licences are countedFree, Pro Personal and Pro Commercial compared; prices, team packs and the Home Lab bundle; how servers, gateways and seats are counted; and where to buy.
- The business case for a private AI serverA cost model for AI Server against per-token cloud AI and per-seat AI subscriptions — the inputs to gather, the formula, the non-financial factors and how to run a pilot that proves it.
- Risk and compliance summaryA one-page summary for executives and reviewers — where data goes, what leaves the network, the controls available, what AI Server does not claim, and the shared responsibilities.
- AI Server roadmapWhat AI Server does today, what we are working on next, what we are considering, and the changes already announced — without promised dates.
Design
Architecture, security, engines, scaling and reference deployments.
- AI Server architectureHow AI Server is built — the server process, its request pipeline, the engines it manages, storage, discovery and the gateway — for architects designing a deployment.
- Security architectureAI Server's security design for architects and reviewers — authentication, key storage, authorisation, transport, licensing integrity, audit, data inventory and every outbound connection.
- Reference architecturesSix proven AI Server deployment shapes — workstation, office server, cloud container, gateway farm, branch office and regulated enterprise — with when to use each and the walkthrough for it.
- Scaling, high availability and sizingSize AI Server hardware for a team, then scale out with AI Gateway — the worker pool, routing strategies, health checks, failover, canary rollouts, backpressure and highly available gateways.
- Engines and modelsThe inference engines AI Server installs and supervises — for chat, embeddings, images, speech and vision — how models are named and stored, and how cloud providers fit in.
- Compatibility, versions and support policySupported platforms, how AI Server is versioned, how long versions are supported, how breaking changes are announced, and the end of the legacy local-AI API.
Install and run
Install on Windows, Docker or Kubernetes; keys, TLS, licensing, monitoring and upgrades.
- Install AI Server on WindowsInstall AI Server from Microsoft Store, serve your own PC for free, then share it with your network as a Windows service with API keys.
- Run AI Server in DockerRun the softwaretailor/aiserver image with a licence key file and an API key, then grow to an AI Gateway farm with Docker Compose.
- Deploy AI Server on Kubernetes with HelmInstall an AI Gateway and a StatefulSet of AI Server workers with the Helm chart — required values, GPU nodes, autoscaling, monitoring and zero-downtime upgrades.
- Configuration reference — environment variables, flags and exit codesEvery AI Server setting for containers, services and scripts — environment variables, command-line flags, defaults and exit codes.
- API keys and scopesIssue, scope, rotate and revoke AI Server API keys — one per person, device or app — and decide which keys may read monitoring and audit data.
- HTTPS and TLS for AI ServerEncrypt traffic to AI Server with its built-in self-signed certificate, your own certificate, or a reverse proxy — and when to trust forwarded headers.
- Licence activation and offline operationHow AI Server licences activate on Windows, Docker and Kubernetes, what activation sends, how seats are counted, and how long a server keeps serving without a connection.
- Governance — rate limits, quotas, budgets, moderation and model lifecycleControl who can use which models and how much — rate limits, daily quotas, monthly budgets, request scheduling, content rules, moderation, model lifecycle and region policy.
- Monitoring — dashboard, Prometheus metrics, usage and logsWatch AI Server and AI Gateway with the built-in dashboard, native Prometheus metrics, per-key usage reports, a signed audit log and redacted logs.
- Upgrading AI ServerHow the Windows app, its service, the container images, the Helm chart and the local AI engines are updated — and how to upgrade a farm without dropping requests.
- Backup and restoreWhat to back up on an AI Server — keys, licence lease, audit log, signing key, settings and governance — how to restore it, and what the settings export does and does not include.
- Hardening checklistA checklist for running AI Server on a network — keys, admin rights, TLS, proxies, network policy, governance, secrets, updates and monitoring.
- Troubleshooting AI ServerFixes for the common AI Server problems — other computers cannot connect, the service does not start, licence errors, 401/403/429/503 responses, slow answers and empty gateway pools.
- Enrol AI Server in AI Admin ConsoleConnect AI Server to your organisation's AI Admin Console so it appears in the organisation's server list with its policy posture.
Build
The OpenAI-compatible API, SDKs, errors, limits and migrating existing code.
- AI Server API overviewBase URLs, authentication, model names, headers and the endpoint list of AI Server's OpenAI-compatible API — everything you need before the first request.
- Chat completions API — streaming, tools, JSON output and visionPOST /v1/chat/completions on AI Server — supported parameters, streaming, tool calling for agents, JSON output, image input, limits and differences from OpenAI.
- Embeddings API — vectors for search and RAGPOST /v1/embeddings and the background batch API on AI Server — models, dimensions, base64 encoding, batching and how to build retrieval-augmented generation.
- Images API — generation, editing and descriptionPOST /v1/images/generations on AI Server — text-to-image, image-to-image, inpainting, outpainting, upscaling and ControlNet — plus describing an image as a prompt.
- Audio API — speech, transcription, translation and live captionsAI Server's audio endpoints — text-to-speech, file transcription with SRT/VTT subtitles, translation to English, live transcription over WebSocket and voice cloning.
- Vision API — detection, faces and background removalAI Server's computer-vision endpoints — object detection, face detection with embeddings, and background removal — and when to use chat with images instead.
- Models API — list, describe and download modelsList installed models, browse the catalogue, download a model with progress events, and read a model's provenance on AI Server.
- API errors — status codes, error codes and retriesEvery error AI Server returns, in the OpenAI error format — status, code, meaning, whether to retry — and how to handle errors in a stream.
- Rate limits, quotas and the Free allowanceHow AI Server limits requests — the Free allowance for third-party tools, per-key and per-client rate limits, daily quotas, budgets, busy-server backpressure — and how clients should respond.
- SDKs and frameworks for AI ServerUse AI Server from Python, JavaScript, .NET, Go, Java and curl with the official OpenAI SDKs, and from LangChain, LlamaIndex, Semantic Kernel and agent frameworks — plus open-source samples.
- Migrate to AI Server from cloud APIs and other local serversMove existing code from OpenAI's cloud API, from another local model server, or from AI Server's legacy local-AI API to AI Server's /v1 API — what changes and what to check.
Look inside
How requests flow, how the gateway routes, how to measure and how to extend.
- Inside AI Server — request flow, engines and stateFor engineers — how a request moves through AI Server, how chat requests are mapped to engines, streaming, tool-call continuation, model resolution, process lifecycle and what is persisted.
- AI Gateway internals — routing, failover and backpressureFor engineers — how the AI Gateway chooses a worker, the five routing strategies, circuit breaking, retries, sticky and canary hashing, discovery, streaming and governance at the edge.
- Benchmarking AI Server on your hardwareHow to measure AI Server on your own hardware — time to first token, generation speed and throughput under concurrency — with a script, and how to read the results.
- Extending and integrating AI ServerThe supported ways to build on AI Server — the OpenAI-compatible API, agent frameworks, AI Suite apps, Prometheus and SIEM integration, custom models through GGUF — and what is not an extension point.
Reference
FAQ, glossary, release notes, support and procurement.
- AI Server FAQShort answers to the questions people ask most about AI Server — what it is, data and privacy, licensing, hardware, models, compatibility and the AI Gateway.
- AI Server glossaryPlain definitions of the terms used in the AI Server documentation — from API key and lease to worker, canary, embedding and quantization.
- AI Server release notesWhat changed in each AI Server release — the Windows app, the container images and the Helm chart.
- Getting support for AI ServerHow to get help with AI Server — reporting a problem from the app, email and the contact form, what to include, security reports, and help for partners.
- Procurement and supplier reviewWhat procurement, legal and security reviewers need for AI Server — the supplier, licence terms, data processing, security answers, invoices and how to buy through the Store or a partner.