Run AI Server in Docker

The container images run the same server as the Windows app, configured entirely by environment variables. A container listens on the network by definition, so it needs two things before it serves: a paid licence key and at least one API key.

Images

Tags are the release version (2.5.2) and latest. Pin the version in production. The images run as a non-root user (UID 1654) and keep all state in /data.

1. Issue an API key

The key is printed once; store it in your password manager. Only its hash is written to the volume.

The images include the aisuite-server-cli tool (from 2.5.2), so you can issue the key straight into the volume:

docker run --rm -v aisuite-data:/data --entrypoint aisuite-server-cli \
  softwaretailor/aiserver:<version> keys add --label my-app

With 2.5.1 or earlier images, issue the key on the Windows app's API keys page (or with aisuite-server-cli keys add --label my-app --data ./seed where the CLI is installed) and copy the resulting credentials/server-keys.json into the volume:

docker run --rm -v aisuite-data:/data -v "$PWD/seed:/seed:ro" alpine \
  sh -c 'mkdir -p /data/credentials && cp /seed/credentials/server-keys.json /data/credentials/ && chown -R 1654:1654 /data'

2. Start the server

Save the licence key in a file and mount it. Never pass it as an environment value: those show up in docker inspect and crash dumps.

docker run -d --name aiserver -p 8080:8080 \
  -v aisuite-data:/data \
  -v "$PWD/license.key:/run/secrets/aisuite-license:ro" \
  -e AISUITE_LICENSE_KEY_FILE=/run/secrets/aisuite-license \
  -e AISUITE_ENGINE=real \
  -e AISUITE_NODE_NAME=ai-1 \
  softwaretailor/aiserver:2.5.2

At first start the node activates its licence over outbound HTTPS and caches a signed lease in /data. See licence activation for what is sent and how long a node keeps serving offline.

3. Check it

curl -fsS http://localhost:8080/readyz
curl -H "Authorization: Bearer <api-key>" http://localhost:8080/v1/models
docker logs aiserver 2>&1 | grep -i license

The log says whether the licence activated. Then download a model with POST /v1/models/pull (see models); a request for a model that is not installed answers 404 model_not_found.

Settings you will usually change

The full list, with defaults and exit codes, is on the configuration reference.

Probes and shutdown

• GET /livez — 200 while the process runs, even while draining. Use it to restart the container.
• GET /readyz — 503 once draining. Use it to route traffic.
• On SIGTERM the server answers 503 on /readyz for AISUITE_DRAIN_SECONDS (8 in the images), then finishes in-flight requests within AISUITE_SHUTDOWN_SECONDS (30). Give Docker at least 40 seconds: stop_grace_period: 40s.

A gateway farm with Compose

A gateway in front of two workers gives you failover and zero-downtime upgrades on one host. Gateway mode needs AI Server Commercial; the gateway itself uses no licence seat, each worker uses one.

Each node needs its own API key store: the client key lives on the gateway, and each worker has the worker key that the gateway presents to it.

# Images after 2.5.2; with 2.5.2 use the copy step from section 1 for each volume.
for node in gateway worker-1 worker-2; do
  docker run --rm -v "aisuite-farm_${node}-data:/data" --entrypoint aisuite-server-cli \
    softwaretailor/aiserver:<version> keys add --label "$node"
done

gateway.json lists the workers and the key the gateway uses for each:

{
  "enabled": true,
  "routing_strategy": "least-connections",
  "workers": [
    { "id": "worker-1", "base_url": "http://worker-1:8080", "bearer_token": "<key issued on worker-1>" },
    { "id": "worker-2", "base_url": "http://worker-2:8080", "bearer_token": "<key issued on worker-2>" }
  ]
}

services:
  gateway:
    image: softwaretailor/aigateway:2.5.2
    ports: ["8080:8080"]
    environment:
      AISUITE_ENGINE: gateway
      AISUITE_LICENSE_KEY_FILE: /run/secrets/aisuite-license
    volumes:
      - gateway-data:/data
      - ./gateway.json:/data/gateway.json:ro
      - ./license.key:/run/secrets/aisuite-license:ro
    stop_grace_period: 40s
  worker-1:
    image: softwaretailor/aiserver:2.5.2
    environment:
      AISUITE_ENGINE: real
      AISUITE_LICENSE_KEY_FILE: /run/secrets/aisuite-license
      AISUITE_NODE_NAME: worker-1
      AISUITE_TRUST_PROXY: "1"
    volumes: [worker-1-data:/data, ./license.key:/run/secrets/aisuite-license:ro]
    stop_grace_period: 40s
  worker-2:
    image: softwaretailor/aiserver:2.5.2
    environment:
      AISUITE_ENGINE: real
      AISUITE_LICENSE_KEY_FILE: /run/secrets/aisuite-license
      AISUITE_NODE_NAME: worker-2
      AISUITE_TRUST_PROXY: "1"
    volumes: [worker-2-data:/data, ./license.key:/run/secrets/aisuite-license:ro]
    stop_grace_period: 40s
volumes: { gateway-data: {}, worker-1-data: {}, worker-2-data: {} }

Run it with the project name used for the volumes above: docker compose -p aisuite-farm up -d. Clients use the gateway's address and the gateway's client key. To check the pool, call GET /v1/gateway/workers with a key issued with --admin.

GPU workers need the NVIDIA Container Toolkit (--gpus all, or deploy.resources.reservations.devices in Compose).

Questions