Zero-downtime upgrades: patching a private AI endpoint without a maintenance window
Upgrading an AI endpoint should not need a maintenance window. AI Server drains a node before it stops accepting work, and the gateway routes around it — so a version rollout drops no requests.
Three mechanisms that ship in every daemon make a rolling upgrade safe by construction. The /readyz probe flips to 503 the moment shutdown begins, so a load balancer or gateway sees the node leave rotation, while /livez stays 200 so the process is not killed as dead [1]. A configurable drain hold keeps the node in that draining state long enough for schedulers to stop sending new work before the process stops accepting it. The gateway probes /readyz, not just liveness, and its per-request failover covers the handful of requests that race the drain window [1].
Draining, not dropping
When a worker receives a stop signal, it enters draining: /readyz returns 503, in-flight requests finish, and only then does the process exit [2]. The drain duration is set with AISUITE_DRAIN_SECONDS, and a separate shutdown budget bounds how long the process waits for in-flight work to complete. The pool entry never changes; the same address comes back healthy on the new version, and the gateway resumes routing to it.
The gateway does the rest
In a farm, you stop one worker at a time. The gateway's dashboard shows that worker turn DRAINING, then DOWN, while traffic continues on its peers; you replace the image and the node returns HEALTHY on the new build before you move to the next one [1]. There is no gateway configuration change during any of this. Health probing, not a config edit, is what moves traffic.
Kubernetes: zero-downtime by construction
On Kubernetes the same probes make kubectl rollout safe without extra tooling: a liveness probe on /livez, a readiness probe on /readyz, and a terminationGracePeriodSeconds set at or above the drain plus shutdown budget [2]. The orchestrator stops routing to a draining pod, waits out the grace period, and rolls the next one.
Why it matters
"Update Tuesday" should not exist for an AI endpoint that other apps depend on. This is also the mechanism that proves the availability position of a gateway farm end to end: patching is not downtime, and it holds under load rather than only on a quiet console [1][3].
References
- Software Tailor. "AI Server — product page." softwaretailor.com/ai-server.htm. Accessed 2026-07-22.
- Software Tailor. "AI Server documentation — Operations." softwaretailor.com/docs/ai-server/operations.htm. Accessed 2026-07-22.
- Microsoft. "AI Server on the Microsoft Store." apps.microsoft.com/detail/9P42956WBWCL. Accessed 2026-07-22.
Related articles
Patch without the downtime.
Start free on one machine and grow into a drain-aware farm. Get it on Windows from the Microsoft Store, or read the feature tour first.