Watching a private AI fleet: dashboards, Prometheus, and per-app chargeback
A running AI Server tells you what it is doing without a monitoring stack to assemble first. A live dashboard, Prometheus metrics, and usage rollups by app and worker are built in.
Every AI Server hosts a live dashboard at /dashboard for humans, exposes the same data as aggregate JSON, and serves native Prometheus metrics for alerting [1]. Usage rollups break down by key, app, model, or worker; the worker pool status is queryable for farm automation; and orchestrator probes cover liveness and readiness. It is all built in, so this is about using it well rather than installing it.
See problems before users do
The dashboard shows every worker's health, error rate, p95 latency, and saturation, so a worker going DOWN or a latency drift is visible before it reaches a user [2]. For alerting, scrape the Prometheus endpoint and start with three rules: an error-rate threshold, a p95 that runs at twice its baseline for ten minutes, and a farm whose healthy worker count is below its total for five minutes.
Chargeback without a metering sidecar
The content-free audit attributes every request to its app and install, and on a farm to the serving worker, which is per-team and per-app chargeback data natively [1]. A usage query over a date range, grouped by app, returns requests, tokens, and errors per application; grouped by worker it splits cost across farm hardware; grouped by key it reports per credential. Per-model pricing can be configured server-side so the rollups include cost estimates.
Numbers that survive due diligence
The dashboard is projector-ready and its legend explains itself to non-engineers, so a live demonstration under load replaces slideware: run a chat and watch requests per second tick, take a worker down and watch failover, then point at the by-app view as the chargeback picture [1]. Every figure is reproducible from these deployment recipes, which is what lets a claim survive a buyer's due diligence.
References
- Software Tailor. "AI Server — product page." softwaretailor.com/ai-server.htm. Accessed 2026-07-22.
- Software Tailor. "AI Server documentation — Operations." softwaretailor.com/docs/ai-server/operations.htm. Accessed 2026-07-22.
- Microsoft. "AI Server on the Microsoft Store." apps.microsoft.com/detail/9P42956WBWCL. Accessed 2026-07-22.
Related articles
Run the demo, not the slides.
A live dashboard, Prometheus metrics, and native chargeback. Get it on Windows from the Microsoft Store, or read the feature tour first.