SDKs and frameworks for AI Server

There is no AI Server SDK to install: the OpenAI-compatible API is the interface. Use the official OpenAI library for your language and set two things — the base URL and the API key. Nothing extra to keep in step with server versions.

Python

from openai import OpenAI
client = OpenAI(base_url="http://ai.example.internal:11436/v1", api_key="<api-key>")
print(client.chat.completions.create(model="llama3.2:3b",
      messages=[{"role": "user", "content": "Hello"}]).choices[0].message.content)

JavaScript and TypeScript

import OpenAI from "openai";
const client = new OpenAI({ baseURL: "http://ai.example.internal:11436/v1", apiKey: process.env.AISERVER_KEY });
const reply = await client.chat.completions.create({ model: "llama3.2:3b", messages: [{ role: "user", content: "Hello" }] });
console.log(reply.choices[0].message.content);

Run this on your server side, not in a web page: see calling from a browser.

.NET

using System.ClientModel;
using OpenAI;
using OpenAI.Chat;

var client = new ChatClient("llama3.2:3b", new ApiKeyCredential(apiKey),
    new OpenAIClientOptions { Endpoint = new Uri("http://ai.example.internal:11436/v1") });
ChatCompletion reply = await client.CompleteChatAsync("Hello");
Console.WriteLine(reply.Content[0].Text);

Microsoft.Extensions.AI, Semantic Kernel and Microsoft Agent Framework accept the same OpenAI client.

Go

client := openai.NewClient(option.WithBaseURL("http://ai.example.internal:11436/v1/"), option.WithAPIKey(key))
reply, err := client.Chat.Completions.New(ctx, openai.ChatCompletionNewParams{
    Model:    "llama3.2:3b",
    Messages: []openai.ChatCompletionMessageParamUnion{openai.UserMessage("Hello")},
})

Java

OpenAIClient client = OpenAIOkHttpClient.builder()
    .baseUrl("http://ai.example.internal:11436/v1").apiKey(key).build();
ChatCompletion reply = client.chat().completions().create(ChatCompletionCreateParams.builder()
    .model("llama3.2:3b").addUserMessage("Hello").build());

curl and PowerShell

curl http://ai.example.internal:11436/v1/chat/completions -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d '{"model":"llama3.2:3b","messages":[{"role":"user","content":"Hello"}]}'

$body = @{ model = "llama3.2:3b"; messages = @(@{ role = "user"; content = "Hello" }) } | ConvertTo-Json -Depth 4
Invoke-RestMethod -Method Post -Uri "http://ai.example.internal:11436/v1/chat/completions" `
  -Headers @{ Authorization = "Bearer $env:AISERVER_KEY" } -ContentType "application/json" -Body $body

Frameworks

Open-source samples

Working code you can run against your own server, MIT licensed:

• ai-server-quickstarts — every endpoint in curl, Python, Node.js, C# and PowerShell, with 401, 429 and 503 handling.
• ai-server-chat-web — a streaming chat page with a small server-side proxy that keeps the key out of the browser.
• ai-server-doc-qa — questions answered from your own documents, with citations.
• ai-server-dropin-recipes — configuration for tools you already use.
• ai-server-agent-frameworks — the same tool-using agent in several agent frameworks.
• ai-server-security-agents — a security-operations triage agent with tool seams.
• ai-server-ops-toolkit — diagnoses why a server cannot be reached.
• ai-server-compat — a protocol probe and a catalogue of tools tested against AI Server.

Questions