POST /v1/chat/completions

Generates a model response to a chat conversation. Schema and streaming match OpenAI; in addition, every response names the model that actually answered in a header.

curl https://sovrgpt.com/api/v1/chat/completions \
  -H "Authorization: Bearer $SOVR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-12b",
    "messages": [
      { "role": "system", "content": "You are a helpful assistant." },
      { "role": "user",   "content": "Explain diffusion models in two sentences." }
    ],
    "temperature": 0.5,
    "max_tokens": 400
  }'

Request body

Other OpenAI fields (seed, logit_bias, user, presence_penalty, frequency_penalty) are passed through to the model server unchanged; whether they take effect depends on the model. n above 1 is not supported.

Reasoning effort (reasoning_effort)

Many models can "think" to different depths per request. You control this the OpenAI way through reasoning_effort (in the Python SDK via extra_body):

curl https://sovrgpt.com/api/v1/chat/completions \
  -H "Authorization: Bearer $SOVR_KEY" -H "Content-Type: application/json" \
  -d '{
    "model": "stackit-qwen3.8-27b",
    "messages": [{ "role": "user", "content": "Find and fix the bug in this stack trace …" }],
    "reasoning_effort": "high"
  }'

GET /v1/models lists, per model, the levels in effort_levels whose effect we have measured. An empty array means there is no effective control; a value sent anyway is not translated. Models that accept a value but ignore it list no levels.

If a model does not offer the level you asked for (some have only low and high), we use the next lower available one, never a higher one.

What reaches the model differs by provider: a token budget instead of a level, other level names, or a reversed scale. The server handles this translation. You always send low, medium or high.

Response (non-streaming)

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1715520000,
  "model": "gemma-4-12b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Diffusion models …"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 42,
    "completion_tokens": 87,
    "total_tokens": 129
  }
}

Which model answered

Every response carries these headers, streaming or not, including the 503 below:

Unknown model IDs: 404

An unknown model ID (a typo, gpt-4o, an embedding ID such as bge-m3) fails with HTTP 404 and the code model_not_found. Switch to an ID from GET /v1/models.

curl -s -X POST https://sovrgpt.com/api/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Hello"}]}'

# {"error":{"message":"The model 'gpt-4o' does not exist. Call GET /v1/models for the list you may use.","type":"invalid_request_error","param":"model","code":"model_not_found"}}

Until 14 October 2026 SovrGPT replaced an unknown ID with the default model instead.

Not affected: aliases and role names (below), a request without any model (it gets the default model as before) and unavailable (a known ID that cannot serve right now).

An alias is not a substitution. coding resolves to qwen3.6-35b-a3b; then only x-sovrgpt-model-served is present. Role names such as coder point to the successor after a model change, again without a substitution header. If the role matters more to you than the specific model, send the role name.

A withdrawn model: 404 model_withdrawn

An ID that existed here and that we have withdrawn is not substituted. It fails with its own code and names the successor:

{
  "error": {
    "message": "The model 'qwen3-coder-next-fp8' has been withdrawn and is no longer served. Zurückgezogen am 2026-09-11 (32 768 Token Kontext, von 'qwen3.6-35b-a3b' überholt). Nachfolger: 'qwen3.6-35b-a3b' … Call GET /v1/models for the list you may use.",
    "type": "invalid_request_error",
    "param": "model",
    "code": "model_withdrawn"
  }
}

HTTP 404, plus the header x-sovrgpt-model-withdrawn with the affected ID. Someone who named a specific model is better served by a clear error than by a general substitute.

The two 404 codes call for different responses:

Streaming (SSE)

With "stream": true:

data: {"id":"chatcmpl-abc","choices":[{"delta":{"role":"assistant"},"index":0}]}
data: {"id":"chatcmpl-abc","choices":[{"delta":{"content":"Diff"},"index":0}]}
data: {"id":"chatcmpl-abc","choices":[{"delta":{"content":"usion"},"index":0}]}
…
data: {"id":"chatcmpl-abc","choices":[{"delta":{},"finish_reason":"stop","index":0}]}
data: [DONE]

The format matches OpenAI's; the streaming mode of the OpenAI SDKs works unchanged. If no token arrives for more than ten seconds, for example while the model is thinking, the server sends the comment line : keep-alive to keep the connection open. SSE clients ignore it.

When the model has to start up first: 503

Self-hosted models scale down to zero when nobody uses them. If we can tell before dispatch that no worker is demonstrably ready, we answer immediately:

HTTP/1.1 503 Service Unavailable
Retry-After: 60

{"error":{"message":"The model 'qwen3.8-27b' is scaled to zero and has no worker ready. A warm-up has been started for you — retry in about a minute. Typical cold start: …","type":"api_error","param":null,"code":"model_cold_start"}}

The start-up has then already been triggered. Wait Retry-After seconds and send the same request again. Depending on the model a cold start takes several minutes, so several attempts are often needed. The order of magnitude per model is in the message and in Models.

When something goes wrong mid-stream

As soon as the first byte is out, the HTTP status is fixed (200). A later error therefore arrives inside the stream, as three frames in this order:

data: {"id":"chatcmpl-err-…","object":"chat.completion.chunk","created":1700000000,"model":"gemma-4-12b","choices":[{"index":0,"delta":{},"finish_reason":"error"}]}

event: error
data: {"error":{"message":"upstream error","type":"api_error","param":null,"code":"upstream_502"}}

data: [DONE]

• The closing frame comes first and sets finish_reason: "error". That is how you know the answer is incomplete.
• The error frame carries the fields message, type, param and code.
• [DONE] closes the stream just as in the success case.

type is one of the usual categories (authentication_error, permission_error, rate_limit_error, invalid_request_error, api_error). For a passed-through provider error, code carries that provider's own code, otherwise upstream_<HTTP status>.

Image input (vision)

Only for models with accepts_vision: true, e.g. gemma-4-12b and gemma-4-26b-a4b:

{
  "model": "gemma-4-26b-a4b",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "What is in this image?" },
        { "type": "image_url", "image_url": { "url": "https://…/photo.jpg" } }
      ]
    }
  ]
}

image_url.url can be a public URL or a base64 data URL (data:image/png;base64,…).

Function calling (tools)

As with OpenAI. accepts_tools in GET /v1/models tells you whether a model handles tools. Example:

{
  "model": "gemma-4-12b",
  "messages": [{ "role": "user", "content": "What does a bitcoin cost?" }],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_price",
      "description": "Fetches the current price",
      "parameters": {
        "type": "object",
        "properties": { "symbol": { "type": "string" } },
        "required": ["symbol"]
      }
    }
  }]
}

The response contains tool_calls as with OpenAI. Your client runs the function and sends the result back as a message with role: "tool".

Reasoning trace (reasoning_content)

Models with a visible chain of thought, such as qwen3.5-9b-deepseek-v4 (reasoning tier), return it in its own field reasoning_content, separate from the answer text in content. In a stream it arrives in the delta. The reasoning trace counts against max_tokens.