POST /v1/chat/completions

Generates a model response to a chat conversation.

curl https://sovrgpt.com/api/v1/chat/completions \
  -H "Authorization: Bearer $SOVR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-12b",
    "messages": [
      { "role": "system", "content": "You are a helpful assistant." },
      { "role": "user",   "content": "Explain diffusion models in two sentences." }
    ],
    "temperature": 0.5,
    "max_tokens": 400
  }'

Request body

Ignored (without crashing): seed, logit_bias, user, n (values above 1 are not supported), presence_penalty, frequency_penalty.

Reasoning effort (reasoning_effort)

Many models can "think" to different depths per request. Control it the OpenAI-idiomatic way through reasoning_effort (in the Python SDK via extra_body):

curl https://sovrgpt.com/api/v1/chat/completions \
  -H "Authorization: Bearer $SOVR_KEY" -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [{ "role": "user", "content": "Find and fix the bug in this stack trace …" }],
    "reasoning_effort": "high"
  }'

Which models evaluate it

GET /v1/models reports effort_levels per model — the levels that have been measured for that model. An empty array means this model has no effective control, and a value sent anyway is not translated.

We only offer levels whose effect we have measured. A model that accepts a value (HTTP 200) but ignores it deliberately lists no levels here — a promise that does not hold is worse than no promise.

If a model does not offer the level you asked for (some have only low and high), it is mapped to the next lower available one — never to a higher one.

What happens internally

The names above are ours; what reaches the model differs, and that is deliberately invisible to callers. Examples from live operation: balanced and coder-mini do not expect reasoning_effort at all but a token budget (thinking_token_budget, a number); on one of the partner-operated models the top level is called xhigh and a high would be rejected there with HTTP 400; on another the scale runs in reverse. The route performs this translation — you send low, medium or high.

An explicitly supplied chat_template_kwargs is passed through verbatim and wins over reasoning_effort — set that field and you get exactly it.

Response (non-streaming)

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1715520000,
  "model": "gemma-4-12b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Diffusion models …"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 42,
    "completion_tokens": 87,
    "total_tokens": 129
  }
}

Which model answered? Three headers

Every response — streaming, non-streaming and the 503 below — carries:

🔴 Why this matters: an unknown or withdrawn model id does NOT fail, it gets substituted. You receive HTTP 200 and a usable answer from the sovereign default model. That is deliberate — an integration that worked yesterday should not die today because the catalog changed — but you should notice it:

curl -sD- -o/dev/null -X POST https://sovrgpt.com/api/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Hello"}]}'

# x-sovrgpt-model-served: gemma-4-12b
# x-sovrgpt-model-requested: gpt-4o
# x-sovrgpt-model-substituted: unknown_model

⚠️ An alias is not a substitution. coding resolves to qwen3.6-35b-a3b; in that case only x-sovrgpt-model-served is present. The substitution headers appear exclusively when you got a different model than you asked for.

🔑 That is precisely the difference between a role name and a model ID, and it became practical on 2026-09-11: when qwen3-coder-next-fp8 was withdrawn, the role name coder was repointed to its successor — send that and you still get a coding model, with no substitution header. ⇒ If the role matters to you more than the specific weights, send the role name.

A withdrawn model: 404 model_withdrawn

An unknown ID gets substituted (see above). An ID that existed here once and that we have withdrawn does not get substituted — it fails, with its own code and the successor spelled out:

{
  "error": {
    "message": "The model 'qwen3-coder-next-fp8' has been withdrawn and is no longer served. Zurückgezogen am 2026-09-11 (32 768 Token Kontext, von 'qwen3.6-35b-a3b' überholt). Nachfolger: 'qwen3.6-35b-a3b' … Call GET /v1/models for the list you may use.",
    "type": "invalid_request_error",
    "param": "model",
    "code": "model_withdrawn"
  }
}

HTTP 404, plus the header x-sovrgpt-model-withdrawn carrying the affected ID.

🔑 Why an error here and not a substitution: a silent substitution is right as long as we still hit the intent of the request — with a role name we do. With a withdrawn ID we would not know: someone who explicitly named a coding model is worse served by a general one than by a clear error.

⚠️ Do not confuse this with model_not_found. The two codes call for different responses:

📌 If you want a one-line safeguard: check for the presence of x-sovrgpt-model-substituted and log it. The model field in the body carries the same truth, but you would have to think of comparing it against what you sent.

Streaming (SSE)

With "stream": true or Accept: text/event-stream:

data: {"id":"chatcmpl-abc","choices":[{"delta":{"role":"assistant"},"index":0}]}
data: {"id":"chatcmpl-abc","choices":[{"delta":{"content":"Diff"},"index":0}]}
data: {"id":"chatcmpl-abc","choices":[{"delta":{"content":"usion"},"index":0}]}
…
data: {"id":"chatcmpl-abc","choices":[{"delta":{},"finish_reason":"stop","index":0}]}
data: [DONE]

The format is identical to OpenAI's — the OpenAI SDK's stream: true mode works without changes.

When the model has to spin up first: 503

The models we host ourselves scale down to zero when nobody is using them. Ask for one where no worker is demonstrably ready, and you get this immediately:

HTTP/1.1 503 Service Unavailable
Retry-After: 60

{"error":{"message":"The model 'qwen3.8-27b' is scaled to zero and has no worker ready. A warm-up has been started for you — retry in about a minute. Typical cold start: …","type":"api_error","param":null,"code":"model_cold_start"}}

The spin-up has already been triggered for you. Wait Retry-After seconds and send the same request again. Depending on the model a cold start takes several minutes, so more than one attempt may be needed. The order of magnitude per model is in the message and in Models.

When something goes wrong mid-stream

As soon as the first byte is out, the HTTP status is fixed (200). An error after that can only be reported inside the stream. You then get three frames, in this order:

data: {"id":"chatcmpl-err-…","object":"chat.completion.chunk","created":1700000000,"model":"gemma-4-12b","choices":[{"index":0,"delta":{},"finish_reason":"error"}]}

event: error
data: {"error":{"message":"upstream error","type":"api_error","param":null,"code":"upstream_502"}}

data: [DONE]

• The closing frame comes first and sets finish_reason: "error" — that is how you know the answer is incomplete.
• The error frame is a frame of its own and carries the four familiar fields message, type, param, code.
• [DONE] closes the stream just as in the success case.

type is one of the usual categories (authentication_error, permission_error, rate_limit_error, invalid_request_error, api_error). For a passed-through provider error, code carries that provider's own code, otherwise upstream_<HTTP status> — so the status is machine-readable rather than buried in prose.

Vision input

Only for models with accepts_vision: true (the vision tier, and the default tier from 3.5 onwards):

{
  "model": "gemma-4-26b-a4b",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "What is in this image?" },
        { "type": "image_url", "image_url": { "url": "https://…/photo.jpg" } }
      ]
    }
  ]
}

image_url.url can be a public URL or a base64 data URL (data:image/png;base64,…).

Function calling / tools

Fully OpenAI-compatible. Example:

{
  "model": "gemma-4-12b",
  "messages": [{ "role": "user", "content": "What does a bitcoin cost?" }],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_price",
      "description": "Fetches the current price",
      "parameters": {
        "type": "object",
        "properties": { "symbol": { "type": "string" } },
        "required": ["symbol"]
      }
    }
  }]
}

The response contains tool_calls exactly as OpenAI's does. The client runs the function and sends the result back as a role: "tool" message.

Reasoning blocks

For models in the reasoning tier, the response additionally contains <think>…</think> blocks before the final answer. UIs can show or hide these — the SovrGPT UI collapses them by default.

Until 2026-08 this heading read "Errors & reasoning" but described only the reasoning blocks and not a single error behaviour. Errors now live where they occur: in the stream and as a regular HTTP status before it.