Skip to main content

Inference (Preview)

POST 
/api/v2/inference/chat/completions

> 🧪 Preview — This endpoint is in preview and its contract may change.

Creates a model completion for the given chat conversation using an OpenAI-compatible Chat Completions API. Models are exposed as OpenAI-compatible endpoints, so you can call them with the OpenAI SDKs by pointing the base URL at this route and passing your Serenity API key as a Bearer token.

Select the model with the standard model property. A bare model code (e.g. qwen/qwen3.6) resolves against the Serenity Cloud catalogue; prefix it with a vendor as vendor:model (e.g. OpenAI:gpt-5.4-mini) to reach any other configured vendor.

You can pass OpenAI-style client (function) tools in the tools array. When the model decides to call one, the response has finish_reason: "tool_calls" and a matching tool_calls entry; run the tool on your side and send the result back on a follow-up request as a role: "tool" message referencing the tool_call_id.

Structured output is supported through response_format: text returns plain text, json_object returns free-form JSON, and json_schema constrains the reply to a JSON schema you supply. The requested format is validated against the selected model's capabilities, so a model that cannot honour it is rejected rather than silently ignoring it.

When stream is true, the response is delivered as Server-Sent Events (SSE).

Request​

Query Parameters

    culture string

    Use this param to override the culture of the response. Options: - en (default) - es

Body

required

    messages

    object[]

    nullable

    required

  • Array [

    • oneOf

    name stringnullable

    content

    object

    required

  • ]

    • model stringnullablerequired
    audio nullable
    frequency_penalty floatnullable
    function_call nullabledeprecated
    functions undefined[]nullabledeprecated

    logit_bias

    object

    nullable

    property name* int32
    logprobs booleannullable
    max_completion_tokens int32nullable
    max_tokens int32nullabledeprecated

    metadata

    object

    nullable

    property name* string
    modalities string[]nullable
    n int32nullable
    parallel_tool_calls booleannullable
    prediction nullable
    presence_penalty floatnullable
    prompt_cache_key stringnullable
    reasoning_effort stringnullable

    response_format

    object

    The format the model must reply in. Exactly one of the three shapes.

    oneOf

    Plain text output. The default when response_format is omitted.

    type stringrequired

    Possible values: [text]

    safety_identifier stringnullable
    seed int64nullable
    service_tier stringnullable

    stop

    object

    store booleannullable
    stream booleannullable
    stream_options nullable
    temperature floatnullable

    tool_choice

    object

    allowedTools

    object

    allowed_tools

    object

    required

    mode stringnullablerequired

    tools

    object[]

    nullable

    required

  • Array [

    • type stringnullablerequired

    function

    object

    name stringnullablerequired

  • ]

    • functionTool

    object

    function

    object

    required

    name stringnullablerequired

    customTool

    object

    custom

    object

    required

    name stringnullablerequired

    tools

    object[]

    nullable

  • Array [

    • oneOf

    function

    object

    required

    name stringnullablerequired
    description stringnullable
    parameters nullable
    strict booleannullable

  • ]

    • top_logprobs int32nullable
    top_p floatnullable
    verbosity stringnullable
    web_search_options nullable

Responses​

OK

Loading...