Skip to main content

Inference (Preview)

POST 
/api/v2/inference/responses

> 🧪 Preview — This endpoint is in preview and its contract may change.

Creates a model response for the given input using an OpenAI-compatible Responses API. Models are exposed as OpenAI-compatible endpoints, so you can call them with the OpenAI SDKs by pointing the base URL at this route and passing your Serenity API key as a Bearer token.

Select the model with the standard model property. A bare model code (e.g. qwen/qwen3.6) resolves against the Serenity Cloud catalogue; prefix it with a vendor as vendor:model (e.g. OpenAI:gpt-5.4-mini) to reach any other configured vendor.

You can pass OpenAI-style client (function) tools in the tools array (the Responses API uses the flat tool shape: { "type": "function", "name", ... }). When the model calls one, the output contains a function_call item (call_id + name + arguments); run the tool on your side and replay it on a follow-up request, appending a function_call_output item with the same call_id.

Structured output is supported through text.format: text returns plain text, json_object returns free-form JSON, and json_schema constrains the reply to a JSON schema you supply. The requested format is validated against the selected model's capabilities, so a model that cannot honour it is rejected rather than silently ignoring it.

When stream is true, the response is delivered as Server-Sent Events (SSE).

Request​

Query Parameters

    stream boolean
    culture string

    Use this param to override the culture of the response. Options: - en (default) - es

Body

required

    input

    object

    required

    agent

    object

    type stringnullable
    name stringnullablerequired
    version stringnullable
    model stringnullable
    instructions stringnullable
    max_output_tokens int32nullable

    reasoning

    object

    Reasoning configuration. Honoured on the AI Proxy routes; the inference routes accept the object but ignore it.

    effort stringnullable

    How much the model should think before answering. The accepted values come from the selected model's own configuration; a model that does not expose the parameter rejects the request.

    detail stringnullable

    Possible values: [auto, none, summary, full]

    How much of the reasoning trace is returned. auto leaves the decision to the vendor and is always accepted; the other values must be supported by the selected model's configuration.

    summary stringnullable

    OpenAI's equivalent of detail, accepted so a standard OpenAI client works unchanged. auto maps to detail: auto, concise to detail: summary and detailed to detail: full. Any other value is ignored. Used only when detail is absent.

    store booleannullable
    stream booleannullable
    previous_response_id stringnullable
    temperature doublenullable
    top_p doublenullable
    parallel_tool_calls booleannullable

    metadata

    object

    nullable

    property name* string
    include string[]nullable

    conversation

    object

    id stringnullable

    metadata

    object

    nullable

    property name* string
    background booleannullable
    max_tool_calls int32nullable
    top_logprobs int32nullable
    safety_identifier stringnullable
    prompt_cache_key stringnullable

    prompt

    object

    id stringnullablerequired

    variables

    object

    nullable

    property name* string
    service_tier stringnullable

    stream_options

    object

    include_obfuscation booleannullable
    truncation stringnullable
    user stringnullabledeprecated
    tools undefined[]nullable
    tool_choice nullable

    text

    object

    format

    object

    nullable

    oneOf

    verbosity stringnullable

Responses​

OK

Loading...