Structured output

Structured output constrains a model’s response to match a schema you define. Write the schema in JSON or generate it from a Pydantic model in Python, then pass it in the response_format field of a chat completions request. When you build an application or agent, your code often needs to act on the model’s response. Structured output lets you process the response using a predefined structure instead of parsing free-form text.

Get a structured response

The following sections show how to define a schema with Pydantic or JSON using an example that triages customer support requests. The schema extracts the order ID, issue, and requested follow-up action from each message. The request returns corresponding structured output.

Pass a Pydantic model

Define a Pydantic model for the response and pass it to client.chat.completions.parse() in the OpenAI Python SDK. The SDK converts the model to a JSON schema, sends it in response_format, and parses the response into an instance of your Pydantic model.

Python

import os
from typing import Literal

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    base_url="https://api.modular.com/v1",
    api_key=os.environ["MODULAR_API_KEY"],
)

class SupportTicket(BaseModel):
    order_id: str
    issue: Literal["damaged", "missing", "late", "other"]
    requested_action: Literal["refund", "replacement", "none"]

completion = client.chat.completions.parse(
    model="google/gemma-4-31b-it",
    messages=[
        {
            "role": "system",
            "content": "Extract the order ID, the issue, and the action the customer wants.",
        },
        {
            "role": "user",
            "content": "Order 4521 arrived with a cracked screen. Can you send me a new one?",
        },
    ],
    response_format=SupportTicket,
    temperature=0,
)

ticket = completion.choices[0].message.parsed
print(ticket)

This prints output similar to the following:

order_id='4521' issue='damaged' requested_action='replacement'

If the response appears truncated and the finish_reason is length, the model reached the token limit before it finished the JSON, so the content is incomplete. Increase max_tokens and send the request again.

Pass a JSON schema

Pass the schema as JSON. Set type to json_schema and include the schema in json_schema.schema.

import json
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.modular.com/v1",
    api_key=os.environ["MODULAR_API_KEY"],
)

completion = client.chat.completions.create(
    model="google/gemma-4-31b-it",
    messages=[
        {
            "role": "system",
            "content": "Extract the order ID, the issue, and the action the customer wants.",
        },
        {
            "role": "user",
            "content": "Order 4521 arrived with a cracked screen. Can you send me a new one?",
        },
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "support_ticket",
            "schema": {
                "type": "object",
                "properties": {
                    "order_id": {"type": "string"},
                    "issue": {"type": "string", "enum": ["damaged", "missing", "late", "other"]},
                    "requested_action": {"type": "string", "enum": ["refund", "replacement", "none"]},
                },
                "required": ["order_id", "issue", "requested_action"],
                "additionalProperties": False,
            },
        },
    },
    temperature=0,
)

ticket = json.loads(completion.choices[0].message.content)
print(ticket)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.modular.com/v1",
  apiKey: process.env.MODULAR_API_KEY,
});

const completion = await client.chat.completions.create({
  model: "google/gemma-4-31b-it",
  messages: [
    {
      role: "system",
      content: "Extract the order ID, the issue, and the action the customer wants.",
    },
    {
      role: "user",
      content: "Order 4521 arrived with a cracked screen. Can you send me a new one?",
    },
  ],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "support_ticket",
      schema: {
        type: "object",
        properties: {
          order_id: { type: "string" },
          issue: { type: "string", enum: ["damaged", "missing", "late", "other"] },
          requested_action: { type: "string", enum: ["refund", "replacement", "none"] },
        },
        required: ["order_id", "issue", "requested_action"],
        additionalProperties: false,
      },
    },
  },
  temperature: 0,
});

const ticket = JSON.parse(completion.choices[0].message.content ?? "{}");
console.log(ticket);

curl -X POST https://api.modular.com/v1/chat/completions \
    -H "Authorization: Bearer $MODULAR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "google/gemma-4-31b-it",
        "messages": [
            {"role": "system", "content": "Extract the order ID, the issue, and the action the customer wants."},
            {"role": "user", "content": "Order 4521 arrived with a cracked screen. Can you send me a new one?"}
        ],
        "response_format": {
            "type": "json_schema",
            "json_schema": {
                "name": "support_ticket",
                "schema": {
                    "type": "object",
                    "properties": {
                        "order_id": {"type": "string"},
                        "issue": {"type": "string", "enum": ["damaged", "missing", "late", "other"]},
                        "requested_action": {"type": "string", "enum": ["refund", "replacement", "none"]}
                    },
                    "required": ["order_id", "issue", "requested_action"],
                    "additionalProperties": false
                }
            }
        },
        "temperature": 0
    }'

Act on the result

Use the parsed response in your application logic. For example, the following code checks ticket.requested_action to determine whether to create a replacement order, issue a refund, or hand off the request to a human:

Python

if ticket.requested_action == "replacement":
    create_replacement_order(ticket.order_id)
elif ticket.requested_action == "refund":
    issue_refund(ticket.order_id)
else:
    hand_off_to_human(ticket)

The application can use the parsed fields to select the appropriate next step.

Validate values before acting on them. The schema constrains the response’s structure and allowed values, but it doesn’t verify business rules. For example, check that an order ID exists before issuing a refund or creating a replacement order.

Example schemas

These examples show different schemas for structured output in application or agent workflows. The Python examples define each schema as a Pydantic model that you pass to client.chat.completions.parse(). The TypeScript and cURL examples pass a JSON schema directly in response_format.

The examples set temperature to 0 to reduce variation in the model’s responses, which is useful for extraction and classification tasks.

Route a request

Have the model choose the next step from a fixed set of options. Your code then sends the request to the matching handler:

class Route(BaseModel):
    reason: str
    destination: Literal["billing", "technical_support", "sales", "other"]

const responseFormat = {
  type: "json_schema",
  json_schema: {
    name: "route",
    schema: {
      type: "object",
      properties: {
        reason: { type: "string" },
        destination: {
          type: "string",
          enum: ["billing", "technical_support", "sales", "other"],
        },
      },
      required: ["reason", "destination"],
      additionalProperties: false,
    },
  },
} as const;

"response_format": {
  "type": "json_schema",
  "json_schema": {
    "name": "route",
    "schema": {
      "type": "object",
      "properties": {
        "reason": {"type": "string"},
        "destination": {"type": "string", "enum": ["billing", "technical_support", "sales", "other"]}
      },
      "required": ["reason", "destination"],
      "additionalProperties": false
    }
  }
}

Extract records

Extract structured records from unstructured text, such as line items from an email, before you write them to a database:

class LineItem(BaseModel):
    product: str
    quantity: int

class Order(BaseModel):
    items: list[LineItem]

const responseFormat = {
  type: "json_schema",
  json_schema: {
    name: "order",
    schema: {
      type: "object",
      properties: {
        items: {
          type: "array",
          items: {
            type: "object",
            properties: {
              product: { type: "string" },
              quantity: { type: "integer" },
            },
            required: ["product", "quantity"],
            additionalProperties: false,
          },
        },
      },
      required: ["items"],
      additionalProperties: false,
    },
  },
} as const;

"response_format": {
  "type": "json_schema",
  "json_schema": {
    "name": "order",
    "schema": {
      "type": "object",
      "properties": {
        "items": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "product": {"type": "string"},
              "quantity": {"type": "integer"}
            },
            "required": ["product", "quantity"],
            "additionalProperties": false
          }
        }
      },
      "required": ["items"],
      "additionalProperties": false
    }
  }
}

Evaluate content

Use the model to evaluate input or output against your rules, such as a moderation step before you show content to users:

from typing import Optional

class ContentCheck(BaseModel):
    is_violating: bool
    category: Optional[Literal["spam", "harassment", "off_topic"]]

const responseFormat = {
  type: "json_schema",
  json_schema: {
    name: "content_check",
    schema: {
      type: "object",
      properties: {
        is_violating: { type: "boolean" },
        category: {
          type: ["string", "null"],
          enum: ["spam", "harassment", "off_topic", null],
        },
      },
      required: ["is_violating", "category"],
      additionalProperties: false,
    },
  },
} as const;

"response_format": {
  "type": "json_schema",
  "json_schema": {
    "name": "content_check",
    "schema": {
      "type": "object",
      "properties": {
        "is_violating": {"type": "boolean"},
        "category": {"type": ["string", "null"], "enum": ["spam", "harassment", "off_topic", null]}
      },
      "required": ["is_violating", "category"],
      "additionalProperties": false
    }
  }
}

A nullable field lets the model return null (None in Python) when no category applies. Use this pattern when a field may not have a meaningful value for every input.

Build a form UI

Use structured output to generate a form definition for information your application still needs, such as travel dates for a booking agent. Your application can use the returned definition to render the form with its own UI components.

from typing import Optional

from pydantic import Field

class FormField(BaseModel):
    name: str = Field(pattern=r"^[a-z][a-z0-9_]*$")
    label: str
    input_type: Literal["text", "textarea", "email", "tel", "number", "date", "checkbox", "select"]
    options: Optional[list[str]]
    required: bool

class Form(BaseModel):
    title: str
    fields: list[FormField]
    submit_label: str

const responseFormat = {
  type: "json_schema",
  json_schema: {
    name: "form",
    schema: {
      type: "object",
      properties: {
        title: { type: "string" },
        fields: {
          type: "array",
          items: {
            type: "object",
            properties: {
              name: { type: "string", pattern: "^[a-z][a-z0-9_]*$" },
              label: { type: "string" },
              input_type: {
                type: "string",
                enum: ["text", "textarea", "email", "tel", "number", "date", "checkbox", "select"],
              },
              options: { type: ["array", "null"], items: { type: "string" } },
              required: { type: "boolean" },
            },
            required: ["name", "label", "input_type", "options", "required"],
            additionalProperties: false,
          },
        },
        submit_label: { type: "string" },
      },
      required: ["title", "fields", "submit_label"],
      additionalProperties: false,
    },
  },
} as const;

"response_format": {
  "type": "json_schema",
  "json_schema": {
    "name": "form",
    "schema": {
      "type": "object",
      "properties": {
        "title": {"type": "string"},
        "fields": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "name": {"type": "string", "pattern": "^[a-z][a-z0-9_]*$"},
              "label": {"type": "string"},
              "input_type": {"type": "string", "enum": ["text", "textarea", "email", "tel", "number", "date", "checkbox", "select"]},
              "options": {"type": ["array", "null"], "items": {"type": "string"}},
              "required": {"type": "boolean"}
            },
            "required": ["name", "label", "input_type", "options", "required"],
            "additionalProperties": false
          }
        },
        "submit_label": {"type": "string"}
      },
      "required": ["title", "fields", "submit_label"],
      "additionalProperties": false
    }
  }
}

The input_type enum restricts the model to input types supported by your UI. The pattern constrains each name to the specified format. For select fields, the model populates options with the available choices.

How the engine enforces your schema

A model generates its response one token at a time. At each step, it scores every token in its vocabulary, and the inference engine selects the next token from those scores. When you send a schema with your request, the inference engine compiles it into a grammar that defines which JSON responses are valid. The engine uses this grammar to determine which tokens to allow at each step, based on the response generated so far. The engine allows the response to end when the generated JSON satisfies the grammar. This process is called constrained decoding.

Next steps

To learn more about what you can do with the chat completions API:

Review the Text generation guide.
See the Create chat completion API reference.

Get started

Inference

Administration

Integrations

Get a structured response

Pass a Pydantic model

Pass a JSON schema

Act on the result

Example schemas

Route a request

Extract records

Evaluate content

Build a form UI

How the engine enforces your schema

Next steps

​Get a structured response

​Pass a Pydantic model

​Pass a JSON schema

​Act on the result

​Example schemas

​Route a request

​Extract records

​Evaluate content

​Build a form UI

​How the engine enforces your schema

​Next steps

Get a structured response

Pass a Pydantic model

Pass a JSON schema

Act on the result

Example schemas

Route a request

Extract records

Evaluate content

Build a form UI

How the engine enforces your schema

Next steps