> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modular.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Structured output

> Format text responses by providing a JSON schema

Structured output constrains a model's response to match a schema you define.
Write the schema in JSON or generate it from a Pydantic model in Python, then
pass it in the `response_format` field of a
[chat completions](/api/inference/create-chat-completion) request.

When you build an application or agent, your code often needs to act on the
model's response. Structured output lets you process the response using a
predefined structure instead of parsing free-form text.

## Get a structured response

The following sections show how to define a schema with Pydantic or JSON using
an example that triages customer support requests. The schema extracts the order
ID, issue, and requested follow-up action from each message. The request returns
corresponding structured output.

### Pass a Pydantic model

Define a Pydantic model for the response and pass it to
`client.chat.completions.parse()` in the OpenAI Python SDK. The SDK converts the
model to a JSON schema, sends it in `response_format`, and parses the response
into an instance of your Pydantic model.

```python Python theme={null}
import os
from typing import Literal

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    base_url="https://api.modular.com/v1",
    api_key=os.environ["MODULAR_API_KEY"],
)

class SupportTicket(BaseModel):
    order_id: str
    issue: Literal["damaged", "missing", "late", "other"]
    requested_action: Literal["refund", "replacement", "none"]

completion = client.chat.completions.parse(
    model="google/gemma-4-31b-it",
    messages=[
        {
            "role": "system",
            "content": "Extract the order ID, the issue, and the action the customer wants.",
        },
        {
            "role": "user",
            "content": "Order 4521 arrived with a cracked screen. Can you send me a new one?",
        },
    ],
    response_format=SupportTicket,
    temperature=0,
)

ticket = completion.choices[0].message.parsed
print(ticket)
```

This prints output similar to the following:

```output theme={null}
order_id='4521' issue='damaged' requested_action='replacement'
```

If the response appears truncated and the `finish_reason` is `length`, the
model reached the token limit before it finished the JSON, so the content is
incomplete. Increase `max_tokens` and send the request again.

### Pass a JSON schema

Pass the schema as JSON. Set `type` to `json_schema` and include the schema in
`json_schema.schema`.

```python Python theme={null}
import json
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.modular.com/v1",
    api_key=os.environ["MODULAR_API_KEY"],
)

completion = client.chat.completions.create(
    model="google/gemma-4-31b-it",
    messages=[
        {
            "role": "system",
            "content": "Extract the order ID, the issue, and the action the customer wants.",
        },
        {
            "role": "user",
            "content": "Order 4521 arrived with a cracked screen. Can you send me a new one?",
        },
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "support_ticket",
            "schema": {
                "type": "object",
                "properties": {
                    "order_id": {"type": "string"},
                    "issue": {"type": "string", "enum": ["damaged", "missing", "late", "other"]},
                    "requested_action": {"type": "string", "enum": ["refund", "replacement", "none"]},
                },
                "required": ["order_id", "issue", "requested_action"],
                "additionalProperties": False,
            },
        },
    },
    temperature=0,
)

ticket = json.loads(completion.choices[0].message.content)
print(ticket)
```

```typescript TypeScript theme={null}
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.modular.com/v1",
  apiKey: process.env.MODULAR_API_KEY,
});

const completion = await client.chat.completions.create({
  model: "google/gemma-4-31b-it",
  messages: [
    {
      role: "system",
      content: "Extract the order ID, the issue, and the action the customer wants.",
    },
    {
      role: "user",
      content: "Order 4521 arrived with a cracked screen. Can you send me a new one?",
    },
  ],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "support_ticket",
      schema: {
        type: "object",
        properties: {
          order_id: { type: "string" },
          issue: { type: "string", enum: ["damaged", "missing", "late", "other"] },
          requested_action: { type: "string", enum: ["refund", "replacement", "none"] },
        },
        required: ["order_id", "issue", "requested_action"],
        additionalProperties: false,
      },
    },
  },
  temperature: 0,
});

const ticket = JSON.parse(completion.choices[0].message.content ?? "{}");
console.log(ticket);
```

```bash cURL theme={null}
curl -X POST https://api.modular.com/v1/chat/completions \
    -H "Authorization: Bearer $MODULAR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "google/gemma-4-31b-it",
        "messages": [
            {"role": "system", "content": "Extract the order ID, the issue, and the action the customer wants."},
            {"role": "user", "content": "Order 4521 arrived with a cracked screen. Can you send me a new one?"}
        ],
        "response_format": {
            "type": "json_schema",
            "json_schema": {
                "name": "support_ticket",
                "schema": {
                    "type": "object",
                    "properties": {
                        "order_id": {"type": "string"},
                        "issue": {"type": "string", "enum": ["damaged", "missing", "late", "other"]},
                        "requested_action": {"type": "string", "enum": ["refund", "replacement", "none"]}
                    },
                    "required": ["order_id", "issue", "requested_action"],
                    "additionalProperties": false
                }
            }
        },
        "temperature": 0
    }'
```

### Act on the result

Use the parsed response in your application logic. For example, the following
code checks `ticket.requested_action` to determine whether to create a
replacement order, issue a refund, or hand off the request to a human:

```python Python theme={null}
if ticket.requested_action == "replacement":
    create_replacement_order(ticket.order_id)
elif ticket.requested_action == "refund":
    issue_refund(ticket.order_id)
else:
    hand_off_to_human(ticket)
```

The application can use the parsed fields to select the appropriate next step.

> [!TIP]
>
> Validate values before acting on them. The schema constrains the response's
> structure and allowed values, but it doesn't verify business rules. For
> example, check that an order ID exists before issuing a refund or creating a
> replacement order.

## Example schemas

These examples show different schemas for structured output in
application or agent workflows.

The Python examples define each schema as a Pydantic model that you pass to
`client.chat.completions.parse()`. The TypeScript and cURL examples pass a JSON
schema directly in `response_format`.

> [!NOTE]
>
> The examples set `temperature` to `0` to reduce variation in the model's responses,
> which is useful for extraction and classification tasks.

### Route a request

Have the model choose the next step from a fixed set of options. Your code then
sends the request to the matching handler:

```python Python theme={null}
class Route(BaseModel):
    reason: str
    destination: Literal["billing", "technical_support", "sales", "other"]
```

```typescript TypeScript theme={null}
const responseFormat = {
  type: "json_schema",
  json_schema: {
    name: "route",
    schema: {
      type: "object",
      properties: {
        reason: { type: "string" },
        destination: {
          type: "string",
          enum: ["billing", "technical_support", "sales", "other"],
        },
      },
      required: ["reason", "destination"],
      additionalProperties: false,
    },
  },
} as const;
```

```json cURL theme={null}
"response_format": {
  "type": "json_schema",
  "json_schema": {
    "name": "route",
    "schema": {
      "type": "object",
      "properties": {
        "reason": {"type": "string"},
        "destination": {"type": "string", "enum": ["billing", "technical_support", "sales", "other"]}
      },
      "required": ["reason", "destination"],
      "additionalProperties": false
    }
  }
}
```

### Extract records

Extract structured records from unstructured text, such as line items
from an email, before you write them to a database:

```python Python theme={null}
class LineItem(BaseModel):
    product: str
    quantity: int

class Order(BaseModel):
    items: list[LineItem]
```

```typescript TypeScript theme={null}
const responseFormat = {
  type: "json_schema",
  json_schema: {
    name: "order",
    schema: {
      type: "object",
      properties: {
        items: {
          type: "array",
          items: {
            type: "object",
            properties: {
              product: { type: "string" },
              quantity: { type: "integer" },
            },
            required: ["product", "quantity"],
            additionalProperties: false,
          },
        },
      },
      required: ["items"],
      additionalProperties: false,
    },
  },
} as const;
```

```json cURL theme={null}
"response_format": {
  "type": "json_schema",
  "json_schema": {
    "name": "order",
    "schema": {
      "type": "object",
      "properties": {
        "items": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "product": {"type": "string"},
              "quantity": {"type": "integer"}
            },
            "required": ["product", "quantity"],
            "additionalProperties": false
          }
        }
      },
      "required": ["items"],
      "additionalProperties": false
    }
  }
}
```

### Evaluate content

Use the model to evaluate input or output against your rules, such as a
moderation step before you show content to users:

```python Python theme={null}
from typing import Optional

class ContentCheck(BaseModel):
    is_violating: bool
    category: Optional[Literal["spam", "harassment", "off_topic"]]
```

```typescript TypeScript theme={null}
const responseFormat = {
  type: "json_schema",
  json_schema: {
    name: "content_check",
    schema: {
      type: "object",
      properties: {
        is_violating: { type: "boolean" },
        category: {
          type: ["string", "null"],
          enum: ["spam", "harassment", "off_topic", null],
        },
      },
      required: ["is_violating", "category"],
      additionalProperties: false,
    },
  },
} as const;
```

```json cURL theme={null}
"response_format": {
  "type": "json_schema",
  "json_schema": {
    "name": "content_check",
    "schema": {
      "type": "object",
      "properties": {
        "is_violating": {"type": "boolean"},
        "category": {"type": ["string", "null"], "enum": ["spam", "harassment", "off_topic", null]}
      },
      "required": ["is_violating", "category"],
      "additionalProperties": false
    }
  }
}
```

A nullable field lets the model return `null` (`None` in Python) when no
category applies. Use this pattern when a field may not have a meaningful value
for every input.

### Build a form UI

Use structured output to generate a form definition for information your
application still needs, such as travel dates for a booking agent. Your
application can use the returned definition to render the form with its own UI
components.

```python Python theme={null}
from typing import Optional

from pydantic import Field

class FormField(BaseModel):
    name: str = Field(pattern=r"^[a-z][a-z0-9_]*$")
    label: str
    input_type: Literal["text", "textarea", "email", "tel", "number", "date", "checkbox", "select"]
    options: Optional[list[str]]
    required: bool

class Form(BaseModel):
    title: str
    fields: list[FormField]
    submit_label: str
```

```typescript TypeScript theme={null}
const responseFormat = {
  type: "json_schema",
  json_schema: {
    name: "form",
    schema: {
      type: "object",
      properties: {
        title: { type: "string" },
        fields: {
          type: "array",
          items: {
            type: "object",
            properties: {
              name: { type: "string", pattern: "^[a-z][a-z0-9_]*$" },
              label: { type: "string" },
              input_type: {
                type: "string",
                enum: ["text", "textarea", "email", "tel", "number", "date", "checkbox", "select"],
              },
              options: { type: ["array", "null"], items: { type: "string" } },
              required: { type: "boolean" },
            },
            required: ["name", "label", "input_type", "options", "required"],
            additionalProperties: false,
          },
        },
        submit_label: { type: "string" },
      },
      required: ["title", "fields", "submit_label"],
      additionalProperties: false,
    },
  },
} as const;
```

```json cURL theme={null}
"response_format": {
  "type": "json_schema",
  "json_schema": {
    "name": "form",
    "schema": {
      "type": "object",
      "properties": {
        "title": {"type": "string"},
        "fields": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "name": {"type": "string", "pattern": "^[a-z][a-z0-9_]*$"},
              "label": {"type": "string"},
              "input_type": {"type": "string", "enum": ["text", "textarea", "email", "tel", "number", "date", "checkbox", "select"]},
              "options": {"type": ["array", "null"], "items": {"type": "string"}},
              "required": {"type": "boolean"}
            },
            "required": ["name", "label", "input_type", "options", "required"],
            "additionalProperties": false
          }
        },
        "submit_label": {"type": "string"}
      },
      "required": ["title", "fields", "submit_label"],
      "additionalProperties": false
    }
  }
}
```

The `input_type` enum restricts the model to input types supported by your UI.
The `pattern` constrains each `name` to the specified format. For `select`
fields, the model populates `options` with the available choices.

## How the engine enforces your schema

A model generates its response one token at a time. At each step, it scores
every token in its vocabulary, and the inference engine selects the next token
from those scores.

When you send a schema with your request, the inference engine compiles it into
a grammar that defines which JSON responses are valid. The engine uses this
grammar to determine which tokens to allow at each step, based on the
response generated so far. The engine allows the response to end when the
generated JSON satisfies the grammar. This process is called
*constrained decoding*.

The model only scores tokens. The inference engine compiles the schema into a grammar, allows only tokens the grammar permits, selects the next token, and repeats until the JSON is complete, then allows the response to end.

## Next steps

To learn more about what you can do with the chat completions API:

* Review the [Text generation](/inference/text) guide.
* See the [Create chat completion](/api/inference/create-chat-completion) API
  reference.