> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modular.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Text generation

> Generate text with an OpenAI-compatible chat completions API

Generate text from a prompt or conversation with the chat completions API. This
API allows you to send text, images, and videos with your request.

Learn about different ways to use the API throughout this page. For information
on all request and response fields, see the [chat completions API
reference](/api/inference/create-chat-completion). For a list of
models and their supported features and modalities, see [Supported
models](/models).

<Tip>
  This API supports the OpenAI Chat Completions interface.

  To try a Modular endpoint with an existing OpenAI SDK integration, set `base_url` to `https://api.modular.com/v1`
  and `api_key` to your [Modular API key](/administration/api-keys).
</Tip>

## Generate a text response

Generate text from a prompt or conversation history.

<Tabs>
  <Tab title="Python">
    Send a chat completion request using the OpenAI Python SDK:

    ```python title="generate-text.py" theme={null}
    from openai import OpenAI

    client = OpenAI(
        base_url="https://api.modular.com/v1",
        api_key="<your-api-key>",  # Load your key
    )

    response = client.chat.completions.create(
        model="google/gemma-4-31b-it",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Who won the world series in 2020?"},
            {"role": "assistant", "content": "The LA Dodgers won in 2020."},
            {"role": "user", "content": "Where was it played?"}
        ]
    )
    print(response.choices[0].message.content)
    ```

    The response should be similar to:

    ```text theme={null}
    The 2020 World Series was played at Globe Life Field in Arlington, Texas. It was a neutral site due to the COVID-19 pandemic.
    ```
  </Tab>

  <Tab title="curl">
    Send the same request using curl:

    ```bash theme={null}
    curl -X POST https://api.modular.com/v1/chat/completions \
        -H "Authorization: Bearer <your-api-key>" \
        -H "Content-Type: application/json" \
        -d '{
            "model": "google/gemma-4-31b-it",
            "messages": [
                {
                "role": "system",
                "content": "You are a helpful assistant."
                },
                {
                "role": "user",
                "content": "Hello, how are you?"
                }
            ],
            "max_tokens": 100
        }'
    ```

    The response should be similar to:

    ```json theme={null}
    {
      "choices": [
        {
          "finish_reason": "stop",
          "index": 0,
          "message": {
            "content": "I'm doing well, thank you for asking. How can I assist you today?",
            "role": "assistant"
          }
        }
      ],
      "model": "google/gemma-4-31b-it",
      "object": "chat.completion",
      "usage": {
        "completion_tokens": 17,
        "prompt_tokens": null,
        "total_tokens": 17
      }
    }
    ```
  </Tab>
</Tabs>

## Stream a text response

Set `stream=True` to receive tokens as they are generated instead of waiting
for the full response.

<Tabs>
  <Tab title="Python">
    Set `stream=True` and iterate over the response chunks as they arrive:

    ```python title="stream-text.py" theme={null}
    from openai import OpenAI

    client = OpenAI(
        base_url="https://api.modular.com/v1",
        api_key="<your-api-key>",  # Load your key
    )

    stream = client.chat.completions.create(
        model="google/gemma-4-31b-it",
        messages=[{"role": "user", "content": "Write a short poem about the sea."}],
        stream=True,
    )

    for chunk in stream:
        if chunk.choices[0].delta.content is not None:
            print(chunk.choices[0].delta.content, end="", flush=True)
    print()
    ```
  </Tab>

  <Tab title="curl">
    Pass `"stream": true` in the request body to receive server-sent events:

    ```bash theme={null}
    curl -X POST https://api.modular.com/v1/chat/completions \
        -H "Authorization: Bearer <your-api-key>" \
        -H "Content-Type: application/json" \
        -d '{
            "model": "google/gemma-4-31b-it",
            "messages": [{"role": "user", "content": "Write a short poem about the sea."}],
            "stream": true
        }'
    ```

    Each streamed chunk is a server-sent event with a `data:` prefix:

    ```text theme={null}
    data: {"choices":[{"delta":{"content":"The"},"index":0}],"object":"chat.completion.chunk"}
    data: {"choices":[{"delta":{"content":" sea"},"index":0}],"object":"chat.completion.chunk"}
    ...
    data: [DONE]
    ```
  </Tab>
</Tabs>

## Analyze an image

Provide an image and a text prompt to generate a description of
or extract information from visual content.

Include the image in the `messages` array using an `image_url` content item. The
`url` field accepts either a publicly accessible URL or a base64-encoded data
URI (for example, `data:image/jpeg;base64,...`).

The following example asks the model to describe the contents of an image.

<Tabs>
  <Tab title="Python">
    Include an `image_url` content block alongside your text prompt:

    ```python title="generate-image-description.py" theme={null}
    from openai import OpenAI

    client = OpenAI(
        base_url="https://api.modular.com/v1",
        api_key="<your-api-key>",  # Load your key
    )

    response = client.chat.completions.create(
        model="google/gemma-4-31b-it",
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "What is in this image?"
                    },
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/0052a70beed5bf71b92610a43a52df6d286cd5f3/diffusers/rabbit.jpg"
                        }
                    }
                ]
            }
        ],
        max_tokens=300
    )

    print(response.choices[0].message.content)
    ```

    The response should be similar to:

    ```text theme={null}
    Here's a breakdown of what's in the image:

    *   **Peter Rabbit:** The main focus is a realistic-looking depiction of Peter
    Rabbit, the character from Beatrix Potter's stories...
    ```
  </Tab>

  <Tab title="curl">
    Pass the image URL and text prompt in the `content` array of the request body:

    ```bash theme={null}
    curl -X POST https://api.modular.com/v1/chat/completions \
    -H "Authorization: Bearer <your-api-key>" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "google/gemma-4-31b-it",
      "messages": [
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "What is in this image?"
            },
            {
              "type": "image_url",
              "image_url": {
                "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/0052a70beed5bf71b92610a43a52df6d286cd5f3/diffusers/rabbit.jpg"
              }
            }
          ]
        }
      ],
      "max_tokens": 300
    }' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '\n' | sed 's/\\n/\n/g'
    ```

    The response should be similar to:

    ```text theme={null}
    Here's a breakdown of what's in the image:

    *   **Peter Rabbit:** The main focus is a realistic, anthropomorphic
    (human-like) rabbit character...
    ```
  </Tab>
</Tabs>

## Analyze a video

Provide a video and a text prompt to generate a natural-language description or
analysis of the video's visual content.

Include the video in the `messages` array using a `video_url` content item. The
`url` field accepts either a publicly accessible URL or a base64-encoded data
URI (for example, `data:video/mp4;base64,...`).

The following example asks the model to describe the contents of a video.

<Tabs>
  <Tab title="Python">
    Pass a video URL and a text question to get a natural-language description of
    the video:

    ```python title="generate-video-description.py" theme={null}
    from openai import OpenAI

    client = OpenAI(
        base_url="https://api.modular.com/v1",
        api_key="<your-api-key>",  # Load your key
    )

    completion = client.chat.completions.create(
        model="google/gemma-4-31b-it",
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "Describe what is happening in this video"
                    },
                    {
                        "type": "video_url",
                        "video_url": {
                            "url": "https://avtshare01.rz.tu-ilmenau.de/avt-vqdb-uhd-1/test_1/segments/bigbuck_bunny_8bit_15000kbps_1080p_60.0fps_h264.mp4"
                        }
                    }
                ]
            }
        ],
        max_tokens=300
    )

    print(completion.choices[0].message.content)
    ```

    The response should be similar to:

    ```text theme={null}
    The video is an animated short film featuring a large, fluffy rabbit in a
    colorful meadow. The rabbit wanders through the environment, encountering
    butterflies and small birds. The animation has a warm, lighthearted tone with
    vibrant natural scenery...
    ```
  </Tab>

  <Tab title="curl">
    Pass the video URL and text prompt in the `content` array of the request body:

    ```bash theme={null}
    curl -X POST https://api.modular.com/v1/chat/completions \
    -H "Authorization: Bearer <your-api-key>" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "google/gemma-4-31b-it",
      "messages": [
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": "Describe what is happening in this video"
            },
            {
              "type": "video_url",
              "video_url": {
                "url": "https://avtshare01.rz.tu-ilmenau.de/avt-vqdb-uhd-1/test_1/segments/bigbuck_bunny_8bit_15000kbps_1080p_60.0fps_h264.mp4"
              }
            }
          ]
        }
      ],
      "max_tokens": 300
    }' | grep -o '"content":"[^"]*"' | sed 's/"content":"//g' | sed 's/"//g' | tr -d '\n' | sed 's/\\n/\n/g'
    ```

    The response should be similar to:

    ```text theme={null}
    The video is an animated short film featuring a large, fluffy rabbit in a
    colorful meadow. The rabbit wanders through the environment, encountering
    butterflies and small birds. The animation has a warm, lighthearted tone with
    vibrant natural scenery...
    ```
  </Tab>
</Tabs>