Video generation

To use a video generation model, you need to use the responses API. The API accepts text and image inputs, and returns generated videos as base64-encoded data.

Notice: We currently don’t offer any shared endpoints for video generation, so the examples below work with dedicated endpoints only.

Requirements

An API key. The code below assumes you set it in an environment variable:
export MODULAR_API_KEY="your_api_key"
$env:MODULAR_API_KEY="your_api_key"
The openai Python package. You can install it with this command:
pip install openai
uv add openai
pixi add openai

Generate a video

To generate a video, set input to a text description of the video and set generation parameters in provider_options.image. The following example generates a video from a text prompt and saves it as output-text-to-video.mp4.

import base64
import os
from pathlib import Path

from openai import OpenAI

client = OpenAI(
    base_url="https://api.modular.com/v1",
    api_key=os.environ.get("MODULAR_API_KEY"),
)

response = client.responses.create(
    model="Wan-AI/Wan2.2-T2V-A14B-Diffusers",
    input=(
        "A campfire crackles in a forest clearing at night, with sparks "
        "spiraling into a star-filled sky."
    ),
    extra_body={
        "provider_options": {
            "image": {
                "height": 512,
                "width": 512,
                "steps": 28,
            }
        }
    },
)

video_data = response.output[0].content[0].image_data
Path("output-text-to-video.mp4").write_bytes(base64.b64decode(video_data))

curl -X POST https://api.modular.com/v1/responses \
  -H "Authorization: Bearer $MODULAR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Wan-AI/Wan2.2-T2V-A14B-Diffusers",
    "input": "A campfire crackles in a forest clearing at night, with sparks spiraling into a star-filled sky.",
    "provider_options": {
      "image": {
        "height": 512,
        "width": 512,
        "steps": 28
      }
    }
  }' \
  | jq -r '.output[0].content[0].image_data' \
  | base64 -d > output-text-to-video.mp4

The generated video should look similar to the following:

Generate a video from an image

To generate a video from an image, set input to a user message containing:

An input_image block with an image URL or base64-encoded data URI.
An input_text block describing how to animate the image.

The following example reads a local image named input.png, encodes it as a data URI, and saves the generated video as output-image-to-video.mp4.

import base64
import os
from pathlib import Path

from openai import OpenAI

client = OpenAI(
    base_url="https://api.modular.com/v1",
    api_key=os.environ.get("MODULAR_API_KEY"),
)

input_data = base64.b64encode(Path("input.png").read_bytes()).decode("utf-8")

response = client.responses.create(
    model="Wan-AI/Wan2.2-I2V-A14B-Diffusers",
    input=[
        {
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": f"data:image/png;base64,{input_data}",
                },
                {
                    "type": "input_text",
                    "text": "Animate this scene with wind moving through the grass.",
                },
            ],
        }
    ],
    extra_body={
        "provider_options": {
            "image": {
                "height": 480,
                "width": 480,
                "steps": 28,
            }
        }
    },
)

video_data = response.output[0].content[0].image_data
Path("output-image-to-video.mp4").write_bytes(base64.b64decode(video_data))

IMAGE_DATA=$(base64 < input.png | tr -d '\n')

curl -X POST https://api.modular.com/v1/responses \
  -H "Authorization: Bearer $MODULAR_API_KEY" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF | jq -r '.output[0].content[0].image_data' | base64 -d > output-image-to-video.mp4
{
  "model": "Wan-AI/Wan2.2-I2V-A14B-Diffusers",
  "input": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_image",
          "image_url": "data:image/png;base64,$IMAGE_DATA"
        },
        {
          "type": "input_text",
          "text": "Animate this scene with wind moving through the grass."
        }
      ]
    }
  ],
  "provider_options": {
    "image": {
      "height": 480,
      "width": 480,
      "steps": 28
    }
  }
}
EOF

Get started

Inference

Administration

Integrations

Requirements

Generate a video

Generate a video from an image

​Requirements

​Generate a video

​Generate a video from an image

Requirements

Generate a video

Generate a video from an image