IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Text to text

MAX makes it easy to generate text with large language models, whether for conversational applications, single-turn prompts, or offline inference workflows. MAX text completion endpoints are fully compatible with the OpenAI API, so you can use familiar tools and libraries.

Text completions let you instruct a model to produce new text based on a prompt or an ongoing conversation. They can be used for a wide range of tasks, including writing content, generating synthetic data, building chatbots, or powering multi-turn assistants. MAX provides two main endpoints for text completions: v1/chat/completions and v1/completions.

Endpoints​

The v1/chat/completions endpoint is recommended as the default for most text use cases and works best with instruction-tuned models. This endpoint supports both single-turn and multi-turn scenarios.

The v1/completions endpoint is also supported for traditional single-turn text generation tasks, which is useful for offline inference or generating text from a prompt without conversational context.

v1/chat/completions​

The v1/chat/completions endpoint is designed for chat-based models and supports both single-turn and multi-turn interactions. You provide a sequence of structured messages with roles (system, user, assistant, or developer), and the model generates a response.

For example, within the v1/chat/completions request body, the "messages" array might look similar to the following:

"messages": [
  {
    "role": "system",
    "content": "You are a helpful assistant."
  },
  {
    "role": "user",
    "content": "Who won the world series in 2020?"
  }
]

Use a combination of roles to give the model the context it needs. A system message can define overall model response behavior, user messages represent instructions or prompts from the end-user interacting with the model, and assistant messages are a way to incorporate past model responses into the message context.

For compatibility with OpenAI SDKs that emit developer for the o1/o3 reasoning models, MAX also accepts developer as a system-equivalent role and normalizes it to system internally.

Some models define extra roles through their chat template. For example, MAX also accepts a root role, but only when the model's tokenizer enables it. Requests that use a role the model doesn't support are rejected.

Use this endpoint whenever you want conversational interaction, such as:

  • Building chatbots or assistants
  • Implementing Q&A systems
  • Supporting multi-turn dialogue in applications

It's also fully compatible with single-turn use cases, making it versatile enough for general text generation workflows.

v1/completions​

The v1/completions endpoint supports traditional text completions. You provide a prompt, and the model returns generated text. This endpoint is ideal when you only need a single response per request, such as:

  • Offline inference workflows
  • Synthetic text generation
  • One-off text generation tasks

Like v1/chat/completions, responses from this endpoint include a usage object reporting prompt, completion, and total token counts.

Quickstart​

Get started quickly serving google/gemma-4-31B-it locally with the max CLI and interact with it through the MAX REST and Python APIs. You'll learn to configure the server and make requests using the OpenAI client libraries as a drop-in replacement.

System requirements:

Set up your environment​

Create a Python project to install our APIs and CLI tools:

  1. If you don't have it, install pixi:
    curl -fsSL https://pixi.sh/install.sh | sh

    Then restart your terminal for the changes to take effect.

  2. Create a project:
    pixi init chat-quickstart \
  -c https://conda.modular.com/max-nightly/ -c conda-forge \
  && cd chat-quickstart
  3. Install max with all dependencies (stableTo get the nightly build, change the version in the website header.):
    pixi add "max-all==26.5"
  4. Start the virtual environment:
    pixi shell

Serve your model​

Use the max serve command to start a local server with the Gemma 4 model:

max serve --model google/gemma-4-31B-it

This creates a server running the google/gemma-4-31B-it large language model on http://localhost:8000/v1/chat/completions, an OpenAI compatible endpoint.

While this example uses the Gemma 4 model, you can replace it with any of the models listed in our supported models. Make sure that the model you choose can fit into the memory of your machine.

The endpoint is ready when you see this message printed in your terminal:

Server ready on http://0.0.0.0:8000 (Press CTRL+C to quit)

For a complete list of max CLI commands and options, refer to the MAX CLI reference.

Generate a text chat completion​

MAX supports OpenAI's REST APIs and you can interact with the model using either the OpenAI Python SDK or curl:

You can use OpenAI's Python client to interact with the model. First, install the OpenAI API:

pixi add openai

Then, create a client and make a request to the model:

generate-text.py
from openai import OpenAI

client = OpenAI(
    base_url = 'http://0.0.0.0:8000/v1',
    api_key='EMPTY', # required by the API, but not used by MAX
)

response = client.chat.completions.create(
  model="google/gemma-4-31B-it",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Who won the world series in 2020?"},
    {"role": "assistant", "content": "The LA Dodgers won in 2020."},
    {"role": "user", "content": "Where was it played?"}
  ]
)
print(response.choices[0].message.content)

In this example, you're using the OpenAI Python client to interact with the MAX endpoint running on local host 8000. The client object is initialized with the base URL http://0.0.0.0:8000/v1 and the API key is ignored.

When you run this code, the model should respond with information about the 2020 World Series location:

python generate-text.py
The 2020 World Series was played at Globe Life Field in Arlington, Texas. It was a neutral site due to the COVID-19 pandemic.

For complete details on all available API endpoints and options, see the REST API documentation.

Reasoning models​

Some models use reasoning, which means they think through a problem before returning the finished output. When you serve a reasoning model, MAX separates the chain-of-thought from the visible answer: the answer appears in the response's content field and the reasoning appears in a dedicated reasoning field. Models with reasoning support include:

  • Gemma 4 (google/gemma-4-31B-it, google/gemma-4-26B-A4B-it)
  • GLM-5.1 and GLM-5.2 (zai-org/GLM-5.1, zai-org/GLM-5.2)
  • Kimi K2.5 and later (nvidia/Kimi-K2.5-NVFP4)
  • MiniMax M2.5 and later (MiniMaxAI/MiniMax-M2.5, MiniMaxAI/MiniMax-M2.7)
  • Nemotron-3 Nano (nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8)
  • Qwen3.5 (Qwen/Qwen3.5-27B)
  • Laguna (poolside/Laguna-M.1-NVFP4)

For example, serve a reasoning model with the max serve command:

max serve --model google/gemma-4-31B-it

Then send a chat request with reasoning turned on:

curl http://0.0.0.0:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "google/gemma-4-31B-it",
        "messages": [
            {
            "role": "user",
            "content": "Where was the 2020 World Series played?"
            }
        ],
        "reasoning": { "enabled": true }
    }'

The response includes the model's reasoning in the reasoning field:

{
  "id": "18b0abd2d2fd463ea43efe2c147bcac0",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning": "The user asks where the 2020 World Series was played.",
        "content": "The 2020 World Series was played at Globe Life Field."
      }
    }
  ],
  "created": 1743543698,
  "model": "google/gemma-4-31B-it",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 64,
    "prompt_tokens": 24,
    "total_tokens": 88,
    "completion_tokens_details": {
      "reasoning_tokens": 42
    }
  }
}

usage.completion_tokens_details.reasoning_tokens reports how many of the completion tokens were spent on reasoning.

Enable reasoning​

Whether a model reasons by default depends on its chat template: some models reason on every request, while others reason only when asked. To turn reasoning on or off for a request, use the reasoning object (or the equivalent thinking object):

"reasoning": {
  "enabled": true
}

Set enabled to false to turn reasoning off.

MAX separates reasoning from the response by default for the models listed above. To check another model, open its architecture source (linked from the supported models table) and look for a reasoning_parser setting. Without one, any thinking markup the model produces stays inline in content.

Rename the reasoning field​

MAX returns reasoning in a field named reasoning. Clients built for other serving APIs, such as vLLM, SGLang, or the DeepSeek API, may look for a field named reasoning_content instead. To use that name in responses, start the server with --emit-reasoning-content:

max serve --model google/gemma-4-31B-it --emit-reasoning-content

Requests accept the reasoning_content name too, so you can send a previous turn's reasoning back to the model.

Next steps​

Now that you have successfully set up MAX with an OpenAI-compatible chat endpoint, check out additional serving optimizations specific to your use case.

Edit this page

Was this page helpful?

Models

Cookbook

Puzzles

Blog

Community

Contact