Function calling

Function calling lets a large language model (LLM) call external functions (also called tools) during inference. This means the model can retrieve data from external systems or run other tasks, then use the results in its response. To make functions available to the model, pass a tools parameter in the request body of the chat completions API. This API is OpenAI-compatible, so you can use the OpenAI SDK without changes. Note that you can’t use the responses API for function calling. To view the list of models that support function calling, see the Supported models page. Function calling only works for models that support it.

When to use function calling

Pass a tools parameter in your chat completions request when you want the model to call your code instead of only returning text. Typical uses include:

Fetching data: Declare tools that read from APIs or databases (for example, weather, prices, or search). The model requests the data; your code runs the call and returns the result for the model to use in its reply.
Performing actions: Declare tools that change state or trigger work (for example, update an app, start a workflow, or call another system). The model chooses the tool and arguments; your code executes the action.

How function calling works

Function calling follows this loop:

You declare tools in the chat completions API request. Each tool describes what the function does and which arguments it accepts.
The model returns a response that includes tool_calls when it wants to use one.
Your code runs the function.
For data-fetching tools, you send the result back in a follow-up request.

The following sections demonstrate this loop using a weather-fetching tool example.

Declare tools

The first step is to declare the tools your model can use. The following example declares a get_weather tool and references the tool in the request:

from openai import OpenAI

def get_weather(location: str) -> str:
    print("Get weather:", location)
    return '{"temperature": 62, "unit": "F"}'

client = OpenAI(
    base_url="https://api.modular.com/v1",
    api_key="<your-api-key>",  # Load your key
)

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current temperature for a given location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country e.g. Bogotá, Colombia"
                }
            },
            "required": [
                "location"
            ],
            "additionalProperties": False
        },
        "strict": True
    }
}]

messages = [
  {
    "role": "user",
    "content": "What's the weather like in San Francisco today?"
  }
]

completion = client.chat.completions.create(
    model="google/gemma-4-31b-it",
    messages=messages,
    tools=tools
)

Take a closer look at each parameter shown in the tools property:

type: The type of tool. Currently, we only support function.
function: The function definition.
- name: The name that the model uses when calling the function.
- description: A description that helps the model decide when to call the function.
- parameters: A JSON Schema for the arguments.
  - type: Must be object.
  - properties: A list of all argument names and types.
  - required: A list of arguments the model must supply to the function.

This shape matches the OpenAI function calling specification.

Control tool use

Use the tool_choice parameter to control tool use. It accepts the following values:

none: Don’t call any tool.
auto (default): Let the model decide whether to call a tool or reply with a message.
required: Require at least one tool call.

A specific function: Force the model to always call a specific function, for example:

"tool_choice": {
  "type": "function",
  "function": {"name": "get_weather"}
}

Handle the model response

If the model chooses a tool, the response includes tool_calls:

print(completion.choices[0].message.tool_calls)

This will print:

[ChatCompletionMessageToolCall(
  id='call_a175692d9ff54554',
  function=Function(
    arguments='{
      "location": "San Francisco, USA"
    }',
    name='get_weather'
  ),
  type='function'
)]

Parse tool_calls and run the matching function in your code:

import json

tool_call = completion.choices[0].message.tool_calls[0]
args = json.loads(tool_call.function.arguments)

result = get_weather(args["location"])

Return results to the model

For tools that fetch data, send another request with the conversation so far, including:

Prior messages
The assistant message that returned tool_calls
A tool message for each result

Set each tool message’s tool_call_id to the id of the matching tool call in that assistant message.

"messages": [
  { "role": "user", "content": "What's the weather in San Francisco?" },
  {
    "role": "assistant",
    "tool_calls": [
      {
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"location\": \"San Francisco\"}"
        }
      }
    ]
  },
  {
    "role": "tool",
    "tool_call_id": "call_abc123",
    "content": "{\"temperature\": 62, \"unit\": \"F\"}"
  }
]

Each tool message’s tool_call_id must match an id from the preceding assistant message. The API returns an HTTP 400 response if a tool_calls argument isn’t JSON, a tool_call_id doesn’t match, or you omit any required tool reply.

Run the example

This walks through the same get_weather example from above as a single, runnable script. If you haven’t already, create an API key and store it securely. Then, send a request with the tools parameter to a model that supports function calling:

Python
curl

Install the OpenAI SDK:

pip install openai

Create a program to send a request specifying the available get_weather() function:

function-calling.py

import json
from openai import OpenAI

def get_weather(location: str) -> str:
    print("Get weather:", location)
    return '{"temperature": 62, "unit": "F"}'

client = OpenAI(
    base_url="https://api.modular.com/v1",
    api_key="<your-api-key>",  # Load your key
)

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current temperature for a given location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country e.g. Bogotá, Colombia"
                }
            },
            "required": [
                "location"
            ],
            "additionalProperties": False
        },
        "strict": True
    }
}]

messages = [
  {
    "role": "user",
    "content": "What's the weather like in San Francisco today?"
  }
]

completion = client.chat.completions.create(
    model="google/gemma-4-31b-it",
    messages=messages,
    tools=tools
)

tool_call = completion.choices[0].message.tool_calls[0]
args = json.loads(tool_call.function.arguments)

result = get_weather(args["location"])

Run it and the get_weather() function should print the argument received:

python function-calling.py

Get weather: San Francisco, USA

Use the following curl command to send a request specifying the available get_weather() function:

curl -X POST https://api.modular.com/v1/chat/completions \
    -H "Authorization: Bearer <your-api-key>" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "google/gemma-4-31b-it",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "What is the weather like in Boston today?"}
        ],
        "tools": [
            {
                "type": "function",
                "function": {
                    "name": "get_weather",
                    "description": "Get the current weather in a given location",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "location": {
                                "type": "string",
                                "description": "The city and state, e.g. Los Angeles, CA"
                            }
                        },
                        "required": ["location"]
                    }
                }
            }
        ],
        "tool_choice": "auto"
    }'

You should receive a response similar to this:

"tool_calls": [
  {
    "id": "call_ac73df14fe184349",
    "type": "function",
    "function": {
        "name": "get_weather",
        "arguments": "{\"location\": \"Boston, MA\"}"
    }
  }
]

Troubleshooting

If your function calling request fails, try the following:

Confirm that function calling is supported by the model.
Check if streaming is enabled. Depending on the model, streaming may interfere with function calling. If you see incomplete or malformed tool-call output while streaming, set stream: false for that request.

Next steps

Find more information in:

The chat completions API reference documentation.
OpenAI’s guide to Function calling.

The same OpenAI tool format works with agent frameworks such as AutoGen, CrewAI, and more.

Get started

Inference

Administration

When to use function calling

How function calling works

Declare tools

Control tool use

Handle the model response

Return results to the model

Run the example

Troubleshooting

Next steps

​When to use function calling

​How function calling works

​Declare tools

​Control tool use

​Handle the model response

​Return results to the model

​Run the example

​Troubleshooting

​Next steps

When to use function calling

How function calling works

Declare tools

Control tool use

Handle the model response

Return results to the model

Run the example

Troubleshooting

Next steps