> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modular.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Function calling

> Use OpenAI-compatible function calling and tool use with the chat completions API

Function calling lets a large language model (LLM) call external functions (also
called *tools*) during inference. This means the model can retrieve data from
external systems or run other tasks, then use the results in its response.

To make functions available to the model, pass a `tools` parameter in the
request body of the
[chat completions API](/api/inference/create-chat-completion). This
API is OpenAI-compatible, so you can use the OpenAI SDK without changes. Note
that you can't use the [responses API](/api/inference/create-response)
for function calling.

To view the list of models that support function calling, see the
[Supported models](/models) page. Function calling only works for models that
support it.

## When to use function calling

Pass a `tools` parameter in your chat completions request when you want the
model to call your code instead of only returning text. Typical uses include:

* **Fetching data**: Declare tools that read from APIs or databases (for
  example, weather, prices, or search). The model requests the data; your
  code runs the call and returns the result for the model to use in its
  reply.
* **Performing actions**: Declare tools that change state or trigger work
  (for example, update an app, start a workflow, or call another system).
  The model chooses the tool and arguments; your code executes the action.

## How function calling works

Function calling follows this loop:

1. You declare tools in the
   [chat completions API](/api/inference/create-chat-completion)
   request. Each tool describes *what the function does* and
   *which arguments it accepts*.
2. The model returns a response that includes `tool_calls` when it wants to use
   one.
3. Your code runs the function.
4. For data-fetching tools, you send the result back in a follow-up request.

The following sections demonstrate this loop using a weather-fetching tool
example.

### Declare tools

The first step is to declare the tools your model can use. The following example
declares a `get_weather` tool and references the tool in the request:

```python theme={null}
from openai import OpenAI

def get_weather(location: str) -> str:
    print("Get weather:", location)
    return '{"temperature": 62, "unit": "F"}'

client = OpenAI(
    base_url="https://api.modular.com/v1",
    api_key="<your-api-key>",  # Load your key
)

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current temperature for a given location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country e.g. Bogotá, Colombia"
                }
            },
            "required": [
                "location"
            ],
            "additionalProperties": False
        },
        "strict": True
    }
}]

messages = [
  {
    "role": "user",
    "content": "What's the weather like in San Francisco today?"
  }
]

completion = client.chat.completions.create(
    model="google/gemma-4-31b-it",
    messages=messages,
    tools=tools
)
```

Take a closer look at each parameter shown in the `tools` property:

* `type`: The type of tool. Currently, we only support `function`.
* `function`: The function definition.
  * `name`: The name that the model uses when calling the function.
  * `description`: A description that helps the model decide when to call the
    function.
  * `parameters`: A JSON Schema for the arguments.
    * `type`: Must be `object`.
    * `properties`: A list of all argument names and types.
    * `required`: A list of arguments the model must supply to the function.

This shape matches the [OpenAI function calling
specification](https://platform.openai.com/docs/guides/function-calling).

### Control tool use

Use the `tool_choice` parameter to control tool use. It accepts the following
values:

* `none`: Don't call any tool.
* `auto` (default): Let the model decide whether to call a tool or reply with a
  message.
* `required`: Require at least one tool call.
* A specific function: Force the model to always call a specific function, for
  example:

  ```json theme={null}
  "tool_choice": {
    "type": "function",
    "function": {"name": "get_weather"}
  }
  ```

### Handle the model response

If the model chooses a tool, the response includes `tool_calls`:

```python theme={null}
print(completion.choices[0].message.tool_calls)
```

This will print:

```js theme={null}
[ChatCompletionMessageToolCall(
  id='call_a175692d9ff54554',
  function=Function(
    arguments='{
      "location": "San Francisco, USA"
    }',
    name='get_weather'
  ),
  type='function'
)]
```

Parse `tool_calls` and run the matching function in your code:

```py theme={null}
import json

tool_call = completion.choices[0].message.tool_calls[0]
args = json.loads(tool_call.function.arguments)

result = get_weather(args["location"])
```

### Return results to the model

For tools that fetch data, send another request with the conversation so far,
including:

* Prior messages
* The assistant message that returned `tool_calls`
* A `tool` message for each result

Set each `tool` message's `tool_call_id` to the `id` of the matching tool call
in that assistant message.

```json theme={null}
"messages": [
  { "role": "user", "content": "What's the weather in San Francisco?" },
  {
    "role": "assistant",
    "tool_calls": [
      {
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"location\": \"San Francisco\"}"
        }
      }
    ]
  },
  {
    "role": "tool",
    "tool_call_id": "call_abc123",
    "content": "{\"temperature\": 62, \"unit\": \"F\"}"
  }
]
```

Each tool message's `tool_call_id` must match an id from the preceding assistant
message. The API returns an HTTP 400 response if a `tool_calls` argument isn't
JSON, a `tool_call_id` doesn't match, or you omit any required tool reply.

### Run the example

This walks through the same `get_weather` example from above as a single,
runnable script.

If you haven't already, [create an API
key](/administration/api-keys#create-an-api-key) and
store it securely.

Then, send a request with the `tools` parameter to a model that supports
function calling:

<Tabs>
  <Tab title="Python">
    Install the OpenAI SDK:

    ```bash theme={null}
    pip install openai
    ```

    Create a program to send a request specifying the available `get_weather()`
    function:

    ```python title="function-calling.py" theme={null}
    import json
    from openai import OpenAI

    def get_weather(location: str) -> str:
        print("Get weather:", location)
        return '{"temperature": 62, "unit": "F"}'

    client = OpenAI(
        base_url="https://api.modular.com/v1",
        api_key="<your-api-key>",  # Load your key
    )

    tools = [{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current temperature for a given location.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "City and country e.g. Bogotá, Colombia"
                    }
                },
                "required": [
                    "location"
                ],
                "additionalProperties": False
            },
            "strict": True
        }
    }]

    messages = [
      {
        "role": "user",
        "content": "What's the weather like in San Francisco today?"
      }
    ]

    completion = client.chat.completions.create(
        model="google/gemma-4-31b-it",
        messages=messages,
        tools=tools
    )

    tool_call = completion.choices[0].message.tool_calls[0]
    args = json.loads(tool_call.function.arguments)

    result = get_weather(args["location"])
    ```

    Run it and the `get_weather()` function should print the argument received:

    ```sh theme={null}
    python function-calling.py
    ```

    ```output theme={null}
    Get weather: San Francisco, USA
    ```
  </Tab>

  <Tab title="curl">
    Use the following `curl` command to send a request specifying the available
    `get_weather()` function:

    ```bash theme={null}
    curl -X POST https://api.modular.com/v1/chat/completions \
        -H "Authorization: Bearer <your-api-key>" \
        -H "Content-Type: application/json" \
        -d '{
            "model": "google/gemma-4-31b-it",
            "messages": [
                {"role": "system", "content": "You are a helpful assistant."},
                {"role": "user", "content": "What is the weather like in Boston today?"}
            ],
            "tools": [
                {
                    "type": "function",
                    "function": {
                        "name": "get_weather",
                        "description": "Get the current weather in a given location",
                        "parameters": {
                            "type": "object",
                            "properties": {
                                "location": {
                                    "type": "string",
                                    "description": "The city and state, e.g. Los Angeles, CA"
                                }
                            },
                            "required": ["location"]
                        }
                    }
                }
            ],
            "tool_choice": "auto"
        }'
    ```

    You should receive a response similar to this:

    ```json theme={null}
    "tool_calls": [
      {
        "id": "call_ac73df14fe184349",
        "type": "function",
        "function": {
            "name": "get_weather",
            "arguments": "{\"location\": \"Boston, MA\"}"
        }
      }
    ]
    ```
  </Tab>
</Tabs>

## Troubleshooting

If your function calling request fails, try the following:

* Confirm that function calling is supported by the model.
* Check if streaming is enabled. Depending on the model, streaming may interfere
  with function calling. If you see incomplete or malformed tool-call output
  while streaming, set `stream: false` for that request.

## Next steps

Find more information in:

* The [chat completions API](/api/inference/create-chat-completion)
  reference documentation.
* OpenAI's guide to
  [Function calling](https://platform.openai.com/docs/guides/function-calling?api-mode=chat\&example=get-weather#handling-function-calls).

<Tip>
  The same OpenAI tool format works with agent frameworks such as
  [AutoGen](https://github.com/microsoft/autogen),
  [CrewAI](https://github.com/crewAIInc/crewAI), and more.
</Tip>