# Function calling

Function calling is like giving an LLM a toolbox with specific tools it can use to help you. When you ask the model to do something that requires one of these tools, it can "call" or use that tool to get the job done.

<img src="https://bentoml.com/llm/assets/images/function-calling-diagram-f6e914ea766607b23681d2dec940c115.png" alt="" height="220.4453125" width="802">

Here is a specific example:

* **You ask**: "What's the current price of Apple stock?"
* **LLM thinks**: "I need current stock data, so I'll use my stock price function"
* **LLM calls**: `get_stock_price("AAPL")`
* **Function returns**: "$195.25"
* **LLM responds**: "The current price of Apple stock is $195.25"

At a technical level, the LLM still predicts the next token, just like any other transformer-based models. The prompt includes well-defined function signatures and descriptions, guiding the model to produce outputs that match those formats. This allows you to build workflows like:

* Calling APIs from a user’s input
* Triggering actions (e.g., send email, retrieve weather)
* Passing outputs back into the model for multi-turn conversations

### FAQs <a href="#faqs" id="faqs"></a>

#### What is the difference between function calling and structured outputs? <a href="#what-is-the-difference-between-function-calling-and-structured-outputs" id="what-is-the-difference-between-function-calling-and-structured-outputs"></a>

These two ideas get mixed up a lot. Here’s the simple difference:

* Structured outputs: You decide the format that the model uses in its responses. For example, you can force the model to output JSON, a list, or a specific object.
* Function calling: You tell the model when to take an action. Function calling tools are often defined in a structured way (e.g., JSON).

Both can be used together.

#### Do all LLMs support function calling? <a href="#do-all-llms-support-function-calling" id="do-all-llms-support-function-calling"></a>

Not all, but many modern models do. Open-source models like Llama, Qwen, and DeepSeek usually work well when served through vLLM or SGLang.

#### Is function calling the same as tools or agents? <a href="#is-function-calling-the-same-as-tools-or-agents" id="is-function-calling-the-same-as-tools-or-agents"></a>

Tools are the actions. Function calling is how the model requests those actions. Agents are systems that can use function calling in a loop with reasoning; they can plan, call multiple tools in sequence, evaluate results, and adjust their approach based on what they learn.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://infronai.gitbook.io/docs/llm-inference-handbook/llm-inference-advanced/unified-api-compatibility/function-calling.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.