Help me understand tool calling in the models framework

Asked on 06/12/2025

1 search

Tool calling in the Foundation Models framework allows the model to execute code you define in your app, providing it with the ability to access external data or perform actions beyond text generation. This feature is particularly useful for integrating live or personal data, such as weather or calendar events, into the model's responses.

Here's a high-level overview of how tool calling works:

  1. Defining a Tool: You start by defining a tool that conforms to the tool protocol. This involves specifying a name and a natural language description, which helps the model understand when to call your tool. The tool also includes a call method, which is invoked by the model when needed. The arguments for this method must be generable types to ensure valid input.

  2. Session Initialization: Tools must be attached to a session at initialization. Once a session is created with tools, the model can autonomously decide when to call these tools based on the context of the user's request.

  3. Tool Invocation: When the model determines that calling a tool can enhance its response, it generates the necessary input arguments and invokes the tool's call method. The tool can perform any action, such as fetching data or executing a function, and its output is incorporated into the model's response.

  4. Parallel and Stateful Execution: Tools can be called multiple times within a single request and can be executed in parallel. They can also maintain state across calls, allowing for more complex interactions.

For a detailed explanation of tool calling, you can refer to the Deep dive into the Foundation Models framework (18:10) session.