What's the easiest way to call the on-device LLM?
Asked on 06/11/2025
32 searches
The easiest way to call the on-device large language model (LLM) using Apple's Foundation Models framework is by following a simple process that involves just a few lines of code. Here's a brief overview:
-
Import the Framework: Start by importing the Foundation Models framework into your project.
-
Create a Session: Set up a session with the on-device LLM. This session will handle the interaction with the model.
-
Send Your Prompt: Send your prompt to the model. The prompt can be a string variable, such as "generate a bedtime story about a fox," and it can be written in any language supported by Apple Intelligence.
This process is highlighted in the session Discover machine learning & AI frameworks on Apple platforms, where it is mentioned that prompting the model is as easy as three lines of code. The framework is designed to work offline, ensuring user data privacy without the need for API keys or accounts.
Explore prompt design & safety for on-device foundation models
Design generative AI experiences that leverage the strengths of the Foundation Models framework. We’ll start by showing how to design prompts for the on-device large language model at the core of Apple Intelligence. Then, we’ll introduce key ideas around AI safety, and offer concrete strategies to make your generative AI features safe, reliable, and delightful.
Discover machine learning & AI frameworks on Apple platforms
Tour the latest updates to machine learning and AI frameworks available on Apple platforms. Whether you are an app developer ready to tap into Apple Intelligence, an ML engineer optimizing models for on-device deployment, or an AI enthusiast exploring the frontier of what is possible, we’ll offer guidance to help select the right tools for your needs.
Meet the Foundation Models framework
Learn how to tap into the on-device large language model behind Apple Intelligence! This high-level overview covers everything from guided generation for generating Swift data structures and streaming for responsive experiences, to tool calling for integrating data sources and sessions for context management. This session has no prerequisites.