I heard foundation models are a big deal. Can you give me a quick summary of their capabilites?

Asked on 06/15/2025

1 search

Foundation models, as presented at Apple's WWDC, are a significant advancement in on-device machine learning and AI capabilities. Here's a quick summary of their capabilities:

  1. On-Device Processing: The Foundation Models framework provides access to a large language model that runs entirely on-device, ensuring user data privacy and allowing offline functionality. This model is integrated into macOS, iOS, iPadOS, and VisionOS.

  2. Core Capabilities: The framework is optimized for tasks such as text extraction, summarization, content generation, and user input analysis. It can be used to enhance existing app features or create new ones, like generating personalized search suggestions or dialogue for game characters.

  3. Tool Calling: This feature allows the model to perform actions beyond text generation by accessing live or personal data, such as weather or calendar events, and even executing code within your app. This extends the model's capabilities significantly.

  4. Guided Generation: This allows developers to get structured output from the model, which is useful for integrating AI-generated content into app interfaces. It uses macros like Generable and Guide to define the types of output expected.

  5. Streaming and Stateful Sessions: The framework supports streaming responses and stateful sessions, which can enhance user experience by providing incremental outputs and maintaining context over multiple interactions.

  6. Integration and Safety: The framework is tightly integrated with Swift and Xcode, and includes safety guardrails to ensure responsible AI use. Developers can also add their own safety rules tailored to specific use cases.

For more detailed information, you can explore sessions like Meet the Foundation Models framework and Deep dive into the Foundation Models framework.