What are the capabilities of Foundation Models?

The Foundation Models framework introduced by Apple at WWDC provides a range of capabilities for on-device large language models. Here are some of the key capabilities:

1. Text Extraction and Summarization: The framework allows for tasks such as text extraction and summarization, which can be used to enhance app features like generating content or summarizing user input.

2. Tool Calling: The model can autonomously call functions defined in your app, allowing it to perform actions like fetching up-to-date content or creating entries in your app. This is useful for integrating live or personal data, such as weather or calendar events, into the model's responses.

3. Structured Data Output: The framework supports generating structured data, which can be directly used in apps. This is achieved through features like guided generation, which allows developers to define types and properties that the model can generate.

4. Privacy and Offline Capabilities: All data processing is done on-device, ensuring user privacy and allowing the model to function offline without increasing app size.

5. Streaming Output: The API supports streaming output, enabling responses to be displayed incrementally, which can enhance user experience by reducing perceived latency.

6. Guardrails and Safety: The framework includes built-in guardrails to prevent harmful content generation, and developers can add their own safety rules for specific use cases.

7. Integration with Swift and Xcode: The framework is designed to be easily integrated into apps using Swift, with support for features like concurrency and tool calling.

For more detailed information, you can explore sessions like Meet the Foundation Models framework and Deep dive into the Foundation Models framework.