What is Foundation Models Framework?

The Foundation Models Framework is a new framework introduced by Apple that provides access to an on-device large language model through a convenient Swift API. It is available on macOS, iOS, iPadOS, and VisionOS. This framework allows developers to enhance existing app features or create new ones by leveraging the capabilities of the language model, such as generating content, summarizing text, and analyzing user input.

The framework is designed to run entirely on-device, ensuring that all data remains private and can operate offline. It is built into the operating system, so it does not increase the app size. The Foundation Models Framework is optimized for performance and efficiency, using state-of-the-art quantization techniques and speculative decoding. It also includes safety features, such as guardrails, to ensure safe app experiences.

For more detailed information, you can refer to the session Meet the Foundation Models framework.