On device models

Apple's WWDC 2025 introduced the concept of on-device models, particularly focusing on large language models (LLMs) that are optimized to run directly on devices like iPhones. These models are designed to handle tasks such as summarization, extraction, and classification, but they are not intended for complex reasoning or tasks requiring extensive world knowledge, which are typically handled by larger server-based models.

The on-device model discussed is a large language model with 3 billion parameters, each quantized to 2 bits, making it significantly smaller than server-scale models. This allows for privacy benefits, as data does not need to be sent to a server, and the model can function offline. The Foundation Models framework, introduced at WWDC, provides developers with tools to integrate these models into their apps, leveraging Apple's machine learning APIs and tools.

For more detailed information on the on-device models, you can refer to the session Meet the Foundation Models framework (02:57) and Explore prompt design & safety for on-device foundation models (02:00).