Can I run llm no my iPhone?
Generated on 9/6/2024
1 search
Yes, you can run large language models (LLMs) on your iPhone. Apple provides tools and frameworks to facilitate this process. You can start with a PyTorch model and convert it into the Core ML format using Core ML tools. These tools offer optimization techniques such as quantization and efficient key-value caching for LLMs. Once converted, you can run your model within your app using the Core ML framework, which optimizes execution across the CPU, GPU, and neural engine of Apple devices. This allows you to run a wide array of models, including large language models, on iOS, iPadOS, and macOS devices.
For more details, you can refer to the Platforms State of the Union (16:37) session.
Platforms State of the Union
Discover the newest advancements on Apple platforms.
Explore machine learning on Apple platforms
Get started with an overview of machine learning frameworks on Apple platforms. Whether you’re implementing your first ML model, or an ML expert, we’ll offer guidance to help you select the right framework for your app’s needs.
Bring your machine learning and AI models to Apple silicon
Learn how to optimize your machine learning and AI models to leverage the power of Apple silicon. Review model conversion workflows to prepare your models for on-device deployment. Understand model compression techniques that are compatible with Apple silicon, and at what stages in your model deployment workflow you can apply them. We’ll also explore the tradeoffs between storage size, latency, power usage and accuracy.