Can I run llm on phone
Asked on 07/30/2024
1 search
Yes, you can run large language models (LLMs) on your phone. Apple provides tools and frameworks to help you import and run AI models, including large language models, on their devices. You can start with any PyTorch model, convert it into the Core ML format using Core ML tools, and then run it within your app using the Core ML framework. Core ML optimizes hardware-accelerated execution across the CPU, GPU, and neural engine, making it possible to run a wide array of models, including Whisper, Stable Diffusion, and Mistral, on iOS devices.
For more details, you can refer to the Platforms State of the Union session at WWDC 2024.
Deploy machine learning and AI models on-device with Core ML
Learn new ways to optimize speed and memory performance when you convert and run machine learning and AI models through Core ML. We’ll cover new options for model representations, performance insights, execution, and model stitching which can be used together to create compelling and private on-device experiences.
Train your machine learning and AI models on Apple GPUs
Learn how to train your models on Apple Silicon with Metal for PyTorch, JAX and TensorFlow. Take advantage of new attention operations and quantization support for improved transformer model performance on your devices.
Explore machine learning on Apple platforms
Get started with an overview of machine learning frameworks on Apple platforms. Whether you’re implementing your first ML model, or an ML expert, we’ll offer guidance to help you select the right framework for your app’s needs.