How to extract text from a screen
Asked on 06/13/2025
1 search
To extract text from a screen using Apple's technologies, you can utilize the Vision framework, specifically the RecognizeDocuments Request API. This API allows you to detect and extract lines of text from an image, as well as recognize structural elements like tables and lists. It can also identify important information such as email addresses, phone numbers, and URLs. This makes it easier to parse documents with fewer lines of code.
For more detailed information, you can refer to the session titled "Read documents using the Vision framework" from WWDC 2025. The session covers how to use the Vision framework to extract text and structural elements from documents. You can start learning about reading documents at the chapter marker Reading documents.
If you need to extract text from a structured document, such as a table, the API provides the ability to detect the table's structure and extract text from each cell, which can then be used to create a list of contacts or other structured data.
Read documents using the Vision framework
Learn about the latest advancements in the Vision framework. We’ll introduce RecognizeDocumentsRequest, and how you can use it to read lines of text and group them into paragraphs, read tables, etc. And we’ll also dive into camera lens smudge detection, and how to identify potentially smudged images in photo libraries or your own camera capture pipeline.
Explore new advances in App Intents
Explore all the new enhancements available in the App Intents framework in this year’s releases. Learn about developer quality-of-life improvements like deferred properties, new capabilities like interactive app intents snippets, entity view annotations, how to integrate Visual Intelligence, and much more. We’ll take you through how App Intents is more expressive than ever, while becoming even easier and smoother to adopt. We’ll also share exciting new clients of App Intents this year like Spotlight and Visual Intelligence, and learn to write app intents that work great in those contexts.
Principles of inclusive app design
Discover how understanding disability can help you create better apps for everyone. Find out how to make your apps more inclusive by supporting multiple types of interaction, providing customization, and adopting accessibility APIs.