What's new in Vision?

What's New in Vision?

Apple introduced several exciting updates to the Vision framework and VisionOS at WWDC 2024. Here are the key highlights:

1. Vision Framework Enhancements:

   • New Swift API: The Vision framework now includes a new API with streamlined syntax designed for Swift, making it easier to integrate computer vision capabilities into your apps. This API supports Swift concurrency and Swift 6, enabling more performant applications.
   • Hand Pose Detection: Vision introduces hand pose detection, allowing for new interactive experiences without touching a screen.
   • Body Pose Requests: Enhanced body pose tracking for health and fitness applications.
   • Text Recognition: The framework can recognize text in 18 different languages, including Korean, Swedish, and Chinese.
   • Face and Landmark Detection: Improved capabilities for detecting faces and facial landmarks like eyes, nose, and mouth.

   For more details, you can check out the session Discover Swift enhancements in the Vision framework.

2. VisionOS Updates:

   • Scene Understanding: Enhanced fidelity in scene understanding capabilities, including the detection of planes in all orientations and the introduction of room anchors that consider the user's surroundings on a per-room basis.
   • Object Tracking API: A new API that allows attaching content to individual objects found around the user, enabling new dimensions of interactivity.
   • Hand Positioning: Developers can now decide if they want the user's hands to appear in front of or behind the content, offering more creative control over app experiences.
   • Spatial Computing: VisionOS 2 allows for the creation of richer spatial experiences, taking full advantage of depth and space. This includes the ability to resize volumes and use SwiftUI scene modifiers for better control over 3D objects and scenes.

   For more information, you can refer to the session Platforms State of the Union.

3. HealthKit in VisionOS:

   • Immersive Spaces: VisionOS allows users to enter immersive spaces where other apps are hidden, and the surrounding pass-through dims to provide a sense of separation from the workspace.
   • Guest User Feature: VisionOS includes a guest user feature that allows others to set up and try out Apple Vision Pro while preserving the owner's data and privacy.

   For more insights, you can watch the session Get started with HealthKit in visionOS.

Relevant Sessions

1. Platforms State of the Union
2. Discover Swift enhancements in the Vision framework
3. Get started with HealthKit in visionOS
4. Create enhanced spatial computing experiences with ARKit
5. Explore machine learning on Apple platforms

These updates make it easier for developers to create more interactive and immersive experiences using Vision and VisionOS.