How does the updated Vision Pro SDK work for creating AR experiences?

The updated Vision Pro SDK offers several new features and improvements for creating AR experiences. Here are some key highlights:

1. Hand Interaction: You can now decide if you want the user's hands to appear in front of or behind the content, providing more creative control over the app experience. This is detailed in the Platforms State of the Union session.

2. Enhanced Scene Understanding: The SDK has significantly extended the fidelity of scene understanding capabilities. Planes can now be detected in all orientations, allowing for anchoring objects on various surfaces. Additionally, room anchors consider the user's surroundings on a per-room basis, and you can detect a user's movement across rooms. This is also covered in the Platforms State of the Union session.

3. Object Tracking API: A new object tracking API for visionOS allows you to attach content to individual objects found around the user. This can be used to attach virtual content like instructions to physical objects, adding new dimensions of interactivity. More details can be found in the Explore object tracking for visionOS session.

4. ARKit Enhancements: The ARKit framework has received several updates, including improvements in room tracking, plane detection, and hand tracking. These updates help customize experiences based on the room and enhance interaction with the surroundings. This is discussed in the Create enhanced spatial computing experiences with ARKit session.

5. Integration with Existing Frameworks: VisionOS is built on the foundation of macOS, iOS, and iPadOS, meaning many tools and frameworks are common across these platforms. SwiftUI, RealityKit, and ARKit are core to developing spatial apps for visionOS. If your app already uses these frameworks, you are well on your way to creating a great spatial computing app. This is highlighted in the Platforms State of the Union session.

For more detailed information, you can refer to the following sessions:

• Platforms State of the Union
• Create enhanced spatial computing experiences with ARKit
• Explore object tracking for visionOS

These sessions provide comprehensive insights into the new capabilities and how to leverage them for creating immersive AR experiences.