What's new on AR APIs from ios15?

What's New in AR APIs from iOS 15?

ARKit Enhancements

1. Room Tracking:

   • ARKit now includes a new room tracking feature, which helps customize experiences based on the room you're in. This can be particularly useful for applications that need to understand the spatial layout of a room to provide more immersive experiences.
   • Create enhanced spatial computing experiences with ARKit

2. Plane Detection:

   • An update to plane detection now includes a new slanted plane alignment for detecting angled surfaces. This is in addition to the existing horizontal and vertical plane detection capabilities.
   • Create enhanced spatial computing experiences with ARKit

3. Object Tracking:

   • ARKit can now track real-world objects that are statically placed in your environment. This allows for the position and orientation of these items to be used as anchors for virtual content.
   • Create enhanced spatial computing experiences with ARKit

4. World Tracking:

   • Improvements have been made to ARKit's world tracking to make it more robust under various lighting conditions.
   • Create enhanced spatial computing experiences with ARKit

5. Hand Tracking:

   • New advancements in hand tracking include display-aligned hand anchor updates for gesture detection and drawing smooth strokes. Hand predictions are also available for content that needs to be attached to the hand.
   • Create enhanced spatial computing experiences with ARKit

RealityKit Enhancements

1. Rendering and Textures:

   • New APIs for low-level mesh and textures work with Metal compute shaders to offer improved control over how your apps look, enabling fully dynamic models and textures in every frame.
   • Platforms State of the Union

2. Character Animation:

   • RealityKit 4 introduces new APIs and tools like blend shapes, inverse kinematics, and animation timelines, which bring expanded character animation capabilities.
   • Platforms State of the Union

3. Introspection in Xcode:

   • Xcode's View debugging now supports introspecting your 3D scene content, allowing you to investigate your scene's object hierarchy and inspect every element's properties.
   • Platforms State of the Union

Summary

These updates to ARKit and RealityKit provide developers with more tools and capabilities to create immersive and interactive AR experiences. From improved scene understanding and object tracking to advanced rendering and animation capabilities, these enhancements are designed to help developers build more engaging and realistic AR applications.