tell me about spatial video

Spatial video is a type of stereoscopic video that enhances the sense of depth and immersion, making it particularly compelling for devices like Vision Pro and iPhone 15 Pro. Here are some key points about spatial video from the session "Build compelling spatial photo and video experiences":

1. Types of Stereoscopic Experiences:

   • 3D Video: Similar to 3D movies, it renders on a flat screen but with depth, making moments in the video appear to pop out or recede.
   • Spatial Video: Captured on Vision Pro and iPhone 15 Pro, it renders through a window with a faint glow around the edges and custom playback controls, enhancing the sense of depth and softening common sources of discomfort. It's great for point-and-shoot formats (Build compelling spatial photo and video experiences).

2. Recording Spatial Video:

   • On the iPhone 15 Pro, the wide and ultra-wide cameras are placed side by side, enabling spatial video capture. This setup mimics the horizontal baseline of human eyes, allowing for accurate depth perception in the recorded video (Build compelling spatial photo and video experiences).
   • The API for recording spatial video is integrated into existing frameworks like AV Foundation, making it easy to add spatial media to apps with just a few lines of code (Build compelling spatial photo and video experiences).

3. Playback and Presentation:

   • Spatial videos can be played back with custom controls and a windowed presentation that allows multitasking with other apps. For full immersion, the video can expand into its native experience, wrapping around the viewer and disabling passthrough (Build compelling spatial photo and video experiences).
   • The Photos app in visionOS displays spatial photos and videos with a feathered border to blend the content with the rest of the scene, enhancing the immersive experience (Optimize for the spatial web).

4. APIs and Tools:

   • Spatial media APIs are integrated into existing Apple frameworks like AV Foundation, Photokit, Quicklook, and Webkit. This allows developers to add spatial media to their apps easily (Build compelling spatial photo and video experiences).
   • For custom spatial media creation, Apple provides sample code projects and articles to help developers understand how to read and write spatial video files with accurate metadata (Build compelling spatial photo and video experiences).

For a deeper dive into spatial video and the new APIs, you can watch the session "Build compelling spatial photo and video experiences".