What information do I need to add to make Spatial Video

To add Spatial Video to your app, you need to follow these steps:

1. Understand Spatial Media:

   • Spatial video is a stereo MVC video with extra spatial metadata.
   • Spatial photos are stereo HEIC with two images in a stereo group along with some spatial metadata.
   • Spatial metadata includes projection, baseline, field of view, and disparity adjustment.

2. APIs and Frameworks:

   • Use existing Apple frameworks like AV Foundation, Photokit, Quicklook, and Webkit.
   • No new framework is required for spatial media.

3. Recording Spatial Video:

   • Ensure you are using an iPhone 15 Pro, as spatial video capture is only supported on this device.
   • Use the built-in dual wide camera for spatial video, which requires the wide and ultra-wide cameras to stream simultaneously.
   • Enable spatial video capture by setting isSpatialVideoCaptureEnabled to true.

4. Code Implementation:

   • Create an AV capture session.
   • Set the AV capture device to the built-in dual wide camera.
   • Pick a video format that supports spatial video.
   • Enable spatial video recording on the output.

5. Improving Quality:

   • Set the preferred video stabilization mode to cinematic extended enhanced for smoother videos.

6. Playback and Detection:

   • Detect spatial assets using APIs like PhotoPicker, Photokit, and AVAsset.
   • Use AVPlayerViewController for 3D video presentation or Preview Application for full spatial presentation.

7. Custom Spatial Media:

   • For custom spatial media, understand the file formats: stereo MVC for video and stereo HEIC for photos.
   • Ensure accurate metadata and perfectly aligned images for a great experience in Vision Pro.

For a detailed walkthrough, you can refer to the session Build compelling spatial photo and video experiences starting at the "Tour of the new APIs" chapter.

Relevant Session

• Build compelling spatial photo and video experiences