What new media features are coming?
Asked on 08/01/2024
1 search
Apple introduced several new media features at WWDC 2024. Here are some of the highlights:
-
Spatial Media:
- Spatial Video and Photos: New APIs in AVFoundation and RealityKit allow for the creation of spatial videos and photos, enhancing media with stereoscopy and real-time 3D content (Build compelling spatial photo and video experiences).
- Multiview Video Playback: VisionOS now supports multiview video playback, enabling simultaneous video watching with features like spatial audio and 3D video (Explore multiview video playback in visionOS).
-
Media Performance Metrics:
- New Metrics in iOS 18: AVFoundation introduces new ways to gather media performance metrics, represented as events, to help triage issues like playback delays and stalling (Discover media performance metrics in AVFoundation).
-
HLS Interstitials:
- Integrated Timeline API: A new AVFoundation API called Integrated Timeline helps manage the timing and sequence of playback content, enhancing user navigation through interstitials (Enhance ad experiences with HLS interstitials).
-
Immersive Media Viewing:
- Custom Environments: Enhancements to media playback experiences in custom environments, including immersive playback and environment pickers (Enhance the immersion of media viewing in custom environments).
-
RealityKit Enhancements:
- Low-Level Mesh and Texture Access: New APIs for low-level mesh and texture access, enabling dynamic models and textures (Discover RealityKit APIs for iOS, macOS and visionOS).
- Animation System: New capabilities in the animation system, including creating animation timelines in Reality Composer Pro (Discover RealityKit APIs for iOS, macOS and visionOS).
These features collectively enhance the media experience across Apple's platforms, particularly with the integration of spatial computing and improved performance metrics.
Explore multiview video playback in visionOS
Learn how AVExperienceController can enable playback of multiple videos on Apple Vision Pro. Review best practices for adoption and explore great use cases, like viewing a sports broadcast from different angles or watching multiple games simultaneously. And discover how to design a compelling and intuitive multiview experience in your app.
Optimize for the spatial web
Discover how to make the most of visionOS capabilities on the web. Explore recent updates like improvements to selection highlighting, and the ability to present spatial photos and panorama images in fullscreen. Learn to take advantage of existing web standards for dictation and text-to-speech with WebSpeech, spatial soundscapes with WebAudio, and immersive experiences with WebXR.
Enhance ad experiences with HLS interstitials
Explore how HLS Interstitials can help you seamlessly insert advertisements into your HLS content. We’ll also show you how to use integrated timeline to tune your UI experience and build SharePlay for interstitials.