Audio & Video
Learn about improvements to AVFoundation, MusicKit, ShazamKit, and more.
Featured
Build device-to-device interactions with Network Framework
Learn how you can create integrated content experiences across multiple devices.
Watch nowSessions
-
Bring your driver to iPad with DriverKit
8:00 a.m.Discover how you can easily connect Thunderbolt and USB accessories to iPad with DriverKit. We’ll show you how to convert your existing Mac drivers without any code changes, learn how to add real-time audio support with AudioDriverKit, and provide best practices and tips for developing drivers for iPad.
-
Create a more responsive media app
8:00 a.m.Discover how you can use AVFoundation to keep people focused on your media app’s content — not your loading spinner. We’ll show you how to support a responsive and fluid interface in your app, all while you create rich audiovisual compositions, load audiovisual assets, and prepare media thumbnails. Find out how you can perform these tasks on your app’s main thread while I/O processes in parallel, learn how to get top-notch playback performance when loading data from custom storage, and more. To get the most out of this session, we recommend first watching "What's new in AVFoundation” from WWDC21.
-
Deliver reliable streams with HLS Content Steering
8:00 a.m.HLS Content Steering dynamically steers clients to different servers based on load and redundancy. We'll take you through the latest updates to this framework and explore how you can introduce dynamically spawned CDNs to existing HLS clients with Pathway Cloning. We'll also share how you can achieve global traffic steering with bucket-based Steering Server rules and more.
-
What's new in SharePlay
8:00 a.m.Join us as we share the latest updates to SharePlay. We’ll show you how you can start SharePlay sessions right from your app, take you through improvements to APIs to create richer experiences, and check out enhancements to GroupSessionMessenger. We’ll also explore best practices for adding SharePlay to your app.
Labs
-
Core Haptics lab
Tuesday @ 9:00 - 11:00 a.m.Core Haptics lets you design fully customized haptic patterns with synchronized audio. Request an appointment with an Apple engineer for guidance and conversation about your own apps and haptic patterns. Bring your code, your AHAP files, and your questions.
-
AirPlay lab
Tuesday @ 2:00 - 5:00 p.m.Have a question about AirPlay? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
ShazamKit lab
Tuesday @ 3:00 - 6:00 p.m.Request an appointment with an Apple engineer for guidance and conversation about exact audio matching with ShazamKit.
-
AVFoundation lab
Tuesday @ 3:00 - 5:00 p.m.Have a question about any aspect of AVFoundation? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
Digital Lounges
-
Meet the Presenter: Add Apple frameworks to your Unity game projects
Tuesday @ 11:00 - 12:00 p.m.Meet the presenter of “Plug-in and play: Add Apple frameworks to your Unity game projects” and join a text-based watch party for the session, followed by a short Q&A. The watch party begins 5 minutes after the start of this activity — so don’t be late!
-
Q&A: Apple Music
Tuesday @ 1:00 - 3:00 p.m.Ask Apple engineers about Apple Music during this 2 hour text-based Q&A. Stop in to request guidance on a code-level question, ask for clarifications, or learn from others in a group setting.
-
Q&A: SharePlay
Tuesday @ 3:00 - 5:00 p.m.Ask Apple engineers about SharePlay during this 2 hour text-based Q&A. Stop in to request guidance on a code-level question, ask for clarifications, or learn from others in a group setting.
Sessions
-
Display ads and interstitials in SharePlay
8:00 a.m.Find out how you can deliver a coordinated playback experience in SharePlay when your app delivers different ad schedules to each participant. We'll explore how to build playback experiences with stitched-in ads and scheduled HLS interstitials, and share tips and best practices.
-
Explore EDR on iOS
8:00 a.m.EDR is Apple's High Dynamic Range representation and rendering pipeline. Explore how you can render HDR content using EDR in your app and unleash the dynamic range capabilities of HDR displays on iPhone and iPad. We'll show how you can take advantage of the native EDR APIs on iOS, provide best practices to help you decide when HDR is appropriate, and share tips for tone-mapping and HDR content rendering. We'll also introduce you to Reference Mode and highlight how it provides a reference response to enable color-critical workflows such as color grading, editing, and content review.
-
Meet Apple Music API and MusicKit
8:00 a.m.Explore the Apple Music API and MusicKit client frameworks. Learn how you can integrate Apple Music features into your web service or Android app, make requests to the Apple Music API to discover and fetch content, and personalize the experience for subscribers.
-
Meet ScreenCaptureKit
8:00 a.m.Learn how ScreenCaptureKit can deliver high-performance screen capture for your macOS screen sharing applications, video conferencing apps, game streaming services, and more. We'll explore the building blocks of this API, learn how to configure streams to capture on-screen video and audio content, and share tips for integrating it into your existing apps.
-
Support multiple users in tvOS apps
8:00 a.m.Discover how you can create personalized, individual experiences in your tvOS app. We’ll show you how you can offer a single checkbox to store profile data, game save states, and more, providing each person with the same level of data separation they'd have on a personal device like iPhone. We’ll also explore how the new user-independent keychain can help you maintain your existing sign on experience for multiple people in the same household.
-
Take ScreenCaptureKit to the next level
8:00 a.m.Discover how you can support complex screen capture experiences for people using your app with ScreenCaptureKit. We’ll explore many of the advanced options you can incorporate including fine tuning content filters, frame metadata interpretation, window pickers, and more. We’ll also show you how you can configure your stream for optimal performance.
-
What’s new in AVQT
8:00 a.m.Discover the latest updates and improvements to the Advanced Video Quality Tool (AVQT). We'll take you through the interactive reports feature and help you learn how to identify video quality-related issues. We'll also explore extended support for raw formats, show you how to evaluate specific scenes within a video, and explore how you can use AVQT for Linux to analyze videos on Linux servers and online in the cloud.
Labs
-
ShazamKit lab
Wednesday @ 9:00 - 12:00 p.m.Request an appointment with an Apple engineer for guidance and conversation about exact audio matching with ShazamKit.
-
Game Center lab
Wednesday @ 9:00 - 11:00 a.m.Request an appointment with an Apple engineer for guidance and conversation about your app, the GameKit API, best practices, and more.
-
AVAudioSession lab
Wednesday @ 9:00 - 11:00 a.m.Have a question about AVAudioSession? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
MusicKit lab
Wednesday @ 9:30 - 12:30 p.m.Have a question about MusicKit? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
Core Audio lab
Wednesday @ 10:00 - 12:00 p.m.Request an appointment with an Apple engineer for guidance and conversation about Audio Units, Workgroups, HAL, Toolbox APIs, CoreMIDI, AVAudioEngine, AVF APIs Catalyst apps, Spatial Audio, SoundAnalysis, or AUVoiceIO.
-
HTTP Live Streaming lab
Wednesday @ 1:00 - 3:00 p.m.Have a question about live or on demand HLS video streaming, Low-Latency HLS, Offline HLS, spatial audio, dynamic ad insertion, or stream authoring recommendations? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
Apple Music API and MusicKit on the Web lab
Wednesday @ 2:00 - 5:00 p.m.Have a question about Apple Music API or MusicKit Web Apps? Request an appointment with an Apple engineer for guidance and conversation about integrating with Apple Music API or MusicKit JS to your Apps, best practices, and more.
-
PHASE Audio Engine lab
Wednesday @ 2:00 - 4:00 p.m.Request an appointment with an Apple engineer for guidance and conversation about the PHASE spatialization audio engine and best practices.
-
Metal lab
Wednesday @ 3:00 - 5:00 p.m.Request an appointment with an Apple engineer for guidance and conversation about your app, the latest Metal APIs, best practices, and more.
-
Perceptual Video Quality lab
Wednesday @ 3:00 - 5:00 p.m.Request an appointment with an Apple engineer for guidance and conversation about perceptual video quality and AVQT.
-
SharePlay lab
Wednesday @ 5:00 - 7:00 p.m.Have a question about SharePlay and Group Activities? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
Digital Lounges
-
Study Hall: Create reactive soundscapes
Wednesday @ 9:00 - 6:00 p.m.Bring on the noise: It's time for a sound design challenge! Today, we're inviting you to experiment with creative ways to manipulate sound on iPhone and iPad using its myriad sensors, inputs, and variable states. Explore over 70 audio files from Apple sound designers and create a sonic experience of your very own! Visit the Design Study Hall to collaborate on the “Create reactive soundscapes” coding & design challenge. Ask questions, connect with other developers and designers, and share your creations.
-
Q&A: Core Image
Wednesday @ 9:00 - 10:00 a.m.Ask Apple engineers about Core Image during this 2 hour text-based Q&A. Stop in to request guidance on a code-level question, ask for clarifications, or learn from others in a group setting.
-
Meet the Presenter: Meet ScreenCaptureKit
Wednesday @ 11:00 - 12:00 p.m.Meet the presenter of “Meet ScreenCaptureKit” and join a text-based watch party for the session, followed by a short Q&A. The watch party begins 5 minutes after the start of this activity — so don’t be late!
-
Q&A: AVFoundation
Wednesday @ 1:00 - 3:00 p.m.Ask Apple engineers about AVFoundation during this 2 hour text-based Q&A. Stop in to request guidance on a code-level question, ask for clarifications, or learn from others in a group setting.
-
Q&A: Coordinated media with SharePlay
Wednesday @ 3:00 - 5:00 p.m.Ask Apple engineers about coordinated media playback with SharePlay during this 2 hour text-based Q&A. Stop in to request guidance on a code-level question, ask for clarifications, or learn from others in a group setting.
-
Meet the Presenter: Take ScreenCaptureKit to the next level
Wednesday @ 3:00 - 4:00 p.m.Meet the presenters of “Take ScreenCaptureKit to the next level” and join a text-based watch party for the session, followed by a short Q&A. The watch party begins 5 minutes after the start of this activity — so don’t be late!
Sessions
-
Create a great video playback experience
8:00 a.m.Find out how you can use the latest iOS and iPadOS system media players to build amazing media apps. We’ll share how we designed the updated player and give you best practices and tips to help you design media experiences of your own. We’ll also explore Live Text for video and show you how to integrate interstitials and playback speed controls into your apps.
-
Display EDR content with Core Image, Metal, and SwiftUI
8:00 a.m.Discover how you can add support for rendering in Extended Dynamic Range (EDR) from a Core Image based multi-platform SwiftUI application. We'll outline best practices for displaying CIImages to a MTKView using ViewRepresentable. We'll also share the simple steps to enable EDR rendering and explore some of the over 150 built-in CIFilters that support EDR.
-
Display HDR video in EDR with AVFoundation and Metal
8:00 a.m.Learn how you can take advantage of AVFoundation and Metal to build an efficient EDR pipeline. Follow along as we demonstrate how you can use AVPlayer to display HDR video as EDR, add playback into an app view, render it with Metal, and use Core Image or custom Metal shaders to add video effects such as keying or color management. Whether you develop games or pro apps, we'll help you decide which frameworks to use and share best practices for selecting transports, colorspaces, and pixelbuffer formats.
-
Make a great SharePlay experience
8:00 a.m.Discover how you can design and build a great SharePlay experience. We'll show you how to identify activities in your app that can help bring everyone together — even while they're apart. Learn best practices to help you design for group activities, create intuitive interactions, and build support for different types of experiences.
-
What’s new in HLS Interstitials
8:00 a.m.HLS Interstitials can help you create seamless transitions in video content between advertisements, other interstitials, and your HLS streams. Learn how you can optimize your ad inventory, fine-tune interstitial presentation with SNAP-IN/OUT when using HLS, and more.
Labs
-
Metal lab
Thursday @ 8:00 - 10:00 a.m.Request an appointment with an Apple engineer for guidance and conversation about your app, the latest Metal APIs, best practices, and more.
-
HDR and EDR lab
Thursday @ 9:00 - 11:00 a.m.EDR is Apple’s High Dynamic Range representation and rendering pipeline. Request an appointment with an Apple engineer for guidance and conversation around rendering HDR video content using EDR in your app, best practices, and more.
-
tvOS lab
Thursday @ 9:00 - 12:00 p.m.Have a question about tvOS? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
Core Audio lab
Thursday @ 10:00 - 12:00 p.m.Request an appointment with an Apple engineer for guidance and conversation about Audio Units, Workgroups, HAL, Toolbox APIs, CoreMIDI, AVAudioEngine, AVF APIs Catalyst apps, Spatial Audio, SoundAnalysis, or AUVoiceIO.
-
AVAudioSession lab
Thursday @ 11:00 - 1:00 p.m.Have a question about AVAudioSession? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
MusicKit lab
Thursday @ 1:00 - 4:00 p.m.Have a question about MusicKit? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
Core Haptics lab
Thursday @ 2:00 - 4:00 p.m.Core Haptics lets you design fully customized haptic patterns with synchronized audio. Request an appointment with an Apple engineer for guidance and conversation about your own apps and haptic patterns. Bring your code, your AHAP files, and your questions.
-
ShazamKit lab
Thursday @ 3:00 - 6:00 p.m.Request an appointment with an Apple engineer for guidance and conversation about exact audio matching with ShazamKit.
-
AVKit lab
Thursday @ 3:00 - 5:00 p.m.Have a question about AVKit? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
AVFoundation playback lab
Thursday @ 3:00 - 5:00 p.m.Have a question related to playback in AVFoundation? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
Digital Lounges
-
Meet the Presenter: Reach new players with Game Center dashboard
Thursday @ 11:00 - 12:00 p.m.Meet the presenter of “Reach new players with Game Center dashboard” and join a text-based watch party for the session, followed by a short Q&A. The watch party begins 5 minutes after the start of this activity — so don’t be late!
-
Meet the Presenter: Create a great video playback experience
Thursday @ 3:00 - 4:00 p.m.Meet the presenter of "Create a great video playback experience" and join a text-based watch party for the session, followed by a short Q&A. The watch party begins 5 minutes after the start of this activity — so don’t be late!
-
Q&A: ScreenCaptureKit
Thursday @ 4:00 - 6:00 p.m.Ask Apple engineers about ScreenCaptureKit during this 2 hour text-based Q&A. Stop in to request guidance on a code-level question, ask for clarifications, or learn from others in a group setting.
Sessions
-
Build device-to-device interactions with Network Framework
8:00 a.m.Learn how you can create integrated content experiences across multiple devices. We’ll introduce you to DeviceDiscoveryUI, which makes it easy to set up communication pathways and connect Apple TV with other devices like iPhone, iPad and Apple Watch. We'll also explore common scenarios and provide best practices to help you enable frictionless device-to-device connectivity.
-
Create custom catalogs at scale with ShazamKit
8:00 a.m.Learn how ShazamKit can help you build custom catalogs and support exact matching of any audio source within your app — all on-device. Find out how you can easily generate audio signatures and build catalogs at scale through the new ShazamKit CLI. We'll also show you how you can quickly update your app to sync with large amounts of audio content like multiple seasons of a TV show or multiple episodes of a podcast, and we'll share updates to the ShazamKit API and SHMediaItems to help your apps respond precisely to key moments in audio sources using time ranges. For more on ShazamKit, we recommend watching "Explore ShazamKit" and "Create custom audio experiences with ShazamKit" from WWDC21.
-
Explore media metadata publishing and playback interactions
8:00 a.m.Learn how you can highlight your app's Now Playing information on every platform. We'll take you through an overview of media metadata, learn how it gets represented in areas like the Lock Screen and Control Center, and show you how to write and publish effective media metadata for your content. We'll also explore how your app can respond to commands from other devices such as HomePod.
Labs
-
ScreenCaptureKit lab
Friday @ 9:00 - 11:00 a.m.Request an appointment with an Apple engineer for guidance and conversation about your app, the ScreenCaptureKit API, best practices, and more.
-
SharePlay lab
Friday @ 9:00 - 11:00 a.m.Have a question about SharePlay and Group Activities? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
Core Image lab
Friday @ 9:00 - 11:00 a.m.Request an appointment with an Apple engineer for guidance and conversation about Core Image, including how to use CIFilters with Extended Dynamic Range (EDR) headroom, methods to add kernels written in the Metal Shading Language, best practices for rendering to MTKViews, and more.
-
Apple Music API and MusicKit on the Web lab
Friday @ 9:00 - 12:00 p.m.Have a question about Apple Music API or MusicKit Web Apps? Request an appointment with an Apple engineer for guidance and conversation about integrating with Apple Music API or MusicKit JS to your Apps, best practices, and more.
-
HTTP Live Streaming lab
Friday @ 9:00 - 11:00 a.m.Have a question about live or on demand HLS video streaming, Low-Latency HLS, Offline HLS, spatial audio, dynamic ad insertion, or stream authoring recommendations? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
ShazamKit lab
Friday @ 10:00 - 1:00 p.m.Request an appointment with an Apple engineer for guidance and conversation about exact audio matching with ShazamKit.
-
AVKit lab
Friday @ 10:00 - 12:00 p.m.Have a question about AVKit? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
AVAudioSession lab
Friday @ 1:00 - 3:00 p.m.Have a question about AVAudioSession? Request an appointment with an Apple engineer for guidance and conversation about the latest APIs, best practices, and more.
-
ReplayKit lab
Friday @ 1:00 - 3:00 p.m.Request an appointment with an Apple engineer for guidance and conversation about your app, the ReplayKit API, best practices, and more.
-
Core Audio lab
Friday @ 2:00 - 5:00 p.m.Request an appointment with an Apple engineer for guidance and conversation about Audio Units, Workgroups, HAL, Toolbox APIs, CoreMIDI, AVAudioEngine, AVF APIs Catalyst apps, Spatial Audio, SoundAnalysis, or AUVoiceIO.
Digital Lounges
-
Meet the Presenter: Create custom catalogs at scale with ShazamKit
Friday @ 9:00 - 10:00 a.m.Meet the presenters of “Create custom catalogs at scale with ShazamKit” and join a text-based watch party for the session, followed by a short Q&A. The watch party begins 5 minutes after the start of this activity — so don’t be late!
-
Meet the Presenter: Building Device-to-Device Experiences with Network Framework
Friday @ 2:00 - 3:00 p.m.Stream a live rewatch of "Building Device-to-Device Experiences" with the presenter. Join a real-time text conversation during the rewatch and participate in a short Q&A afterwards. The rewatch begins 5 minutes after the start of this activity — so don’t be late!
-
Q&A: AVKit
Friday @ 3:00 - 5:00 p.m.Ask Apple engineers about AVKit during this 2 hour text-based Q&A. Stop in to request guidance on a code-level question, ask for clarifications, or learn from others in a group setting.