Meet the Now Playing framework

Meet the Now Playing framework

Get a first look at Now Playing — a Swift framework that connects your app's media playback to system surfaces like the Lock Screen, Control Center, Dynamic Island, and CarPlay. Discover how to publish playback state and respond to commands using its observable API. Explore remote playback sessions, a new capability that lets your app represent media playing on external devices and bring full playback controls to those same system surfaces.

Chapters
- 0:00 - Introduction
- 1:08 - Media sessions
- 5:03 - Remote media sessions
- 10:31 - Media sharing extensions
Resources
Hi, my name is Leo Formaggio, and I'm an engineer on the Media Frameworks team.
Media has such an important presence in our daily lives. It's a podcast on the drive home. A high-energy playlist during a workout. Or watching a movie on a long flight.
Whether we are spending time by ourselves, or connecting with people, media is always around us. On iPhone, it appears right on the lock screen, Control Center, and in the Dynamic Island.
When the iPhone is set down and charging, it's glanceable in StandBy.
When getting in the car, it appears front and center in CarPlay.
That's the system now-playing experience, and it's available on all Apple platforms. Including Apple Watch, Apple Vision Pro, and Apple TV! I'll show you how NowPlaying framework makes it easy to bring media from your app into the system.
I'll start with the media sessions API, showing how to surface your content in the system now-playing experience.
Then, I'll explain how to bring content playing on other devices into the system using remote media sessions.
And finally, I'll go over how Media Sharing Extensions simplify playing media from iPhone to other devices.
I'll use the example of an app I developed. It plays ambient sounds to help people focus and relax. From my app, I can select different sounds, and I can also pause and resume the audio.
To surface my content in the system, I used the media sessions API from NowPlaying. Next, I'll show you how I did it. Here's my PlayerModel – an @Observable class with a reference to my app's audio engine, and a property that tracks which sound is currently playing. The MediaSessionRepresentable protocol is like a contract between my app and the system. When PlayerModel conforms to it, the system will be able to understand what my app is playing – and how to handle media interactions, like skip or pause. Each session representation needs a unique identifier. The content property is used to describe the sound that's playing. NowPlaying offers content-specific types, like Music, Podcast, and MovieContent, to describe what kind of media your app is playing. GenericContent is a good fit for my use case, so I used that.
Each content is identified by the current sound.id. I used the sound.name as the content title and a short sound.description as the subtitle. Media type can be either .audio or .video, but my app only plays audio. I set the duration to .continuous, because my app plays ambient sounds indefinitely. I provide an Artwork with an async closure. The system calls it whenever it needs an image at a specific size.
Now, I'll show you what this looks like on the lock screen. The sound name and description appear along with the artwork.
The PlaybackSnapshot property is used to display the current playback state. Since ambient sounds are continuous, I just need to indicate if content isPlaying. For content with a defined duration, you should also specify an elapsedTime parameter in the snapshot.
Through the commands property, I define all the actions my app supports. Each command has a closure that the system calls when someone performs that action. For example, when I tap the pause button on the lock screen, the pause command closure is called, and I can pause the player.
Notice the button has changed to reflect the paused state. Now, if I tap play on the phone, the play command closure is called, and I resume the player. Similarly, when I tap the next button on the lock screen, the next command closure is called. I can skip to the next sound, and the lock screen updates to reflect the new content.
After I adopted MediaSessionRepresentable, I had to do one more thing to make my content available to the system. MediaSession is what connects the session representation with the system. I initialize it with my PlayerModel, in the same place where I set up my audio engine. Once that's done, MediaSession starts observing the model, keeping the now-playing surfaces up to date automatically. That's how I integrated my app's content with the system now-playing experience using media sessions.
For more information, check out the article "Publishing Media Sessions" on Apple Developer Documentation.
In addition to playing on iPhone, I made my audio engine available on smart speakers, which can be controlled by my app.
From my app's device picker menu, I choose the speaker I want to control. I tap the Living Room Speaker to start controlling it. And, through a web server, my app connects to the selected speaker to request the playback state, and send commands.
To surface the content playing on that speaker in the system, I used the remote media sessions API.
This API uses an app extension and push notifications to receive updates about the speaker. I'll show you what this interaction looks like.
When someone interacts with a speaker, the speaker communicates the state is changed to the server. The server then uses Apple Push Notification service, APNs, to send a push notification to the iPhone with the updated state. The system launches the app extension with the updated state from the push notification payload. The app extension then provides the system with an updated representation of that session. For more information on sending push notifications with APNs, check out the article "Setting up a remote notification server" on developer.apple.com When the interaction originates from iPhone system UI, the system calls a command handler in the app extension. The app extension sends the command to the server. And the server notifies the speaker, which reacts to the change.
Here is how I adopted remote media sessions in my app.
First I created an app extension conforming to the RemoteMediaSessionExtension protocol. To set it up, I used NowPlaying's RemoteMediaSessionExtensionConfiguration and the remote-media extensionPoint identifier. The session(:) method is called by the system whenever it needs to interact with a remote session representation, for example, to update the user interface, or to handle an interaction. Here, I can use the RemotePlayerState to create my model, and return it.
With the app extension configured, I'll show how I used my model to represent a Remote Media Session. This is my RemotePlayerModel. It's an @Observable class with a reference to ServerClient, the class I use to communicate with my server. It also keeps track of the server state. I'll use this as a foundation to build my Remote Media Session representation. Each Remote Media Session representation needs a unique identifier. I used the sessionID from my server state.
The content property is used to describe the sound playing on the speaker. Once again, I used GenericContent by giving it the sound identifier, the sound name and the sound description.
The media type is .audio and the duration is .continuous.
I provided an Artwork object that loads an image for the current sound.
The server state indicates if the speaker isPlaying. I can use that to create a PlaybackSnapshot with the corresponding state.
Since I'm controlling playback on a remote device, each command closure sends a request to the server with the corresponding action.
For example, if I tap play on iPhone, the play command closure is called. I send a play request to my server, which resumes playback on the speaker. Similarly, when I tap the next button, a request is sent to the server, and the speaker moves to the next sound.
So far, the adoption of RemoteMediaSessionRepresentable feels very similar to what we saw in media sessions for local playback. Next, I'll cover the remaining properties and methods that are specific to remote sessions.
The devices property tells the system about devices playing in that session. I map my server's device list into MediaDevice values. Each one needs a unique identifier that is stable across different sessions. I provide the name of the device, a device type, like .speaker in my case, and a list of capabilities, such as the device's volume control type.
This is what that looks like in ControlCenter. The device name appears along with the volume level.
When I change the volume using the system volume slider, the volume change closure is called with the updated volume level. Here, I can send a volume change request to my server.
The update(:) function is called when a push notification is received with a new state. For example, when the content changes on the speaker. RemotePlayerState is a struct I defined, that conforms to RemoteMediaSessionAttributes. It represents my server state and the push notification payload. Here, I update my state variable with the new data. Because my model is observable, NowPlaying detects the change and updates the system automatically.
And that's how I integrated my app's remote media session with the system. For more information, check out the article "Publishing remote media sessions".
I also want to talk about Media Sharing Extensions – a set of APIs for playing media from iPhone to other speakers and TVs, all through a unified system interface. Media Sharing Extensions allow you to use the system device picker for all the media protocols your app supports.
This simplifies media device selection in your app, and the selection is reflected on system surfaces, like Control Center.
Traditionally, supporting a media protocol meant embedding its SDK into your app bundle.
With Media Sharing Extensions, the protocol implementations live outside your app and are managed by the system.
Your app can focus on the media content rather than the playback technology.
As more protocols become available, apps built with Media Sharing Extensions can use them without adopting another SDK.
That covered how to bring local and remote media sessions into the system now-playing experience using NowPlaying framework, and how Media Sharing Extensions can simplify sending media to other devices.
For apps that play media locally or control playback on remote devices – adopt NowPlaying to bring your content into the Lock Screen, Control Center, and beyond. It's a straightforward integration that gives people control over media even outside your app.
Learn more about Media Sharing Extensions. They let your app use the system media device picker and expand your reach when playing media to other devices. For more information about Media Sharing Extensions, check out the article "Routing media to third-party devices".
I can't wait to see your app extended to the system now-playing experience. Thank you for watching, and blue skies!

1:57 - Existing PlayerModel implementation

import Observation

@Observable
final class PlayerModel {
    let player: SoundPlayer
    var sound: Sound { player.currentSound }

    init(player: SoundPlayer) {
        self.player = player
    }
}

2:06 - Adopt MediaSessionRepresentable

import NowPlaying

extension PlayerModel: MediaSessionRepresentable {
    var id: String { "ambient-sound-session" }

    var content: (any MediaContentRepresentable)? {
        return GenericContent(
            id: sound.id,
            title: sound.name,
            subtitle: sound.description,
            type: .audio,
            duration: .live,
            artwork: Artwork(id: sound.id) { size in
                let data = try await self.artworkData(size: size)
                return try ArtworkRepresentation(data: data)
            }
        )
    }

    var playbackSnapshot: MediaPlaybackSnapshot? {
        MediaPlaybackSnapshot(
            state: player.isPlaying ? .playing() : .paused
        )
    }

    var commands: [MediaCommand] {[
        .play { self.player.play() },
        .pause { self.player.pause() },
        .previous { self.player.previous() },
        .next { self.player.next() }
    ]}
}

4:31 - MediaSession initialization

import NowPlaying

struct PlayerController {
    let player: SoundPlayer
    let model: PlayerModel
    let session: MediaSession<PlayerModel>

    init() {
        self.player = SoundPlayer()
        self.model = PlayerModel(player: player)
        self.session = MediaSession(model)
    }
}

6:42 - App extension entry point

import ExtensionFoundation
import NowPlaying

@main
final class SampleAppExtension: @MainActor RemoteMediaSessionExtension {
    var configuration: some AppExtensionConfiguration {
        RemoteMediaSessionExtensionConfiguration(extension: self)
    }

    var extensionPoint: AppExtensionPoint {
        AppExtensionPoint.Identifier(host: "com.apple.nowplaying", name: "remote-media")
    }

    func session(_ state: RemotePlayerState) async throws -> RemotePlayerModel {
        RemotePlayerModel(state: state)
    }
}

7:23 - Existing RemotePlayerModel implementation

import Observation

@Observable
@MainActor
final class RemotePlayerModel {
    let client: ServerClient
    var state: RemotePlayerState

    init(state: RemotePlayerState) {
        self.client = ServerClient(sessionID: state.sessionID)
        self.state = state
    }
}

7:40 - Adopt RemoteMediaSessionRepresentable in app extension

import NowPlaying

extension RemotePlayerModel: @MainActor RemoteMediaSessionRepresentable {
    var id: String { state.sessionID }

    var content: (any MediaContentRepresentable)? {
        GenericContent(
            id: state.sound.id,
            title: state.sound.name,
            subtitle: state.sound.description,
            type: .audio,
            duration: .live,
            artwork: Artwork(id: state.sound.id) { size in
                let data = try await self.artworkData(size: size)
                return try ArtworkRepresentation(data: data)
            }
        )
    }

    var playbackSnapshot: MediaPlaybackSnapshot? {
        MediaPlaybackSnapshot(
            state: state.isPlaying ? .playing() : .paused
        )
    }

    var commands: [MediaCommand] {[
        .play { try await self.client.send(.play) },
        .pause { try await self.client.send(.pause) },
        .previous { try await self.client.send(.previous) },
        .next { try await self.client.send(.next) }
    ]}

    var devices: [MediaDevice] {
        state.devices.map { device in
            MediaDevice(
                id: device.id,
                name: device.name,
                type: .speaker,
                capabilities: [
                    .absoluteVolume(device.volume) { volume in
                        // send volume change to server
                    }
                ]
            )
        }
    }

    func update(_ state: RemotePlayerState) {
        self.state = state
    }
}

- 0:00 - Introduction
- Discover the Now Playing system experience, available across all Apple platforms. It allows apps to surface currently playing media info on system surfaces like the Lock Screen, Dynamic Island, and CarPlay.
- 1:08 - Media sessions
- Learn how to use the media sessions API to bring audio or video from your app into the system's Now Playing experience by adopting the MediaSessionRepresentable protocol.
- 5:03 - Remote media sessions
- Discover how to extend playback control to devices like smart speakers by adopting RemoteMediaSessionRepresentable and utilizing Apple Push Notification service (APNs).
- 10:31 - Media sharing extensions
- Find out how Media Sharing Extensions simplify routing media from iPhone to other devices by leveraging the system device picker without needing to embed additional SDKs.

Explore Get Started

Stay Updated

Explore Platforms

Featured

Explore Technologies

Featured

Explore Community

Featured

Explore Documentation

Release Notes

Explore Downloads

Featured

Explore Support

Featured

Quick Links

Chapters

Resources