AVKit on tvOS offers an intuitive user interface for interaction with your media on this powerful new platform. AVKit is built on AVFoundation, which gives you access to the full potential of platform services for operating on time-based audiovisual media. Learn how to leverage the new content proposal APIs, and how to create a seamless interstitial playback experience. Management of media metadata and chapter navigation will also be covered.
How many of you have apps in the App Store or working
on apps that play video?
That's quite a few of you.
Good. Whether for sports, news, entertainment or education.
What's the most important part of your video playback?
Is it the Play/Pause control?
Maybe the audio and subtitle settings?
No. Of course not.
It's the video presentation.
So it's important when users are using your application
and watching your video that they focus entirely
on your video presentation.
And they aren't distracted by trying to figure
out your user interface.
That's why it's important that video playback be easy,
consistent and predictable for viewers as they move
between apps on their Apple TV.
Video playback on Apple TV appears simple,
but it's no easy task.
There are many different tools for user interaction,
including the buttons and touch surface of the Siri remote,
Siri voice commands, older Apple TV remotes, the iOS remote app,
Bluetooth keyboards, game controllers, and of course,
infrared universal remotes.
Wouldn't it be great if you didn't have to deal with this?
I'm Dan Wright and today I'll be talking
about how you can accomplish all this easily with AVKit.
AVKit provides modern playback
with a consistent user interface.
Let's take a quick look at some of the features AVKit provides.
Here is AVKit playing a video
with a transport bar visible showing the elapsed time,
the position in the video and the time remaining.
Touching the surface of the remote reveals hints.
And clicking on the edges will skip forward or back.
Or by holding down the touch surface,
viewers can fast forward or rewind.
And when paused, swiping side to side
on the remote will scrub quickly through the entire video.
Swiping down reveals info panel, which includes information
about the video and navigation markers as well as access
to settings related to audio and subtitles.
Finally, AVKit supports Siri voice commands automatically.
And as I go back to the beginning,
or "What did she say?"
to skip back 30 seconds and replay
with captions turned on temporarily.
AVKit uses the modern media stack,
the same as on iOS and macOS.
AVKit provides the user experience.
AVFoundation and CoreMedia provide the core playback tools.
And UIKit or on macOS AppKit provides interface elements.
Today we're going to talk about three things.
First, we'll show you how easy it is to get started with AVKit.
Second, we'll look at several
of the ways you can extend the playback experience
with features unique to tvOS.
And third, we'll talk about some best practices.
All right, let's get started.
Let's talk about AVPlayerViewController.
AVPlayerViewController uses an AVPlayer, AVPlayerItem
and AVAsset provided by your application.
The AVAsset represents the audio visual media.
AVPlayerItem represents the presentation state of an asset.
The AVPlayer controls playback.
And AVPlayerViewController sits on top, providing the UI.
Let's look at how you'll provide your media to AVKit.
There are four steps.
First, create an asset from the URL.
Second, create a playerItem of the asset.
Third, create a player with the playerItem.
And fourth, associate the player with the playerViewController.
Now, you can simplify all that to one line just
to create the player directly from the URL and assign it
to the player property of the playerViewController.
Next, let's see how you can embed an inline player view.
First, you'll set up your playerViewController.
Next you're going to set the frame to the inline view.
We're setting the frame directly here,
but you should use constraints, of course.
And third, add the view from the playerViewController
to your view.
And add the playerViewController itself
as a child view controller of your view controller.
Now, once you're ready
for interactive full-screen presentation,
you can just present the view controller
and it will automatically animate
from the inline playback to the full screen.
Let's talk about extending the playback experience.
AVKit for tvOS was introduced in tvOS 9, last fall,
with several enhancements.
Namely, the ability to add non-interactive overlays.
Restrict playback interaction.
Provide informational metadata.
And identify interstitial content.
With tvOS 10, we will introduce a couple new enhancements:
The ability to modify skipping behavior
and presents content proposals.
And now let's talk about each of these.
Overlays are for logos and other overlaid graphics.
The playback overlay view lies above the video,
but below the controls.
Views may be static or animated.
But views will not receive focus or events.
Let's move on to restricting playback.
The requiresLinearPlayback property
of the player view controller limits user interaction
when set to Play and Pause.
Things like fast forward, scrubbing, skipping,
chapter navigation and so on are all forbidden when this is set.
Typically you'll set it to true when you want to enforce viewing
of a particular section of video,
for example ads or a legal notice.
The external metadata property supplements
or replaces information embedded in your asset.
Things like title, description, genre, media content and rating,
PG-13 or R, that sort of thing, or poster artwork.
Let's go back to our screenshot
that showed the info panel we saw earlier.
And I want to direct your attention
to this top section right here.
This is where the informational metadata is displayed on tvOS.
Now, in addition, it'll also be displayed in the iOS remote app.
Here we see the artwork, which in this case looks
like a still from the video.
We have the title.
We have the duration, which comes directly from the asset,
a media content rating and a description.
Now, let's look at how we're going
to create external metadata items.
And we will create a couple of helper functions
to make it a little easier.
The first one called metadataItem takes an identifier
and a value.
And it's going to return an AVMetadataItem.
And so metadataItem has a whole bunch of properties.
You really need to set three, these three: The value,
the identifier, and often overlooked,
the extended language tag.
Now, you see here I've set it to the string und which is short
for undefined and it acts like a wildcard.
So if you do not have a more specific language translation
available, this is the version that AVKit will display.
If you don't have any translations, if everything,
for example, is in English or everything's in French,
then just use und so that users,
regardless of their language, will see something.
Now, artwork is a little bit different.
The value of the artwork metadataItem is an
So it's raw data.
You can use PNG or JPEG.
You also need to set the data type field to identify what kind
of data you're providing.
And once again, you set the identifier
and the extended language tag.
Now we're going to bring it all together.
We're going to create several external metadata items.
We create an array.
Then we're going to use our helper function to add a title,
a description and the poster image to that array --
oh, and sorry, also the genre.
And then finally we're going to assign that array
to the external metadata property of the playerItem.
Back to our info panel.
Now we're going to look down here at the bottom part.
This is the navigation section.
And this is displaying a navigation marker group.
Navigation markers most often are used for chapters.
But they allow for easy access
to different parts of your video.
Navigation marker groups are for chapters or events.
Examples of events would be
in a sports video you might have game highlights, for example.
A navigation marker group is defined by an object
of class AVNavigationMarkersGroup.
And it contains an array of navigation markers,
in this case chapters.
Every marker has a title.
And in most cases an image as well,
usually a thumbnail from the video.
But that's optional.
An event group itself also has a title, a chapter group does not.
Another little helper function, the create navigation marker.
This takes a title, a description and a time range.
Now, the time range is the time range of the thing
that you're navigating to.
However, the duration is optional.
If you don't really care to set it, you can just leave it zero.
Here again, we're creating metadata items.
In this case, for a title, description.
And then finally we're going to create the timed metadata group
with our metadata and the time range.
Now let's talk about interstitials.
Here we see a video with several interstitials as indicated
by the dots on the transport bar.
The user is watching an interstitial here.
And the time above the transport bar is counting
down the time remaining.
The time below is the elapsed time into the video.
Interstitial content is content
that is unrelated to the main video.
So for example, advertisements, legal notices,
other things like that.
An interstitial time range identifies the portion
of an asset that contains interstitial content.
And interstitial time ranges collapse.
The dots on the transfer bar are [inaudible] holes as we saw
in the previous slide.
And finally, during scrubbing, interstitial content is hidden.
So the user just is navigating through the entire main video
and they're not getting distracted
by ads popping ups as they're scrubbing.
Now here we have a diagram comparing the timelines
of your asset on the top and the transfer bar as seen
by the user on the bottom.
We have a nine minute asset with a couple of interstitials,
both of which are going to be collapsed into dots.
And you'll notice that the duration
on the transfer bar has been reduced by the length,
or rather the duration of those interstitials.
Now let's look at creating and declaring interstitial content.
First, on your server you should stitch the interstitial content
directly into your HLS asset.
Then, in your client app,
you will declare the interstitial time ranges.
And implement a few delegate methods
to enforce playback policy.
Here's a short snippet showing how
to declare the interstitial time ranges.
For each interstitial create an AVInterstititalTimeRange.
Then set the interstitialTimeRanges property
of your player item to this array.
In this case we're just creating a single interstitial
Next, the delegate methods.
There are three important ones I'm going to talk about.
The first is willPresent.
willPresent is called when one
of your interstitials begins to play.
Typically, this is where you're going
to change requiresLinearPlayback to restrict navigation.
So for example, if this is an advertisement,
you probably need, for contractual reasons,
to require the user to watch the entire interstitial once
You set it to true.
Then they can't skip out.
They could still pause, head to the kitchen
or something and come back.
But they can't navigate, they can't skip
over it once they're inside.
didPresent is called at the end of your interstitial.
And usually you set requiresLinearPlayback back
to False so that the users can once again navigate.
timeToSeekAfterUserNavigatedFrom time to some target time.
So this is called when the user navigates or indicates
that they want to navigate to a different part of your video.
For example, they skipped forward or back.
They scrub and hit Play again.
Or they use the navigation markers to jump
to another part in your video.
The oldTime is the time that they were playing
at just before they skipped.
The targetTime is the time that they want to watch next.
The time returned by this delegate is
where you want playback to resume.
So by default, if you didn't implement this method,
it would resume at the target time.
AVKit would seek to the target time and start playback there.
If you provide the start time of an interstitial instead,
AVKit will automatically redirect to that interstitial.
Now, if you do redirect to an interstitial,
unlike in our very simple sample here, you're going to want
to save the target time.
And when your interstitial is complete,
seek back to the target time
so that the user can once again resume at the time
that they wanted to watch.
Now let's turn to skipping behavior.
Here we see the skipping indicator.
And this is a new skipping indicator in tvOS 10,
the skip by item indicator which can replace the older indicator,
which is the skip forward or back 10 seconds.
In tvOS 10 you can choose
between these two skipping behaviors.
If your app is displaying something like a movie
or a TV show, it's probably most useful for users to be able
to navigate skip 10 seconds back and forward at a time
to more easily find the scene that they're looking for.
If you have a series of short videos,
for example exercise videos, it probably makes more sense
to let users easily skip to the next video, the next exercise,
or the previous video, previous exercise, then to move
around a little bit within an exercise.
So, to accommodate that, you can change the skipping behavior.
There are two skipping behaviors defined today.
The default, which is to skip plus
or minus 10 seconds, and skip item.
And there are three properties
in the PlayerViewController related to this:
The skipping behavior and properties to enable
and disable the skip forward and skip backward hint indicators.
So let's look at skipping by item instead
of skipping a few seconds.
First thing we're going to do is we're going
to set the three properties.
Select the new skipping behavior.
In this case we're just going
to always enable skip forward and skip back.
I'm being a little lazy.
And then we're going to implement two delegate methods.
Skip to next item, all we're going
to do is replace the current item with the new video.
And skip to previous item.
Same thing, but going backwards.
Okay. Now for a demo of all these features,
I'd like to welcome Jonathan Long to the stage.
My name is Jonathan Long.
I work with Dan on AVKit for tvOS.
And today, we are going
to be demoing an app called AVKit Player.
I have the project open here in Xcode.
And this is the main view controller.
It is the subclass of UIViewController.
And we have a reference to an AVPlayerViewController.
We obtain this reference in the prepare for segue callback.
In addition, we also create an AVAsset, an AVPlayerItem
and an AVPlayer, which we assign
to the PlayerViewController's player property.
I'll go ahead and build and run this
so we can see what it looks like.
[ Music ]
So here's our playerViewController
in full screen with all of the standard user interactions.
If I swipe down to reveal the info panel,
you can see that there's not a whole lot
of interesting information here.
Most notably we are missing the info tab.
So the first thing we're going to do is we are going
to add some external metadata and some navigation markers
so that info tab shows up.
Jumping back into Xcode, the first thing that we're going
to do is we are going to add some external metadata.
To do this we need to create some AVMetadataItems,
add them to an array and assign this array
to the playerItemExternalMetadata
I'll go ahead and add some helper functions here.
These probably look very familiar,
as they are the same ones
that Dan showed you in slides earlier.
The first creates and AVMetadataItem for artwork,
setting the data type to be JPEG.
The second creates an AVMetadataItem
for all other value types,
setting the extendedLanguageTag to be undefined.
Now, I have one more helper function here
that takes a dictionary as its argument.
And it simply maps the key value pairs of this dictionary
to create AVMetadataItems
and returns an array of AVMetadataItems.
So now with these helper functions,
we can set the playerItem.externalMetadata
property to be our array of AVMetadataItems.
Cool. So we created AVMetadataItems,
added them to an array, assigned that array
to the playerItem.externalMetadata
So now let's move on and add some navigation markers.
I have some helper functions here for that.
The first creates an AVTimedMetadataGroup
with a title, a description and a time range.
The second creates an AVNavigationMarkersGroup
with a name and four time metadata groups
that represent different time ranges in our content.
So now with these functions,
I can set the navigationMarkerGroups property
on the playerItem to be an array of AVNavigationMarkerGroups.
We'll name this one Additional Group.
Cool. So we created an AVNavigationMarkersGroup
with some time metadata groups with a name.
We added that to an array.
And then we assigned the array
to the navigationMarkerGroups property on the playerItem.
So, let's build and run and see what happened.
So here is our Player View Controller again.
And I can swipe down to see the Info tab.
And now we see in the metadata space all the metadata
that we've specified as well as the artwork for this title.
And if I select down at the bottom,
one of the navigation markers that we've added,
we can skip to that point in the content.
So with just a little bit of code it's easy
to add external metadata and navigation markers
to provide a rich user experience in your info panel.
Now I'll hand it back over to Dan to talk
about new APIs in AVKit for tvOS.
Thank you, Jonathan.
Content proposals are about suggesting what to watch next.
You may have seen Up Next suggestions in some
of your favorite apps such as Netflix, Hulu, HBO.
When you reach the end of each episode of, say,
"The West Wing" you'll see a suggestion pop
up to watch the next episode and so on and so
on until it's 3:00 in the morning.
Many apps implemented their own version of this for tvOS 9.
And then it was kind of tricky.
No more. With tvOS 10 AVKit provides a standard way
to present your own customized Up Next experience.
We call these Content Proposals.
Here is one example
Now, you have the entire screen to work with.
So you're not limited to anything here.
This is just an example.
In the lower right we have some metadata
and some big buttons that you can press.
Up here we have the currently playing video.
And here is the thumbnail representing the proposed video.
Now, a content proposal is represented by an object
of class AVContentProposal.
This is basically a model object.
Now, there are several properties on here.
I'm not going to spend a lot of time on all of them.
But I want to highlight a few important ones.
The first is contentTimeForTransition.
This is the time within the currently playing video
that you want your proposal to be presented to the user.
So, if you specify the duration of your video or zero
as a shortcut, that means that you want it to appear
at the very end of the video.
However, if you're presenting a TV show or a movie or something
like that, you probably have a bunch of end credits
that the user may not want to sit through.
And you may want to present your proposal at the beginning
of those end credits, in which case you can set this
Next automaticAcceptanceInterval allows you to specify a timeout
for your presentation.
You can say, for example, 30 seconds.
Which means that if the user does absolutely nothing,
they take no action for 30 seconds,
then the proposal will be automatically accepted
If you don't set this, then your proposal will just stay onscreen
until the user takes some sort of action.
Title and preview image would be the bare minimum bit
of information that you should provide to a user.
You can, of course, provide as much additional information
as you like: Description, maybe a little widget
to set a user rating,
maybe additional suggestions for alternate videos.
And finally, the way you specify your content proposal is
to set the next contentProposal property
on the current playerItem.
So let's look at creating the content proposal.
Well, this is pretty easy.
We just create an instance of AVContentProposal.
We're going to specify a time of zero
for our contentTimeForTransition.
This means we want it to show up at the very end.
This is a nice shortcut if you don't have the duration yet.
For example, if your playerItem hasn't loaded.
And we're going to set the title and a preview image.
In this case, our proposal is for Episode 2 of Season 9
of a fictional TV show called "Happy Hijinks."
In this case we're going set one of the optional properties
of the content proposal, a URL.
And we're going to set it to the URL for this episode,
for our own use later.
Now we assign the proposal to the playerItem,
which should show the proposal.
Naturally, this is the playerItem
for Season 9, Episode 1.
And, once again, we've got three interesting delegate methods.
shouldPresent is called immediately before your proposal
is to be presented to the user.
Now, you could take this opportunity to return False
and prevent the proposal from being presented.
But you can also use it as a last minute opportunity to set
up to prepare for your presentation.
And we'll show an example of that in just a minute.
didAccept is called when the user accepts your proposal.
That means they want to watch the video that you've suggested.
It will also be called if the timeout occurs.
And didReject is called if the user has indicated
that they want out of your proposal.
They want out of the video.
They want to get back to your menu.
Now finally, you will implement your presentation
by creating a custom subclass of AVContentProposalViewController.
There are a few interesting things in the subclass.
Let's look at two,
preferredPlayerViewFrame is the frame onscreen
where you want the video to appear during your presentation.
Now, the base class simply returns the frame
of the entire screen, in which case the video will continue
to fill the screen.
And your presentation controls will be overlaid on top.
However, you can specify a smaller rectangle,
maybe off to the side in a corner or something,
to give yourself lots of space, the rest of the screen to work
with for your controls.
And dismissContentProposal is how content proposals
You can specify one of three actions:
Accept, Reject and Defer.
Defer means the user simply wants to hide your presentation,
get back to watching the end credits so they can,
I don't know, maybe they want to spot their name
in the end credits if they worked on the video.
And then as the name implies, Defer, it will reappear
at the end of the video.
Now let's look at responding to those delegate notifications.
In this case, we're going to create an instance
of our custom contentProposalViewController
subclass and assign it
to the contentProposalViewController
of the playerViewController and return true.
This lets us set that up at the very last minute
when we know we're going to actually need it
and not allocate a bunch of memory
that the user may never get to.
Here all we're going to do is we're going
to replace the current playerItem
with a proposed content playerItem.
We've seen this code a few times already.
You'll notice here I'm making use of the URL property
of the proposal that I set when I created the contentProposal.
Now, if this is literally all you're going to do here,
you don't need to implement this.
Because AVKit, if you provide a URL on your proposal,
AVKit will automatically do this
if you don't implement the delegate method.
All right, and now for a demo with content proposals.
So, let's continue with AVKitPlayer and add an
Up Next experience
by subclassing AVContentProposalViewController.
So the first thing we need to do is we need
to create a custom subclass of AVContentProposalViewController.
I have one right here that I will go ahead
and add to our project.
So, this is our UpNextContentViewController.
It is a subclass of AVContentProposalViewController.
And we have some UI properties here
such as a UIImageView, a UILabel and others.
We override the preferredPlayerViewFrame
to return the CGRect that we want our playerViewController's
view to animate to when the contentProposal is presented.
Now, the rest
of the viewController simply handles the layout
of its view and subviews.
So we'll jump back into our main viewController.
So there's two more things that we need to do.
First, we need to create an instance of AVContentProposal
and assign it
to the playerItem's next content proposal property.
So, here we are creating an instance of AVContentProposal.
We have a transition time, a title and a preview image.
We're also setting the automaticAcceptanceInterval
to be 15 seconds.
In addition, we set the URL and the metadata
on this content proposal that corresponds to the next item
that will be played if the user chooses to accept this proposal.
Finally, we set the contentProposal we just created
as the next contentProposal property on our AVPlayerItem.
So the next thing we need to do, and the last thing we need
to do, is implement two playerVIewController
You can see here that we set the playerViewController delegate
And the two methods that we need
to implement are shouldPresentProposal
So in shouldPresentProposal we create an instance
of our custom subclass of AVContentProposalViewController
and we assign that as the playerViewController's
In addition we return true to indicate
that the contentProposal should be presented.
And finally, in didAccept,
we need to handle transitioning our player
to the next playerItem.
So, to do this, we get the URL from the contentProposal.
We create an AVPlayerItem from that URL.
And then we simply replace the current item on our playerItem.
And we're done.
So let's build and run and see what happens.
So I'll go ahead and skip closer to the end of the content.
[ Music ]
And here is our contentProposal.
As you can see, the metadata that we specified is here.
The preview image as well as the title.
And if I select Play Next, we would transition
to our next playerItem.
So I think if you have implemented this
on tvOS 9 you'll be very happy with this API and you will find
that it's much easier to use.
I'd like to invite Dan back up to talk
about some best practices of the AVKit.
Okay. Best practices.
I'm going to talk about a few things.
Things to do and things to avoid doing.
First, some things that we recommend that you do.
Once again, present or present view controler will handle
zooming automatically from an inline view.
So there's no reason to implement your own animation.
Second, remember Playback is only interactive
when full screen.
Third, use the new content proposal API.
Some of you wrote it the hard way before.
And we encourage you to switch over to doing it the new way.
I think you'll find your code is a lot simpler.
It's easy to understand the flow.
And you may avoid some bugs and other things
that you may have had trying to get it
to work just perfectly before.
And observe the error property of the player in playerItem.
This is how you find out about errors
that occur during playback.
And I'm going to talk about one particular error in a minute
that you should handle.
But you should watch for any errors that occur
and respond appropriately to the user.
Now a few things to avoid.
Avoid toggling showsPlaybackControls.
This property is not for temporarily showing
or hiding the controls.
This is for indicating your permanent intent.
So when you set this to false,
you're saying we don't want the playback controls at all.
So if you toggle this back and forth,
what you're doing is you're telling AVKit
to destroy all the controls and recreate them.
And it's not very efficient, as you can imagine.
Second, some people hate this,
but avoid adding supplemental gestures to playback
because it's going to confuse users.
People will not discover them.
And by the same token, do not overload the Select button
or touch service gestures.
Not only will that confuse users,
but you'll likely break in the future.
Now, some more general tips.
Replace your asset if you see this error:
If you don't respond to this
and the media services are reset during playback,
video playback will fail.
And the user won't be able
to do anything except, hopefully, get out.
So what they'll have to do is get out,
go back to your menu and go back in.
If you catch this error
and automatically replace your asset, your player item
and your player, basically your AV foundation objects,
then you can more seamlessly handle this case.
And the user will barely even know that anything happened.
Now, some other sessions with best practices
for playback include this year,
Advances in AVFoundation Playback.
It was earlier this week.
You can catch the video in the WWDC app.
And two years ago we had an excellent session,
Mastering Modern Media Playback, which includes tips
for both AVKit and AVFoundation and getting them
to work well together.
So in summary, AVKit provides standard playback controls
and behaviors, support for remotes, game controllers
and Siri, full access to the media stack, powerful new APIs
and it's easy to get started.
Be sure to check out the sample code for more in-depth examples.
And we're excited to see your app.
So, for more information, go to the URL on the screen.
We'll have links to the sample code and other resources.
And check out some of the related sessions.
We also have on here the HTTP Live Streaming session also
earlier this week.
And a pair of sessions on using TVMLKit,
which is an alternative approach to using AVKit directly.
So, thank you.
Looking for something specific? Enter a topic above and jump straight to the good stuff.
An error occurred when submitting your query. Please check your Internet connection and try again.