ReplayKit is an easy-to-use framework that enables players to share gameplay recordings or broadcast live games to players and viewers online. Watch how live broadcasting can help build community and create new fans for your apps and games. Understand the APIs to include player commentary via the microphone and front-facing camera on iPhone and iPad, and check out new support for ReplayKit on Apple TV.
[ Music ]
Welcome everyone to Go Live with ReplayKit.
In this session, we are going to tell you
about all the new features that we've added to the framework.
I am Ben Harry, a software engineer
on the Game Technologies Team here at Apple.
And later, you'll be hearing from Edwin Iskander.
Now we know people love to record their gameplay
and share those recordings on social media.
ReplayKit gives our users those abilities
with little effort required from you, the developer.
To get started today, I am going to take a few minutes to talk
about the existing features that are offered with ReplayKit.
ReplayKit provides the ability
to record your app's audio and visual content.
In addition, you may enable microphone recording.
By doing so, your users can provide voice commentary while
playing their games.
After a user trays a recording,
they may share the recording directly
from their game using the System Share sheet.
ReplayKit has a simple API, and as you're going to see today,
a small amount of code is required
to bring these features to your games.
Moving along, ReplayKit will generate HD-quality videos
with little impact on your game's performance,
and uses a minimal amount of power from the device's battery.
Here at Apple, user privacy is very important to us,
so we have implemented some privacy safeguards.
For example, we present a user prompt before recording begins,
so user's can give permission before recording their gameplay.
Furthermore, recording will exclude system UI,
Therefore, when you receive a text message while you're
playing a game, that message is not going to be recorded.
Finally, ReplayKit was made available in iOS 9.
So this is a list of the current features
that we offer with ReplayKit.
Now, I am going to take a minute to list the new features
that have been added to ReplayKit framework.
First up, we are bringing ReplayKit support to Apple TV.
So now, you'll be able to record your gameplay
and share it directly from your TV OS games.
The next new feature we've added is live broadcasting.
Now, users will be able to broadcast for gameplay
in real-time to third party broadcast services.
And for our third and final feature, we're adding support
to record the Face Time camera.
In addition, we've made enhancements
to the microphone recording API.
So before I get into ReplayKit and Apple TV,
I'm going to take a minute to give you an overview
of the ReplayKit architecture.
I want to show you how your app uses the framework to interact
with the operating system to create a recording.
So here we have your application.
Your application needs to notify the operating system
when it wants to begin recording.
So ReplayKit provides an RPScreenRecorder class.
You will tell this class to start recording.
At this moment, a message is sent to the Replay Daemon.
The Replay Daemon will start writing your app's data
to a movie file.
When you tell the RPScreenRecorder
to stop recording, the Replay Daemon will finish writing your
movie, and now we have a movie on the system.
Now this movie needs to get back up to your application
so your users can preview the recording they just made.
To do so, we provide the RPpreviewViewController.
You present this view controller in your game,
so now your users have an interface
where they can preview, they can edit
and they can share the video that they just made.
As we just saw, we have the RPScreenRecorder class.
This is the class you will use to start,
stop and to discard recordings.
You will also use it to check the availability
to record on the device.
There is a delegate property on the RPScreenRecorder,
and it will notify you if the availability to record changes.
Also it will let you know of recording stops due to an error.
The second class I brought up was the RPPreviewController.
You present this view controller in your game
to give your users ability to preview the recording on iOS,
and to share that recording directly from the game.
We also have a delegate method on this class,
and this delegate will let you know when the user is finished
with the preview user interface.
So this is the reflected architecture
and the classes you will use to record and share content
on both iOS and tvOS, which now brings me
to ReplayKit and Apple TV.
And to begin I'm going to show you a demo.
So here I have a game called Fox.
Fox was developed for WWDC, 2015.
It was developed using SceneKit.
You may also recognize it because it is available
as sample code on our developer website.
The goal of this game is to walk on the level,
and collect these flowers like the one I just got.
I'm going to make my way through the level,
and collect the flowers.
There's three on the level, and I got the first.
You can optionally collect these pearls along the way,
but I'm just going to focus on the flowers.
I got two.
I know the third flower is on the top, and is in the middle,
on the topo of the rock.
So I'm over here, so here I'm going to take,
I'm going to bring up my game menu,
because I want to record this.
There is an obstacle at the head,
and I've been practicing this a lot this past week,
so I wanted to share this with my friends to show them
that I can get through this most of the time now.
So I'm going to start my recording,
and here is the consent prompt I mentioned.
And I am going to give permission.
So now I'm recording.
So notice the recording indicator
at the top of the screen.
This lets me know that our recording is taking place.
Okay, so I made it through the flier.
So all I have to do now is make my way to the end,
to do the final flower, so I can finish the level,
and ultimately finish the recording.
Okay so that's it.
I'm going to stop my recording.
Now, once I'm stopped, I have the option where I can preview
or share the video I just made, so I'm going
to start off by previewing.
So we have a video playback,
and we have a timeline we can scrub forward and backward.
I'm just going to start playing it.
Notice here that recording indicator is not included
in the recording.
I'm going to show you how this was accomplished
in just a few minutes.
So I have a video, I'm happy with how it turned out.
So now I want to share it.
So I'm going to go over here and select share.
And now we are presented with an AirDrop interface.
So I'm going to grab my phone.
I'm going to select my phone.
Anyway, when I do accept the file on my phone
and the file is transferred, once it's on my phone,
I do have the ability where I can preview the video,
I can edit and trim the video if I wanted to,
and finally I can share it to my favorite social media sites.
So that was how ReplayKit looks on Apple TV.
I now want to take a few moments to call out four main points
from the demo, and then I'm going to show you the code
that corresponds with each of those points in a minute.
So first I'm going to begin.
I'm going to have the end game menu
where I started my recording.
Then we have the gameplay where we have the recording indicator
on screen, back to the game menu to stop the recording,
and finally when I stop the recording I have the options
to preview or share the recorded video.
So I'm going to walk through each one of these,
and show you the code that corresponds to each one.
We began or start our recording.
So when I press my start recording button,
first I'm going to grab the shared instance
of the RPScreeningRecorder.
And I simply am going to tell it to start recording.
Now once I'm recording,
I'm going to present the indicator view.
As I mentioned in the video,
the indicator was not included in the recording.
This is because ReplayKit will only record the application's
Therefore, I'm going to create a new UI window.
And then the indicator view.
The indicator view will be your responsibility to create.
This is because now you can make it match the style of your game.
So we have an indicator view,
I'm going to instantiate an instance of that indicator view,
and simply add it as a sub view to that main window I created,
to the window I created above.
Now the indicator view is on screen,
and we know we're recording.
When we are done recording, we press the stop record button.
Again, I'm going to grab the shared instance
of the RPScreenRecorder.
Now notice we return the preview view controller,
which is the RPPreviewController class I mentioned earlier.
I'm going to hide my indicator view.
And here it's important that we hold onto a reference
of the preview view controller, because we are going to use it
to present in the next step for previewing
and sharing the video.
Finally I'm going to set the delegate.
So we saw that we have one preview controller,
but we have two interfaces,
one for previewing, and one for sharing.
So at Apple TV, we have introduced a new mode property.
When we want a preview, we simply set the mode to preview,
and then present the view controller.
Similarly, to share, we have the mode property.
We are going to set the mode to share, and again,
present the view controller.
Now we will be presented with the AirDrop interface.
In both cases, when the user is done with each interface,
we have a delegate method that is going to be called.
Preview Controller did finish.
It's important when this is called that you just--
that you dismiss the preview controller,
since your app was responsible for presenting it.
Finally, when we are done with the recording and we know
that we no longer need it, it's good practice
to discard that recording.
Now ReplayKit will automatically discard the previous recording
when a new recording begins.
This is because one recording is allowed at a time per app.
You could also explicitly discard the recording
when you know that the preview will no longer be available.
For example, maybe at the end of a level,
there could be no more opportunity
to present a preview.
So now you can explicitly discard it
by calling RPScreenRecorder's discard recording method.
Through these last few slides, I've shown you the small amount
of code that is required
to bring these features to your tvOS games.
I encourage you all to add ReplayKit to your tvOS apps,
and also to your iOS apps, if you have not done so already.
So now, let me take a minute
to summarize ReplayKit and Apple TV.
Now, with ReplayKit and Apple TV,
you can record your app audio and video content.
Under Apple TV, the microphone is reserved by the system,
so you cannot provide voice commentary on the platform.
But you can allow users to preview their videos,
and they can share their videos directly from their games.
As you saw, we have a very simple API.
In fact, it's the same API that we offer for iOS.
All these features for Apple TV are going
to be available new in tvOS 10.
So that was ReplayKit and Apple TV.
So now it's time to move on to the second of our new features,
which I personally am very excited about.
So, at this time, please welcome Edwin Iskandar up to the stage.
Alright, thanks guys.
Hey, how is everyone doing?
My name is Edwin Iskandar.
I am a Software Engineer
on the Game Technologies Team here at Apple.
And like Ben, I'm super excited to talk to you guys
about our next feature, Live Broadcast.
So with Live Broadcast, players will be able
to broadcast gameplay to third party streaming services
directly from their iOS or tvOS device.
So this is really exciting, because this is the first time
that this will be possible without any additional hardware
or third party SDKs, will also be allowing your users
to take advantage of their device's powerful Face Time
camera and microphone to provide rich commentary all
And finally, we are going to do this all securely and ensure
that all audio and video data is only accessible to the system
and the broadcast service.
Alright, so to illustrate how this works, we've been working
with flair games to implement the broadcast feature,
and the flagship game, Olympus Rising.
So it's a really awesome game.
It's available in the app store today.
And it has this unique blend of strategy,
RPG and action elements.
Okay, now that we have a game to broadcast from,
we also need somewhere to broadcast to.
So we've also been working with Mob Crush, a broadcast service
that specializes in mobile games.
Alright, so now that we have a game
and a service, we're ready to roll.
So let's see this in action.
Alright so in Olympus Rising,
the Flair developers have implemented an end game button
to start broadcasting.
To initiate a broadcast, the user taps this button,
and are pesented with broadcast services
that they have pre-installed on their device.
In this particular example,
the user has installed the Mob Crush app,
so it appears on this list.
So at this point, the user taps on the Mob Crush icon,
and they are presented with Mob Crush's UI
to set up the broadcast.
The user then goes ahead and adds a title for the broadcast,
and once they're done, they simply press
on the start broadcast button, which returns to the game,
which then presents a countdown,
so that the player can get ready, and finally, go live.
And now that they're live, and the user is playing,
video and audio data is streamed to the broadcast service.
Spectators all around the world can now launch the Mob Crush app
from their device, and watch the gameplay in near real time.
Within the Mob Crush app,
spectators can also discuss the gameplay as they watch.
And even cooler, broadcasters can even be instantly notified
when this happens while playing their game.
So it's just like in sports,
there is something really special
about watching an event live.
Now, this will be possible natively with iOS
and tvOS games as well.
Alright so now that we've seen the entire flow,
as a game developer, you're probably wondering what it takes
to implement this feature in your games?
Here is the player flow broken down.
As you can see, there are many steps.
Initiating the broadcast, selecting a broadcast service,
setting up the broadcast, starting
and stopping the broadcast, indicating a broadcast is
in progress, and also uploading that video and audio data
to the back-end servers.
So this might look a little bit daunting at first,
but the good news for game developers is three
of these steps are handled by ReplayKit
and the broadcast service.
So let's take a look at the code for each of these steps.
So to initiate a broadcast programmatically,
we use a new class called RPBroadcastActivity
ViewController, and call its load method
to get an instance of it.
We then simply present it as you would any UI view controller,
which will then present the user with a list
of broadcast services, and ultimately allow the user
to set up the broadcast.
Just before we do that, though,
we also set the ActivityViewController's
Because we want to be notified when the setup is complete.
At this point, the user can select the service
to broadcast to, sets up the broadcast,
and once we you done setting up,
the activity view controller's delegate method is fired,
and this delegate method is supplied an incidence
of another new class, RPBroadcastController,
which allows us to start a broadcast.
But before we do that, though, we also want
to dismiss the ActivityViewController,
since we are the ones that presented it.
Start our in game countdown timer,
and when the countdown timer ends,
we finally start the broadcast by calling Start Broadcast
on the new RPBroadcastController instance, that was passed in.
And now we are live.
And since we're live broadcasting,
we need to clearly indicate this to the user.
So Olympus Rising does this very well
by animating the broadcast button.
Since the game has fairly complex controls,
they're reusing the button UI to also indicate an
in progress broadcast,
maximizing the available screen space.
Something to note is that the indicator is strictly required
during broadcasting and will be enforced during app review
before you add this to your games.
So programmatically check if broadcast is in progress,
simply query is broadcasting prompt
of the broadcast controller.
You can use the value of this property to start
or stop animating your indicator UI.
To allow users to finish broadcasting,
the Flair developers have implemented a fly out UI
that reveals a sub menu which includes a stop button.
When the player presses on the stop button,
we simply call finish broadcast on the controller.
When broadcasting is finished, we update our UI.
Alright, now that we covered the basic flow,
let's look at a few more details, like error handling.
So since there are a lot of moving parts to this feature,
it's possible that things could go wrong.
So it is important to handle this gracefully.
And so let's take a look at how we do this during a broadcast.
To handle errors, simply set the delegate
to the broadcast controller.
Once set, your delegate will now have its did finish
with error method called and when this is triggered,
you should let the user know about it
and update any necessary UI.
So now another detail to consider is
when the user backgrounds the app,
or the app is interrupted during a broadcast,
like when receiving a phone call, or example.
So when the application goes to the background,
ReplayKit will automatically pause the broadcast.
So in this particular example,
when the application is reactivated and comes back
to the foreground, we prompt the user and ask if they would
like to resume the broadcast.
If the user wants to resume, call,
the resume broadcast method, if they don't,
simply call the finish broadcast method.
And that's it.
For the entire flow, the game developer needs to only interact
with two classes, and its delegates.
to present broadcast services, and allow users to set
up a broadcast, and RPBroadcastController to start,
pause, resume and finish a broadcast.
As you can see, it couldn't be easier to add broadcasting
into your games, and I urge you to do so,
because the benefits are huge.
There is really no better way to spread the word,
increase player engagement
and even build a community around your games.
So that covers game implementation,
but what about the broadcast services
that players are streaming to, like Mob Crush?
Let's take a moment to talk briefly
about what those services are responsible for.
So for all the game developers in the audience,
or watching online, I just want to make it extra clear
that the topics discussed
in the next section are not your responsibility
to implement in your games.
So let's go back to that flow diagram.
We've seen that the gamer is responsible
for initiating the broadcast, and controlling
when to start and stop a broadcast.
We have also seen that the UI
for broadcast selection is taken care
of by ReplayKit's BroadcastActivityViewController.
So this leaves us with two remaining tasks.
Setting up the broadcast, and uploading video and audio data
to the back-up servers.
So to allow broadcast services to accomplish these steps,
we're introducing a new pair
of app extensions specifically for this feature.
A UI extension, to allow the user to set up a broadcast,
and a non-UI extension for processing
and uploading the audio and video data.
For those of you unfamiliar with extensions,
they're target embedded in your parent application and are a way
to extend your app so that you can execute alongside
They run in a separate process from your parent app,
but can share data with your parent application,
which can be handy or sharing things
like authentication data, for example.
So one thing to keep in mind, though,
is that extensions are limited
in resources compared to applications.
So try to refrain from doing resource intensive tasks
We made it incredibly easy to set
up these extensions via an exotemplate.
These are available as new targets for both iOS and tvOS.
If created from this template,
the extensions will be pre-configured so they will show
up in the broadcast activity control list.
So let's take a look at the two new extensions
that a broadcast service needs to implement,
starting with the broadcast UI extension.
UI extension has a few key responsibilities,
first it's responsible for authenticating the user,
and providing sign up if the user is not yet registered.
Ideally, this should all be done within the extension
so that the user's experience is not interrupted.
But is also perfectly acceptable to link
up to the parent application as long as the user has a way
to get back to the game.
So, during sign up it is also required
that the user be shown terms and conditions
for using the service, and be provided the ability to accept
or decline those terms and conditions.
So as we saw earlier, UI extension is also responsible
for allowing the user to set up their broadcasts,
like adding a title to it, and it can also allow the user
to notify others that broadcasting is
about to begin via social media.
Finally, its final responsibility is notifying
ReplayKit, and ultimately the game
that broadcast set up is complete.
So now that we looked at the UI extension, let's switch
over to the upload extension.
It is responsible for receiving and processing video
and audio data, and uploading that data
to the back-end servers.
So we are aware there are many different implementations
of data processing and uploading to live streaming.
So I won't be getting into any specific implementations during
Instead, if you are a broadcast service, please get in touch
with us, so we can work directly with you.
This way we can implement a solution
that delivers the best experience to our customers.
So going back to our flow diagram, now we have looked
at the broadcast set up and upload.
We have gone through the entire broadcast slow.
And we've clearly broken down the responsibilities
of the game, ReplayKit and the broadcast service,
and that was live broadcasting.
We really feel this is a game changing feature
that you players are going to love,
and for developers it is going to introduce a new way
for your fans to experience your games, not only by playing them,
but by watching them as well.
So before we go, I'd like to end
with a few new enhancements we made to the API
around commentary for iOS.
Namely, support for the front-facing camera,
and also a new capability for the microphone.
So first up, we've added Face Time camera support,
making it incredibly convenient for you to add picture
in picture video commentary to your games.
Player reaction is priceless, and always adds
that special flavor to recordings and broadcasts.
So to enable this in Olympus Rising,
the Flair developers have added a button to toggle the camera
in their fly out menu.
Pressing this button enables the camera displaying a preview
at the top left.
So let's look at the API they used to achieve this.
To toggle the camera on,
we simply set the IsCameraEnabled property to true
on RPScreenRecorder, this turns on the camera hardware,
and populates a new RPScreenRecorder called
CameraPreviewView, which is a subclass of UI view.
Since it is UI view, you can simply add it as a sub view
to your game view, and even freely position it
so that it does not obstruct gameplay.
You can even attach a Adjust or Recognize It to it,
and allows you to position it manually if you wanted.
So looking at the code here,
we're simply setting the IsCameraEnabled property
to True during recording of broadcast.
Then we get a CameraPreviewView instance, then set its geometry
to position it, and then simply add it
as a sub view to your game view.
So now that we have video commentary, we also need
to pair that with audio.
So microphone recording was included last year in iOS 9,
but now we are including the ability
to mute it during recordings as well.
So we all know broadcasters always have something
interesting to say,
but sometimes even they need a break.
So Olympus Rising makes this really easy
by including a microphone toggle in their fly out menu.
Implementing the action
to microphone toggle couldn't be simpler.
The existing IsMicrophoneEnabled property
on RPScreenRecorder can be set to true if you want
to enable the microphone, and false when you want to mute it.
Again this can all be done during a broadcast
or recording session.
Alright so that concludes our session,
and I really hope you enjoyed it.
With the powerful new broadcast feature and now support
for Apple TV, we can't wait to play
and watch your games live on both iOS and tvOS.
For more information, visit Developer.apple.com,
session 601, and during the week, also make sure to check
out our other game technology sessions,
what's new in GameplayKit, SpriteKit and Game Center.
Okay, so that's it for me.
Thanks so much for attending this session and I hope
that you enjoy the rest of your week here at WWDC.
Thanks so much.
Looking for something specific? Enter a topic above and jump straight to the good stuff.
An error occurred when submitting your query. Please check your Internet connection and try again.