Introduction to TVMLKit

We're going to kick off this afternoon's session by introducing you to a brand new framework available on tvOS called TVMLKit.

And this is really kind of continuing what we were talking before lunch. It's really about media apps and you'll see why.

But it's actually this, hold on, that's the app store. But it's actually a TVML based app. The app store is a TVML based app.

And it's also this, the iTunes Video App. In fact, several of the apps that have shipped with tvOS are actually TVML apps under the covers.

And if we take a step back to consider what these apps are all about, it's really a common goal.

It's about browsing and finding and then playing back and consuming content, media content, but we want to be able to do it in a really kind of consistent and still unique way.

Another example is actually Apple Music. As we designed tvOS and the apps that it contains, we actually found some really consistent patterns in the way the apps are built. And so we built then a flexible framework to allow you to build these sort of native apps based on a template system that then can be dynamically populated with your content. So as you're kind of considering how you want to go about developing your app for tvOS, if you find that your focus, or the focus or value, of your app is really around a catalog of content, and especially media content, and if you want a native, familiar and instantly intuitive experience, you might want to actually consider TVMLKit. So going into some more detail, TVMLKit is a template based layout engine that's especially geared towards media apps where people browse and consume media. It enables really rapid development by offering preformed templates that mirror the experience of the system apps on tvOS and this enables you to rapidly create apps that offer catalogs of media content for viewing but with much less effort. It also offers quite a degree of customizability and this allows you to bring your brand into the apps but while still really maintaining a really consistent look and feel with other apps on the platform. So why would we do this? Well first it's to enable some consistency in presentation and experience between different media apps on the platform, both those downloaded from the app store but also those provided by Apple. We also really wanted to help reduce the amount of development work required to bring one of these apps to market, which means of course that you can actually deliver the app even sooner. Many of you already have a web server in place that you can use to provide maybe a JSON feed of content or perhaps serve up TVML documents alongside your HTML ones. And you may also have perhaps a deep strength in your company with some JavaScript developers. And actually TVML apps actually most run on JavaScript. But we wanted to enable the above advantages while still offering a truly native experience for people using the apps and this is basically what TVML offers.

So let's go back to that videos example and see how this template works. Here you can see an example of a template layout that offers a carousal at the top with beautiful wide banners and then shelves of content below. So if we take a look at what this looks like from a TVML perspective, the document simply defines a stackTemplate, which is this overall style of template.

And then inside of the stackTemplate it lists a carousal for the top and then a series of shelf elements below for each row of content. The list of items on the shelf are contained within a collection list.

And then if we look at one of those items on the collection list, it's described in TVML as a lockup, which is a means of actually binding several elements together. In this case it's binding together the image and title elements.

Here's another variation on that stack template. In this case it offers actually a completely different look and feel than the prior but as you can see it's still about having a series of rows. As before, the root of it is the stack template element. The top element is known as a banner which encompasses a background and description element and then below we have a series of shelf elements containing those rows of content.

One more variation to show you on the stack template. It starts out the same way with the stack template root element and then leverages shelf elements for each row of content below but I want to hone in more on that top section, the banner. The banner element in this case contains the background, the title and the subtitle elements. On the right hand side we have what's known as a row element and this contains three button lockup elements handling things such as favorites, ratings or the like. As you can see there's actually a fair bit of flexibility in terms of the layouts for these templates.

And what you're actually doing is establishing native objects such as UICollectionView and UICollectionViewCells, UIButtons and UILabels and you're just describing the layout of them using these TVML documents. Here's the search screen, something nearly universal to all media apps but we've made this one really easy to layout.

Simply declare a search template element. This establishes the keyboard and search field at the top and then the results area at the bottom.

You can then customize what placeholder text exists in the text field using a search field element. And then the results are shown in a shelf element, just as we saw in those prior templates. Different template, the product bundle template, the root element is a product bundle which sets up the overall template style. The background is actually contained inside of this root element providing this beautiful full screen cinematic background. The system is now taking care of this blur effect behind the stack at the bottom automatically. The upper section is a banner, as we've seen with our other templates, but in this case there's a lot of elements though. We've got a title, subtitle, an arbitrary text element. In this case it could be used to describe maybe the film or the genre of the film. We also have a description field which has a special attribute allowing you to make the text focusable and clickable, allowing for the longer description text to be presented in a modal view rather than trying to jam it all into this one screen. Below that we have a row element containing a few button lockups, just as we saw in the prior template.

I'd also like to point out that one of the lockups in that lower shelf contains an overlay element with a progress bar which allows you to show how much of a given piece of content has already been viewed. I'm also now going to show you just a few more templates that are kind of more utility in nature, just to give you an idea of the breadth of what is available. For instance, one for handling ratings of your content. Maybe you need an alert with some descriptions, a couple paths on how to proceed. Or just a simple loading screen. Let's dig a bit more into how this actually works.

A TVML application is simply a native application that uses TVML documents to describe the layouts of the pages of an app.

Those documents generate UI using UIKit objects. Now it's typically based on a client's server model, although that's not an absolute requirement.

You could be generating these documents and these layouts on the device. The logic and lifecycle of the TVML part of the application is actually handled in JavaScript but you can bridge back to native code if you find the need to do so.

You can also actually create custom native elements and then bend them to TVMLKit for use inside of templates. Before we go much further there's a few things though that I want to cover that have commonly come up in our discussions with developers. First, this is not a web browser so even though it follows a client server model and it's leveraging in XML like format to describe the layouts, you can't actually display web pages using the TVML framework.

As discussed, you're actually creating native layouts by describing them in TVML. You can leverage native API's by bridging over to native code.

For example, although you cannot actually directly validate an in app purchase receipt in the JavaScript layer, it's really straightforward to bridge across to native code to do this. Another example might be some custom image processing or compositing that you want to do to your images.

Well you could bridge to native code to leverage core image, for instance, to maybe apply a blur effect or gradient. And finally, although templates can be stylized to some degree in order to align with your brand preferences and colors, the templates aren't infinitely flexible. So if your layouts need to be drastically different than what the templates offer, you might be better served by building out your app in UIKit or perhaps just building out those unique view controllers in UIKit.

What does this look like architecturally? So as mentioned earlier, a typical application that leverages TVML for its layout is architected as a client/server application where the client is a native application running on a tvOS device.

And there's also a web server hosting your media, image, JavaScript files as well as TVML documents perhaps. And these TVML documents can either be static or they can be dynamically generated upon request. You can also generate these TVML documents locally on the device in JavaScript or native code perhaps based on a JSON feed from your web server. The native app fires up as app delegate, just as it does for all apps on tvOS.

The app delegate then is responsible for initializing the appropriate classes from the TVML framework. This offers a bidirectional communication path with native app code throughout the application lifecycle. The TVML framework then bootstraps its main JavaScript file which is often downloaded from a remote web server.

This JavaScript file handles the lifecycle of the TVML component of the app and as required it can call back to the web server for additional resources which might include more JavaScript, images or TVML documents which are then presented to the person using the app in a similar way that view controllers are presented in a native application. So in your app delegate did finish launching with options method you instantiate your TVML context with this pattern described in the app programming guide for tvOS. First you create your TVML application controller context.

You create NSURL object pointing to your main JavaScript file, either on the device or maybe it's on a remote server. And then you set the JavaScript application URL property of your TVML app controller context to this URL. You can then pass along some launch options over to the JavaScript environment.

This is really important for things like deep linking. The app program and guide offer some sample code on how to do this.

Finally, create your TVML app controller, passing at the context that you created, the window, as well as assigning a delegate.

And at this point logics just going to continue in that main JavaScript file. Your main JavaScript file is going to receive some callbacks that are going to help you handle the lifecycle of that JavaScript portion of your app. And I'm going to call up two of them here as they're the ones you're most likely to interact with. On launch is going to get called when your JavaScript file is initially bootstrapped.

On exit is called if you call the stop method of the TV application controller. Note that should the app be, you know, jetsammed or otherwise abruptly quit, this might not actually be called. Now implementation of methods and callbacks to handle other lifecycle events in your app are up to you to put in place though. Now once your app is up and running you're going to want to be able to present your first TVML document so that you've got something on screen. Navigation document is the main JavaScript class that you're going to want to work with here and it's effectively acting as the UINavigationController in TVML. TVML exposes a global variable in the JavaScript context pointing to its navigation document instance. You can't actually create instances of this object yourself. Presenting a new document, you first create a JavaScript object that contains your TVML document.

And then you use the push document method of the navigation document to present that document. And follow this pattern every time that you want to push a new document onto the stack such as maybe you've received an event letting you know that the person using your app has selected a piece of media that they would like to see more details about. Typically you don't tend to programmatically pop view controllers off the stack.

That's usually the result of the press of the menu button and the framework will handle this automatically. But there are, you know, other methods on there as well to handle that case if you need to. Now since we're dealing with documents and resources that are being transmitted over the web, it's really important to take a brief look at how caching works. In short, caching is entirely controlled using HTTP response headers.

So set your values appropriately because you want to ensure that we'll only hold on to those assets for as long as they should actually be valid.

For instance, if you want to make sure that every request fetches a fresh copy of a given resource, you should set the cache control header to no cache.

Note though that this can be really expensive for your backend services so if an asset should actually be valid for longer, make sure you set the value higher or more appropriate. Now since TVML is all about browsing through and selecting media assets for playback, it's really important to know how to play one of those back. The great thing is it leverages the same native player experience that we discussed earlier in that section on media playback. And so that way the experience is immediately intuitive. Let's take a look at how you might load up and playback a streaming asset. First, we're going to create a new player object. Now in this case the player object encompasses both the playback engine as well as the player view itself. Then we create a play list object and assign it to the player's property.

Whether you're only playing back a single media item or you're going to playback several in sequence, you still have to encapsulate it into a playlist.

Then we're going to create a new media item object which is going to represent a single video as well as its metadata. There are some exposed properties for setting the metadata such as title and description. Now very often you're also going to want to offer people the ability to resume a video where they left off. Well you can do this by setting the resume time property on your media item object.

Then we're going to add that media item object to the playlist and then trigger presentation of the player, beginning the playback experience by calling play.

And that's it. The video player is going to be presented and the video will start playing back.

Now what if you want to call into the JavaScript environment from your native code? Well it's actually quite straightforward.

You might want to do this to update a variable, trigger an event or maybe just pass some data across. Well the way to do this is by calling the evaluate JavaScript context completion method of your TV application controller object that you established when initializing your TVML app.

This is going to allow you to evaluate blocks in the JavaScript execution queue. The method calls for two blocks or functions to be passed into it. The first is going to provide you with the context in which the JavaScript will be executed.

And in this case we're going to use it to evaluate a string of JavaScript using the evaluate script method. This method does return a JS value object so if you're expecting result from your expression that you would like to inspect or otherwise use, make sure you capture that.

The completion block simply hands back a Boolean value indicating whether or not the provided block was actually evaluated fully.

Providing the path back from the JavaScript context into your native code is a bit more complicated though as we don't expose any methods or properties into that context by default. So during the initialization of your TV application controller you're delegates going to receive a callback appController:evaluate AppJavaScript:InContext:. This is going to be your opportunity to add objects and classes to your JavaScript context. And you can do that by calling setObject:forKeyedSubscript: against the provided context and we'll take a look in more detail on this now. First, we're going to declare a protocol that conforms to JSExport.

The methods and properties that you list in the protocol are going to be interpreted by JavaScript core to be exported into the JavaScript context.

In this example we've named it my JSExportProtocol and we've declared a single function that we're going to use to write some strings and to NSLog.

Now you could do this for a class that already exists, just pick a few methods and properties and expose those. But in this case we're just going to create a small class that implements the protocol that we've just defined. And in this case we're going to call this new class MyJSBridge and we'll declare its conformance the MyJSExport Protocol that we've just defined. This simple class has just a single function implementation that takes the provided string and then logs it to the console using NSLog. It's really important to note that these functions are going to be executed on the JavaScript context thread.

So be careful to dispatch to a more appropriate thread when necessary, especially if you're expecting a change in UI. Now having established this protocol in class, we're going to implement the appController :evaluateAppJavaScript :inContext: method that we discussed earlier.

So in your TV application controller delegate, implement this method. You can call said object for keyed subscript against the provided JSContext object which is going to allow you to then an instance of that my JSBridge class that we created.

And then also provide the object name that you want it represented with inside of the JavaScript context, in this case JSBridge.

Having done this you can now actually call the log something method from your TVML JavaScript context. All of the functions that we defined in that protocol are now effectively accessible from the JavaScript Context. And that's how you can provide a bridge back from the JavaScriptContext into your own native classes, it will go both ways now. Now that you've written the app, how are you going to analyze its performance? Well like any other app on tvOS, you of course can leverage the Xcode Debugger for debugging native code and then instruments for your performance tuning. But for debugging and tuning your JavaScript code, you can actually attach to the device using the Safari Web Inspector, just as you would for debugging web pages in Safari or web views on iOS.

In Safari and OS 10, under the develop menu you can actually attach to a running process or even set it up so that it will automatically display an instance of the web inspector whenever it detects a JSContext to debug. This allows you to do things like view the JavaScript console log or perhaps set break points in your JavaScript code. So between instruments and the Safari Web Inspector you're actually able to easily debug and analyze your code throughout your entire application. In this section we discussed TVMLKit, a brand new framework that is part of tvOS.

TVMLKit enables the rapid development of your media centric applications by allowing you to leverage native UIKit based templates describing the layouts using TVML. The native media player is also available to you in TVMLKit and we've discussed how to playback a piece of HLS media.

Finally we discussed how the JavaScript application layer is setup and configured and how to enable interaction between your native and JavaScript code and then how to debug and analyze your app. For more information, please take a look at the listed reference guides.

We've also got two sample projects for you. TVML catalog, which walks through the various templates available.

As well as TVMLAudioVideo which demonstrates how to play back audio and video media from within TVMLKit.