CloudKit JS and Web Services

CHRIS EDSTROM: Good afternoon. My name is Chris. I'm the manager of ICloud Web Services and I couldn't be happier to be here today to tell you about the number one feature request we received last year for CloudKit, which is web services.

So this talk expects a little bit of familiarity with CloudKit I will be giving you a brief overview of the architecture but if you want a more deep dive we had a couple of sessions last year. You can go back and review those sessions.

So in CloudKit, all of your data is stored within a container. Inside of this container we have a public database for your app data and a private database where your user date is stored. In the public database, you have the default zone, which has records, and in the private database, you also have a default zone, but you can also create custom zones which add sync semantics which allow you to answer the question what has changed since a given point in time. Of course, you can create records in both of these zones. From the API perspective, we have the CloudKit Server, and on the native side we have the CloudKit Framework, which your app uses to interface with the server.

We are introducing CloudKit Web Services today, which we have used to bill the Notes Web app, and today you can use to build your web app.

So what exactly is the CloudKit Web Service? Well, it's three things. Number one, it's a JSON interface to CloudKit.

Number two, it's a web sign-in with Apple ID and number three, it's a JavaScript library.

So let's take a look at the JSON API features.

So your full public and private database access, you can, of course, create, read, update, and delete records. You can upload and download assets.

You can query your data.

You can create subscriptions and receive call backs when your data changes, you have user discoverability where you can find users' real names. You can do sync and we provide authentication. So this is a long list, and in fact it means that we have feature parity between the native API and the JavaScript API.

So everything you can do in the native API you can do through the JSON API and we have gone so far to make sure things like field names and parameters have the same names in JavaScript.

And JavaScript promises to provide completions.

Your code ends up looking similar between JavaScript and Swift. I know many of you in the audience may already have a CloudKit application, but if you don't, let's look quickly about how to get started with a new CloudKit application. So the first thing to do is to create a container.

You can do this either through the WWDR portal or through Xcode.

Once you have the container you need to create a schema. You can do this either through the iCloud dashboard or your native Apple create it on demand.

So now you have your CloudKit application up and running.

Let's enable web access.

This is a three-step process. Number one, you need to generate a web service API token.

Number two, you need to set your login call back mechanism, and number three, you need to define your domain restrictions. So let's take a closer look at these three steps.

So here is the CloudKit dashboard. As you can see, we have added a new pane called API tokens and I have already created my first token here.

So at the top is the token ID, which is what you pass in when you make a web service call, below that I have chosen post message as my login call back. It is for JavaScript applications I will get a JavaScript call back when the user logs in.

Alternatively I could have chosen to have a URL redirect.

Below that I can set the origins that are allowed to access my CloudKit data through web services. I have here chosen that any domain can access my data but for added security I could have restricted to viewer domains so now we have enabled web services.

Let's get some users to log in.

So authentication is handled transparently. What I mean by that is when a user calls an API that requires authentication, we will return a specially formed payload with a redirect URL that you can send your users to. They then authenticate with an existing Apple ID or they can create a new Apple ID right in the sign-in flow.

And to protect users' privacy we don't provide user name or password to your app.

Instead we give you a stable anonymous identifier for the user. So let's put all of these things together and let's build a quick example app.

So we are going to build a tasks app. It stores tasks, and the com dot example dot tasks container.

So let's create our first task.

So what I will do is I will call the records modify API, and pass in a JSON body. This JSON body says to create a new task it has a unique identifier task one and the value of the task is buy milk.

Let's go ahead and send that over to the server. So we didn't have an authenticated user.

So in the response we get a redirect URL where we can send the user to authenticate. Let's go ahead and follow that URL. So here is the sign-in UI. You will notice it has the app name and the app icon and the ability to create a new Apple ID right here.

We already have one, so let's go ahead and sign in.

Now, once the user is signed in, you will get a session ID back to your application through the call back mechanism you defined earlier. So we can go ahead and append this session ID to the URL and send the same request again.

Much better.

Now we receive back a successful creation. You can see that the record name is there, the change tag is there, the fields are there and some more metadata about the record.

This is a lot of JSON parsing and you have to worry about error handling and throttling and we want to make this API as easy to use as possible. So we are proud to introduce CloudKit JS a wrapper on top of the JSON. To talk about this I would like to invite Onar Vikingstad to the stage.

I'm.

ONAR VIKINGSTAD: Engineering Manager at iCloud. And I'm here to talk about CloudKit JS. CloudKit JS lets you connect to CloudKit from JavaScript and it's a small self-contained library that works well with any framework or library you might already be using on the web.

It's also important to note that this is a low level transport API which means that it doesn't really intend to be a data store. It doesn't allow a lot of abstraction on top of the JSON API so if you are already using a data store please keep using that and you should be able to plug CloudKit JS straight into it.

Also it allows for an easy transition from CloudKit Framework. So you might already be using CloudKit Framework on iOS and OS X in the terms and concepts, method names and class names are going to be the same so it will be an easy transition.

That said, there will be some differences. This is the web, after all, and it's JavaScript so there will be a few differences, for instance we are using JavaScript promises for all of the asynchronous behavior and we will see examples of that later on.

In terms of browser support, we support all of the major browsers you would expect Safari, FireFox, Chrome, Internet Explorer and the new Microsoft Edge.

We support Note JS if you want to do server to server connections on behalf of the user. So before we start looking more at the API let me give you a quick demo. So what we are looking at here is the CloudKit catalog sample code project.

This is available both as a downloadable zip file, you can get this on the developer website, but what we are looking at here is the hosted version which is also available. So it is providing live code examples that you can execute in any browser.

You will see a list of features on the left. We are looking at the ReadMe right now. This is giving you some instructions on how to get started on CloudKit JS. Let's move to the first section called authentication. So you will notice each of these sections have a description on top and this example is telling you how to use authentication, and below that you have a code example. So this code snippet specifically is checking for authentication state of the user. If the user is not already signed in, it's going to present a signed in button. And you can actually go ahead on top here and run this code. So if I click this run code button, it's going to execute this code, and you will see it scrolls down through a section and it tells us we aren unauthenticated user at this point and we look at the sign-in button. I can go ahead and click the sign-in button.

I type in any Apple ID and password.

And you notice the window closed. And that was the Apple hosted authentication page. So at this point your web application is signed in. This is the sample app and you see we have the name of the user, first and last name as well as the sign-out button instead of the sign-in button. This is another feature of CloudKit which is discoverability. If the user has opted in, the user can choose to share first and last name with a developer as well as being discoverable in the container for other users to find them.

So now that we have the first and last name we are using that here to populate the top right corner. All right. So now that we are signed in, we can do some operations so let's go to records.

And this is showing you how we can save a record in the private database.

So we see a few fields on the top here. We have the record name, which is the unique identifier of the record.

We have a zone name which is the default zone but it will be any custom zone that you set up, and if you have three custom field names that are defined in the CloudKit schema.

So in this example, we have a name field which is the string value. We have a location field, which is the geo location value and then we have an asset field which lets you upload a file. So I will go ahead and choose a file. I will choose this green field landscape J Peg.

And I'm going to go back up on top and give this the name landscape 1. This is the record name which is the unique identifier. And I will change the name here to just called it green field. So now that I have the field filled out I can run the code and it will save the record to the database as well as upload assets to asset storage.

So you see when it's completed we are left with a result and this is actually what the server responded with, effectively with the JSON we are representing it here in the table. We have a few metadata fields as well as the three custom feel fields. The green field, the location and the link to download the asset. So just to show you this actually got stored we can go to fetch record and I can type in landscape 1, run this code, and we have the same record again. That's how simple it is to save records with CloudKit, with CloudKit JS. Now, let's go and do another feature called querying. So to do that, I will first go ahead and sign the user out.

So queries allow you to get subset of data from the database, and in this code example, we are showing you how we can query the public database and if you set up the security roles appropriately on the schema, you can actually query for data in the public database even if the user is not authenticated that we are showing here. I can go ahead and run this code and this example is basically using the current location in a Monscone center to send a query and sort results based on proximity to where we are currently. I can go ahead and run the code.

And I get back two records. The first one being Iceland, and the second one being Italy. And Iceland is closer to where we are than Italy, so that makes sense. And so I'm not going to go through all of the additional features but we encourage you to check it out. You can go to developer@Apple.com/CloudKit and you can run this on any browser yourself.

So CloudKit is the foundation for a lot of our new services at Apple including iCloud drive and iCloud photo library.

This year we also introduced a notes application for iOS and OS X. This is also coming to the CloudKit platform, but in addition to the notes application on native we have a notes web application on Apple.com. In fact, it's using CloudKit JS in exactly the same way as you as a developer can now use it. There is no additional Apple magic here.

So hopefully you are as excited about CloudKit JS as we are and see a lot of new possibilities here and want to know how to get started.

So let's take a look.

So first off, we need to make sure you insert a script tag on your web page, and you do this by pointing to the CD unhosted version of CloudKit JS. So this will get CloudKit JS included, and then secondly, you need to configure the CloudKit instance. So you are going to have the CloudKit name space available and you can call the CloudKit configure method and you give it a container identifier that your application is using. The environment you want to target, either production or development, and then you specify the API token that you saw Chris generate through the CloudKit dashboard. That's all you need. At this point you are ready to start making CloudKit API calls. Next, you probably want to authenticate your users just like we saw in the sample app.

So on iOS and OS X this is straight forward with CloudKit.

All you need to do since most users are already signed into iCloud on the device is to call the fetch user method on the container and you get back a user record and this user record is going to have a stable identifier.

So this is all you really need to do to authenticate users through CloudKit on the native devices. There is no explicit login, there is no UI. It's all seamless. On the web, however, it's going to work a little bit differently in that the browser is not already going to be signed in so you have to provide a location for the sign-in button. Like we saw on the sample app, clicking it will open the new window. There is an Apple hosted authentication page and you notice on this page we have both the iCon of the application as well as the name of the application, and this is coming out of ITunes Connect and it's published in the App Store. And the user types in their user name and credentials. This obviously only goes back to Apple. There is also a two-step verification if the account is set up for that. And when this process completes, the window is going to close and it's going to send back a post message to the opener window and it will contain the CloudKit session. So at this point your CloudKit JS, your web application is going to be able to start making authenticated calls to CloudKit.

So to implement authentication on your page, you first have to provide a location for a sign in button. So you set a development to give it the ID Apple sign-in button.

Secondly, you have to call the method called set up off, and this is going to check for the authentication state of the user. So if the user is already signed in, a user info object is going to come back, and that's all you need to look for. In addition to user info object is going to have a record name and this is the stable identifier that we just looked at. So this stable identifier is going to be the same now on the web as it is on iOS and OS X so you have a way of referencing the same user across all of the platforms. So in addition to checking for authentication state, you also need to handle what happens when user signs in or signs out.

So you call when user signs in on the container, and you get back a promise, and it's going to resolve as soon as the user signs in. And it's exactly the same for when the user signs out.

So beyond those default capabilities in CloudKit JS around authentication. We also provide toggles and ways to customize how you handle authentication. So this next example is showing you how you can actually persist the AUTH between sessions so CloudKit is not going to write cookies on your domain so if you want a cookie to be written to store the AUTH session, you have to set the persist property to true in the configuration.

Now, instead of cookies, you might actually want to store the session in some other way. So this is showing you a couple of hooks in the configuration. So we have a put token and a get token method and these are going to get called anytime for put token anytime cloud JS receives token from the server. So one example here might be you want to store the session in local storage or perhaps you might want to store it in your own server so you can make server to server calls based on behalf of the user.

So next let's talk about record operations.

And just like Chris showed you earlier with the JSON.

A record in CloudKit JS is basically an object like this. So we have a record with a record named task 1.

We have record type tasks, and we have some metadata fields created and modified which will contain the user that created and modified the record as well as the time stamp. The record change tag, which is the version of this record for conflict resolution, and then comes all of your custom fields below that that you have defined in the schema. In this example we just have one task name, it's a type string and the value of buy milk. Now, obviously you might want to store other types of fields than strings so this is giving you a comparison of what those look like compared to iOS and OS X.

So in CloudKit framework we have a string but in CloudKit JS we just use JavaScript strings. Same thing for number, for NS data we use basic code 64 and coded binary. NS data we have a JavaScript number which is unit time in milliseconds.

So for the more complex objects like CL location, we have a location object, and a reference object and also an asset object, and we ask you to look in the documentation for how each one of these are defined. So in terms of creating a record, you can go ahead and define one of these object literals. In this example we're just saying record type is tasks and we have one field task name with a value of buy milk and you can go ahead and call the save record method on the database object. You notice though in this case we didn't actually give it the record name. So the record name is unique identifier of a record. So by not specifying that the server is actually going to go ahead and generate one for us. Also we don't specify the string type for this particular field, so CloudKit JS will do some type inference here and it will figure out that this value is fully a string to you have to specify this for all of the basic types we have.

So next we want to talk about queries, and this is going to work a little bit different on the web through the JavaScript API.

So on iOS and OS X in foundation framework we have NSPredicate.

And NSPredicate is great but it's not really that natural for the web and it doesn't really conform well to the JSON API we are exposing through the web services so we have something that looks very similar to Java literals for records in that you just define a query like this. In this example we are saying record type is tasks and we call perform query on the database, and we are going to get back all of the records of type tasks. Now, you can do something a bit more sophisticated. So in this next example we are providing some options to the perform query method. So we are saying this hard key is task name. We are only going to get back the task name field from every record. We are limiting the result set to ten and we will limit records in the all task custom zone. So queries also allow you to go and specify a filter.

So in this next example, we are saying still, you know, we want to look for record types of tasks but we only want to get back tasks of priority one.

And we call it perform query and we get back all task with priority one.

You notice here we are using the equals comparators and we have a lot more comparators available and it depends on the field type you are querying on so look in the field documentation for where each one of those are defined. As you saw earlier, we were limiting the result set and so the question is how do you paginate through the results. So this is showing one example of that. So results limits at 10. So we will get back the first 10 records when this promise resolves but there is more coming property on the response object. So this is telling us more records to fetch beyond the 10 we just got so we can pass the response object back to perform query and CloudKit JS will do all of the work here for us in terms of fetching the next set of ten records from where we are.

And in this example we are showing should we can chain these promises together. So next assets. So assets work a little bit differently in CloudKit than the other field types.

So assets are actually not stored with the record itself, assets are stored outside in something we call asset storage. So the record simply has a reference to the asset and asset storage.

The good news is CloudKit JS handles this seamlessly for you.

In this example we have a web page with input element and lets users select a file like this one in the CloudKit catalogue app.

So when a file is selected, the handle file select function is going to get called, and all we need to do is just get a reference to the file, and we can use that as the value for the asset field we are saving. So CloudKit JS is here going to do the work for us in terms of uploading the asset to asset storage and then save the record in the database with a reference to the asset. Downloading assets even simpler in that when you are asking for a field that has an asset, you are going to get back a download URL. In this example we are fetching one record, a user record, and you notice here the value has a download URL property, and you can go ahead and fetch the bits from there or maybe use this as the source of your image element, for instance.

So another great feature of CloudKit is subscriptions and push notifications.

And all pushes to Apple devices go through Apple Push Notification Service.

Now, traditionally you had to set up your own third party server for this to work. So you had to get your own server with your own certificate with your own server logic and you had to push, send pushes through that way.

However, with CloudKit, there is no longer necessary. You can simply set up CloudKit Subscriptions and these act like triggers in a database. So these are going to look for changes in data and send out pushes accordingly when they happen.

And now with CloudKit JS the web will be another client in this mix. So we are going to able to receive the exact same pushes to the web as we are to iOS and OS X. So to receive pushes you have to set up subscriptions.

This first example is showing you a zone subscription.

To subscription type is zone, subscription ID is change tasks and we give is zone ID of all tasks. I would go ahead and save the subscription with the database.

So this will start looking for changes so any time something changes in the all tasks custom zone. It will send pushes to all clients connected to the same account.

So subscriptions also allow you to base them on a query so this next example we are setting up a subscription, a query subscription that will fire anything that's created, updated or deleted in the all task custom zone and then we specify a query so this particular query is like when we fetch for records any time a task of priority one is created, updated or deleted, a push will get sent out and that's all we have to do. So on the website, you have to implement two things, first you have to implement register for notifications. This will go ahead and fetch a token from the server that we can use with Apple Push Notification Service and also park the connection with Apple Push Connection Service.

And secondly, you need to call add notification listener which is going to get called any time a notification is received on a web page.

So last, let me cover a few best practices with CloudKit JS. So as we saw earlier, you should dynamically link to the CDN hosted version of CloudKit JS. This is important to you get the latest updates and security fixes as we make them available and it also insures a low latency delivery of this asset through our vast accomplished distribution network and it's all for free which is great.

Secondly, you should consider loading CloudKit JS asynchronously on your page. This is a good suggestion for any third party library you might load on the web page and it will avoid any blocking and impact on user experience and last you want to make sure you handle the request responses so the new free limits of CloudKit are really generous including the limits for requests.

That said you might be retrieving bursts of traffic to your website, so you want to make sure that your client is properly backing up. If it's retrieving requests through all responses.

So you can also look into how it can batch multiple operations in a single request and we do have a batch API with CloudKit JS and if you are interested in topic you should go to the CloudKit tips and tricks session tomorrow. They will talk more about the request throttling across all of CloudKit and not just web specifically.

So that's it. That's CloudKit JS. We really look forward to seeing what you are going to be building with it. Please stop by the labs with any questions you might have. And now I will hand it back to Chris.

CHRIS EDSTROM: Thanks Onar.

So in summary, the CloudKit Web Services provide a full parity interface to interact with your CloudKit data on the web.

They are often on a per container basis. We handle authentication, metadata storage, and asset upload and download. You handle hosting your static assets and your JavaScript application code.

And finally we have brought you CloudKit JS to make this as easy as possible. You can visit us on the web at developer.Apple.com/cloudKit and for any questions you have you can visit the Apple Developer Forums or Apple developer technical support. If you still have questions beyond that you can reach us at CloudKit@Apple.com. There are a few related sessions. We had what's new in CloudKit yesterday which is available on line and tomorrow as Onar mentioned we have CloudKit tips and tricks. I hope to see you there. And thank you very much. [Applause]