Networking with NSURLSession

LUKE CASE: Good morning! Thanks for coming out and welcome to "Networking with NSURL Session." I'm Luke Case and I'm an engineer on the CF Network team. We have got some great stuff lined up for you guys this morning. First, we're going to talk a little bit about app transport security and following that we will talk about some new protocol support we've added in NSURL Session.

Now, we all heard about watchOS on Monday. We'll talk about the NSURL Session features we've added for watchOS. Following that, we will wrap things up with some API changes and some new additions we've added to the NSURL Session. Now, to begin, I would like to start with a little bit of background on NSURL Session and the HTTP protocol. NSURL Session is a networking API that is mainly used to download content, HTTP content, from the web.

It has a rich set of delegate methods that can be used for authentication and other important event handling. And one of the greatest features about NSURL Session is that it allows you to do networking even while your app is not running via the background download APIs. Now, for those of you that aren't familiar with NSURL Session, it's recommended that you go back a few years to the previous WWDC sessions on the introduction of NSURL Session. So now let's talk a little bit about the hypertext transfer protocol. It's a very well-known protocol and many of you x probably are familiar with it. Essentially at its core, what you are doing is you are making requests to the server and you are pulling in data in response. Now, HTTP in itself, as some of you may know, is sent out over cleartext. So it's inherently insecure.

And in today's environment, there's many hostile parties going after the data that's leaving your app and going out onto the network.

Now, there was a time when using cleartext HTTP alone was perfectly reasonable. But that time has passed. Now, fortunately for all of us, this problem has been solved for many years and there's a tried and true solution known as HTTPS. Now, HTTPS is essentially HTTP layered on top of another protocol known as transport layer security. And transport layer security performs a multi-leg handshake using public key cryptography and, when complete, creates a secure connection. Now this connection is considered secure because of three properties. The first being that the data that's leaving your app goes over the network encrypted. So it can't be read.

The second that it provides message integrity so the message can't be altered without detection.

And finally, the third, it provides authentication so you can actually prove the identity of exactly who you are talking to. Now, NSURL Session has excellent HTTPS support embedded within in it, and so often in your client code, it's as easy as using HTTPS instead of using HTTP. Now, do note that some additional server support is required, but HTTPS is supported by most server vendors today. Also keep in mind that most data should be considered sensitive, and this is because even if you might not think that the data that's leaving your app going out onto the network is not sensitive, your customer may, in fact, think it is. For example, if you have a TV streaming app, you may think, well, it's just TV. You know? He's just watching TV. He or she is just watching TV, but to them, it may mean that they don't want people to know what they are watching, essentially. Let's put it that way. So -- now, why is it important to use HTTPS? Well, it's essentially, it comes down to this: your customers trust you with their data and their privacy.

And we at Apple, we really want to work together with you guys to make sure that we build upon and maintain that trust. And today, I'm proud to introduce App Transport Security. Now, App Transport Security is a new feature from Apple in iOS 9 and OS X, El Capitan. And essentially, at its core it helps prevent accidental disclosure of sensitive customer data.

Now, ATS also strengthens the default policy of NSURL Session.

And perhaps the most important aspect of that strengthening is that now NSURL Session, by default, will disallow cleartext HTTP loads. It'll only use HTTPS connections. Now, these connections that it does make uses today's best practices. So ATS places restrictions on TLS versions, cipher suites used, certificate trusts, and certificate key sizes that are used in that transaction. Now ATS is simply configured via your apps Info.plist. And you can see an example here.

Essentially what we are asking you to do is to declare your intended network behavior within your app.

So at its core, ATS wants you to really describe what you intend to do with the network, and we really want you to worry less about the security of your app and rely more heavily on the system to do the right thing. So describing your network transaction is, of course, easiest when it's all secure.

So if your app exclusively uses secure connections, and they only use best practice properties, then there's nothing else you have to do to configure ATS within your app.

So if you are writing a new app, this is exactly where we want you to start, and if you have an existing app or you are supporting legacy code, this is where you should aim to be. Now, we understand that this may not be always the case that you have a server that can comply with these new restrictions on TLS versions or cipher suites, so we allowed that to -- through the use of exceptions. And essentially, with ATS you can declare exceptions and let us know what versions of TLS you want to use, or if you want to opt out of forward secrecy, or other options. Now, we do understand that existing apps may have different constraints. Let's say for the most part you do use secure connections, but in some cases, you have servers that just don't support HTTPS or the best practices. So, for example, if you have a media server that doesn't support HTTPS and only HTTP, you can easily describe this behavior using an exception.

Essentially all you have to do is declare which domain needs to load over HTTP and NSURL Session will still allow that cleartext load.

Now we also know that some of you may be supporting a general purpose web browser or another app that loads URLs based off dynamic user input, and we've allowed for that case. In this case here, you can't possibly describe what secure connections you need ahead of time, and so you can simply opt out by declaring an allow-all policy.

Now the allow-all policy can be used to allow NSURL Session to load any HTTP or HTTPS resource.

Now, do note that this is a useful tool for debugging that I believe is already out on the Internet. So we ask, if you do use it for debugging, to do it temporarily for obvious reasons. Now, even if you are in this state, you can still protect specific servers and domains.

So you are in this state where you are allowing everything, but you have your own servers, say, that you pull the configuration data from or metadata from, and to do this, again, is just an exception within ATS.

You simply declare this domain that you want to protect as an exception to the allow-all policy. And this will allow NSURL Session to continue to protect this resource by only loading over HTTPS and with best practices.

Now, the SDK has been out since Monday, and there's probably a few of you that woke up at this awful hour, just to find out what ATS is all about and why your network loads are failing.

Well, we understand that and we want to work together with you to get you up and running.

So, do note that ATS is only active if you build against a current SDK. If you are targeting the previous release, ATS rules do not apply. NSURL Session will transfer all HTTP resources or loads URLs to HTTPS automatically. And, again, as I said earlier, some of you already know about the allow-all key, and we ask you to use this temporarily to find out if the new network failures in your app are because of ATS or not. And this will kind of help you narrow things down. So the next step, if you allow-all ATS loads, or allow-all loads, and you've determined that the problem is ATS, the next step would be to log the NSURL Session errors that you are seeing, so you can try to determine and narrow down which load is actually failing, and what the underlying error is.

There is also a great debugging tool that's an environment variable known as CF Network Diagnostics.

If you set CF Network Diagnostics to level 1, all the failing URL loads -- so, all the loads that failed will log the URLs and the underlying TLS error.

And so what you can do in the next step is take that underlying TLS error, and look up and secure transport.h, and see exactly what the underlying issue is. Usually it's a failure in negotiation between client and server at the TLS layer.

So now, do understand that this is a new API, and it's still under development. So we strongly encourage you guys to pay attention to the seed notes and release notes, and also, please file radars so we can track any issues that you guys are hitting and help you guys get up and running. So I hope it's clear that the time for secure networking is now. And we really want to work with you in order to protect your customers' data.

So, again, if you are writing a new app, start with HTTPS and try to get your servers up and running with the best practices. Now, for those of you with existing apps, we ask that you first start by moving what you can to HTTPS, and for the places where you can't, go ahead and use the exceptions that are available through your Info.plist.

Now, do keep in mind that your customers trust you with their data, and it is truly sensitive in all cases. And so Apple wants to work together with you to provide a more secure environment for your customers. So, again, please give us feedback via radar, come see us down at the lab after this session and tomorrow. We really want to work with you to protect our customers.

Thank you all for being here and I will be followed up by Andreas who will talk about new protocol support and NSURL Session. Have a great conference.

ANDREAS GARKUSCHA: Thanks, Luke. Good morning, everyone! My name is Andreas and I'm going to talk about new protocol support in NSURL Session.

Yes, NSURL Session supports HTTP/2 protocol.

Your apps are ready to communicate using HTTP/2 and it's very easy to adopt. Let me play it one more time. All right! If you are already using NSURL Session in your code, you are automatically a part of all of that. Future of the web, major milestone in the web's evolution, your apps are running faster. You don't need to change your source code. Everything works automatically.

So today, we are going to talk about three things.

Why do we need a new protocol and what are the common problems of the current HTTP/1.1 protocol? We will learn about the most important HTTP/2 features.

And finally, we will talk about HTTP/2 protocol adoption in your apps.

So, why another new protocol? We already have a lot of protocols for every kind of communication.

The reason is that most of the protocols were designed many years ago for the needs of their time.

Look at one of the first Apple websites.

Today's needs look significantly different than those 15 years ago. So it is time for an update. HTTP has been around since the beginning of the web, and it is not a secret that it has many issues.

The most famous HTTP issue is the problem of only one outstanding request per TCP connection.

The solution for this was HTTP pipelining, but HTTP pipelining is not available on all servers or networks. In fact, it's disabled on most popular desktop web browsers.

Another solution was to open multiple connections to a host.

This could help to fetch multiple resources faster, but together with other things like textual protocol overhead, the lack of header compression, it just adds up to higher system requirements and lower performance on both the client and the server. Last year, we introduced SPDY Support in NSURL Session.

SPDY was an attempt to make the web faster. It was an experimental protocol, however, it was chosen as a basis for a new version of HTTP. The specification for the new protocol version went through the IETF standardization, and officially got an RFC number assigned last month.

So today, as you already know, NSURL Session is extended to support HTTP/2 protocol. Let's take a look at the key differences between HTTP/1.1 and HTTP/2.

As opposed to HTTP/1.1, HTTP/2 opens only one TCP connection to a host.

It's network friendly and requires less system resources on both the client and the server.

HTTP/2 is fully multiplexed. That means that a new request does not need to wait until the server sends the response for the previous request.

HTTP/2 has request priorities, so that more important resources can be delivered at a higher priority to the client.

Let's take a look at how HTTP/2 multiplexing resolves the Head-of-Line Blocking problem.

We have three requests for resources on a web server.

We sent out the first request and we get a response.

Only after this, we can send out the second request and get the second response. Same happens with the third request. Now, with pipelining enabled, we can send out all the requests, one after another, without waiting for the previous responses to arrive.

But we still get the responses in the same order.

And you see that the first response for the image in blue, blocks the following two responses. With HTTP/2, we have the same three requests, with different priorities.

We still can send out all the requests at the beginning, but we are receiving all the responses at the same time. Moreover, the requests with the higher priority, I get and deliver faster to the client.

You see that the second response was a medium priority and the third response for request was a high priority, arrived prior to the first response even though they were scheduled later. The image does not block them anymore and this is great. Great for your applications and the performance.

Let's continue with the comparison. HTTP/2 is a binary protocol.

That makes data processing and parsing faster. HTTP/1.1 does not use header compression. SPDY also could not use header compression because of a security exploit.

HTTP/2 uses HPACK, a more secure mechanism for header compression.

Let's talk about HPACK.

HPACK header compression is based on two tables, a static table and a dynamic one.

The static table contains the most used HTTP headers and is unchangeable.

The headers, which are not included in the static table, can be added to the dynamic table.

The headers from the tables can be referenced by index.

As an example, you see a simple HTTP/1.1 request.

Highlighted is the data which is going to be sent to the server. And here's HTTP/2 representation of the same request. So let's encode this request.

The pseudo headers, method, scheme, and path, can be referenced using the static table.

The authority header is included in the static table, but without its value.

So to encode this request, we need three bytes for the first three headers, plus an additional byte, which tells that we want to add the authority header to the dynamic table and the value of the authority with its length.

And this is what is going to be sent to the server plus additional overhead for the header frame. Now with the second request, and you see that the authority header goes in the dynamic table.

So with the second request, HTTP/1.1 would send the same headers over and over again.

In HTTP/2 case, in that particular case, we can reference all the headers using the static and the dynamic table. We are using only one byte for each header. It is a huge savings of the bandwidth and it's remarkable how few bytes are needed to encode a request or response header in HTTP/2.

So let's talk about what you need to adopt HTTP/2 protocol in your apps.

There's not much work to do. HTTP/2 protocol is seamlessly integrated into NSURL Session API.

If you are already using NSURL Session in your code, your apps and OS X programs will get this functionality automatically.

You will not need to write any new code or provide any additional configuration to turn it on. Let's take a look at the source code example. This source code looks exactly like the code you guys already have in your apps. You see, there's no difference, no new configuration flags. It just works. Yes, you only need an HTTP/2 server.

But it's okay.

Your apps are ready to communicate using HTTP/2 protocol. If you do not deploy an HTTP/2 server yet, then your apps will use HTTP/1.1 directly, or the best available protocol will be selected automatically for the network communication. Once you start using a web server which supports HTTP/2, there is no additional work needed.

Your apps will use HTTP/2 protocol automatically.

Please keep in mind that NSURL Session supports HTTP/2 protocol only over an encrypted connection.

And that your HTTP/2 server requires to support ALPN or NPN for protocol negotiation.

Currently at Apple, HomeKit remote access via iCloud is using HTTP/2 protocol for communication between HomeKit accessories and iCloud.

Many big companies are already using HTTP/2 protocol. Google provides its services in HTTP/2. Twitter is using HTTP/2 as well.

There are many HTTP/2 open source web servers out there, and finally, some CDN service providers are working on the HTTP/2 protocol support today.

We worked hard to provide HTTP/2 protocol support in NSURL Session so that you guys can adopt and use it as easy as possible. HTTP/2 is available today in WWDC seed. It's seamlessly integrated into NSURL Session API and it's enabled in Safari, on OS X, 10.11, and iOS 9.

Thank you.

And now I would like to invite Dan up to the stage. Dan? DAN VINEGRAD: Good morning, everyone. My name is Dan. I'm a software engineer on the CF Network team at Apple, and the first thing I would like to talk about today is using NSURL Session on watchOS.

With the WatchKit SDK that was released alongside watchOS 2 in beta earlier this week, I'm pleased to say that HTTPS loads are fully supported on watchOS.

And this means that everything we've already talked about today, like App Transport Security and HTTP/2, are built into this support.

One major difference between using NSURL Session on watchOS and using it on other platforms is that underneath the hood, we actually will use the best connectivity mechanism that's available to us at the time. And this means that in most circumstances, if the users' watch is nearby their paired iPhone device, then we will actually leverage the Bluetooth connection between them, perform the HTTP loads on the phone itself, and deliver the results back to the watch over Bluetooth.

If the user happens to be out and away from their phone, but with their watch, and the watch is connected to a known Wi-Fi network, then we can actually use that Wi-Fi network directly.

But the good news for you is that this is all abstracted away from the API itself. You can use the API just as you have been on other platforms. You don't need to worry about how we are connecting. It just works like magic. So, with that said, there are a few best practices and things to keep in mind when using NSURL Session on watchOS. The first is that you should really try to just download the minimal size assets that are actually required for your app to function. Keep in mind that the watch has a very small screen. So if you are downloading images you don't really need to download the full resolution images that you would want to display on an iPhone 6 Plus or a Mac with a retina display. The screen's a lot smaller; you can download smaller images. And also keep in mind that the watch has a lot less processing power than a phone or a Mac, and additionally will often be limited by the bandwidth and latency constraints of the Bluetooth connection to your phone. So we're not really going to be able to get you bytes as quickly to the watch as we would on other platforms. So keep that in mind as well. Another thing to note is that apps on watch will generally run for a much shorter period of time than iPhone apps or definitely Mac apps. You will mostly be limited by how long the user wants to sit there standing with his wrist raised staring at the watch and interacting with it.

So if you are using a default session configuration or an ephemeral session configuration, keep in mind that these networking transfers will happen only while your app is actually running. So this is totally fine if you are sending small amounts of data like fetching stock quotes or weather data or social network status updates, but for any larger content like videos, for instance, you 'll want to use background uploads or downloads, which can continue out of process while your app is no longer running. And for more information on background uploads and downloads, I highly encourage you to check out the WWDC sessions on Foundation Networking from previous years. Next up, I would like to talk about some API changes that we have made in this year's releases.

The first thing I would like to talk about is NSURL Connection, something we have not talked about today so far. So, this year, I'm announcing that NSURL Connection is deprecated on OS X, El Capitan, and iOS 9.

And let's just take a moment to think about what that means. Deprecation does not mean that NSURL Connection is going away entirely. We know that there are a lot of apps out there that are using NSURL Connection, and we're not just going to break them. So it will still work. Those transfers will still work, but keep in mind that new features are really only going to be added to NSURL Session at this point. We highly encourage you to switch over existing code from NSURL Connection to NSURL Session if you haven't already, and if you are writing new code, we would really hope that you would only use NSURL Session and not Connection. Another thing to keep in mind is that NSURL Connection is not supported at all on watchOS. So if you need to load HTTP content from a WatchKit extension, you have to use NSURL Session. But luckily for you, if you haven't done so already, switching from NSURL Connection to NSURL Session is actually very easy. So I would like to walk through an example of that now. Here's a simple use of NSURL Connection to perform an asynchronous HTTP request, which probably looks similar to the things that a lot of you have in your apps. Here, we're connecting to www.example.com over HTTPS, and we have an NSURL object to represent that.

We then construct an NSURLRequest, wrapping that URL, and we use NSURL Connection's 'send asynchronous request' method to fire off that request asynchronously. And we receive the result in the form of this closure, which includes an NSURL Response object representing the HTTP response headers that are received, an NSData object for the response body data, and an error if a transmission error occurred.

So let's take a look at what that would look like with NSURL Session.

It's very similar. You will notice that the NSURL and NSURL request objects are still in use and this is true for a lot of the other NSURL family of objects, like NSURL Credential Storage and NSHTTP Cookie Storage. The main difference here is that instead of using NSURL Connection to send an asynchronous request method, we are using the 'data task with request' method on the NSURL Session shared session. Then once we resume the task, we would again receive the results of that transaction, asynchronously, in the form of the response data, the response headers, and an error if a transmission error occurred. So that's how easy it is to switch from NSURL Connection into NSURL Session in your apps.

So, next I would like to switch gears and talk about some new additions that we've added to the NSURL Session family of APIs this year.

The first thing I would like to talk about deals with cookies.

Last year at WWDC we introduced a great new feature called app extensions, which lets you embed parts of your app's functionality other places on the system like notification center.

But applications and their extensions have different data containers by default, which means that even if you are using NSURL Session and already leveraging our great built-in cookie handling support, those cookies are actually being stored in different places. But you can use what's called an application group to actually get access to a shared data container, which both your app and its extensions can access. And this year we have introduced new API to let you create a cookie storage associated with that group container. And I would like to show you how to do that. So what you want to use here is NSHTTP cookie storage's new 'shared cookie storage for group container identifier' method. And you just simply create a cookie storage with the passing in the name of your application group, and application groups can be configured while editing your project's build settings in Xcode and going to the Capabilities tab.

Once you've created the cookie storage, you just need to set it as the HTTP cookie storage property on an NSURL Session configuration object, create an NSURL Session from that configuration, and then any tasks that you create in that session will use that cookie storage in the group container.

So for the entirety of the presentations today, we have been talking about using NSURL Session in the context of loading HTTP content.

But there are some cases where you really need a protocol other than HTTP or HTTPS. So if you are implementing a chat application, a video calling app, or -- or something like that, you really need -- you might need a protocol that's not HTTP, and you want to do something custom directly on top of TCP/IP networking. So NSURL Session Stream Task is a new API this year, which is a Foundation extraction, directly over a TCP connection.

Now, in the past, you might have used NSInput Stream and NSOutput Stream to do something similar, but we think that NSUSRLSession Stream Task has a few key advantages over the NSStream APIs. First of all, it offers a very simple, convenient asynchronous read and write interface, whereas with NSStream you had to set up a delegate to listen for events or when to read, and the read and writes could block, and it was kind of a mess. But this is a lot easier and cleaner.

Secondly, NSURL Session has great built-in support for automatically getting through HTTP proxies, and NSURL Session Stream Task can leverage this support so that you can connect to a remote server even if there's an HTTP proxy between it, and NSStream can't really do this. And there are a few other new API goodies that I will talk about later as well.

We also know that NSStream is a very pervasive API. There are a lot of other frameworks and APIs that accept and work with NSStream objects. So we also have some support to be compatible with NSStreams as well and I will show you that later on.

So as I said, NSURL Session Stream Task supports TCP/IP connections, which you can create explicitly with a host name and port, or if you are using the NSNetService's APIs to discover Bonjour services in your app, then we can actually accept that NSNetService and resolve that for you automatically.

Stream Task uses the existing NSURL Session configuration options and delegate methods to communicate events to you, and, of course, we support TLS-secured connections and you can even dynamically change this once you've actually established a connection to a server. So let's take a look at how you would perform a read operation with a Stream Task. First, to create a Stream Task you can simply use the 'Stream Task with Host Name and Port' method. And you just pass in the host name and port that you want to connect to. And then after resuming the task, you can use this 'Read Data of Min Length, Max Length, Timeout' method.

And what this will do is you can pass in a range of bytes that you want to read and a timeout for that operation. So if we managed to read something within that range or we hit an EOF, or there's a transmission error or timeout that occurs, we will invoke this closure with the results.

Writing is very similar. We -- unlike -- with NSStream we can work directly with NSData, you just pass in the NSData object that you want us to write and a timeout for that operation, and, again, this closure will be invoked with a nil error if that occurred successfully or an error if a timeout or a transmission error occurred. Enabling TLS is as simple as calling the 'Start Secure Connection' method on the task. I mentioned before that we also have some built-in support to work with NSStream, NSURL Session Stream Task. So the reason we do this is because we know that there are a lot of APIs out there that work with NSStream objects already and we want to give you something that can still be compatible with those. So, you can actually convert an NSURLSession Stream Task to NSStreams, and I will show you the very simple code to do that on the next slide. Just keep in mind that when you do this, any pending async reads or writes that you have enqueued on the Stream Task will complete before we give you the NSStreams. And that doing this also detaches the task from the session. And so its connection, its underlying connection will no longer count against a limit that you may have set on the session's maximum number of connections per host, and it will no longer be in the session's set of outstanding tasks.

So to convert a Stream Task to Streams, you simply call the 'Capture Streams' method on the Task, and then those input streams and output streams will be delivered to your delegate with a new delegate message.

We also have a few other informational delegate messages that we have introduced on Stream Task. The first is interesting. It's a better route discovered for Stream Task. So let's say, you have a Stream Task that's connected to your server over a cellular data connection. If the user then joins a Wi-Fi network, then we would deliver this notification to your app to tell you that there might be a better way to connect. And what you might want to do in this situation, if you want to, is tear down your existing Stream Task and create a new one to that host and port, and try to connect over the better connection. And we leave it up to you to decide whether or not you want to do this. You know, if you are 99% of the way through transferring a lot of data it might not make sense to actually tear down the existing connection and create a new one. And we also have some informational delegate methods to tell you when to read or write sides of the underlying TCP connections have been closed.

I also talked before about our automatic support for dealing with HTTP proxies in Stream Task. And the way you can do deal with this is actually by starting with an existing NSURL Session Data Task, which generally deals with HTTP content, and converting it to a Stream Task.

And this you can do when the response for that Data Task is received.

So to convert a Data Task to a Stream Task, and let you communicate directly over the TCP connection without the HTTP framing, you would simply respond to this completion handler in the 'Did Receive Response' delegate method with the new 'Become Stream' disposition. And then we will inform your delegate that the Data Task has been converted to a Stream Task with the 'URL Session Data Task Did Become Stream Task' method.

Before we finish up today, I would just like to go over a few things that we discussed this morning. We first talked about how you can use App Transport Security to guarantee that only secure connections are made from your app and how you can declare exceptions to those as needed. We then talked about how you can make your apps faster by supporting HTTP/2 on your servers, which requires no additional changes on the client.

We talked about best practices for using NSURL Session on watch apps, and went over the new API changes this release, the deprecation of NSURL Connection and how to migrate to Session, how to share cookies between apps and extensions, and how to get a great foundation level abstraction if you need to communicate using a TCP connection without HTTP to your servers. As always, I would like to refer you to the documentation that we have available on our website, and remind you about the developer forums that you can participate in, and any questions that you might have while you are here at WWDC this week, Paul Danbold is our Evangelist and you can talk to him. There are a few other related sessions that you might want to attend. I highly encourage you to attend "Your App and Next Generation Networks," tomorrow, which will be a lower level discussion of networking. And the networking labs occur this week as well and you can talk to us directly for any questions you have. With that, I would like to thank you for coming out today, and we look forward to seeing the amazing apps that you create with iOS 9, OS X, El Capitan, and watchOS 2. Thanks. [Applause]