What's New in Network Extension and VPN

JAMIE WOOD: Thank you.

Good morning, everyone. Welcome to session 717. My name is Jamie Wood and I'm here today with my colleague, Tommy Pauly. We are Network Engineers in the Core OS networking group at Apple, and we are excited to tell you about what is new in Network Extension and VPN. So you may be wondering what is Network Extension Framework? What capabilities does it have to offer to me for use in my application? Network Extension Framework provide APIs to use in your app to customize and extend the Core networking features of Apple's platforms. Let's look at specific examples of kinds of apps you can create using the Network Extension APIs.

If your company makes Wi-Fi Hotspots that are deployed in public places like hotels, airports, coffee shops, you can use the any Hotspot helper API to create apps that connect to your Wi-Fi Hotspots.

If your company creates a personal VPN service that your users use to securely browse the Internet you can use the NEVPN manager API to create apps that connect to your personal VPN service.

If your company creates a remote access VPN server, you can use the any tunnel provider family of APIs to create apps that connect to your VPN server. And finally if your school creates a network content filtering solution for use by schools to protect the students' Internet browsing, you can use the any filter provider family of APIs to create apps that filter network content.

So these are all of the Network Extension APIs that we are going to talk about today during the session.

So let's go ahead and dive into the any Hotspot helper API.

So the any Hotspot helper API is used to create apps that connect to Wi-Fi Hotspots. Now, some of you may be aware that there actually are already APIs that you can use to create apps that connect to Wi-Fi Hotspots. The CNSetSupportedSSIDs and CNMarkPortalOnline APIs, but they have limitations.

One problem is that the Hotspot list you can pass to CNSetSupportedSSIDs is limited in size. If you have a Global Network of Wi-Fi Hotspots you're trying to connect your app to you can quickly run into the size limitation.

Another draw-back is that in order to call CNMarkPortalOnline, users have to manually run your application.

This is exacerbated by the fact that there's no indication to the user they need to do this. It is not a great user experience. The users don't know that they have to go into the app to connect to the Wi-Fi Hotspot.

So we've introduced the any Hotspot helper API to address some of these limitations.

So the way this works is you first register your app with the system as a Hotspot helper.

Then as the device comes within range of Wi-Fi networks, scanning for Wi-Fi networks or the user selects a Wi-Fi network to connect to, the system will call into your app, run your app in the background, call into your app and give your app the opportunity to claim the Wi-Fi Hotspot with a level of confidence, high, medium or low. If you claim a Hotspot with a high level of confidence, the system will call you to actually perform the authentication with the Wi-Fi Hotspot.

And it will periodically call you to maintain the authentication session.

The Hotspot helper API also allows you to annotate Wi-Fi networks that show up in the Wi-Fi manager in the settings app and you can annotate these Wi-Fi networks with the name of your app or the name of your company.

So that's the any Hotspot helper API.

You can use this API to create apps that seamlessly connect to Wi-Fi Hotspots. Next let's look at the NEVPN manager API. You can use this API to create apps that connect to a personal VPN service that users will use to securely browse the Internet.

Now, the NEVPN manager API is not new in iOS 9.

We introduced this API in iOS 8.

We made some enhancements to the API and made the API available on OS X, OS X El Capitan.

The way this works is you can use the NEVPN manager API to configure and control the built-in IPSEC client on the platform.

You can create a single personal VPN configuration. I'll talk about what a personal VPN configuration is in a few minutes here. The built in IPSEC clients supports IKE v1 and IKE v2 key exchange protocols. You can use either of these.

You can configure the VPN to connect automatically under certain network conditions.

New in iOS 9 and OS X El Capitan, you can configure http proxies for http traffic traversing your VPN tunnel. And personal VPN configurations coexist and cooperate with enterprise VPN configurations. Suppose the user is using your app to connect to your personal VPN service, and they are securely browsing the Internet and want to access something in their company's internal network.

So they go ahead and connect to their enterprise VPN and all of the traffic that's destined for those internal resources will go ahead and use that enterprise VPN. But everything else that is not going to use the enterprise VPN will continue to go through the personal VPN. Personal VPN remains connected and will continue to tunnel traffic.

So there's a brief run down of the features that the NEVPN manager API provides.

Another way that we've enhanced NEVPN manager is we've enhanced our IKE v2 protocol support.

We think that's a great tunneling protocol and made it better in this release. We now support MOBIKE.

We will now seamlessly transition your IKE v2 tunnel from one network interface to the other. For example, if your tunnel is connected over the Wi-Fi interface but the user walks out of range of that Wi-Fi network, so now the device switches over to use the cellular interface, the IKE v2 tunnel will seamlessly transition over.

Very cool.

JAMIE WOOD: We now have full support for IPv6, both inside and outside of the IKE v2 tunnel.

We now support IKE fragmentation. This allows the IKE v2 tunnel to work more reliably in a variety of network environments. We have IKE redirect supports allowing you to load balance your IKE v2 server.

So with all of these enhancements it is now better than ever. IKE v2 works better than ever. It works more reliably in a wider number of network environments. So now let's take a look at some code. How do you use the NEVPN manager API? I want to show you some code that will create this personal VPN configuration.

So each app that uses the NEVPN manager API gets access to a single personal VPN configuration, which is represented by a single NEVPN manager object. So first I get a reference to my shared manager object and all of the VPN configurations on the system are stored in the network extension preferences. Before I can work with my configuration, in my code, I need to load it from those preferences using this load from preferences with completion handler call.

In the completion handler, the first thing I want to do is check and see if my configuration exists or not. I look at the protocol property on the manager object. If that's nil, it means I don't have a configuration yet. I want to set one up. I'll configure a IKE v2 configuration here and I fill it out with all the IKE v2 specific configuration parameters.

When I'm done with that I'll set the protocol property on the manager object and I'll enable the VPN configuration. You can only have one personal VPN configuration enabled on the system at a time. And then when I'm all done, go ahead and call save to preferences to save the configuration into the network extension preferences and apply the configuration of the system so they can actually be used.

From here you can use the API to manually start and stop the VPN tunnel and introspect the status of the VPN tunnel to know when you're connected, disconnected, et cetera.

But for great user experience another thing to do is to configure connect on-demand. So with connect on-demand you can set up rules that state when the VPN should connect automatically.

So, for this example, I'm going to go ahead and set up a rule that says that my VPN should connect whenever the device is on a Wi-Fi network.

So the first thing to do, I'm going to create an on-demand rule connect object.

So this is a rule that says when this rule matches, connect the VPN. Next I'm going to set an interface type match condition on my rule.

So it says when the device is on a particular type of network interface, then this rule will match. I'm go to set that to Wi-Fi.

This rule will match when the device is on a Wi-Fi network.

Go ahead and set the on-demand rules array to contain this new rule and save the result to the preferences.

So now my VPN will connect automatically whenever the device is on a Wi-Fi network.

Okay. So that's the NEVPN manager API. You can use this API to create apps that connect to your personal VPN service. Let's go ahead and talk about the any tunnel provider family of APIs.

You use these APIs to create apps that connect to enterprise VPN servers. So the way this works is you create a custom VPN protocol provider.

This is essentially the client side implementation of your own custom tunneling protocol. These protocol providers run as app extensions. They run in the background handling background traffic.

You can create two types of tunnel providers. You can create a Packet Tunnel Provider.

These tunnel traffic at the IP layer.

You can create, or you can create an App Proxy provider that tunnels traffic at the app layer.

The tunnel provider family also has APIs that allow you to configure and control your protocol provider from your app. So similar to the VPN manager API where you can configure and control the IPSEC client, these APIs allow you to configure and control your own custom protocol provider.

Let's take a closer look at the NEPacket Tunnel provider API. How does this work? So suppose we have an NEPacket Tunnel Provider here running on the system and it's connected to a VPN server and providing a tunnel to some internal network.

So you've got an app that's trying to connect to a resource in the internal network.

So the app will open up a network connection that will create a socket and create a TCP/IP connection and the packets for that TCP/IP connection will be routed to the UTUN0 interface. This is a virtual interface, so instead of sending the packets out over the physical network it will divert them to the NEPacket Tunnel Provider.

The Packet Tunnel Provider then can take the packets, encapsulate them in your tunneling protocol, send them over to the tunneling server. The tunneling server will decapsulate them, inject those packets into the internal network and they will be delivered to the ultimate destination, and the return packets will be encapsulated by the server, sent back to the provider, the provider decapsulates them and injects them back into the networking stack via the UTUN0 interface, they'll be delivered back up through the TCP/IP stack to the application.

This is the basic way that this works.

NEPacket Tunnel Provider has a lot of control over the UTUN0 interface.

Most importantly, it can specify the routes. The IP destinations that will be routed to the UTUN0 interface and through the tunnel.

The Packet Tunnel Provider can specify the virtual address to assign to the interface, as well as the DNS and proxy settings that should be used for traffic routed through the tunnel. So next Tommy Pauly is going to come up and give us a brief demo of how to create an NEPacket Tunnel Provider.

TOM PAULY: Thank you, Jamie. We are excited to have you write your own custom VPN protocol providers. To help you do that I want to show you quickly the steps that are involved in building a Packet Tunnel Provider. I'm going to start with an Xcode project in which I have an app already. It is using the manager APIs to create a configuration. So we are going to skip over that part and we're going to assume that I have a framework that is implemented to do my own custom protocol negotiation. I want to show how to build this new extension and kind of put all the pieces together. All right.

So here I have my project.

The first thing you need to do is add a new target for your extension.

To get this, in OS X El Capitan we've added it into the network extension framework.

So on your system if you go to system, library, Frameworks, network extension Framework and you go into the resources folder, we've provided an installer for the templates for this, these target types.

So you can go through the installer.

I'm going to install it just for this user.

And now I have the templates. If I go into my project and I try to create a new target, under application extensions I now have four new target types.

I have App Proxy providers for application layer VPNs. I have two filter control and data providers. These allow me to do content filtering which we'll talk about later and I have Packet Tunnel Provider. My protocol in this case uses IP level tunneling for my VPN. I'll choose the Packet Tunnel Provider. Going to write in Swift. Let's just call it packet tunnel. Great. Now I have a new target and I can start building my VPN provider. So let's jump ahead to the actual process of writing this. My protocol happens to be based around UDP. It uses UDP to talk to the server and tunnel traffic from the system. I have subclassed the NEPacket Tunnel Provider class and this is the class going to be called in my extension to start and stop my VPN.

The most important function that I'm going to overwrite here is the start tunnel with options. This will get called whenever the user or the system automatically wants to start your VPN. And if you notice, it calls and passes a completion handler. This completion handler is what you can use to call and tell the system that you are done setting up your VPN. When you call this, the system will know you fully connected and it's ready to go. So I mentioned already that my protocol is based around UDP. I want to in my start create a UDP connection to my server. In order to do this I'm going to create an NWUDP session object. This is one of many convenience networking APIs that we've added to the Network Extension Framework to help you make good and very efficient connections. So UDP session allows me to create a connection to a host name. It will do all the DNS resolution for me and asynchronously let me know when it's ready to go. I can do reading and writing of multiple data grams at a time for the most efficient connections possible. Let's implement start tunnel with options now.

The first thing I need to do is actually figure out what I'm trying to connect to. I look at my configuration to get my server address. This might be an IP address or a host name.

I then wrap it up in an NW host end point. This is just a container object that takes an address or a host name and puts it alongside a port that belongs to my protocol.

I then call create UDP session 2 end point to start the process of doing DNS resolution and creating the connection to my server.

At this point the system starts doing that and I am waiting for the event that I want to read and write. I do KeyVO to watch the state property of the UDP session to get these transitions. So at this point I'm pretty much done with my start call. The last thing I need to do is save that completion handler that I got at the beginning because I'm not yet done completing my VPN connection.

So I'm going to declare a pending start completion. And I'm going to save that for later.

All right. So I'm going to skip over the part where we actually negotiate with our server. I assume your protocol already knows how to do that. I'm going to skip ahead to where we have been assigned an address and other network settings. We want to apply it to the system and let the system know that we're done bringing up the VPN.

This is just a custom function that I wrote that says that I'm done bringing up my connection.

What I'm going to do is call, create NEPacket Tunnel network settings object, and this is a bundle of settings that I can apply into the system.

I can set the IPv4 settings, I have an assigned address and subnet.

I have route settings. In this case I want to be the default route for the system but I can also have a long list of split tunnel routes.

I can define the overhead bytes that my tunnel has. For every packet, how much do I incur for encryption and encapsulation. This helps the system know how large it should send packets. And lastly, I can set the DNS settings. I can set my assigned DNS server or domain.

To apply this to the system, I call set tunnel network settings. I get a callback handler when it is done and that lets me know it has been successfully applied and I can call the start completion handler letting the system know that I'm done bringing up the VPN. This is great. I have a full connection up.

But the thing I'm missing is being able to route traffic back and forth. I'll give you a look at what that looks like. So as Jamie mentioned, the way you're doing this, you have a virtual interface that is going to be sending packets up into your protocol provider and then you're going to send that off to your server. When you receive packets back from the server you re-inject them back into the stack. I want to start first with the outbound flow.

A packet tunnel provider has a property called a packet flow. This represents the virtual interface that you can read and write into.

If I call read packets with completion handler, I will get called back with an array of packets that have been sent out into the network.

I can send these into my protocol to encapsulate and to encrypt them and generate a new array of encrypted payloads. And very easily I can write these as multiple data grams at one time into the UDP session to send over to the server.

That's all we need to do to send packets out.

Coming back in is very similar.

Here I have a function that is taking packets in from the UDP server.

And I can simply send these to my protocol. I can decapsulate them, generate IP packets which I then write into my packet flow property.

Great. So now I have a connection that can be established in read and write. Let's see how this looks in practice. All right.

So here I have my device. If you notice I've installed already my VPN app.

First I want to show you that if I go into Safari and try to access the internal website, I do not have access to it right now. I now can go into my VPN app and I hit start VPN. And what this did, in the background it kicked off the tunnel packet provider.

If you can see now, I do have the VPN badge up in the status bar. It actually did come up and establish the connection.

I can then go back into Safari and I see that my internal page has now loaded. So we have a fully working VPN protocol provider all written in Swift. That simple.

TOM PAULY: Back to Jamie. JAMIE WOOD: All right. Thank you, Tommy. So Tommy just showed us how to create a can packet tunnel provider. It's easy and powerful.

So while we are on the subject of Packet Tunnel Provider I want to take a moment and talk about Per-App VPN for managed apps. Per-App VPN is great for BYOD or bring your own device use cases.

In BYOD you have company employees who want to bring their own personal device, connect it to the company's internal network and access network resources. They also don't want all of their Internet traffic to be routed through the company's VPN. And then you have enterprises that want to give their employees access, but they want to restrict the applications that can access their internal network.

So this is where Per-App VPN is great.

So with Per-App VPN you can set up the VPN so that only certain managed applications can use the VPN.

The way to set this up is using Apple's own MDM or mobile device management protocol. You enroll devices with an MDM service. The MDM service has the ability to push down configurations including VPN configurations as well as managed applications.

And the MDM service can also link those managed applications with Per-App VPN configurations to send up a Per-App VPN deployment. Now, we support Per-App VPN with, we've always supported it with custom App Proxy providers.

In iOS 9 and OS X El Capitan, we now support Per-App VPN with custom packet tunnel providers and support Per-App VPN with a built-in IPSEC clients on the platform.

So let's go ahead and take a look at how Per-App VPN works in the network data path. We have the same Packet Tunnel Provider that we had before. We have the UTUN0 interface again that the traffic is routed to. Now we have a managed application that is linked to the VPN configuration.

Instead of having the traffic routed to the UTUN0 interface by the destination IP, the traffic is routed to the UTUN0 interface by the source application. Only this managed app can access the VPN.

If the user uses another app that is not managed, it is going to connect to the network out the physical interface. Even if it's trying to connect to the same IP that the managed app is connecting to. All right.

So that's how Per-App VPN works.

Tommy is going to give us another brief demo of how to configure Per-App VPN. TOM PAULY: Great. Thank you, Jamie.

We are really excited to have... Now, everyone who is writing a custom protocol or even using the built in IPSEC protocols will be able to deploy Per-App VPN to make great UID solutions. As Jamie mentioned, we create these Per-App configurations with MDM. MDM is pushing down a configuration profile that defines the VPN and also pushes down the managed apps to associate with that VPN. I want to show how to modify an existing VPN profile to make it work with Per-App VPN.

I'm going to do it with a profile that configures the same custom app that I just built.

So here is a look into a VPN profile. There's a lot on here. You don't have to get it all right now. I want to highlight the parts that are different. The three tweaks that you need to make to have this profile work with Per-App VPN. The first is the payload type. Normally it is com.Apple.vpn.managed simply add a .applayer to the end of that to make that a Per-App VPN. There are multiple types of providers that work with Per-App VPN. So we need to specify which type you're using. Within the VPN payload, we have a new key that is the provider type.

And you can specify either packet tunnel or App Proxy. In this case as you saw, my app uses a packet tunnel protocol. I'll specify that one. The last field you need to add is called the VPN UUID. Which is an arbitrary string that acts as the glue between a VPN configuration and the managed apps. When an MDM server pushes down managed apps, it will mark them with the same VPN UUID so the system knows those apps can only route traffic through that VPN. This profile should work for Per-App VPN. So because we are so eager to have you guys make apps that work with Per-App VPN, we wanted to make it really easy to develop with it. You may not have access to a full MDM solution while you are developing.

So we've enabled for development builds of your app only. Not distribution builds.

The ability to specify in an info Plist apps to associate your VPN configurations with to make it easier to test Per-App VPN. Here is a look at an info Plist of your app, it's small here, but we have a new key called NETest App Mapping. This is a dictionary of arrays, the keys are the same UUID I mentioned before, the glue to hold configurations and apps together. Within the array you can specify the string bundle identifiers of apps that you want to force to go through your VPN. In this case I'm going to use the box app.

Great to show you what this looks like. I installed the profile and the info Plist on my device. And so here I have a configuration for my Per-App VPN. And you can see that I have my custom Packet Tunnel protocol.

And listed are the included apps and it has now associated Box with my VPN configuration.

That's all it takes to making Per-App VPN work with your protocol. Thank you.

JAMIE WOOD: All right. Thank you, Tommy. So we really think that Per-App VPN is a great deployment for BYOD, provides a great user experience for company employees. We highly encourage you to deploy it.

So next I want to talk about the NEApp Proxy Provider API.

Now, any App Proxy providers work exclusively with Per-App VPN.

So let's take a look and see how these work, how they are different from the packet tunnel providers. We have a managed application, an App Proxy Provider running on the system and the managed app is going to connect to an internal network resource.

Instead of having IP packets routed to the UTUN0 interface, the data the managed app writes to its socket is diverted directly to the proxy provider.

So from there the NEApp Proxy Provider can send the data to the proxy server and the return data from the proxy server will be sent back to the proxy provider, it can inject it into the socket to be delivered to the application.

The reason why we added the NEApp Proxy Provider API is really about the servers. So you're connecting to these transparent network proxy servers.

The servers are usually easier to deploy than a fully fledged IP layer VPN. You don't have to provision virtual IPs to assign to the UTUN0 interface. They usually scale a little better. We've given you the NEApp Proxy Provider API to connect to these servers. And a new thing that we added for App Proxy Provider in iOS 9 and OS X El Capitan is the ability to proxy UDP traffic in addition to TCP traffic.

Oh, here we have a managed app to illustrate those are still connecting out the physical interface. Okay. So that's the NETunnel provider family of APIs. Use these APIs to create custom clients for your tunneling protocol, to connect to enterprise VPN servers. Now let's take a look at the NEFilter provider family of APIs. Use these APIs to create the network content filtering solutions for schools.

There are currently some ways that schools can do network content filtering with iOS devices.

They can deploy an on-site content filter, put a device on their local network and route all their Internet traffic through that content filter so that they can filter the content, right? The draw back with this is that it's only available on the school's local network. If the students want to take the schools iPads or iPhones home they either can't browse the Internet at all when they are home or browse the Internet unfiltered. This is not so great. To sort of try to solve some of that problem, the school can deploy global proxy, put it on the Internet, route all of the Internet traffic through the proxy to perform the content filtering.

The draw back, the school has to deploy and maintain the proxy, and not all the schools have the resources to do that. Another solution is to use a full tunnel VPN. That has a lot of the same problems as global proxy and may be even harder to deploy and maintain.

The best solution for schools is to filter the network content on the device. Before it leaves the device and just before it's actually delivered to the user.

So this is exactly what the NEFilter provider family of APIs allows you to do.

Now, the NEFilter provider APIs are only available on iOS.

And using API you can perform dynamic evaluation of network content. You have the ability to update filtering rules on the fly, for instance downloaded from the Internet.

And you have the ability to send back a customizable block page to the user when they try to access something that they are not allowed to access.

Now, NEFilter provider only works on supervised devices. We are targeting this at schools, schools that own their devices, are locking them down so that users can't install new apps or change settings.

This is where filter provider works.

So let's take a look at how filter provider works in the data path. So we have an NEFilter data provider extension running on the system. And its job is to make pass and block decisions about network content that is flowing through the system. Now, because it has access to all this network content, we've locked down this data provider so it runs in a read only sand box.

It can't access the network and it has read only access to the disk.

So we've also provided another extension, the NEFilter control provider, and its job is to feed information to the filter data provider so that it can do its job. For instance, download rules over the Internet, write them to a location on disk where the data provider can access them.

Now, suppose that the user is running an app using Web kit, right? They have a UI Web view in their UI and so all the content that is being rendered into the Web kit is passed off to the filter data provider, which makes a pass or block decision on that data. So obviously not all applications use Web kit.

There's some applications that are using other networking APIs such as NSURL session, NSURL connection, some of the CFNetwork APIs.

For those APIs we added a hook into the socket layer to send that traffic over to the filter data provider so it can make a pass/block decision.

As you can see the filter control provider and data provider together provide a comprehensive content filtering solution on the device. So for Web kit applications, when the data provider makes a block decision, decides to block some access to the content, the data provider can send a, instruct the Web kit to display a block page. Here is an example of that block page. You can, using the API you can customize this block page in a number of ways. You can change the organization that's displayed. For example, you can display the name of the school and you can customize this request access link. You can choose to not display the link at all, not give the user the option of gaining access to the content. If you display it, you can display the text of the link and target of the link. Point to the Web service where users can go and request access to the blocked content. So that's the NEFilter provider family of APIs. You can use these APIs to create powerful comprehensive, on-device network content filtering solutions for schools. Here is all the Network Extension APIs we've talked about today.

And here's some of the great apps that you can create using these APIs. You can create apps for Wi-Fi Hotspot, personal VPN services, enterprise remote access VPNs and you can create on-device network content filtering solutions for schools.

As you can see these are powerful APIs.

They allow you to extend and customize the core networking features of Apple's platforms.

And because these are very powerful APIs we do require some special entitlements for you to be able to use these.

To use the NEVPN manager API, you need to select the personal VPN capability in your project settings' Xcode. To use the any Hotspot helper API, the NETunnel family of APIs, and the NEFilter provider family of APIs you need access to other special entitlements.

The way to get these is to send a request to networkextension@Apple.com.

We'll send you back a questionnaire that we'll ask you to fill out and send back to us and we'll process your request. We're excited to be delivering these APIs to you. We are looking forward to getting all of your requests and giving you access to these APIs.

So what should you do? Port your Captive Network apps to use the NEHotspot helper, which is seamless. Adopt the NEManager API in your personal VPN app to create seamless user experience for those using your personal VPN service. Use the NETunnel provider to create powerful custom enterprise VPN apps that connect to your VPN server. Use the MDM to deploy Per-App VPN for great BYOD user experiences. And use the NEFilter provider APIs to create dynamic on-device network content filtering for schools. Here is where you can get more information. Unfortunately, the documentation is not yet published. Coming soon. Please keep searching for it on the developer website. And the sample code is also not quite ready yet. Keep searching for the simple tunnel sample. We have a great sample going. We haven't quite finished it yet. Please keep looking for it.

For more information you are welcome to participate in the dev forums, contact developer technical support.

Reach out to Paul Danbold our Core evangelist. And here is the address you can send your network extension request to.

Here are related sessions. We encourage you to check these out. Especially want to give a plug for the other Core OS networking session today happening 11:00 a.m., Your app and next generation networks.

Thank you all so much for coming. Have a great day. [Applause]